0

Let's take this unit test. Unit testing guidelines state that I should only have 1 assert per test, unless I'm testing the state of an object. In this case, Muxer.Muxe is a wrapper around FFMPEG that generates the right command line and executes it. Having the function return "Success" really tells me nothing about whether the FFMPEG commandline was correctly generated for a variety of scenarios, which is what requires more testing. (For example an extra tag needs to be added for AAC files).

So, based on unit testing correct practices, should I include the last part of the test, which tests the internal work being done within the method, or not?

[Theory]
[InlineData("video.mkv", "audio.aac", "dest.mp4")]
[InlineData("video.MKV", "audio.AAC", "Dest.MKV")]
[InlineData("video", "audio", "dest")]
public void Muxe_AudioVideo_Success(string videoFile, string audioFile, string destination) {
    var Muxer = SetupMuxer();

    var Result = Muxer.Muxe(videoFile, audioFile, destination);

    Assert.Equal(CompletionStatus.Success, Result);

    Assert.Single(factory.Instances);
    IProcessManagerFFmpeg Manager = factory.Instances.FirstOrDefault() as IProcessManagerFFmpeg;
    Assert.NotNull(Manager);
    output.WriteLine(Manager.CommandWithArgs);
    Assert.Contains(audioFile, Manager.CommandWithArgs);
    Assert.Contains(videoFile, Manager.CommandWithArgs);
}
Improve this question
10
  • 7
    " Unit testing guidelines state that I should only have 1 assert per test" And pragmatism states that rules like that are crap and that there should be as many asserts as is necessary to make the test valuable and clear in intent. Commented Feb 10, 2019 at 6:43
  • @Euphoric - "crap" is to much. With multiple assertions per test, will tell you only first failed reason. After you fix it it, test will fail with next one. So you will be forced to run tests suite multiple times. Where pragmatism with one assertion will know actual reason only by reading test name. And of course it depend on testing framework. Commented Feb 10, 2019 at 7:43
  • 3
    @Fabio And? What is problem running test suite multiple times? Running test suite should be really fast. Or you can just re-run the same test until it works. Commented Feb 10, 2019 at 10:59
  • @Euphoric, difference is that when first assertion fails, you don't know does rest of assertions pass or fails. Multiple or single assertions is as usually in programming simple tradeoff. Single assertion provide more information, where multiple assertions can reduce amount of times same configuration need to be setup for the test. Pragmatism != write less, pragmatic programmer will use "right tool for the job". Which tool is right is your/team tradeoff. Commented Feb 10, 2019 at 11:12
  • 1
    The reason why this is okay is because you are testing the result of a single unit of work. The biggest concern with multiple asserts would be if you performed one step, asserted, did another, asserted again, and so on. Single asserts per test is a decent guideline, but I wouldn't write the exact same test with the same steps multiple times just to make sure each test had its own single Assert. Tests are code and code requires maintenance. Code duplication is a problem in unit tests for many of the same reasons it's a problem anywhere else. Commented Feb 11, 2019 at 20:33

3 Answers 3

Reset to default
3

based on unit testing correct practices, should I include the last part of the test, which tests the internal work being done within the method, or not?

Short answer: yes, it looks to me as though what are you testing is fine.

The important thing to understand: the responsibility of the unit test is to independently verify that the test subject as satisfied its own postcondition.

So if the correct behavior of a method is supposed to include side effects, then verifying those side effects (or a proxy for them) should be part of the test.

Improve this answer
2

Think about why you are doing unit testing. (And "because my boss tells me", or "because this book told me" doesn't count).

You do unit tests for these reasons: To verify that one part of your software (the "unit") works as intended, to get a warning when the unit stops working as intended for whatever reason, and to get some pointer to the area where the unit failed, saving you some work.

And when you look at "guidelines", what you need to consider is that any unit test is better than no unit tests. This "one assert per unit test" means you have to do ten tests instead of one for ten asserts. Which is a lot more work. Which is why it may not be done at all. One test with ten asserts is better than doing one test with one assert, then another test with one assert, and then stopping because the work was too boring.

And with your particular problem: You need to check that calling your function does what it is supposed to do. Just checking that it returns "success" is rather pointless. Imagine a customer reports a failure, and you say "unit tests ran fine, so the problem must be elsewhere". Now you are on the completely wrong track and it will take you a lot longer to fix the problem. That's worse than having no unit tests at all.

Improve this answer
1

The meaning of 'a single assertion' can be confusing due to human language overloading / ambiguity interacting with framework jargon.

As I understand it, your assertion would be the 'Command (line) is well formed' or, if looking for specific error handling, the 'Command (line) is malformed because X is malformed'.

To assess the 'is Well-Formed' meta property may require a compound statement of smaller result / state / property checks. For most non-functional methodologies (i.e., OO), this is the normal, not the exception.

You could create a single assertion with the compound boolean logic in one Assert() call. But assuming your verification statement is in in conjunctive normal form, sequential asserts of the clauses, possibly in descending order of likely failure, is a straight forward way to test your 'Command (line) is well formed' assertion, with the added benefit of explicitly identifying which clause of the verification procedure failed.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.