21

JUnit4 has @FixMethodOrder annotation which allows to use alphabetical order of test methods execution. Is there analogous JUnit5 mechanism?

1

4 Answers 4

23

Edit: JUnit 5.4 is officially released now, so no need to use snapshots anymore.

This is now possible with JUnit 5.4.

https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-execution-order

To control the order in which test methods are executed, annotate your test class or test interface with @TestMethodOrder and specify the desired MethodOrderer implementation. You can implement your own custom MethodOrderer or use one of the following built-in MethodOrderer implementations.

Alphanumeric: sorts test methods alphanumerically based on their names and formal parameter lists.

OrderAnnotation: sorts test methods numerically based on values specified via the @Order annotation.

Sign up to request clarification or add additional context in comments.

4 Comments

It works for me, yet adding only dependencies { testImplementation "org.junit.jupiter:junit-jupiter-api:5.4.0-SNAPSHOT" testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:5.4.0-SNAPSHOT" } is enough
I use maven pom.xml. At this moment I can see 5.4.0-M1 version. As I know Milestone Dependacies not good for developments. How do you guys get a different version? :O
How can I order test classes?
@dmatej Currently you cannot. But you can add a comment if you need it as well: github.com/junit-team/junit5/issues/1948
13

No, not yet. For unit tests, execution order should be irrelevant. For more complex tests, JUnit is aiming to provide explicit support - test ordering would be part of that.

7 Comments

From a technical perspective, it's true that execution order should be irrelevant, but from a human perspective, the number of times that the random order has tripped me up is too much. I want them to execute in the same order from test run to test run.
I'd argue that if the non-deterministic order trips you up, then the code you're testing has implicit temporal dependencies that make it very hard to use. I'd make fixing that a high priority, which has the added benefit to stabilize the test suite.
Disagree, from the UI perspective. The human brain. E.g. I'm running a class of ten tests and 2 are failing, so my immediate training tells me to look at the two tests that are failing, in order from the top, like the 2nd one down and the 4th one down, when I run it again. But then when they don't come back in the same order, I have to (a) realise that there's not some bizarre non-deterministic bug causing random tests to fail, they're just not in the same order and (b) remember the usually dumb test names that failed and run it again to make sure they are actually failing consistently.
So technically I totally agree with you - running in random order is good - but for the sake of sanity, I'd like a little button on the IDE unit test dialog which allows me to impose order on the test run - preferably by order of appearance in code.
Yes I use Intellij at the moment, and I like the re-run failed tests feature - but the 'run-in-code-order' is my ideal and I end up naming the test methods so their code order is also alphabetical.
|
0

With version 5.8.0 onwards, test classes can be ordered too.

src/test/resources/junit-platform.properties:

# ClassOrderer$OrderAnnotation sorts classes based on their @Order annotation
junit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer$OrderAnnotation

Other Junit built-in class orderer implementations:

org.junit.jupiter.api.ClassOrderer$ClassName
org.junit.jupiter.api.ClassOrderer$DisplayName
org.junit.jupiter.api.ClassOrderer$Random

For other ways (beside junit-platform.properties file) to set configuration parameters refer here.

You can also provide your own orderer. It must implement ClassOrderer interface:

package foo;
public class MyOrderer implements ClassOrderer {
    @Override
    public void orderClasses(ClassOrdererContext context) {
        Collections.shuffle(context.getClassDescriptors());
    }
}
junit.jupiter.testclass.order.default=foo.MyOrderer

Note that @Nested test classes cannot be ordered by a ClassOrderer.

Refer to JUnit 5 documentations and ClassOrderer api docs to learn more about ordering test classes.

Comments

0

The answer to this question is sub.version dependent. In my environment, junit-jupiter-5.5.2, the implementation of ASCII test ordering that works for me is:

import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.api.MethodOrderer.Alphanumeric;

@TestMethodOrder(Alphanumeric.class)

Related orderings (e.g. explicit numbering) are documented in the JUnit 5.5.2 API.

As of JUnit 5.7, I believe the amended declarations are:

import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.api.MethodOrderer.MethodName;

@TestMethodOrder(MethodName.class)

If somebody with Junit 5.7 or later wants to test this, confirmation or correction would be welcome.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.