JUnit test method ordering

Biju KunjummenJanuary 10th, 2013Last Updated: January 10th, 2013

1 94 1 minute read

Junit until version 4.10 uses the order of test methods in a test class as returned by the reflection API as the order of test method execution – Class.getMethods(). To quote the Javadoc of getMethods() api:

The elements in the array returned are not sorted and are not in any particular order.

thus the order of test method execution in a test class is not predictable. This in a way is good as it encourages us as developers to write test methods which can stand on its own and to not depend on the order of test method execution.

With version 4.11 of Junit the order of test method execution is not so unpredictable anymore, by default the order though not specified will be deterministic for every run. The order can further be enforced by adding a new annotation @FixMethodOrder to the test class with the following values:

1. @FixMethodOrder(MethodSorters.DEFAULT) – deterministic order based on an internal comparator

2. @FixMethodOrder(MethodSorters.NAME_ASCENDING) – ascending order of method names

3. @FixMethodOrder(MethodSorters.JVM) – pre 4.11 way of depending on reflection based order

Consider the following test case:

public class ATest {
 @Test
 public void b_test_1() {
  System.out.println('b_test_1 called..');
 }

 @Test
 public void r_test_2() {
  System.out.println('r_test_2 called..');
 }

 @Test
 public void z_test_3() {
  System.out.println('z_test_3 called..');
 }

 @Test
 public void l_test_4() {
  System.out.println('l_test_4 called..');
 }
}

Pre 4.11 running this test prints the following in my machine

Running testorder.ATest
r_test_2 called..
z_test_3 called..
b_test_1 called..
l_test_4 called..

With NAME_ASCENDING:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class ATest

this is the output:

Running testorder.ATest
b_test_1 called..
l_test_4 called..
r_test_2 called..
z_test_3 called..

There is still no way to explicitly specify other custom sorting methods which is good as the purpose of providing a predictable order is just that – to make it predictable, not to use it to add dependencies between test methods.