In the beginning of February, the JUnit 5 (aka JUnit Lambda) team has published an alpha release. Since JUnit 4 is among the most used items in my toolbox I thought it might be worth to have a look at the next major release.
I took the latest build for a spin and noted down the changes that I found noteworthy here.
Installing JUnit 5
It is probably needless to say that a project titled JUnit Lambda requires Java 1.8 or later. If that is given then including the library is straightforward. The latest revision of the current alpha release channel is available from Sonatype’s snapshots repository at https://oss.sonatype.org/content/repositories/snapshots/org/junit/
The artifacts can be consumed with Maven and Gradle. If you prefer to manually maintain dependencies, there is also a zip distribution available that contains everything to compile and run JUnit 5.
At development time it is sufficient to depend on the org.junit:junit5-api module.
Note that when specifying the snapshot repository it should be configured to never cache artifacts so that always the latest version is used.
Cutting Loose from JUnit 4
As far as I can see, the new version is a complete rewrite of the library with no dependencies whatsoever on older versions. Thus you can enjoy legacy free testing (for a while at least;-).
But of course there is a migration path that allows for both versions to coexist and will enable you to maintain the existing test code base while writing new tests with JUnit 5. More on this later.
Same but Different
But let’s finally look at how JUnit 5 tests look like. At first sight, not much has changed. A simple test class …
class FirstTests {
@Test
void firstTest() {
fail();
}
}… is barely distinguishable from a JUnit 4 test.
But did you spot the little difference? Right, tests don’t need to be public any more, but if you prefer still can be of course.
Though annotations are still used to identify methods to set up and tear down the test environment, their names have changed. What was @BeforeClass/AfterClass is now @BeforeAll/AfterAll and @Before/After are now named @BeforeEach/AfterEach.
Ignoring tests is also still possible with the @Disabled annotation.
@Test vs. @Test
As you have seen already, tests are still tagged with the @Test annotation. But be careful if you happen to also have JUnit 4 on your class path. JUnit 5 brings its own @Test annotation, thus make sure to import org.junit.gen5.api.Test which is the right one. Otherwise the JUnit 5 test runner won’t find your tests.
Another thing to note is that the new @Test annotation does not offer other services. If you were used to use timeout or expected from time to time, you will need to replace them in JUnit 5.
Running Tests with JUnit 5
It’s no wonder that there is no IDE support yet to run JUnit 5 tests. Therefore I used the ConsoleRunner to execute my experiments. Three more modules are required to run tests this way:
org.junit:junit5-engineorg.junit:junit-launcherorg.junit:junit-console
My IDE of choice is Eclipse, and in order to run tests with the ConsoleRunner from there I had to manually extend the Classpath of the launch configuration. Only after adding the test-classes output folder that contains the compiled tests, they would be picked up. But this quirk may as well be due to my meager Maven knowledge or due to a particularity in the Eclipse Maven integration.
The JUnit 5 team also provides basic plug-ins to execute tests in Maven and Gradle builds. See the Build Support chapter if you want to given them a try.
Assertions
At first sight, assertions haven’t changed much, except that they are now homed in the org.junit.gen5.api.Assertions class.
But a closer look reveals that assertThat() is gone, and with it the unfortunate dependency on Hamcrest. These methods actually duplicated API provided by MatcherAssert and tied previous versions of JUnit to the Hamcrest library. This dependency led occasionally to class resolution conflicts. In particular when used with other libraries, that – even worse – include a copy of Hamcrest on their own.
Another change is the new assertAll() method that is meant to group assertions. For example
assertAll( "names", () -> {
assertEquals( "John", person.getFirstName() );
assertEquals( "Doe", person.getLastName() );
} );will report a MultipleFailuresError containing all failed assertions within the group.
It is then the test executors responsibility to display this failure in a suitable way. The current ConsoleRunner implementation however doesn’t yet regard grouped failures and simply reports the first one:
Finished: testNames [junit5:com...GroupAssertionsTest#testNames()]
=> Exception: names (1 failure)
expected: <John> but was: <Mary>My first, unfiltered thought was that if grouping assertions were needed it might be a sign to divide the code into multiple tests instead. But I haven’t used grouped assertions yet for real and there may as well be places where they perfectly make sense.
Testing Exceptions
Testing exceptions has been unified. To replace expected and ExpectedException there is now an expectThrows assertion that evaluates a lambda expression and verifies that it throws an exception of the given type.
For example,
@Test
void testException() {
Foo foo = new Foo();
Throwable exception = expectThrows( IllegalStateException.class, foo::bar );
assertEquals( "some message", exception.getMessage() );
}… will fail if calling foo::bar() does not throw an IllegalStateException. Otherwise the thrown exception will be returned and can be further verified. If the thrown exception is of no interest, there is also an assertThrows() method that returns void.
Goodbye Runner, Rule and ClassRule
JUnit 5 doesn’t know runners, rules, or class rules any more. These partially competing concepts have been replaced by a single consistent extension model.
Extensions can be used declaratively by annotating a test class or test method with @ExtendWith. For example a test that wishes to have some fields initialized with mocked instances could use a Mockito extension like this:
@ExtendWith(MockitoExtension.class)
class MockedTest {
@Mock
Person person;
// ...
}If you are interested in more on this topic, stay tuned for a separate post about extensions and how to migrate existing rules to custom extensions that I am planning to write.
Test Method Parameters
In JUnit 5, methods are now permitted to have parameters. This allows to inject dependencies at method level.
In order to provide a parameter, a so-called resolver is necessary, an extension that implements MethodParameterResolver. Like with all other extensions, to use a resolver for a given method or class it needs to be declared with @ExtendWith. There are also two built-in resolver that don’t need to be explicitly declared. They supply parameters of type TestInfo and TestReporter.
For example:
class MethodParametersTest {
@Test
// implicitly uses TestInfoParameterResolver to provide testInfo
void testWithBuiltIntParameterResolver( TestInfo testInfo ) {
// ...
}
@Test
@ExtendWith( CustomEnvironmentParameterResolver.class )
// explicit resolver declared, could also be placed at class level
void testWithCustomParameterResolver( CustomEnvironment environment ) {
// ...
}
}If no matching parameter resolver can be found at runtime, the engine fails the test with a corresponding message.
The documentation states that there are plans to provide additional extensions, also one for dynamic test registration among them. With this extension in place it would be possible to have parameterized tests. And given that test methods already accept parameters it seems likely that parameterized tests will also work at method level.
Backwards Compatibility
To bridge the gap until IDEs support JUnit 5 natively there is a JUnit 4 Runner that is able to execute tests written for JUnit 5. Use the @RunWith(JUnit5.class) annotation to run test classes and test suites.
Through this runner, is is possible to run JUnit 4 and 5 tests classes side by side. What is certainly out of scope is mixing old and new concepts in a single test, for example having @Rules coexist with @ExtendWith or the like.
Test utilities like Mockito and AssertJ, will continue to work with the new version without change. They interact with JUnit by raising an exception which is still considered a test failure, even in JUnit 5 :)
Open Test Alliance for the JVM
The JUnit Lambda team has also started the Open Test Alliance for the JVM with the goal to establish a standard that facilitates the interaction between test frameworks, assertion libraries, mock libraries, build tools, and IDEs.
The primary goal is to provide a library that defines a common set of exceptions to be used by testing frameworks (e.g. JUnit, TestNG, Spock, etc.) as well as assertion libraries. Build tools and IDEs would also benefit in that they could rely on the same set of types regardless of the test framework.
A draft implementation is available in the form of the org.opentest4j library, which is – you guess it – used by JUnit 5.
Outlook
My impression is that basic concepts of the new version are established. Things like @Test, set up and tear down annotations, the concept of a single extension model will probably remain in their current shape.
But many details seems to be unresolved yet and APIs are likely to change, which I think is quite understandable at this stage in the development cycle. Each part of the API is tagged with an @API annotation that indicates how stable it is.
If this post caught your interest and you may want to browse the documentation for more, there is plenty more to explore, for example:
- Tags to filter test execution
- Nested tests to group test and express relations among groups of tests
- Ideas for extensible validation rules for tests (e.g.
@Testand@BeforeEachshould not occur on the same method) - Dynamically register tests at runtime
- Annotations to aid with running tests in parallel
The first milestone is planned to be due by the end of Q1 2016. A tentative list of items to be addressed in this release is available here.
| Reference: | JUnit 5 – A First Look at the Next Generation of JUnit from our JCG partner Rudiger Herrmann at the Code Affine blog. |
