All Your Tests Belong to You: Maintaining Mixed JUnit 4/JUnit 5 and Testng/JUnit 5 Test Suites
If you are seasoned Java developer who practices test-driven development (hopefully, everyone does it), it is very likely JUnit 4 has been your one-stop-shop testing toolbox. Personally, I truly loved it and still love: simple, minimal, non-intrusive and intuitive. Along with terrific libraries like Assertj and Hamcrest it makes writing test cases a pleasure.
But time passes by, Java has evolved a lot as a language, however JUnit 4 was not really up for a ride. Around 2015 the development of JUnit 5 has started with ambitious goal to become a next generation of the programmer-friendly testing framework for Java and the JVM. And, to be fair, I think this goal has been reached: many new projects adopt JUnit 5 from the get-go whereas the old ones are already in the process of migration (or at least are thinking about it).
For existing projects, the migration to JUnit 5 will not happen overnight and would probably take some time. In today’s post we are going to talk about the ways to maintain mixed JUnit 4 / JUnit 5 and TestNG / JUnit 5 test suites with a help of Apache Maven and Apache Maven Surefire plugin.
To have an example a bit more realistic, we are going to test a UploadDestination class, which basically just provides a single method which says if a particular destination scheme is supported or not:
1 2 3 4 5 6 7 8 | import java.net.URI;public class UploadDestination { public boolean supports(String location) { final String scheme = URI.create(location).getScheme(); return scheme.equals("http") || scheme.equals("s3") || scheme.equals("sftp"); }} |
The implementer was kind enough to create a suite of JUnit 4 unit tests to verify that all expected destination schemes are indeed supported.
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | import static org.junit.Assert.assertTrue;import org.junit.Before;import org.junit.Test;import org.junit.runner.RunWith;import org.junit.runners.Parameterized;import org.junit.runners.Parameterized.Parameters;@RunWith(Parameterized.class)public class JUnit4TestCase { private UploadDestination destination; private final String location; public JUnit4TestCase(String location) { this.location = location; } @Before public void setUp() { destination = new UploadDestination(); } @Parameters(name= "{index}: location {0} is supported") public static Object[] locations() { } @Test public void testLocationIsSupported() { assertTrue(destination.supports(location)); }} |
In the project build, at very least, you need to add JUnit 4 dependency along with Apache Maven Surefire plugin and, optionally Apache Maven Surefire Reporter plugin, to your pom.xml, the snippet below illustrates that.
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.13.1</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.0.0-M5</version> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-report-plugin</artifactId> <version>3.0.0-M5</version> </plugin> </plugins> </build> |
No magic here, triggering Apache Maven build would normally run all unit test suites every time.
01 02 03 04 05 06 07 08 09 10 11 12 13 | ...[INFO] -------------------------------------------------------[INFO] T E S T S[INFO] -------------------------------------------------------[INFO] Running com.example.JUnit4TestCase[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 s - in com.example.JUnit4TestCase[INFO][INFO] Results:[INFO][INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0[INFO][INFO] ------------------------------------------------------------------------... |
Awesome, let us imagine at some point another teammate happens to work on the project and noticed there are no unit tests verifying the unsupported destination schemes so she adds some using JUnit 5.
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 | package com.example;import static org.junit.jupiter.api.Assertions.assertFalse;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.params.ParameterizedTest;import org.junit.jupiter.params.provider.ValueSource;class JUnit5TestCase { private UploadDestination destination; @BeforeEach void setUp() { destination = new UploadDestination(); } @ParameterizedTest(name = "{index}: location {0} is supported") public void testLocationIsNotSupported(String location) { assertFalse(destination.supports(location)); }} |
Consequently, another dependency appears in the project’s pom.xml to bring JUnit 5 in (since its API is not compatible with JUnit 4).
1 2 3 4 5 6 7 8 | <dependencies> <dependency> <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter</artifactId> <version>5.7.0</version> <scope>test</scope> </dependency> </dependencies> |
Looks quite legitimate, isn’t it? But there is a catch … the test run results would surprise this time.
01 02 03 04 05 06 07 08 09 10 11 12 13 | ...[INFO] -------------------------------------------------------[INFO] T E S T S[INFO] -------------------------------------------------------[INFO] Running com.example.JUnit5TestCase[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.076 s - in com.example.JUnit5TestCase[INFO][INFO] Results:[INFO][INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0[INFO][INFO] ------------------------------------------------------------------------... |
The JUnit 4 test suites are gone and such a behavior is actually well documentented by Apache Maven Surefire team in the Provider Selection section of the official documentation. So how we could get them back? There are a few possible options but the simplest one by far is to use JUnit Vintage engine in order to run JUnit 4 test suites using JUnit 5 platform.
01 02 03 04 05 06 07 08 09 10 11 12 | <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.0.0-M5</version> <dependencies> <dependency> <groupId>org.junit.vintage</groupId> <artifactId>junit-vintage-engine</artifactId> <version>5.7.0</version> </dependency> </dependencies> </plugin> |
With that, both JUnit 4 and JUnit 5 test suites are going to be executed side by side.
01 02 03 04 05 06 07 08 09 10 11 12 13 | [INFO] -------------------------------------------------------[INFO] T E S T S[INFO] -------------------------------------------------------[INFO] Running com.example.JUnit5TestCase[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 s - in com.example.JUnit5TestCase[INFO] Running com.example.JUnit4TestCase[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 s - in com.example.JUnit4TestCase[INFO] [INFO] Results:[INFO][INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0[INFO][INFO] ------------------------------------------------------------------------ |
The lesson to learn here: please watch carefully that all your test suites are being executed (the CI/CD usually keeps track of such trends and warns you right away). Especially, be extra careful when migrating to latest Spring Boot or Apache Maven Surefire plugin versions.
Another quite common use case you may run into is mixing the TestNG and JUnit 5 test suites in the scope of one project. The symptoms are pretty much the same, you are going to wonder why only JUnit 5 test suites are being run. The treatment in this case is a bit different and one of the options which seems to work pretty well is to enumerate the test engine providers explicitly.
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 | <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.0.0-M5</version> <dependencies> <dependency> <groupId>org.apache.maven.surefire</groupId> <artifactId>surefire-junit-platform</artifactId> <version>3.0.0-M5</version> </dependency> <dependency> <groupId>org.apache.maven.surefire</groupId> <artifactId>surefire-testng</artifactId> <version>3.0.0-M5</version> </dependency> </dependencies> </plugin> |
The somewhat undesired effect in this case is the fact that the test suites are run separately (there are other ways though to try out), for example:
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | [INFO] -------------------------------------------------------[INFO] T E S T S[INFO] -------------------------------------------------------[INFO] Running com.example.JUnit5TestCase[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.074 s - in com.example.JUnit5TestCase[INFO] [INFO] Results:[INFO][INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0[INFO][INFO] [INFO] -------------------------------------------------------[INFO] T E S T S[INFO] -------------------------------------------------------[INFO] Running TestSuite[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 s - in TestSuite[INFO] [INFO] Results:[INFO]INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0[INFO][INFO] [INFO] ------------------------------------------------------------------------ |
To be fair, I think JUnit 5 is a huge step forward towards having modern and concise test suites for your Java (and in general, JVM) projects. These days there are seamless integrations available with mostly any other test framework or library (Mockito, TestContainers, … ) and the migration path is not that difficult in most cases. Plus, as you have seen, co-existence of JUnit 5 with older test engines is totally feasible.
As always, the complete project samples are available on Github: JUnit 4/JUnit 5, TestNG / JUnit 5.
| Published on Java Code Geeks with permission by Andrey Redko, partner at our JCG program. See the original article here: All Your Tests Belong to You: Maintaining Mixed JUnit 4/JUnit 5 and Testng/JUnit 5 Test Suites Opinions expressed by Java Code Geeks contributors are their own. |
