Programmatically Listen to JUnit Test Results

Sebastian DaschnerFebruary 15th, 2023Last Updated: February 14th, 2023

0 688 2 minutes read

There are many ways available to run JUnit tests, from the IDE, build tool, or development modes. To process and use the test results, however, one has to either read the output or parse result files (such as the XML surefire reports). It’s also possible to define result listeners programmatically in JUnit.

You can register a custom TestExecutionListener, that will be notified at certain events that happen during testing, such as when the whole test plan as well as individual executions start or stop. We define our own implementation of this — TestResultNotifier — that gathers and uses the overall result. This class can reside under src/test/java, as any other test class:

import org.junit.platform.engine.TestExecutionResult;
import org.junit.platform.launcher.TestExecutionListener;
import org.junit.platform.launcher.TestIdentifier;
import org.junit.platform.launcher.TestPlan;

public class TestResultNotifier implements TestExecutionListener {

    private boolean passed = true;

    @Override
    public void executionFinished(TestIdentifier testIdentifier,
            TestExecutionResult testExecutionResult) {

        if (testExecutionResult.getStatus() == TestExecutionResult.Status.FAILED)
            passed = false;
    }

    @Override
    public void testPlanExecutionFinished(TestPlan testPlan) {
        if (!passed) {
            System.out.println("!!!");
            System.out.println("Tests NOT passed!");
            System.out.println("!!!");
        } else {
            System.out.println("All tests passed!");
        }
    }

}

In order to automatically register the JUnit listener, we add a file under src/test/resources/META-INF/services/ named org.junit.platform.launcher.TestExecutionListener. The Content of the file should point to our class, so a single line containing:

com.sebastian_daschner.coffee.TestResultNotifier

Then, we can execute the tests in any way we like, and we will see the output. The following shows the Maven output for the example project:

[...]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ quarkus-playground ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.sebastian_daschner.coffee.CoffeeShopTest
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.098 s <<< FAILURE! - in com.sebastian_daschner.coffee.CoffeeShopTest
[ERROR] test  Time elapsed: 0.028 s  <<< FAILURE!
org.opentest4j.AssertionFailedError:

Expecting:
 <false>
to be equal to:
 <true>
but was not.
        at com.sebastian_daschner.coffee.CoffeeShopTest.test(CoffeeShopTest.java:18)

!!!
Tests NOT passed!
!!!
[INFO]
[INFO] Results:
[INFO]
[...]

Great, this already works. Now, you might rightfully ask what’s the point.

Extreme Feedback Device Example

Well, a System.out example of course is boring, but now we have all possibilities of the Java world to locally react to the test results, every single run. For example, for some complex UI test suites that run for a longer time, I’m using an “extreme feedback” device, such as a blinking LED (the blink(1) USB LED to be precise) to notify me.

To get such an integration, you can code some connection in Java, or also call a command line executable:

@Override
public void testPlanExecutionFinished(TestPlan testPlan) {
    String arg = passed ? "--green" : "--red";

    try {
        Process process = new ProcessBuilder("blink1-tool", arg).start();
        process.waitFor();
    } catch (IOException | InterruptedException e) {
        System.err.println("Could not execute blink1-tool");
        e.printStackTrace();
    }
}

The blink1-tool is an executable that controls the LED, and in our case it will either blink red or green. Now, the cool thing is that this reacts pretty quickly, so if you run your tests via a development mode such as quarkus:dev with continuous testing, you will see the results even before you quickly switch the window or glance at the command line. If you’re interested, I can create a short video that demos this.

You can check out the example project on GitHub.

Published on Java Code Geeks with permission by Sebastian Daschner, partner at our JCG program. See the original article here: Programmatically Listen to JUnit Test Results

Opinions expressed by Java Code Geeks contributors are their own.