JUnit 5 is pretty impressive, particularly when you look under the covers, at the extension model and the architecture. But on the surface, where tests are written, the development is more evolutionary than revolutionary – is there no killer feature over JUnit 4? Fortunately, there is (at least) one: parameterized tests. JUnit 5 has native support for parameterizing test methods as well as an extension point that allows third-party variants of the same theme. In this post we’ll look at how to write parameterized tests – creating an extension will be left for the future.
Overview
This post is part of a series about JUnit 5:
- Setup
- Basics
- Architecture
- Migration
- Dynamic Tests
- Parameterized Tests
- Extension Model
- Conditions
- Parameter Injection
- …
This series is based on the pre-release version Milestone 4 and will get updated when a new milestone or the GA release gets published. Another good source is the JUnit 5 user guide. You can find all code samples on GitHub.
Throughout this post I will use the terms parameter and argument quite a lot and in a way that do not mean the same thing. As per Wikipedia:
The term parameter is often used to refer to the variable as found in the function definition, while argument refers to the actual input passed.
Hello, Parameterized World
Getting started with parameterized tests is pretty easy but before the fun can begin you have to add the following dependency to your project:
- Group ID: org.junit.jupiter
- Artifact ID: junit-jupiter-params
- Version: 5.0.0-M4
Then start by declaring a test method with parameters and slap on @ParameterizedTest instead of @Test:
@ParameterizedTest
// something's missing - where does `word` come from?
void parameterizedTest(String word) {
assertNotNull(word);
}It looks incomplete – how would JUnit know which arguments the parameter word should take? Well, since you defined zero arguments for it, the method will be executed zero times and indeed JUnit reports Empty test suite for that method.
To make something happen, you need to provide arguments, for which you have various sources to pick from. Arguably the easiest is @ValueSource:
@ParameterizedTest
@ValueSource(strings = { "Hello", "JUnit" })
void withValueSource(String word) {
assertNotNull(word);
}Indeed, now the test gets executed twice: once word is “Hello”, once it is “JUnit”. In IntelliJ that looks as follows:
And that is already all you need to start experimenting with parameterized tests!
For real-life use you should know a few more things, though, about the ins and outs of @ParamterizedTest (for example how to name them), the other argument sources (including how to create your own), and about a so far somewhat mysterious feature called argument converters. We’ll look into all of that now.
Ins And Outs of Parameterized Tests
Creating tests with @ParameterizedTests is straight-forward but there are a few details that are good to know to get the most out of the feature.
Test Name
As you can tell by the IntelliJ screenshot above, the parameterized test method appears as a test container with a child node for each invocation. Those node’s name defaults to “[{index}] {arguments}” but a different one can be set with @ParameterizedTest:
@ParameterizedTest(name = "run #{index} with [{arguments}]")
@ValueSource(strings = { "Hello", "JUnit" })
void withValueSource(String word) { }An arbitrary string can be used for the tests’ names as long as it is not empty after trimming. The following placeholders are available:
- {index}: invocations of the test method are counted, starting at 1; this placeholder gets replaced with the current invocation’s index
- {arguments}: gets replaced with {0}, {1}, … {n} for the method’s n parameters (so far we have only seen methods with one parameter)
- {i}: gets replaced by the argument the i-th parameter has in the current invocation
We’ll be coming to alternative sources in a minute, so ignore the details of @CsvSource for now. Just have a look at the great test names that can be built this way, particularly together with @DisplayName:
@DisplayName("Roman numeral")
@ParameterizedTest(name = "\"{0}\" should be {1}")
@CsvSource({ "I, 1", "II, 2", "V, 5"})
void withNiceName(String word, int number) { }
Non-Parameterized Parameters
Regardless of parameterized tests, JUnit Jupiter already allows injecting parameters into test methods. This works in conjunction with parameterized tests as long as the parameters varying per invocation come first:
@ParameterizedTest
@ValueSource(strings = { "Hello", "JUnit" })
void withOtherParams(String word, TestInfo info, TestReporter reporter) {
reporter.publishEntry(info.getDisplayName(), "Word: " + word);
}Just as before, this method gets called twice and both times parameter resolvers have to provide instances of TestInfo and TestReporter. In this case those providers are built into Jupiter but custom providers, e.g. for mocks, would work just as well.
Meta Annotations
Last but not least, @ParameterizedTest (as well as all the sources) can be used as meta-annotations to create custom extensions and annotations:
@Params
void testMetaAnnotation(String s) { }
@Retention(RetentionPolicy.RUNTIME)
@ParameterizedTest(name = "Elaborate name listing all {arguments}")
@ValueSource(strings = { "Hello", "JUnit" })
@interface Params { }Argument Sources
Three ingredients make a parameterized test:
- a method with parameters
- the @ParameterizedTest annotation
- parameter values i.e. arguments
Arguments are provided by sources and you can use as many as you want for a test method but should have at least one (or the test will not get executed at all). A few specific sources exist but you are also free to create your own.
The core concepts to understand are:
- each source must provide arguments for all test method parameters (so there can’t be one source for the first and another for the second parameter)
- the test will be executed once for each group of arguments
Value Source
You have already seen @ValueSource in action. It is pretty simple to use and type safe for a few basic types. You just apply the annotation and then pick from one (and only one) of the following elements:
- String[] strings()
- int[] ints()
- long[] longs()
- double[] doubles()
Earlier, I showed that for strings – here you go for longs:
@ParameterizedTest
@ValueSource(longs = { 42, 63 })
void withValueSource(long number) { }
There are two main drawbacks:
- due to Java’s limitation on valid element types, it can not be used to provide arbitrary objects (although there is a remedy for that – wait until you read about argument converters)
- it can only be used on test methods that have a single parameter
So for most non-trivial use cases you will have to use one of the other sources.
Enum Source
This is a pretty specific source that you can use to run a test once for each value of an enum or a subset thereof:
@ParameterizedTest
@EnumSource(TimeUnit.class)
void withAllEnumValues(TimeUnit unit) {
// executed once for each time unit
}
@ParameterizedTest
@EnumSource(
value = TimeUnit.class,
names = {"NANOSECONDS", "MICROSECONDS"})
void withSomeEnumValues(TimeUnit unit) {
// executed once for TimeUnit.NANOSECONDS
// and once for TimeUnit.MICROSECONDS
}Straight forward, right? But note that @EnumSource only creates arguments for one parameter, which in conjunction with the fact that a source has to provide an argument for each parameter means that it can only be used on single-parameter methods.
Method Source
@ValueSource and @EnumSource are pretty simple and somewhat limited – on the opposite end of the generality spectrum sits @MethodSource. It simply names the methods that will be called to provide streams of arguments. Literally:
@ParameterizedTest
@MethodSource(names = "createWordsWithLength")
void withMethodSource(String word, int length) { }
private static Stream createWordsWithLength() {
return Stream.of(
ObjectArrayArguments.create("Hello", 5),
ObjectArrayArguments.create("JUnit 5", 7));
}Argument is a simple interface wrapping an array of objects and ObjectArrayArguments.create(Object… args) creates an instance of it from the varargs given to it. The class backing the annotation does the rest and this way withMethodSource gets executed twice: Once with word = “Hello” / length = 5 and once with word = “JUnit 5” / length = 7.
The method(s) named by @MethodSource must be static and can be private. They must return a kind of collection, which can be any Stream (including the primitive specializations), Iterable, Iterator, or array.
If the source is only used for a single argument, it may blankly return such instances without wrapping them in Argument:
@ParameterizedTest
@MethodSource(names = "createWords")
void withMethodSource(String word) { }
private static Stream createWords() {
return Stream.of("Hello", "Junit");
}As I said, @MethodSource is the most general source Jupiter has to offer. But it incurs the overhead of declaring a method and putting together the arguments, which is a little much for simpler cases. These can be best served with the two CSV sources.
CSV Sources
Now it gets really interesting. Wouldn’t it be nice to be able to define a handful of argument sets for a few parameters right then and there without having to go through declaring a method? Enter @CsvSource! With it you declare the arguments for each invocation as a comma-separated list of strings and leave the rest to JUnit:
@ParameterizedTest
@CsvSource({ "Hello, 5", "JUnit 5, 7", "'Hello, JUnit 5!', 15" })
void withCsvSource(String word, int length) { }In this example, the source identifies three groups of arguments, leading to three test invocations, and then goes ahead to take them apart on commas and convert them to the target types. See the single quotes in “‘Hello, JUnit 5!’, 15”? That’s the way to use commas without the string getting cut in two at that position.
That all arguments are represented as strings begs the question of how they are converted to the proper types. We’ll turn to that in a minute but before I want to quickly point out that if you have large sets of input data, you are free to store them in an external file:
@ParameterizedTest
@CsvFileSource(resources = "word-lengths.csv")
void withCsvSource(String word, int length) { }Note that resources can accept more than one file name and will process them one after another. The other elements of @CsvFileSource allow to specify the file’s encoding, line separator, and delimiter.
Custom Argument Sources
If the sources built into JUnit do not fulfill all of your use cases, you are free to create your own. I won’t go into many details – suffice it to say, you have to implement this interface…
public interface ArgumentsProvider {
Stream<? extends Arguments> provideArguments(
ContainerExtensionContext context) throws Exception;
}… and then use your source with @ArgumentsSource(MySource.class) or a custom annotation. You can use the extension context to access various information, for example the method the source is called on so you know how many parameters it has.
Now, off to converting those arguments!
Argument Converters
With the exception of method sources, argument sources have a pretty limited repertoire of types to offer: just strings, enums, and a few primitives. This does of course not suffice to write encompassing tests, so a road into a richer type landscape is needed. Argument converters are that road:
@ParameterizedTest
@CsvSource({ "(0/0), 0", "(0/1), 1", "(1/1), 1.414" })
void convertPointNorm(@ConvertPoint Point point, double norm) { }Let’s see how to get there…
First, a general observation: No matter what types the provided argument and the target parameter have, a converter will always be asked to convert from one to the other. Only the previous example declared a converter, though, so what happened in all the other cases?
Default Converter
Jupiter provides a default converter that will be used if no other was applied. If argument and parameter types match, conversion is a no-op but if the argument is a String it can be converted to a number of target types:
- char or Character if the string has length 1 (which can trip you up if you use UTF-32 characters like smileys because they consist of two Java chars)
- all of the other primitives and their wrapper types with their respective valueOf methods
- any enum by calling Enum::valueOf with the string and the target enum
- a bunch of temporal types like Instant, LocalDateTime et al., OffsetDateTime et al., ZonedDateTime, Year, and YearMonth with their respective parse methods
Here’s a simple example that shows some of them in action:
@ParameterizedTest
@CsvSource({"true, 3.14159265359, JUNE, 2017, 2017-06-21T22:00:00"})
void testDefaultConverters(
boolean b, double d, Summer s, Year y, LocalDateTime dt) { }
enum Summer {
JUNE, JULY, AUGUST, SEPTEMBER;
}It is likely that the list of supported types grows over time but it is obvious that it can not include those specific to your code base. This is where custom converters enter the picture.
Custom Converters
Custom converters allow you to convert the arguments a source emits (often strings) to instances of the arbitrary types that you want to use in your tests. Creating them is a breeze – all you need to do is implement the ArgumentConverter interface:
public interface ArgumentConverter {
Object convert(
Object input, ParameterContext context)
throws ArgumentConversionException;
}It’s a little jarring that input and output are untyped but there’s really no use in being more specific because Jupiter knows the type of neither. You can use the parameter context to get more information about the parameter you are providing an argument for, e.g. its type or the instance on which the test method will eventually be called.
For a Point class that already has a static factory method for strings like “(1/0)” the convert method is as simple as this:
@Override
public Object convert(
Object input, ParameterContext parameterContext)
throws ArgumentConversionException {
if (input instanceof Point)
return input;
if (input instanceof String)
try {
return Point.from((String) input);
} catch (NumberFormatException ex) {
String message = input
+ " is no correct string representation of a point.";
throw new ArgumentConversionException(message, ex);
}
throw new ArgumentConversionException(input + " is no valid point");
}The first check input instanceof Point is a little asinine (why would it already be a point?) but once I started switching on type I couldn’t bring myself to ignoring that case. Feel free to judge me.
Now you can apply the converter with @ConvertWith:
@ParameterizedTest
@ValueSource(strings = { "(0/0)", "(0/1)","(1/1)" })
void convertPoint(@ConvertWith(PointConverter.class) Point point) { }Or you can create a custom annotation to make it look less technical:
@ParameterizedTest
@ValueSource(strings = { "(0/0)", "(0/1)","(1/1)" })
void convertPoint(@ConvertPoint Point point) { }
@Target({ ElementType.ANNOTATION_TYPE, ElementType.PARAMETER })
@Retention(RetentionPolicy.RUNTIME)
@ConvertWith(PointConverter.class)
@interface ConvertPoint { }This means that by annotating a parameter with either @ConvertWith or your custom annotation JUnit Jupiter will pass whatever argument a source provided to your converter. You will usually apply this to sources like @ValueSource or @CsvSource, that emit strings so you can then parse them into an object of your choice.
Reflection
That was quite a ride, so let’s make sure we got everything:
- We started by adding the junit-jupiter-params artifact and applying @ParameterizedTest to test methods with parameters. After looking into how to name parameterized tests we went to discussing where the arguments come from.
- The first step is to use a source like @ValueSource, @MethodSource, or @CsvSource to create groups of arguments for the method. Each group must have arguments for all parameters (except those left to parameter resolvers) and the method will be invoked once per group. It is possible to implement custom sources and apply them with @ArgumentsSource.
- Because sources are often limited to a few basic types, the second step is to convert them to arbitrary ones. The default converter does that for primitives, enums, and some date/time types; custom converters can be applied with @ConvertWith.
This allows you to easily parameterize your tests with JUnit Jupiter!
It is entirely possible, though, that this specific mechanism does not fulfill all of your needs. In that case you will be happy to hear that it was implemented via an extension point that you can use to create your own variant of parameterized tests – I will look into that in a future post, so stay tuned.
| Reference: | JUnit 5 – Parameterized Tests from our JCG partner Nicolai Parlog at the CodeFx blog. |
