Rules add special handling around tests, test cases or test suites. They can do additional validations common for all tests in the class, concurrently run multiple test instances, set up resources before each test or test case and tear them down afterwards.
The rule gets complete control over what will done with the test method, test case or test suite it is applied to. Complete control means that the rule decides what to do before and after running it and how to deal with thrown exceptions.
First chapter shows how to use rules and second shows what build-in rules can do. The third chapter describes third party rules libraries I found and the last one explains how to create new rules.
Using Rules
This chapter shows how to declare and use rules inside a test case. Most rules can be applied to each test method separately, once to the whole test case or once to the whole test suite. Rules run separately for each test are called test rules and rules applied to the whole test case or suite are called class rules.
We will use temporary folder rule as an example, so first subchapter explains what it does. Second subchapter declares it as test rule and third one as class rule. Last subchapter shows how to access the folder from inside the tests.
Example Rule – Temporary Folder
Temporary folder rule creates new empty folder, runs test or test case and then deletes the folder. You can either specify where to create the new folder, or let it be created in system temporary file directory.
Temporary folder can be used as both test rule and class rule.
Declaring Test Rules
Test rules e.g., rules that run for each test method separately, have to be declared in public field annotated with @Rule annotation.
Declare test rule:
public class SomeTestCase {
@Rule
public TemporaryFolder folder = new TemporaryFolder();
}The above folder rule creates new folder before every test method and destroys it afterwards. All tests are able to use that directory, but they are not able to share files through it. Since we used constructor with no parameters, the folder will be created in system temporary file directory.
Test rule does its work before methods annotated with @Before and after those annotated with @After. Therefore, they will have access to temporary folder too.
Declaring Class Rules
Class rules e.g., rules that run once for the whole test case or test suite, have to be declared in public static field and annotated with @ClassRule annotation.
Declare test case rule:
public class SomeTestCase {
@ClassRule
public static TemporaryFolder folder = new TemporaryFolder();
}The above folder rule creates new folder before running the first test method and destroys it after the last one. All tests are able to use that directory and they are able to see files created be previously running tests.
Class rules are run before anything inside that class. E.g. methods annotated with @BeforeClass or @AfterClass will have access to temporary folder too. The rule runs before and after them.
Using Rules Inside Tests
Rules are classes as any other and tests are free to call their public methods and use their public fields. Those calls are used to add test specific configuration to the rule or read data out of it.
For example, temporary folder can be accessed using newFile, newFolder or getRoot methods. First two create new file or folder inside the temporary folder and the getRoot method returns temporary folder itself.
Create temporary file and folder:
@Test
public void test1() {
// Create new folder inside temporary directory. Depending on how you
// declared the folder rule, the directory will be deleted either
// right after this test or when the last test in test case finishes.
File file = folder.newFolder("folder");
}
@Test
public void test2() {
// Create new file inside temporary folder. Depending on how you
// declared the folder rule, the file will be deleted either
// right after this test or when the last test in test case finishes.
File file = folder.newFile("file.png");
}Default Rules
JUnit comes with five directly useable rules: temporary folder, expected exception, timeout, error collector and test name. Temporary folder have been explained in previous chapter, so we will briefly explain only remaining four rules.
Expected Exception
Expected exception runs the test and catches any exception it throws. The rule is able to check whether the exception contains the right message, the right cause and whether it was thrown by the right line.
Expected exception has private constructor and must be initialized using static none method. Each exception throwing test has to configure expected exception parameters and then call the expect method of the rule. The rule fails if:
- the test throws any exception before the
expectmethod call, - the test does not throw an exception after the
expectmethod call, - thrown exception does not have the right message, class or cause.
The last test line throws an exception. Expected exception rule is configured right before causing the exception:
@Rule
public ExpectedException thrown= ExpectedException.none();
@Test
public void testException() {
// Any exception thrown here causes failure
doTheStuff();
// From now on, the rule expects NullPointerException exception
// to be thrown. If the test finishes without exception or if it
// throws wrong one, the rule will fail.
thrown.expect(NullPointerException.class);
// We well check also message
thrown.expectMessage("Expected Message.");
// this line is supposed to throw exception
theCodeThatThrowsTheException();
}Bonus: the expected message method accepts also hamcrest matcher argument. That allows you to test the message prefix, suffix, whether it matches some regular expressions or anything else.
Timeout
The timeout rule can be used as both test rule and class rule. If it is declared as test rule, it applies the same timeout limit to each test in the class. If it is declared as class rule, it applies the timeout limit to the whole test case or test suite.
Error Collector
Error collector allows you to run multiple checks inside the test and then report all their failures at once after the test ends.
Expected-vs-actual value assertions are evaluated using the checkThat method exposed by the rule. It accepts hamcrest matcher as an argument and thus can be used to check anything.
Unexpected exceptions can be reported directly using addError(Throwable error) method. Alternatively, if you have an instance of Callable to be run, you can call it through checkSucceeds method which adds any thrown exception into errors list.
Test Name
Test name rule exposes test name inside the test. It might be useful when you need to create custom error reporting.
Third Party Rules Libraries
Rules are decoupled from the test class, so it is easy to write libraries of general purpose rules and share them between projects. This chapter describes three such libraries.
System rules is rules collection for testing code that uses java.lang.System. It is well documented, available in maven and released under Common Public License 1.0 (the same as jUnit). System rules allows you to easily:
- test content of
System.errandSystem.out, - simulate input in
System.in, - configure system properties and revert their values back,
- test
System.exit()calls – whether it was called and what return value was, - customize java
SecurityManagerand revert it back.
A big set of useful rules is available on aisrael account on github. Its documentation is somewhat limited, but you can always look at the code. All rules are released under MIT license:
- starting and stopping in-memory derby database,
- starting and stopping default java HttpServer,
- starting and stopping Jetty server,
- running stub jndi,
- some support for dbUnit tests.
Another undocumented set of rules on github. I will not list them here, because their names are self-explanatory and they do not have specified license. Look at the rules directory to see their list.
Custom Rule
This chapter shows how to create new rules. They can be implemented from scratch by implementing the TestRule interface or by extending one of two convenience classes ExternalResource and Verifier available in jUnit.
We will create a new rule from scratch and then rewrite it using ExternalResource class.
New Rule
New rule ensures that all files created by tests are properly deleted after each test finishes its work. The tests themselves have only one responsibility: report all new files using the ensureRemoval(file) method exposed by the rule.
How to declare and use the DeleteFilesRule rule:
@Rule
public DeleteFilesRule toDelete = new DeleteFilesRule();
@Test
public void example() throws IOException {
// output.css will be deleted whether the test passes, fails or throws an exception
toDelete.ensureRemoval("output.css");
// the compiler is configured to create output.css file
compileFile("input.less");
checkCorrectess("output.css");
}From Scratch
Each rule, including class rules, must implement the @TestRule interface. The interface has exactly one method:
public interface TestRule {
Statement apply(Statement base, Description description);
}Our job is to take statement supplied in the base parameter and turn it into another statement. The statement represents a set of actions e.g., test, test case or test suite to be run. It might have already been modified by other declared rules and includes before and after test or class methods.
The second description parameter describes the input statement. It can tell test class name, test name, annotations placed on it, it knows whether we are dealing with test or test suite etc. We will not need it.
We need to create a new statement which will do three things:
- Empty the list of files to be deleted.
- Run underlying test, test case or test suite represented by the
baseparameter. - Delete all files reported by tests inside previously run statement.
The statement is a class with one abstract method:
public abstract class Statement {
public abstract void evaluate() throws Throwable;
}Since underlying statement can throw an exception, the code to delete all files must run from finally block:
public class DeleteFilesRule implements TestRule {
public Statement apply(final Statement base, final Description description) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
emptyFilesList(); // clean the list of files
try {
base.evaluate(); // run underlying statement
} finally {
removeAll(); // delete all new files
}
}
};
}
}Both referenced methods emptyFilesList and removeAll are declared outside of new statement, directly inside the DeleteFilesRule class:
public class DeleteFilesRule implements TestRule {
private List<File> toDelete;
private void emptyFilesList() {
toDelete = new ArrayList<File>();
}
private void removeAll() {
for (File file : toDelete) {
if (file.exists())
file.delete();
}
}
/* ... the apply method ... */
}The last thing we need is a public method able to add files to be deleted:
public void ensureRemoval(String... filenames) {
for (String filename : filenames) {
toDelete.add(new File(filename));
}
}Full Class
public class DeleteFilesRule implements TestRule {
private List<File> toDelete;
public void ensureRemoval(String... filenames) {
for (String filename : filenames) {
toDelete.add(new File(filename));
}
}
private void emptyFilesList() {
toDelete = new ArrayList<File>();
}
private void removeAll() {
for (File file : toDelete) {
if (file.exists())
file.delete();
}
}
public Statement apply(final Statement base, final Description description) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
emptyFilesList(); // clean the list of files
try {
base.evaluate(); // run underlying statement
} finally {
removeAll(); // delete all new files
}
}
};
}
}Extending Build-in Classes
JUnit contains two convenience classes ExternalResource and Verifier meant to simplify the above process even more.
External Resource
The ExternalResource helps when you need to do some kind of preprocessing and postprocessing around the underlying test statement. If you need preprocessing, override the before method. If you need postprocessing, override the after method. The
after is called from finally block, so it will be run no matter what.
Our DeleteFilesRule could be rewritten like this:
public class DeleteFilesRule2 extends ExternalResource {
/* ... list, ensureRemoval and removeAll methods ... */
@Override
protected void before() throws Throwable {
toDelete = new ArrayList<File>();
}
@Override
protected void after() {
removeAll();
}
}Verifier
The Verifier has only one method verify to override. That method runs after the wrapped test finished its work and only if it did not thrown an exception. As the name suggests, the verifier is good if you want to run additional checks after the test.
More About jUnit
Previous post about jUnit 4 features:
| Reference: | jUnit: Rules from our JCG partner Maria Jurcovicova at the This is Stuff blog. |
