Despite the existence of books and articles about JUnit testing, I still meet quite often programmers, who at most have a vague understanding of the tool and its proper usage. Hence I had the idea to write a multi-part tutorial, that explains the essentials from my point of view.
Maybe the hands-on approach taken in this mini-series might be appropriate to get one or two additional developers interested in unit testing – which would make the effort worthwhile.
Last time I introduced the very basics of a test – how it is written, executed and evaluated. While doing so I outlined that a test is more than a simple verification machine and can serve also as kind of low level specification. Therefore it should be developed with the highest possible coding standards one could think of.
This post will continue with the tutorial’s example and work out the common structure that charactarizes well written unit tests, using the nomenclature defined by Meszaros in xUnit Test Patterns [MES].
The Four Phases of a Test
A tidy house, a tidy mind
Old Adage
The tutorial’s example is about writing a simple number range counter, which delivers a certain amount of consecutive integers, starting from a given value. Beginning with the happy path the last post’s outcome was a test which verified, that the NumberRangeCounter returns consecutive numbers on subsequent invocations of the method next:
@Test
public void subsequentNumber() {
NumberRangeCounter counter = new NumberRangeCounter();
int first = counter.next();
int second = counter.next();
assertEquals( first + 1, second );
}Note that I stick with the JUnit build-in functionality for verification in this chapter. I will cover the pro and cons of particular matcher libraries (Hamcrest, AssertJ) in a separate post.
The attentive reader may have noticed that I use empty lines to separate the test into distinct segments and probably wonders why. To answer this question let us look at each of the three sections more closely:
- The first one creates an instance of the object to be tested, referred to as SUT (System Under Test). In general this section establishs the SUT’s state prior any test related activities. As this state constitutes a well defined test input, it is also denoted as fixture of a test.
- After the fixture has been established it is about time to invoke those methods of the SUT, which represent a certain behavior the test intends to verify. Often this is just a single method and the outcome is stored in local variables.
- The last section of the test is responsible to verify whether the expected outcome of a given behavior has been obtained. Although there is a school of thought propagating a one-assert-per-test policy, I prefer the single-concept-per-test idea, which means that this section is not limited to just one assertion as it happen to be in the example [MAR1].
This test structure is very common and have been described by various authors. It has been labeled as arrange, act, assert [KAC] – or build, operate, check [MAR2] – pattern. But for this tutorial I like to be precise and stick with Meszaros’ [MES] four phases called setup (1), exercise (2), verify (3) and teardown (4).
- The teardown phase is about cleaning up the fixture in case it is persistent. Persistent means the fixture or part of it would survive the end of a test and may have bad influence on the results of its successor.
Plain unit tests seldomly use persistent fixtures so the teardown phase is – as in our example – often omitted. And as it is completely irrelevant from the specification angle, we like to keep it out of the test method anyway. How this can be achieved is covered in a minute.
Due to the scope of this post I avoid a precise definition of a unit test. But I hold on to the three types of developers’ tests Tomek Kaczanowski describes in Practical Unit Testing with JUnit and Mockito and can be summarized to:
- Unit tests make sure that your code works and have to run often and therefore incredibly quickly. Which is basically what this tutorial is all about.
- Integration tests focus on the proper integration of different modules, including code over which developers have no control. This usually requires some resources (e.g. database, filesystem) and because of this the tests run more slowly.
- End-to-End tests verify that your code works from the client’s point of view and put the system as a whole to the test, mimicking the way the user would use it. They usually require a signification amount of time to execute themselves.
- And for an in-depth example of how to combine these testing types effectively you might have a look at Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce.
But before we go ahead with the example there is one question left to be discussed:
Why is this Important?
The ratio of time spent reading (code) versus writing is well over 10 to 1…
Robert C. Martin, Clean Code
The purpose of the four phases pattern is to make it easy to understand what behavior a test is verifying. Setup always defines the test’s precondition, exercise actually invokes the behavior under test, verify specifies the expected outcome and teardown is all about housekeeping, as Meszaros puts it.
This clean phase separation signals the intention of a single test clearly and increases readability. The approach implies that a test verifies only one behavior for a given input state at a time and therefore usually does without conditional blocks or the like (Single-Condition Test).
While it is tempting to avoid tedious fixture setup and test as much functionallity as possible within a single method, this usually leads to some kind of obfuscation by nature. So always remember: A test, if not written with care, can be a pain in the ass regarding maintenance and progression.
But now it is time to proceed with the example and see what this new knowledge can do for us!
Corner Case Tests
Once we are done with the happy path test(s) we continue by specifying the corner case behavior. The description of the number range counter states that the sequence of numbers should start from a given value. Which is important as it defines the lower bound (one corner…) of a counter’s range.
It seems reasonable that this value is passed as configuration parameter to the NumberRangeCounter‘s constructor. An appropriate test could verify that the first number returned by next is equal to this initialization:
@Test
public void lowerBound() {
NumberRangeCounter counter = new NumberRangeCounter( 1000 );
int actual = counter.next();
assertEquals( 1000, actual );
}Once again our test class does not compile. Fixing this by introducing a lowerBound parameter to the counter’s constructor, leads to an compile error in the subsequentNumber test. Luckily the latter test has been written to be independent from the lower bound definition, so the parameter can be used by the fixture of this test, too.
However the literal number in the test is redundant and does not indicate its purpose clearly. The latter is usually denoted as magic number. To improve the situation we could introduce a constant LOWER_BOUND and replace all literal values. Here is what the test class would look like afterwards:
public class NumberRangeCounterTest {
private static final int LOWER_BOUND = 1000;
@Test
public void subsequentNumber() {
NumberRangeCounter counter = new NumberRangeCounter( LOWER_BOUND );
int first = counter.next();
int second = counter.next();
assertEquals( first + 1, second );
}
@Test
public void lowerBound() {
NumberRangeCounter counter = new NumberRangeCounter( LOWER_BOUND );
int actual = counter.next();
assertEquals( LOWER_BOUND, actual );
}
}Looking at the code one may notice that the fixture’s in-line setup is the same for both tests. Usually an in-line setup is composed of more than a single statement, but there are often commonalities between the tests. To avoid redundancy the things in common can be delegated to a setup method:
public class NumberRangeCounterTest {
private static final int LOWER_BOUND = 1000;
@Test
public void subsequentNumber() {
NumberRangeCounter counter = setUp();
int first = counter.next();
int second = counter.next();
assertEquals( first + 1, second );
}
@Test
public void lowerBound() {
NumberRangeCounter counter = setUp();
int actual = counter.next();
assertEquals( LOWER_BOUND, actual );
}
private NumberRangeCounter setUp() {
return new NumberRangeCounter( LOWER_BOUND );
}
}While it is debatable if the delegate setup approach improves readability for the given case, it leads to an interesting feature of JUnit: the possibility to execute a common test setup implicitly. This can be achieved with the annotation @Before applied to a public, non static method that does without return value and parameters.
Which means this feature comes to a price. If we want to eliminate the redundant setUp calls within the tests we have to introduce a field that takes the instance of our NumberRangeCounter:
public class NumberRangeCounterTest {
private static final int LOWER_BOUND = 1000;
private NumberRangeCounter counter;
@Before
public void setUp() {
counter = new NumberRangeCounter( LOWER_BOUND );
}
@Test
public void subsequentNumber() {
int first = counter.next();
int second = counter.next();
assertEquals( first + 1, second );
}
@Test
public void lowerBound() {
int actual = counter.next();
assertEquals( LOWER_BOUND, actual );
}
}It is easy to see that implicit setup can remove a lot of code duplication. But it also introduces a kind of magic from the view point of a test, which can make it difficult to read. So the clear answer to the question ‘Which kind of setup type should I use?’ is: it depends…
As I usually pay attention to keep units/tests small, the trade off seems acceptable. So I often use the implicit setup to define the common/happy path input and supplement it accordingly by small in-line/delegate setup for each of the corner case tests. Otherwise as in particular beginners tend to let tests grow to large, it might be better to stick with in-line and delegate setup first.
The JUnit runtime ensures that each test gets invoked on a new instance of the test’s class. This means the constructor only fixture in our example could omit the setUp method completely. Assignment of the counter field with a fresh fixture could be done implicitly:
private NumberRangeCounter counter = new NumberRangeCounter( LOWER_BOUND );
While some people use this a lot, other people argue that a @Before annotated method makes the intention more explicit. Well, I would not go on war over this and leave the decision to your personal taste…
Implicit Teardown
Imagine for a moment that NumberRangeCounter needs to be disposed of for whatever reason. Which means we have to append a teardown phase to our tests. Based on our latest snippet this would be easy with JUnit, as it supports implicit teardown using the @After annotation. We would only have to add the following method:
@After
public void tearDown() {
counter.dispose();
}As mentioned above teardown is all about housekeeping and adds no information at all to a particular test. Because of this it is very often convenient to perform this implicitly. Alternatively one would have to handle this with a try-finally construct to ensure that teardown is executed, even if a test fails. But the latter usually does not improve readability.
Expected Exceptions
A particular corner case is testing expected exceptions. Consider for the sake of the example that NumberRangeCalculator should throw an IllegalStateException if a call of next exceeds the amount of values for a given range. Again it might be reasonable to configure the range via a constructor parameter. Using a try-catch construct we could write:
@Test
public void exeedsRange() {
NumberRangeCounter counter = new NumberRangeCounter( LOWER_BOUND, 0 );
try {
counter.next();
fail();
} catch( IllegalStateException expected ) {
}
}Well, this looks somewhat ugly as it blurs the separtion of the test phases and is not very readable. But since Assert.fail() throws an AssertionError it ensures that the test fails if no exception is thrown. And the catch block ensures that the test completes successfully in case the expected exception is thrown.
With Java 8 it is possible to write cleanly structured exception tests using lambda expressions. For more information please refer to
Clean JUnit Throwable-Tests with Java 8 Lambdas.
If it is enough to verify that a certain type of exception has been thrown, JUnit offers implicit verification via the expected method of the @Test annotation. The test above could then be written as:
@Test( expected = IllegalStateException.class )
public void exeedsRange() {
new NumberRangeCounter( LOWER_BOUND, ZERO_RANGE ).next();
}While this approach is very compact it also can be dangerous. This is because it does not distinct whether the given exception was thrown during the setup or the exercise phase of a test. So the test would be green – and hence worthless – if accidently an IllegalStateException would be thrown by the constructor.
JUnit offers a third possibility for testing expected exceptions more cleanly, the ExpectedException rule. As we have not covered Rules yet and the approach twists a bit the four phase structure, I postpone the explicit discussion of this topic to a follow-up post about rules and runners and provide only a snippet as teaser:
public class NumberRangeCounterTest {
private static final int LOWER_BOUND = 1000;
@Rule
public ExpectedException thrown = ExpectedException.none();
@Test
public void exeedsRange() {
thrown.expect( IllegalStateException.class );
new NumberRangeCounter( LOWER_BOUND, 0 ).next();
}
[...]
}However if you do not want to wait you might have a look at Rafał Borowiec‘s thorough explanations in his post JUNIT EXPECTEDEXCEPTION RULE: BEYOND BASICS
Conclusion
This chapter of JUnit in a Nutshell explained the four phase structure commonly used to write unit tests – setup, exercise, verify and teardown. It described the purpose of each phase and emphasized on how it improves readability of test cases when consistently used. The example deepened this learning material in the context of corner case tests. It was hopefully well-balanced enough to provide a comprehensible introduction without being trivial. Suggestions for improvements are of course highly appreciated.
The next chapter of the tutorial will continue the example and cover how to deal with unit dependencies and test isolation, so stay tuned.
References
- [MES] xUnit Test Patterns, Chapter 19, Four-Phase Test, Gerard Meszaros, 2007
- [MAR1] Clean Code, Chapter 9: Unit Tests, page 130 et seqq, Robert C. Martin, 2009
- [KAC] Practical Unit Testing with JUnit and Mockito, 3.9. Phases of a Unit Test, Tomek Kaczanowski, 2013
- [MAR2] Clean Code, Chapter 9: Unit Tests, page 127, Robert C. Martin, 2009
| Reference: | JUnit in a Nutshell: Test Structure from our JCG partner Frank Appel at the Code Affine blog. |
