1. Introduction
Since Kent Beck coined the idea of test-driven development (TDD) more than a decade ago, testing became an absolutely essential part of every software project which aims for success. Years passed, the complexity of the software systems has grown enormously so did the testing techniques but the same foundational principles are still there and apply.
Efficient and effective testing is a very large subject, full of opinions and surrounded by never ending debates of Dos and Don’ts. Many think about testing as an art, and for good reasons. In this part of the tutorial we are not going to join any camps but instead focus on testing the applications which are implemented after the principles of the microservice architecture. Even in such narrowed subject, there are just too many topics to talk about so the upcoming parts of the tutorial will be dedicated to performance and security testing respectively.
Table Of Contents
But before we start off, please take some time to go over Marin Fowler’s Testing Strategies in a Microservice Architecture, the brilliant, detailed and well-illustrated summary of the approaches to manage the testing complexity in the world of microservices.
2. Unit Testing
Unit testing is the probably the simplest, yet very powerful, form of testing which is not really specific to the microservices but any class of applications or services.
A unit test exercises the smallest piece of testable software in the application to determine whether it behaves as expected. – https://martinfowler.com/articles/microservice-testing/#testing-unit-introduction
Unit tests usually should constitute the largest part of the application test suite (as per practical test pyramid) since they supposed to be very easy to write and fast to execute. In Java, JUnit framework (JUnit 4 and JUnit 5) is the de-facto pick these days (although other frameworks like TestNG or Spock are also widely used).
What could be a good example of unit test? Surprisingly, it is very difficult question to answer, but there are a few rules to follow: it should test one specific component (“unit”) in isolation, it should test one thing at a time and it should be fast.
There are many unit tests which come as part of service test suites of the JCG Car Rentals platform. Let us pick the Customer Service and take a look on the fragment of the test suite for AddressToAddressEntityConverter class, which converts the Address data transfer object to corresponding JPA persistent entity.
public class AddressToAddressEntityConverterTest {
private AddressToAddressEntityConverter converter;
@Before
public void setUp() {
converter = new AddressToAddressEntityConverter();
}
@Test
public void testConvertingNullValueShouldReturnNull() {
assertThat(converter.convert(null)).isNull();
}
@Test
public void testConvertingAddressShouldSetAllNonNullFields() {
final UUID uuid = UUID.randomUUID();
final Address address = new Address(uuid)
.withStreetLine1("7393 Plymouth Lane")
.withPostalCode("19064")
.withCity("Springfield")
.withStateOrProvince("PA")
.withCountry("United States of America");
assertThat(converter.convert(address))
.isNotNull()
.hasFieldOrPropertyWithValue("uuid", uuid)
.hasFieldOrPropertyWithValue("streetLine1", "7393 Plymouth Lane")
.hasFieldOrPropertyWithValue("streetLine2", null)
.hasFieldOrPropertyWithValue("postalCode", "19064")
.hasFieldOrPropertyWithValue("city", "Springfield")
.hasFieldOrPropertyWithValue("stateOrProvince", "PA")
.hasFieldOrPropertyWithValue("country", "United States of America");
}
}
The test is quite straightforward, it is easy to read, understand and troubleshoot any failures which may occur in the future. In real projects, the unit tests may get out of control very fast, become bloated and difficult to maintain. There is no universal treatment for such disease, but the general advice is to look at the test cases as the mainstream code.
3. Integration Testing
In reality, the components (or “units”) in our applications often have dependencies on other components, data storages, external services, caches, message brokers, … Since unit tests are focusing on isolation, we need to go up one level and switch over to integration testing.
An integration test verifies the communication paths and interactions between components to detect interface defects. – https://martinfowler.com/articles/microservice-testing/#testing-integration-introduction
Probably the best example to demonstrate the power of the integration testing is to come up with the suite to test the persistence layer. This is the area where frameworks like Arquillian, Mockito, DBUnit, Wiremock, Testcontainers, REST Assured (and many others) take the lead.
Let us get back to the Customer Service and think about how to ensure that the customer data is indeed persistent in the database. We have a dedicated RegistrationService to manage the registration process, so what we need is to provide the database instance, wire all the dependencies and initiate the registration process.
@RunWith(Arquillian.class)
public class TransactionalRegistrationServiceIT {
@Inject private RegistrationService service;
@Deployment
public static JavaArchive createArchive() {
return ShrinkWrap
.create(JavaArchive.class)
.addClasses(CustomerJpaRepository.class, PersistenceConfig.class)
.addClasses(ConversionService.class, TransactionalRegistrationService.class)
.addPackages(true, "org.apache.deltaspike")
.addPackages(true, "com.javacodegeeks.rentals.customer.conversion")
.addPackages(true, "com.javacodegeeks.rentals.customer.registration.conversion");
}
@Test
public void testRegisterNewCustomer() {
final RegisterAddress homeAddress = new RegisterAddress()
.withStreetLine1("7393 Plymouth Lane")
.withPostalCode("19064")
.withCity("Springfield")
.withCountry("United States of America")
.withStateOrProvince("PA");
final RegisterCustomer registerCustomer = new RegisterCustomer()
.withFirstName("John")
.withLastName("Smith")
.withEmail("john@smith.com")
.withHomeAddress(homeAddress);
final UUID uuid = UUID.randomUUID();
final Customer customer = service.register(uuid, registerCustomer);
assertThat(customer).isNotNull()
.satisfies(c -> {
assertThat(c.getUuid()).isEqualTo(uuid);
assertThat(c.getFirstName()).isEqualTo("John");
assertThat(c.getLastName()).isEqualTo("Smith");
assertThat(c.getEmail()).isEqualTo("john@smith.com");
assertThat(c.getBillingAddress()).isNull();
assertThat(customer.getHomeAddress()).isNotNull()
.satisfies(a -> {
assertThat(a.getUuid()).isNotNull();
assertThat(a.getStreetLine1()).isEqualTo("7393 Plymouth Lane");
assertThat(a.getStreetLine2()).isNull();
assertThat(a.getCity()).isEqualTo("Springfield");
assertThat(a.getPostalCode()).isEqualTo("19064");
assertThat(a.getStateOrProvince()).isEqualTo("PA");
assertThat(a.getCountry()).isEqualTo("United States of America");
});
});
}
}
This is an Arquillian-based test suite where we have configured in-memory H2 database engine in PostgreSQL compatibility mode (through the properties file). Even in this configuration it may take up to 15-25 seconds to run, still much faster than spinning the dedicated instance of PostgreSQL database.
Trading integration tests execution time by substituting the integration components is one of the viable techniques to obtain feedback faster. It certainly may not work for everyone and everything so we will get back to this subject later on in this part of the tutorial.
If your microservices are built on top of Spring Framework and Spring Boot, like for example our Reservation Service, you would definitely benefit from auto-configured test slices and beans mocking. The snippet below, part of the ReservationController test suite, illustrate the usage of the @WebFluxTest test slice in action.
@WebFluxTest(ReservationController.class)
class ReservationControllerTest {
private final String username = "b36dbc74-1498-49bd-adec-0b53c2b268f8";
private final UUID customerId = UUID.fromString(username);
private final UUID vehicleId = UUID.fromString("397a3c5c-5c7b-4652-a11a-f30e8a522bf6");
private final UUID reservationId = UUID.fromString("3f8bc729-253d-4d8f-bff2-bc07e1a93af6");
@Autowired
private WebTestClient webClient;
@MockBean
private ReservationService service;
@MockBean
private InventoryServiceClient inventoryServiceClient;
@Test
@DisplayName("Should create Customer reservation")
@WithMockUser(roles = "CUSTOMER", username = username)
public void shouldCreateCustomerReservation() {
final OffsetDateTime reserveFrom = OffsetDateTime.now().plusDays(1);
final OffsetDateTime reserveTo = reserveFrom.plusDays(2);
when(inventoryServiceClient.availability(eq(vehicleId)))
.thenReturn(Mono.just(new Availability(vehicleId, true)));
when(service.reserve(eq(customerId), any()))
.thenReturn(Mono.just(new Reservation(reservationId)));
webClient
.mutateWith(csrf())
.post()
.uri("/api/reservations")
.accept(MediaType.APPLICATION_JSON_UTF8)
.contentType(MediaType.APPLICATION_JSON_UTF8)
.body(BodyInserters
.fromObject(new CreateReservation()
.withVehicleId(vehicleId)
.withFrom(reserveFrom)
.withTo(reserveTo)))
.exchange()
.expectStatus().isCreated()
.expectBody(Reservation.class)
.value(r -> {
assertThat(r)
.extracting(Reservation::getId)
.isEqualTo(reservationId);
});
}
}
To be fair, it is amazing to see how much efforts and thoughts the Spring team invests into the testing support. Not only we are able to cover the most of the request and response processing without spinning the server instance, the test execution time is blazingly fast.
Another interesting concept you will often encounter, specifically in integration testing, is using fakes, stubs, test doubles and/or mocks.
4. Testing Asynchronous Flows
It is very likely that sooner or later you may face the need to test some kind of functionality which relies on asynchronous processing. To be honest, without using dedicated dispatchers or executors, it is really difficult due to the non-deterministic nature of the execution flow.
If we rewind a bit to the moment when we discussed microservices implementation, we would run into the flow in the Customer Service which relies on asynchronous event propagation provided by CDI 2.0. How would we test that? Let us find out one of the possible ways to approach this problem by dissecting the snippet below.
@RunWith(Arquillian.class)
public class NotificationServiceTest {
@Inject private RegistrationService registrationService;
@Inject private TestNotificationService notificationService;
@Deployment
public static JavaArchive createArchive() {
return ShrinkWrap
.create(JavaArchive.class)
.addClasses(TestNotificationService.class, StubCustomerRepository.class)
.addClasses(ConversionService.class, TransactionalRegistrationService.class, RegistrationEventObserver.class)
.addPackages(true, "org.apache.deltaspike.core")
.addPackages(true, "com.javacodegeeks.rentals.customer.conversion")
.addPackages(true, "com.javacodegeeks.rentals.customer.registration.conversion");
}
@Test
public void testCustomerRegistrationEventIsFired() {
final UUID uuid = UUID.randomUUID();
final Customer customer = registrationService.register(uuid, new RegisterCustomer());
await()
.atMost(1, TimeUnit.SECONDS)
.until(() -> !notificationService.getTemplates().isEmpty());
assertThat(notificationService.getTemplates())
.hasSize(1)
.hasOnlyElementsOfType(RegistrationTemplate.class)
.extracting("customerId")
.containsOnly(customer.getUuid());
}
}
Since the event is fired and consumed asynchronously, we cannot make the assertions predictably but take into account the timing aspect by using Awaitility library. Also, we do not really need to involve the persistence layer in this test suite so we provide our own (quite dumb to be fair) StubCustomerRepository implementation to speed up test execution.
@Singleton
public static class StubCustomerRepository implements CustomerRepository {
@Override
public Optional<CustomerEntity> findById(UUID uuid) {
return Optional.empty();
}
@Override
public CustomerEntity saveOrUpdate(CustomerEntity entity) {
return entity;
}
@Override
public boolean deleteById(UUID uuid) {
return false;
}
}
Still, even with this approach there are opportunities for instability. The dedicated test dispatchers and executors may yield better results but not every framework provides them or supports easy means to plug them in.
5. Testing Scheduled Tasks
The work, which is supposed to be done (or scheduled) at specific time, poses an interesting challenge from the testing perspective. How would we make sure the schedule meets the expectations? It would be impractical (but believe it or not, realistic) to have test suites running for hours or days waiting for task to be triggered. Luckily, there are few options to consider. For application and services which use Spring Framework the simplest but quite reliable route is to use CronTrigger and mock (or stub) TriggerContext, for example.
class SchedulingTest {
private final class TestTriggerContext implements TriggerContext {
private final Date lastExecutionTime;
private TestTriggerContext(LocalDateTime lastExecutionTime) {
this.lastExecutionTime = Date.from(lastExecutionTime.atZone(ZoneId.systemDefault()).toInstant());
}
@Override
public Date lastScheduledExecutionTime() {
return lastExecutionTime;
}
@Override
public Date lastActualExecutionTime() {
return lastExecutionTime;
}
@Override
public Date lastCompletionTime() {
return lastExecutionTime;
}
}
@Test
public void testScheduling(){
final CronTrigger trigger = new CronTrigger("0 */30 * * * *");
final LocalDateTime lastExecutionTime = LocalDateTime.of(2019, 01, 01, 10, 00, 00);
final Date nextExecutionTime = trigger.nextExecutionTime(new TestTriggerContext(lastExecutionTime));
assertThat(nextExecutionTime)
.hasYear(2019)
.hasMonth(01)
.hasDayOfMonth(01)
.hasHourOfDay(10)
.hasMinute(30)
.hasSecond(0);
}
}
The test case above uses fixed CronTrigger expression and verifies the next execution time but it could be also populated from the properties or even class method annotations.
Alternatively to verifying the schedule itself, you may find it very useful to rely on virtual clock and literally “travel in time”. For example, you could pass around the instance of the Clock abstract class (the part of Java Standard Library) and substitute it with stub or mock in the tests.
6. Testing Reactive Flows
The popularity of the reactive programming paradigm has had a deep impact on the testing approaches we used to employ. In fact, testing support is the first class citizen in any reactive framework: RxJava, Project Reactor or Akka Streams, you name it.
Our Reservation Service is built using Spring Reactive stack all the way and is great candidate to illustrate the usage of dedicated scaffolding to test the reactive APIs.
@Testcontainers
@SpringBootTest(
classes = ReservationRepositoryIT.Config.class,
webEnvironment = WebEnvironment.NONE
)
public class ReservationRepositoryIT {
@Container
private static final GenericContainer<?> container = new GenericContainer<>("cassandra:3.11.3")
.withTmpFs(Collections.singletonMap("/var/lib/cassandra", "rw"))
.withExposedPorts(9042)
.withStartupTimeout(Duration.ofMinutes(2));
@Configuration
@EnableReactiveCassandraRepositories
@ImportAutoConfiguration(CassandraMigrationAutoConfiguration.class)
@Import(CassandraConfiguration.class)
static class Config {
}
@Autowired
private ReservationRepository repository;
@Test
@DisplayName("Should insert Customer reservations")
public void shouldInsertCustomerReservations() {
final UUID customerId = randomUUID();
final Flux<ReservationEntity> reservations =
repository
.deleteAll()
.thenMany(
repository.saveAll(
Flux.just(
new ReservationEntity(randomUUID(), randomUUID())
.withCustomerId(customerId),
new ReservationEntity(randomUUID(), randomUUID())
.withCustomerId(customerId))));
StepVerifier
.create(reservations)
.expectNextCount(2)
.verifyComplete();
}
}
Besides utilizing Spring Boot testing support, this test suite relies on outstanding Spring Reactor test capabilities in the form of StepVerifier where the expectations are defined in terms of events to expect on each step. The functionality which StepVerifier and family provide is quite sufficient to cover arbitrary complex scenarios.
One more thing to mention here is the usage of Testcontainers framework and bootstrapping the dedicated data storage instance (in this case, Apache Cassandra) for persistence. With that, not only the reactive flows are tested, the integration test is using the real components, sticking as close as possible to real production conditions. The price for that is higher resource demands and significantly increased time of the test suites execution.
7. Contract Testing
In a loosely coupled microservice architecture, the contracts are the only things which each service publishes and consumes. The contract could be expressed in IDLs like Protocol Buffers or Apache Thrift which makes it comparatively easy to communicate, evolve and consume. But for HTTP-based RESTful web APIs it would be more likely some form of blueprint or specification. In this case, the question becomes: how the consumer could assert the expectations against such contracts? And more importantly, how the provider could evolve the contract without breaking existing consumers?
Those are the hard problems where consumer-driven contract testing could be very helpful. The idea is pretty simple. The provider publishes the contract. The consumer creates the tests to make sure it has the right interpretation of the contract. Interestingly, the consumer may not need to use all APIs but just the subset it really needs to have the job done. And lastly, consumer communicates these tests back to provider. This last step is quite important as it helps the provider to evolve the APIs without disrupting the consumers.
In the JVM ecosystem, Pact JVM and Spring Cloud Contract are two the most popular libraries for consumer-driven contract testing. Let us take a look on how the JCG Car Rentals Customer Admin Portal may use Pact JVM to add the consumer-driven contract test for one of the Customer Service APIs using the OpenAPI specification it publishes.
public class RegistrationApiContractTest {
private static final String PROVIDER_ID = "Customer Service";
private static final String CONSUMER_ID = "JCG Car Rentals Admin";
@Rule
public ValidatedPactProviderRule provider = new ValidatedPactProviderRule(getContract(), null, PROVIDER_ID,
"localhost", randomPort(), this);
private String getContract() {
return getClass().getResource("/contract/openapi.json").toExternalForm();
}
@Pact(provider = PROVIDER_ID, consumer = CONSUMER_ID)
public RequestResponsePact registerCustomer(PactDslWithProvider builder) {
return builder
.uponReceiving("registration request")
.method("POST")
.path("/customers")
.body(
new PactDslJsonBody()
.stringType("email")
.stringType("firstName")
.stringType("lastName")
.object("homeAddress")
.stringType("streetLine1")
.stringType("city")
.stringType("postalCode")
.stringType("stateOrProvince")
.stringType("country")
.closeObject()
)
.willRespondWith()
.status(201)
.matchHeader(HttpHeaders.CONTENT_TYPE, "application/json")
.body(
new PactDslJsonBody()
.uuid("id")
.stringType("email")
.stringType("firstName")
.stringType("lastName")
.object("homeAddress")
.stringType("streetLine1")
.stringType("city")
.stringType("postalCode")
.stringType("stateOrProvince")
.stringType("country")
.closeObject())
.toPact();
}
@Test
@PactVerification(value = PROVIDER_ID, fragment = "registerCustomer")
public void testRegisterCustomer() {
given()
.contentType(ContentType.JSON)
.body(Json
.createObjectBuilder()
.add("email", "john@smith.com")
.add("firstName", "John")
.add("lastName", "Smith")
.add("homeAddress", Json
.createObjectBuilder()
.add("streetLine1", "7393 Plymouth Lane")
.add("city", "Springfield")
.add("postalCode", "19064")
.add("stateOrProvince", "PA")
.add("country", "United States of America"))
.build())
.post(provider.getConfig().url() + "/customers");
}
}
There are many ways to write the consumer-driven contract tests, above is just one favor of it. It does not matter much what approach you are going to follow, the quality of your microservice architecture is going to improve.
Pushing it further, the tools like swagger-diff, Swagger Brake and assertj-swagger are very helpful in validating the changes in the contracts (since it is a living thing in most cases) and making sure the service is properly implementing the contract it claims to.
If this is not enough, one of the invaluable tools out there is Diffy from Twitter which helps to find potential bugs in the services using running instances of new version and old version side by side. It behaves more like a proxy which routes whatever requests it receives to each of the running instances and then compares the responses.
8. Component Testing
On the top of the testing pyramid of a single microservice sit the component tests. Essentially, they exercise the real, ideally production-like, deployment with only external services stubbed (or mocked).
Let us get back to Reservation Service and walk through the component test we may come up with. Since it relies on the Inventory Service, we need to mock this external dependency. To do that, we could benefit from Spring Cloud Contract WireMock extension which, as the name implies, is based on WireMock. Besides Inventory Service we also mock the security provider by using @MockBean annotation.
@AutoConfigureWireMock(port = 0)
@Testcontainers
@SpringBootTest(
webEnvironment = WebEnvironment.RANDOM_PORT,
properties = {
"services.inventory.url=http://localhost:${wiremock.server.port}"
}
)
class ReservationServiceIT {
private final String username = "ac2a4b5d-a35f-408e-a652-47aa8bf66bc5";
private final UUID vehicleId = UUID.fromString("4091ffa2-02fa-4f09-8107-47d0187f9e33");
private final UUID customerId = UUID.fromString(username);
@Autowired private ObjectMapper objectMapper;
@Autowired private ApplicationContext context;
@MockBean private ReactiveJwtDecoder reactiveJwtDecoder;
private WebTestClient webClient;
@Container
private static final GenericContainer<?> container = new GenericContainer<>("cassandra:3.11.3")
.withTmpFs(Collections.singletonMap("/var/lib/cassandra", "rw"))
.withExposedPorts(9042)
.withStartupTimeout(Duration.ofMinutes(2));
@BeforeEach
public void setup() {
webClient = WebTestClient
.bindToApplicationContext(context)
.apply(springSecurity())
.configureClient()
.build();
}
@Test
@DisplayName("Should create Customer reservations")
public void shouldCreateCustomerReservation() throws JsonProcessingException {
final OffsetDateTime reserveFrom = OffsetDateTime.now().plusDays(1);
final OffsetDateTime reserveTo = reserveFrom.plusDays(2);
stubFor(get(urlEqualTo("/" + vehicleId + "/availability"))
.willReturn(aResponse()
.withHeader("Content-Type", "application/json")
.withBody(objectMapper.writeValueAsString(new Availability(vehicleId, true)))));
webClient
.mutateWith(mockUser(username).roles("CUSTOMER"))
.mutateWith(csrf())
.post()
.uri("/api/reservations")
.accept(MediaType.APPLICATION_JSON_UTF8)
.contentType(MediaType.APPLICATION_JSON_UTF8)
.body(BodyInserters
.fromObject(new CreateReservation()
.withVehicleId(vehicleId)
.withFrom(reserveFrom)
.withTo(reserveTo)))
.exchange()
.expectStatus().isCreated()
.expectBody(Reservation.class)
.value(r -> {
assertThat(r)
.extracting(Reservation::getCustomerId, Reservation::getVehicleId)
.containsOnly(vehicleId, customerId);
});
}
}
Despite the fact that a lot of things are happening under the hood, the test case is still looking quite manageable but the time it needs to run is close to 50 seconds now.
While designing the component tests, please keep in mind that there should be no shortcuts taken (like for example mutating the data in the database directly). If you need some prerequisites or the way to assert over internal service state, consider introducing the supporting APIs which are available at test time only (enabled, for example, using profiles or configuration properties).
9. End-To-End Testing
The purpose of end-to-end tests is to verify that the whole system works as expected and as such, the assumption is to have a full-fledge deployment of all the components. Though being very important, the end-to-end tests are the most complex, expensive, slow and, as practice shows, most brittle ones.
Typically, the end-to-end tests are designed after the workflows performed by the users, from the beginning to the end. Because of that, often the entry point into the system is some kind of mobile or web frontend so the testing frameworks like Geb, Selenium and Robot Framework are quite popular choices here.
10. Fault Injection and Chaos Engineering
It would be fair to say most of tests are biased towards a “happy path” and do not explore the faulty scenarios, unless trivial ones, like for example the record is not present in the data store or input is not valid. How often have you seen test suites which deliberately introduce database connectivity issues?
As we have stated in the previous part of the tutorial, the bad things will happen and it is better to be prepared. The discipline of chaos engineering gave the birth to many different libraries, frameworks and toolkits to perform fault injection and simulation.
To fabricate different kind of network issues, you may start with Blockade, Saboteur or Comcast, all of those are focused on network faults and partitions injection and aim to simplify resilience and stability testing.
The Chaos Toolkit is a more advanced and disciplined approach for conducting the chaos experiments. It also integrates quite well with most popular orchestration engines and cloud providers. In the same vein, the SimianArmy from Netflix is one of the earliest (if not the first) cloud-oriented tools for generating various kinds of failures and detecting abnormal conditions. For services built on top of Spring Boot stack, there is a dedicated project called Chaos Monkey for Spring Boot which you may have heard of already. It is pretty young but evolves fast and is very promising.
This kind of testing is quite new for the most organizations out there, but in the context of the microservice architecture, it is absolutely worth considering and investing. These tests give you a confidence that the system is able to survive outages by degrading the functionality gradually instead of catching the fire and burning in flames. Many organizations (like Netflix for example) do conduct chaos experiments in production regularly, proactively detecting the issues and fixing them.
11. Conclusions
In this part of the tutorial we were focused on testing. Our coverage it is far from being exhausting and complete, since there are some many different kind of tests. In many regards, testing individual microservices is not much different, the same good practices apply. But the distributed nature of such architecture brings a lot of unique challenges, which Contract Testing along with Fault Injection and Chaos Engineering are trying to address.
To finish up, the series of articles Testing Microservices, the sane way and Testing in Production, the safe way are the terrific sources of great insights and advices on what works and how to avoid the common pitfalls while testing the microservices.
12. What’s next
In the next section of the tutorial we are going to continue the subject of testing and talk about performance (load and stress) testing.
