In the last year or two, the pace of change in software architecture has rapidly advanced with new approaches like DevOps and Microservices becoming hot topics overnight.
In this post, I want to introduce you to a project that I’ve been working on which combines two of the stand out architectural advances of the last few years: Microservices and the Command and Query Responsibility Separation pattern (or CQRS for short).
The microservice sample we’re discussing here revolves around a fictitious `Product` master data application, similar to that which you’d find in most retail or manufacturing companies. Products can be added, stored, searched and retrieved using a simple RESTful API.
The source code for the application is open and available on Github. If you download it you’ll have an application that demonstrates how to implement several of the features required for ‘cloud native’ Java including:-
- Microservices that use Java and Spring Boot;
- Command and Query Responsibility Separation (CQRS) and Event Sourcing (ES) using the Axon Framework v2, MongoDB and RabbitMQ;
- Build, Ship and Run anywhere using Docker containers;
- Centralised configuration and service registration using Spring Cloud;
- plus API documentation using Swagger and SpringFox.
In this first installment, I’m going to use pre-built docker containers to show you how to get the sample code working with the minimum effort. There’s no need to write any code, you can simply try out the components for yourself then have a look at the code later. In later posts I’ll go into more detail so you can see how the system was built, piece by piece.
If you simply want to dive straight into the code, you can find it all on Github at https://github.com/benwilcock/microservice-sampler
How it works
The application is built using the CQRS architectural pattern. In CQRS commands like `ADD` are physically separated from queries like `VIEW (where id=1)`. In this example the domain’s codebase has been quite literally split into two separate components – a command-side microservice and a query-side microservice.
Like most other 12 factor apps, these microservices both have a single responsibility; feature their own datastores; and can be deployed and scaled independently of each other. This is CQRS and microservices in their most literal interpretation. Neither CQRS or microservices have to be implemented in this way, but for the purpose of this demonstration I’ve chosen to create a very clear separation of the read and write concerns.
The logical architecture looks like this:-
Both the command-side and the query-side microservices have been developed using the Spring Boot framework for Java. All communication between the command and query microservices is purely `event-driven`. The events are passed between the microservice components using RabbitMQ messaging. Messaging provides a scalable means of passing events between processes, microservices, legacy systems and other parties in a loosely coupled fashion.
Notice how neither of the services shares it’s database with the other. This is important because of the high degree of autonomy it affords each service, which in turn helps the individual services to scale independently of the others in the system. For more on CQRS architecture, check out my Slideshare on CQRS Microservices which the slide above is taken from.
More about the Command-side Microservice
Commands are “actions which change state“. The command-side microservice contains all the domain logic and business rules. Commands are used to add new Products, or to change their state. The execution of these commands on a particular Product results in `Events` being generated which are persisted by the Axon framework into MongoDB and propagated out to other processes (as many processes as you like) via RabbitMQ messaging.
In event-sourcing, events are the sole record of state for the system. They are used by the system to describe and re-build the current state of any entity on demand (by replaying it’s past events one at a time until all previous events have been re-applied). This sounds slow, but actually because events are simple, it’s really fast and can be tuned further using rollups called ‘snapshots’.
In Domain Driven Design (DDD) the entity is often referred to as an `Aggregate` or an `AggregateRoot.`
More about the Query-side Microservice
The query-side microservice acts as an event-listener and a view. It listens for the `Events` being emitted by the command-side and processes them into whatever shape makes the most sense (for example a tabular view).
In this particular example, the query-side simply builds and maintains a ‘materialised view’ or ‘projection’ which holds the latest state of the individual Products (in terms of their id and their description and whether they are saleable or not). The query-side can be replicated many times for scalability and the messages held by the RabbitMQ queues can be made to be durable, so they can even temporarily store messages on behalf of the query-side if it goes down.
The command-side and the query-side both have REST API’s which can be used to access their capabilities.
For more information, see the Axon documentation which describes how Axon brings CQRS and Event Sourcing to your Java apps as well as lots of detail on how it’s configured and used.
Running the Demo
Running the demo code is easy, but you’ll need to have the following software installed on your machine first. For reference I’m using Ubuntu 16.04 as my OS.
- Docker (I’m using v1.8.2)
- Docker-compose (I’m using v1.7.1)
If you have both of these, you can run the demo by following the process outlined below.
If you have either MongoDB or RabbitMQ already, please shut down those services before continuing in order to avoid a port availability clash.
Step 1: Get the Docker-compose configuration file
In a new empty folder, at the terminal execute the following command to download the latest docker-compose configuration file for this demo.
$ wget https://raw.githubusercontent.com/benwilcock/microservice-sampler/master/docker-compose.yml
Try not to change the file’s name – Docker defaults to looking for a file called ‘docker-compose.yml’. If you do change the name, use the -f switch in the following step.
Step 2: Start the Microservices
Because we’re using docker-compose, starting the microservices is now simply a case of executing the following command.
$ docker-compose up
You’ll see lots of downloading and logging output in the terminal window as the docker images are downloaded and run.
There are six docker images in total, they are ‘mongodb’, ‘rabbitmq’, ‘config’, ‘discovery’, ‘product-cmd-side’, and ‘product-qry-side’.
If you want to see which docker instances are running on your machine at any time, open a separate terminal and execute the following command:-
$ docker ps
Once the instances are up and running (this can take some time at first) you can have a look around immediately using your browser. You should be able to access:-
- The Rabbit Management Console on port `15672`
- The Eureka Discovery Server Console on port `8761`
- The Configuration Server mappings on port `8888`
- The Product Command Side Swagger API Docs on port `9000`
- An empty Product view on the query-side on port `9001`
Step 3: Working with Products
So far so good. Now we want to test the addition of products. In this manual system test we’ll issue an `add` command to the command-side REST API.
When the command-side has processed the command a ‘ProductAdded’ event is raised, stored in MongoDB, and forwarded to the query-side via RabbitMQ. The query-side then processes this event and adds a record for the product to it’s materialised-view (actually a H2 in-memory database for this simple demo). Once the event has been processed we can use the query-side microservice to lookup information regarding the new product that’s been added. As you perform these tasks, you should observe some logging output in the docker-compose terminal window.
Step 3.1: Add A New Product
To perform test this we first need to open a second terminal window from where we can issue some CURL commands without stopping the docker composed instances we have running in the first window.
For the purposes of this test, we’ll add an MP3 product to our product catalogue with the name ‘Everything is Awesome’. To do this we can use the command-side REST API and issue it with a POST request as follows…
$ curl -X POST -v --header "Content-Type: application/json" --header "Accept: */*" "http://localhost:9000/products/add/1?name=Everything%20Is%20Awesome"
If you don’t have ‘CURL’ available to you, you can use your browser to add a product by navigating to the simple web-form that comes with the command-side’s Swagger documentation.
You should see the following response.
* Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 9000 (#0) > POST /products/add/1?name=Everything%20Is%20Awesome HTTP/1.1 > Host: localhost:9000 > User-Agent: curl/7.47.0 > Content-Type: application/json```bash > Accept: */*$ http://localhost:9000/products/1 < HTTP/1.1 201 Created < Date: Thu, 02 Jun 2016 13:37:07 GMTThis < X-Application-Context: product-command-side:9000 < Content-Length: 0 < Server: Jetty(9.2.16.v20160414)
The response code should be < HTTP/1.1 201 Created.` This means that the MP3 product “Everything is Awesome” has been added to the command-side event-sourced repository successfully.
Step 3.2: Query for the new Product
Now lets check that we can view the product that we just added. To do this we use the query-side API on port `9001` and issue a simple ‘GET’ request.
$ curl http://localhost:9001/products/1
You should see the following output. This shows that the query-side microservice has a record for our newly added MP3 product. The product is listed as non-saleable (saleable = false).
{
name: "Everything Is Awesome",
saleable: false,
_links: {
self: {
href: "http://localhost:9001/products/1"
},
product: {
href: "http://localhost:9001/products/1"
}
}
}That’s it! Go ahead and repeat the test to add some more products if you like, just be careful not to try to reuse the same product ID when you POST or you’ll see an error.
If you’re familiar with MongoDB you can inspect the database to see all the events that you’ve created. Similarly if you know your way around the RabbitMQ Management Console you can see the messages as they flow between the command-side and query-side microservices.
| Reference: | Microservices with Spring Boot, Axon CQRS/ES and Docker from our JCG partner Ben Wilcock at the Ben Wilcock’s blog blog. |
