Before we start
Some time ago I wrote a tutorial on Spring Integration to demonstrate how to use Spring Integration in sample application inspired by real-world invoices processing system. I got quite positive feedback on that one so I decided that I will show you how can we build very same application using Apache Camel – the greatest competitor of Spring Integration.
The application that we are about to build will be pretty much the same and so will be lot of the text in this tutorial. The new parts will mostly be focusing on the Apache Camel and its usage.
You can either follow this tutorial step-by-step and create application from the scratch yourself, or you can go ahead and get the code from github: DOWNLOAD SOURCES HERE: https://github.com/vrto/apache-camel-invoices
Please note that there are more ways how to write Camel “routes” – Java DSL, Spring XML, Scala DSL … In this tutorial I will be using Java DSL, but for those interested – you can also find same project configured via Spring XML here. Whichever way you prefer, it’s time to get started!
Application for processing invoices – functional description
Imagine that you’re working for some company that periodically receives a large amount of invoices from various contractors. We are about to build a system that will be able to receive invoices, filter out relevant ones, create payments (either local or foreign) and send them to some banking service. Even though the system will be rather naive and certainly not enterprise-ready, we will try to build it with good scalability, flexibility and decoupled design in the mind.
Apache Camel is an integration framework – it means that it provides meaningful abstractions for the complex systems you’re integrating.
There are plenty of important concepts to understand for this tutorial. Let me sum them up for you:
- Camel Context: Runtime system that keeps all pieces together.
- Message: Fundamental entity – core principle of messaging. It consists of Headers, Body and Attachments.
- Exchange: Container for messages (abstraction of what is actually sent over the system). It contains In and optionally also Out message.
- Route: Chain of processors. If you like more “academic” explanation then route is a graph, where “node” is represented by some Processor and “line” is represented by some Channel.
- Processor: Uses/modifies incoming exchange. Output of one processor is connected to the input of another one.
- Endpoint: Models end of channel. Configured using URI. Example:
file:data/inbox?delay=5000
- Component: Factory for endpoints. Referred with prefixes (jms:, file:, etc.).
If you want to know more details about core camel concepts I highly encourage you to read very good book Camel In Action.
Now that we have some understanding of basic concepts, let’s take a look at the following picture which is a summary of the system, and walk over important pieces:
On the picture you can see a route that illustrates our messaging structure and core components of the system – they are marked with red numbers. Let’s walk over those (we will get back to each component in more detail later):
- Invoices Gateway – this is the place where we will put new invoices so they can enter the messaging layer.
- Splitter – the system is designed to accept a collection of invoices, but we will need to process each invoice individually. More specifically, message with body of Collection type will be split to the multiple messages, where each message will have individual invoice as a body.
- Filter – Our system is designed to automatically process only those invoices that issue less than $10,000.
- Content-based router – Some invoices use IBAN account numbers and we have two different accounts – one for the local transactions and one for the foreign transactions. The job of a router component is to send a message carrying invoice to the correct channel – either for local invoices, or for the foreign invoices.
- Transformers – While we accept Invoices in to the system, our banking APIs work with other types – Payments. Job of the transformer component is to take some message and transform it to another message according to provided logic. We want to transform the payload of original message (invoice) to the new payload – payment.
- Banking Service Activator – After we have processed invoices and generated some actual payments we’re ready to talk to the external banking system. We have exposed service of such systems and when message carrying payment enters the correct (banking) channel, we want to activate some logic – passing the payment to the bank and let the bank do further processing.
Creating the project
By now you should have a high level overview of what the system does and how is it structured. Before we start coding you will need an actual Maven project and set up the structure and required dependencies. If you’re familiar with Maven then see pom.xml file below, else if you want to save some time you’re welcome to use a project template I’ve created for you: download the Maven project template.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>apache-camel-invoices</groupId>
<artifactId>apache-camel-invoices</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<camel-version>2.12.0</camel-version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>${camel-version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring</artifactId>
<version>${camel-version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-quartz</artifactId>
<version>2.9.5</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-test-spring</artifactId>
<version>${camel-version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>14.0.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<!--
- uncomment when InvoicesApplication class will exist
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<configuration>
<mainClass>com.vrtoonjava.invoices.InvoicesApplication</mainClass>
</configuration>
</plugin>
-->
</plugins>
</build>
</project>Bootstraping Camel
Before using Apache Camel, we need to bootstrap it a little. First, we will create package com.vrtoonjava.routes where we will put our route configurations. Second, we will create class InvoicesRouteBuilder which will be the place for configuring our routes. So create class just like this:
package com.vrtoonjava.routes;
import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;
@Component
public class InvoicesRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
//TODO configure route
}
}Last thing to do is plug Camel to Spring (Camel will be using Spring as a registry for beans). Add file camel-config.xml to your src/main/resources folder with following contents:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://camel.apache.org/schema/spring"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<camelContext xmlns="http://camel.apache.org/schema/spring">
<routeBuilder ref="invoicesRouteBuilder"/>
</camelContext>
</beans>Let’s now walk over the six major components of the system in more details and get hands on the actual code.
1. Invoices Gateway
First, let’s see the code for Invoice – which will be one of the core classes in our system. I will be using package com.vrtoonjava as root package, and invoices and banking as sub-packages:
package com.vrtoonjava.invoices;
import com.google.common.base.Objects;
import java.math.BigDecimal;
public class Invoice {
private final String iban;
private final String address;
private final String account;
private final BigDecimal dollars;
public Invoice(String iban, String address, String account, BigDecimal dollars) {
this.iban = iban;
this.address = address;
this.account = account;
this.dollars = dollars;
}
public boolean isForeign() {
return null != iban && !iban.isEmpty();
}
public String getAddress() {
return address;
}
public String getAccount() {
return account;
}
public BigDecimal getDollars() {
return dollars;
}
public String getIban() {
return iban;
}
@Override
public String toString() {
return Objects.toStringHelper(this)
.add("iban", iban)
.add("address", address)
.add("account", account)
.add("dollars", dollars)
.toString();
}
}Imagine that we’re getting invoices from an another system (be it database, web-service or something else), but we don’t want to couple this part to the integration layer. We will use Gateway component for that purpose. Gateway introduces a contract that decouples client code from the integration layer (Apache Camel dependencies in our case). Let’s see the code for InvoiceCollectorGateway:
package com.vrtoonjava.invoices;
import java.util.Collection;
/**
* Defines a contract that decouples client from the Apache Camel framework.
*/
public interface InvoiceCollectorGateway {
void collectInvoices(Collection<Invoice> invoices);
}We have defined this interface in order for client to depend only on the contract. Actual implementation will be Camel-specific and we can create it like this:
package com.vrtoonjava.invoices;
import org.apache.camel.Produce;
import org.apache.camel.ProducerTemplate;
import org.springframework.stereotype.Component;
import java.util.Collection;
@Component
public class CamelInvoiceCollectorGateway implements InvoiceCollectorGateway {
@Produce(uri = "seda:newInvoicesChannel")
ProducerTemplate producerTemplate;
@Override
public void collectInvoices(Collection<Invoice> invoices) {
producerTemplate.sendBody(invoices);
}
}Note the @Produce annotation. This annotation tells Camel that field producerTemplate is a producer to endpoint seda:newInvoicesChannel. When client calls collectInvoices method, gateway will send a new message (containing List body) to the seda:newInvoicesChannel channel. That leaves client decoupled from the messaging facilities, but lets us place the result to the real messaging channel.
Why SEDA?
Usually, when building messaging systems in Java, you end up using something like JMS that provides lot of useful features, like high reliability or message persistence. In this tutorial we will be using lighter alternative based on SEDA component – it allows us to utilize asynchronous in-memory messaging with pretty much zero configuration.
2. Invoices Splitter
From the Gateway we’re sending one big message to the system that contains a collection of invoices – in other words – Message has body of Collection type. As we want to process invoices individually, we will get the result from the seda:newInvoicesChannel and use a splitter component, that will create multiple messages. Each of these new messages will have a body of Invoice type. We will then place messages to the new channel – seda:singleInvoicesChannel. This is how we define splitter (add following code to the configure method of your InvoicesRouteBuilder):
from("seda:newInvoicesChannel")
.log(LoggingLevel.INFO, "Invoices processing STARTED")
.split(body())
.to("seda:singleInvoicesChannel");3. Filtering some invoices
A business use case of our system requires us to automatically process only those invoices that that issue us less than $10,000. For this purpose we will introduce a Filter component. We will grab messages from the seda:singleInvoicesChannel, apply our filtering logic on them, and then write matched results to the new seda:filteredInvoicesChannel channel. In Apache Camel you can plug in your own predicates that contain filtering logic. First, let’s define such predicate (by extending Camel’s org.apache.camel.Predicate):
package com.vrtoonjava.invoices;
import org.apache.camel.Exchange;
import org.apache.camel.Predicate;
public class LowEnoughAmountPredicate implements Predicate {
public static final int LOW_ENOUGH_THRESHOLD = 10_000;
@Override
public boolean matches(Exchange exchange) {
Invoice invoice = exchange.getIn().getBody(Invoice.class);
boolean lowEnough = invoice.getDollars().intValue() < LOW_ENOUGH_THRESHOLD;
System.out.println("Amount of $" + invoice.getDollars()
+ (lowEnough ? " can" : " can not") + " be automatically processed by system");
return lowEnough;
}
}For the sake of brevity, I am not pasting unit tests in this tutorial – but if you’re interested go ahead and download github project and see the tests for yourself.
Now we need to connect this predicate to our route, so add following code to your configure method:
from("seda:singleInvoicesChannel")
.filter(new LowEnoughAmountPredicate())
.to("seda:filteredInvoicesChannel");4. Routing invoices
So far, we’ve splitted and filtered out some invoices. Now it’s time to inspect contents of the each invoice more closely and decide, whether it is an invoice issued from the current country (local), or from an another country (foreign). Apache Camel allows use to define Content-Based Router by using choice() method in Java DSL. We can even directly access body of the message in Java DSL and perform some simple evaluations. Add following code to your configure method (and note how we are accessing body by using ${body.isForeign} standard expression):
from("seda:filteredInvoicesChannel")
.choice()
.when().simple("${body.isForeign}")
.to("seda:foreignInvoicesChannel")
.otherwise()
.to("seda:localInvoicesChannel");We will continue developing this application in the second part of this tutorial.
