1. Introduction
In this post, we feature a comprehensive article on Microservices API Gateways and Aggregators. In the last part of the tutorial we were talking about the different means of how services in the microservices architecture discover each other. Hopefully it was a helpful discussion, but we left completely untouched the topic of how other consumers, like desktop, web frontends or mobile clients, are dealing with this kind of challenge.
The typical frontend or mobile application may need to communicate with dozens of microservices, which in case of REST(ful) service backends for example, requires the knowledge of how to locate each endpoint in question. The usage of service discovery or service registry is not practical in such circumstances since these infrastructure components should not be publicly accessible. This does not leave many options besides pre-populating the service connection details in some sort of configuration settings (usually configuration files) that the client is able to consume.
By and large, this approach works but raises another problem: the number of round-trips between clients and services is skyrocketing. It is particularly painful for the mobile clients which often communicate over quite slow and unreliable network channels. Not only that, it could be quite expensive in case of cloud-based deployments where many cloud providers charge you per number of requests (or invocations). The problem is real and needs to be addressed, but how? This is the moment where API gateway pattern appears on the stage.
There are many formal and informal definitions of what API gateway actually is, the one below is trying to encompass all aspects of it in a few sentences.
API gateway is server that acts as an API front-end, receives API requests, enforces throttling and security policies, passes requests to the back-end service and then passes the response back to the requester. A gateway often includes a transformation engine to orchestrate and modify the requests and responses on the fly. A gateway can also provide functionality such as collecting analytics data and providing caching. The gateway can provide functionality to support authentication, authorization, security, audit and regulatory compliance. https://en.wikipedia.org/wiki/API_management
Table Of Contents
A more abstract and shorter description of the API gateway definition comes from excellent blog post An API Gateway is not the new Unicorn, a highly recommended reading.
The API gateway is a way to solve the problem of how clients consume their use-cases in a microservice-based ecosystem within the microservice pattern – https://www.krakend.io/blog/what-is-an-api-gateway/
Along the rest of the tutorial we are going to talk about different kind of API gateways available in the wild and in which circumstances they can be useful.
2. Zuul 2
Zuul was born at Netflix and serves as the front door for all requests coming to their streaming backends. It is a gateway service (also sometimes called edge service) that provides dynamic routing, monitoring, resiliency, security, and a lot more. It went through a major revamp recently and has been rebranded as Zuul 2, the next generation.
Essentially, Zuul provides basic building blocks but everything else, like for example routing rules, is subject of customization through filters and endpoints abstractions. Such extensions should be implemented in Groovy, the scripting language of choice. For example, the JCG Car Rentals platform heavily uses Zuul to front the requests to all its services by providing its own inbound filter implementation.
class Routes extends HttpInboundSyncFilter {
@Override
int filterOrder() {
return 0
}
@Override
boolean shouldFilter(HttpRequestMessage httpRequestMessage) {
return true
}
@Override
HttpRequestMessage apply(HttpRequestMessage request) {
SessionContext context = request.getContext()
if (request.getPath().equals("/inventory") || request.getPath().startsWith("/inventory/")) {
request.setPath("/api" + request.getPath())
context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)
context.setRouteVIP("inventory")
} else if (request.getPath().equals("/customers") || request.getPath().startsWith("/customers/")) {
request.setPath("/api" + request.getPath())
context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)
context.setRouteVIP("customers")
} else if (request.getPath().equals("/reservations") || request.getPath().startsWith("/reservations/")) {
request.setPath("/api" + request.getPath())
context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)
context.setRouteVIP("reservations")
} else if (request.getPath().equals("/payments") || request.getPath().startsWith("/payments/")) {
request.setPath("/api" + request.getPath())
context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)
context.setRouteVIP("payments")
} else {
context.setEndpoint(NotFoundEndpoint.class.getCanonicalName())
}
return request
}
}
Zuul is very flexible and gives you a full control over the APIs management strategies. Beside many other features, it integrates very well with Eureka for service discovery and Ribbon for load balancing. The server initialization is pretty straightforward.
public class Bootstrap {
public static void main(String[] args) {
Server server = null;
try {
ConfigurationManager.loadCascadedPropertiesFromResources("application");
final Injector injector = InjectorBuilder.fromModule(new RentalsModule()).createInjector();
final BaseServerStartup serverStartup = injector.getInstance(BaseServerStartup.class);
server = serverStartup.server();
server.start(true);
} catch (final IOException ex) {
throw new UncheckedIOException(ex);
} finally {
// server shutdown
if (server != null) {
server.stop();
}
}
}
}
It is battle-tested in production for years and its effectiveness as API gateway and/or edge service is proven at Netflix’s scale.
3. Spring Cloud Gateway
Spring Cloud Gateway, a member of the Spring platform, is a library to facilitate building your own API gateways leveraging Spring MVC and Spring WebFlux. The first generation of Spring Cloud Gateway was built on top of Zuul but it is not the case anymore. The new generation has changed the power train to Spring’s own Project Reactor and its ecosystem.
Let us take a look on how JCG Car Rentals platform could leverage Spring Cloud Gateway to have an edge entry point for its APIs.
server:
port: 17001
spring:
cloud:
gateway:
discovery:
locator:
enabled: true
routes:
- id: inventory
uri: lb://inventory-service
predicates:
- Path=/inventory/**
filters:
- RewritePath=/(?.*), /api/$\{path}
- id: customers
uri: lb://customer-service
predicates:
- Path=/customers/**
filters:
- RewritePath=/(?.*), /api/$\{path}
- id: reservations
uri: lb://reservation-service
predicates:
- Path=/reservations/**
filters:
- RewritePath=/(?.*), /api/$\{path}
- id: payments
uri: lb://payment-service
predicates:
- Path=/payments/**
filters:
- RewritePath=/(?.*), /api/$\{path}
eureka:
instance:
appname: api-gateway
preferIpAddress: true
client:
register-with-eureka: false
fetch-registry: true
healthcheck:
enabled: true
service-url:
defaultZone: http://localhost:20200/eureka/
As you can spot right away, we have used a purely configuration-driven approach along with Eureka integration for service discovery. Running the server using Spring Boot requires just a few lines of code.
@SpringBootApplication
@EnableDiscoveryClient
public class GatewayStarter {
public static void main(String[] args) {
SpringApplication.run(GatewayStarter.class, args);
}
}
Similarly to Zuul 2, Spring Cloud Gateway allows you to slice and dice whatever features your microservices architecture demands from the API gateway. However, it also becomes your responsibility to maintain and learn how to operate it.
One of the benefits to building your own API gateway is the freedom to perform aggregations and fan-outs over multiple services. This way the number of round-trips which clients have to perform otherwise could be reduced significantly since API gateway would be responsible for stitching the multiple responses together. There is the dark side of this path though, please stay tuned.
4. HAProxy
In the previous part of the tutorial we talked about HAProxy primarily as a load balancer, however its capabilities allow it to serve as API gateway as well. If HAProxy already made its way into your microservice architecture, trying it in a role of API gateway is worth considering.
5. Microgateway
Microgateway by StrongLoop is a great illustration of the innovations happening in the world of JavaScript and particularly Node.js ecosystem.
The Microgateway is a developer-focused, extensible gateway framework written in Node.js for enforcing access to Microservices and APIs. – https://strongloop.com/projects/
6. Kong
Kong is among the first API gateways which emerged at Mashape to address the challenges of their microservice deployments.
Kong is a scalable, open source API Layer (also known as an API Gateway, or API Middleware). Kong runs in front of any RESTful API and is extended through Plugins, which provide extra functionality and services beyond the core platform. – https://konghq.com/about-kong/
Written in Lua, Kong is built on solid foundation of nginx (which we have talked about in the previous part of the tutorial) and is distributed along with OpenResty, a full-fledged nginx-powered web platform.
7. Gravitee.io
From the focused API gateway solutions we are gradually moving towards more beefy options, starting from Gravitee.io, an open-source API platform.
Gravitee.io is a flexible, lightweight and blazing-fast open source API Platform that helps your organization control finely who, when and how users access your APIs. – https://gravitee.io/
The Gravitee.io platform consists of three core components: in the center is API Gateway, surrounded by Management API and Management Web Portal.
8. Tyk
Tyk is yet another example of lightweight and comprehensive API platform, with the API gateway in the heart of it.
Tyk is an open source API Gateway that is fast, scalable and modern. Out of the box, Tyk offers an API Management Platform with an API Gateway, API Analytics, Developer Portal and API Management Dashboard. – https://tyk.io/
Tyk is written in Go and is easy to distribute and deploy. It has quite large list of the key features, with the emphasis on API analytics and access management.
9. Ambassador
The hyper-popularity of Kubernetes led to the rise of API gateways which could natively run on it. One of the pioneers in this category is Ambassador by Datawire.
Ambassador is an open source Kubernetes-native API Gateway built on Envoy, designed for microservices. Ambassador essentially serves as an Envoy ingress controller, but with many more features. – https://github.com/datawire/ambassador
Since most of the organizations are leveraging Kubernetes to deploy their microservices fleet, Ambassador has occupied the leading positions there.
10. Gloo
Yet another notable representative of the Kubernetes-native API gateways is Gloo, open-sourced and maintained by solo.io.
Gloo is a feature-rich, Kubernetes-native ingress controller, and next-generation API gateway. Gloo is exceptional in its function-level routing; its support for legacy apps, microservices and serverless; its discovery capabilities; its numerous features; and its tight integration with leading open-source projects. – https://gloo.solo.io/
Gloo is built on top of the Envoy. The seamless integration with a number of serverless offerings makes Gloo a truly unique solution.
11. Backends for Frontends (BFF)
One of the challenges that many microservice-based platforms face these days is dealing with the variety of different types of consumers (mobile devices, desktop applications, web frontends, …). Since every single consumer has own unique needs and requirements, it clashes with the reality to run against one-size-fit-all backend services.
The API gateway could potentially help, often trading the convenience for the explosion of the APIs, tailored for each consumer. To address these shortcomings, the Backends For Frontends (or BFF) pattern has been emerged and gained some traction. In particular, backed by GraphQL, it becomes a very efficient solution to the problem.
Let us quickly look though the JCG Car Rentals platform which includes the BFF component, based on GraphQL and Apollo GraphQL stack. The implementation itself uses REST Data Source to delegate the work to Reservation Service, Customer Service or/and Inventory Service, transparently to the consumer which just asks for what it needs using GraphQL queries.
query {
reservations(customerId: $customerId) {
from
to
}
profile(id: $customerId) {
firstName
lastName
}
}
BFF, especially GraphQL ones, may not be classified as traditional API gateway however it is certainly a very useful pattern to consider when dealing with the multitude of different clients. The major benefit BFFs bring on the table is the ability to optimize for specific client or platform but they may also sidetrack to the danger zone easily.
12. Build Your Own
In case none of the existing approaches look appealing to your microservice architecture needs, there is always the possibility to build your own. Leveraging the full-fledged integration frameworks like Apache Camel or Spring Integration could be the best and shortest path to get you there. Moreover, if you are already betting on these frameworks, using the familiar paradigm is much more efficient than learning yet another technology (spoiler alert, do not let hype to mislead you).
13. Cloud
Every major cloud provider has at least one kind of API gateway offering. It may not be the best in class but hopefully is good enough. On the bright side, it has seamless integration with numerous other offerings, specifically related to security and access control.
One interesting subject to touch upon is to understand what is the role of API gateways in the serverless computing? It will not be an overstatement to say that the API gateway is a very core component in such architecture since it provides one of the entry points into the serverless execution model. If a trivial microservices deployment may get along without an API gateway, the serverless may not get far without it.
14. On the Dark Side
The fight for leadership in the niche of the API gateways forces the vendors to pour more and more features into their products, which essentially reshapes the definition of what an API gateway is and leads straight to the identity crisis. The problem is exceptionally well summarized by ThoughtWorks in their terrific technology radar.
We remain concerned about business logic and process orchestration implemented in middleware, especially where it requires expert skills and tooling while creating single points of scaling and control. Vendors in the highly competitive API gateway market are continuing this trend by adding features through which they attempt to differentiate their products. This results in overambitious API gateway products whose functionality — on top of what is essentially a reverse proxy — encourages designs that continue to be difficult to test and deploy. API gateways do provide utility in dealing with some specific concerns — such as authentication and rate limiting — but any domain smarts should live in applications or services. – https://www.thoughtworks.com/radar/platforms/overambitious-api-gateways
The issues are not imaginable and this is indeed happening in many organizations. Please take it seriously and avoid this trap along your journey.
15. Microservices API Gateways and Aggregators – Conclusions
In this part of the tutorial we have identified the role of the API gateway, yet another key piece in the modern microservice architecture. Along with BFF, it takes care of many cross-cutting concerns and removes unnecessary burden from the consumers, but at the cost of increasing complexity. We have also discussed the common pitfalls the organizations fall into while introducing API gateways and BFFs, the mistakes other did and you should learn from and avoid.
16. What’s next
In the next section of the tutorial we are going to talk about deployment and orchestration, specifically suited for microservices.
