Recently in one of my project I got a requirement to execute a fallback call for a failing webservice call. To implement the same I was looking for some implementation of circuit breaker pattern and finally came across Netflix Hystrix library which I found is the best suited library as per our application.
In this post I tried to showcase a thin example of our problem and how Hystrix solved the same using a single microservice and a client to access it along with Hystrix Dashboard. Before diving into coding, let’s understand in brief what Hystrix is and how it works internally.
What is Hystrix?
Hystrix is a library that helps us control the interactions between the distributed services by adding latency tolerance and fault tolerance logic. It does this by isolating points of access between the services, stopping cascading failures across them, and providing fallback options, all of which improve our system’s overall resiliency.
It implements the circuit breaker pattern which work on circuit-breaker transitions from CLOSED to OPEN when a circuit meets a specified threshold and error percentage exceeds the threshold error percentage. While it is open, it short-circuits all requests made against that circuit-breaker. After some amount of time, the next single request is let through (this is the HALF-OPEN state). If the request fails, the circuit-breaker returns to the OPEN state for the duration of the sleep window. If the request succeeds, the circuit-breaker transitions to CLOSED and all requests made against that circuit-breaker are passed through to the service. More you can explore here.
Now, let’s start creating employee-service microservice running on port 8090 and client to access the same along with Hystrix Dashboard following below steps:
Step 1: Go to start.spring.io and create a new project employee-service adding the Web starters, based on the following image:
Step 2: Edit EmployeeServiceApplication.java to add a method which returns a list of employee, as follows:
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
@RestController
public class EmployeeServiceApplication {
public static void main(String[] args) {
SpringApplication.run(EmployeeServiceApplication.class, args);
}
@RequestMapping(value = "/list")
public String list() {
return "Arpit, Sanjeev, Abhishek";
}
}Step 3: Edit application.properties to specify the application name and port number of a service, as follows:
server.port=8090 spring.application.name=employee-service
Step 4: Move to employee-service directory and run command: mvn spring-boot:run. Once running, open http://localhost:8090/list.
Next, we will create hystrix-client which will access our newly created employee-service and if it is down will return the response from fallback method.
Step 5: Go to start.spring.io and create a new project hystrix-client adding the Web, Hystrix and Actuator starters, based on the following image:
Step 6: Edit HystrixClientApplication.java to add a method which calls employee-service to get a response and if service is down or unavailable because of any reason return a response from fallback method, as follows:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.hystrix.EnableHystrix;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
import com.test.service.IEmployeeService;
@EnableHystrix
@EnableCircuitBreaker
@SpringBootApplication
@RestController
@ComponentScan(basePackages = { "com.test.service" })
public class HystrixClientApplication {
@Autowired
private IEmployeeService employeeService;
public static void main(String[] args) {
SpringApplication.run(HystrixClientApplication.class, args);
}
@RequestMapping("/list")
public String list() {
return employeeService.list();
}
static class ApplicationConfig extends WebMvcConfigurerAdapter {
@Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
}Step 7: Create interface IEmployeeService and it’s implementation class EmployeeServiceImpl under com.test.service package and edit them as follows:
IEmployeeService.java
package com.test.service;
public interface IEmployeeService {
String list();
}EmployeeServiceImpl.java
package com.test.service.impl;
import java.net.URI;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import com.test.service.IEmployeeService;
@Service
public class EmployeeServiceImpl implements IEmployeeService {
@Value("#{'${employee.service.url}'}")
private String employeeServiceUrl;
@HystrixCommand(commandProperties = {
@HystrixProperty(name = "execution.isolation.strategy", value = "THREAD"),
@HystrixProperty(name = "execution.timeout.enabled", value = "true"),
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "500"),
@HystrixProperty(name = "execution.isolation.thread.interruptOnTimeout", value = "true"),
@HystrixProperty(name = "fallback.enabled", value = "true"),
@HystrixProperty(name = "circuitBreaker.enabled", value = "true"),
@HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10"),
@HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "1000"),
@HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "10"),
@HystrixProperty(name = "circuitBreaker.forceOpen", value = "false"),
@HystrixProperty(name = "circuitBreaker.forceClosed", value = "false") }, fallbackMethod = "fallback", commandKey = "list", groupKey = "EmployeeServiceImpl", threadPoolKey = "thread-pool-employee-service", threadPoolProperties = { @HystrixProperty(name = "coreSize", value = "5") }, ignoreExceptions = { IllegalAccessException.class })
public String list() {
RestTemplate restTemplate = new RestTemplate();
URI uri = URI.create(employeeServiceUrl + "/list");
return restTemplate.getForObject(uri, String.class);
}
public String fallback() {
return "Fallback call, seems employee service is down";
}
}@HystrixCommand specified above is used to wrap code that will execute potentially risky functionality with fault and latency tolerance, statistics and performance metrics capture, circuit breaker and bulkhead functionality.
@HystrixProperty specified above is used to control HystrixCommand behavior. All available options are listed here.
Step 8: Edit application.properties to specify the application port on which hystrix-client should be running and url on which employee-service is available, as follows:
server.port=8080 employee.service.url=http://localhost:8090
Step 9: Move to hystrix-client directory and run command: mvn spring-boot:run. Once running, open http://localhost:8080/list.
Is Hystrix working?
Shut down the employee-service application. Fallback message should be seen : Fallback call, seems employee service is down.
Next, we will create Hystrix Dashboard which will provide us the graphical view of success and failure requests, circuit status, host, cluster and thread pool status of an application.
Step 10: Go to start.spring.io and create a new project hystrix-dashboard adding the Hystrix Dashboard starters. Once created edit HystrixDashboardApplication.java, as follows:
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;
@SpringBootApplication
@EnableHystrixDashboard
public class HystrixDashboardApplication {
public static void main(String[] args) {
SpringApplication.run(HystrixDashboardApplication.class, args);
}
}Step 11: Edit application.properties to specify the application port on which hystrix-dashboard should be running, as follows:
server.port=8383
Step 12: Move to hystrix-dashboard directory and run command: mvn spring-boot:run. Once running, open http://localhost:8383/hystrix and enter http://localhost:8080/hystrix.stream in stream textbox and click Monitor Stream. Once dashboard is loaded we will see image similar to below:
The complete source code is hosted on github.
| Reference: | Microservices fault and latency tolerance using Netflix Hystrix from our JCG partner Arpit Aggarwal at the Arpit Aggarwal blog. |
