Initially, I wanted to call this blog – ‘Flexible timeouts with interceptor driven retry policies‘ – but then I thought it would be too ‘heavy’. This statement, along with the revised title should (hopefully) give you an idea of what this post might talk about ;-)
The trigger
This post is primarily driven by one of the comment/question I received on one of my earlier posts about which briefly discussed timeout mechanisms and how they can be used to define ‘concurrency policies’ for Stateful and Singleton EJBs.
The Problem
While timeouts are a good way to enforce concurrency policies and control resource allocation/usage in your the EJB container, there problem arises when the timeouts are inconsistent and unpredictable. How do you configure your timeout policy then ?
Of course, there is no perfect solution. But, one of the work around which popped into my mind was, to ‘retry‘ the failed method (this might not be appropriate or possible for your given scenario, but can be applied if the use case permits). This is good example of a ‘cross-cutting‘ concern or in other words, an ‘aspect‘. The Java EE answer for this is – Interceptors. These are much better than the default ‘rinse-repeat-until-xyz with a try-catch block‘ because of
- code reuse
- flexibility
The gist (of the solution)
Here is the high level description (code available on Github)
- Define a simple annotation which represents the ‘retry policy metadata’ e.g. number of retries
- Define an interceptor with implementation to retry the target method – this would use the above mentioned ‘retry policy’ metadata and behave accordingly
- Attach this interceptor to the required method (caller)
- Optionally, use @InterceptorBinding
The sample code
- Uses a Singleton EJB to simulate a sample service and introduces latency via the obvious Thread.sleep() [which of course is forbidden inside a Java EE container]
- Uses a JAX-RS resource which injects and calls the Singleton EJB and is a candidate for ‘retry’ as per a ‘policy’
- Can be tested by deploying on any Java EE (6 or 7) compatible server and using Apache JMeter to simulate concurrent clients/requests (Invoke HTTP GET on http://serverip:port/FlexiTimeouts/test)
Without the retry (interceptor) configuration, the tests (for concurrent requests) will result in a HTTP timeout (408).
Once retry interceptor is activated, there will be some latency because the task will be automatically retried once it fails. This of course will depend on the volume (of concurrent requests) and the threshold would need to be tuned accordingly – higher threshold for a highly concurrent environment (usually, not ideally)
Additional thoughts
- It is not mandatory to have the threshold or retry policy defined in the code. It can be externalised as well (to make things more flexible) e.g. use the @RetryPolicy to point to a file which contains required policy metadata
- A Retry Threshold is not the only configurable attribute. You can have other criteria and use it in your Interceptor logic
- One can expose statistics related to success/failure/retries. Its better to do this in an async fashion (push it to JMX via an @Async EJB?) so that it does not hamper the Interceptor performance itself
Cheers!
| Reference: | Implementing auto retry in Java EE applications from our JCG partner Abhishek Gupta at the Object Oriented.. blog. |
