In previous articles (here and here) I showed that creating non-blocking asynchronous applications could increase performance when the server is under a heavy load. EJB 3.1 introduced the @Asynchronous annotation for specifying that a method will return its result at some time in the future. The Javadocs state that eithervoid or a Future must be returned. An example of a service using this annotation is shown in the following listing:
Service2.java
@Stateless
public class Service2 {
@Asynchronous
public Future<String> foo(String s) {
// simulate some long running process
Thread.sleep(5000);
s += "<br>Service2: threadId=" + Thread.currentThread().getId();
return new AsyncResult<String>(s);
}
}The annotation is on line 4. The method returns a Future of type String and does so on line 10 by wrapping the output in an AsyncResult. At the point that client code calls the EJB method, the container intercepts the call and creates a task which it will run on a different thread, so that it can return a Future immediately. When the container then runs the task using a different thread, it calls the EJB’s method and uses the AsyncResult to complete the Future which the caller was given. There are several problems with this code, even though it looks exactly like the code in all the examples found on the internet. For example, the Future class only contains blocking methods for getting at the result of the Future, rather than any methods for registering callbacks for when it is completed. That results in code like the following, which is bad when the container is under load:
//type 1
Future<String> f = service.foo(s);
String s = f.get(); //blocks the thread, but at least others can run
//... do something useful with the string...
//type 2
Future<String> f = service.foo(s);
while(!f.isDone()){
try {
Thread.sleep(100);
} catch (InterruptedException e) {
...
}
}
String s = f.get();
//... do something useful with the string...This kind of code is bad, because it causes threads to block meaning that they cannot do anything useful during that time. While other threads can run, there needs to be a context switch which wastes time and energy (see this good article for details about the costs, or the results of my previous articles). Code like this causes servers that are already under load to come under even more load, and grind to a halt.
So is it possible to get the container to execute methods asynchronously, but to write a client which doesn’t need to block threads? It is. The following listing shows a servlet doing so.
@WebServlet(urlPatterns = { "/AsyncServlet2" }, asyncSupported = true)
public class AsyncServlet2 extends HttpServlet {
@EJB private Service3 service;
protected void doGet(HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException {
final PrintWriter pw = response.getWriter();
pw.write("<html><body>Started publishing with thread " + Thread.currentThread().getId() + "<br>");
response.flushBuffer(); // send back to the browser NOW
CompletableFuture<String> cf = new CompletableFuture<>();
service.foo(cf);
// since we need to keep the response open, we need to start an async context
final AsyncContext ctx = request.startAsync(request, response);
cf.whenCompleteAsync((s, t)->{
try {
if(t!=null) throw t;
pw.write("written in the future using thread " + Thread.currentThread().getId()
+ "... service response is:");
pw.write(s);
pw.write("</body></html>");
response.flushBuffer();
ctx.complete(); // all done, free resources
} catch (Throwable t2) {
...Line 1 declares that the servlet supports running asynchronously – don’t forget this bit! Lines 8-10 start writing data to the response but the interesting bit is on line 13 where the asynchronous service method is called. Instead of using a Future as the return type, we pass it a CompletableFuture, which it uses to return us the result. How? Well line 16 starts the asynchronous servlet context, so that we can still write to the response after the doGet method returns. Lines 17 onwards then effectively register a callback on the CompletableFuture which will be called once the CompletableFuture is completed with a result. There is no blocking code here – no threads are blocked and no threads are polled, waiting for a result! Under load, the number of threads in the server can be kept to a minimum, making sure that the server can run efficiently because less context switches are required.
The service implementation is shown next:
@Stateless
public class Service3 {
@Asynchronous
public void foo(CompletableFuture<String> cf) {
// simulate some long running process
Thread.sleep(5000);
cf.complete("bar");
}
}Line 7 is really ugly, because it blocks, but pretend that this is code calling a web service deployed remotely in the internet or a slow database, using an API which blocks, as most web service clients and JDBC drivers do. Alternatively, use an asynchronous driver and when the result becomes available, complete the future as shown on line 9. That then signals to the CompletableFuture that the callback registered in the previous listing can be called.
Isn’t that just like using a simple callback? It is certainly similar, and the following two listings show a solution using a custom callback interface.
@WebServlet(urlPatterns = { "/AsyncServlet3" }, asyncSupported = true)
public class AsyncServlet3 extends HttpServlet {
@EJB private Service4 service;
protected void doGet(HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException {
...
final AsyncContext ctx = request.startAsync(request, response);
service.foo(s -> {
...
pw.write("</body></html>");
response.flushBuffer();
ctx.complete(); // all done, free resources
...@Stateless
public class Service4 {
@Asynchronous
public void foo(Callback<String> c) {
// simulate some long running process
Thread.sleep(5000);
c.apply("bar");
}
public static interface Callback<T> {
void apply(T t);
}
}Again, in the client, there is absolutely no blocking going on. But the earlier example of the AsyncServlet2 together with the Service3 class, which use the CompletableFuture are better for the following reasons:
- The API of
CompletableFutureallows for exceptions / failures, - The
CompletableFutureclass provides methods for executing callbacks and dependent tasks asynchronously, i.e. in a fork-join pool, so that the system as a whole runs using as few threads as possible and so can handle concurrency more efficiently, - A
CompletableFuturecan be combined with others so that you can register a callback to be called only when severalCompletableFutures complete, - The callback isn’t called immediately, rather a limited number of threads in the pool are servicing the
CompletableFutures executions in the order in which they are due to run.
After the first listing, I mentioned that there were several problems with the implementation of asynchronous EJB methods. Other than blocking clients, another problem is that according to chapter 4.5.3 of the EJB 3.1 Spec, the client transaction context does not propagate with an asynchronous method invocation. If you wanted to use the @Asynchronous annotation to create two methods which could run in parallel and update a database within a single transaction, it wouldn’t work. That limits the use of the @Asynchronous annotation somewhat.
Using the CompletableFuture, you might think that you could run several tasks in parallel within the same transactional context, by first starting a transaction in say an EJB, then creating a number of runnables and run them using the runAsync method which runs them in an execution pool, and then register a callback to fire once all were done using the allOf method. But you’re likely to fail because of a number of things:
- If you use container managed transactions, then the transaction will be committed once the EJB method which causes the transaction to be started returns control to the container – if your futures are not completed by then, you will have to block the thread running the EJB method so that it waits for the results of the parallel execution, and blocking is precisely what we want to avoid,
- If all the threads in the single execution pool which runs the tasks are blocked waiting for their DB calls to answer then you will be in danger of creating an inperformant solution – in such cases you could try using a non-blocking asynchronous driver, but not every database has a driver like that,
- Thread local storage (TLS) is no longer usable as soon as a task is running on a different thread e.g. like those in the execution pool, because the thread which is running is different from the thread which submitted the work to the execution pool and set values into TLS before submitting the work,
- Resources like
EntityManagerare not thread-safe. That means you cannot pass theEntityManagerinto the tasks which are submitted to the pool, rather each task needs to get hold of it’s ownEntityManagerinstance, but the creation of anEntityManagerdepends on TLS (see below).
Let’s consider TLS in more detail with the following code which shows an asyncronous service method attempting to do several things, to test what is allowed.
@Stateless
public class Service5 {
@Resource ManagedExecutorService mes;
@Resource EJBContext ctx;
@PersistenceContext(name="asdf") EntityManager em;
@Asynchronous
public void foo(CompletableFuture<String> cf, final PrintWriter pw) {
//pw.write("<br>inside the service we can rollback, i.e. we have access to the transaction");
//ctx.setRollbackOnly();
//in EJB we can use EM
KeyValuePair kvp = new KeyValuePair("asdf");
em.persist(kvp);
Future<String> f = mes.submit(new Callable<String>() {
@Override
public String call() throws Exception {
try{
ctx.setRollbackOnly();
pw.write("<br/>inside executor service, we can rollback the transaction");
}catch(Exception e){
pw.write("<br/>inside executor service, we CANNOT rollback the transaction: " + e.getMessage());
}
try{
//in task inside executor service we CANNOT use EM
KeyValuePair kvp = new KeyValuePair("asdf");
em.persist(kvp);
pw.write("...inside executor service, we can use the EM");
}catch(TransactionRequiredException e){
pw.write("...inside executor service, we CANNOT use the EM: " + e.getMessage());
}
...Line 12 is no problem, you can rollback the transaction that is automatically started on line 9 when the container calls the EJB method. But that transaction will not be the global transaction that might have been started by code which calls line 9. Line 16 is also no problem, you can use the EntityManager to write to the database inside the transaction started by line 9. Lines 4 and 18 show another way of running code on a different thread, namely using the ManagedExecutorService introduced in Java EE 7. But this too fails anytime there is a reliance on TLS, for example lines 22 and 31 cause exceptions because the transaction that is started on line 9 cannot be located because TLS is used to do so and the code on lines 21-35 is run using a different thread than the code prior to line 19.
The next listing shows that the completion callback registered on the CompletableFuture from lines 11-14 also runs in a different thread than lines 4-10, because the call to commit the transaction that is started outside of the callback on line 6 will fail on line 13, again because the call on line 13 searches TLS for the current transaction and because the thread running line 13 is different to the thread that ran line 6, the transaction cannot be found. In fact the listing below actually has a different problem: the thread handling the GET request to the web server runs lines 6, 8, 9 and 11 and then it returns at which point JBoss logs JBAS010152: APPLICATION ERROR: transaction still active in request with status 0 – even if the thread running line 13 could find the transaction, it is questionable whether it would still be active or whether the container would have closed it.
@Resource UserTransaction ut;
@Override
protected void doGet(HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException {
ut.begin();
...
CompletableFuture<String> cf = new CompletableFuture<>();
service.foo(cf, pw);
...
cf.whenCompleteAsync((s, t)->{
...
ut.commit(); // => exception: "BaseTransaction.commit - ARJUNA016074: no transaction!"
});
}The transaction clearly relies on the thread and TLS. But it’s not just transactions that rely on TLS. Take for example JPA which is either configured to store the session (i.e. the connection to the database) directly in TLS or is configured to scope the session to the current JTA transaction which in turn relies on TLS. Or take for example security checks using the Principal which is fetched from EJBContextImpl.getCallerPrincipal which makes a call to AllowedMethodsInformation.checkAllowed which then calls the CurrentInvocationContext which uses TLS and simply returns if no context is found in TLS, rather than doing a proper permission check as is done on line 112.
These reliances on TLS mean that many standard Java EE features no longer work when using CompletableFutures or indeed the Java SE fork-join pool or indeed other thread pools, whether they are managed by the container or not.
To be fair to Java EE, the things I have been doing here work as designed! Starting new threads in the EJB container is actually forbidden by the specs. I remember a test I once ran with an old version of Websphere more than ten years ago – starting a thread caused an exception to be thrown because the container was really strictly adhering to the specifications. It makes sense: not only because the number of threads should be managed by the container but also because Java EE’s reliance on TLS means that using new threads causes problems. In a way, that means that using the CompletableFuture is illegal because it uses a thread pool which isn’t managed by the container (the pool is managed by the JVM). The same goes for using Java SE’s ExecutorService as well. Java EE 7’s ManagedExecutorService is a special case – it’s part of the specs, so you can use it, but you have to be aware of what it means to do so. The same is true of the @Asynchronous annotation on EJBs.
The result is that writing asynchronous non-blocking applications in a Java EE container might be possible, but you really have to know what you are doing and you will probably have to handle things like security and transactions manually, which does sort of beg the question of why you are using a Java EE container in the first place.
So is it possible to write a container which removes the reliance on TLS in order to overcome these limitations? Indeed it is, but the solution doesn’t depend on just Java EE. The solution might require changes in the Java language. Many years ago before the days of dependency injection, I used to write POJO services which passed a JDBC connection around from method to method, i.e. as a parameter to the service methods. I did that so that I could create new JDBC statements within the same transaction i.e. on the same connection. What I was doing was not all that different to what things like JPA or EJB containers need to do. But rather than pass things like connections or users around explicitly, modern frameworks use TLS as a place to store the “context”, i.e. connections, transactions, security info, etc. centrally. As long as you are running on the same thread, TLS is a great way of hiding such boilerplate code. Let’s pretend though that TLS had never been invented. How could we pass a context around without forcing it to be a parameter in each method? Scala’s implicit keyword is one solution. You can declare that a parameter can be implicitly located and that makes it the compilers problem to add it to the method call. So if Java SE introduced such a mechanism, Java EE wouldn’t need to rely on TLS and we could build truly asynchronous applications where the container could automatically handle transactions and security by checking annotations, just as we do today! Saying that, when using synchronous Java EE the container knows when to commit the transaction – at the end of the method call which started the transaction. If you are running asynchronously you would need to explicitly close the transaction because the container could no longer know when to do so.
Of course, the need to stay non-blocking and hence the need to not depend on TLS, depends heavily on the scenario at hand. I don’t believe that the problems I’ve described here are a general problem today, rather they are a problem faced by applications dealing with a niche sector of the market. Just take a look at the number of jobs that seem to be currently on offer for good Java EE engineers, where synchronous programming is the norm. But I do believe that the larger IT software systems become and the more data they process, the more that blocking APIs will become a problem. I also believe that this problem is compounded by the current slow down in the growth hardware speed. What will be interesting to see is whether Java a) needs to keep up with the trends toward asynchronous processing and b) whether the Java platform will make moves to fix its reliance on TLS.
| Reference: | Is Asynchronous EJB Just a Gimmick? from our JCG partner Ant Kutschera at the The Kitchen in the Zoo blog. |
