Concurrency best practices

This article is part of our Academy Course titled Advanced Java.

This course is designed to help you make the most effective use of Java. It discusses advanced topics, including object creation, concurrency, serialization, reflection and many more. It will guide you through your journey to Java mastery! Check it out here!

1. Introduction

The multiprocessor and multicore hardware architectures greatly influence the design and execution model of applications which run on them nowadays. In order to utilize the full power of available computational units, the applications should be ready to support multiple execution flows which are running concurrently and competing for resources and memory. Concurrent programming brings a lot of challenges related to data access and non-deterministic flow of events which can lead to unexpected crashes and strange failures.

In this part of the tutorial we are going to look at what Java can offer to the developers in order to help them to write robust and safe applications in concurrent world.

2. Threads and Thread Groups

Threads are the foundational building blocks of concurrent applications in Java. Threads are sometimes called lightweight processes and they allow multiple execution flows to proceed concurrently. Every single application in Java has at least one thread called the main thread. Every Java thread exists inside JVM only and may not reflect any operating system thread.

Threads in Java are instances of class Thread. Typically, it is not recommended to directly create and manage threads using the instances of Thread class (executors and thread pools covered in section Futures and Executors provide a better way to do that), however it is very easy to do:

public static void main(String[] args) {
    new Thread( new Runnable() {
        @Override
        public void run() {
            // Some implementation here
        }
    } ).start();
}

Or the same example using Java 8 lambda functions:

public static void main(String[] args) {
    new Thread( () -> { /* Some implementation here */ } ).start();
}

Nevertheless creating a new thread in Java looks very simple, threads have a complex lifecycle and can be in one of the following states (a thread can be in only one state at a given point in time).

Not all thread states are clear right now but later in the tutorial we will go over most of them and discuss what kind of events cause the thread to be in one state or another.

Threads could be assembled into groups. A thread group represents a set of threads and can also include other thread groups (thus forming a tree). Threads groups intended to be a nice feature however they are not recommended for use nowadays as executors and thread pools (please see Futures, Executors and Thread Pools) are much better alternatives.

3. Concurrency, Synchronization and Immutability

In mostly every Java application multiple running threads need to communicate with each other and access shared data. Reading this data is not so much of a problem, however uncoordinated modification of it is a straight road to disaster (so called racing conditions). That is the point where synchronization kicks in. Synchronization is a mechanism to ensure that several concurrently running threads will not execute the specifically guarded (synchronized) block of application code at the same time. If one of the threads has begun to execute a synchronized block of the code, any other thread trying to execute the same block must wait until the first one finishes.

Java language has synchronization support built-in in the form of synchronized keyword. This keyword can be applied to instance methods, static methods or used around arbitrary execution blocks and guarantees that only one thread at a time will be able to invoke it. For example:

public synchronized void performAction() {
    // Some implementation here
}

public static synchronized void performClassAction() {
    // Some implementation here
}

Or, alternatively, the example which uses the synchronized with a code block:

public void performActionBlock() {
    synchronized( this ) {
        // Some implementation here
    }
}

There is another very important effect of synchronized keyword: it automatically establishes a happens-before relationship (http://en.wikipedia.org/wiki/Happened-before) with any invocation of a synchronized method or code block for the same object. That guarantees that changes to the state of the object are visible to all threads.

Please notice that constructors cannot be synchronized (using the synchronized keyword with a constructor raises compiler error) because only the thread which creates an instance has access to it while instance is being constructed.

In Java, synchronization is built around an internal entity known as monitor (or intrinsic/monitor lock, http://en.wikipedia.org/wiki/Monitor_(synchronization)). Monitor enforces exclusive access to an object’s state and establishes happens-before relationships. When any thread invokes a synchronized method, it automatically acquires the intrinsic (monitor) lock for that method’s instance (or class in case of static methods) and releases it once the method returns.

Lastly, the synchronization is Java is reentrant: it means that the thread can acquire a lock which it already owns. Reentrancy significantly simplifies the programming model of the concurrent applications as the threads have fewer chances to block themselves.

As you can see, concurrency introduces a lot of complexity into the Java applications. However, there is a cure: immutability. We have talked about that many times already, but it is really very important for multithreaded applications in particular: immutable objects do not need the synchronization as they are never being updated by more than one threads.

4. Futures, Executors and Thread Pools

Creating new threads in Java is easy, but managing them is really tough. Java standard library provides extremely useful abstractions in the form of executors and thread pools targeted to simplify threads management.

Essentially, in its simplest implementation, thread pool creates and maintains a list of threads, ready to be used right away. Applications, instead of spawning new thread every time, just borrows the one (or as many as needed) from the pool. Once borrowed thread finishes its job, it is returned back to the pool, and becomes available to pick up next task.

Though it is possible to use thread pools directly, Java standard library provides an executors façade which has a set of factory method to create commonly used thread pool configurations. For example, the code snippet below creates a thread pool with fixed number of threads (10):

ExecutorService executor = Executors.newFixedThreadPool( 10 );

Executors could be used to offload any task so it will be executed in the separate thread from the thread pool (as a note, it is not recommended to use executors for long-running tasks). The executors’ facade allows customizing the behavior of the underlying thread pool and supports following configurations:

In some cases, the result of the execution is not very important so executors support fire-and-forget semantic, for example:

executor.execute( new Runnable() {            
    @Override
    public void run() {
        // Some implementation here
    }
} );

The equivalent Java 8 example is much more concise:

executor.execute( () -> {
    // Some implementation here
} );

But if the result of the execution is important, Java standard library provides another abstraction to represent the computation which will happen at some point in the future, called Future<T>. For example:

Future< Long > result = executor.submit( new Callable< Long >() {
     @Override
     public Long call() throws Exception {
        // Some implementation here
        return ...;
     }            
} );

The result of the Future<T> might not be available immediately so the application should wait for it using a family of get(…) methods. For example:

Long value = result.get( 1, TimeUnit.SECONDS );

If result of the computation is not available within specified timeout, the TimeoutException exception will be raised. There is an overloaded version of get() which waits forever but please prefer to use the one with timeout.

Since the Java 8 release, developers have yet another version of the Future<T>, CompletableFuture<T>, which supports addition functions and actions that trigger upon its completion. Not only that, with introduction of streams, Java 8 introduces a simple and very straightforward way to perform parallel collection processing using parallelStream() method, for example:

final Collection< String > strings = new ArrayList<>();
// Some implementation here
        
final int sumOfLengths = strings.parallelStream()
    .filter( str -> !str.isEmpty() )
    .mapToInt( str -> str.length() )
    .sum();

The simplicity, which executors and parallel streams brought to the Java platform, made the concurrent and parallel programming in Java much easier. But there is a catch: uncontrolled creation of thread pools and parallel streams could kill application performance so it is important to manage them accordingly.

5. Locks

Additionally to the monitors, Java has support of the reentrant mutual exclusion locks (with the same basic behavior and semantics as the monitor lock but with more capabilities). Those locks are available through ReentrantLock class from java.util.concurrent.locks package. Here is a typical lock usage idiom:

private final ReentrantLock lock = new ReentrantLock();
    
public void performAction() {
    lock.lock();
        
    try { 
        // Some implementation here
    } finally {
        lock.unlock();
    }
}

Please notice that any lock must be explicitly released by calling the unlock() method (for synchronized methods and execution blocks Java compiler under the hood emits the instructions to release the monitor lock). If the locks require writing more code, why they are better than monitors? Well, for couple of reason but most importantly, locks could use timeouts while waiting for acquisition and fail fast (monitors always wait indefinitely and do not have a way to specify the desired timeout). For example:

public void performActionWithTimeout() throws InterruptedException {
    if( lock.tryLock( 1, TimeUnit.SECONDS ) ) {
        try {
            // Some implementation here
        } finally {
            lock.unlock();
        }
    }
}

Now, when we have enough knowledge about monitors and locks, let us discuss how their usage affects thread states.

When any thread is waiting for the lock (acquired by another thread) using lock() method call, it is in a WAITING state. However, when any thread is waiting for the lock (acquired by another thread) using tryLock() method call with timeout, it is in a TIMED_WAITING state. In contrast, when any thread is waiting for the monitor (acquired by another thread) using synchronized method or execution block, it is in a BLOCKED state.

The examples we have seen so far are quite simple but lock management is really hard and full of pitfalls. The most infamous of them is deadlock: a situation in which two or more competing threads are waiting for each other to proceed and thus neither ever does so. Deadlocks usually occur when more than one lock or monitor lock are involved. JVM often is able to detect the deadlocks in the running applications and warn the developers (see please Troubleshooting Concurrency Issues section). The canonical example of the deadlock looks like this:

private final ReentrantLock lock1 = new ReentrantLock();
private final ReentrantLock lock2 = new ReentrantLock();
    
public void performAction() {
    lock1.lock();
        
    try {
        // Some implementation here            
        try {
            lock2.lock();                
            // Some implementation here
        } finally {
            lock2.unlock();
        }              
        // Some implementation here
    } finally {
        lock1.unlock();
    }
}
    
public void performAnotherAction() {
    lock2.lock();
        
    try {
        // Some implementation here            
        try {
            lock1.lock();                
            // Some implementation here
        } finally {
            lock1.unlock();
        }              
        // Some implementation here
    } finally {
        lock2.unlock();
    }
}

The performAction() method tries to acquire lock1 and then lock2, while the method performAnotherAction() does it in the different order, lock2 and then lock1. If by program execution flow those two methods are being called on the same class instance in two different threads, the deadlock is very likely to happen: the first thread will be waiting indefinitely for the lock2 acquired by the second thread, while the second thread will be waiting indefinitely for the lock1 acquired by the first one.

6. Thread Schedulers

In JVM, the thread scheduler determines which thread should be run and for how long. All threads created by Java applications have the priority which basically influences the thread scheduling algorithm when it makes a decision when thread should be scheduled and its time quantum. However this feature has a reputation of being non-portable (as mostly every trick which relies on specific behavior of the thread scheduler).

The Thread class also provide another way to intervene into thread scheduling implementation by using a yield() method. It hints the thread scheduler that the current thread is willing to yield its current use of a processor time (and also has a reputation of being non-portable).

By and large, relying on the Java thread scheduler implementation details is not a great idea. That is why the executors and thread pools from the Java standard library (see please Futures, Executors and Thread Pools section) try to not expose those non-portable details to the developers (but still leaving a way to do so if it is really necessary). Nothing works better than careful design which tries to take into account the real hardware the application is running on (f.e., number of available CPUs and cores could be retrieved easily using Runtime class).

Want to be a Java Master ?

Subscribe to our newsletter and download the Advanced Java Guide right now!

In order to help you master the Java programming language, we have compiled a kick-ass guide with all the must-know advanced Java features! Besides studying them online you may download the eBook in PDF format!

Download Now

Thank you!

We will contact you soon.

7. Atomic Operations

In multithread world there is a particular set of instructions called compare-and-swap (CAS). Those instructions compare their values to a given ones and, only if they are the same, set a new given values. This is done as a single atomic operation which is usually lock-free and highly efficient.

Java standard library has a large list of classes supporting atomic operation, all of them located under java.util.concurrent.atomic package.

The Java 8 release extends the java.util.concurrent.atomic with a new set of atomic operations (accumulators and adders).

8. Concurrent Collections

Shared collections, accessible and modifiable by multiple threads, are rather a rule than an exception. Java standard library provides a couple of helpful static methods in the class Collections which make any existing collection thread-safe. For example:

final Set< String > strings = 
    Collections.synchronizedSet( new HashSet< String >() );

final Map< String, String > keys = 
    Collections.synchronizedMap( new HashMap< String, String >() );

However the returned general-purpose collection wrappers are thread-safe, it is often not the best option as they provide quite a mediocre performance in real-world applications. That is why Java standard library includes a rich set of collection classes tuned for concurrency. Below is just a list of most widely used ones, all hosted under java.util.concurrent package.

Those classes are specifically designed to be used in the multithreaded applications. They exploit a lot of techniques to make the concurrent access to the collection as efficient as possible and are the recommended replacement to synchronized collection wrappers.

9. Explore Java standard library

The java.util.concurrent and java.util.concurrent.locks packages are real gems for the Java developers who are writing concurrent applications. As there are a lot of the classes there, in this section we are going to cover most useful of them, but please do not hesitate to consult Java official documentation and explore them all.

Unfortunately, the Java implementation of ReentrantReadWriteLock was not so great and as of Java 8, there is new kind of lock:

10. Using Synchronization Wisely

Locking and synchronized keyword are powerful instruments which help a lot to keep the data model and program state consistent in multithreaded applications. However, using them unwisely causes threads contention and could dramatically decrease application performance. From the other side, not using the synchronization primitives may (and will) lead to a weird program state and corrupted data which eventually causes application to crash. So the balance is important.

The advice is to try to use locks or/and synchronized where it is really necessary. While doing so, make sure that the locks are released as soon as possible, and the execution blocks which require locking or synchronization are kept minimal. Those techniques at least should help to reduce the contention but will not eliminate it.

In the recent years, a lot of so called lock-free algorithms and data structure have emerged (f.e., atomic operations in Java from Atomic Operations section). They provide much better performance comparing to the equivalent implementations which are built using synchronization primitives.

It is good to know that JVM has a couple of runtime optimizations in order to eliminate the locking when it may be not necessary. The most known is biased locking: an optimization that improves uncontended synchronization performance by eliminating operations associated with the Java synchronization primitives (for more details, please refer to http://www.oracle.com/technetwork/java/6-performance-137236.html#2.1.1).

Nevertheless JVM does its best, eliminating the unnecessary synchronization in the application is much better option. Excessive use of synchronization has a negative impact on application performance as threads will be wasting expensive CPU cycles competing for resources instead of doing the real work.

11. Wait/Notify

Prior to the introduction of the concurrency utilities in the Java standard library (java.util.concurrent), the usage of Object’s wait()/notify()/notifyAll() methods was the way to establish communication between threads in Java. All those methods must be called only if the thread owns the monitor on the object in question. For example:

private Object lock = new Object();

public void performAction() {
    synchronized( lock ) {
        while( <condition> ) {
            // Causes the current thread to wait until
            // another thread invokes the notify() or notifyAll() methods.
            lock.wait();
        }

        // Some implementation here        
    }
}	

Method wait() releases the monitor lock the current thread holds because the condition it is waiting for is not met yet (wait() method must be called in a loop and should never be called outside of a loop). Consequently, another thread waiting on the same monitor gets a chance to run. When this thread is done, it should call one of notify()/notifyAll() methods to wake up the thread (or threads) waiting for the monitor lock. For example:

public void performAnotherAction() {
    synchronized( lock ) {              
        // Some implementation here
            
        // Wakes up a single thread that is waiting on this object's monitor.
        lock.notify();
    }
}

The difference between notify() and notifyAll() is that the first wakes up a single thread while second wakes up all waiting threads (which start to contend for the monitor lock).

The wait()/notify() idiom is not advisable to be used in the modern Java applications. Not only is it complicated, it also requires following a set of mandatory rules. As such, it may cause subtle bugs in the running program which will be very hard and time-consuming to investigate. The java.util.concurrent has a lot to offer to replace the wait()/notify() with much simpler alternatives (which very likely will have much better performance in the real-world scenario).

12. Troubleshooting Concurrency Issues

Many, many things could go wrong in multithreaded applications. Reproducing issues becomes a nightmare. Debugging and troubleshooting can take hours and even days or weeks. Java Development Kit (JDK) includes a couple of tools which at least are able to provide some details about application threads and their states, and diagnose deadlock conditions (see please Threads and thread groups and Locks sections). It is a good point to start with. Those tools are (but not limited to):

• JVisualVM (http://docs.oracle.com/javase/7/docs/technotes/tools/share/jvisualvm.html)
• Java Mission Control (http://docs.oracle.com/javacomponents/jmc.htm)
• jstack (https://docs.oracle.com/javase/7/docs/technotes/tools/share/jstack.html)

13. What’s next

In this part we have looked through very important aspect of modern software and hardware platforms – concurrency. In particular, we have seen what kind of instruments Java as a language and its standard library offer to developers to help them dealing with concurrency and asynchronous executions. In the next part of the tutorial we are going to cover serialization techniques in Java.

14. Download

You can download the source code of this lesson here: advanced-java-part-9

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!

1. JPA Mini Book

2. JVM Troubleshooting Guide

3. JUnit Tutorial for Unit Testing

4. Java Annotations Tutorial

5. Java Interview Questions

6. Spring Interview Questions

7. Android UI Design

and many more ....

I agree to the Terms and Privacy Policy

Sign up

Thank you!

We will contact you soon.