Oracle’s Latest Java 8 Update Broke Your Tools — How Did it Happen?

Niv SteingartenAugust 12th, 2014Last Updated: August 11th, 2014

1 39 3 minutes read

If you’ve been keeping up with the news in the Java world lately, you’ve probably heard that the latest Java 8 build released by Oracle, Java 8u11 (and Java 7u65), introduced errors and broke some popular 3rd party tools such as ZeroTurnaround’s JRebel, Javassist, Google’s Guice, and even Groovy itself.

The errors spewed by the JVM are long and verbose, but in essence they look something like this:

Exception in thread "main" java.lang.VerifyError: Bad method call from inside of a branch
Exception Details:
   Location:
   com/takipi/tests/dc/DepthCounter.()V @10: invokespecial
   …

The reason these errors suddenly started appearing stems from the fact that the bytecode verifier in the latest updates is a bit more strict than that of previous versions. Unlike previous versions, it does not allow calls to super-constructors from within branched code.

Let’s break it down.

Java bytecode and the bytecode verifier

Bytecode is the intermediate language the JVM actually executes, and in which compiled .class files are written. The JVM’s machine code, if you will.

All JVM-based languages are compiled into bytecode, from Java, through Scala, Groovy, Clojure, and so on. The JVM doesn’t know and doesn’t care what the source language was — it only knows bytecode.

I’m not going to go into how bytecode works, as it is a subject worthy of a post (or several posts) of its own, but just to get a feel for what bytecode looks like — take this simple Java method for example:

int add(int x, int y) {
   int z = x + y;
   return z;
}

When compiled, its bytecode looks like this:

ILOAD x
ILOAD y
IADD
ISTORE z
ILOAD z
IRETURN

When the JVM loads a class file from the classpath into memory, it must first make sure the bytecode is valid and that the code is structured correctly. It basically checks if the code that’s being loaded can actually be executed. If the bytecode is good, the class is successfully loaded into memory; otherwise, a VerifyError is thrown, just like the one at the beginning of the post.

This process is called bytecode verification, and the part of the JVM that’s responsible for it is the bytecode verifier.

Why did it break?

In order for bytecode to pass verification, it must adhere to a set of rules defined in the class file format specification. As the JVM was originally designed with the Java programming language in mind, many of these rules are directly derived from Java rules and constraints.

One such well-known constraint in the Java language, is that the very first thing you must do in a constructor, before doing anything else, is call either super(…) or this(…). Any piece of code before that — and your code won’t compile. Even when you don’t explicitly write super(), the compiler implicitly inserts it for you at the very beginning of the constructor.

The same constraint exists, at least on paper, in the bytecode verification rules. However, it turns out that up until these recent JDK updates, this constraint had not been fully enforced. This means, that although no Java compiler would ever let you compile this code:

public static class ClassyClass {
   public ClassyClass() {
      if (checkSomething()) {
         super();
      } else {
         super(getSomething());
      }
   }
}

… The equivalent bytecode would pass verification!

ALOAD this
    INVOKESTATIC checkSomething() : boolean
    IFEQ L2
    INVOKESPECIAL super() : void
    GOTO L2
L1: INVOKESTATIC getSomething() : int
    INVOKESPECIAL super(int) : void
L2: RETURN

You can see in the simplified bytecode above that there are both an invocation (INVOKESTATIC) and even a branch (IFEQ — “if equal”) taking place before the first call to the super-constructor (INVOKESPECIAL).

Bear in mind that although the above code is not legal Java, and thus no Java compiler would ever produce the equivalent bytecode — there are plenty of other tools which potentially could, such as compilers of other JVM languages which do not follow Java’s constraints, and many other tools such as bytecode instrumentation libraries. The ability to execute code before the call to super can be pretty useful!

However, Java 8 update 11 brought with it a stricter bytecode verifier, one which rejects classes that use such constructs in their bytecode, and causes verification errors to be thrown and JVMs to crash.

On one hand, the new verifier is loyal to the spec, making sure our JVMs are safe from bad code. On the other hand, many tools which utilize bytecode instrumentation, such as debuggers and aspect weavers (AOP) often make use of constructs such as the above.

How to solve it?

A fix to the bytecode verifier has already been committed, but it hasn’t been released yet. However, many of the affected tools and projects are already releasing fixed versions and workarounds.

In the meantime, if you happen to encounter one of these errors, you can try starting your JVM with the -noverify command line argument. This option instructs the JVM to skip bytecode verification when loading classes.

Reference:

Oracle’s Latest Java 8 Update Broke Your Tools — How Did it Happen? from our JCG partner Niv Steingarten at the Takipi blog.