This post may not make me any new friends. Oh well, I was never really popular at school anyway. But let’s get to the point. Java 8’s biggest feature in terms of the language is undoubtedly Lambda expressions. It’s been a flagship feature for functional languages such as Scala and Clojure for a few years, and now Java has finally joined in.
The second biggest feature (depending of course on who you ask) is Nashorn – the new JVM JavaScript engine that’s supposed to bring Java up to par with other JS engines such as V8 and its node.js container.
But these new features have a dark side to them.
I’ll explain. The Java platform is built out of two main components. The JRE, which JIT compiles and executes bytecode, and the JDK which contains dev tools and the javac source compiler. These two components are fairly (but not fully) decoupled, which is what enables folks to write their own JVM languages, with Scala rising to prominence in the last few years. And therein lies some of the problem.
The JVM was built to be language agnostic in the sense that it can execute code written in any language, as long as it can be translated into bytecode. The bytecode specification itself is fully OO, and was designed to closely match the Java language. That means that bytecode compiled from Java source will pretty much resemble it structurally.
But the farther away you get from Java – the more that distance grows. When you look at Scala which is a functional language, the distance between the source code and the executed bytecode is pretty big. Large amounts of synthetic classes, methods and variables are added by the compiler to get the JVM to execute the semantics and flow controls required by the language.
When you look at fully dynamic languages such as JavaScript, that distance becomes huge.
And now with Java 8, this is beginning to creep into Java as well.
So why should I care?
I wish this could be a theoretical discussion, that while interesting, has no practical implication on our everyday work. Unfortunately it does, and in a very big way. With the push to add new elements into Java, the distance between your code and the runtime grows, which means that what you’re writing and what you’re debugging will be two different things.
To see how let’s (re)visit the example below.
Java 6 & 7
This is the traditional method by which we would iterate over a list of strings to map their lengths.
// simple check against empty strings
public static int check(String s) {
if (s.equals("")) {
throw new IllegalArgumentException();
}
return s.length();
}
//map names to lengths
List lengths = new ArrayList();
for (String name : Arrays.asList(args)) {
lengths.add(check(name));
}This will throw an exception if an empty string is passed. The stack trace will look like –
at LmbdaMain.check(LmbdaMain.java:19) at LmbdaMain.main(LmbdaMain.java:34)
Here we see a 1:1 correlation between the stack trace we see and the code we wrote, which makes debugging this call stack pretty straightforward. This is what most Java devs are used to.
Now let’s look at Scala and Java 8.
Scala
Let’s look at the same code in Scala. Here we’ve got two big changes. The first is the use of a Lambda expression to map the lengths, and the second is that the iteration is carried out by the framework (i.e. internal iteration).
val lengths = names.map(name => check(name.length))
Here we really start to notice the difference between how the code you wrote looks, and how the JVM (and you) will see it at run time. If an exception is thrown, the call stack is an order of magnitude longer, and much harder to understand.
at Main$.check(Main.scala:6) at Main$$anonfun$1.apply(Main.scala:12) at Main$$anonfun$1.apply(Main.scala:12) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at Main$delayedInit$body.apply(Main.scala:12) at scala.Function0$class.apply$mcV$sp(Function0.scala:40) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App$$anonfun$main$1.apply(App.scala:71) at scala.App$$anonfun$main$1.apply(App.scala:71) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32) at scala.App$class.main(App.scala:71) at Main$.main(Main.scala:1) at Main.main(Main.scala)
* Remember, this example is very simple. With real-world nested Lambdas and complex structures you’ll be looking at much longer synthetic call stacks, from which you’ll need to understand what happened.
This has long been an issue with Scala, and one of the reasons we built the Scala Stackifier.
And now in Java 8
Up until now Java developers were pretty immune to this. This will change as Lambda expressions become an integral part of Java. Let’s look at the corresponding Java 8 code, and the resulting call stack.
Stream lengths = names.stream().map(name -> check(name)); at LmbdaMain.check(LmbdaMain.java:19) at LmbdaMain.lambda$0(LmbdaMain.java:37) at LmbdaMain$$Lambda$1/821270929.apply(Unknown Source) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.LongPipeline.reduce(LongPipeline.java:438) at java.util.stream.LongPipeline.sum(LongPipeline.java:396) at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:526) at LmbdaMain.main(LmbdaMain.java:39)
This is becoming pretty similar to Scala. We’re paying the price for shorter, more concise code with more complex debugging, and longer synthetic call stacks.
The reason is that while javac has been extended to support Lambda functions, the JVM still remains oblivious to them. This has been a design decision by the Java folks in order to to keep the JVM operating at a lower-level, and without introducing new elements into its specification.
And while you can debate the merits of this decision, it means that as Java developers the cost figuring out these call stacks when we get a ticket now sadly lies on our shoulders, whether we want to or not.
JavaScript in Java 8
Java 8 introduces a brand new JavaScript compiler. Now we can finally integrate Java + JS in an efficient and straightforward manner. However, nowhere is the dissonance between the code we write and the code we debug bigger than here.
Here’s the same function in Nashorn –
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("nashorn");
String js = "var map = Array.prototype.map \n";
js += "var a = map.call(names, function(name) { return Java.type(\"LmbdaMain\").check(name) }) \n";
js += "print(a)";
engine.eval(js);In this case the bytecode code is dynamically generated at runtime using a nested tree of Lambda expressions. There is very little correlation between our source code, and the resulting bytecode executed by the JVM. The call stack is now two orders of magnitude longer. In the poignant words of Mr.T – I pity the fools who will need to debug the call stack you’ll be getting here.
Questions, comments? (assuming you can scroll all the way below this call stack). Let me know in the comments section.
LmbdaMain [Java Application] LmbdaMain at localhost:51287 Thread [main] (Suspended (breakpoint at line 16 in LmbdaMain)) LmbdaMain.wrap(String) line: 16 1525037790.invokeStatic_L_I(Object, Object) line: not available 1150538133.invokeSpecial_LL_I(Object, Object, Object) line: not available 538592647.invoke_LL_I(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 2150540.interpret_I(MethodHandle, Object, Object) line: not available 538592647.invoke_LL_I(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 92150540.interpret_I(MethodHandle, Object, Object) line: not available 38592647.invoke_LL_I(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 731260860.interpret_L(MethodHandle, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LL_L(MethodHandle, Object[]) line: 1108 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 2619171.interpret_L(MethodHandle, Object, Object, Object) line: not available 1597655940.invokeSpecial_LLLL_L(Object, Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLLL_L(MethodHandle, Object[]) line: 1118 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 2619171.interpret_L(MethodHandle, Object, Object, Object) line: not available 1353530305.linkToCallSite(Object, Object, Object, Object) line: not available Script$\^eval\_._L3(ScriptFunction, Object, Object) line: 3 1596000437.invokeStatic_LLL_L(Object, Object, Object, Object) line: not available 1597655940.invokeSpecial_LLLL_L(Object, Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLLL_L(MethodHandle, Object[]) line: 1118 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 484673893.interpret_L(MethodHandle, Object, Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLLLL_L(MethodHandle, Object[]) line: 1123 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 282496973.interpret_L(MethodHandle, Object, Object, Object, long, Object) line: not available 93508253.invokeSpecial_LLLLJL_L(Object, Object, Object, Object, Object, long, Object) line: not available 1850777594.invoke_LLLLJL_L(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 282496973.interpret_L(MethodHandle, Object, Object, Object, long, Object) line: not available 293508253.invokeSpecial_LLLLJL_L(Object, Object, Object, Object, Object, long, Object) line: not available 1850777594.invoke_LLLLJL_L(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 1840903588.interpret_L(MethodHandle, Object, Object, Object, Object, long, Object) line: not available 2063763486.reinvoke(Object, Object, Object, Object, Object, long, Object) line: not available 850777594.invoke_LLLLJL_L(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 82496973.interpret_L(MethodHandle, Object, Object, Object, long, Object) line: not available 220309324.invokeExact_MT(Object, Object, Object, Object, long, Object, Object) line: not available NativeArray$10.forEach(Object, long) line: 1304 NativeArray$10(IteratorAction).apply() line: 124 NativeArray.map(Object, Object, Object) line: 1315 1596000437.invokeStatic_LLL_L(Object, Object, Object, Object) line: not available 504858437.invokeExact_MT(Object, Object, Object, Object, Object) line: not available FinalScriptFunctionData(ScriptFunctionData).invoke(ScriptFunction, Object, Object...) line: 522 ScriptFunctionImpl(ScriptFunction).invoke(Object, Object...) line: 207 ScriptRuntime.apply(ScriptFunction, Object, Object...) line: 378 NativeFunction.call(Object, Object...) line: 161 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available 1740189450.invokeSpecial_LLL_L(Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLL_L(MethodHandle, Object[]) line: 1113 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 2619171.interpret_L(MethodHandle, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLL_L(MethodHandle, Object[]) line: 1113 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 323326911.interpret_L(MethodHandle, Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLLL_L(MethodHandle, Object[]) line: 1118 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 323326911.interpret_L(MethodHandle, Object, Object, Object, Object) line: not available 263793464.invokeSpecial_LLLLL_L(Object, Object, Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLLLL_L(MethodHandle, Object[]) line: 1123 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 1484673893.interpret_L(MethodHandle, Object, Object, Object, Object, Object) line: not available 587003819.invokeSpecial_LLLLLL_L(Object, Object, Object, Object, Object, Object, Object) line: not available 811301908.invoke_LLLLLL_L(MethodHandle, Object[]) line: not available 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 484673893.interpret_L(MethodHandle, Object, Object, Object, Object, Object) line: not available LambdaForm$NamedFunction.invoke_LLLLL_L(MethodHandle, Object[]) line: 1123 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available LambdaForm$NamedFunction.invokeWithArguments(Object...) line: 1147 LambdaForm.interpretName(LambdaForm$Name, Object[]) line: 625 LambdaForm.interpretWithArguments(Object...) line: 604 323326911.interpret_L(MethodHandle, Object, Object, Object, Object) line: not available 2129144075.linkToCallSite(Object, Object, Object, Object, Object) line: not available Script$\^eval\_.runScript(ScriptFunction, Object) line: 3 1076496284.invokeStatic_LL_L(Object, Object, Object) line: not available 1709804316.invokeExact_MT(Object, Object, Object, Object) line: not available FinalScriptFunctionData(ScriptFunctionData).invoke(ScriptFunction, Object, Object...) line: 498 ScriptFunctionImpl(ScriptFunction).invoke(Object, Object...) line: 207 ScriptRuntime.apply(ScriptFunction, Object, Object...) line: 378 NashornScriptEngine.evalImpl(ScriptFunction, ScriptContext, ScriptObject) line: 544 NashornScriptEngine.evalImpl(ScriptFunction, ScriptContext) line: 526 NashornScriptEngine.evalImpl(Source, ScriptContext) line: 522 NashornScriptEngine.eval(String, ScriptContext) line: 193 NashornScriptEngine(AbstractScriptEngine).eval(String) line: 264 LmbdaMain.main(String[]) line: 44
| Reference: | The Dark Side Of Lambda Expressions in Java 8 from our JCG partner Tal Weiss at the Takipi blog. |
Nice. Java joined the “write once – f*** you” group of languages.
Starting today, I’m your friend.
At last there is someone else who looks beyond the marketing; I totally agree with this post and think lambdas simply aren’t for general (developers) public.
Hi Tal, I was involved in commenting on some very early implementations of lambda functions is Java. I totally agree with you – but it’s even worse than you think. In languages that are truly functional you can have functions as objects. Java 8 lambdas makes you think that this is the case with method references. Here is a simple abstract example. Say you have an event producer object to which you can add listeners (maybe they are stored in a HashMap in the producer). With Java 8 you could ass a listener like this (using the new method reference… Read more »
Thanks Conrad! beautiful comment.
I just learned this the hard way. I wrote a bunch of code expecting removeListener to work, but it doesn’t :/ Unbelievable. There were millions of posts about adding listeners with method references and I loved the possibility… But none of them mentioned you can’t remove the damn thing. This makes method references completely useless in this scenario. I wish I read this comment earlier. So sad :((
Easy workaround:
EventListener listener = this::eventListener;
Pass that into eventProducer’s methods and viola!
You WOULD kind of hope that every time you reference a method, it would be the same object, but it’s not that big of a deal to actually set it into a variable (you know, like almost every other object you use). Make it final if you’re worried that it might accidentally get changed somewhere.
Sorry- the formatting got screwed up in my post – I hope it still makes sense
My point of view is that debugger has to show current stack trace in a smarter way, for instance with a small set of entities to be processed in each stream, a counter about how many entries has been processed, watch point and break point to intercept when a filter is applied on an entity… and so on. Sincerely that is a complex dynamic UI design that has to scale with the complexity of the lambda expression under debug, to avoid a developer to split his expression into parts for debugging, just losing the “benefits” of this new support in… Read more »
You can add new friend in your list. Totally agree with the post. I would be glad to write boilerplate code till it makes my life easier while debugging. Thumbs mate for the post !
Nice article. I think we should have race condition for JVM programming languages.
Wow. If a little noise in stack traces is the greatest expense, it seems Java 8 Lambdas were very cheap. Java developers have long been used to long stack traces from: web/app servers and various frameworks. The stack traces are very clear about which lines are your application code and which aren’t, so I can’t imagine any Java developer spending more than a couple extra seconds to figure out what the offending lines were. And, did you know that when you can get an exception, you can get the stack trace data programmatically? You can easily write a stack trace… Read more »
Plus, there seams to be something that no one is taking into account: almost all of that stack trace has to do with the streams API, NOT the lambda stuff. Yes, there’s a lot going on behind the scenes with streams, but you gotta show me a big stack trace just from lambdas to throw me off. Even that barely will, because of what you said.
Just looked into it: there is only one additional class in the stack trace when it comes to a simple lambda call.
at LambdaStackTraceTest.lambda$0(LambdaStackTraceTest.java:19)
at LambdaStackTraceTest$$Lambda$2/1523554304.run(Unknown Source)
at LambdaStackTraceTest.referenceRunner(LambdaStackTraceTest.java:29)
at LambdaStackTraceTest.main(LambdaStackTraceTest.java:19)
The first one is the actual lambda I wrote, the second is the added thing. The third line is the method calling the lambda, and the last is the main.
If you use a method reference, the top line is simply replaced by the method being referenced.
Those huge stack traces are due to the complexity of streams, not because of lambdas.
Thank for this post. Hard times for maintainers come…
2 cents
if (s.equals(“”)) is not correct –
if (“”.equals(s)) is correct . You avoid nullpointer exception
Thanks Dmytro. I’d still get the exception two lines later when I do s.length(), but in general you are right that there should be an additional check to protect against null values.
I would agree with Riccardo that the general developer won’t embrace lambdas. Of course time will tell if this was a good addition to the Java language or not. Perhaps Java has grown too big and we need to remove some more things. Thanks for sharing Tal!
Tom
I am learning Scala now in preparation for Apache Spark. I noticed what Tal brings up in the Map Reduce also. Whenever there is an error, the system presents a huge list of errors that scans several pages.
I think Java 8 is very good. I think lambdas are an okay way to save a few lines of code but I don’t see them as super clarifying. Generics are much more confusing and I can only use them in simple way. I learnt Java around 1.5 if I remember correctly, it was somewhat uphill but doable. I wouldn’t like try starting out with Java now. It would take a massive effort to get to a basic industry level. On the other hand javafx 8 is easy and very good for GUI’s and there are a lot of helper… Read more »
Totally agree.
When the code looks simpler, it almost always hard to read it.
It becomes nightmare to understand logic, when you read the code, written not by you, or even by you, but while ago.
I would rather have some boilerplate code, than headache while trying to debug what is this code doing….
Basically, when the Java compiler sees a Lambda expression, it recursively searches through the list of interfaces known to it until it finds an interface that defines a single method whose arguments match the types of those used by the lambda. It takes the compiler (or IDE) a fraction of a second to do a brute-force search through the API to find a match. Humans, unfortunately, aren’t quite so agile at things like that, which is why we invented source code in the first place 50 years ago. Behind the scenes, the compiler is just automatically assembling an anonymous class… Read more »