Trip down the memory lane
Back in 1998, when I was a C/C++ developer, trying my hands on Java, a few things about the language were, to put it mildly – irritating – for me. I remember fretting about these quite a lot
- Why isn’t there a decent editor for this? C/C++ had quite a few. All that I had for Java was the good old notepad.
- Why do I have to make a class, when all I want is a function? Why wasn’t a function an object as well?
- Why can’t I just package everything into one zip/jar and let the end user launch with a double click?
and a few others. Back then, I found me frequently chiding myself for not being able to let go of my ‘C/C++ way of thinking’ and embracing ‘Java way’ of doing things. Now, writing this article in 2013, about a decade and a half later, surprisingly all of those early irritations are gone. Not because I have embraced ‘Java’ way, but because java has changed.
Idle chit chatting aside, the point of this article is to talk about one of these questions – ‘Why can’t I just package everything into one zip/jar and let the end user launch with a double click?’.
Why do we need this – one zip/jar – that is executable?
If you are a developer, coding away happily on your IDE (I despise you all who have coded java on Eclipse, NetBeans from day one and have not had to code on Notepad), assisted by Google (I positively totally hate all of you all who did not have to find stuff on internet before Google), there is probably no convincing case.
However, have you faced a situation when
- You have been pulled into the data centre because the guy there have followed your deployment steps but your application / website will simply not work?
- All of a sudden the environment variables are all messed up, when ‘nobody at all so much as touched’ the production boxes, and you are the one who has to ‘just make it work’.
- You are sitting with your business stakeholder and staring incredulously at a ‘ClassNotFound exception’ and were convinced that Java did not like you at all.
In short, what I am trying to say is, when you are in the ‘relative’ sanity of your dev box / environment, a one executable jar does not really do anything for you. But the moment you step into the twilight zone of unknown servers and situations (sans the IDE and other assortment of tools) you start appreciating just how much a single executable jar could have helped.
Ok, I get it. But, what’s the big deal? We can make such a package / zip / jar in a jiffy if we have to. Isn’t that so.
In all my naivety, I thought so and found out the answer the hard way. Let me walk you through it. Fire up your editors folks. Let’s create a executableJar project. I use jdk1.7.0, STS, Maven 3.0.4. If you are new to Maven or just not hands on, I recommend you read
this and this.
File: C:\projects\MavenCommands.bat
ECHO OFF REM ============================= REM Set the env. variables. REM ============================= SET PATH=%PATH%;C:\ProgramFiles\apache-maven-3.0.4\bin; SET JAVA_HOME=C:\ProgramFiles\Java\jdk1.7.0 REM ============================= REM Standalone java application. REM ============================= call mvn archetype:generate ^ -DarchetypeArtifactId=maven-archetype-quickstart ^ -DinteractiveMode=false ^ -DgroupId=foo.bar ^ -DartifactId=executableJar001 pause
After you run this batch file, you will have a fully compilable standard java application. Go ahead compile it and build a jar (mvn -e clean install). You will end up with a executableJar001-1.0-SNAPSHOT.jar at C:\projects\executableJar001\target. Now lets go ‘
java -jar jarFileName‘. And here you stumble the first time. In geeky vocabulary it tells you that there were no class with a main method and hence it did not know what to execute. Fortunately this is an easy one. There are standard java process to solve it. And there is a Maven plugin to solve it. I will use the latter.
Updated File: /executableJar001/pom.xml
...
<dependencies>
...
</dependencies>
<build>
<plugins>
<!-- Set main class in the jar. -->
<plugin>
<groupid>org.apache.maven.plugins</groupid>
<artifactid>maven-jar-plugin</artifactid>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<mainclass>foo.bar.App</mainclass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
...
You can compile and assemble the application again (mvn -e clean install). It is will create a jar file in target folder. Try running the jar from command line again. This time you will get intended result. So, we are all sorted, right? Wrong. Very wrong.
Why? Everything seems fine.
Let’s dig in a bit deeper and we will find why everything is not as sorted as it looks at the moment. Let’s go ahead and add a dependency e.g. let’s say we want to add logging and for that we want to use a third party jar i.e. logback. I will let Maven handle dependencies in the development environment.
Updated File : /executableJar001/pom.xml
... <dependencies> <!-- Logging --> <dependency> <groupid>ch.qos.logback</groupid> <artifactid>logback-classic</artifactid> <version>1.0.9</version> </dependency> </dependencies> <build> ... </build>
Updated File: /executableJar001/src/main/java/foo/bar/App.java
package foo.bar;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class App
{
private final static Logger logger = LoggerFactory
.getLogger(App.class);
public static void main( String[] args )
{
System.out.println( 'Hello World!' );
logger.debug('Hello world from logger.');
}
}Now let’s compile and run the jar from command prompt using jar command. Did you see what happened?
Exception in thread 'main' java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
Basically it is saying that the class (i.e the actual code) of the LoggerFactory (i.e. the 3rd party jar that we had added in development environment) was not found.
Oh, but surely we should be able to tell java to pick up the 3rd party libraries from some folder.
Definitely. It is almost a certainty – if you are asking that question – that for most of your applications you tell the JVM where the 3rd party / dependency libraries are. You tell this by setting classpath. You could possibly be using some application server e.g. Tomcat / Jetty and that could be picking up some dependencies itself. And that is exactly where the problem originates.
As a developer, I provide a x.jar that works. However, for it to work, it depends on a.jar (which in turn might depend upon b.jar and c.jar … you get the point). When I, as a developer, bundle up my deliverable, a x.jar, there is a dependency – on whoever I am handing this out to – to make sure that the classpath is correctly set in the other environment where x.jar is supposed to work.
It is not that big a deal, mostly. However, it is not trivial either. There are multitude of ways that the dependencies on target environment could get messed up. There might be routine updates. There might be some other application deployed in the same production box, that needed an update on a jar that nobody thought would impact yours. We can discuss and debate the multitude of ways that these kind of mishaps can be stopped, but bottom line is x.jar (the developers responsibility) has dependencies (that the developer do not directly control). And that leads to mishaps.
Of course, if you add into this mix the whole lot of variables that comes in because of different versions, different application servers, etc etc. the existing solution of providing x.jar only, quickly starts looking very fragile.
So, what do we do?
Say thanks to Dr. P. Simon Tuffs. This gentleman explains how he catered to this problem in this link. It is a good read, I recommend it. What I have explained in very laymen terms (and have barely scratched the surface), Simon takes a deep dive into the problem and how he solved it. Long story short, he coded a solution and made it open source. I am not going to replay the same information again – read his article, it is quite informative – but I will call out the salient point of his solution.
- It allows folks to create a single jar that contains everything – your code, resources, dependencies, application server (potentially) – everything.
- It allows the end use to run this entire humongous jar by the simple java -jar jarFileName command.
- It allows developers to develop the same way they have been developing e.g. if it is a web application, the war file structure, remains same. So there are no changes in the development process.
Fine. So how do we go about doing it?
There are many places where it is detailed out. The One-JAR website. Ant with One-JAR. Maven with One-JAR.
Let’s see it in action on our dummy code. Thankfully there is also a Maven plugin for this. Sadly it is not in the Maven Central repository (Why? Folks why? You have put in 98% of work. Why be sluggish about the last 2%?). It comes with nice
usage instructions.
Updated file: /executableJar001/pom.xml
...
<dependencies>
...
</dependencies>
<build>
<plugins>
...
<!-- If you wanted to bundle all this in one jar. -->
<plugin>
<groupid>org.dstovall</groupid>
<artifactid>onejar-maven-plugin</artifactid>
<version>1.4.4</version>
<executions>
<execution>
<goals>
<goal>one-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<!-- Required only if you are usng onejar plugin. -->
<pluginrepositories>
<pluginrepository>
<id>onejar-maven-plugin.googlecode.com</id>
<url>http://onejar-maven-plugin.googlecode.com/svn/mavenrepo</url>
</pluginrepository>
</pluginrepositories>
Now all you need to do is run mvn -e clean package. You will get, apart from the normal jar, a fat self sufficient jar as well. Go ahead, do the java -jar jarFileName from command prompt again. It should work.
Hmm.. that sounds good. Why isn’t everybody going for this? And this One-JAR seems to be around since 2004. Why are we not seeing more players in this market?
You know what they say about free lunches? There are none. While the concept is quite neat and very practical, it does not mean that every other player have decided to join in. So if your website ‘needs’ to be hosted on one of the biggie paid application servers (I don’t know why you want to keep paying for those proprietary software and folks that understand them. Should you not pay for only quality folks and rely on the open source apps that do not lock you in) One-JAR might not be a feasible solution for you. Also, I hear unconfirmed murmurs about how things might get sluggish (during load up if your app is big. So, before you decide to commit to using this, I recommend you do a POC and make sure that other bits of your techstack are not unhappy with One-JAR.
My personal opinion is, 2004 was perhaps a little too early for this kind of thing. People were still struggling with stuff like standardization of build and release process, getting clear player in ORM area, closing on a clear player for MVC framework etc. Not that those questions have been answered yet, or will be any-time soon. But I think the flavour of current problems in IT world are around
- How to make DevOps work.
- How to make the entire build and release automated.
- How to leverage the open source libraries to provide solid dependable software while ensuring there are no heavy proprietary software causing lock-in and hence make the solution less agile for future business requirement.
And in my mind, One-JAR plays very nicely in that area. So, I definitely expect to see more of this tool and / or more tools around this concept. And, to be fair, there are more players in this area. Thanks to Christian Schlichtherle for pointing this out. There is Maven Assembly Plugin and Maven Shade Plugin which cater to this exact same problem. I have not tried them yet but from the documentation they look quite alright, feature wise. Dropwizard, although not the same thing, but in essence is very similar. They have extended the whole one jar concept with embedded app server, out of the box support for REST, JSON, Logback, sort of in a nice neat package, that you could just use straight off the shelf.
So, as I keep saying, these are nice exciting times to be in technology business, particularly if you like tinkering around with software.
Reference: One jar to rule them all from our JCG partner Partho at the Tech for Enterprise blog.
And there is also gradle …
How it handles merging the jars with same config files inside jars?
For ex: Many of the Spring jars have META-INF/spring.handlers file which contain the namespace handler mappings of that particular jar. When all these jars are merged will those config files merged properly?
OneJar puts the dependencies as JAR files inside the application JAR file. Thus you don’t have these conflicts in contrast to the Maven Assemble or Shade plugins. OneJar is using a custom classloader, though, to handle this JAR-in-JAR case.