What follows is a practical introduction to a specific type of memory problems in Java applications. Namely – we will analyze the errors that cause the java.lang.OutOfMemoryError: PermGen space symptom in the stack trace.
First of all we will go through the core concepts required to understand the subject – and explain what objects, classes, classloaders and the JVM memory model are. If you are familiar with the basic concepts, you can jump directly to the next section where I will describe two typical cases for the error in question alongside with hints and suggestions for solving it.
Objects, Classes and ClassLoaders
Well, I will not start with the very basics. I guess if you have already found us, you should be familiar with the concept that everything in Java is an Object. And that all Objects are specified by their Class. So every object has a reference to an instance of java.lang.Class describing the structure of that object’s class.
But what actually happens under the hood, when you create a new object in your code? For example if you write something truly complicated like
Person boss = new Person()
The Java Virtual Machine (JVM) needs to understand the structure of the object to create. To achieve this, the JVM looks for the class called Person. And if the Person class is accessed for the first time during this particular execution of the program, it has to be loaded by the JVM, normally from the corresponding Person.class file. The process of seeking for the Person.class file on the drive, loading it into memory and parsing it’s structure is called class loading. Ensuring proper class loading process is the responsibility of a ClassLoader. ClassLoaders are instances of java.lang.ClassLoader class and each and every class in a Java program has to be loaded by some ClassLoader. As a result we now have the following relationships:
As you can see from the next diagram every classloader holds references to all the classes it has loaded. For the purpose of our article these relationships are very interesting.
Remember this image, we will need it later.
Permanent Generation
Almost every JVM nowadays uses a separate region of memory, called the Permanent Generation (or PermGen for short), to hold internal representations of java classes. PermGen is also used to store more information – find out the details from this post if you are interested – but for our article it is safe to assume that only the class definitions are being stored in PermGen. The default size of this region on my two machines running java 1.6 is not a very impressive 82MB.
As I have explained in one of my earlier posts, a memory leak in Java is a situation where some objects are no longer used by an application, but the Garbage Collector fails to recognize them as unused. This leads to the OutOfMemoryError if those unused objects contribute to the heap usage significantly enough that the next memory allocation request by the application cannot be fulfilled.
The root cause of java.lang.OutOfMemoryError: PermGen space is exactly the same: the JVM needs to load the definition of a new class but there is not enough space in PermGen to do it – there are already too many classes stored there. A possible reason for this could be your application or server using too many classes for the current size of PermGen not to be able to accommodate them. Another commone reason could be a memory leak.
Permanent Generation Leak
But still, how on earth it is possible to leak something in PermGen? It holds definitions of java classes and they cannot become unused, can they? Actually, they can. In case of a Java web application deployed into an application server all those classes in your EAR/WAR become worthless when the application is undeployed. The JVM continues to run as the application server is still alive, but a whole bunch of class definitions are not in use anymore. And they should be removed from PermGen. If not, we will have memory leak in the PermGen area.
As a nice sample on the reasons – the Tomcat developers have set up a Wiki page describing different leaks found and fixed in the Apache Tomcat versions 6.0.24 and above.
Leaking Threads
One possible scenario for a classloader leak is through long running threads. This happens when your application or – as often was the case in my experience – a 3rd party library used by your application, starts some long running thread. An example of this could be a timer thread whose job is to execute some code periodically.
If the intended lifespan of this thread is not fixed, we are heading directly into a trouble. When any part of your application ever starts a thread, you must make sure that it is not going to outlive the application. In typical cases the developer either is not aware of this responsibility or simply forgets to write the clean-up code.
Otherwise, if some thread continues to run after the application is undeployed, it will, usually, hold a reference to a classloader of the web application it was started by, called context classloader. Which in turn means that all classes of the undeployed application continue to be held in memory. Remedy? If it is your application that starts new threads, you should shut them down during undeployment using a servlet context listener. If it is a 3rd party library, you should search for its own specific shutdown hook. Or file a bug report if there is none.
Leaking Drivers
Another typical case of a leak can be caused by database drivers. We have encountered this leak in our own demo application that we ship with Plumbr. It is a slightly modified Pet Clinic application shipped along with Spring MVC. Let us highlight some things that happen when this application is being deployed to the server.
- The server creates a new instance of java.lang.Classloader and starts to load the application’s classes using it.
- Since the PetClinic uses a HSQL database, it loads the corresponding JDBC driver, org.hsqldb.jdbcDriver
- This class, being a good-mannered JDBC driver, registers itself with java.sql.DriverManager during initialization, as required per JDBC specification. This registration includes storing inside a static field of DriverManager a reference to an instance of org.hsqldb.jdbcDriver.
Now, when the application is undeployed from the application server, the java.sql.DriverManager will still hold that reference, as there is no code in the HSQLDB library nor in the Spring framework nor in the application to remove that! As was explained above, a jdbcDriver object still holds a reference to an org.hsqldb.jdbcDriver class, which in turn holds a reference to the instance of java.lang.Classloader used to load the application. This classloader now still reference all the classes of the application. In case of our particular demo application, during application startup almost 2000 classes are loaded, occupying roughly 10MB in PermGen. Which means that it takes about 5-10 redeploys to fill the PermGen with default size to reach the java.lang.OutOfMemoryError: PermGen space crash.
How to fix that? One possibility is to write a servlet context listener, which de-registers the HSQLDB driver from DriverManager during application shutdown. This is pretty straightforward. But remember – you will have to write the corresponding code in every application using the driver.
Download our latest version of Plumbr with our demo application and play with it to find out how the leak occurs, how Plumbr finds it and how do we explain the cause.
Conclusion
There are many reasons why your application might encounter a java.lang.OutOfMemoryError: PermGen space. The root cause for the majority of them is some reference to an object or a class loaded by the application’s class loader that has died after that. Or a direct link to the class loader itself. The actions you need to take for remedy are quite similar for most of these causes. Firstly, find out where that reference is being held. Secondly, add a shutdown hook to your web application to remove the reference during application’s undeployment. You can do that by either using a servlet context listener or by using the API provided by your 3rd party library.
Finding those leaking references has never been easy. We ourselves have spent countless hours trying to trace down why some applications require 20MB of PermGen on each redeploy. But as of version 1.1,, Plumbr will show you the cause of the leak and give you a hint on how to fix it. If you want to try it out, register and download the tool. If you are running an older version of Plumbr, we strongly recommend downloading an upgrade.
Reference: What is a PermGen leak? from our JCG partner Nikita Salnikov Tarnovski at the Plumbr Blog blog.
getting a 404 from the download page for the demo app.
very interesting
good job man!!1