Analyze and respond to a Tomcat heap dump

Home > Search
  by

Get the PID

If a heap dump is unexpectedly or automatically created, ensure that the JVM associated with the heap dump is still running properly. The heap dump files can be used to determine the PID associated with the heap dump, and then the PID can be used to determine the JVM associated with the heap dump. The heap dump files are in this format:

java_pid<pid>.hprof

 

Is the PID in use

For example, let's say the PID is 12345. The ps command can be used to determine if the PID is still associated wth the JVM.

~]# ps -ef | grep 12345

 

Kill the PID if in use

If the output of the ps command only displays the grep command, the PID is no longer in use, which means that the JVM that was associated with the PID is no longer running, or was automatically restarted as part of the heap dump. If there is a significant amount of output, the JVM is still running. However, the JVM is probably in a bad way. For example, the JVM may be out of memory. When the JVM is in a bad way, you first will want to kill the PID, and the start the JVM.

~]# kill -9 12345

 

Determine the JVM associated with the PID

You can search the logs with the PID to determine what JVM was associated with the PID. This command will usually produce quite a bit of output, as this command searches every file at and below the specified directory for the string (12345 in this example). This may help you find the JVM that had the PID associated with the heap dump.

~]# grep -R /path/to/logs/directory -ie '12345'

 

Ensure the JVM is running

Once you know the JVM that had the PID with the heap dump, determine if the JVM was restarted. You can check the catalina.log file for the event "Starting service Catalina" to determine when the JVM was last restarted.

~]# cat catalina.log
Mar 16, 2018 1:25:56 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina

 

Check for out of memory perm gen space

If event OutOfMemory Perm Gen space is in the catalina.out log before the heap dump event, this means that the JVM heap dumped because the JVM ran out of perm gen memory. In this situation, there is no need to analyze the heap dump. Instead, you would look into the perm gen event, due to the perm gen event being the cause of the heap dump.

~]# cat catalina.out | grep Perm
java.lang.OutOfMemoryError: PermGen space

 

Transfer the heap dump

If the heap dump was not caused by an out of memory perm gen space event, then you will want to transfer the heap dump from the Tomcat server to another PC, such as your personal laptop. You never want to analyze a heap dump on the Tomcat server, as this will use significant resources, such as memory and CPU, which will impact the performance of the Tomcat server.

 

Check for a memory leak

If you have a heap visualizer tool, use the tool to determine if there is a memory leak. In this example, there is clearly a memory leak. The minimum bytes in use gradually increases. A steady incline is a visual representation of a garbage collector inability to remove some of the objects from the heap. Given enough time, a memory leak will cause all of the heap space to be used, causing an out of memory situation. A memory leak means there is some issue with the code of an application, and a code change will be needed to resolve the memory leak. For example, one possibility is that the code is constantly putting very large PDF files into the heap.

When the minimum heap space reaches the total available heap space, the JVM will run out of heap space, and an out of memory java heap space event should appear in the JVM logs.

java.lang.OutOfMemoryError: Java heap space

A memory leak will eventually lead to an out of memory Java heap space situation, which will create the heap dump, which means that one common cause of a heap dump is a memory leak.

 

On the other hand, this example shows good heap utilization. After the JVM was restarted, there was some activity and eventually the heap reached a steady state.

 

Analyze the heap dump

IBMs Heap Analyzer can be used to analyze a heap dump. IBMs Heap Analyzer can be installed as a tool in IBMs Support Assistant Team Server

 

In IBMs Heap Analzyer, select File > Open, select the java_pid<pid>.hprof file, and select Open. The analysis will identify the Java classes that contain objects that are taking up heap space. In this example, the java/lang/Object class is using 27.59% of the heap.

 

A class is a container that contains one or more objects. In this example, there is a class called Dog that contains an object called Puppy with a value of Old Yeller. The programmer that created the application that has classes taking up heap space will need to determine if a code change can be made to the application to reduce the heap space being used by the class.

Public class Dog {
  Public static void main(String []args){
    Puppy myPuppy = new Puppy( "Old Yeller" );
  }
}

 

 



Add a Comment




We will never share your name or email with anyone. Enter your email if you would like to be notified when we respond to your comment.




Please enter in the box below so that we can be sure you are a human.




Comments