Kelly O'Hair's Blog

Heap Dump Snapshots

Alan recently blogged about heap dumps are back and Sundar blogged about OQL. These all use the old and historic HPROF binary heap dump format as the "heap dump snapshot", which is essentially a complete dump of all the objects in the Java heap at a given time.

If you are familiar with core dumps, and the Solaris utility gcore, you'll understand what we are trying to do here, at least in a basic way. This is not a new concept by any means. By creating "snapshots in time" of the heap, this allows you to browse the data without disturbing a running process, or allows you to do some postmortem processing on a crashed process.

We also added Bill Foote's valuable but historic Heap Analysis Tool (HAT) to view, inspect, and browse these heap dumps. Binary dumps aren't much good without some kind of viewer. HAT has been released in the latest Mustang JDK as jhat. So if you have the latest Mustang and you can use HPROF, you can very easily browse the heap state at the time your java program exits with:

java -Xrunhprof:format=b,file=snapshot1.hprof Classname
jhat snapshot1.hprof

Then startup your browser and go to http://localhost:7000, or replace localhost with your machine name and browse on any machine you want. If you haven't tried this, you should give it a spin. If you don't want the heap dump at exit, then send your Java process a ^C (control-C), and HPROF will get you a heap dump file on demand.

Don't want to use the HPROF startup option? You don't have to. Try using the 'jmap' utility:

jmap -dump:format=b,file=snapshot2.jmap PID_OF_PROCESS
jhat snapshot2.jmap

It doesn't matter how the JVM was started, jmap can get you a heap dump snapshot. The jmap heap.bin files should contain all the primitive data, but won't include any stack traces as to where the objects have been created.

The HPROF in Build 55 of Mustang should include all the primitive data and stack traces.

Prior to Build 55 and in JDK 5.0 the primitive data (characters, integers, etc.) was not included in the HPROF generated files. This was understood in the JDK 5.0 release as a limitation we needed to fix.

So in doing this work, we have leveraged the original HPROF binary heap dump file format as the basic snapshot of the heap. Beyond Mustang, we'd like to do a new and more detailed specification of a binary heap format file, making it more available, and fixing some of the basic problems with the format. This format hasn't changed, and in theory HPROF binary format files from JDK 1.2 could still be used as input HAT or the Mustang jhat, not that I'd recommend it. The versions of HPROF prior to JDK 5.0 used JVMPI, and the quality of the file contents can't be guaranteed.

An improved format with a set of APIs and tools around it could serve as a basis for IDE's or 3rd party tools to provide help in tracking down heap related problems. We'd like to hear from anyone that might like to participate in defining this new file format or APIs, and will probably start a small java.net project as a way to get more involvement from the Java community. Stay tuned.

One last plug, sorry. :^)

The JDK 5.0 Troubleshooting Guide also contains lots of information in troubleshooting problems in JDK 5.0 (applies to JDK 6.0 too). If you have never looked at this document, it's worth a scan. It might answer some of those age old questions you might have had when your Java process core dumped or crashed, which hopefully doesn't happen very often anymore.

Alan has recently requested input on the new JDK 6.0 Troubleshooting Guide. We really want to hear from users as to what is important in this area, so send us your ideas! Tell us what we can improve or document to make your Java development job easier.

Comments

• I found out the Hat can't handle big files (~500m), the problem that most heap dump files are this big...

  Is there any way to overcome this problem?

  Posted by: g_korland on August 21, 2006 at 02:42 AM

• Some changes were made to JDK 6 jhat handle larger dumps, but it's basic design need some changes to completely fix this problem.

  Posted by: kellyohair on October 20, 2006 at 01:53 PM

• 
  Hello, I have tried profiler 6.0beta - it is really good. Especially the heap dump's Instances/references view - it helps to find the exact source of the memory leak (which were impossible without this view in 5th version). For example, at some moment I know that the instance of MyObject class should be garbage collected, but in the "live allocated objects" view I can see, that this reference is live. So I make a heap dump, open this object in Instances View, and in the "references view" start expanding the referent tree nodes one by one trying to find the object which holds the reference to MyObject instance and prevents it from being garbage collected. If I know that in my program another object of type PersistentObject should be alive at this moment, and if would see this object at some level of the MyObject's "references view" tree, I can understand, that this is the source of the memory leak.

  What I really miss in profiler at this moment is effective tools for working with the references view tree - for navigating through it and finding the object of type that I need. For the above example, MyObject is complex custom gui component, it has a big number a anonymous classes defined inside it (listeners) which are registered here and there, so the "references view" tree has a big number of the following entries which can be expanded:

  -this$0
  -this$0
  -this$0
  ...
  -this$0

  
  The PersistentObject reference is somewhere under these nodes (and not on the 1st level of deepness), but in order to find it I have to look through 20 tree nodes like the above, recursively expanding each node, one by one manually. This might take forever. If I had a simple text field "filter", where I could put the name of the PersistentObject class, so after applying it the referent tree would only show me tree paths, which contain PersistentObject references, this would save hours from manual routine work. This might be implemented in some other way - like search engine or something else - the main goal is to solve the described problem and to add some automation for this tree.
  

  Posted by: benderamp on October 27, 2007 at 06:20 PM

This work is licensed under a Creative Commons License.