Skip to main content

Needing a New Heap Dump Format

Posted by kellyohair on November 11, 2005 at 2:54 PM PST

I spent quite a bit of time working on

HPROF
in the
JDK 5 (Tiger) release,
converting it from

JVMPI to JVM TI
.
But I also spent a great deal of time making sure the new code
created output files that matched the HPROF file formats
byte for byte.
This was not an easy task, not only were these file formats
poorly specified, HPROF itself was considered "experimental",
built on an "experimental" JVMPI which influenced the file format design,
and we were severely limited as to the available
tests to verify the file formats.
In addition, we had no idea who used the file formats,
or what we could
safely change in them.
Usually, nothing was changed in these files because of this.

Turned out with a few minor problems,
most tools just worked with the files created by the JDK 5 HPROF.
But it was obvious that this was less than an ideal
situation, for everyone.
We needed a better specification, and the file format
needed some updating.
At the same time it was obvious that changing these formats
could create some severe compatibility problems for some vendors.

During the
JDK 6 Mustang
development, some industrious co-workers
(Alan and Sundar) started creating
heap dumps from running VM applications, without HPROF, but using
the historical HPROF binary format
(it was decided that creating a new format at the time wasn't a good idea).
Then also in JDK 6
the original
HAT
utility that reads HPROF binary files,
was transformed into jhat
and added to the JDK.
Now we had multiple ways to create HPROF binary files, but
we had added a utility that could at least be used in tests
to verify the format had the correct syntax.
But wait, there's more!

In

Sundar's Blog

you can see that he
has taken it even further, merging some scripting work he has
done with the functionality of jhat.
If you haven't looked at Sundar's work, or tried what is available
in Mustang, you should give it a try.

And all this is based on that ancient ill-specified HPROF binary file format.
It was obvious that these heap snapshot files had some un-tapped value.

So I've started a small JDK project
to define a new heap dump format at:


Hopefully we can come up with a format that is as good as the old
HPROF binary format and ideally much better, with a formal
specification.

If you are interested, please join the project and let's have some discussions about this.
If we can standardize a format it should serve as a solid basis for
more tools and support in helping analyze and diagnose heap related
problems in java applications.

Related Topics >>