Ben Galbraith's Blog

Heaps of Memory

While profiling a Java EE web app, I found myself wondering how much memory a process required. I attached a low-instrusion profiler to the JVM and watched a pattern repeat itself several times: the heap grew by ~15 MB, got collected, grew again by ~15 MB, got collected, etc.

My first reaction was to conclude that the process required 15 MB of heap in order to complete. But of course, that's an erroneous conclusion. With different settings or constraints, perhaps the process would have added only 7 MB of garbage before the collector cleaned the heap.

What I really need to know is the size of the largest set of strongly referenced object data placed on the heap as a result of the process at any one time. Because I can reasonably isolate the process, I'd settle for a definite measurement of the strongly referenced objects on the heap at any one time.

So how do I figure this out? I've used a number of tools that show me the size of the heap over time. Some let me take a snapshot of the heap and remove all weakly-referenced data. But I have yet to find a tool that will, over a period of time, tell me what the largest amount of the uncollectable heap data is (in bytes).

Anyone know of tool or process to figure this out? I know I could reduce the heap size until the process throws an OutOfMemoryException. I'm looking for something a little more automated.

Comments

• How about a hack in your app that calls System.gc() really really often in a separate thread? It will slow the app down to a crawl but the max memory shown in the profiler will be the max strongly referenced (approximately). It's only a hack though.

  Cheers,
  Mikael Grev

  Posted by: mikaelgrev on October 21, 2006 at 01:29 PM

• I've been looking at sum(memory collected at gc) / (number of lifecycles) in these cases. I.e. 10 collections collecting 80M each / 260 Requests = 3M/Objects generated per request. Interestingly, I don't think you can get that information from the platform MXBeans nor JVMTI, so you need to look at the -Xloggc: output.

  Matthias

  Posted by: mernst on October 23, 2006 at 06:15 AM

• you could try using YourKit Java Profiler. It allows you to take snapshots, then do deltas between 2 snapshots. have you already tried that?

  Posted by: echoi1975 on October 23, 2006 at 12:22 PM

• Hi,

  How about approach the memory profiling from a capacity/consumption perspective which I believe is much more natural and effective and does not have you setting JVM arguments based on particular JVM implementations. The following article might convince you to look at the issue in a different light.
  http://www.jinspired.com/products/jxinsight/outofmemoryexceptions.html
  By the way JXInsight is the only tool on the market providing allocation sizes for profiled traces alongside GC, CPU and Clock times.

  William Louth, JXInsight Product Architect, JInspired
  http://www.jinspired.com
  

  Posted by: williamlouth on October 24, 2006 at 12:21 PM

This work is licensed under a Creative Commons License.