Memory leaks in Java are fortunately pretty rare, and when they do exist, off-the-shelf tools like JProfiler and OptimizeIt normally make short work of them. So when I discovered that an application I'm working on had a memory leak -- a BIG one -- I wasn't too worried. At first. Should just be a simple matter of spending some quality time with a profiler, shouldn't it?
This particular leak, though, seemed to defy all effort to actually catch it in the act. It never happened while I was profiling it, and for that matter it seemed to never happen while I was actually paying attention to the application. I would spent two hours unsuccessfully trying to make the leak happen, walk away from my computer for ten minutes, and come back to find an additional five megabytes of memory gone. And when I tried the same thing (so I thought) in a profiler, it didn't happen.
As it was such a big leak, megabytes at a time, I knew it had to be image-related -- there just isn't anything else that leaks in such big chunks, and this particular application indeed processes gigantic images. I carefully checked my usage of all such images, and never found anything the least bit problematic. The bug was a ghost, striking only when my back was turned.
After a couple days of dedicated hunting, I finally managed to catch the leak in a profiler. Sure enough, it was a five-megabyte int which belonged to an Image. But, curiously, it wasn't my image -- it belonged to RepaintManager. I felt a sinking sensation in the pit of my stomach as I realized it was a double buffer image.
Ten seconds later, I was looking at Bug 6209673: Memory leak repainting Swing UI caused by non-java process using 3D graphics. The summary is misleading, as it has nothing to do with 3D graphics. What actually happens is that any time your computer changes screen resolution (and therefore GraphicsConfiguration) and a paint occurs, the RepaintManager allocates a new double buffer for the new GraphicsConfiguration. And keeps the old one.
Other than the obvious "manually changing screen resolution" mechanism of doing it, there are a lot of other things that will trigger this bug. Screen savers or games might change the resolution. Hibernating your system will generally result in a new GraphicsConfiguration, so a Swing program running on Java 5.0 can leak megabytes of memory every time the system is hibernated. In some cases you can defend yourself by never repainting in the background: by never painting while the screen saver or game's GraphicsConfiguration is in effect, you prevent Java from allocating a new buffer, but this isn't always practical and doesn't defend against all occurrences of the bug.
In my case, I'm on a laptop, so hibernation was the culprit. That is also the reason I could never get it to happen while I was actively looking for it -- why would I hibernate my laptop while I'm in the middle of hunting for a bug?
The workaround (subclassing RepaintManager and changing the caching code in getVolatileOffscreenBuffer) was straightforward; the real difficulty was in tracking the bug down, because by its very nature it doesn't happen while you're actively looking for it. Since this bug is pure evil to find, can affect any Swing program, usually leaks five or more megabytes at a time, and has a very misleading summary, I think it's important to be aware of it. Hopefully you'll at least figure it out faster than I did.