Skip to main content

NIO file backed buffered images

Posted by timboudreau on May 31, 2009 at 11:37 PM PDT

Over the years, a few people have come across and used a bit of code I wrote for Imagine. You basically have the problem that Java image data is stored on the heap as giant byte[] arrays and you quickly run out of memory.

One of the JDK team guys assured me about two years ago that with JDK 7 this would no longer be a problem - but I got no sense that he either understood what the problem was, and I couldn't think of a way you could really solve it without doing your own memory management.

Say you're writing an image editor application. "Tools" draw on the surface of an image. You want to provide undo support. You have a few options:

    Make copy of the whole image - but this is silly, the user changed part of the image, not the whole thing

    Notice the minimal rectangle, and save the previous version of that plus its coordinates - this is easy - wrap the Graphics2D that is drawn to in a proxy that records the bounds that are actually modified - then save a snapshot of just those bounds

Strategy 2 definitely makes more sense. But what to do with the old image data you might never reuse? You could use ImageIO and write it out in some format, but that turns out to be deathly slow.

The solution was ByteNIOBufferedImage - it's a regular BufferedImage you can treat like any other. But its backing data is a index into a memory-mapped file (there are still fragmetation issues to be solved - this *is* doing home-grown memory management, the thing Java is supposed to save us from). You don't want to take one of these images and paint it straight to the screen - it will be non-accellerated and pretty much deathly slow.

But it does provide a way to store semi-unlimited amounts of image data for a Java app without requiring huge heap sizes.

Anyway, it's definitely not for everyone but I've gotten enough private emails from people who wanted to use it that it seemed worth mentioning.

Related Topics >>

Comments

Most of the time the simple answer is to just ask for a big heap --- the garbage collectors now manage such heaps far better than used to be the case. On 32 bit Windows the achievable limit is usually at least 900M (i.e. -Xmx900m) and often up to 1400m or so. Note this just reserves the address space, it doesn't actually allocate the memory to your process until it needs to. An often requested feature is for a segmented heap that would not require an up front declaration of maximum heap size. Unfortunately I don't see any sign that such an implementation is imminent for the HotSpot jvm (there is another JVM which has implemented such a feature).

...and yes, I've looked at and used JAI, and it offers some improvements, but seems more server-side oriented and still doesn't (as far as I know) solve the heap limit problm.

Unfortunately there is an awkward problem with NIO memory mapping --- you can't control when unmapping occurs. So if you need more than 1400MB or so you will need to unmap some data, but you don't know when this will actually happen and thus allow you to map some more into memory. http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038 In my experience it is very difficult to use memory mapping effectively if you are pushing the limit of available address space.

Say we're editing an 12 megapixel image - 3504 * 2336. 3504 * 2336 = 8185344 pixels 8185344 pixels * 24 bits = 196448256 bytes (but most likely it's stored in memory as 32 bit ints, so realistically...) 8185344 pixels * 32 bits = 261931008 bytes 261931008 / 1024 = 25,5792 megabytes to store the image Now perform an operation affects the whole image, such as brightening it. Your undo buffer needs a backup of the original (yes there are clever things you can do and some transforms are truly reversible). Okay, that's 50 megabytes. Do that a few times, and 1. That 900 megabyte heap is full fairly quickly 2. You might actually have more than one image open with more than one undo stack 3. A 900 or 1400Mb heap is a recipe for horrific garbage collection pauses Now think about doing this for true 600dpi print resolution images (when I did 3d graphic design, it was not uncommon for me to be delivering images 6000+ pixels wide - one render could take several days on a network of 386s). For these kinds of volumes of data, the OS's memory manager is simply a better tool for the job than the Java heap.