Chris Campbell's Blog

400 Horsepower: Image I/O Improvements in Mustang

It's been a while since my last post, but I wanted to kick off 2006 by reporting some nice performance improvements in Image I/O. As you may know, we're constantly finding ways to boost performance in various parts of the JDK. In JDK 5 we did some performance work in Image I/O, specifically to avoid finalization in the JPEGImageReader, which made for huge gains in scalability and performance of reading JPEG images. We heard some great feedback about those changes, but we knew there was more work to be done.

So a couple months ago, I spent a few solid weeks digging deep into the Image I/O framework and the core plugins trying to find ways to improve performance. I've identified a few long term things we can do to optimize certain plugins even more, perhaps in the Dolphin timeframe. But along the way, I uncovered (and picked off) some low hanging fruit and fixed them in Mustang b63. I won't bore you with all the details, but if you're interested, you can read the evaluations of the following bug reports, all of which were fixed in b63:

• 6347575: FileImageInputStream.readInt() and similar methods are inefficient
• 6348744: PNGImageReader should skip metadata if ignoreMetadata=true
• 6354056: JPEGImageReader could be optimized
• 6299405: ImageInputStreamImpl still uses a finalize() which causes java.lang.OutOfMemoryError
• 6354112: increase compiler optimization level for libjpeg to improve runtime performance

Also in those evaluations you'll find lots of raw J2DBench numbers and analysis, but ASCII can be so boring. What blog entry would be complete without a Ross Perot-style chart? [Did I really just make a Ross Perot reference? How pathetic... Maybe next time I'll make a clever remark about Dan Quayle's potatoe?] Feast yr eyes:

That 20x20 JPEG score for JDK 1.4.2 isn't a glitch; it really was that bad (yet another demonstration of the evils of finalizers and explicit System.gc() calls)... Note that this chart only shows results for small images just because I'm tired and lazy, but similar improvements can be seen for larger images as well. For example, I've measured a 53% improvement in reading 4000x4000 pixel JPEG images between Mustang b62 and b63.

And yet somehow I'm still not satisfied... We'll continue to work on this stuff as time goes on. In the meantime, please download the latest Mustang snapshot and let us know if these Image I/O enhancements are helping your application.

While we're on the subject, just yesterday I received some exciting feedback from Jeffrey Moore and James Todd from the JXTA project. You may have read recently that MyJXTA now has a video conferencing feature, which uses MJPEG as the video protocol. It's not the most efficient video format out there, but it was easy for them to use the JPEGImageReader in Image I/O to assist in decoding the JPEG frames. They were asking about JPEG decoding performance and I suggested that they try the latest Mustang builds because of the fixes I mentioned above. It turns out that with the latest Mustang snapshots they are seeing CPU usage cut in half when decoding an MJPEG stream at 30 fps (compared to JDK 5)! The fixes above had a lot to do with it, but I know there have been other performance improvements across the stack (in HotSpot and elsewhere) that are probably contributing as well. This chart tells all:

When he's not working on one of his cool JOGL/Java2D demos, you can find Romain snapping some photos of the desktop Java geeks that work here on Sun's Santa Clara campus. If you've ever wondered about the faces behind the names, now's your chance to satisfy your curiosity. Or maybe you have a favorite Java2D or Swing bug that still isn't fixed? Here's what you do: print out the responsible engineer's photo, tape it to your office dartboard, and then play a little game of cricket... It may not get that bug fixed any sooner, but what better way is there to relieve the frustration of waiting?

Well, perhaps that was a tad sinister. I would never condone violence against my fellow Sun engineers (except maybe Chet, but I kid, I kid). If you really do want to see a languishing JDK bug fixed, let me once again recommend the JDK Collaboration project (aka Project Peabody). We've had lots of external developers digging into the JDK source code and submitting complete bug fixes, but we'd love to see more, so why not get involved? There are plenty of open bugs on the "starter list" that you can hack on to get your feet wet.

On that note, I recently had the pleasure of working with one such external developer, Rémi Forax, who submitted a really nice patch for this Image I/O RFE. This was a model submission: it wasn't just a cut-and-paste job, but instead included a more complete fix (using JDK 5 features, no less) to make the existing code even better, and to top it off it included the requisite testcase. Thanks for the great fix, Rémi! I hope to see more submissions just like that in the near future...

In my ears-uh: The Fall, "Bend Sinister"
In my eyes-uh: Peter Høeg, "Smilla's Sense of Snow"

Comments

• Hi,

  Does this part of the JVM share code with jai-imageio ?
  Could it benefit from this patch ? https://jai-imageio-core.dev.java.net/issues/show_bug.cgi?id=10

  Posted by: ycoupin on January 13, 2006 at 07:47 AM

• I really appreciate the performance improvements, but does libjpeg used vectorized assembler to produce optimal results for JPEG encoding and decoding?
  It seems that the JPEG code is still extremely slow. Intel used to have a free JPEG Image Library that had very fast code, now they have rolled that into a generic performance toolkit of some sort and I think it must be purchased (though still worth it for Sun I'm sure).
  

  Posted by: swpalmer on January 13, 2006 at 09:46 AM

• @ycoupin: I just checked out your patch, and it turns out that we don't have the same block of code in the core JPEGImageReaderSpi code, so fortunately the JDK is unaffected by that bug. Thanks for the tip though.

  Chris.

  Posted by: campbell on January 13, 2006 at 10:12 AM

• @swpalmer: Hey Scott, I saw your comment about the Intel JPEG libraries on one of the JDC bugs, so it did not go unnoticed :) Right now the native IJG libraries we use do not make use of vector instructions, so I agree that we would likely benefit from using an alternative library like the one from Intel. This is one of those things that is on the list of possibilities for Dolphin, but it's a potentially big change (lots of testing would be required). Last time I checked there was a trial version available from Intel, so perhaps you or someone else from the community would like to see what is involved in replacing IJG with Intel and investigating the performance impact? Sounds like a great "JDK research" project to me...

  Thanks, Chris.
  

  Posted by: campbell on January 13, 2006 at 10:22 AM

• Chris--I been trying out the Mustang builds occassionally (more freq. the last few weeks), with several Swing programs (jEdit and NetBeans among them) and so far it all looks good and yes, feels snappier. I ran the Java2D demos in the SDK and those also appear somewhat better, though it seems (sadly) that it's still easy to push the uses of J2D too far and get flickering in the images--this in the J2D demos as well as in well-publicized demos like Romain Guy's Twinkle. All of the JDK image stuff looks like it is getting better, looking forward to seeing where you all take it. Cheers! Patrick

  Posted by: pdoubleya on January 14, 2006 at 01:30 AM

• Some month ago I had to create some (good quality) jpeg previews of large jpeg photos (8Mpx+). With 1.5 the performace was terrible compared to the asociation of some linux programs (djpeg, pnmscale, cjpeg):

  djpeg -max 150m -pnm 8000.jpg | pnmscale -xysize 500 500 | cjpeg -quality 30 -progressive -maxmemory 150m -outfile 500.jpg

  I don't remember the exact figures but in Java the performace was 6 to 10 times worse on the same machine (the server VM had -Xmx250M). I'll try the mustang build some day but from the graph I would guess that the performance gap would be reduced to 3 - 5 times slower.
  I wish I could do everything in Java but right now the performance gap is too large.
  

  Posted by: imjames on January 14, 2006 at 02:20 AM

• @pdoubleya: The flickering you see in Twinkle is NOT due to Java2D. This is because it was my first time coding with OpenGL. It also depends a lot on you graphics card and video drivers. Blame the hardware, the vendo, OpenGL, JOGL or me, but not Java2D :)

  Posted by: gfx on January 14, 2006 at 01:32 PM

• @pdoubleya: Hey Patrick, thanks for trying out the Mustang builds. I'm not really sure what you mean by "flickering in the images"... Is this reproducible in Java2Demo? If so, drop me an email and let me know what we'd need to reproduce this. We always test all of the JDK demos and have not seen anything like this, so it would be good to find out if this is something specific to your configuration.

  Thanks, Chris.

  Posted by: campbell on January 14, 2006 at 01:54 PM

• @imjames: I should have mentioned in my blog entry that I compared IIO JPEG performance to the existing Java decoders (such as the Toolkit JPEG decoder) as well as some native decoders and encoders (such as the GdkPixBuf decoder). I found that the IIO JPEG decoder has lower overhead than other native solutions (so is usually faster for decoding small images), but is a bit slower than some native decoders for larger images (maybe 2x at worst). I did not see the performance differential you are seeing, but I didn't try djpeg or cjpeg. I will give that a try soon, maybe I could write a follow-up blog comparing them someday.

  I agree that we still have work to do. I won't be satisfied until the Java IIO decoders/encoders are as fast as, or faster than, native solutions. For some formats, such as PNG, I think we're already there, but JPEG still needs some work.

  Thanks, Chris.

  Posted by: campbell on January 14, 2006 at 02:14 PM

• I've noticed the improvement, and am very thankful for it. I noticed in b65 that my mapping application which uses large quantities of 500x500 and 1000x1000 jpeg images was faster to display the images.
  I've also noticed that my application starts significantly faster with b65 than it did with b55.
  Overall the 2d rendering speed has made enormous leaps and bounds from Java1.4 to Java5 and is now wickedly fast in Java6.
  Keep up the good work guys. This application was headed for the retirement heap, to be replaced with a .Net application, but now is back on the agenda.

  Cheers Oz

  Posted by: oznog on January 15, 2006 at 03:02 PM

• @romain: well, I can't believe your coding skills are the problem :), but OK, maybe it's not J2D. It's just that I have a laptop from last year, ATI x700 with 128M of video RAM, and I expect the graphics to be pretty smooth. For regular Swing apps, it is. With Twinkle, the animations are very smooth except for the scaling as images "fly away" from the screen. This has been my experience with graphics apps in Java generally: there's a comfort zone for the API, and when you step outside of that, you notice rendering problems. Unfortunately, that comfort zone is smaller than for native executables. But I'm really just a layperson with little education in graphics-related things, so will take your word for it.

  @campbell: I will send some notes by email about the "flickering" I'm seeing--I'm referring to rendering that is not smooth, some frames are dropped I think so the effect is not as good as it could be. This is on the J2D demo. I'll send my specs.

  Thanks for the replies, and btw I'm very happy with the progress in the toolkits, NB 5.0 on a laptop with Mustang looks a heck of a lot better.
  Patrick

  Posted by: pdoubleya on January 16, 2006 at 12:07 AM

• Note: The link to 6354056 is actually pointing at 6348744. Great to hear about all the performance improvements. How did jpeg manage to be faster than png, tho?

  Posted by: dconrad on January 26, 2006 at 12:40 PM

• It all sounds promosing.
  But can someone tell me what performance to expect when encoding a 1024*600 BufferedImage to PNG (simple vector graphics) ?

  On my AMD64 3000 machine, it takes about 1 second, and on my old Pentium2 400Mhz, it takes 7-8 seconds.
  The encoded PNG file ends up with a size of about 16kb.
  I've tested on JDK1.5.
  So I think this is not very impressive.

  I just downloaded an GifEncode from the 1999, and then the performance is 3 times better. (I know I'm comparing apples and oranges)

  I'm still investigating my numbers on jdk 1.6b70, I'll come back with them.

  Does anyone have an opnion on how milliseconds to expect to encode a 1024*600 PNG image on an AMD3000 or similar CPU ?

  Posted by: hogemark on February 08, 2006 at 12:17 PM

• I'm writing an application that needs fast JPEG reading to show MJPEG streams, so I'm particularly interested in the improvements described here. I made some tests with Mustang b71 loading 640x480 images but I didn't notice significant improvements respect to JDK 1.4.1. Maybe the code I'm using doesn't call the optimized JPEGImageReader.
  I obtain an ImageReader with this instruction
  ImageReader reader =
  (ImageReader) ImageIO.getImageReadersByFormatName("jpeg").next();
  then I set it's input to an ImageInputStream and finally I get a BufferedImage calling
  reader.read(0);
  Now my question is: am I using the optimized JPEGImageReader? If not, can anybody send me the right code?
  Thanks in advance.

  Posted by: lucac on February 23, 2006 at 06:48 AM

• Hi,
  I have similar problems to hogemark. The PNG encoder
  almost take 1 sec to encode a 1280x1024 BufferedImage. is this acceptable?

  JPEG encoding seams 4-5 times faster than PNG, it takes 150ms to encode the same image. is this ok?

  I am using ImageIO.write() , am I getting this new improvements
  automatically?

  Thanks

  Posted by: omerboyaci on March 06, 2006 at 04:48 PM

• Chris,
  I remember the first time I used ImageIO it was significantly slower than Toolkit for loading JPEGS. Is it still the case? If so any idea whne we it will be resolved.

  Posted by: navinkjha on March 08, 2006 at 02:07 PM

• It’s very good article. Great site with very good look and perfect information. I like it too.
  

  Posted by: teamagazine on November 05, 2007 at 12:09 AM

• It’s very good article. антимаулнетизм

  Posted by: poplop on June 13, 2008 at 02:19 AM

• Thanks so much for this! This is exactly what I was looking for.
  ///////////////////////////
  phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex mootzie
  and other video to SWF, FLV ecoded by Java.
  May it Gadgets helpful To your lifestyle... Enjoy！！

  Posted by: dante52 on June 19, 2008 at 06:33 AM

• _

  Posted by: dante52 on June 19, 2008 at 06:34 AM

• Thanks so much for this! This is exactly what I was looking for.
  ///////////////////////////
  phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex phone sex mootzie
  and other video to SWF, FLV ecoded by Java.
  May it Gadgets helpful To your lifestyle... Enjoy！！

  Posted by: dante52 on June 19, 2008 at 06:34 AM

• _

  <xmp -

  Posted by: dante52 on June 19, 2008 at 06:35 AM

This work is licensed under a Creative Commons License.