Chet Haase's Blog

August 2003 Archives

BufferedImage as Good as Butter, Part II

Let's dive briefly into some of the performance implications with BufferedImages (because I'm writing this and I tend to wind up in the performance arena no matter what I'm talking about).

There is currently only one image type that we guarantee acceleration on (if possible on the runtime platform): VolatileImage. When you create one of these images, we try to stash it in accelerated memory (such as VRAM on Windows, or an accelerated pixmap on Unix) and then perform rendering operations to and from that image using any available hardware acceleration.

VolatileImages work well for things like back buffers (ala the Swing back buffer, which is now a VolatileImage), where you obviously want to render to them frequently and copy from them as fast as possible. But for your average image, managing a VolatileImage can be tiresome (you have to make sure it's there before and after you use it), and you can't get all the flavors of images you want (currently only opaque volatiles exist).

But think about your average application: there's the back buffer and screen that you write to often, so you want those rendering-to operations to go really really fast. But then there's a bunch of other images like icons, sprites, whatever which you really only write to once or occasionally, but from which you would like to copy often.

This is where Managed Images come in (our new catch-phrase which, roughly translated, means "we will try our darnedest to accelerate this for you"). Here, you create an image however you need to, start working with it, and internally we will recognize that these copying operations can go much faster using an accelerated version, so we will just create that cached version for you. You don't need to manage the image and you don't need to know how these operations are happening; you just keep calling your rendering operations and let us take care of the pesky details.

Now, the fine print: as of 1.4.*, we hooked out only certain parts of the API to create these managed images. Specifically, you need to create an image either by calling one of the create*Image methods:

GraphicsConfiguration.createCompatibleImage(w, h)
GraphicsConfiguration.createCompatibleImage(w, h, transparency)
Component.createImage(w, h)

So in the current implementation of Java2D, the advice posted by ajsutton in response to Part I of this BufferedImage article is well-taken: if you want to take advantage of possible acceleration for your image, use a compatible image (or one of the other means above). Then we will attempt to accelerate this for you. Another benefit of using a compatible image is that you will get an image that is "compatible" (thus the name of the method above) with the display device you are rendering to, which saves pixel format conversion during the copy loops.

(A further caveat is that not all image types that you get from the above methods are acceleratable. For example, if you create an image with the flag Transparency.TRANSLUCENT then we do not currently accelerate that image and you end up going through software rendering loops regardless. Look for this to change as the library evolves and we try to accelerate more and more standard yet nifty features of the API).

Okay, so that's the state of things now: use BufferedImage for all of your condiment image needs, and if performance is particularly important to you, then use one of the variants mentioned above is the way to go. But what about the future?

Gosh, I'm glad you asked that. What a great question.

You may already be wondering, in reading the above explanations and caveats: "Why can't they just accelerate everything? Why are only portions of the API managed?" In fact, this is totally correct; there is nothing preventing this from happening (other than the most obvious of reasons: time to implement and lots of other stuff that we've been working on in the meantime). For example, let's say you have a 16-bit BufferedImage you created from scratch and you want to copy it to a 32-bit display. This means that we have to do a pixel-format conversion, so we can't cache the 16-bit version, right? But we can cache a new 32-bit version; we just copy the 16-bit version to our new 32-bit cached version, and then simply use the cached version thereafter.

Starting in jdk1.5 (currently in the oven, baking for a while, available at some unspecified (by me) date in the future), we will manage a much wider array of images. In fact, most of the images you can create or load will be managed for you. The code is in there, I've seen it working with my own eyes: BufferedImage objects running as fast as compatible images. It's pretty sweet...

That's all for now. I've glossed over many of the details of images and acceleration, but hopefully I've given a taste of how accelerated images work today and in the future. And hopefully you will be able to use this information to get the fastest and tastiest BufferedImage applications possible.

BufferedImage as Good as Butter

Recently there was a brief discussion on the java2d interest list. We were talking about some of uses of BufferedImage and someone replied to the group with their epiphany on these objects with just a brief: "BufferedImage as good as butter!"

The Java2D team has spent the last few days trying to understand and come to terms with this phrase. Was it praise? Irony? Sarcasm? An epigram? A typo? Does it mean that BufferedImage is yellow? Or creamy? Or that it spreads well if left out of the fridge, but melts if it is too hot outside? Is it a condiment? Does it go well with bread? Is it a substitute for vegetable oil?

I have decided it was actually one of those opaque philosophical phrases that you might see in a fortune cookie (albeit one that you got from a very geeky restaurant). I do not yet understand its true meaning, but I hope to walk far enough on the path to enlightenment to one day glean its essence.

In the meantime, I will arbitrarily choose one possible meaning for the purposes of this article: BufferedImages are slick.

One of the things that we and our developers run up against again and again (and again) is the vast array of possibilities when dealing with images and image operations in our APIs. In particular, confusion appears to reign when developers first learned Java in the 1.0/1.1 days (whose APIs are still working and prevalent in some browsers' implementations of Java) but they are now trying to apply that knowledge in the current API and implementation.

So, for example, someone could easily assume that ImageProducer/ImageConsumer is still the way to deal with creating and using images. Or that PixelGrabber is a great way to get at the actual pixel data in an image. Or that MediaTracker is a really cool way to ensure that an image is loaded before you use it.

Then these developers run into the new (as of 1.4) ImageIO APIs and wonder how they get from their Image to a BufferedImage.

Might we suggest, instead, trying to use BufferedImage for most, if not all, of your condiment image needs. There are very few situations where you really want or need to use the old Image APIs and thus translating between that world and the new world of BufferedImage is really unnecessary. ImageProducer/Consumer is an interesting way of dealing with Images that are coming over the net and won't arrive until some nondeterministic time in the future, but if you have the image on disk and simply want to load it in, that's a pile of work to go through (and some pretty non-obvious API calls) just to get your hands on the data. Or if you want an image that you can access the pixel data on, you can sure use PixelGrabber, but wouldn't it make more sense to simply call get/setRGB on a BufferedImage? Or if you want an offscreen image, wouldn't it be nice to simply be able to create an image of a specific type (or one compatible with the display) and simply give it the width and height to use?

In particular, check out the simple commands like:

new BufferedImage(int width, int height, int type)

Or how about:

GraphicsConfiguration.createCompatibleImage(int width, int height)

Then there's always the entire ImageIO package, which deals exclusively in BufferedImage objects. Instead of using the old Toolkit APIs to load an Image, you can use ImageIO to read images of more varied types (such as PNG) which will then return a BufferedImage.

And remember too that converting from one image type to another is usually as easy as a simple copy command. So, for example, if you happen to have one of these old Image objects lying around and you really want to get at the pixels, or use ImageIO, or just revel in the power and glory of having a BufferedImage representation of that image, you can always do something like this:

// your other image type
Image img;

// new, cool representation of that image, created by one
// of the means above, with width/height equal to img's
BufferedImage bImg;

// Get the Graphics object for the BufferedImage
Graphics g = bImg.getGraphics();

// Copy the old image into the new BufferedImage object
g.drawImage(img, 0, 0, null);


So store it in the fridge, keep it out on the counter, spread it on your favorite wheat products and applications; BufferedImage is as good as butter ... and maybe even better.

ps: Did you notice I didn't touch on that hot "performance" topic anywhere in this article? Stay tuned for the next article, in which I'll talk about some of the performance implications and implementation details of BufferedImages.

Patterns, Shmatterns

Seeing as how this is my first blog, I thought it was reasonable to go on a complete tangent instead of spending my soapbox time rambling on about upcoming features and cool Java graphics things. Maybe I'll get into that stuff in a later column...

For years now, we've been hearing about how Patterns and Components will make our code much better, easier, reusable, etc. etc. etc. (etc.). But I have yet to see patterns or components created for the most commonly used and recreated things of all: bugs.

I find myself continually writing new code with completely new features, yet I invariably end up inserting the same type of bugs in it. There's the array overrun error (at least in native code). Then there's the "Whoops! I used the wrong local variable to dereference my data structure inside of this loop bug." There's the "I forgot to initialize that variable bug." Or the "I forgot to Unlock that thing that I had previously Locked" bug.

Think how much easier and better and cooler our code (and lives) would be if we no longer had to worry about creating these problems from scratch, but could just rely on standard patterns, or even reusable components to do them for us?

"I need to fail out of this loop right about here; what if I use the BadDereference pattern?"

"I see you have a BadLoopVariableIncrementor pattern at work here. Let's try taking that out and see if the code works."

"My bugfinder script found 87 bugs in the 4 new files you've checked in. This is 6.7% more than last month; we need to make that number go down, not up."

Life will be so much better with BugPatterns and BugObjects. The promises of object reusability will finally come true; after all, what single object in the software world is more used than a bug?

And why do you think they called it Oops to begin with?

I'll get to work soon on these patterns, and then a library of BugObjects. I'll try to open source the project so that everyone can contribute. If we can get everyone to participate, then maybe we can nail the whole spectrum of bugs and we won't have to write any of our own bugs anymore!

First I have to finish working on my jdk1.5 features, which includes fixing various bugs. Man, if I could only fix them by grep'ing through the code for BugObject instances...

Chet.
(Currently working on 1.5 features and bug fixes, including new VolatileImage API and fixing various things related to hardware acceleration on Windows. Wish me luck.)

This work is licensed under a Creative Commons License.