Chet Haase's Blog

VolatileImage: Now you See it, Now you Don't

Way back when we were first implementing the VolatileImage API, I had asked to come to the Swing staff meeting so that I could explain about the new VolatileImage API and why Swing should start using it.

I got up to the whiteboard, drew some pictures (probably horrible, but it's just a crutch for me when I explain things). I described this new image type, boiling it down to its basics:

1. It's going to be hardware-accelerated (depending on the runtime platform)
2. Rendering-to and Copying-from should be way faster than the current BufferedImage objects
3. It can go away at any time
4. Swing should use it for their back buffer

Then they asked me to go back to point #3; the whole disappearing-image thing. Apparently, they thought I was kidding.

It's true: VolatileImages can (and quite often will) go away and the API for VolatileImage was developed specifically to deal with that problem. All of the loss situations currently occur only on Windows.

You would think that we would actually be notified prior to these problems, so that we could prevent the loss, or backup the image, or something. But you'd be wrong; we only find out from Windows that there is a problem the next time we actually try to use the image. Surface loss situations arise from situations such as:

• Another app going into fullscreen mode
• The display mode changing on your screen (whether caused by your application or the user)
• TaskManager being run
• A screensaver kicking in
• The system going into or out of StandBy or Hibernate mode
• Your kid just yanked the power cord to the machine

Let's look at how you use the API to manage this situation. Then we'll step back and talk about other issues regarding these lossy beasts.

There are two parts to the API that need to be used in dealing with image loss:

	int valCode = validate(GraphicsConfiguration gc);
	boolean contentsLost();

validate() is used prior to using a VolatileImage. This tells us that you are about to use the image, so we need to make sure that it is in a state ready to be used. In most cases, all we do is check that things are fine and return IMAGE_OK. But in some situations, we need to do more:

• SurfaceLoss: One of the situations above that cause surface loss may have occurred since the last time you called validate(), in which case the underlying native surface memory is no longer there. In this case, validate() tells us to reallocate that memory or, failing that (there may not be enough VRAM, or some other failure might occur), allocate something in system memory that can also do the job. One of these is guaranteed to succeed and we will then return IMAGE_RESTORED from validate.
• Surface incompatibility: the reason that we take a GraphicsConfiguration argument to validate is that we need to know what you intend to do with the image before we can do the right thing to validate it. For example, you may have a dual-head system. Perhaps you created the VolatileImage on the primary graphics device. Later, you wish to copy that image to a Component on the secondary device. Surfaces are stored in graphics-device-specific memory, so we cannot copy an image from the VRAM on one device to another device. Or if we can, you probably don't want us to, because it would be a very slow operation. Instead, we ask you to tell us the target device for the operation and then we will tell you whether this is possible for the given image. If the GraphicsConfiguration that you pass in is compatible with the one that the image was created with, then we return IMAGE_OK, otherwise we return IMAGE_INCOMPATIBLE.

When you get a RESTORED error, you know that the image was in a funky state, but has now been reinstated and can be used. If you are about to write to the image and clobber the contents (as you would a back buffer, for example), this is not an error that you need to do anything about; the image memory exists and that's all you need to know. But if it's an image that you are copying from, or in some other way depending on the contents, then you need to restore those contents prior to depending on them.

For example, let's say you have an icon stored in a VolatileImage. At some point in your app, the image memory gets lost. The next time you go to copy from that image to the back buffer, you get the RESTORED code. Before doing the copy that you wanted to do, you need to recreate the icon using whatever means you need to do do that.

When you get an INCOMPATIBLE error, you know that your image is not in an appropriate state to do whatever operation you want to do with it. So, for example, if you are about to copy that image to a window with the GraphicsConfiguration that you passed in, that operation may not succeed. The only way to handle this error is to simply recreate the image on the new GraphicsConfiguration.

Now that you know what to do prior to using your image (call validate() and handle the return code appropriately), let's look at what you do afterwards. ("Afterwards? I'm done rendering, aren't I? How much do I have to do here? I thought Java was supposed to be easy, dangit!")

Notice that you had to call validate() prior to doing any rendering in case anything happened to your image since the last time you used it. But what happens if something happened to your image during your use of it? You might just want to know that some failure occurred. For example, let's say you are using a VolatileImage for a back buffer in some double-buffered application. You're in your rendering loop, you've called and handled the validate() call for the buffer, and you've done all the rendering to that buffer, and then copied the buffer to the screen. You're done, right?

Nope; what about if your image became lost during either the rendering to that buffer or the rendering from the buffer to the screen? Then chances are pretty good that either the copy to the screen simply didn't happen, or that some of the rendering either to or from that buffer is undefined. Do you want to just let it go and catch it later? Probably not, especially if this is some static application that only renders its contents based on changes in the app (versus some animation that's rendering one frame after another constantly); you don't want garbage hanging out on the screen or some required paint event just dropped on the floor. Instead, you want to make sure you get the right bits to the user as soon as possible.

This is where contentsLost() comes in; at the end of your rendering loop (or wherever you are doing rendering operations to and from your VolatileImage(s), you need to call contentsLost(). The boolean return value from this function will tell you whether the rendering operations from the iamge in question happened successfully, or whether anything went amiss during rendering. If the function returns false, everything is cool and you can just continue about your normal routine. if the function returns true, this means that the contents on that image are undefined and you need to:

• re-validate() that image
• redo whatever rendering you had just done to make sure it gets done correctly

So these are the two crucial pieces of the VolatileImage puzzle: you call validate() before you do anything with the image and contentsLost() after you are supposedly finished. And based on the return values from those functions, you do some other stuff to handle the errors.

We can put this into a simple loop which should probably look very similar in most situations. At first, I wrote some code that just showed the inner rendering loop. But to avoid confusion and allow easy copy/paste/compile/test, I've posted an entire (and functional) test app below. Sure it's boring (unless you really dig apps that draw single lines to the screen), but it shows the basic VolatileImage management we're after:

import java.awt.Color;
import java.awt.Component;
import java.awt.Frame;
import java.awt.Graphics;
import java.awt.image.VolatileImage;

public class VImageDemo extends Component {
	
    VolatileImage backBuffer = null;

    void createBackBuffer() {
	if (backBuffer != null) {
	    backBuffer.flush();
	    backBuffer = null;
	}
	backBuffer = createVolatileImage(getWidth(), getHeight());
    }

    public void paint(Graphics g) {
	if (backBuffer == null) {
	    createBackBuffer();
	}
	do {
	    // First, we validate the back buffer
	    int valCode = backBuffer.validate(getGraphicsConfiguration());
	    if (valCode == VolatileImage.IMAGE_RESTORED) {
		// This case is just here for illustration
		// purposes.  Since we are
		// recreating the contents of the back buffer
		// every time through this loop, we actually
		// do not need to do anything here to recreate
		// the contents.  If our VImage was an image that
		// we were going to be copying _from_, then we
		// would need to restore the contents at this point
	    } else if (valCode == VolatileImage.IMAGE_INCOMPATIBLE) {
		createBackBuffer();
	    }
	    // Now we've handled validation, get on with the rendering

	    //
	    // rendering to the back buffer:
	    Graphics gBB = backBuffer.getGraphics();
	    gBB.setColor(Color.white);
	    gBB.fillRect(0, 0, getWidth(), getHeight());
	    gBB.setColor(Color.red);
	    gBB.drawLine(0, 0, getWidth(), getHeight());

	    // copy from the back buffer to the screen
	    g.drawImage(backBuffer, 0, 0, this);

	    // Now we are done; or are we?  Check contentsLost() and loop as necessary
	} while (backBuffer.contentsLost());
    }

    public static void main(String args[]) {
	Frame f = new Frame();
	f.setSize(500, 500);
	f.add(new VImageDemo());
	f.setVisible(true);
    }
}

Stay tuned for my next blog; now that we've gone over the basics in how to use the VolatileImage API, we'll address some questions that you may have about using VolatileImages, or about this sample code in particular.

Comments

• bug found and fixed in sample code above
  Thanks to Tasos for finding and pointing out the bug in the sample code: I was mistakenly calling createImage() for the backBuffer instead of createVolatileImage().

  That'll teach me for just whipping out some code without actually compiling and running it...

  Posted by: chet on September 12, 2003 at 11:50 AM

• Infinite loop
  Thank you for the article, but if you try the code you'll get into
  a neverending loop: the code in the paint method shold be:
  ...
  } while (backBuffer.contentsLost());

  otherwise we'll exit as soon as there is a problem, and
  stay forever if everything works fine...
  Moreover, I see in the JComponent source code that there
  is a limit RepaintManager.VOLATILE_LOOP_MAX to the
  number the repaint on contentsLost is done (see line 4740).

  Thanks again for the article :-)

  Posted by: aaime on October 12, 2003 at 05:55 AM

• Infinite loop
  Whoops!

  Thanks for the pointer. Obviously, I was using an Infinite Loop Pattern here (see my earlier blog Patterns, Shmatterns...).

  The point about JComponent is more of an implementation detail; the Swing code chooses to loop a max number of times to avoid any possibility of inifinite loops. You could certainly choose to do the same when you manage VImages manually in your code. But given what I said above (and the way the code works internally) you should not have a problem with our code sending you into an infinite loop. Whether your code (like my example above before I fixed it...) spins on the wrong value is, of course, another thing entirely...

  Posted by: chet on October 13, 2003 at 08:05 AM

• Thanks for the article - the Javadocs for this are cabalistic.
  If you call super.paint() at the end you'll save some of your readers the pain of wondering why their children aren't showing up

  Posted by: anniepoo on February 06, 2007 at 04:51 PM

This work is licensed under a Creative Commons License.