 |
VolatileImage Q&A
Posted by chet on September 17, 2003 at 07:57 AM | Comments (11)
I originally wrote this as one single blog, but as I found out with
my previous BufferedImage articles, I tend to go on much longer
than I inteded at first, so I had to split it up. So now there's
2 parts to this blog: last time's VolatileImage usage discussion,
and this week's VolatileImage FAQ. This week, I thought I'd write
down various reasonable questions about either VolatileImage
objects and usage in general, or questions that might arise from
last week's example. These are questions that come up all the time,
or that might reasonably surface from the discussion so far. If you have questions
of your own, feel free to post them in the talk-back section at the end.
First, let's see that sample code again:
import java.awt.Color;
import java.awt.Component;
import java.awt.Frame;
import java.awt.Graphics;
import java.awt.image.VolatileImage;
public class VImageDemo extends Component {
VolatileImage backBuffer = null;
void createBackBuffer() {
if (backBuffer != null) {
backBuffer.flush();
backBuffer = null;
}
backBuffer = createVolatileImage(getWidth(), getHeight());
}
public void paint(Graphics g) {
if (backBuffer == null) {
createBackBuffer();
}
do {
// First, we validate the back buffer
int valCode = backBuffer.validate(getGraphicsConfiguration());
if (valCode == VolatileImage.IMAGE_RESTORED) {
// This case is just here for illustration
// purposes. Since we are
// recreating the contents of the back buffer
// every time through this loop, we actually
// do not need to do anything here to recreate
// the contents. If our VImage was an image that
// we were going to be copying _from_, then we
// would need to restore the contents at this point
} else if (valCode == VolatileImage.IMAGE_INCOMPATIBLE) {
createBackBuffer();
}
// Now we've handled validation, get on with the rendering
//
// rendering to the back buffer:
Graphics gBB = backBuffer.getGraphics();
gBB.setColor(Color.white);
gBB.fillRect(0, 0, getWidth(), getHeight());
gBB.setColor(Color.red);
gBB.drawLine(0, 0, getWidth(), getHeight());
// copy from the back buffer to the screen
g.drawImage(backBuffer, 0, 0, this);
// Now we are done; or are we? Check contentsLost() and loop as necessary
} while (backBuffer.contentsLost());
}
public static void main(String args[]) {
Frame f = new Frame();
f.setSize(500, 500);
f.add(new VImageDemo());
f.setVisible(true);
}
}
And now, on with the questions:
Q: So when do I use VolatileImage objects? Should I use them instead of
BufferedImages, or managed images?
A: Ah yes, the plethora of image types and the confusion that
quickly sets in. The quick answer is "You probably do not need
to ever use a VolatileImage object directly in your application."
This answer assumes that most applications:
- Are Swing apps, and therefore are already using a VolatileImage
back buffer because Swing does it for them
- Do not need or want the extra complexity inherent in using these
images that must be managed.
- Do not need every ounce of performance and runtime details
available
So much for the quick answer. The long and much more descriptive and rewarding answer is "You might want to use VolatileImage objects sometimes."
(Actually, this long answer is much shorter than the quick answer
above, but you'll see that the underlying explanation behind this
glib response is not so brief).
The main reason for anyone to ever use a VolatileImage object is
when you need a dynamic (frequently updated) image that you would
really like to be accelerated if possible, including both
rendering to the image and copying from the image. This is the
basic usage model of a back buffer; you want it accelerated and
you are rendering to the image frequently (at least once per
screen update, by definition). This is why Swing uses a VolatileImage
for its back buffer.
Using a BufferedImage or a managed image in this kind of scenario will instead give you software-rendering performance. Because of the volatile nature of accelerated memory (particularly on Windows), any image that is not explicitly managed for loss (which means any image that is not a VolatileImage) must reside primarly in system memory. That way, any rendering operation to that image is captured in that system memory version and cannot be destroyed through the loss of an accelerated version of the image. This implies that all rendering to the image must occur first through software rendering loops (since we cannot use hardware acceleration to render to a system memory image). Then, if it makes sense, we may copy that image into accelerated memory for future copies from that image. There are two main problems with this for a back-buffer scenario:
-
Performance: for simple rendering, hardware acceleration will generally win out over software rendering performance. There are various reasons for this, including the speed of the graphics processors, the inherent parallelism of offloading operations to the graphics chip, etc. But in the end, just remember that's why they call it "accelerated". So if your back buffer resides in system memory (if you are not using a VolatileImage), then you must make do with software rendering performance.
-
Performance: Although the render-to performance is important for the back buffer, sometimes the bigger win comes in the simple copy-to-screen operation that back buffers do on every frame. Ideally, this copy happens between an accelerated back buffer living in VRAM to the screen, also living in VRAM, and can happen at amazing speeds due to the memory bandwidth in the graphics system. But if your back buffer lives in system memory, this copy must happen over the system bus: reading from system memory (hopefully the on-chip cache, at least) and copying to VRAM down the PCI or AGP bus. Imagine trying to render 60 frames per second in an animation application, with a screen size of 1280x1024 and 32bpp screen depth; that's about 5 megs of data per frame * 60 frames = 300 MBytes of data moving across the bus, just to copy the buffer to the screen, not including anything else that needs to happen over the bus or through the CPU/cache/memory. "But what if the back buffer is a managed image?" Unfortunately, this is no better than an un-managed image; because you are rendering to the back buffer every frame, there is no benefit to our copying that buffer down to a VRAM cached version; we'd have to do it every frame (since the contents of the primary version of the image are being updated every frame), which is no better than simply copying it directly to the screen every frame.
Q: Couldn't the contenstLost() rendering loop become an infinite loop?
A: No. At least, not in theory. validate() will attempt to allocate
an accelerated surface. Failing that, it will allocate the surface
memory in system memory. So when you get either an IMAGE_OK or
and IMAGE_RESTORED, you can guarantee that there is something
there to render to (and from). And if you get an INCOMPATIBLE
error, you must recreate the image completely, which again will
result in at least something you can render to and from.
So if nothing happens to that surface between that validate and
the contentsLost(), then contentsLost() will return false and
you can continue. The only situation I can foresee where you
might get stuck in this loop more than one time around would be
some weird system thrashing case where we are able to allocate
VRAM, but then it's clobbered by the time we get around to the
end of the loop, then we can get the memory again by the next
validate, then it gets clobbered, and so on; this is pretty
unlikely in reality, and if this is really happening on the system, there are probably more disturbing things to worry about than spinning in this rendering loop too many times.
Q: What's with the extra null-check condition in createBackBuffer()?
A: This is an optimization tweak for those interested. There
are a couple of things going on here:
backBuffer.flush();
This call tells us to release any memory associated with this image.
This will happen anyway when we reassign backBuffer to a new
value; the old object will eventually be GC'd (Garbage Collected). But the key word
here is "eventually"; we do not know exactly how long it will
take for that GC operation to kick in. When dealing with the
constraint of small and finite video memory, we need to be as
proactive as possible and make sure that we do not have
anything wasting valuable VRAM space. For example, what if there was
space for exactly one back buffer? If we had an old version
of the back buffer around and then went to create a new one without
first removing the old one, we would not have space in video memory
for the new one and would be forced to create it in system memory.
Eventually, the old one would go away and we would re-create the
new one in accelerated memory, but why not get it right the first
time?
The second item is this condition is even tweakier:
backBuffer = null;
This one is more superfluous, given the flush() call above. I put it
in just to bring up an interesting point (and one that was a complete
mystery to me until I ran into it). In general, the garbage collector
will not collect any objects until they are dead; that is, until there
is no reference to them. By assigning null to backBuffer, we ensure
that the next time the garbage collector runs, the object formerly
pointed at by backBuffer (sounds like the monicker of a rock star...)
is up for collection. If we took the more obvious and usual
approach of simply reassigning backBuffer to the new object
we create, the end result is the same, but the timing is not.
Let us say that the operation of assigning the new object to
backBuffer causes the collector to run to try to get more memory.
It can only get memory from objects that are already dead; but
since the old backBuffer object is still alive until this
reassignment is done, that object is not available for
collection.
As I said, this extra step is somewhat useless here; since we
already nuked the video memory taken up by this image by using the
more definitive and direct approach of calling flush(). The only
possible memory savings here is that Java heap memory taken
up by the image data structure. This is not enough to worry about.
But I brought it up because I have run into problems with this in
other image contexts. For example, let's say (for some unknown
and perhaps suspect reason) that our rendering loop creates a
new BufferedImage every time through the loop for some intermediate
calculation. Images can get pretty big, so this one object can
be enough to swamp the young generation of the collector; but if
we have to have 2 of these objects live simultaneously, then there's
a fair chance that we're going to start chucking objects into the
tenured generation and thus requiring Full (aka, "expensive") GC
operations to get that memory back. Much better to get rid of the
old one before asking for another one.
I don't want to get too deeply into GarbageCollection issues and
implementations (and I think I've reached the limit of my knowledge
on the subject in any case), but I did want to mention the cases above because
as graphics and image users, we tend to run into unique garbage and memory
situation, so it's worth understanding more about this stuff.
Q: If I have several VolatileImage objects, can I just call the
image-loss methods on one of them? Won't I get the same answers from
all of them and thus working with just one of them will be just as good.
A: Yes, but NO.
I say NO with Capital Letters because I have, in fact, run
into this exact bug in a simple demo app I was writing recently.
Whoops!
It is correct that the images will probably all return the same
values (if one image gets lost, all will be lost). At least this
is probably true now, but it is not guaranteed to always
be true. For example, at some point we will implement image
priorities and may actually force some images to be punted out
of memory, thus causing selective loss instead of the wholesale
loss we now see due to operating-system funkiness.
However, even if we could always count on all of the image attributes
being the same, that does not mean that reinstating any one image
will automatically cause all images to be reinstated.
Example time: Let's say we have one VImage sprite and a VIimage
backBuffer. We could write a buggy rendering loop like this:
int valCode = backBuffer.validate(comp.getGraphicsConfig());
if (valCode == VolatileImage.IMAGE_RESTORED) {
// backBuffer does not need to be restored,
// but sprite does - initContents() is a method
// that re-renders the contents into the sprite
initSpriteContents();
} else if (valCode == VolatileImage.IMAGE_INCOMPATIBLE) {
// recreate all volatile images
createBackBuffer();
createSprite();
// also restore contents of sprite since we now have a
// completely new image
initSpriteContents();
}
// rendering to the back buffer:
Graphics gBB = backBuffer.getGraphics();
gBB.drawImage(sprite, 0, 0, this);
// copy from the back buffer to the screen
g.drawImage(backBuffer, 0, 0, this);
// Now we are done; or are we? Check contentsLost() and loop as necessary
} while (!backBuffer.contentsLost());
This code is exactly the same as the code way above except that
we also use the return values from validate() on
the backBuffer to do whatever is necessary to the sprite.
Here's the problem: images only get reinstated (put into a reasonable
state to render to/from) when you call validate() on that image.
So if you get a IMAGE_RESTORED value from backBuffer.validate(), then
chances are pretty good that the sprite is also lost ... and
is staying lost. The code above calls initSpriteContents()
to re-render the contents of the sprite, but these operations
will fail because the sprite is not in a state that will accept
rendering.
The single contentsLost() call will probably work for now, but as I
said above, if any images are every selectively punted that could
change and the above code would be wrong under the mythical future
implementation.
A more correct version of the above looks something like this:
int valCodeBB = backBuffer.validate(comp.getGraphicsConfig());
int valCodeSprite = sprite.validate(comp.getGraphicsConfig());
if (valCodeSprite == VolatileImage.IMAGE_RESTORED) {
// note that we do not care about the back buffer
// for restoration since the buffer will be
// re-rendered here anyway.
initSpriteContents();
}
if (valCodeBuffer == VolatileImage.IMAGE_INCOMPATIBLE) {
createBackBuffer();
}
if (valCodeSprite == VolatileImage.IMAGE_INCOMPATIBLE) {
createSprite();
// also restore contents of sprite since we now have a
// completely new image
initSpriteContents();
}
// rendering to the back buffer:
Graphics gBB = backBuffer.getGraphics();
gBB.drawImage(sprite, 0, 0, this);
// copy from the back buffer to the screen
g.drawImage(backBuffer, 0, 0, this);
// Now we are done; or are we? Check contentsLost() and loop as necessary
} while (!backBuffer.contentsLost() && !sprite.contentsLost());
Bookmark blog post:  del.icio.us  Digg  DZone  Furl  Reddit
Comments
Comments are listed in date ascending order (oldest first) | Post Comment
-
A few questions
Hi Chet!
Those two articles about the VolatileImage are really helpful!
Thanks a lot!
But I have still two questions:
I don't know why I should call contentsLost() at all. If my contents may be lost at ANY time, they may be lost right after I started to feel comfortable because my call to contentsLost() returned "false". I know that I must be wrong, so can you tell me what this call does guarantee me? Shouldn't the call to contentsLost() and the command to make the backbuffer visible be a single atomic operation, so that the contents may not be lost in between?
The second question is about BufferStrategy. I'd like to use page-flipping for my game engine, so I am using this class. Is the backbuffer automatically a VolatileImage? If not, how can I use VolatileImage (for a backbuffer) with page-flipping?
Regards, Patric
Posted by: gwydion on September 17, 2003 at 11:41 AM
-
A few questions
I keep meaning to reply to this, but there's a big freeze this week for our 1.5 work, so my blog efforts tend to fall prey to my priority scheme...
The quick answers here are:
- contentsLost() should be called after every series of calls dealing with the VImage(s); this ensures that if something happened _after_ you validated the surfaces, during your rendering operations, then you'll find out that something went kaphlooey and you need to loop around again to handle it. Yes, you can lose the image contents right after you call contenstLost(), but if you do things properly and do not have any code _outside_ the contentsLost() loop that deals with the VImage(s), then you should not care about this loss until the next time you hit your rendering loop.
- An atomic operation does not do the same thing: you need to make sure the images exist _before_ you render (hence the call to validate at the beginning of the loop), then you need to make sure everything during rendering went okay (hence the call to contenstLost() at the end of the loop).
- BufferStrategy vs VolatileImage: This is a good Q to fold into the article. In fact, I had every intention of doing so before I posted the blog (the same question came up on a javagaming forum recently), but I opted for just posting it sooner instead. In the meantime, until I take the time to fold it into the article: if you can use BufferStrategy, do so; there is no reason to use VolatileImage instead or in addition to BufferStrategy. In fact, BufferStrategy is wrapped around VolatileImage objects, handling some of the management for you (note that you can query whether the contents were lost, but the validate() is done for you automatically, whenenver you request the drawGraphics of the BufferStrategy).
Posted by: chet on September 19, 2003 at 08:01 AM
-
-Dsun.java2d.noddraw?
It is interesting how these flags affect volatile images:
-Dsun.java2d.ddoffscreen=false
-Dsun.java2d.noddraw
-Dsun.java2d.d3d=false
Posted by: xcheffo on November 24, 2003 at 07:38 AM
-
-Dsun.java2d.noddraw?
Here's what those flags actually do in the library:
1) noddraw: This one disables ddraw (actually, all of DirectX) acceleration completely. That means:
- No VRAM-cached surfaces (images live in system memory only)
- no ddraw for onscreen copies (currently used for scrolling; ddraw is way fast at copying rectangles around on the screen)
- no ddraw locking/unlocking for copying pixel data around (this can be a fast way to get bits onto the screen from system memory images, but actually we stopped using this on win2k and XP as of 1.4.1 because of cursor flickering and other animation artifacts (like jittery menu fades)).
So if you use noddraw, VolatileImages are actually not Volatile, and not accelerated.
2) ddoffscreen=false: like noddraw, except we may still use ddraw for onscreen copies and for some lock/copy/unlock operations. But no VRAM-cached surfaces are used.
So if you use ddoffscreen=false, VolatileImages are actually not Volatile, and not accelerated (like noddraw in that respect).
3) d3d=false: This one just disables d3d. We currently only use d3d for drawing lines to a VRAM-cached DirectX surface. This is way faster in general than drawing the lines via our fallback renderer (currently GDI) because of data copying and context switching. However, d3d can be flaky on some cards. We run simple tests to make sure that line quality is okay in general, but it's nice to have an out if necessary.
We also use d3d for accelerating translucent images via texture mapping, but that capability is not enabled by default (gotta use another flag for that one for now). So d3d=false will also disable that capability.
Chet.
Posted by: chet on November 24, 2003 at 07:46 AM
-
java.awt.VolatileImage and transparency question
Couple of months ago I tried to convert my old applet
(small 2D scrolling game) to use the new copyArea() method in java.awt.Graphics[my tests show that blitting with copyArea() is apx. 10times faster than copying gfx between two java.awt.Images with drawImage() ].
It works fine [for backgound/background tiles], however I got problems with transparent images - sprites in my game [I use both transparent GIFs and PNGs].
The transparency is lost - my sprites appear on a
black rectangle. Therefore I use copyArea() only to create the backgorund, and the old drawImage()
to copy the sprites [into the backbuffer, the result is later copied to the frontbuffer].
what I wanted to do was:
1. create a backbuffer - java.awt.VolatileImage
with size of the fronbuffer[screen] + extra [hidden]
part to store my gfx [backgroud tiles, sprites]
2. in the game rendering loop using the copyArea()
copy the tiles and sprites to the backbuffer "screen area"
3. blit the backbuffer "screen area" into the frontbuffer
as mentioned above the transparency got lost once I copied the sprite Image into my backbuffer's
[java.awt.VolatileImage] "hidden area"
cheers,
hacklberry
my game [preview] could be viewed at:
http://members.iinet.net.au/~grimble/test_tile_engine_20031120.html
[cursor keys + CTRL for fire]
Posted by: hacklberry on December 15, 2003 at 10:10 PM
-
Scrollpane Acceleration
I'm developing a digital logic simulator for a senior project. Basically it displays logic gates (AND, OR, etc.), and connections between them. Different inputs can be tested in real time (Flip a switch, see if a lightbulb lights up). So I'm displaying a few images of gates over and over, some text, and a bunch of lines (mostly horizontal or vertical). Graphically, it's really similar to a flowchart. Here's my issue: These diagrams could span several screens. The obvious implementation for a large diagram is just a big JScrollPane. Will this approach kill all hope of acceleration and good graphics performance? Would it be better to implement my own scrolling window, with algorithms for where things should be drawn, or if they should be drawn at all?
Posted by: kj12345 on January 14, 2004 at 05:45 PM
-
Here's a suggestion for kj12345 (by the way, I think I know your uncle thx1138 - he's in
the movies, right?)
Instead of a JScrollPane, you could layout your wiring diagram on a JTabbedPane.
The diagram would flow from left-to-right, and the individual panels could be the height
of nearly the entire screen.
Just a suggestion to consider.
Cheers,
Peter
Posted by: pvdl on September 30, 2004 at 07:24 AM
-
java.awt.VolatileImage and transparency question 2
I create VolatileImages like this:
GraphicsConfiguration gc = g.getDeviceConfiguration();
img = gc.createCompatibleVolatileImage( orig.getWidth(null), orig.getHeight(null), Transparency.TRANSLUCENT);
But when i try to draw them on screen, they will loss the whole content. They are painted simply white.
Is there a way to get around this? I need the speed at all cost and of course TRANSCLUENT Images.
Posted by: tobain on December 20, 2004 at 03:10 PM
-
Is it possible to accelerate an image that's in native memory? For example creating an image from a direct java.nio.ByteBuffer and outputting it directly to the graphics card without copying it into a byte[]. It would be very useful for video, where you can't afford to copy every frame. I think it would be feasible by simply adding a new DataBuffer and using regular BufferedImages, but I guess it would have to be accelerated internally. Could this become a feature in Mustang?
Posted by: sat1196 on February 01, 2005 at 02:41 PM
-
do you know if there is any issue in using volatile and linux?
Posted by: ilredeldomani on April 24, 2006 at 08:45 AM
-
2 ilredeldomani:
There are no issues with using VolatileImages on linux, but they are accelerated in a different way - by using Pixmaps, which may or may not be placed into VRAM by the video board driver.
Dmitri
Java2D Team
Posted by: trembovetski on April 24, 2006 at 03:38 PM
|
java.net RSS Feeds
