Chet Haase's Blog

February 2006 Archives

Make Your Animations Less Ch-Ch-Choppy

This article is Part One of a 2-part series. In Part One, I look at some of the problems contributing to choppy animations. In Part Two, Smooth Moves, I look at some of the possible solutions and offer a demo application for trying out some of the results.

The Problem

I was working on some animation code the other day and ran into some artifacts that made the animation look choppy. In this particular example, there were both fading and moving/scaling animations. The fade looked great, but the animation with movement looked stuttery. This was particularly odd since the fade happened over the entire screen, whereas the scrolling animation occupied only a small portion of the screen.

It seemed obvious that the fading animation must be quicker than the other one and that we were hitting some performance bottleneck on the other animation.

I inserted some simple timing code in both animation loops to verify this easy assumption. Strangely, I found that the fade was happening at a rate of about 10 frames per second (fps); not very fast, considering that minimal frame rates for animation are usually considered to be 30 fps, although 60 fps is a more usual minimal for highly interactive content such as games. Stranger still, I found that the other animation was getting about 15-20 fps; faster than the visually smoother fading animation.

So here was the quandary; why did the large fading animation with a lower frame rate actually look better than the small scrolling animation with the higher frame rate? It just didn’t compute.

I had stumbled, once again, across another example of perceived performance. This topic comes up often in the GUI space, where the true performance of an application is often not as important as how fast the user thinks it is. This is why, for example, you should run your long non-graphical operations on a separate thread from the GUI thread, so that if your app has to go out and query the database for several seconds, at least the GUI is not frozen waiting for that to finish. Make the application GUI snappy, and the user will be happier with the performance of the application - even if the data takes exactly the same time to actually populate the GUI. Perceived performance also relates to the gray rect fix in Mustang that Scott Violet and I blogged about; the application may not actually be any faster, but it certainly looks faster to the user, and that’s worth a lot. In this case, I perceived the fading animation to be more performant than the scrolling animation simply because it looked smoother.

After this realization, I figured that it was time to investigate the factors that could affect the smoothness of an animation; what makes an animation appear choppy, and what could be done to reduce or eliminate the choppiness.

What Makes an Animation Choppy

There are various elements that contribute to choppy animations, some of which were factors in my application, but all of which are worth considering whenever you develop animation code.

I’ll break down the factors into three main categories: timing, color, and vertical retrace.

1) Timing is (Nearly) Everything

In general, you want your animations to be speedy. I don’t mean that they should zip across the screen as fast as possible (you have to give the player the chance to actually hit the alien spaceship, don’t you?), but rather that the animations should move quickly in small increments. There are various elements related to the speed at which an animation can run: raw performance, framerate, consistency, and realistic movement.

Performance

Raw performance is one of the most important factors affecting animation smoothness, and probably the most obvious; if you run your animation faster, it will tend to look smoother. This is because the amount of change between each frame of your object is smaller, and thus the eye will tend to perceive the steps as being smoother. The way to improve the performance of your application is by making your rendering faster. Are you wasting time drawing anything in a sub-optimal way? Are you drawing anything complex that could be rendered into an intermediate image instead? Are you limiting your rendering to only the area affected by the animation, or are you re-rendering the entire window? All of these factors, and more, can contribute to slower rendering and, therefore, a slower frame rate.

Framerate

Framerate is closely related to Performance: You want to make the time between sequential frames of your animation as short as possible. One way to affect framerate is by improving the performance of your rendering code. Another way is by running your animation at a different “resolution”. At the core of your animation is a timer, which will kick off timing events at some specified interval. You can set this timer up to call you every half-second, which would result in a very unanimated animation running at a whopping 2 frames per second (fps). Or you can set it up to call you every 30 milliseconds, which would give you a more reasonable 30 fps (assuming your rendering loop could do its job in that small window of time to maintain this frame rate).

Consistency

Ideally, you will have as small a timer resolution as possible so that your animation can move to the next state very quickly. However, if there are some states that require significantly more time to render, or if other things are happening in your program that make this frame rate unachievable, then you will end up with intermittent pauses in your animation. These pauses do not have to be very long to be noticeable to the user. Let’s say that your animation runs swimmingly at 30 fps most of the time, but takes a time hit of about 100 ms every 4th frame. The net result is that the user would see your animation run relatively smoothly and then pause and jump at frequent intervals, several times per second.

It is far better to set a frame rate that you know is achievable in most situations than to have a jumpy animation; an animation running at a consistent 20 fps is better than one running at 30 fps with occasional noticeable lags and jumps.

Related to consistency is using time-based animation, rather than speed-based animations. A very simple approach to animation is to update the animation state to the next step the next time you are called. This works on systems where timing is very predictable, but breaks down when you run on different systems, or when things happen to perturb the timing between steps. It is far better to base your animation state on the time that has elapsed; that way the animation will always proceed in a logical fashion, regardless of how much time passes between steps. I discuss this more in the “Animation Fraction” section of Timing is Everything.

Realistic timing

The simplest motion to calculate is linear interpolation. That is, for any fraction of the elapsed animation, you might move the object by that fraction between the starting and ending points of the animation. It's easy in code:

                x = x0 + (x1 - x0) * fraction
                

2) Color: What’s the Difference?

I think the Timing section above details probably the most obvious element here: if you want to make your animation look better, run it faster, with smaller incremental steps, at a consistent and realistic rate. This next section is about a factor that is perhaps less obvious, but one which contributed greatly to the choppiness I was seeing in my particular application.

The apparent smoothness of an animation is strongly related to the rate of change of the pixel colors affected. That is, the display pixels affected by each of the animations in my application (the pixels being modulated in the fade and those being altered as the scrolling animation moved across the window) were changing at some rate, and that rate was perceived to be smoother in the case of the fading animation.

Let's take a step back and think about what makes an animation smooth in general. It's pretty simple to understand that having a higher frame rate will give (or render, if you're one for topical and dull puns) a smoother animation. Imagine trying to animate an image from the left side of this page to the right over the course of one second. You could simply move it from left to right in one step; the image sits over on the left until the animation begins, at which time you erase it from the left side and draw it on the right side. You've just animated that movement at an enormous 1 frame per second. Of course it's only “animated” in the technical sense; it didn't really animate at all, it just performed a single movement.

I think you'll agree that that animation would be just a bit choppy.

Let's try for something a little better. Let's divide the distance into 10 steps and perform the animation by moving the image a smaller distance in each of those steps, ending up at the same place as before. This looks a little better; at least it looks like the image is moving this time, instead of just appearing in the new place. Now we're getting a more reasonable ten frames per second, which is a rate that at least allows us to consider it an animation ... but it's still pretty choppy.

Now let's take it up another order of magnitude; imagine dividing the space into 100 equal parts, and doing the same as before, where we copy the image to each of these places along the way toward the final goal. Now we're getting somewhere; the image is moving much more smoothly than before, at a rate of 100 frames per second. You may still notice some choppiness (due to the same factors that I'll get to eventually here), but you can at least consider this an animation. And if the image was an evil droid sprite, you might even be inclined to crank up your power blaster and nail the sucker.

What made this image movement an “animation” was increasing the frame rate, and decreasing the amount of movement per interval, so that each step was small enough that overall movement began to appear smooth. This interval decreasing was done in the time and space dimensions (decreasing the time between steps and the space between each movement). Now imagine doing the same for color.

In particular, think about the color of each of the pixels being affected by an animation. How much do they change between each step? Do we have large numbers of pixels changing drastically every step? Or small numbers of pixels undergoing minor changes each time?

In the first case above, where we moved the object from the left to the right in one single frame, we changed all of the pixels in the place the object came from (we erased them to the background color) and in the place the object moved to (we colored them from the background color to the image color). That's a lot of pixels that underwent significant change in one step. In the final animation, however, where we moved the object gradually from the left to the right, the only pixels affected in each step were those on the leading and trailing edges of the object. The changes to those pixels may have been just as drastic (which is what makes that final animation still look choppy), but the fact that there were far less pixels affected by each step made the animation appear much more smooth than the first attempt.

In order to smooth out an animation, we want to minimize the amount of color change per pixel in each step of the animation. Let's think about the original animation comparison. In the fading animation of my application, each pixel was modulating between the original color and the new color at a rate of 10 frames per second. In the extreme case of a pixel moving between black ((0,0,0) in RGB space) and white ((255, 255, 255) in RGB space), the change for every step would be 10% of that color difference, or (25.6, 25.6, 25.6). We can calculate this Euclidean distance in RGB space as sqrt(25.6^2 + 25.6^2 + 25.6^2) = 44.34. Meanwhile, the moving animation was changing pixels between the background color and the object color in one step (for however many pixels were affected in each step of that animation). In the extreme case (black object and white background), pixels would change as much as (256, 256, 256) in each step, for a Euclidean distance of ten times the fading amount, or 443.4 in RGB space.

One of the interesting things to me in this investigation is that the amount of change of individual pixels appears to have much more visual impact that the amount of change overall. That is, a large color change happening in a small number of pixels is far more detectable than a small change happening in a large number pixels. In the fading case, that animation took over the entire 800x800 window. If that animation were running at 10 frames per second fading from black to white, this means there would have been 640,000 pixels changing by 44.34 each frame in RGB space. You would think that all of that color change would be pretty easily detectable by our eyes. Meanwhile, the scrolling animation affected only a tiny portion of the window in comparison; the animation happened in an area of about 200x50, or about 10,000 pixels total. Also, the scrolling animation did not affect every pixel in that area, since it just involved some objects shifting slightly every time. Assuming that the scrolling animation affected only a quarter of the pixels in that region, this means that there were only about 2,500 pixels affected each frame. In the extreme black-white case again, we have a color change of 443.4 in RGB space for each of these pixels. So that's 44.34/pixel for 640,000 pixels versus 443.4/pixel for 2,500 pixels. By numbers alone, it seems like the fade is much more significant. Nevertheless, the fade looked far smoother than the scrolling animation.

I don’t think I’m going too far out on a limb here in claiming that the rate of change of any single pixel is more significant than the total change over a group of pixels. If you've got a whole screen of a million pixels changing by just one incremental color value, that will be far less noticeable than a single pixel flickering betwen black and white in the middle of the screen. If you imagine it this way, I think you'll agree; that huge change of a single pixel's color value is far more discernible to our eyes.

Maybe it comes from our ability to edge-detect very well. Or our carnivore instinct, evolved through millenia of hunting flickery pixels in the wastelands of Cro-Magnon times. I have no idea, but I do know that’s what it looks like, and in graphics it’s all about how it looks.

3) Vertical Retrace: That Syncing Feeling

Another major factor that I noticed in my animations is the impact of the “vertical retrace” of the display.

A typical computer display (traditional CRT or modern LCD display) will update the screen from video memory at some frequency (typically 60 Hz for LCD isplays, and typically anywhere from 60-90 Hz for CRTs). You can think of this as a linear process where every pixel on the screen is updated one-by-one, left-to-right and top-to-bottom. It’s done so fast and seamlessly that you would not usually notice this happening. But fast animations on the screen can cause an artifact called “tearing” that make vertical retrace an issue.

Imagine that the vertical retrace is in the middle of refreshing the screen, and it happens to be refreshing the pixels in the area where you are copying pixels to the screen. The effect will be that the pixels below and to the right of the refresh location will show up in their new location, but the pixels to the left and above that location will still be shown in their old location; this is because the refresh is only updating the pixels to the right and below on this refresh. This effect does not last long; it lasts exactly until the next refresh, which is in 1/60th of a second for an LCD display. Since the artifact is not permanent, you might not even notice or be disturbed by it. But if you are animating things constantly, and if those animations are happening in large increments, then the tearing will be so obvious that this will add to any perception of a choppy animation.

It might be easier to see this effect with a picture.

Suppose we are trying to move our image between frame n and frame n+1 like so:

Now suppose that the vertical retrace (represented by the red line below) is happening right in the middle of this area as we are doing this copy:

The net result is that the user will see “Frame 2” as having a broken object in it very quickly, until it is fixed by the next refresh.

The problem here is that copies to the screen do not, by default, pay any attention to the state of the vertical retrace process, so it is entirely possible, even likely, that animations will run into this rather jarring (if transient) artifact.

Segue

Now for the cliff-hanger: please check out the article Smooth Moves on java.net for the stunning conclusion of this series. That article examines possible solutions to the color difference and vertical retrace problems, and offers an applet and sample code the demonstrates both the problems and solutions.

These Are Some of my Favorite [Mustang] Things

I've written about Mustang a few times already, but I thought I'd take the opportunity of the Mustang Beta Release to wax poetic about some of my favorite features in the release. This view is most certainly skewed, biased, and even subjective; there are a lot of cool things in the platform, but I specifically wanted to talk about the stuff that strikes my personal fancy, either selfishly because I had a hand in the feature or because I've used it already or look forward to using it someday (when I get time to write all the code I haven't gotten to yet).

My Favorite Features

If someone accosted me in the street, shoved a very large weapon in my face, and screamed “Tell me your favorite Mustang feature! Now!”, there would be no hesitation; I'd pass out.

But when I woke up, I'd definitely say...

Gray Rect Fix: This Swing feature is not only inherently cool from a graphics perspective, but it has the double advantages of real and perceived performance improvements. Real performance improvements kick in when Swing applications no longer need to repaint the GUI due to some invalidation on the screen; they simply copy the persistent back buffer to the screen. Perceived performance improvements, often more important than the real ones, are in the way that Swing windows are automatically updated from the back buffer instead of going through an erase/update cycle; even if the update cycle takes an insiginficant amount of time, the fact that the user did not see an erase (the gray rect) followed by the update is huge. Now, the “gray rect” erasure is replaced by a simple copy of the back buffer contents. This is one of the last holdouts of the tired old “Swing is slow” reputation; because it looked different than typical native applications during the erase/update cycle, users would interpret this as being slower. Now we're on a par with native applications (even faster in some situations, as many native applications don't do anything as sophisticated as double-buffering, and suffer flashing and slower updates as a result).

Faster, cleaner, more elegant, and cooler; how could you get a better feature than this?

For more on this feature, check out ScottViolet's and my blogs

Peabody: Okay, so this isn't a “feature” per se, but it's a really big change with Mustang from previous releases. Now, developers can get ahold of our development builds at the same time as we create them, see all the source code, build it, and even submit proposed fixes. This is way beyond where we've been in any previous release. Prior to Peabody, only licensees could see all of the source code, not many people on the planet could build it (or at least we didn't make it very easy), and proposed fixes would be necessarily limited to the code that people could view without being able to actually make the change and test it out themselves. I'm positive that this has contributed to both more awareness of what's in Mustang (since people are able to try it out) as well as more quality of the bits (as people try it out and are able to report problems). Sure, we're going into public Beta now, but in some sense, we've been in a rolling Beta since we started posting the Peabody builds on the site. This has reduced the lag between making a change, having it run in the wild, and finding out about problems that need to be addressed. Typically, this process takes months and involves layer upon layer of indirection between the engineers that did the work and the customers that tried it out. Now, we're able to implement a feature, promote the build, blog about it to tell people it's there, have it downloaded, tested, and get feedback directly to the developers; all within days of integrating the code. For example, check out the comments to Scott's Gray Rect blog; the same day that he posted the blog (which was just a few days after the Swing code was putback into the workspace) people were trying it out and giving us feedback on the feature. (Did I mention how much I like the Gray Rect feature?)

For more on Peabody, check out the mustang web site.

Single Threaded Rendering (STR): This feature moves Java 2D from a model of rendering immediately on aribtrary threads to one where we queue up requests at the Java level and execute them (slightly) later at the native level on a single thread. This feature is not as visible as many of the others in Mustang because it's not enabled by default. STR is implemented in the OpenGL rendering pipeline only, which is not on by default (due to various driver and hardware issues). But besides being extremely cool from a graphics software perspective (there are lots of issues of graphics state and large and complex primitive handling that had to be solved along the way), STR offers significant improvements both for the current OpenGL pipeline and for future rendering pipelines that will use this approach. OpenGL rendering benefits both in robustness (multi-threaded rendering in OpenGL tends to be a driver killer for many video cards) and performance (batching up rendering requests saves significantly in both JNI and OpenGL API overhead). In the future, we will use the STR approach in other rendering pipelines (such as the default DirectX renderer on Windows) and expect to see similar benefits.

For more on this feature, check out Chris Campbell's STR-Crazy and STR-Crazier blogs.

ImageIO Performance Improvements: This feature was a latecomer to the party; Chris Campbell did this work recently (and blogged about it recently as well). Our strong wish in Java 2D is for developers to start using the Image IO APIs for all image loading/saving operations and to move away from the old Toolkit approach. The ImageIO APIs are much easier to use (see my blog on using Image IO for easy image conversion, for example), much more feature rich, and are also where we are putting our current and future efforts in image loading and saving. The Toolkit APIs are old and crufty. Unfortunately, the Toolkit image loaders also happen to be faster in some important cases (such as loading JPEGs). Until we fix the performance issues, it's hard to convince people that Image IO is really a better approach. Finally, Chris looked into some of the issues here and made JPEG loading significantly faster than it used to be. Check out the nifty bar charts in his blog for the quick take on this.

For more on this feature, check out Chris Campbell's blog .

Native Look & Feel: There are basically two camps of Swing GUI developers: those that want a cross-platform look and feel and think that the Metal/Steel theme is dated and those that want a native look & feel and feel that we don't do a good enough job there. Okay, there's actually a third camp as well; there's also a silent majority of people that don't care deeply and are happy with whatever we offer.  But we're not talking about those folks here since their problems are solved. The Metal/Steel issue was addressed in Tiger, with the release of the Ocean theme for Metal (no more purply/bumpy widgets; it's all about cool, smooth gradients now).

The Native Look & Feel issue has continued to dog us for the last several releases.  In 1.4.2 we finally introduced both the XP and GTK look & feels and in Tiger we kept tweaking and improving things. In Mustang, we've made a major shift in both GTK and Windows to use the native widget rasterizer to get a truly native look. No, we're not using heavyweight widgets on these systems; Swing still has a lightweight widget model. But now, instead of Swing doing its own rendering of these lightweight widget images, we ask the native operating system to render the images for us, so that these lightweights look exactly the same as heavyweight widgets would. Combined with improvements in our text support (see below), I believe we're finally “there” for native look & feel support.

For more on this feature, check out Bino George's blog.

LCD Text: People have asked for some time for us to upgrade our font support to do better anti-aliasing on LCD displays.  In Mustang, we've done this. Not only do we now support this more complex anti-aliasing algorithm (which uses the RGB striping of LCD displays to enable sub-pixel rendering of the fonts; see my article Anti-Aliasing on the Fringe for geeky details on how this works); we also do more work at the Swing level to detect the desktop properties for anti-aliasing and set our default Swing anti-aliasing setting appropriately. Also, we now ask the font for information on appropriate anti-aliasing information given the size of the font; you'll notice that enabling anti-aliasing on the native desktop results in some text being anti-aliased and some not; this is because fonts actually have information about which sizes are appropriate for anti-aliasing and which are not (generally, small sizes of fonts don't look great with standard anti-aliasing, so fonts may specifically ask to be aliased at these sizes).

For more information on this feature, check out this blog on Phil's Font Fixes.

Table Sorting and Filtering: This is a handy feature that seems obvious once it's there; JTable now provides built-in support for sorting and filtering. This is functionality that many developers end up doing on their own in their JTable code. When JDNC began, they realized they needed this feature as well, so they did it on their own. It's like part of the initiation ritual for users of JTable, I think. Now Swing has folded in that capability into core so that everyone can just get it automatically.

For more on this feature, check out Scott's blog and this tech tip on java.sun.com.

New Modality: Modal dialogs in AWT have always had a behavior that was unexpected, when compared to typical native desktop applications. Now, through the new modality APIs, you get the behavior you expect.

For more information on this feature, check out this article on java.sun.com.

Splash screen: Applications end up implementing their own splash screen functionality, to mitigate the pause between launching an application and the application window actually coming up. Now, AWT offers this capability for free; you pass in the image and they'll display the splash screen. Not only that, but they'll do it in native code so that the splash screen can be displayed while the VM is still being initialized.

For more information on this feature, check out this article and tech tip on java.sun.com.

Desktop API: Integration with the native desktop is increasingly important for Java applications. The Desktop API allows a Java application to launch one of several typical native applications, all from within Java code. Your application can launch the native browser, the native mail client, any native document viewer, and more.

For more information on this feature, check out this article on java.sun.com.

Deployment Dialogs: A lot of the user experience issues around deployment have been vastly improved, from the splash screen we use for Java Web Start to the security dialogs that pop up when the user needs to approve some behavior. Scary security dialog no more!

For more on this feature, check out Stanley Ho's blog.

Wrap Up

As I said at the beginning, this is a purely subjective view of Mustang from my skewed perspective.  I haven't scratched the surface of the wealth of features offered in Mustang, from the whole host of features in the core libraries to improvements in the VM to other desktop features.  I would encourage you to read more about these features in the Core Java Technology Features article, in the Desktop Features article, on the Mustang site, and in the host of blogs and articles on the subject. 

But most of all, I would encourage you to download the Beta, download the regular snapshots, play with the release, tell us about any problems you have, and help us deliver a great release.

This work is licensed under a Creative Commons License.