Chris Campbell's Blog

Performance Archives

STR-Crazy: Improving the OpenGL-based Java 2D Pipeline

What was causing these crashes? If you're familiar with OpenGL, you probably know that it's important to do all your rendering from one (and only one) thread. While it is possible to render from other threads (if you're careful), OpenGL drivers are optimized for the single-threaded case. Games almost always play by these rules; they may use other threads for things like AI and physics calculations, but they do all their rendering from one display thread.

Well, we're not quite as lucky in Java 2D... Rendering requests can come from any number of threads (the EDT, a user thread, etc) even though we try to teach developers to avoid doing heavy rendering to hardware surfaces from many threads. In 5.0, we dealt with this by taking precautions, such as:

• ensuring only one thread is calling an OpenGL method at any given time
• using OpenGL rendering contexts only from the thread on which they were created

This makes OpenGL drivers reasonably happy, but it requires more labor on our part (meaning reduced performance), and despite all our efforts, drivers can still crash when there are many rendering threads in use by an application.

Clearly, something needed to be done if we ever wanted to see the OGL pipeline become reliable enough for day-to-day use by end users. For the past couple years, Chet and others on our team have discussed the idea of "single-threaded rendering" (STR), which would allow us to interact with native graphics libraries (such as OpenGL) from a single thread, thus making those libraries and drivers happy.

The idea originally came up as a possible solution to a number of threading problems we've dealt with in the past on the Windows side. But what better way to test these ideas than to implement them using the OGL pipeline? So that's what we set out to do in Mustang (JDK 6.0). Instead of this mess:

we'll now have the following, more elegant solution:

As you have probably noticed, I could go on and on about this project, but I'll spare you further pain. (If you really want more details, see this RFE.) The bottom line is that the project was a big success (from our perspective). The OGL pipeline is now more robust since we only interact with the native OpenGL libraries from a single "queue flushing" thread.

Going into this project, we were a bit unsure of the performance implications. We had our reservations, but surprisingly, there are some significant performance gains attributed to these changes, mostly because we can avoid going down through JNI on each and every rendering operation. Instead, rendering operations are buffered up on a NIO buffer, which is flushed periodically (or as necessary). Another benefit of STR is that it makes it much easier to "batch up" similar operations. OpenGL prefers to see lots of similar operations (like lines or rectangles) batched up within a glBegin()/glEnd() pair, so it is now easier for us to track which operations are being enqueued, and therefore easier for us to batch.

I'll save the detailed performance numbers for another day, but here are some quick numbers from my Linux configuration (JDS, 2x 2.6GHz P4, Nvidia GeForce FX 5600), but the numbers look similar on Solaris and Windows:

• SwingMark (internal Swing benchmark) is approximately 15% faster
• drawString() is about 250% faster (according to J2DBench)
• fillRect() and drawLine() are up to 2500% faster (according to J2DBench)
• with FireStarter (an internal demo), we can render about 2600 transformed/filtered/blended sprites per frame at 30 fps (as opposed to only 1600 sprites before STR)

(Note: these results are from microbenchmarks, so take them with a grain of salt...)

Now, why did I choose today to post this blog entry? If you said, "well, because the STR changes have just been integrated into Mustang build 27", give yourself a big pat on the back. It's true... The b27 snapshot was just posted on java.net today. Please download the binaries for your favorite platform and take an app or two for a test drive with the OGL pipeline enabled (-Dsun.java2d.opengl=True). Let me know how it goes... But first, a couple caveats...

Everytime I write about the OGL pipeline, the first question that is asked (without fail) is "Will the OGL pipeline be enabled by default?"... Well, the answer is still "no". It will never be enabled by default on the Windows platform (the DirectX-based pipelines are a better bet on Windows). However, I certainly hope that someday (Mustang? Dolphin?) the OGL pipeline will be enabled by default on Solaris and Linux, but only if we detect a "compatible" system, meaning hardware accelerated drivers are installed, bug-free, and performant.

Even with STR in place for the OGL pipeline, there are still just a few outstanding driver issues (more so with ATI's drivers than with Nvidia's) that prevent us from enabling the OGL pipeline by default. We've filed bugs with the respective companies, and we're still working with their driver teams to get the bugs fixed. I hope to post a follow-up soon to report progress on the remaining driver issues. For now, if you're using a video card from ATI, you may see some inverted colors and/or gradients being rendered incorrectly, and maybe even a crash on exit. These issues have been filed with ATI. If you're using a video card from Nvidia, you may see reduced performance when rendering software-based images on Windows, but otherwise, things should be looking pretty good.

If you're one of those people hoping to see hardware accelerated Java 2D on Solaris and Linux someday, please download the Mustang snapshots and continue to provide feedback on the OGL pipeline. The more feedback we get, the better the chances that we can enable the OGL pipeline by default in a future release.

Finally, A Micro-Benchmark For Java 2D...
A number of folks on the javagaming.org and JavaLobby forums have requested a way to benchmark Java 2D performance on various platforms and configurations. We've had an internal microbenchmarking application for quite some time called J2DBench that we use on a daily basis. In response to requests from developers, we wanted to make J2DBench available to the public, but it took quite a while to get permission from our lawyers to do so (you know how it goes).

Finally I'm happy to say that J2DBench is available as part of the JRL source bundle for Mustang build 27. To access the J2DBench sources, download the source bundle and cd to j2se/src/share/demo/java2d/J2DBench. In there, you'll find a README file, Ant build.xml file, and NetBeans 4.0 project file to get you started. In the future, we should be able to accept J2DBench fixes and improvements (e.g. new tests) through the JDK Collaboration (or similar) project on java.net... I'll blog with more details in the future once we figure out how best to handle that issue.

Thanks to Dmitri Trembovetski for taking the time to prepare the J2DBench source files for public consumption. I hope it will be useful for developers trying to performance tune their application, or as a convenient way to submit performance bugs to us on the Java 2D team. (For example, use the sample options file included with J2DBench to compare the performance of the OGL pipeline in JDK 5.0 with the improved pipeline in JDK 6.0-b27.) Good luck!

Our Work Is Never Done
Another improvement coming soon in a Mustang snapshot near you: a fix for the dreaded "gray rect" problem. This is one of Chet's favorite battles, so I'll let him describe the problem, and the solution, in an upcoming blog. However, I wanted to quickly mention that due to the way this fix was implemented, Swing is able to use BufferStrategy for their doublebuffering needs.

What makes this interesting is that it opens the door for the Swing backbuffer to be stored in the native OpenGL backbuffer, which is usually pre-allocated for us anyway, and often goes unused in the case of Swing applications. Making use of the OpenGL backbuffer means improved Swing performance (when the OGL pipeline is enabled), and also results in a big reduction in VRAM usage.

Prior to this change, the Swing backbuffer would be allocated in an OpenGL pbuffer. Pbuffers allow for hardware accelerated offscreen rendering, but they can be expensive in terms of VRAM footprint (they often use 12-20 bytes of VRAM per pixel!). So by eliminating the need for pbuffers for the Swing backbuffer, there will be plenty more VRAM available for accelerating things like icons and text. I'll provide more details about this change in a future blog entry.

There are plenty of other projects on my plate for Mustang, including:

• Fullscreen/DisplayMode changing on Linux (finally)
• OGL pipeline improvements (of course)
• Hardware acceleration for LCD-optimized text (important!)
• Wide line performance improvements (maybe)
• Applying STR techniques to our DirectX-based pipelines on Windows (Dolphin?)
• Other miscellanea (as always)

In my ears: Robert Pollard, "Speak Kindly Of Your Volunteer Fire Department"
In my eyes: Cao Xueqin, "The Story Of The Stone (Vol. 2)"

And This One Is Better Than Ever!

Do not expect any order or rhyme or reason for the content that follows. I've been away from the blogging game for several weeks, so I have a lot of random thoughts waiting to jump out of my dome (in typical fashion, none of them are well formed). Chet and Hans have been cranking out new blogs left and right, so I feel compelled to keep up with my end-of-the-hall-mates.

On the new OpenGL-based Java 2D pipeline: Work on this exciting new pipeline doth proceed apace. We've been making some excellent performance improvements over the past few days, and we're working with video board manufacturers to increase reliability, compatibility, and performance. I think desktop developers should be pleasantly surprised with the improvements offered by this pipeline, and I look forward to your feedback when 1.5 beta ships in a few months.

This new pipeline has been getting some publicity on various forums like Java Gaming and JavaLobby in recent weeks (check out those threads for some more detailed information; the JL thread was interesting until it devolved into a discussion on open sourcing Java, which seems to be par for the course). Now might be a good time to answer some frequently asked questions regarding the new OGL pipeline:

Q: You suck. Why isn't this pipeline being developed for Windows too (Sun is evil)? Why aren't you using a shared code base? Is it because you're dumb?
A: I'm rubber, you're glue... But seriously, we would very much like to offer this pipeline as an alternate implementation on Windows, but it won't make it in for 1.5... Hopefully we'll enable it in a future release, because it could really open some doors for interoperability with JOGL, and various other factors. The reality is that we have hundreds of video boards and driver combinations to support on Windows, and our effort is best concentrated right now on making our DirectX pipeline even more stable and performant. Driver flakiness is a big concern (just ask Chet and Dmitri!), so making sure that the OGL pipeline works on all Windows configurations will be a challenging task. We'd rather concentrate right now on making the OGL pipeline rock on Linux and Solaris, where graphics acceleration has traditionally been more dismal. (And yes, we do have a shared code base, with small platform-dependent modules to support GLX and WGL. It's a nice streamlined design, in my obviously biased opinion.)

Q: Why aren't you making use of UMD's Agile2D project? Is it because Sun sucks and hates open source software?
A: Agile2D is another interesting project out of UMD, but the downside is that it sits outside the guts of Java 2D and Swing, so you have to "port" your application to use Agile2D. Writing the OGL pipeline from the ground up allowed for much tighter integration with Java 2D internals, so you reap all sorts of benefits like using texture memory for caching glyphs and "managed images", in addition to all the performance gains reported by the Agile2D guys. If you tried out Agile2D, you should get all the same benefits and many more just by using core Java 2D and Swing in 1.5...

On the new "Sun Java Desktop System" branding strategy: Okay, so this one is taking a while to sink in, I'll admit. Some folks on JavaLobby are complaining that Java doesn't play a large enough role to warrant the name. Granted, a majority of the apps are not written in Java, but the biggies like Mozilla and StarOffice surely are Java-enabled. The important thing to keep in mind is that this is just the first release; prepare to see many more pure Java apps added to the package as time marches on.

I'm glad that the name doesn't mention the word Linux or Solaris... There's no reason for it. We're at an important cusp in the software world where the underlying OS has become irrelevant. Application developers and end-users alike no longer care what operating system they're using (yes, yes, gross generalization) as long as it runs their favorite cross-platform apps that help them get their job done, whether they are written in Java (NetBeans, Limewire) or not-Java (Mozilla, Evolution). If Sun decided tomorrow to base the Java Desktop System on OS/2 or FreeBSD or whatever, it wouldn't make a lick of difference to end-users (gross generalizations are fun). This general trend in the industry towards platform-agnosticism is really cool, and it's only going to accelerate.

On the state of the "Mu" project: I haven't had much time for Mu this summer, but that hasn't stopped others like Mr. Rob Ross from adding cool new features, like video playback. I made some minor enhancements to the "lite" client to improve ease-of-use (i.e. act more like iTunes), but those haven't yet been committed to CVS. Soon we'll be ripping out some of the internals and replacing them with an exciting new media content/metadata system about which I've been conversing with the Gas/JRec/Ptarmigan author. There's also talk about adding new pure Java codecs, live TV viewing, TV listings, and even some video conferencing functionality. Hopefully we'll make some more progress this autumn... Help from interested parties is always appreciated!

On Giants/A's playoff tickets: Should I stay or should I go? I could stay here and get some work done, or I could go sit in line at the ballpark with the rest of San Francisco. It's a beautiful day in the city, but I wonder if they have wireless down there? Better yet, I could bribe my sickly roommate into sitting outside all night for tickets. Yes, that sounds like a fantastic idea to me (hint hint nudge nudge)...

On Maven: I still haven't had a chance to play with Maven yet (beyond the few articles I've read), but it sure seems cool. Anything to eliminate the drudgery of build/doc/test systems is fine with me. And I used to think Ant was high-level...

On the genius of Bob Pollard: November 4th brings us a definitive box set from legendary lo-fi drunk-rockers Guided By Voices, entitled "Hardcore UFOs". Whenever I drop money on GbV, I come closer to understanding how crack addicts feel when they make their purchase -- 2% guilt and 98% elation. How can you not admire song titles like "I Invented The Moonwalk (And The Pencil Sharpener)"? The energetic, start-stop, repetitive-F-chord mania that is "Postal Blowfish"? The buzzing-guitar-string simplicity and effortless crooning of "Goldheart Mountaintop Queen Directory"?

On the virtues of vacationing (or "how to boost your debugging effectiveness"): This was a great summer. I took various long breaks (well, a few days at a time) away from work to visit with nature, friends, and family. My personal rule number one on these trips was "do not under any circumstances think about work", and then a funny thing happened. I'd be sweating my way up Half Dome or just plain sweating on a beach somewhere, and then I'd have a vision. No, it wasn't Yeti (what an odd assumption on your part)... It was the design flaw that's been nagging me for months, but this time it came with a solution. It's that old psychological notion of receiving inspiration when you least expect it. So there you go, next time you're working on some tough project, tell your boss you'll be taking a few days off to work-but-not-really on that important project.

On a related note, I was just reading James Gosling's latest blog entry about trusting (or better put, not trusting) yourself when debugging. I do this to myself all the time, and I never seem to learn my lesson. Just yesterday I was reading an old email I sent about the OGL pipeline, where I said something to the effect of "Q doesn't work even though I know that we're definitely doing Z, which should enable Q". So for months I built up this notion that the bug couldn't possibly be related to Z. And then what happens? I go revisit Z to discover a subtle flaw that had been there all along, and now of course when I fixed the bug in Z, Q is working just as I had always expected! Damnation! I could smite myself! (In fact, I just did that just to add a little drama...) Moral of the story? Don't trust yourself, but do listen to James, he's a smart guy.

On the coolness of iPods and iSights: I broke down and bought one of each, after hemming and hawing for months. The iSight is great, and the ease of iChat is unparalleled. Yet it just kills me that I'm able to video/voice chat with all of two people (Mac owners of course). So, good for Apple, they'll convince a couple more percent of the earth to buy their machines, and meanwhile everyone else (Windows/Linux users) is missing out on a fantastic hardware/software solution. Everytime a company ties people into a specific OS with some custom built application, the industry takes one step backwards. Argh like Charlie Brown.

I haven't yet received my iPod. I've been making all the necessary preparations, like converting my entire CD collection, encoded in the FLAC format, to MP3. I updated an old tagging/converting Java app that I had sitting around to handle the conversion, and it seems to be working like a charm. Yay Java. Note that by purchasing this iPod, I've admitted to myself that maybe the world isn't quite ready (yet!) for my idea of always-on, instant media transcoding/streaming from your home media server to your MIDP2-enabled cell/PDA (running a mobile version of Mu). But mark my words, this will be the future of personal, mobile entertainment! You just wait!

In my ears: The Super Friendz, "Love Energy" and Neil Young, "On The Beach"
In my eyes: Haruki Murakami, "The Wind-up Bird Chronicle"

This work is licensed under a Creative Commons License.