Chet Haase's Blog

Tiger on the Desktop

I know Tiger's getting a lot of press lately (after all, we don't ship a new major Java release that often) and that there's a lot more about the new release than the graphics end of things. In particular, with a platform whose language changes so little and so infrequently, the types of changes introduced in Tiger are pretty substantial and attention-grabbing.

But still, there were some features that we worked on in the desktop client area of the platform for this release and if I can push! just squeeze shove! my way through the teeming throng of Tiger admirers, I'd like to mention a few of the highlights.

First, I should point out that on nearly every release, most of the work we do on the desktop client goes into things that don't percolate up into "features". This basically means fixing bugs, speeding things up, supporting any new platforms, and just making the existing platform run better every release. We've heard, loud and clear, that stability is one of the top requirements for our users, so the number one priority around here is quality (and everything that goes along with it, including making performance better and better, but not at the cost of core functionality).

Nonetheless, we do eventually get to work on new, cool stuff to enhance the platform. It's that stuff that I'll be discussing today.

The main things on Swing's feature parade for Tiger were some new Look & Feel features as well as table printing. I also have to add here my favorite feature: JFrame.add().

• Look & Feels

  There were two significant improvements in the Look & Feel area: Ocean and Synth.

  • Ocean

    The first was an update to the cross-platform look & feel. This one hasn't changed since the introduction of the Metal l&f. It had been a few years, that l&f was beginning to look a bit dated, so we worked on a more modern theme called "Ocean".

    One of the primary goals with Ocean was to Not Break Stuff. So now adjustments were made to the l&f which would have broken existing applications; the sizing of widgets is the same and everything should just work out of the box; it's just the skin of that l&f that's different (and, I think you'll agree, better).

    Users will get the Ocean l&f for free; an app that is run against 5.0 will pick up the Ocean l&f. Note that an app that subclasses Metal, or has other dependencies upon that look & feel, will still get the old Metal look & feel; again, backwards compatibility is the Prime Directive here.

  • Synth

    In addition to simply updating the cross-platform Java look & feel, the Swing team wanted to make it easier for developers to create newer, cooler look & feels for their applications. So they came up with synth, which is an architecture that makes it much easier to create new skins for applications, either through static XML definitions or programmatically through Java. I will leave the details of this approach to the experts; for more information on Synth, check out The Synth Look and Feel by Scott Violet on javadesktop.org.

• JTable Printing

  Don't you hate it when you've got a great GUI with some awesome data in it and you can't output that data in any other place that the screen? The Swing team thought so too, so they made printing work with JTable. Enough said.

• JFrame.add()

  The Swing team didn't call out this item when I asked them for their Tiger highlights, but it's probably my favorite Tiger feature in Swing and maybe the entire Java platform. Okay, so perhaps that's overstating it a bit. But I like it a lot in any case.

  Previously, you had to do the following to add a component to JFrame:

  	JFrame f = new JFrame();
  	f.getContentPane().add(someComponent);
  

  This is because of the Swing containment hierarchy; you cannot add a component to the JFrame itself, but only to the pane inside the JFrame which is tasked with being a container for sub-components. However, it was always awkward to have to remember that nuance and have to deal with Swing's containment hierarchy at this level. Well, they've fixed it. Now you can simply do this:

  	JFrame f = new JFrame();
  	f.add(someComponent);
  

  Internally, it's doing the same thing as before, only it's now handling the pesky containment details for you instead of your having to deal with it in your code.

Here are a few of the more significant features that you might notice about the graphics engine in Tiger. For more information on 2D Tiger features, check out our Release Notes.

• OpenGL

  This is an item that's not going to be apparent to most users right now, but should be an exciting thing for some developers and a taste of the great things to come, in terms of Java2D hardware accelerated graphics.

  We worked hard to enable nearly all Java2D functionality through hardware-accelerated OpenGL graphics on all Sun Java platforms (Solaris, Linux and Windows). This means we get accelerated graphics performance for such things as simple GUI operations (filling rects, drawing lines, copying buffers, caching and copying text glyphs as images, and so on), but we also get acceleration for way more interesting (and currently time-consuming) operations such as gradients, translucency, anti-aliased text, and transforms. This means that typical Swing applications work well, but more advanced 2D graphics apps just fly!

  There's a pretty big BUT here that I have to stick in at this point: this stuff all works extremely well ... except for the fact that OpenGL support on various platforms, graphics cards, and driver versions can be, er, less than optimal. Some of the problem lies with the way that we use the API and the types of features and performance we need; for example, we require different capabilities for Swing's back buffer than a typical GL app requires for its double-buffering operations. Some of the problem lies with the cards or drivers themselves; they simply have bugs that we tend to trigger (because we use the GL API in different ways than, say, Doom3 or other typical OpenGL applications).

  The net result is that we have an excellent implementation whose vast powers ... lie untapped by default. You can play with the OpenGL implementation by passing in a command-line argument to tell us to enable it:

  	java -Dsun.java2d.opengl=true 
  

  or you can set the same system property in your main() method:

  	System.setProperty("sun.java2d.opengl", "true");
  

  But otherwise, our OpenGL acceleration is disabled by default.

  But don't fear; I wouldn't have bothered mentioning it if it weren't way cool. Not only does it provide excellent benefits in the right situations now, but it also provides the necessary building blocks for us to have this kind of acceleration enabled by default on future releases. So it only gets better from here...

  For more information on this OpenGL renderer, check out the information in The OpenGL section of our Release Notes.

• Managed Images Everywhere!

  We introduced the concept of Managed Images in jdk 1.4. Basically, if you happened to create your images in certain specific ways (including Toolkit.getImage(), Component.createImage(w, h), and GraphicsConfiguration.createCompatibleImage(w, h)) then we would try to create accelerated, cached versions of the images underneath from which we could then do hardware-accelerated image copies.

  This system worked great for users of those particular APIs; developers didn't have to hassle with the management aspects of VolatileImage in order to enjoy the benefits of hardware accelerated image copies.

  But why the restrictions on which method calls you had to use? Why couldn't developers get the same benefit regardless of how the image was created?

  It was all a matter of just getting around to it. So in Tiger, we finally did.

  Now, you can create an image any way you like (including new BufferedImage(...)); if we can accelerate it, we will.

  Note that there are other things that can complicate the life and performance of a managed image; if this is a topic that interests you might check out these blogs on Images and this article on Intermediate Images for more details about image performance.

• Font Rearchitecture

  This is another "internal implementation" detail, but one which some users will definitely notice.

  The main work here was to take crufty old native code and move it up into elegant new Java code. This doesn't benefit the developers directly except that it makes the code much cleaner and easier to maintain (read: you'll get your bugs fixed faster and better).

  The real noticeable benefit of this change was that we removed threading constraints in the old code at the same time. Previously, much of the text processing was done in a synchronized fashion so that only one thread could access that key code at a time. This meant, for example, that an application rendering text to many different images at the same time would bottleneck in this code waiting for access.

  In Tiger, this constraint is removed and the font code is now Multithread Happy. Imagine the super-scalar servers that can now take advantage of all of those processors cranking away on text rendering simultaneously...

• CUPS Printing Support

  For the Unix printing fans out there, we now have CUPS support!

• ImageIO: plugins and performance

  We added a couple of key image format plugins in Tiger: BMP and WBMP (apparently popular image formats on some niche platforms; who knew?). We also worked on some major performance bottlenecks in image loading so the library is faster in some key areas.

• Startup performance

  The platform overall enjoys some serious startup improvements in Tiger. Small parts of those improvements came in the graphics arena, in the way that we were loading and initializing things that could have been done faster (and now it is).

  There's still a ways to go toward Startup Performance Nirvana, but we're making progress.

• Miscellaneous Performance Tweaks

  There were fixes in some rendering niches that are worth mentioning:

  • Primitive batching on Windows: we are smarter now about how we batch up line rendering calls when using our Direct3D renderer on Windows. This reduced the (significant) overhead per line and got us up to a 10x improvement in micro benchmarks (your mileage will vary; the biggest improvement will be seen when you issue all line calls together, not interspersed with other types of rendering operations).

  • Complex clip rendering speed was significantly improved in Tiger.

  • Footprint reduction for Toolkit images: We used to create intermediate images in our Toolkit image code which tended to bulk out the Java memory heap footprint. We've reduced much of that unnecessary garbage and got some pretty big footprint reductions as a result. Images tend to take up a pretty huge amount of memory (imagine: a buffer the size and depth of an average screen today takes about 5 megs of memory!), depending on the size and depth of the image, so saving the number of copies sitting around on the heap can help things out quite a bit.

There were lots of bug fixes in the AWT arena for 5.0. There were also a couple of areas of major functionality work. For more information on all of the AWT-related fixes and features in Tiger, check out the 1.5 AWT Release Notes. Here's a quick take on some of the larger items implemented:

• XAWT

  AWT was reimplemented on both Solaris and Linux to avoid being tied to the old Motif widget library system. The new XAWT toolkit was implemented using only Xlib, with a minimum of native code (and thus a minimum of native platform dependencies). XAWT is now the default on Linux, but Solaris still defaults to the Motif implementation for now. The default for either platform can be altered by use of the AWT_TOOLKIT environment variable or the -Dawt.toolkit command-line parameter; values for both can be set to either sun.awt.X11.XToolkit (for XAWT) or sun.awt.motif.MToolkit (for Motif). You can read more about XAWT at http://java.sun.com/j2se/1.5.0/docs/guide/awt/1.5/xawt.html

• X Drag and Drop

  Prior to Tiger, only Motif Drag and Drop (Motif DnD) protocol was supported on X11 platforms; now we also support the XDnD protocol. XDnD is a common feature on Linux and Java Desktop System platforms because of its support by GTK and Qt.

There were a few interesting (and huge) features added to our internationalization offerings in Tiger. For more information on these features than the blurbs below, check out http://java.sun.com/j2se/1.5.0/docs/guide/intl/enhancements.html. You can also get information on internationalization support in general at http://java.sun.com/j2se/corejava/intl/index.jsp.

• Multilingual Fonts

  In the past, logical fonts were monolingual or maybe bilingual. This means, for example, if you were running in a Japanese environment, you could display Japanese and English text, but not Arabic or Chinese. To use a different language, you'd have to quit the application, change a few environment settings (the LANG environment variable on Unix or the language selection in the Regional Options control panel on Windows), then restart the application. This made life difficult, for example, if the application you were using was an IDE, and you were using it to develop software that was localized into multiple languages.

  With the multilingual font configurations in Tiger, all you have to do is install fonts for all the languages you care about on your system and the JRE will use them all together when rendering with logical fonts. Note that today's operating systems usually come with support for many languages on their installation CDs - all you have to do is ask for that support to be installed. The list of writing systems that the JRE will support when provided with the necessary fonts and input methods is at http://java.sun.com/j2se/1.5.0/docs/guide/intl/locale.doc.html.

• Unicode 4.0 Support

  Character handling is now based on Unicode version 4.0 and there is also support for supplementary characters. For more information on supplementary character support, check out the article at http://java.sun.com/developer/technicalArticles/Intl/Supplementary/.

• Big Things

  The DecimalFormat class now supports BigDecimal and BigInteger.

• Vietnamese Support

  After a successful grass-roots campaign on the bug database (2517 votes on one of the bugs!), we have finally added support for Vietnamese in the java.util and java.text packages.

• Native Unicode Usage on Windows

  We now use the Unicode features of the native platform on Windows, which means we can now handle text in some components without restrictions on Windows locale settings.

That's about it, or at least that's the most visible part of our work on Tiger. We hope you enjoy it, and continue to send us feedback on what else you'd like to see in the future.

Happy Hacking!

Comments

• HTML editing/rendering support is quite inadequate in Swing. I wish the Siwng team made it a priority. If Sun wants wide spread use of Swing for desktop application development, they need to do a better job.
  

  Posted by: surigade on September 28, 2004 at 10:47 AM

• Chet -- nice blog. I didn't realize that you folks had mapped such complex operations to the hardware in the OpenGL pipeline! Definitely going to start playing around with that.
  However, you left out one of my favorite new features: support for Adobe Type 1 fonts! Very cool... and keep up the good work!

  Posted by: javaben on September 28, 2004 at 12:10 PM

• Thanks for the great article Chet. It is great to see that Sun remains commited to Java on the desktop and continues to make performance improvements and feature enhancements in this area. I have already noticed big performance improvements in load times, display times, etc. since I started using Tiger with my desktop apps.

  Posted by: serengetilion on September 28, 2004 at 12:30 PM

• wow :)

  i had no idea so much work had been done in this area :) :)

  
  

  Posted by: asjf on September 29, 2004 at 03:01 AM

• But otherwise, our OpenGL acceleration is disabled by default.

  Good thing it's disabled by default. On my box, the JVM crashes with -Dsun.java2d.opengl=true.

  Regards,
  Horia

  P.S. My box is a quite recent equipement (HP OEM), is running Win XP SP1 brought to latest levels, I am running the latest JRE from Sun. (Tiger RC).

  I remember trying some jogl examples few weeks ago and the machine rebooted instantly. This never happend before and sometimes I run some junk apps on it.

  Posted by: sevenm on September 29, 2004 at 09:30 AM

• Chet:
  Excellent info as usual. Thanks heaps.

  surigade:
  I think they are doing a better job. : )
  Isn't this half the point of Chet's post?
  BTW: Joshua Marinacci is developing a pure-Java XHTML & CSS renderer called Flying Saucer

  sevenm:
  Maybe check you've got an OpenGL-capable video card?
  Try one of the OpenGL screensavers (e.g. pipes) maybe?
  Poor attempt at support ends here.
  I suggest you enter a bug, though (if you haven't already) - IMO no Java app should ever make your system crash or reboot.

  Posted by: grlea on September 29, 2004 at 07:58 PM

• Great changes! As a Java desktop developer I really apprectate the changes. Another nice change is the ability to make windows stay on top. Unfortunately the focus subsystem has not been made to accomodate the change but I filed a bug for that: 5091224.
  But I agree with surigade that HTML rendering is way too primitive to be useful. What about waking up the HotJava Browser code again?

  Posted by: perkr116 on September 30, 2004 at 08:15 AM

• Hi Chet, thanks for putting up such a good blog on the Java2D side of things!

  The biggest problem I find with using Java2D is that there seems to be a lack of clear definition of the rendering pipeline and what the optimised paths are. Maybe a flow diagram will help, like the OpenGL State Machine Diagram. The lack of info on optimised paths also makes it hard to use Java2D in a real-time environment, like video-editing+effects, which is the industry I work in. For example, I wrote a simple Flash (SWF) player in Java2D (the bitmap features only, no vector shapes), then ported the code to Windows' GDI+ in C++. I got identical rendering results since the two APIs are quite similiar. However, the GDI+ version is easily twice as fast as the original Java2D version. This is on JDK1.4.2 so it may be a pixel-pushing speed issue than an overall rendering pipeline thingy.

  I look forward to testing the code using JDK1.5 with OpenGL acceleration, but like you said, there're many factors affecting the real speed we can get, so I wonder if there's a mechanism to query if a certain Java2D configuration is optimised or not. Certainly I do not wish to resort to things like using an external OpenGL debugger like GDebugger (www.gremedy.com) since it's really low-level.

  Another issue I have with how Java2D's image objects are designed, is that there's so much abstraction that it is hard to do direct pixel manipulation but still stay within the Java2D rendering pipeline, in a -fast- way. To be fair, GDI+ has the same problem too. To give an example, I've had to implement a class called LayerARGB that wraps a BufferedImage. The LayerARGB provides direct access to the pixels using a simple int[], and the BufferedImage uses the same int[] which has been wrapped up by manually created Raster objects. This way, I can easily read/write the pixels while enjoying Graphics2D features. Needless to say, this approach doesn't run very fast. But I've had to do it because Java2D doesn't support advanced things like BlendModes (found in Photoshop). Even a simple Saturation Add blending function can't be easily implemented within the Java2D design. Given the advent of OpenGL Shading Language (which really enables advanced 2D image processing as well as the usual 3D stuff that it's meant for), how will Java2D be extended to make use of something so useful like this? I think the implementing the compositing pipeline of Photoshop in Java2D (with OpenGL acceleration) is a nice and appropriate problem to solve.

  Lastly, how well does OpenGL-accelerated Java2D work with JOGL? Is it possible for them to work on the same canvas (or whatever surface it may be)? I think their integration is the answer to making Java/Java2D great and desirable for real-time media applications, since Java2D offers high-level rendering, while JOGL offers the low-level to-the-metal rendering.

  .rex
  

  Posted by: rexguo on October 01, 2004 at 11:37 PM

• Good to hear the desktop side is also moving forward. One note though, if the font rendering was reimplemented, why the fonts under Windows XP still look broken ?

  Posted by: vahur on October 02, 2004 at 08:00 AM

• Rex,

  Wow - lot's of good ideas and input.

  
  We realize that it's a bit tough to see what's going on as far as acceleration. I hope to address these mysteries through my blogs, bit by bit (like what makes managed images accelerated and not). Obviously we have a ways to go to get complete clarity.

  The diagram approach is interesting; I'll mull it over.

  One problem with talking too much about our acceleration paths is that they are just implementation details which change over time. So if we diagrammed exactly what happened in 1.4, we'd have to update it for 1.4.1 and then again for 1.4.2 and 1.5. Anyway, I'll see if there's a way I can give at least a solid overview of what's what on the various platforms.

  I hear you about the performance of direct pixel manipulation. Actually, we (the 2D team) were just talking about this yesterday. The main problem here is that the method that gives you the fastest access to the pixel data (grabbing the array and setting the data directly) is exaclty the same trigger we use internally to punt on accelerating that image. I go into this in some of the other blogs (such as the BufferedImage blogs), but basically we need to be able to tell when the data has changed in order to know when to update the accelerated version of that managed image. So if you grab a direct reference to the array, all bets are off.

  One thing we are talking about providing in future API is a means for you to get fast access to the pixel data without grabbing the array. This could be done, for example, by having us call you with a temporary pointer to the data; after the method is finished, we know that you cannot access the data anymore, so we retain info on when the data changes. Inside the method, you should be able to set the data just as fast as you currently do with the pixel arrays of DataBuffer. Just a thought, needs to be refined, but we need something like this to address the double-concern of fast-data-manipulation and managed-image-integrity.

  In the meantime, there are 2 alternate approaches that can let you both manipulate the pixel data and retain performance. You'll have to play with them to see if they suit your needs:

  
  1) Use the APIs: Instead of grabbing the DataBuffer, use the APIs in Raster and/or BufferedImage to set the RGB data. These avoid our internal punt while still letting you set the pixel data. There is more overhead using this approach than your current array-setting approach, so you'll have to see if it works for you.

  
  2) Use an intermediate image: Do your pixel manipulation on one image, and then call drawImage() into an intermediate image. Then do your copies from that intermediate image (onto the back buffer, screen, whatever). This will let you have fast pixel setting on the original image but also fast copies from the intermediate image. This approach will probably work best in a situation where you copy from the image more than you change the data in the image. In your case (video editing), I don't know if this is appropriate, but it's worth considering.

  As for buffer sharing, this is another thing on our TODO list. The same feature applies to Java3D (so that we can share our buffer internally with that library), JOGL, DirectX, and anyone wishing to get their mitts on our back buffer. We can't just throw open the gates here, but we should try to address the larger concerns, especially if we want developers to try to intergrate technologies such as Java2D and JOGL.

  On the GL support for Java2D side, look for better and more robust use of GL going forward; this is definitely a focus of acceleration efforts on the non-Windows platform, and the Windows port shold benefit from all of that work as well. Querying mechanisms to find out what's going on under the hood would be useful as part of this work.

  Posted by: chet on October 02, 2004 at 09:11 AM

• re: surigade's comment "HTML editing/rendering support is quite inadequate in Swing".

  Well, I think that's what JDIC is here for. :)

  I haven't tried it, but supposedly it allows you to embed the system default web browser (IE in Windows, Mozilla in Linux, etc....) from Java in a seamless way.

  I think it is much better to follow this JDIC approach rather than having to invest a ton of money in designing and maintaining a pure-java web browser (remember the abandoned Hotjava?) and html rendering engine component, always playing catch-up with the market share leaders (IE and Gecko -under its different flavors Mozilla Firefox Netscape 7.x).

  I wish JDIC is integrated into a future JVM revision (1.6 ?).

  Check the JDIC project page, it looks pretty impressive.

  https://jdic.dev.java.net/

  Just my $0.02
  Fernando
  

  Posted by: fcassia_at_gmail on October 11, 2004 at 10:26 AM

• Hi.I'm very interested in the opengl-enabled 2D operations. But, is there a list of supported videocards? I had no luck until now, both on Linux and Windows, to find an application that can enable openlg.
  

  Posted by: manoweb2 on October 27, 2004 at 02:19 AM

• Hi Chet, I'm doing a simple follow up to my earlier post with this thread in the JOGL forums at JavaGaming.org:

  Relationship btw Sun's OpenGL in 1.5 and JOGL

  Posted by: rexguo on November 15, 2004 at 07:26 PM

• Hi Chet,

  Are there any plans for adding more Composite classes to Java2D?

  Currently there exists only AlphaComposite. Is AlphaComposite accelerated with MMX/SSE in software mode? Does the OpenGL pipeline in Tiger accelerate it ?

  One other very common type of composite is the Saturation Add and that's very easy to accelerate using only MMX. You will frequently see it used in particle effects in games.

  What I'm hoping for is to have the entire set of 'blend modes' provided in Photoshop implemented as core Composite classes in Java2D, since they are fundamental to creating image-editing applications. There's a nice page by Jens Gruschel that describes most of these blend modes in great detail, as well as their exact formulas. The open-source GIMP also implements most of it, so it's a good reference too. The OpenGL Shading Language book (called the Orange book) also talks about implementing these blend modes as shaders.

  Hope that's not a lot to ask for! :-)

  .rex

  Posted by: rexguo on November 15, 2004 at 08:41 PM

• 
  Chet, I hope you don't mind, but you're my new best friend. Your posts have been extremely informative and helpful.I have an application that allows you to zoom and pan an image in a JFrame. It uses setClip and drawImage. I noticed a major performance drop with the 1.5.0 release. I just tried it with the 1.6.0 early access release, and it's back to normal (very responsive). Do you know what might be responsible and if it's possible to achieve the same performance with 1.5.0 with a method call or JVM switch or something?Thanks,-Andy

  Posted by: detorres on December 17, 2004 at 03:18 PM

• 
  Andy,

  
  Thanks, I can always use a friend...

  
  I don't know off-hand what would be causing the 5.0 regression. I know there was work done on how we handled clips, but most of the changes were actual fixes, not bugs.
  If you send us a simple example that demonstrates the problem (and tell us what platform you're running on) we can take a look. You can submit a bug on the jdc site or send it in to the java2d comments page off our java2d page on java.sun.com.

  Posted by: chet on December 18, 2004 at 02:05 PM

This work is licensed under a Creative Commons License.