Fernando Lozano's Blog

When Applets are not WORA

During the end of 2005 I had a customer who could not run a Java Applet on his desktops, despite having the latest update from Sun. And the desktops ran the fastest-growing OS and browser in the market today

This customer ran Fedora Linux 4 and Firefox 1.5. We tried other Firefox and Mozilla releases (but could not try old Netscape 4 because the portal uses recent CSS standards and they were planning to start using AJAX). We also tried other releases of Sun Java as far as 1.4.0, besides IBM and BEA JREs for Linux. None of them worked for that particular application.

And what the application did to break compatibility? Nothing so uncommon: it used JSObject to direct the browser window load another page. The applet was a virtual keyboard for user login and we tested it worked flawlessly under Windows with both Firefox, Mozilla and the virus / worms farm (IE). Applets that didn't use JSObject worked fine in both Linux and Windows.

The applet used only standard Java SE / Java Plugin classes, no com.sun.* stuff, no JNI calls and no third party libraries. It is a very basic Swing application.

Digging deeper into the problem, we found the cause was that Fedora 4 packages were compiled using GCC 4.0 while Sun, IBM and BEA Java for Linux were compiled using GCC 3.3 (and older releases with GCC 2.9). So memory layout differences between the code compiled with each GCC release made the interaction between the JVM and the browser lock both, but without generating a SIGSEGV.

We confirmed the problem running the applet under Fedora Core 3 (that uses GCC 3.4) and all went fine. So the customer was left with the option of re-engineering the web app to not use Applets at all, or downgrade the Linux desktop to an earlier Fedora release, that is about to have no support anymore. Unfortunately, free software JVMs like GCJ, Kaffe and JamVM do not yet support reliable applet execution inside the browser.

We would not have had this problem (and the stress it caused) if Sun had a more liberal license that allowed Sun Java to be bundled and distributed with community-supported Linux (like old proprietary but freeware Netscape 4 did) and recompiled / repackaged and redistributed by those distribution mantainers. I hope Harmony or some of the other free software JVM matures quickly enough to give me a real choice over Sun Java.

Comments

• As far as I know, you can download Sun's jvm sources and compile them. Still, it could be even easier, but I am not sure it is all Sun's fault that gcc is not backwards compatible...

  Posted by: tobega on January 27, 2006 at 12:36 PM

• Hi tobega, you could to pay more attention to licenses and non-java software restrictions: :-)

  
  You can download sun sources and compile them, but you cannot redistribute the result. The applicaton servers also people outside the company and we would not allowed to distribute our custom Sun Java build to them. I need either Sun or one of its licensees (IBM, BEA, Blackdown) to provide me a JRE binary compiled using GCC 4.0
  Backwards binary compatibility with native code is a very tricky issue. Of course we would not have any problem at all if GCC 4.0 had the same memory layout as previous releases. But I remember the days I did lots of C programming and how hard it was to change anything without having to recompile *everything*.

  GCC is compatible with a wide array of processors and OSes, using the same code-generation back-end. It's not realistic to expect them to support all that, keep supporting newer CPUs and provide optimum performance without changing data structures. Most native code in the Windows word simply ignores capabilities of any CPU beyond the 386 or first Pentiums.

  
  But my point is, it takes more than a standards commitee and a single vendor to guarantee cross-platform compatibility. Sometimes you need just to allow outsiders do help themselves.

  Posted by: flozano on January 27, 2006 at 01:33 PM

• I certainly agree with most you say, but you don't have to redistribute a binary JRE since your client could just as well compile it for themselves :-) Not an ideal situation, but still...

  Posted by: tobega on January 28, 2006 at 02:59 PM

• Hi tobega, the application has users outside of my client (his business partners) so there would be a need to distribute the custom-build jre to them. But as they could also compile for thenselves, why not just end this trend of providing ready binaries for end users? Sun, Apache, Red Hat and IBM could provide just the sources and everyone compiles what they want.

  Posted by: flozano on January 29, 2006 at 03:43 PM

• Excuse me. I don't think your applet did just use "only standard Java SE / Java Plugin classes." JSObject, the peice you had trouble with is not a standard part of J2SE, and I could be wrong but I don't think it's a promised par tof the Java plugin either. Unlike JavaScript, Java's never really been designed to interact with the web page it's embedded in. If that's what you're trying to so, I'm not surpriused you're having troubles.

  Posted by: elharo on January 30, 2006 at 06:02 AM

• Hi elharo, that's a good point. I have to check if JSObject is really part of Java SE or if it's specified as part of any JSR.

  But it's a fact that many developers do use JSObject as if it were part of "standard Java" and it's not fair to consider Applets are not supposed to interact with the host web browser. If you do this you have no choice over Flash and ActiveX, Applets loose much of their reason to exist.

  Anyway JSObject is documented on Sun JRE and if it doesn't work it's a Sun product fault.

  Posted by: flozano on January 30, 2006 at 06:45 AM

• I hope it was justified from the beginning to: implement website login using Java applet. Since when usual forms of authentication in a web application require an applet? Something's rotten here...

  Posted by: denka on January 30, 2006 at 10:03 AM

• Hi denka, the applet purpose was to provide a "virtual keyboard" to prevent keyboard capture program on the user computer, like many internet banking sites do. The keyboard could have been build other ways, using flash, javascript, but it has to cause the browser to load another page after the user clicks his password. That's when the binary incompatibilities between Sun JRE and Firefox on Linux hurts.

  Posted by: flozano on January 30, 2006 at 10:21 AM

• If all you need is to load another page in the browser, there's been a standard method to do that since the beginning: AppletContext.showDocument

  Posted by: timothymoore on January 30, 2006 at 10:50 AM

• Hi timothymoore, thanks for the pointer, but we tried that also, locking the browser the same way.

  Posted by: flozano on January 30, 2006 at 11:12 AM

• I wonder how Microsoft deals with this sort of issue? They don't let you recompile IE or Flash, yet I've never seen an issue like these before with them. Does Microsoft never update compilers, or are they just a lot more anal about backwards compatibility than the FSF?

  Posted by: afishionado on January 30, 2006 at 02:02 PM

• Hi afishionado, sure Microsoft does a better job regarding backwards binary compatibility, but they have a much simpler problem to manage, after all they don't target a lot of different processor architectures and OS.

  Posted by: flozano on January 30, 2006 at 03:39 PM

• Ah, good point. One OS and three processor architectures (x86, x86-64, and PPC on the XBox, and four with the Cell thingy in the Xbox 360 now) are simpler to deal with. ;-)

  Posted by: afishionado on January 30, 2006 at 06:49 PM

• Hi afishionado, looks like you are mistaking me for one of those radicals that just hate Microsoft and everything corporate-like. No problem.

  Sure it's easier to support just x86, x86-64 and PPC than adding to this Sparc, Mips, Alpha, ARM, Prism, 68K, Zilog, 390 and a plentora of microcontroller processors, and each one for dozen different OSes, specially when you do not provide the same software for all them.

  But none of this changes the fact that Microsoft did a better job keeping binary compatibility with newer releases of its C compiler, and that they have a much easier scenario to manage. Nothing changes also the fact that binary compatibility problems happen from time to time, whatever the platform or product, and we ISVs or consultants have our hands tied unless we get some help from vendors.

  Better knowing about the problem, so you can be prepared and pledge for solutions, or look at workarounds, than be caught by surprise, isn't it?

  Posted by: flozano on January 31, 2006 at 07:05 AM

• I'm way outside of commenting on sources I have actually seen here, but something doesn't seem quite right. It seems more than a little odd that this problem would just silently fail... one would expect to see a seg fault or something if there were truly any mis-alignment.

  
  Ok, so the java plugin "works" in that it loads, displays ui's, etc. This is certainly no small feat considering the amount of libraries that are involved in such an interaction, but the API's go largely one way and involve C-style apis (X11, glibc, etc). Also, I believe that this part of the plugin infrastructure is purposefully very simple. However, the plugin coming back into the browser to interact with the DOM/JavaScript is another beast entirely. This would (at least underneath it all) require some non-trivial interactions with the native mozilla XPCOM architecture if I'm not mistaken. It would seem that this is where the problem lies. What is odd in this situation is that XPCOM (like MS COM) was designed for binary compatibility (this is in fact why IE, et al fairs so well in terms of BC). The data structures are very simple and unless if the alignment of compiler generated structures changed (which is usually tied to a platform/OS instead of compiler), it should work.

  
  Now, I know that it is offensive to run into a seemingly unsolvable problem in something that should "just work" and you are rightfully hesitant to take the deep dive into debugging it, but my guess is that the behavior you are describing is in fact a bug, not an inherent flaw. There are a whole host of library differences in FC3 vs FC4. This may be the root of the problem instead of a pure GCC issue.

  
  Now, you're actually not in a bad debugging situation. Firefox/Mozilla and GCC are fully open source. You can also get the Sun sources and build them yourself (if not distribute them). These types of issues can be the worst if it comes down to some small discrepency in an assembly listing, but I bet this isn't the case. And you're not on your own. If you come armed with some facts (specifically some GDB sessions where you discovered that "X" should be happening, but in fact "Y" is happening), I know that the people on the GCC list can be most helpful. I would imagine the same is true of Firefox/Mozilla.

  
  It is true that if the bug turns up in the Sun code that they will gladly accept the bug but you will still be in no better situation (not being able to distribute your changes), but there may be something you can do in the Firefox/GCC side of things to ease the pain.

  
  Here's another though: Have you tried building a version of Firefox with GCC 3.x?

  Posted by: tlaurenzo on February 07, 2006 at 07:22 AM

• Hi tlaurenzo, it's not completelly odd that the app silently fail. Actually it freezes (both browser and jvm are locked up) and this is a possible outcome of a memory aligmnent error from C code. Segmentation faults are much more common, but they are not garanteed to happen. An infinite loop is also likely.

  We assumed the cause was moving from GCC 3 to GCC 4 because similar problems happened with GCC 2.9 to GCC 3.0 and previous releases of Java and Netscape / Mozilla. But we did not investigated if other library discrepancies would be the real cause.

  You are right that GCC and Firefox allow plenty of room for debugging, and there's a possibility we could find a workaround that didn't touch Sun code. But the customer was not willing to do that, as we could not provide any guarantees. Besides that, the customer (and my team) found the license terms for using Sun Java sources unacceptable.

  The customer is using FC3 while he hopes a future update to the JREs will solve the issue, but if it takes long they will simply drop the Applet and use Flash or Javascript. :-(
  

  Posted by: flozano on February 07, 2006 at 10:33 AM

• Ah, I didn't catch the part about freezing up. I misread the symptoms to be that basically nothing happens when you call the methods. While that could in some strange universe be caused by alignment problems too, those types of problems rarely happen in such a benign way (as you noted).

  It is frustrating when you can *see* the problem but can't do anything about it because some lawyers in the back office of your software vendor prohibited you from doing so. Or even worse, when some evangelical crusade about purity of the platform, etc keeps the tool from actually WORKING in the real world.

  Posted by: tlaurenzo on February 07, 2006 at 02:14 PM

This work is licensed under a Creative Commons License.