Osvaldo Pinali Doederlein's Blog

Opening Java

I’ve been a close observer of the evolution of the Java platform, and one of the aspects that attracted me since the beginning was its openness. Back in 1996, the JDK 1.0 bundled source code for all public API classes (378Kb!). It was also multiplatform in respect to OS & hardware platforms, and soon a strong multi-vendor platform. As a binary standard, it succeeded where paper-only or source-level standards failed. Java was a big step forward.

But that was ten years ago, when free / open-source software (FOSS) was also warming up. For example, Eric Raymond’s TCATB would appear only 1+ year later  (May ‘97), and it was visionary at that time. Today, Java is huge, the top business app development platform. Meanwhile, FOSS increased in importance, volume and mindshare by orders of magnitude, and it has also become serious business.

Unfortunately, Java is increasingly seen as a problem in the POV of FOSS users and developers. This is despite many significant improvements in openness since ‘96:

Full source code is available, including Sun’s crown jewels like HotSpot.

The first SCSL releases used to be made available several months late, and updated only for major releases, so only licensees who pay a big fee had full access to up-to-date sources. But now with the Mustang projects, the public sources are updated weekly, so anybody can track its development as closely as for any traditional open source project.

The JCP is an effective forum to drive Java standards in an open way.

It’s less open than projects driven by public forums and more relaxed steering committees (no fees for influential positions, no strong dependency on proprietary software vendors, pure meritocracy, etc.). But every model has its tradeoffs, and not everybody is convinced that a “Pure Open” model would scale to the technical and business realities of J2SE.

The relationship between Sun, the JCP, and other commercial backers of Java, and the FOSS community, is very strong.

The latest version of the JCP’s process (JCP 2.6) certifies that open source groups can implement any JSR, and makes the JSR works more transparent for outsiders. The major Java vendors have strong participation in FOSS projects like those from Apache and Eclipse, not to mention others not directly related to Java, like Linux or OpenSolaris. In fact many JSRs have their Reference Implementations delegated to FOSS projects, historically under the umbrella of Apache Jakarta and more recently in java.net.

Java was always a great player with open standards in general.

Many non-Java developers have the impression that we like to reinvent the wheel, creating our own incarnations of everything. The truth is that we just create new implementations of everything. For example, a Java webserver like Tomcat doesn’t reinvent any wheel or break any standard: it strictly adheres to relevant standards, like HTTP. Somebody could ask why don’t we simply use (and contribute to) an existing free httpd, adding new modules for Java-specific technologies like JSP. But the reason for not doing this is not just the WORA mission. The full vertical integration of the software stack, possible when everything is written in Java and optimized for Java, is advantageous in many fronts, from performance to ease of development and extensibility. The major reasons why other people don’t do the same are that their languages don’t have sufficient performance for systems / middleware programming, or their communities are not big enough to support such projects.

Having said that, Java is not yet true Open Source or true Free Software. Sun’s licenses, SCSL and JRL, don’t grant all rights required by the FOSS communities (Bruno Souza explains this in detail). Sun says they want to avoid a proliferation of dialects of the platform. You may think this is very unlikely because few people will be interested in making changes to the Java language or creating new core APIs. But these are not the only sources of ‘dialectization’. You just have to introduce a new bug, one that would create a JCK failure, and the damage is done: it’s not Java anymore.

How justified is this argument? We can answer this by checking current FOSS Java projects, like Classpath, GCJ and Kaffe. These projects do a pretty good job in respecting Java standards. They don’t add any new public APIs, and their compatibility issues are related to insufficient resources to catch up with the proprietary implementations and to lack of access to the JCK under acceptable terms (this is just fixed with J2SE 5.0). So the community has even started its own free replacement for the JCK, Mauve – a strong evidence of the commitment of the FOSS developers to Java compliance. (But I would be too forgiving to ignore GCJ’s CNI, a proprietary native interface that competes with JNI, and has no reason to exist, except for the immaturity of GCJ. Microsoft also did have good technical excuses for the extensions they created long ago. Dear GCJ developers, please kill this bastard; CNI gives reason to all the paranoids that think opening Java is killing its WORA.)

Java has become too important to be ignored even by those old-school FOSS developers who still love lower-level tools like C. FSF leader Richard Stallman’s Java Trap paper puts this in good perspective. RMS doesn’t seem to be a big fan of Java (“people who write free software often feel Java is sexy…”), but he admits that just not using Java is not an option. The world is now full of Java code and Java programmers, and the only viable solution is having a Free Java implementation. This debate is getting hotter by the day. We recently saw the FSF complaining about the growing use of Java in OpenOffice. I think that if this community didn’t have resources to compete alone in this field, they should be happy that Sun has made OO possible. Of course Sun will put Java wherever it’s a good fit. I was amused by the FSF’s initial reaction, hinting towards a fork of OO to drop, or re-implement in other languages, the pieces that use Java. This is IMHO something that should not happen in a healthy FOSS community – and those who value the Free Software ideology over features could easily pick other choices. Fortunately, it seems that the FSF is going to do the right thing: tweak OO and/or improve GCJ until they work together, so OpenOffice 2.0 offers a 100% Free build for those who care. I am one of those who won’t use that (when OO2 goes FCS, I will certainly run it atop any JVM that deliver the best stability  and performance) so I don’t like the additional delay imposed by this solution. But looking from the Forest view, I think it’s a good thing: OO is not the only software that will benefit from improved Free Java implementations, and these improvements may increase the popularity of Java with Free software users. After all, there is a whole boatload of high-quality Free Java software out there. Any effort spent in converting Java to C for specific apps, even important ones like OO, is wasted effort, in perspective.

The other big news is Apache’s newly-incubated Harmony. Everybody is blogging over Harmony, so I will try to add some new thoughts. Harmony intends to build a complete, compliant J2SE implementation, which is a step forward compared to VM- or library-only projects. They are talking to the FSF over Classpath, because without reusing that, the project is not viable, as the J2SE APIs are too big and fast-moving. I’m very curious about the reaction of some players, like IBM, who has been pushing Sun publicly towards open-sourcing Java. Well, IBM owns at least two completely clean-room, production-quality JVMs (IBM JDK and J9), and it seems they have reimplemented large chunks of the APIs too, especially the most difficult ones like CORBA and Security. AFAIK they can open these sources without breaking contract with Sun. And it’s not just IBM; BEA has also became a competitive player recently with JRockit, and Apple, while not in the clean-room team, has made many significant improvements to HotSpot and APIs like Swing.

But I don’t think this is a simple case of cheap talk. It’s more likely a case of “we’re not giving away our competitive advantage, at least not alone”. For major Java players, having their own proprietary JVM is strategic: to have control over its implementation (e.g. to fix bugs that are important to their own customers), add optimizations for your hardware or application server and vice-versa. This is also the reason why just pledging Sun to unilaterally opensource Java has little chance to work. These companies will not open the code for their Java technology before they sit together and find a common solution, one that doesn’t risk anybody’s business plans. (And Sun must be part of the solution, because nobody else has complete clean-room APIs.) Perhaps the best alternative is endorsing a new, independent Free Java project like Harmony, as those big companies know that such project, even with significant resources, will take several years to start competing with proprietary JVMs, if not in compliance certainly in performance. Then Sun, IBM, BEA and others can play nice, even donating resources and developer time to Harmony, while still boasting their superior proprietary implementations whenever it’s good for business. Some day, a great J2SE implementation will be a commodity, but this day is not today. And if you ask me, there’s nothing wrong with this strategy.

Should a full, top-notch J2SE implementation go Free today, we wouldn’t need Harmony, but then we might worry about potential “misuse” of these sources. The risk of forking has been discussed to death and I don’t think it’s big: the Java community has a strong culture of following its standards, and the current status of Free implementations that are not compliant because they can’t is worse than any purposeful forking. A second possibility is that Java code could be useful for competing platforms. This risk is significant, as .NET is a strong competitor that’s also similar enough to Java to allow easy porting. I don’t think Microsoft would do that, but I wonder about FOSS .NET projects, like Mono or DotGNU. These projects are way behind the proprietary VMs (either Java or .NET); and their developers obviously like .NET better than Java, or perhaps for DotGNU, value their Free agenda more than any of the competing platforms. There are VM projects like RMTk, and library projects like NUnit, providing evidence that Java code can be used to accelerate the production of similar .NET software. Not to mention “hybrid” solutions like IKVM, that allows running unchanged Java code on .NET runtimes. Even if the Java code is eventually replaced by new code, written from scratch to better explore .NET’s style and frameworks, the Java code is very useful for prototyping and to make initial versions available quickly. In an ideal world, both communities could benefit from each other’s work, but in practice, the numbers of Java are still vastly superior to .NET’s, so we’re just giving away and not receiving anything back. Companies with a large investment in Java might worry about giving away technology to the competition. They could use viral licenses like the GPL or CDDL, so the Java code cannot be used in proprietary .NET products that are their main competition, but this won’t prevent that code to foster the opensource .NET offerings. But in the bright side (and now I’m dropping my Java-advocate hat to say this), the migration of code from Java to NET, and eventually vice-versa, is interesting from the interoperability point of view, as we tend to have more compatibility between software of both worlds.

The whole debate is even more interesting when we inspect Sun Microsystem’s overall behavior towards open source. We have to commend Sun not only for past glory like NFS, but for very important, high-profile open source efforts like OpenOffice, NetBeans and OpenSolaris. The latter, btw, is supposed to deliver the promised source code by mid-June. (Solaris X will be very appealing to advanced Java developers as soon as DTrace supports Java.) And the weirdest thing is that in these other opensource efforts, Sun doesn’t have any problem with real FOSS licenses. In OpenSolaris, Sun even adopted a “viral”, MPL-like license, perhaps with the same logic of not helping competitors with proprietary OSes, or even Free ones. And while some people inside Sun are very vocal in favor of open source, James Gosling is not, saying that many customers would “freak.. screaming to the hills” over the prospect of opensourcing Java. I admire James for his brave words, in a time when being anti-opensource is almost politically incorrect – I’m just waiting for the Free zealots comparing JG to Bill Gates due to this quote. I would certainly “freak out” if Sun just opened CVS access to the J2SE project so anybody could check code in. But no serious opensource project works that way, they have layers of leadership and code reviewers to validate contributions; and James knows it, and this didn’t stop Sun from opening Solaris. Check the OpenSolaris roadmap for this, they have planned a process to make external contributions possible without wrecking the quality of the Solaris product. In the Mustang project, even at this stage (without full opensourcing), Sun has also put together the basic process and infra-structure for contributions. I say “basic” because they won’t have a significant volume of contribution before they have a FOSS license, but it’s a solid step towards full opensourcing if this ever happens. Now, with respect to freaking customers, Sun should do some customer education before they opensource Java, just like (I guess) they have/will do for Solaris. It’s just a matter of making clear that the huge engineering and QA effort that was put into J2SE will not be compromised. This implies that, if Sun opensources their implementation of J2SE, they should do that in a way that keeps enough control so they can enforce their standards and vision. Look no further than Eclipse, which organization and process was recently criticized for being less open than other FOSS projects… well, as a satisfied user of Eclipse, I think its governance is ideal like it is, and it seems that OpenSolaris won’t deviate much from that model: Sun will obviously keep a very strong leading position... anyway, Free Java is probably not going to happen that way.

So, what about Harmony? It’s an excellent move from the Free community, probably the best (or only) viable way to produce a Free implementation of J2SE, and then increase even further the penetration of the Java platform. Now, if they have as much success as I wish, and in a few years we have a complete, high-performance Free Java, then we will have real answers to the big questions posed here: whether a “pure” FOSS project can deal with something as complex as a production-quality J2SE platform, and whether any of the FUD against Free Java is justified.

Comments

• Nicely put Osvaldo,

  It feels to me like an eruption is about to occur. Many big companies need a free common Java implementation, but no one is willing to offer up his own. Even the GNU project wants one. This is not at all unlike the OS wars, of the last millenium. ;-)

  History is ready to repeat itself. There will be a Linux effect on Java, undoubtedly. However, there is one missing catalyst: The GPL.

  With Linux, the GPL allowed big companies to pour in precious, closely held technology, knowing nobody could 'embrace and extend'; then strongly market a proprietary knock off. Unfortunately Harmony, being under the Apache license, is a perfect setup for this. As such, I doubt many big companies will get involved. The GPL even allowed Linux to incorproate significant technologies from BSD, but goven their supposedly 'freer' license, they were unable to use any technology from Linux.

  Currently a GPL VM and RE would only permit applications to run on it, which were themselves GPL. This is a complete show stopper. The GNU Classpath project is trying to work around this problem by providing an explicit GPL override, known as the Linking Exception. However in Harmony, being under the Apache license, Classpath would also become helpless to exploitation.

  The GPL3 should directly address this issue: Providing something along the lines that anyone can use, extend, and implement GPL Java code, without becoming governed under its license. However, any modifications to the original code itself, are fully subject to the GPL.

  I believe, given these terms, there will be an explosion to create a free implementation, and it will totally rock!

  I wonder; Who will be the Linus Torvalds of Java?

  Posted by: cajo on May 21, 2005 at 11:07 AM

• cajo: You raise an important issue that is the existence of multiple, often incompatible, free/open licenses. It's been also subject of recent debate as the OSI attempts to reduce the proliferation of official Open Source licenses.

  This situation remembers me from Monty Python's masterpiece The Life of Brian, in the scene showing the Jewish opposition to the Romans, split into several competing parties, that preferred attacking each other to joining forces agains their common oppressor and in favor of their common goals. The movie's script was an obvious shot at the left-wing political parties worldwide, but it fits pretty well to the FOSS movement.

  I hope the Harmony project comes out with a very good solution, like dual-licensing of Classpath, as I'm not sure if the improvements of GPL3 will be sufficient to make everybody happy and make sure that everybody contributes to a single project. And the full J2SE is a very big challenge, so if Harmony becomes Yet Another Free Java project, they will miss the critical mass of developers to face that challenge.

  Posted by: opinali on May 21, 2005 at 06:44 PM

• Cajo wrote:
  "Many big companies need a free common Java implementation, but no one is willing to offer up his own."
  
  Might I suggest that the big companies that need a free common Java implementation are not the same companies as those who already own a Java implementation.
  
  Convince a company that already has a Java implementation that it is in their best interest for one to be free, and we'll see one the next day.

  Posted by: johnreynolds on May 23, 2005 at 06:35 AM

• Cajo says: "Currently a GPL VM and RE would only permit applications to run on it, which were themselves GPL. "

  That's wrong. The GPL explicitely lets you run a GPLd application like a VM for any purpose, that is what we call freedom 0. That freedom includes running a GPLd VM in order to run non-GPLd code. :)

  cheers,
  dalibor topic

  Posted by: robilad on May 23, 2005 at 07:12 AM

• I always wonder wether these java guys are just soo stupid or wether they just don't get it??
  SUN HAS FULL CONTROL OVER JAVA - 100%. THATS THE PROBLEM!
  If SUN descides to take money for java all other implementations can go away, cause even with the new SCSL Sun has 100% control.

  Posted by: linuxhippy on May 23, 2005 at 09:46 AM

• Good point Dalibor, my mis-statement. The point I wanted to make was that to distribute an application with a GPLd VM & RE currently would make the application subject to the terms of the GPL: Hence the need for the linking exception workaround.

  Posted by: cajo on May 23, 2005 at 11:02 AM

• There are reasons for FOSS community to demand for a truely Free License. One example that readily comes to my mind, and Sun itself is the reason behind it is the Forte (the original development environment) product that Sun bought. The Forte was being used by lot many financial institutions before Sun bought the company. After having bought it, Sun proceeded to kill it, stopping support for it from 2008. The financial institutions that spend millions of dollars developing their in-house applications using Forte, have to now rewrite the whole damn thing. Complete waste of money, time and effort. This is what happens when you depend on a proprietory development platform.

  This may not happen to Java, because huge amount of industry depend on it. But, it sets a bad precedence, in that a non-Free development platform can influence the whole industry.

  I think this is the reason for FOSS advocates to push for a FOSS development platform.

  Posted by: vhi on May 23, 2005 at 11:58 AM

• Cajo, the GPL affects other works only if they are derived (or copied, or modified) from the GPLd work. If a GPLd work is distributed alongside works that are not derived from it, that's mere accumulation of works, and the GPL can not 'propagate' to those other, unrelated works. If it did, it would not be a free software license :)As long as an application uses the 'standard' APIs, it can not be considered a derived work of a particular GPLd implementation, as the programmer is writing to the spec.If on the other hand, the application is an extension or modification of the GPLd VM, then it needs to be licensed under the same terms.The need for an exception comes from two areas: in embedded space, when we are talking about ROMs and similar deployment options, some provisions of the GPL regarding relinking are a bit problematic to fulfill reasonably well. Secondly, when you are using Ahead-Of-Time compilers, large, copyrightable parts of the libraries are copied verbatim into the resulting binary, and using the GPL without exception for those verbatim copied parts would put the whole binary under the GPL, which most people would not want :)

  cheers,
  dalibor topic

  Posted by: robilad on May 23, 2005 at 01:43 PM

• With due respect, the true advantage of Java is that it allows me to be OS agnostic. I just don't care if the client is running windows, linux or os-x.

  So as a result of that I'm finding it quite hard to feel any sympathy for the open source people bleating about how they can't have Java. Because its an 'own goal'. Yes, yes you can have Java, you just choose not to.

  I do see what people are saying about Sun having control over Java. And wanting to make money off it. But I think its a lot of a red herring. If Sun decides tomorrow that they're going to start charging money for Java... nothing will change.

  I mean, its not as though millions of JREs are going to cry out in anguish and then magically uninstall themselves. I'll still have my various JDKs and will be happily able to keep developing with them.

  The only bad thing Sun could do is Fork Java... so that the free (as in beer) installs of the JDK/JRE are not compatible with the new improved not free (as in Porsche) JavaN... much like various Open Source people say they will do the second Java becomes Open Source.

  If anything, my sympathies lie in the other direction, I think that Sun has given too much control over the future of Java to 'the community'. The problems with the current JCP being that they are dominated by special interest groups (eg IBM) and there is no way to put the brakes on, or for the wider community to vote no.

  The 1.5 fork is a prime example. I think Generics are an incredibly bad idea, but what could I do? Nothing! The special interest groups wanted them, and so they are now rammed down our throats. Thanks muchly.

  I wonder how long it will be before various things like Eclipse and SWT get bundled into the JRE and rammed down our throats... and when I contemplate that I am certain it would be far worse if Java were 'really open' as compares to 'mostly open'.
  

  Posted by: rickcarson on May 23, 2005 at 10:38 PM

• Linuxhippy, I really think you are missing the point.

  You first said : "SUN HAS FULL CONTROL OVER JAVA - 100%. THATS THE PROBLEM!"

  Ever heard of JCP ? Ever read the debate on the various JSR ? Ever vote for JCP elections ? Look at who actually got the control ...

  Then you clamed : "If SUN descides to take money for java all other implementations can go away, cause even with the new SCSL Sun has 100% control."

  This shows a complete misunderstanding of what is Java and who has realy build it. First point if, according to JSPA all the JSR specification are publically available for Free. Also, now with the latest JSPA any Reference Implementation (Sun's J2SE for instance) or Test Compatibility Kit are available at no charge for non-profit organisations (like Apache Group for instance). This is absolutly guaranteed by the JSPA. This is a constract man, I mean, this is absolute legal terms !

  Ok, so now you know that Sun can not make non-profit organisation pay for J2SE (what you maybe think when saying "Java"), but for comercial use, can they ? Actually, no, it is not possible anymore, mainly because of the code contributes from various sources and especially IBM. Be sure, that IBM will protect its Websphere business ;-)

  Posted by: bjb on May 24, 2005 at 01:35 AM

This work is licensed under a Creative Commons License.