David Herron's Blog

The Unix wars and Java compatibility

There was a little throwaway idea in my previous posting, Java is doomed to failure, which is growing on me. While writing I had this thought about the Unix wars.. and really that period, the fallout from which is still with us, is a great analogy of what could happen with Java if the various implementations were allowed to differ greatly.

Compatibility matters. That's more than just a slogan on a T-Shirt, it's something we put a lot of effort into. We have lots of engineers working on compatibility tests as a specific focus, performing compatibility tests, and we contractually require the Java licensees to perform compatibility testing.

The Unix wars were a failure of compatibility. For those young enough to have not lived through the Unix wars... Unix is this software platform developed by AT&T Bell Labs engineers which has had a long and circuitous history of arcane proportions. Originally Unix was this cool little thing, when I first encountered it (V7 on a PDP11) Unix was this fun thing, and a huge breath of fresh air from the DECSYSTEM-10 (TOPS-10) on which I'd cut my teeth. Comparing the two was night and day, and I was hooked. I even have a xerox copy of the Lyons book, an annotated copy of the V6 source.

V7 came before the commercialization that led to a couple dozen companies selling hardware running a variant of Unix. As Unix spread to various implementations incompatibilities grew. The incompatibilities made it hard to port software, and fortunately C/C++ has a preprocessor and the #ifdef constructs sort of saved the day by filling our code with arcane tests and twisting threads of duplicated code that implements the same things in different ways.

4.2BSD was 'better', wasn't it? It sure seemed better but, heck, it was incompatible.

Linux came along offering a clean room implementation of Unix, and it too was incompatible, and later on the dizzying varieties of Linux distributions all grew serious incompatibilities with each other. And the Mach/NeXT/MacOS thread of Unix implementations offered their own incompatibilities.

One problem which arose is in the early-to-mid 90's it appeared Unix was gonna die. The incompatibilities made it tough for the Unix market to be taken seriously, and Microsoft made inroads because like 'em or hate 'em, one thing they do is compatibility (of a sort). In a former job I had access to Windows source code and was astonished one day reading Windows NT source to find a comment saying something like "this is a bug, but if we fix it then application XYZZY from WHOZIT, Inc. will break". Indeed many of the Unix's of that time did die.

Another effect of the incompatibilities was the ./configure script. It's now ubiquitous, but I'm reasonably sure the first one was in the Perl distribution. If you're Unix-experienced you know the drill, ./configure, make, make install. The ./configure scripts are enormous shell scripts which check all sorts of system characteristics, often by writing and compiling small C programs to check various things. The most amusing of checks I saw in ./configure scripts was that Perl's checked to see if you're running Eunice, and printed "Congratulations, you aren't running Eunice" .. that is, unless you were running Eunice, and I used to work for the company who developed Eunice.

Anyway I'm sure many people see the ./configure script as a savior ... and for the longest time that's how I saw that script. The ./configure was certainly easier than the alternative of figuring out on your own the settings for conditional compilation choices. But if you stop and think about it, as I've done recently, I see the ./configure script as a symptom of a deeper problem.

The ./configure script points to the compatibility problems which have been endemic in the Unix/Linux camp for over 20 years. Wasn't POSIX supposed to save us from compatibility problems? Bzzzzt, nice try.

There it is.. compatibility.

I think one higher purpose for software is for it to run on whatever hardware you have available. Isn't it frustrating to have a Mac and e.g. want to run Quicken and find that Quicken for the Mac is so bad compared to Quicken for Windows? (BTW, I heard on the Javaposse podcast this week that Intuit will launch soon a cross-platform cross-browser version of Quicken built using Java Server Faces)

In other words I think there is a universal desire for software portability, which in turn requires very strong software compatibility.

But how is this going to be implemented? Do we allow Microsoft to take over the computer industry and make us all use Windows? Or, what?

Comments

• I already find that Java gives me most of what I need to port between all the platforms that I work on, which happen to be Windows and several flavours of *nix (Solaris, Linux, OSX). It remains a pleasant astonishment, having spent over a decade keeping production C++ ported between platforms (actually most of my working life one way or another), how it's hardly necessary for Java at all, and a WAR or JAR built anywhere runs anywhere (bar silly filesystem issues).

  Rgds

  Damon

  Posted by: damonhd on September 04, 2007 at 12:12 AM

• I always have to laugh when I compile unix code and run the ./configure script. I find it funny that a C/C++ program first has to run a script that checks for things like if I have Forth available. The unix environment truly is a disaster and the funny thing is that so many of the unix programmers can't see it, even though there it is staring them in the face every time they run ./configure.
  If you are programming in C on unix, why can't you just write to the ANSI C spec and be done? That is of course how it should be. Let's make sure that with Java it always will be that simple.

  Posted by: swpalmer on September 04, 2007 at 06:45 AM

• allowing microsoft to usher a new era of compatability would be utter madness. i use to develop on the microsoft platform using C/C++, delphi (no java). and in the end there are even bigger incompatabilities on the windows side. certain API only work with certain flavours of the OS (and even only certain service packs). there are many ways (such as in perl) to do a thing wich should have only been one API from the beginning. in that sence java is the fresh air in the computing industry. it forces a reasonable binary translation layer on software. not that all are just peachy in java land. just that given reasonable technical boundries i can get my work done, keep my employer happy and in the end put bread on my table.

  what should concern us now is not the java environment per se, but the question of how to protect java from one (or more) vendor(s) bend on adding features to play the game of vendor locking all over again. incedently this is what killed unix in the first instance. this a far greater danger than some exotic langauge feature.

  there is already some kind of vendor locking within the java platform, jdbc. specifically the fact that the jdbc api does not enforce a standard on the error codes. this means that while an jdbc based application may be cross platform from the java side, it still locked in by the vendor in terms of SQL features and error codes. already the very existance of frameworks such as spring jdbc, jpa etc.. indicates that the using straight jdbc puts us right back into vendorland (and at the database vendor's mercy). now just phone oracle and ask for quote.

  
  

  Posted by: wildfirexy on September 04, 2007 at 06:45 AM

• @wildfirexy: I don't do much with databases, and don't know this issue with error codes. Another part of what you say is the niggling differences in SQL from vendor to vendor. But as for the frameworks, I thought hibernate etc existed because raw JDBC is rather tedious to write and it's helpful to have a simpler way to build the JDBC code.

  Posted by: robogeek on September 04, 2007 at 06:52 AM

• 
  why can't you just write to the ANSI C spec and be done?

  
  You don't know much about the ANSI C standard, don't you? ANSI C lacks everything you need to write real-world applications. Thats why one can't use pure ANSI C on Windows or on Unix for non-trivial tasks. Thats why you have to have additional APIs. Unix has the POSIX standard on top, plus thousands of vendor-specific enhancements. And Windows went from a few hundred API calls in Windows 1.0 to more than 10 000 in NT 4.0 (I lost track with later Windows systems). While POSIX and the Unix extensions at least look like natural ANSI C counterparts that can't be said about the Windows API. MS did some rather tasteless things.

  Java could have been the answer to the incompatible API problem. Wouldn't it have been for Sun neglecting Java for the desktop. So one has to resort to JNI for many real-world Java desktop applications. The moment JNI has to be used is the moment the Java cross-platform feature has left the building. If you write turn-key Java applications you have to provide JNI implementations for all supported platforms. And by definition the platforms for which you don't provide JNI implementations are unsupported. I'd rather like to have a configure script instead of not being able to run an application on some platform.

  Posted by: ewin on September 04, 2007 at 01:17 PM

•  C/Unix/X is still relatively compatible in the sense that it is always possible to port, and maintain ports, that run on various flavours of unix, or server apps and X windows apps.
        The "splintering" of unix is a necessary inconvience to allow for innovation, surely? In opensource, we have linux vs freebsd (vs solaris), so the splintering allows for different licensing philosophies, independent innovation, healthy competition, and ultimately mutual benefit. Similarly with different linux distros. What has changed is that the vendor's software that one is locked into, is opensource.
        These days in java we have vendor framework lock-in, eg. glassfish vs jboss vs spring, but at least those frameworks are opensource, which keeps the vendors honest and hard working.
        What is great about java, is that it popularised the virtual machine, to negate any hardware/OS incompatibilities. I don't expect we'll ever lose that, or get a splintering of opensource java, because it is opensource? I mean, the linux kernel hasn't splintered, GNU hasn't splintered, GNOME and KDE haven't splintered. Java vendors are too busy writing frameworks to splinter java itself ;)
        PS. So i thought i'd comment because i've got some time to kill before being collected for fishing, in Russia, where i'm holidaying right now. I fear that "fishing" involves more vodka than fishing, so i'm bracing myself ;)

  Posted by: evanx on September 05, 2007 at 01:19 AM

• The core problem is that C/C++, POSIX, and many other standards committees set their compatibility goals way too low, and the standards committees where established long after the implementations existed. In the end, the most they could agree on was to really just document what the existing implementations had in common, and what wasn't common.

  See Chapter 6 "Standardization" in "The Design and Evolution of C++", where Stroustrup discusses this issue really well.

  He says this:
  "Thus, 100% portability of standards-conforming programs is not in general an achievable or desirable ideal for C++. A program that conforms to a standard is not necessarily 100% portable because it may display implementation-dependent behavior. Actually, most do."

  People really need to get past the naive idea that standards, as written by committees, ensure (or even help with) compatibility.
  In reality, if you have a standard, it means you already have more than one incompatible implentation, and it's time to do your best to document the mess you're in.
  

  Posted by: atripp on September 05, 2007 at 08:01 AM

• Yes, there is a lot of incompatibility in the unix world.
  Some people say that its good to have different distro's of linux because its good for inovation. I would agree if there would only were a hand full of distro's but a few hundred will not help.

  @ robogeek,
  David, your post has a very odd ending. Can't you write a post without blaming Microsoft for something? You write about the compatibility problems in the unix world and then for no reasons you write about Microsoft. Please act like a grown up man and not like a linux kiddie.

  Posted by: carmello on September 09, 2007 at 03:58 AM

• interesting is it, how people always claim that "Microsoft has a monopoly" and then to on to say they don't want "Microsoft taking over the world". If they had a monopoly they'd no longer need to take over the world, they'd already have succeeded in doing so...
  @wildfirexy: you can hardly expect Microsoft to retain full upwards compatibility in everything they create. Yes, some things work only on some versions of Windows, but that's a generational timeline rather than parallel timelines that claim to be one and the same. If I write something for Windows XP using XP specific APIs I can't expect it to work without modification (if at all) on NT 3.5 (for example). And even if I don't use XP specific APIs I'd not expect it to automatically work on all older versions. After all, Microsoft is constantly fixing bugs (as any company does) in their products so behaviour may change over time in subtle ways. The problem with Unix is quite different. You can write something that works fine on one machine running a 2.6.12 kernel based Debian, but fails utterly on an identical machine running a 2.6.12 kernel based RedHat with identical packages installed (according to the package list). It's the same with every other Unix variant out there, on paper they look identical but in reality they're quite different. And many are different even on paper, the name has become to mean many different things to different people to the extent where you have to state the exact operating system you're using rather than just "Unix ".

  Posted by: jwenting on November 08, 2007 at 11:08 PM

• @jwenting, amen bro. However a little nitpick is that Linux is not certified as Unix. Yeah, Linux and the userspace stuff usually integrated with Linux (Redhat, SuSE, etc) is highly inspired from and highly compatible with Unix. But it hasn't been tested and certified as being Unix. @carmello, yeah I probably shoulda removed the last couple paragraphs. I remember when writing this I had a long rant in mind and decided the rant was a tangent. heh, "linux kiddie", ah gee, I've been using Unix since 1981 and using Linux since 1993. Sooo... ?

  Posted by: robogeek on November 09, 2007 at 06:56 AM

This work is licensed under a Creative Commons License.