David Herron's Blog

Community: JDK Archives

OpenJDK Regression Test Harness, also known as jtreg, now available as open source

Go to openjdk.java.net and scroll your eyes down to the Tools section of the navigation bar. You will see a link that's been there a long time, jtreg harness. There is new stuff behind that link now available. Today we have made this harness open source under the GPL+Classpath license combo. (Jonathan Gibbons' announcement)

The OpenJDK Regression Test Harness, also known as jtreg, is the test harness used in the OpenJDK for running unit and regression tests. Jtreg is a wrapper around JT Harness which simplifies its use. Documentation is on the jtreg harness home page.

How is this important? Previously jtreg had been available under the most liberal closed source binary license you could imagine. The license said you can do essentially anything you want with the binary jtreg distribution. No matter how liberal that binary license, it interfered with the ability of some Linux distros to make use of the unit and regression tests. I wish to thank the project members of these Linux distros for explaining the importance and distinction between the most liberal binary license you can imagine and a proper open source license. Once I groked the distinction it deepened my understanding of the open source culture.

In the end what is important is that OpenJDK builds are as high quality as possible. Enabling the distro's to run the unit and regression tests help them have comfort in knowing the tests passed.

Open media and open screens

On an earlier blog posting a commenter asked: "I would like to know how to use the VLC media player stack as the media handler for OpenJDK.." so, yeah, I hear you, there are many asking for better media support in the Java platform. It's with interest that I read Adobe's announcement today of the Open Screen Project. Hey, cool, if you dig around there are documents describing the byte level details of the SWF and FLV and F4V formats. I feel as Nick Main does in 'SWF Spec is now less restrictive !' I've also thought it would be great to have a SWF player implemented in Java and using the newest graphics improvements in the latest JRE's. And, hey, the documents are right there that says in details what the SWF/FLV/etc formats are, especially as they are saying "removing restrictions on use of the SWF and FLV/F4V specifications".

We should be clear, however, that this appears not to be an open source announcement despite Adobe's use of the word 'Open'. Not being a lawyer the following is my best understanding - the documents are under a restrictive license that allows you to print the document once, and you cannot redistribute those documents. So the specification is openly published, as they say, but not open source. I couldn't find any software source code, FWIW.

On hacking the OpenJDK

I'm giving a session at JavaOne this year titled "Hacking the OpenJDK" and it's been very interesting sitting with this topic these last few months. Much of the presentation is an overview of the developer guide, source repositories and other infrastructure on openjdk.java.net which anybody 'hacking' the OpenJDK will need to get started. All that is pretty straightforward and obvious what to cover and how to present it. What's been interesting is what might be some common hacking scenarios with the OpenJDK code.

My co-presenter is the one preparing the demos, and while we have some interesting demos lined up he wanted to know if y'all had any thoughts on "hacks" you'd like to see. The comment box below is available for y'all to leave us ideas. And please remember that we have 50 minutes for the session, 30 minutes will be taken with me talking through the OpenJDK project overview, leaving 20 minutes for demos and questions.

The most interesting and most controversial idea in the session is: how to remove CORBA. I thought it's obvious, the most common change people want in the Java platform is to ditch CORBA. That is, unless you're one of the two people who actually use CORBA, I suppose. Feedback from colleagues was a "are you sure it's our party line to suggest making incompatible changes?" While that's a fair criticism I a) wanted to make it clear the OpenJDK is an open source project and any change is fair game, and b) that some changes create incompatibilities and that there is value in maintaining compatibility with the platform spec. In any case it's not too hard to hack the OpenJDK to remove CORBA, but I'll leave the details until my session at JavaOne.

The session ID is: TS-5230

Interplanetary migrations

I've been subscribed to Planet Classpath and Planet JDK for a couple years. This blog has been aggregated into Planet JDK for a long time, and Planet Classpath was always a "them" aggregation. But, yeah, as Mark Reinhold says, we've been working closely together for awhile and as Mark Wielaard said we now have an ambassador from Planet Classpath in the Planet JDK team. As odd as it is to see my blog being aggregated into Planet Classpath, I welcome this change. May it lead to greater harmony in the movement to Free Java.

Maybe all the Doctor Who episodes I've been watching recently has something to do with this interplanetary exchange.

OpenJDK 6, tastes great, less filling!

It seems the java world is in a bit of an uproar right now with a bit of news which I've seen blogged and newsed about in several places. First, Ubuntu Hardy Heron (8.04; not 'Hardy Herron' as some have been spelling it, sigh) was released this morning, and it does include the OpenJDK. This is part of a larger effort to have several of Sun's products integrated with Ubuntu, and Roman Stroble wrote about it in 'Java and NetBeans on Ubuntu 8.04'. It's really cool to see it there. Even though we've now had DLJ derived bundles in Ubuntu's Multiverse for 2 yrs now having real honest to goodness open source derived builds is so much better, and is a nice result to get to after the last two years of work since the initial announcement at JavaOne 2006. The goodness doesn't stop there as Fedora is still planning to include OpenJDK derived builds in Fedora 9. And there are some hints other distros are beginning work on OpenJDK builds. Perhaps it will be a tsunami as cgwalters suggested? I just think that with the opportunity now for any open source operating system to pick up the OpenJDK, that Java has a bright future in the open source world.

"??Less Filling??" .. that's modern flotsom that popped into mind as I thought about the encumbrance issues. We are on the cusp of having the binary encumbrances cleared. While the OpenJDK represented a huge chunk of open source code, it was only 95% open source. The remaining bits coming from source we hadn't been able to open source. In the months since we've been working on replacement code or on gaining the rights sufficient to open source those encumbered pieces. And the last of those is on the verge of being resolved.

6u10beta is available.. please test it..!

Recently we made Java SE 6 update 10 available for beta testing. Beta testing is a period in product release cycles where testing is taken to people outside the product team, and those "external" testers bang on it with their applications and let the product team know what's wrong (or not).

There is a lot of exciting stuff in 6u10 (formerly known as 6uN) .. there is a lot more here than the typical update release. Unlike most update releases where the work is limited to critical fixes, for 6u10 the changes are pretty dramatic, will be affecting a lot of things, which leaves us wanting to hear about regressions or other kinds of bugs we may have created along the path of getting to 6u10.

(um.. before the wags get ahold of what I just wrote -- any time you write new code it's common to write some bugs as well)

Let's go over some of the new features:-

Java Deployment Toolkit:- hooboy the description is pretty hairy but says a few interesting things. "allows developers to easily deploy applets and applications to a large variety of clients with JavaScripts.... makes it possible to automatically install Java Platform for Java Plug-in applets and Java Web Start applications. The script exposes a single object, named deployJava" and the deplyJava object has a buncha useful functions to reduce the complexity of deploying an applet across multiple browsers.

The Java Kernel (FAQ) makes for a tiny Java platform which can be more quickly downloaded & installed while still allowing the full Java SE platform to be available, requiring downloads of missing pieces as required.

The next generation Java plugin looks really cool and ought to drastically improve applet and javawebstart functionality and behavior. It is a ground-up reimplementation of the venerable plugin which uses a more intelligent architecture which immediately fixes several outstanding glaring bugs and problems. I think that while in most cases incremental improvements and bug fixes are good, there are times where it's better to just start over from scratch and that this is one of those times.

The most critical part of 6u10 to test is this new plugin.

re: Into the light

I think 'Into the light' is the name of an album by Ian Anderson, who is also the a band that's too old to rock'n'roll and too young to die. Anyway it is also Dalibor's way of announcing something really kewl. I am so tickled to have the opportunity to work with him more closely, so here's a few thoughts ..

In the pre-open source days .. the old days, back when we were arguing it wasn't necessary to open source the Java implementation .. to be honest I thought of Dalibor as a bit of a nuisance. Always asking embarrassing questions and I also thought the name 'Dalibor' had to be one of these made up screen names, and always wondered what his real name was. However over time we made our open source the Java implementation announcement. That made his questions less embarrassing, and I grew to know Dalibor, I learned that's his real name and I learned that he is a really nice guy with what I think of as an amazing approach to manifesting his vision for the world.

Dalibor, welcome to the team. I predict great fun.

Duchesses, FOSDEM, International Womens Day, and diversity

Has it already been two weeks since FOSDEM? Gosh. Anyway, I had a great time there and I will probably post another blog entry based on my recollections. I want to mention specifically one of the sessions, the presentation by the Duchesses. They made a very interesting presentation about under-representation and diversity.

Clearly women are under-represented in software development jobs. What I did not know is that women are even more under-represented in open source software development. This is a little surprising given that Free and Open Source Software (FOSS) stems from a yearning for freedom, but at the same time the FOSS community seems overly stereotyped to a male-dominated environment. They claimed that only 3% of open source developers are female. A bit of yahoogling turns up We assume FOSS benefits all equally. But does it really? and Ontario Linux Fest 2007 - Women In Open Source, Angela Byron (2007) and Opening doors to open source for women and linuxchix.org/ and Debian Women and Bridging the gender digital divide in FOSS and LXer Feature: Survival Tactics For Women In FOSS, part 1 ...etc...

Why is this important? I think the clearest answer is one they gave -- diversity. Diversity of experience should probably lead to better solutions than if the people involved have a narrow range of experience. Diversity means having people of all kinds within an organization, of all races, genders, intelligence levels, interests, sexual orientations, etc.

It's also kind of a strange situation -- there's nothing I can think of which makes men more suitable for software development than women. It's not a matter of upper body strength nor who can pee standing up. It's a matter of people possessing intelligence, the ability to conceptualize abstract thingymajobs, and describe those abstractions in software. Of course women are capable of these things.

This reminds me of a blog posting by Lillian Angel last fall ... how are women supposed to feel comfortable entering this male dominated field knowing they'll just be hit on because they're the only woman in the room? Lillian was sitting in the room and it was funny when Mark, the Classpath project leader, recounted how some Classpath project members just assumed Lillian was a guy. Ah, shades of Neo meeting Trinity... ("You're Trinity..the Trinity? I always thought you were a guy... Most guys do")

However there was an interesting angle to their presentation. The statistics presented were about percentages of software developers in software projects. However as in any endeavor there is more involved with creating a software project, whether it's FOSS or not, than just writing the software. Some critical pieces to a successful software project is documentation, testing, quality, marketing, graphics, logos, etc. Their claim is that while Women are under-represented in software development, they do tend to hold other jobs inside software org's like QA and documentation. I am struggling over whether this is good or bad or just is what it is. Yeah, that women tend to be under-represented in coding jobs is a concern and it would be great to even this out a bit, however in the bigger picture women are present in software development organizations. I wondered listening to their talk if they carried an idea that coding jobs have higher prestige or importance to the resulting software than the other jobs.

In any case as tomorrow is International Womens Day I thought to assist making this issue more widely known.

C U @ FOSDEM?

It's that time of year again. The time when free java advocates migrate to Belgium to imbibe in talking. I will be there and mjw tells me he's going to make me talk about something.

The thing I'm pondering the most right now is under which model we can collaborate about Quality in the OpenJDK. The immediate issue is the OpenJDK 6 project, and clearly it is not (yet) beta quality and does not (yet) meet JCK6a conformance. The quality is pretty good but there are some known bugs and it would be nice to fix these.

There is: Exchanging bug data up/down-stream .. test execution on OpenJDK, and sharing test results .. test development ..

At FOSDEM I hope to discuss these and other ideas ...

The end of the beginning of OpenJDK6

I heard Joe Darcy use this phrase .. it's the end of the beginning .. so, yeah, that's about right. "The code is coming! The code is coming!" -- "The code is here!!" is the official announcement of something we have been working hard to achieve. The source drop for OpenJDK 6 is now available. See http://download.java.net/openjdk/jdk6/

This is a very early release, there are bugs, and if you build from these sources they do not yet produce a compliant binary. The latter is being worked on.

Another recent related and important bit of news is the clearing of a major encumbrance .. [OpenJDK 2D-Dev] imaging and color classes binary plugs no longer needed (soon) discusses how we've recently gotten the rights necessary to relicense under the GPL the source for some classes critical to image and graphics rendering. The code is not yet in the OpenJDK6 release, we expect to get it there soon.

This 'end' is just the beginning of a new phase, because there is more work ahead in this. We have made an important step, but we are on a journey and this is only one of the steps along this path.


(11:09:40 AM) mr: http://mail.openjdk.java.net/pipermail/announce/2008-February/000056.html
...
(12:23:11 PM) rkennke: roman@mercury:~/src/openjdk6/control/make$ ../build/linux-i586/bin/java -cp ~/src/test HelloWorld
(12:23:11 PM) rkennke: Hello World
(12:23:18 PM) rkennke: hurray!


JSON versus XML

Is JSON better than XML? is an interesting read. JSON is the new kid for portable data transfer and while JSON derives from Javascript there is JSON parser support in other languages. The posting goes over pro's and con's of using JSON or XML. It appears in this analysis that the big primo advantage of JSON is that browsers have an easier time using it, since XML support on browsers is "spotty" and XML support anywhere means using the DOM model which is a bit klunky. However XML has some glaring advantages what with XPath and XSLT offering powerful ways to sift through XML datasets and transform them to other formats.

Suggests to me a model of using XML for server-server interchange, and then using XSLT to transform it to JSON when squirting data to a browser.

But then he gets to security. JSON relies on using eval() to run javascript commands and that squirts out a javascript object. Um.. that looks like a gaping security problem because that JSON could contain anything and a nefarious website operator (phishing anyone?) could wreak havoc. Maybe.

He does claim that XML data is secure because There is never a possibility that parsing XML data will result in code being executed. ... uh, gee, I have been looking at SVG files recently and the W3C official SVG test suite contains SVG files that use Javascript to do animation. It appears to me that in some instances XML files do contain executable code.

Freeze Java?

The latest Javaposse podcast (Java Posse #158 - Newscast for Jan 9th 2008) is out and has some interesting discussion of a question posed by Bruce Eckel (Java: Evolutionary Dead End). Bruce's title is a little misleading because he's not claiming Java is Dead.. but that some things have been done to the Java language to add unnecessary featuritis some of which are evolutionary dead ends. The suggestion is to freeze the Java language ... and to do language experimentation using other languages that run on top of the JVM.

I'm generally agreeable with this.. I think the moves we've made to make for stronger support of other languages on the JVM is brilliant. I think the important thing is the platform, and really the virtual machine and the ecosystem of existing libraries that run on this platform. That fabled 'network effect' is helping us here.

But 'freeze' the Java language? Hmm.. unsure about that. The language changes which have been made have a poor history of causing adoption problems. But I think it's quite possible to add features to a language where they slide in and feel like a natural part of the langauge. I don't think, as Bruce suggests, that the only time to do language design is at the very beginning of a new language. I do think the early years of a language is critical to forming the language, but that to freeze any changes later may be a mistake. In general I think that frozen things, things which cannot change, are in an inevitable death spiral and that 'life' always means the ability to change and grow.

A couple thoughts I have are:-

Other languages running on the JVM ought to do a good job of integrating with existing Java classes.. the big value of the Java ecosystem is the existing stuff, and that's the big value the JVM offers to a language designer.

Changes to the Java language could be coupled with tools that help to convert source code to the new language features. This could especially make use of the compiler API that is now available and/or the jackpot tooling that is now in Netbeans. I think one of the issues making previous language changes difficult is the existing body of source code, and that if tools existed which robustly and simply made code changes then the migration ought to be easier.

Bruce made an interesting point about compatibility -- as a former member of the CCC team here inside Java SE engineering, I had a few years experience reviewing and approving platform changes. So, heck, compatibility when taken to the extreme we've taken it is really a double edged thingymajob isn't it? It provides some of the value to the Java ecosystem, that theoretically (absent bugs) you can take Java class files compiled in the stone age and still run them today. But it also provides resistance to fixing bugs or adding features, in some cases. This kind of compatibility may well be a heavy weight that over time interferes with the necessary change required to keep the Java ecosystem a living and breathing entity.

But this compatibility requirement goes beyond the language. It includes the runtime libraries of the core classes (java.* and javax.*) and behavior and signatures of the classes and methods therein. Even other languages running on the JVM are affected by binary compatibility requirements.

Hurm.. it occurs to me off the top of my head that a conversion tool could help here as well. Say for Java 8 the goal were to remove all deprecated methods and a few other changes. It could be declared that class file version n corresponds with that change. The JVM could then recognize "oh that's an old class file" and do something automatic like pull in a compatibility library, or do byte code transformations that changes the class files before execution, etc.

In any case the core thing I want to say is one general principle I understand is that things which cannot change are in a death pattern, and things which can change are in a living pattern. Which tells me that to suggest freezing a thing (the Java language) means to say this thing (the Java language) needs to die.

The Java programmer is without property

Todays Editors daily blog references Properties Get No Respect .. and something clicked in my mind and communism started running through my mind. Property is highly valued by the ruling class ... something like: The bourgeoisie keeps more and more doing away with the scattered state of the population, of the means of production, and of property. It has agglomerated population, centralized the means of production, and has concentrated property in a few hands. The necessary consequence of this was political centralization. Independent, or but loosely connected provinces, with separate interests, laws, governments, and systems of taxation, became lumped together into one nation, with one government, one code of laws, one national class interest, one frontier, and one customs tariff.

So maybe this is a brain fart..

I think the analogy might be that if Java-the-language doesn't support 'properties' then the only ones who can code in a property-like fashion are those programmers witty enough to implement the property-like features themselves. Those programmers could be analogized to be the 'bourgoisie' of marx's philosophy, and that there is a conspiracy preventing the proletariat from enjoying property in Java-land.

Or something like that... or it could be a brain fart on the last day before christmas break.

button.text = "Click me" as syntactic sugar for button.setText("Click me!") seems like a small improvement but it would set the stage for larger capabilities. Joe Nuxoll has talked about this at length in javaposse podcasts, especially regarding component-oriented programming. You can browse all the proposals on one wiki site and even vote on them.

Myself I find it tedious to continually write the boilerplate ..


private String name;
/**
* Gets the name of this widget
* @return the name of this widget
*/
public String getName() { return name; }
/**
* Sets the name of this widget
* @param name the new name of this widget
*/
public void setName(String name) { this.name = name; }


This kind of thing seems like prime territory for syntactic sugar.

Should 'Java' stay 'Java'?

I've got a few minutes before heading off to a meeting, and see a few blog postings on approximately the same topic. Should Java (the language) strive to be the perfect be-all-end-all language? Or should it simply strive to be good enough?

On the #openjdk IRC channel one day last week we had an impromptu debate along these lines. One or more people were claiming Java (the language) sucks, that it should have several different uber-features, and basically that since it's not Haskell or some such it's a complete failure. As a language, that is.

Java the platform, Java the runtime environment, that is clearly one of the most successful cross platform runtime systems of all times. Cross-platform runtime systems have a long history, and my earliest knowledge of them is with Forth and with the UCSD p-System Pascal (which I ran, not on an AppleII but on a Terak PDP-11). Java the platform is near-ubiquitous, offers ever-increasing capabilities and performance, supports dozens of different programming languages, and does a great (not perfect) job of behaving well across platform.

Java should stay Java (?|!): Draws an interesting distinction between the uberprogrammers ("übergeeks that think in monads") and the rest of us. Tim Bray in On Closures draws a similar distinction. "My argument is simple: Java worked so well because it hit the 80/20 point; for my money one of the loudest, cleanest 80/20 technology victories ever. Subsequent attempts to fill in the 20% were, well, mostly harmless. Until generics, which are a disaster; they have made Java harder to learn and to understand, and you can’t avoid them."

I am very much in agreement with the points raised by these two. I can't help but remember Beta-versus-VHS being on the losing end by having bought the superior Beta VCR. This is my example of how the 'Best' is not necessarily 'Best'.

Just because some particular language has wonderful ubergeek features, does that make it 'Best'? There was a distinction Graham Hamilton promoted to the Java SE -- that there are a lot more average programmers than there are ubergeek programmers. If this thing we call Java were to be targeted only to the ubergeek programmers it would limit the potential audience for Java. Now, why is that important to anybody but Sun? Clearly this is important to Sun because the more Java appeals to a broad spectrum of developers the bigger a market we have to sell services into. However to the rest of y'all, this is also important because the bigger the Java market is the more work there is for Java programmers. It serves everybody if the Java market were to be as large as possible.

The nice thing about the Java ecosystem architecture is that it supports multiple languages. One can have a range of languages for the average programmer, and one can have language(s) for the ubergeeks, and through the magic of the underlying platform they can work together.

A practical example ... there are a couple ways to run PHP programs on top of the Java platform. This offers you an opportunity to hire PHP programmers to do the front end of a web application, while the business logic and enterprise scalability is written as Java services somewhere in the back.

So.. back to the debate of 'Should Java stay Java'... I don't have the technical background to properly evaluate the different Closure proposals. I have recently learned Groovy and like the way Closures are implemented there, and that's about as far as my preferences go.

But ... I think there's a distinction to be drawn here between Java-the-Language and Java-the-Platform. This distinction is, I think, sometimes lost on people.

New PDF renderer project

So, hey, there's a new open source project, PDF Renderer: a 100% Java PDF renderer and viewer which was just announced by Joshua Marinacci. This is the big secret he's been hinting at for awhile. Okay, that's cool. I remember seeing something in the JavaFX demo's about a PDF renderer there, and with the new scenegraph project one should be able to make a very interesting PDF viewer.

But here's something which I think is a gap ...

The high level question is:- How does one do an online presentation using an open source toolchain? And have that presentation have the whizzy special effects you can do with a tool like Powerpoint or Open Office? Or even whizzier special effects?

Often presentations are exported to PDF ... Open Office has an 'Export to PDF' choice, for example. PDF is, as Josh notes, a widely used format and (news to me) is soon going to be OSI approved. But I believe there isn't a way to encode whizzy effects from a presentation tool into PDF files. One of my readers will no doubt correct me if I'm wrong.

One option I see as a possibility -- Open Office files are a zip file containing XML and other artifacts required by the file. This makes it possible for a viewer tool to be written that understand the OOo format files. This makes it possible, I think, for a Java viewer to take those things and if it understands the whizzy effects OOo can encode in the files then the presentation could go as the author intended -- without the viewer being required to have OOo installed on their machine.

It could be as cool or cooler than a PDF renderer (as important as this renderer is, that is).

What is Needed for the Next Level of Internet Applications?

I found an InfoQ article, What is Needed for the Next Level of Browser Applications?, which brought to mind a couple recent postings I made. The InfoQ article is very well thought out and discusses how to improve the state of applications running inside a web browser.

Ian Roughley starts with: The current level of Ajax toolkits are a result of IE winning the browser race, and all browsers vendors entering a period of stability. Stagnation in browsers features meant that to provide new application features, libraries needed to be developed. And the libraries today have provided good solutions to the holes in browser functionality, as well as providing a browser compatibility layer for developers. and ends with Fortunately the web has already gained huge momentum, being the preferred delivery mechanism of today's applications. And for the web, developers are providing mass and the number of deployed browsers providing velocity.

See, what he's described is the quandry I discussed a couple weeks ago. The Internet is being trapped within a jail called the Web Browser. The limitations inherent in the web browser are limiting the abilities of application developers to provide applications for their customers.

Yes some amazing things have been accomplished with the web, with web browsers, and with the DHTML toolkits. My point is that one can only go so far in application features using an application development framework hosted inside a web browser. Stagnation in browsers, as Ian put it, means application developers have had stagnancy in the capabilities they could code into their applications.

So let's dive into this a bit. An application always runs in a container, whether that container is an appserver, a web browser, or an operating system. We don't write code in machine language (any more) because high level languages make application development easier, and to make high quality multimedia-rich applications requires a high level of abstraction from the underlying machine. To write an application is to use and exercise the capabilities of the container.

You can point to any number of causes for Stagnation in browsers ... the fact is the capabilities of web browsers haven't appreciably increased over time. How can I say that while at the same time being a Firefox user with several extensions installed? Well, the issue is the underlying capabilities Firefox can put into a browser frame are limited by the overall definition of 'what is a Browser'. Also the firefox extensions I find useful are outside the browser frame, and in the surrounding chrome. Like the Web Developer toolbar, or Firebug, etc. The Mozilla Foundation has a bit more freedom in terms of what the XUL application framework can do, but what happens within the browser frame is what I'm discussing here.

Ian discussed 'Features' saying 'new features' are required but they need to be added with care. That could well be, but it seems to me that the programming model in a web browser isn't low level enough. An advantage of a desktop application GUI platform like Swing is that while it provides you with similar basic components (JTextArea, JButton, etc) you can also dive beneath the covers and do your own painting or start with a raw JComponent and build from there. Comparatively in a browser you can start with a <SPAN> or <DIV> tag and use CSS and Javascript to build up look and behavior but some things are difficult because you don't have strong graphics capabilities underneath you (like Java2D) and you don't (so far as I know) have the capability of designing an animation framework like https://timingframework.dev.java.net/ or https://animatedtransitions.dev.java.net/.

While application development is easier with a high level of abstraction, there are times you have to get close to the metal to make the app do what you want. There's a level where the browser-based application development platform simply prevents you from going.

Ian discussed 'Trust' and this is a big thing for Internet applications, whether they live inside a web browser or not. An Internet application takes part in exchanging data over the Internet, uses several protocols, etc, and it doesn't take a rocket scientist to realize a nefarious person could send out a naughty Internet application. Fortunately the Internet community has several years experience with developing these things. For example in the Java app development model you can have several levels of trust with security managers etc. However a desktop Java app running outside a browser generally doesn't have a security manager installed, and the end user probably doesn't have the expertise or capability to add a security manager to a desktop Java app that doesn't have one.

This is something a web browser does bring to this problem, that there is an assumed level of security when an application runs inside a web browser container. Applications running outside the web browser container don't have that assumed security container around them.

Closed device jail, and platform security

A couple weeks ago I jailbroke my iPod Touch and had this strange epiphanette looking at a root shell prompt on a fracking iPod. This morning I allowed iTunes to update it to v1.1.2 and now my iPod is trapped again as being just an iPod. All the extra icons went away, I can't play Pig Shooter, Labrynth, can't use Stumbler to see nearby WiFi access points, etc. Hurm. And I did a bit of yahoogling this morning and see there's some methods available to keep the iPod jailbroken, it seems I have to revert it back to 1.1.1 however.

One interesting aspect of this has to do with the method used to jailbreak the thing. At jailbreakme.com they host a tiff file which contains a security exploit which causes Safari to crash, and once crashed they are able to squirt into the iPod (or iPhone) an installer which then loads up other software. So.. um.. while it's cool they've done this, it's a security concern, right? That exploit could be used by nefarious folk to do nasty things via a broken iPod. It's fortunate that jailbreakme.com is run (I hope) by honest people who (I hope) didn't do anything nefarious with my iPod.

While you might think "what's on an iPod that could be dangerous" .. ah, let's see, iPods get sync'd to a desktop computer regularly and a nefarious bit of software could be transferred into the desktop when sync'd. Also iPod Touch's are Unix systems (remember that shell prompt) and that means the software capabilities are just as endless as Unix allows.

Which just makes me think of this IcedTea bug , 'Bug 78: Use pure Java implementation of zlib from Classpath instead of zlib for java.util.zip' related to a native zlib being embedded in the OpenJDK implementation. It's even an old version of zlib. In the bug discussion they refer to the sort of buffer overrun bugs that is used by jailbreakme.com to crack into an iPod/iPhone. Generally they suggest C code is unsafe, and Java code is safe(r).

Native code can have buffer overrun attacks which can trigger a security problem in a JVM using that native code. In Java code on the other hand it's really hard (if not impossible) to have buffer overrun attacks and in general there is an excellent security model in Java. Hence if things like zlib or image codecs were implemented in Java rather than native code, the code would be safer. The question of course is accuracy (can we reimplement with quality equal to the existing native libraries) and performance (will the JIT do a good-enough job) and would the implementation perform well in an embedded device (JavaME)?

Maybe for security sake it would be good to remove dependencies on native libraries and reimplement them as Java code. But the bug report shows a little investigation - namely, a Java zlib implementation is shown to have less performance than the existing native zlib implementation.

I can't help but also think about the gphone (android) now that I have an iPod that's been put back in jail. The last two weeks has seen a lot of hype regarding the android (gphone) platform. But the reality of what Google has delivered didn't live up to the hype.

It seems the hype was a lot of wishful thinking and that there was similar hype before the iPhone was launched. So in the hope of building some hype around Sun's product line, hey everybody, we have a cell phone platform we announced a few months ago. JavaFX Mobile is supposedly going to be the neatest thing since sliced bread, supposedly gonna have an iPhone-like UI, but be written in Java(JavaFX) and be available on cell phones and as a stack for cell phone makers. Many of the ideas attributed to the gphone before Google revealed Android could also be said of JavaFX Mobile. But we also haven't revealed our concrete plans, so those sentences were an exercise in building hype rather than revealing actual reality.

I think the gphone/iphone hype was related to a freedom that some people desire. Since cell phones are becoming more powerful they "ought to" be more like generalized computing devices rather than single-function devices whose featureitis is tightly controlled. My iPod Touch was more useful as a generalized computing device than it is right now as an iPod.

It's been said that cell phone makers haven't been allowing this to happen. Instead that cell phones have been generally tightly controlled walled gardens. It remains to be seen whether any company will be able to make this dream come to reality. The most interesting article I've seen about this, Yes, Google is trying to take over the world. - By Tim Wu - Slate Magazine, suggests that the U.S. communications infrastructure has been trapped in a multigenerational lock perpetrated by AT&T and GTE (Verizon). The article gives me a whole new perspective on the network neutrality debate of the last couple years.

Cell phones need to be a reliable service. Whatever nefarious things one might say about AT&T and other telecomm companies, they are offering a service which provides a critical communications infrastructure. When there's a disaster don't you want to call for help? The fire dept, police dept, etc, all provide essential services and your way to ask for those services is by placing a phone call. I think part of this tight control stems from their need to provide such reliable service. Or is that just an excuse?

Remember that in the old days AT&T refused to allow third party telephones to be sold, and held us in a lock of having to rent our phones. Their claim? That a third party phone might damage the network. I think it's the same claim they trot out concerning flexible cell phones, so it seems like a suspicious claim, on the other hand cell phone which are really generalized computing devices can truly do more damage if nefarious software gets installed. Think of the damage a Windows virus can do in terms of denial of service attacks. A cell phone virus could launch a denial of service attack which could block cell phone service.

I believe the sweetspot is somewhere in between the shackles of todays walled gardens, and the utopia represented by platforms like OpenMoko.

Java freedom day

Funny how we all have different names for this day. For Tom Marble it's l'anniversaire du Java Libre, for Rich Sands it's November 13 is Java Liberation Day, the same name used by Mark Weilaard ... but I like to think of it as Java Freedom Day. In my mind there are two Java Freedom Days, the first is November 13 (today) and the second would be the opening day of JavaOne.

Java more than any other software platform belongs in the open source camp, but this wasn't so obvious to me until after we did it. The issue of maintaining compatibility is tricky for an open source project, but is vital to the success of Java as a platform. I've written about this several times recently so I won't belabor the point but instead relate a personal experience of the power of Java's compatibility.

I've used a Mac OS X system at home since 10.0. Prior to OS X I made fun of MacOS (Cooperative multitasking? what a joke) and ignored their system, but having spent several years with it now I now appreciate the value of proper design esthetics. In any case that Apple was committed to Java made it all the more compelling to me. (oh, and this is my cue for: 13949712720901ForOSX)

Early on in using OS X I saw an announcement of an MP3 player clone of the WinAmp player, but written in Java (?jlgui?). I downloaded it and even though they didn't claim anything about Mac OS X the thing ran and played music. I sent them an email and they responded saying that gosh they'd not even thought about testing on OS X and were surprised it ran properly. The author of Moneydance (a personal finance app written in Java) made a similar discovery, I remember he described his porting process to support OS X as copying over the .jar file and running it. It just ran.

That's the power of Java compatibility at work.

As an open source project we have the opportunity before us to become integrated with Linux distributions as a core part of the system. That presents untold opportunities as developers would then be able to assume Java's presence rather than the existing situations today which give more questions than certainty. However we are not yet at that point and we have a lot of work facing us to reach that goal. It is of high interest, however.

Here's to a bright future.

A Brave New World

I really am beginning to think the idea I posted a few days ago is very real. In Freeing the Internet from the Web 'jail' I talked about how The Internet can be so much more than The Web ever was. I mean The Web to be confined by The Browser with the capabilities offered by The Browser. The Internet has a whole range of other protocols besides HTTP, and The Internet can transfer tons of other data types.

Let's look at a couple examples:-

Adobe: Online Photoshop coming this year ... it appears from the article to be functionality inspired by Photoshop, but implemented as an AIR application.

WebTools: eBay Desktop (desktop.ebay.com) Is a slicker UI into the eBay universe than you get in the traditional HTML version. I haven't tried it, but the demo in this video looked like better UI responsiveness and featureitis than could be (easily) implemented even with the fanciest of DHTML techniques.

Weiqi Gao: JavaScript To Fragment, The End Of The Reign Of The Browsers he's noting that Microsoft and Mozilla are disagreeing over the future of Javascript. Which means it's likely the disconnect of Javascript features in the major web browsers will continue. And which Weiqi Gao suggests makes it more likely other languages and platforms will come to dominate and supplant the preeminence of the browser.

Well, most of the growth in 'Rich Internet Applications' are based on the FLEX/AIR platform. However as Weiqi Gao points out, there are other platforms which can easily take a role here. He suggests JavaFX and of course we are still working on developing that platform.

JavaFX is a new language (JavaFX Script) with some runtime components including greatly improved graphics and animation layers. It is supposed to run on "the three screens" (desktop, mobile and TV settop) etc. It looks really slick.

That ol' Java java jing jing jing

The Javaposse has been using a specific song as their theme song. It is credited to "Loose" Bruce Kerr, one of Sun's lawyers. I found this blog post: Java Jingle from 1997 which gives an MP3 of the whole thing.

The song was written in 1997.. as I noted yesterday that was in the thick of the issues with Microsoft. It's interesting what the rest of the song is, that is the part after where the javaposse cuts it off.

Freeing the Internet from the Web 'jail'

I've recently been studying our plans for JavaFX -- and at the same time looking at the big picture of where the Internet is going. It's giving me some interesting ideas to ponder.

The immediate idea I'm looking at is a customer scenario perspective on the quality of JavaFX as it's being developed. That's having me look at what sort of web applications are being developed, how RIA's are being used, the broad range of Internet applications on desktop and handset and TV's etc. So, it's unavoidable that I'd end up looking at the big picture, especially as I'm a big picture kinda guy.

Where I've gotten to is an idea that the Internet has been trapped in this prison-like shell we call a 'Web Browser'.

As someone who grew up (Internet wise) in the mid 80's while the Internet was this cozy little village (compared to what it is today) with collegial sharing of everything.. one certainty I built during the early days is that the Internet has a huge array of potential services, and that the Internet has a huge potential in connecting people across all sorts of boundaries.

The Web is a distinct entity which rides on the Internet, and the Web has a relatively limited set of abilities. Those limits come from the behavior of the browser and the programming languages used to build browser capabilities.

For example UI's built with the basic HTML components are very limited and you end up with the old-style web applications where to make the page look different requires a GET request and a complete refresh of the page. AJAX makes this nicer and the DHTML/AJAX camp are doing lots of fun things. There are issues with using CSS and Javascript, which is what I mean by the behavior of the browser is limited to the programming languages available. Such as Javascript being not as cross-browser compatible as one might like, leading programmers to tearing out their hair dealing with the inconsistencies, or else trusting the AJAX framework makers to deal with the inconsistencies. I was reading the other day about the plans for ECMAscript 4. If I understood it right, the Mozilla guy said "hey, let's drastically beef up this language" and the Microsoft guy said "hey, let's use a different language instead". Sigh. The browser wars are destined to be with us for awhile, and cross-browser compatibility is a difficult goal to reach.

But, wait a minute.. consider the web browser as the container for this. I've seen the articles oohing and aaahing over the idea that you should store all your data out on the cloud, that the Internet Service Providers have the backup procedures etc to ensure your data is safe etc, that it's a hassle to keep a computer up-to-date and properly configured... and therefore you should simply live in your web browser and do everything through the web browser.

That is the Web 'jail'. To limit what the Internet can do to what the web browser is capable of doing.

Let me offer a suggested definition... An Internet Application participates in network protocols over the Internet, and utilizes data from one or more Internet protocols, and may additionally use data stored on a local storage. A Rich Internet Application in turn is an Internet Application which has a rich GUI, rich user experience, multimedia, etc etc..

As a Mac user I've had access to a few Apple-supplied Internet applications.. iTunes, iCal, Sherlock, iChat, Audium ...

iTunes is an interesting example. The iTunes Music Store lets you browse a whole slew of stuff, buy stuff, subscribe to podcasts, etc, all without hitting a web browser. There may be HTML components used within iTMS but that's transparent to the user. However at the same time I find some aspects of iTunes clumsy especially the section for browsing podcasts.. there's no way to view 'show notes' for example.

I have a Mini in the living room and use it as the entertainment center. I don't have a TV and instead do couch-potato-like behaviors using Front Row to browse the days podcasts as they get downloaded. It also lets me browse the latest movie trailers. The UI is a little klunky but the overall experience is great in that it's seamlessly querying data over the network but giving me an across the room remote control driven experience of Internet based entertainment.

There's so much more that can be done, and I think this represents a potential flowering of the capabilities of the Internet. Notice I didn't say "the capabilities of the Web" but of the Internet, because it seems to me that the Internet has been held back by the web browser.

The capabilities of a rich desktop GUI toolkit far outstrip what you can do with DHTML. A minor example is the act of editing web content using a content management system or blog application or wiki or the like. Editing in a TextArea using any of the wiki/bbcode/html markup languages is a throwback to the early 1980's when 'vi' and 'troff' were king. I use this (as I'm doing now) because that's what blogging on java.net requires of me. I don't pretend to like it, but I'm putting up with it because blogging on java.net has its advantages... My personal websites are built with Drupal (also using simple textarea's in a browser.. sigh) and recently I have looked at several of the DHTML pseudo-WYSIWYG editors that are integrated with Drupal. See: Improved methods for editing content on a Drupal site. There ends up being two problems endemic to all of them. The main problem to discuss is that the DHTML editors do not understand which operating system they're running on.

All my testing of these pseudo-WYSIWYG editors is in Firefox on a Mac. None of them understood COMMAND-B and the like to turn on bold or invoke other toolbar buttons. In some cases they did detect it and showed me an error dialog telling me that I'm a loser for using a Mac and Mac-specific keyboard shortcuts are not supported. And none of them knew about desktop UI preferences and adjusted their widgets to the native look and feel or native UI behavior.

Which.. ah... that just reminds me of the complaints about Swing L&F behavior across operating systems. Ah.. why isn't as big a stink being made this as the issues of Swing cross OS behavior. Ah well, I suppose it's such a feat of magic to make a web browser do pseudo-WYSIWYG as well as FCKeditor does, that people forget what the real issue is. And I'm digressing.

Another Internet Application I use is Blogbridge to aggregate RSS/Atom news feeds. It's such a cool application and much better than using the web sites that offer similar capabilities. They seem so klunky compared to what Peter Salus and friends have accomplished in BB. My only gripe is sometimes when you click on a link, it takes a looooooong time for the browser to display the page. It seems to only happen with links on digg feeds. I wish that blogbridge had a good HTML component in it so it could directly display the web page rather than show it in the web browser. But then that kinda leads to maybe BB should be running inside the browser...? In BB v6 they added an interesting new ability, to let you set up a 'Smart Feed' which is a search on some service like Technorati, but you don't have to go to your browser to build the search. In the past you could go to those services, enter a search, see the RSS link on the search results page, copy that link into blogbridge, and subscribe that way. Now you say 'Create Smart Feed' and it gives you a list of services and you enter search terms and that's it.

One gaping hole in Blogbridge is podcast support. While it does show you the attachment and you can click on the attachment and play the content, I think they could do a better job of integrating content playback directly into Blogbridge. But.. probably.. they're limited by the limited media support in Java. Hopefully JavaFX will cure this.

It's hard to see what the future is, where all this is going.

I do see that the major Internet platform providers ... Mozilla, Microsoft, Adobe and Sun ... that we all seem to see this. That Rich Internet Applications do not need to be trapped within the web jail.

We are all working on application platforms which offer a way to build rich internet applications which run outside the web browser.

You might be thinking.. Mozilla? .. building an app that lets applications run outside the browser? What kinda crack is this dude smoking? What I mean is their recent announcement of .. ah .. I forget the app name, but it's derived from WebRunner. Basically the Mozilla architecture has for a long time had a bottom layer which lets you build applications using XUL/XBL/XPCOM/Javascript/etc. Firefox and Thunderbird and other apps are built using that foundation.

As I said, I don't know precisely where this is going.. The details are up to individual app developers.

The If's and When's of Java 6 for Mac OS X 10.5

Okay: I don't know either ('if' or 'when', that is), and I'm pissed it wasn't with 10.5. I just wanted to get that out of the way first.

Seems that a lot of other people are also if not pissed are not happy.

However I just read a timeline of Java releases correlated with OS X releases. Seems like there really is a pattern of Java updates shipping slightly after an OS X major release update. Maybe, as Eric Burke points out, we really should calm down a bit and have some patience and expect an update soon. I do hope that's the case.

Really.

Web 3.0 ??

I've been doing some research on the coming wave of technology for "Rich Internet Applications" (my del.icio.us links tell the story).. This JavaFX thing is very interesting and intriguing, as well as the other developments we're working on.

InfoQ has a really good overview of JavaFX and other client side Java improvements that's pretty much on the mark. I believe they're summarizing the information presented at last weeks Client Java Drinks & Snacks Event.

What inspired me to write is WealthFly talking about web 3.0. Treading into this web2.0/web3.0 territory is fraught with marketeerese so please bear with me.

The "Web 2.0" moniker is so vague that how can we adequately talk about it. WealthFly's blog posting is a case example in point. S/he defines Web 2.0 as "the ability to multi-thread and the ability to update browser HTML without refreshing the browser. It lets HTML apps act more like desktop apps and less like simple document viewers." That misses several game changing aspects of web2.0 such as user generated content, simplified read/write of web content, and the whole mashup, microformat, and API phenomenon.

In any case the DHTML (a.k.a. AJAX) aspect of web2.0 is interesting. It has improved user experience of websites but as a technology it leaves a lot to be desired in terms of GUI/UI usability and performance.

Traditional desktop application development methods tend to provide better usability and performance. To take an example of usability, I'm reviewing the rich editing widgets available for the Drupal content management system. I use Drupal for my personal websites (such as davidherron.com) and I've come to recognize that editing web pages in a textarea is a throwback to the usability of the early 80's. Editing in a textarea is very much like using 'vi', and writing HTML or BBCODE or any of the various WIKI markup languages is, well, very much like using 'nroff'. There are several pseudo-WYSIWYG HTML editors written in javascript (such as fckeditor) and the Drupal community has made modules to incorporate many of these.

The last two I evaluated have the same problem. Clicking on a toolbar button (e.g. to select bold) means the textarea looses focus. In a normal word processor such as Open Office clicking on a toolbar button does not shift keyboard focus away from the textarea, and it's a jarring user experience to click on a toolbar button then start typing again and realize your text isn't going into the text area.

Getting back to WealthFly's blog posting ... s/he goes on to discuss "Rich Internet Applications" (a.k.a. web 3.0) as a flashy internet-aware and internet-centric GUI written with traditional desktop GUI techniques.

I suppose this vision is to free the Internet from being trapped in a Web Browser.

I've been on the Internet for over 20 years and remember what it was like before Web Browsers came along. In fact the first impressions of web browsers was as a nicer user interface to browsing FTP sites ... sigh. In any case there are tons of other services the Internet can run besides HTTP and HTTP is hardly the be-all-end-all of what the Internet could be.

For example .. at home I do not own a TV and instead have a Mac Mini in my living room along with a 22" LCD computer monitor. Okay, I'm a geek, but I hate TV and still want to watch DVD's in the living room. The Mini is running iTunes and I've poured a zillion audio CD's into it, and subscribed it to 3 dozen different podcasts (video and audio).

An interesting feature in Mac OS X is Front Row. With the Apple Remote I can sit on the couch or stand across the room and use Front Row to browse the content in iTunes. Podcasts become something akin to a slew of TV channels, in that Front Row makes browsing downloaded podcasts a simple experience.

Front Row also has access to the Apple's movie trailer collection. This also is a simple remote control experience to viewing movie trailers.

What's going on behind the scenes? The Front Row UI hides the complexity, and I haven't taken a look, but of course it's simply a presentation layer for several underlying technologies that retrieve stuff over the Internet. The important thing is there's not a web browser in sight.

OpenJDK Encumbrances being cleared

The OpenJDK project is about 95% of a complete Java implementation. That last 5% or so is what we've called 'encumbrances', code which we weren't able to open source.

The last two weeks has seen a lot of interesting movement to clearing the encumbrances. I don't know whether a comprehensive list of the encumbrances have been posted, unfortunately, but what's been cleared is significant. The following are what I found in a brief scan of the mailing list archives.

[OpenJDK 2D-Dev] Open Source rasterizer is an announcement from Jim Grahm of a new rasterizer, being called Pisces, that will replace the Ductus rasterizer currently being used. Pisces has known quality and performance problems, which the community can pitch in to help with.

In [OpenJDK 2D-Dev] Remove warnings in sun.java2d Mark Reinhold mentioned the the "internal Mercurial cutover" will happen at the end of October (this month) and that shortly afterward there will be external Mercurial workspaces. Also Kelly O'Hair has posted an update on the Mercurial transition.

[OpenJDK 2D-Dev] Freetype font rasteriser discusses the inclusion of the FreeType font rasterizer. There doesn't seem to be an announcement for this. status of freetype work is a status/update from Igor Nekrestyanov.

[Audio-engine-dev] Unencumbered part of JavaSound implementation has been opened .. it is still not completely open source. They've narrowed down what's remaining to be done to a software sound synthesizer and 'OSS mixer' (for Solaris and Linux). Further discussion has occurred in the list on suitable replacements.

[security-dev 00016]: Crypto has been added to OpenJDK "We have just added the cryptographic code (aka JCE) to the OpenJDK source tree. This includes the framework (javax.crypto and friends), the SunJCE provider, and the crypto portions of the SunPKCS11 and SunMSCAPI providers."

status of 6532373 (xcb_xlib.c:50: xcb_xlib_unlock: Assertion 'c->xlib.lock' failed.) is not the clearing of an encumbrance, but it is significant nonetheless. The XCB team has redesigned Xlib and it has been included in X.org's latest release, which will then be included in future versions of various operating systems. AWT and certain other applications have been crashing with those Xlib implementations, which is forcing a rewrite these applications.

Re: Stop the Insanity

A few weeks ago Phil Toland wrote Stop the Insanity about the "rise" in popularity of languages other than Java. (but it didn't pop up in my feed reader until today) Last year it seemed you couldn't turn around without reading another blog entry saying Ruby was going to kill Java, etc. Now the named languages are Haskell or Erlang, but the story is the same. I think the rise of JRuby demonstrates something, however.

'Java' isn't just a language.. it's an architecture that includes a virtual machine, a cross-platform application package format (.jar, etc), zillions of available libraries, a cross-platform GUI toolkit, oh and a language.

Since the architecture allows for executing on the JVM any language which can be expressed in Java bytecodes, this makes for huge flexibility. If you don't like the Java language you can choose among the other languages, and still use all the other features the environment offers.

For example I've recently been playing with Groovy. I wanted to be exposed to closures and dynamic languages and there are many things about Groovy to like. One of the coolest things about it is since it compiles to Java classes, it easily can incorporate any Java library. So I've been having great fun using the Rome library in Groovy to play with blog and podcast aggregation. Groovy makes it simple, especially due to the hugely simplified syntax for expressing an HTML document, while Rome does all the heavy lifting of making RSS and Atom simple.

If Groovy weren't running on the JVM then I'd have to spend a lot more time reimplementing Rome-like functionality in Groovy.

Really Idiotic Acronyms: RIA Approaches

I came across this article by Chris Keene: Really Idiotic Approaches to RIA: Flex, Silverlight and JavaFX .. in which he describes Adobe's FLEX, Microsoft's Silverlight, and Sun's JavaFX as an idiotic timewarp approach to developing Rich Internet Applications. Hmm...

I've been wondering myself somethings about the growth in complexity that browser-based applications offer. Clearly this represents a competitive challenge to some organizations who have been offering certain ways of developing applications. And that offers one way to interpret the Flex/Silverlight/JavaFX movement which this guy is criticizing. That we are each seeing the javascript writing on the wall and trying to aim towards having a continued role in the world.

But I think that's not the entire story. Why should the browser be the king of the world? Why should the browser be the only application someone runs in order to do their work and run applications? Why should the browser be the only way to deliver a rich internet application? And for that matter what does "RIA" mean as an acronym anyway?

For example all desktop GUI application frameworks from Win32/MFC to GTK to Java etc have for ages been able to query for data from over the Internet and could integrate that data into an application. e.g. I use blogbridge as my RSS aggregator and much prefer that it's a desktop application than a web application. Why should it be expected that only a browser can implement a "rich" client that integrates Internet information into a "rich" GUI experience?

What's so special about browsers and Javascript?

Javascript is one language among many.. it implements one programming paradigm which may or may not be a good way to develop applications. Why should the javascript programmers be the only ones who get to play in this field? Oh, and if you want to write javascript programs, there's support for that in Sun's JRE.

In a way, the development of JavaFX or Flex or Silverlight should be seen as a good thing. It represents a broad range of choice, rather than the lockhold of the only way to develop a "rich" application is javascript in a browser.

P.S. I've been a good boy this whole blog posting and wrote Silverlight rather than Silverfish.

Dilbert mentions Java

So of course that means Java geeks across the blogosphere are posting todays Dilbert in their blog. Which led to the following.

As the screenshot shows, I use blogbridge to track the blogosphere.

The Unix wars and Java compatibility

There was a little throwaway idea in my previous posting, Java is doomed to failure, which is growing on me. While writing I had this thought about the Unix wars.. and really that period, the fallout from which is still with us, is a great analogy of what could happen with Java if the various implementations were allowed to differ greatly.

Compatibility matters. That's more than just a slogan on a T-Shirt, it's something we put a lot of effort into. We have lots of engineers working on compatibility tests as a specific focus, performing compatibility tests, and we contractually require the Java licensees to perform compatibility testing.

The Unix wars were a failure of compatibility. For those young enough to have not lived through the Unix wars... Unix is this software platform developed by AT&T Bell Labs engineers which has had a long and circuitous history of arcane proportions. Originally Unix was this cool little thing, when I first encountered it (V7 on a PDP11) Unix was this fun thing, and a huge breath of fresh air from the DECSYSTEM-10 (TOPS-10) on which I'd cut my teeth. Comparing the two was night and day, and I was hooked. I even have a xerox copy of the Lyons book, an annotated copy of the V6 source.

V7 came before the commercialization that led to a couple dozen companies selling hardware running a variant of Unix. As Unix spread to various implementations incompatibilities grew. The incompatibilities made it hard to port software, and fortunately C/C++ has a preprocessor and the #ifdef constructs sort of saved the day by filling our code with arcane tests and twisting threads of duplicated code that implements the same things in different ways.

4.2BSD was 'better', wasn't it? It sure seemed better but, heck, it was incompatible.

Linux came along offering a clean room implementation of Unix, and it too was incompatible, and later on the dizzying varieties of Linux distributions all grew serious incompatibilities with each other. And the Mach/NeXT/MacOS thread of Unix implementations offered their own incompatibilities.

One problem which arose is in the early-to-mid 90's it appeared Unix was gonna die. The incompatibilities made it tough for the Unix market to be taken seriously, and Microsoft made inroads because like 'em or hate 'em, one thing they do is compatibility (of a sort). In a former job I had access to Windows source code and was astonished one day reading Windows NT source to find a comment saying something like "this is a bug, but if we fix it then application XYZZY from WHOZIT, Inc. will break". Indeed many of the Unix's of that time did die.

Another effect of the incompatibilities was the ./configure script. It's now ubiquitous, but I'm reasonably sure the first one was in the Perl distribution. If you're Unix-experienced you know the drill, ./configure, make, make install. The ./configure scripts are enormous shell scripts which check all sorts of system characteristics, often by writing and compiling small C programs to check various things. The most amusing of checks I saw in ./configure scripts was that Perl's checked to see if you're running Eunice, and printed "Congratulations, you aren't running Eunice" .. that is, unless you were running Eunice, and I used to work for the company who developed Eunice.

Anyway I'm sure many people see the ./configure script as a savior ... and for the longest time that's how I saw that script. The ./configure was certainly easier than the alternative of figuring out on your own the settings for conditional compilation choices. But if you stop and think about it, as I've done recently, I see the ./configure script as a symptom of a deeper problem.

The ./configure script points to the compatibility problems which have been endemic in the Unix/Linux camp for over 20 years. Wasn't POSIX supposed to save us from compatibility problems? Bzzzzt, nice try.

There it is.. compatibility.

I think one higher purpose for software is for it to run on whatever hardware you have available. Isn't it frustrating to have a Mac and e.g. want to run Quicken and find that Quicken for the Mac is so bad compared to Quicken for Windows? (BTW, I heard on the Javaposse podcast this week that Intuit will launch soon a cross-platform cross-browser version of Quicken built using Java Server Faces)

In other words I think there is a universal desire for software portability, which in turn requires very strong software compatibility.

But how is this going to be implemented? Do we allow Microsoft to take over the computer industry and make us all use Windows? Or, what?

Java is doomed to failure

In the wake of The Rise of JAVA - The Retirement of SUNW I thought something I discovered on osnews.com was more than interesting. (for the record, I hate this change, but after sleeping on it overnight I realized it's just a ticker symbol and has little real effect ... unless marketing decides to return to naming misappropriation such as Java Desktop System)

Today is the 10th anniversary of OSNEWS, one of the news sites I click on every day. For their anniversary they've dug up some of their original pages and at the top of the news for 30 October 1997 they have this gem: C/net is exposing the future Betamaxes and 8 track tapes of the technology world in a column entitled "10 Technologies that Don't Stand a Chance." Judge for yourself. BTW, one of those doomed technologies is Java.

Looking at that 10 years later I can only think the rumors of Java's death are greatly exaggerated.

The snippet contains dead links but thanks to the wayback machine we can read the original articles.

Why, 10 years ago, did C|NET think Java was dead? ... To quote them we have this familiar promise (write once run anywhere) as C|NET described it back then: Imagine a world without computer incompatibilities. You can write a program on one platform, then run it on a Sun workstation, a Mac, and a PC with any version of Windows. The cost of developing software plummets, software becomes cheaper to buy, and a new age of computing dawns. The lion lies down with the lamb. The land flows with milk and honey. Sounds like a great story, eh?

They then went on to describe how, if Java doesn't run on Windows, then we might as well kiss that dream goodbye because Java is doomed to failure. Or, rather, as they say, Microsoft must also use Java and unfortunately we lived through that history which, at the time, was still very fresh, and having gone through those times we have reached a time when Microsoft is not using Java (or, did they fork Java to create C#/.NET?) and Java is still here and taking a significant role in the world of computing. And the real spectre of that time, which is still affecting our thinking today, is compatibility: Everyone will claim to be supporting it under the banner of "open systems," while adding their own proprietary technologies that completely undermine the goal of universal compatibility. Unix has now splintered into the famous 31 flavors: Solaris, Irix, AIX, DG-UX, HP-UX, Ultrix, Linux, and more. That's why Unix is not proving much of a rival for Windows NT. The same thing will happen with Java, only the Java flavors will undoubtedly have cuter names. This splintering is precisely what Sun is suing to stop. But it's hard to successfully sue someone for not agreeing with your philosophical ideals.

You may recall that Microsoft provided an incompatible Java seemingly in an effort to splinter the Java market and turn it into an equivalent of the splintered Unix market. And the suit was about an inability to comply with a contract Microsoft signed, not philosophical ideals.

I think the general consensus is that proprietary extensions to Unix, which led to the splintered market, greatly damaged Unix (especially in the mid 90's) because the divided house of Unix could not compete against the unified house of Windows NT. It's worth pondering the effect of incompatibilities and splintered platforms, because the issue of Java compatibility is still with us today.

Clearly the splintered Unix market has some problems, and most of those flavors of Unix have indeed died or are barely alive. Some have flourished, as has Windows NT (even as it's not called that any longer). But what of Linux? Linux distributions have a lot of incompatibilities between them, but Linux overall is a big deal in the computer industry. I think the splintered Linux market is itself a hindrance and a blessing. That Linux implementations aren't in lock step total compatibility with each other has allowed a lot of innovation and experimentation. But it's at the same time frustrating e.g. if you're browsing HOWTO or tutorials that describe how to do something, but the article is written for a different Linux flavor than the one you're using all the details will be wrong even down to the names of kernel modules to modprobe. Or if you want to download and run some code the incompatibilities mean a huge ./configure script, which might or might not work, installation of a compiler, and enough technical expertise to know how to deal with compilation and installation. Oh, okay, maybe your linux distribution maker has already done the dirty work for you, but the amount of effort going into the distro's build systems etc is yet another symptom of the incompatibility disease.

That you can relatively successfully download a .jar file and it just runs, that's a great and wonderful thing. Someone can click on a JNLP link or enter a page with an APPLET and have a very high chance it will just work. This is the result of the efforts to maintain compatibility, even when challenged by the 800 pound gorilla of the computer industry to create incompatibilities and a splintered market.

Don't get me wrong about Linux. I've been using Linux since 1993 and am rather happy with it overall. But at the moment I'm thinking of how I see tutorial articles like Setting default editor in ubuntu .. or CPU Frequency Scaling In Ubuntu .. or Hyperic HQ On Ubuntu 7.04 .. and while it's cool there's a lot of tutorials on doing X or Y or Z with ubuntu (my preferred distro) why is the tutorial specifically for Ubuntu? Why isn't the tutorial for Linux? Why are tutorials for Ubuntu incompatible with Redhat or Suse or the other distros?

It's unlikely the incompatibilities between Linux flavors are going to kill Linux.. just as the splintered Unix market did not in the end kill Unix. Hey, even Apple is part of the Unix camp (who, at the time of the C|NET article) was getting ready to release Rhapsody... shudder. But I think incompatibilities do hinder adoption of a software platform.

JDepend and code complexity

I found a blog posting, Hidden dangers of code quality tools, discussing static analysis and code quality, using JDepend as their example. The point they make is that software quality metrics tools can give a false impression, unless you understand what you're looking at. They suggest it's better if you interpret the results of a code quality tool by understanding the context of your code, and understanding what's being measured by the tool.

I'm interested in static analysis tools as a way for the Java SE SQE team to better focus the tests we write. For example it's likely that complex code has more bugs than does simple code, so if we could focus our tests on the complex code we could find more bugs, our managers would be happy with our bug counts, and the developers might grumble with the bug fixing work we're sending their way, but in the end you guys, our customers, would hopefully have a better Java platform to run on.

Interested, I downloaded JDepend and gave it a spin. But maybe my target, to use it to analyze the JDK's rt.jar, isn't a good candidate.

They say: JDepend traverses Java class file directories and generates design quality metrics for each Java package, but actually it also can analyze JAR files (as I asked it to analyze rt.jar and it handled that without problem). It did it's analysis pleasantly fast and up popped a Swing GUI. Actually, what came up was two windows with identical contents, and I'm not at all understanding why it gave me two windows. Since I do not use eclipse I'm not trying out that plugin, but it appears from Enerjy's blog entry the eclipse plugin can graph the results.

The window has a bunch of lines of JTree entries with cryptic notations of various code quality metrics. These are: Number of Classes and Interfaces, Afferent Couplings (Ca), Efferent Couplings (Ce), Abstractness (A), Instability (I), Distance from the Main Sequence (D), and Package Dependency Cycles. Most of the entries are marked with "folder" icons, and double clicking the entry opens up a hierarchy of what appear to be a twisty maze of package names all alike, and I haven't found the pirate yet but I suspect the magic phrase xyzzy is not going to help here.

Maybe to a code quality metrics expert these labels make sense. While there is a guy named David Herron who is an expert on code quality metrics, that guy is not me, he's some other David Herron. But what I gather is there are two kinds of couplings (a coupling is a dependency between a package and some class), and some other measures related to complexity. Abstractness is a simple ratio of abstract classes to concrete classes in a package. Instability is a ratio of the coupling, so a package with a high degree of instability (by JDepend's measure) has dependencies on many other packages.

Clearly this is an expert friendly tool. It doesn't do much to help me understand what I'm looking at and doesn't present the data in a very friendly format. It does have a command line interface, and it can produce XML which can be processed for various purposes, one of which is to generate graphs using the GraphViz package. It also has an API and the author shows examples of running JDepend as a JUnit test, despite the fact that static analysis is not a unit test.

I've added a link in the Javapedia Testing area, specifically on CodeAnalysisApps.

Code for Freedom

A couple years ago I ran a contest asking y'all to find regression bugs in JDK6 (the project formerly known as Mustang) and while that was fun I'm glad I'm not involved in running this new contest we've launched. The new contest, Code For Freedom is really cool, and I'm excited to learn about it. But it's a six month contest and as I said I'm glad someone else is doing it... we're asking students in India to make contributions to certain open source projects, with prizes for the best ones. The prizes are pretty significant, and the opportunity is there for the students to learn some cool technologies, to learn about open source project participation, and to get their feet wet in software development.

In India there is a movement to embrace open source software in a big way.

Java-in-browser availability

One of the questions/concerns about Java is, can a web site author assume that it's there? Part of the popularity of Flash or Javascript is it's in all browsers, but the belief is that Java probably is not installed.

I happen to publish a popular non-technical website, and I think the traffic statistics on that site probably illustrate the capabilities of the average computer. I was just reviewing the usage statistics (using Google Analytics) and am astonished by this (reported over the last month)

Unfortunately Google doesn't break down the version numbers, unlike what they do for Flash support (BTW, Flash v9 is around 65%). In the past I'd always seen 95% or more, but to see 99% is quite refreshing.

There are some indications that modern computers are becoming prevalent. For example it used to be that 800x600 displays using 8bit graphics were overwhelmingly popular. In the same results, 800x600 comes in 3rd (11%) with 1024x768 1st (50%), 1280x1024 2nd (12%), 1280x800 4th (11%), and 1440x900 and other resolutions starting at 5th (4% and downward). 32bit graphics on 87% of the displays, and 8bit on less than 1%.

And the browser chart is interesting:

Which makes me wonder, what can we do to end the misery of people using IE on Mac?

So anyway, this is hardly a comprehensive survey and I'm sure someone will post a comment to definitive results. But I find this very interesting.

Source code isn't text

This is a thought that popped out of my mouth yesterday, and the more I think about it the truer it seems. "Source code isn't text, despite how much it looks like text". Simple text editors (/bin/ed) can edit source programs, so therefore source code must be text, right? er...

What I'm thinking is - that the form we are accustomed to writing programs is simply a textual representation. Just like XML isn't really text, it's a textual representation of a data structure, so too is source code a textual representation of program instructions, so too is SQL a textual representation of set theory.

One byproduct of this line of thinking is it explains a fundamental difficulty with writing software. It has to do with translating between an abstract model into the textual representation. To write software requires cognitively grasping the data structures and operations to be done, and understanding the way a particular language represents those as text.

Another byproduct is explaining the time a newcomer to a project requires before they can be productive. Since the text is just a textual representation, the newcomer has to spend time reading text and translating from the textual representation into the abstract model.

Another byproduct is explaining some sorts of bugs. Since the textual representation of the source code is not in its native format, everybody who deals with it has to translate from the textual representation to their own mental cognitional system. Anybody with experience talking among speakers of a second language know that non-fluent speakers have a difficult time with word choice and grammar and pronunciation, and it just makes communication "interesting". Especially in a conversation among multiple people all of whom are speaking a second (non-fluent) language.

I've been seeing discussion of "Domain Specific Languages" and while I'm sure these are a help .. e.g. I'd think SQL is an example of a DSL, right? It's sure a lot easier to describe set theory operations in SQL than in a general programming language. But these languages are all text based, and hence will all ultimately leave the programmer frustrated because textual representation isn't the best modeling of software.

Most of my programming experience is with textual representation but I can think of some examples where non-textual representation is used.

WYSIWYG-like HTML editors sure make it a lot easier to write HTML code. I much prefer them over writing straight HTML, even when the editor makes mangled looking HTML. (e.g. N|VU is a decent editor but the HTML looks not-so-good)

GUI builders such as the one in Netbeans make it easier to do GUI design. I've written GUI's in textual representation and in GUI builders both, and doing it in a GUI builder is much easier and straightforward.

Some drawing programs (e.g. Inkscape) produce SVG. It's sure a lot easier to make pretty pictures in Inkscape than it is by editing the SVG directly. Ditto with Flash and other similar graphics editors.

Music editors (of which I only know Deluxe Music Construction Set from the Amiga) which show staffs, and notes, and rests, and clefs, and whatnot.. they're sure a lot easier than e.g. making MIDI instructions.

There are no doubt more examples, these are what come to mind. I don't know how true is any of what I've just written. However, these ideas seem to be true.

Re: Java: One Platform To Rule Them All?

On Javalobby Michael Urban asks Java: One Platform To Rule Them All? noting an article, Use Java to Improve Drupal's Scalability. In that article an exploration of running Drupal on the Java platform is done, and he is looking for greater scalability than the regular PHP platform offers. Drupal is a "content management system" written in PHP, and I happen to use Drupal as the basis for some web sites that I run.

So, hmm, PHP. Isn't PHP in competition with Java? Well, so is/was Ruby, right? But as JRuby is exemplifying the ability to run multiple languages on the JVM, it's worthwhile to consider the benefits of recasting even more languages so they run on the JVM. And, to be real, running multiple languages on the JVM is not new at all. What's new is that we (the Java team) are being more open and supporting of this concept. scripting.dev.java.net has information on the multiple languages that run on the JVM.

First, why? The JVM, and the overall Java ecosystem, offers an interesting facility to language authors. The community developing a language has to develop the grammar of their language, code generation, code execution, and useful libraries to facilitate writing applications in that language. The JVM offers a preexisting platform for code execution, that's cross platform, and (now that the OpenJDK project exists) offers the potential of ubiquity. The Java ecosystem is rich with libraries and facilities. And we have over 12 yrs of refinement and optimization of the JVM, both inside and outside Sun, with a wide array of system optimization tools, etc.

For example.. One of the key limitations in Drupal (and other PHP based webapps) is it runs best on top of MySQL. So, sure, MySQL is pretty darn popular but it's by far not the only database out there. Perhaps another database would offer some additional capabilities or scalability. In PHP cross-database support is pretty weak, compared to the rich array of cross-database capabilities offered by the Java ecosystem. If Drupal were implemented knowing PHP could access backend Java-implemented libraries for cross-database support, would make available EJB or Hibernate or the variety of other technologies developed to support enterprise application development. The Drupal developers could continue using their familiar PHP, but have access to very capable backend facilities.

Another key limitation in Drupal is image handling. Like with many other PHP webapps, to handle images Drupal depends on external apps like ImageMagick being installed on the server. The image handling includes autoresizing or watermarking. If Drupal were implemented knowing PHP could access Java facilities, it could simply use Java2D functions to do whatever image manipulation they want, with no dependency on an external tool. Java2D can operate on headless servers (using the Headless AWT toolkit) and operate on offscreen images.

In Geert Bevin's article he discusses using Terracotta to provide system performance improvements. The Java ecosystem includes a wide variety of enterprise-quality solutions scaling webapps, including Terracotta.

At the moment the possibilities for executing PHP on top of the JVM include:

• PHP (www.caucho.com/resin-3.0/quercus/) - pure Java implementation of PHP. The Resin appserver has this bundled
• PHP (php-java-bridge.sourceforge.net/) - uses native PHP implementation through remoting.

On the naming of Java releases

J2EE or JEE, Java 5 or Java 1.5 - Is SUN Crazy?.. Maybe we are crazy, who knows. But it's not unknown in the history of marketing to have product name changes for various purposes. At OSCON last week one presentation was an overview of branding, and the presenter discussed how GTE (General Telephone) had a horrible reputation in the 60's, they realized it would take a generation or more of good behavior to fix that reputation, so instead they changed their name to Verizon.

I'm not sure who 'nitinpai' is but s/he is showering frustration upon the various names Sun's Java implementation has gone under. As an engineer I can only shake my head in bewilderment at some of the things Marketing has done. In this case it just makes me think of the naming changes from SunOS 4.x to Solaris 2.1, 2.2, etc, which eventually became Solaris 8, Solaris 9, Solaris 10, Solaris Developer Express, Solaris 11, etc.

In the beginning there was Java and all was good. Then there was Java2 (just be thankful it didn't become named Java2000). And now there's Java5, Java6, etc. What happened to Java3 and Java4? It's the same thing which happened to Solaris3, Solaris4, Solaris5, Solaris6?

And, this is not an unknown practice. Consider if you will the naming of Windows releases. I believe that Win2000 and WinXP and Windows Vista all have an underlying version of WinNT that you could think of as the "Engineering Version" versus the "Marketing Version". XP is the Marketing version of what Engineers might call Windows NT 6.x (if I've counted right).

JavaTM SE 6, Platform Name and Version Numbers discusses the thinking at the release of Java 6. This is the current definitive statement of how to interpolate the version numbering. The following links are presented as a history of prior forms of versioning Java.

J2SE Code Names is an out-of-date listing of the release version names and project code names.

J2SE SDK/JRE Version String Naming Convention gives you a decoder methodology to help decode the version numbering, except we aren't using that particular set of conventions right now.

Java SE Naming and Versions covers the naming changes which came with Java 6.

Version 1.5.0 or 5.0? discusses the thinking at the release of Java 5.

AJAX application testing

This IBM Developer Works article, In pursuit of code quality: Unit testing Ajax applications came up on my blog crawl today. Andrew Glover says "the emergence of Ajax has essentially invalidated a host of test frameworks and tools that weren't designed to test asynchronous Web applications" and goes on to discuss a methodology to creating unit tests for the GWT application framework.

The approach is to ignore "interaction testing", that is to ignore automation of the interactions with the GUI elements. Instead the approach suggested is to make sure the application is factored with a clean separation of GUI elements and "business logic", and then to test the business logic that's called from the GUI elements.

A similar approach can be taken with any GUI application toolkit. As Andrew says in the article, this is a good approach to building applications in the first place, and it is a recommended best practice to cleanly separate business logic from the UI elements. Doing so makes it easier to refactor the GUI application as your user experience team refines the interactionality of the application.

Testing GUI's at the UI layer can be done, though it requires specialized tools. For example GWT is written with Java language syntax, and one might think one of the Java GUI test tools might do the trick. However those tools only work with Java GUI's, and in GWT the application is compiled to Javascript and executed in the web browser, and the java.awt.Robot class does not exist in the web browser. To automate web browser tests usually test engineers turn to the commercial tools (Silk, WinRunner, etc) and while these will do the job there are a couple issues. First, those tools only exist for Windows so if you want to test the user experience on other platforms (e.g. OS X is gaining in popularity) you're out of luck. Second, the core issue in testing GUI interaction is what happens when the GUI changes? The tests all have to be rewritten to accommodate the GUI change.

This second issue, GUI changes that invalidate tests, is something which strongly encourages the recommendation of cleanly separating UI from business logic. One could refactor the GUI all you want, and so long as the business logic doesn't refactor the tests wouldn't change. And, it's clear that in certain parts of the application development process the user experience people must be able to quickly refactor the GUI, try out new ideas, and make refinement after refinement.

I've been helping the Java SE SQE team with test automation for several years now, and one thing I've learned is that the return-on-investment of a test is better the longer you can reuse that test without having to rewrite it. If your test is coded to relatively unchanging interfaces or system attributes it should be reusable for years and years giving great ROI.

Getting back to the question of testing AJAX applications... I think there's approximately four levels to test:

• The server end which provides the datasourcing used by the AJAX app
• The protocol(s) between the AJAX app(s) and the back end(s)
• The business logic inside each of the AJAX app(s)
• The GUI of the AJAX app(s)

A similar separation of layers could be made for other over-the-network applications, it's just that AJAX application construction techniques present some interesting challenges. While one could automate testing the first two pretty readily, it's an interesting question whether the 'J' part of the AJAX app works correctly. Javascript implementations apparently vary so much in behavior that it's difficult to have them work correctly across web browsers. Testing in every web browser combination you can afford to test is vital, due to these variances.

But this leaves us with a problem. It leaves out testing one of the application layers. How are you going to validate that the GUI behaves properly? If you don't automate GUI tests, that leaves you with manual GUI testing. As I said earlier the tools for editing GUI interaction with a web browser, they're all commercial testing tools, and they only exist for Windows. If someone knows different please let me know. What this means is there's a likelihood the GUI application will be better tested on Windows than on the other platforms, because the these commercial tools can automate tests there.

ANNOUNCING: Sun has open sourced its Java implementation, named OpenJDK

SANTA CLARA, CA (Nov 13, 2006) - (Somewhat tongue in cheek...) Today is a day which will be henceforth known as Java Freedom Day. Today Sun announces the immediate availability of portions of the source of its Java SE and ME implementations. It is available under the GPLv2. You can find out more at openjdk.java.net and at mobileandembedded.org.

SAN FRANCISCO, CA (JavaOne 2007) - Today is the second day which shall henceforth be known as Java Freedom Day. On this, the Second Java Freedom Day, Sun announces the immediate availability of even more of the source of its Java SE implementation, still available under the GPLv2. Sun also announces an interim governance board, interim contribution policies, etc. You can find out more at openjdk.java.net.

Okay, so if this seems like it's coming from the wayback machine ... I have been attending OSCON this week, and am somewhat surprised at the number of people who did not know that we, Sun, have open sourced our Java implementation. Given this I thought it would be nice to make the announcements all over again.

Information about the OpenJDK project is on the project web site, and sun.com/software/opensource/java/ contains another view to describing the open sourcing of Java.

Our current status is that we managed to open source 95% of the JDK source. The code is the train of software leading toward the eventual JDK7 release, however the Java7 project has not officially spun up (no platform JSR for Java7 has been started, for example). We have been rather busy with creating the OpenJDK project, ple