Chris Adamson's Blog

April 2005 Archives

Java Media without Mediocrity

In JMF, wherefor art thou?, Mason Glaves wrote of his frustration with the status of Java Media Framework. Hung up on unfixed bugs and the lack of rival implementations of the spec, he writes:

At this point Sun really needs to consider taking action. Either they need to begin active development on the RI again, or they need to come out, officially, and tell the world that they have dropped support for JMF's Reference Implementation. They could sit on the code, or present it to the Open Source community for further development.

I admire the open-mindedness of Mason's blog — the simple reaction would be to say "start working on it again", but Mason realizes that there would also be some upside to an explicit abandonment of JMF. But please oblige me as I take another step back to try to get some perspective.

Why are we even doing media in Java?

There are two ways to attack this question:

1. We're already doing Java, so why do we need media?
2. We're already doing media, so why do we need to do it in Java?

Everyone in these kinds of discussions seems to be a Java programmer, so they look at it from POV 1: they assume that the choice of Java has been made, and then media ends up being something you typically want sooner or later. Heck, we've done this in Swing Hacks: we assume that if you're delivering a polished and/or cool desktop application, that you may need to incorporate sound for various reasons, and we do some stuff with JavaSound, JMF, and QuickTime for Java.

But I think this POV is a trap. How can you really assume Java when, almost 10 years in, very few people are writing and delivering J2SE applications to end-users? (and don't you dare jump to the talkbacks and type "of course Java on the desktop matters - I use Eclipse every day"; that just proves my point of J2SE irrelevance and developer myopia.) Maybe that was the right mindset back in 1997, when all sorts of desktop apps were being furiously ported to Java, and a "build it and they will come" mentality made sense. But it's not true today.

Interesting illustration of this: JMF'ers have been clamoring for an upgrade of JMF's Flash support, since it only handles Flash 2. A Sun response from 2002 is atypically aboveboard in spelling out that:

• Macromedia wrote the original JMF Flash support
• They aren't inclined to do commit further resources to update it
• Sun is not inclined to update it either, and is unsure of its legal ability to do so anyways

And if Sun won't support it either, then we might as well just give up. This is the road that Mason, and everyone else who's used JMF, has travelled.

But what about Plan B? Assume media, and then get to Java? How do we do that?

By being cross-platform. That's the Java value-proposition that's always mattered. That's why Sun, to their credit, is so determined to keep Java from forking and to enforce compatibility.

But then again, there's always been a cross-platform version of JMF... if cross-platform media matters, then why hasn't that taken off? Well, there are some technical reasons. For one thing, its list of supported formats sucks. As user zander said in a talkback to Mason: right now you really have to look for a move to actually play on the darn thing.

The funny thing is, JMF's very open, very extensible architecture should make it easy to add support for more formats and more codecs. It doesn't have a "favorite son" format, and treats all its Players and Processors equally

And I think that may be the key problem. Mediocrity through equality.

QuickTime has a favorite format, and so does Windows Media. These API's see the world through the eyes of their favorite son formats. For example, when you open an MP3 in QuickTime, it ceases to be an MP3, and becomes a Movie with an audio track of MPEG audio. The ID3 tags are stripped from the front of the stream and become user data items.

Is that a bad thing? No, it's a good thing. It's the only way QuickTime can have an editing API — something utterly missing from JMF (despite the fact that JMF has a capture API... and what's the point of that if I can't edit what I capture?!). Import from popular formats, export to popular formats, but have a preferred format, something rich, robust, and widely supported.

There's one really good choice for such a format today: MPEG-4. It's an open, if patent- and license-encumbered, standard, and it's fabulously capable, with support for world-class audio, video, and even interactivity. The latter has been a focus of IBM, whose MPEG-4 Toolkit offers not only an all-Java MPEG-4 video player, but also supports 2D graphics and interactivity via the SMIL-like XMT-Ω markup, and has tools to compile the markup into its binary form. This is very useful to MPEG-4 developers, is frequently mentioned on the MP4-Tech list and would be great if there weren't a big ol' license fee associated with it.

So, is this worth doing? I try not to spend other people's money — I'm a libertarian after all — so let's consider the possible self-interest to Sun or other parties:

• Great MPEG-4 support might help legitimize desktop Java. Many of the things that desktop java was built to do have been done with web applications instead. But I wouldn't want to try to manage my media collection or do any serious media work with a web app, even with flavor-of-the-month AJAX or equivalent DHTML cheese. Done right in Java, this works on Windows, Mac, and Linux from day one (plus the emerging J2SE-on-the-device space, as recently noted by Ted Kosan).
• If you're going to support anything new, support MPEG-4. iPods play MP3 and AAC, both products of MPEG specs. The Sony PSP plays MPEG-4 videos off its memory card. DVD and direct broadcast satellite are moving to the compelling H.264 codec. Media is still fragmented, but MPEG-4 makes most people happy. Java-based (and Java-branded!) tools help people use this media.

A well-thought out approach to MPEG-4 might mean leaving JMF behind: in JMF, there's no object to represent the thing being played or processed (like a Movie in QuickTime for Java), just a DataSource that represents where its media comes from (or a DataSink that represents where it goes). That's probably not a good metaphor for supporting editing, interacting with the variables of an MPEG-4 scene, manipulating metadata, etc.

This isn't necessarily a magic bullet: such an entry would be coming very late to the MPEG-4 party, would compete with QuickTime's embrace of MPEG-4 and Windows Media's attempts to make MPEG-4 irrelevant, and I still can't say with any certainty that there's enough value in cross-platform MPEG-4 support to pay for all the MPEG-4 license fees, to say nothing of the engineering. Maybe it's something that could be explored by say, doing an open-source port of MPEG-4 reference implementations to Java and seeing how it performs and if it merits further work.

But I do think that JMF is a dead-end, and that if there's a future to Java Media, it's because media people will want something that only Java can provide, not because "Java people" want media.

The urge to svn merge

Joshy and I have just sent the final copy for Swing Hacks to the editor, along with over 200 screenshots to the production department (yeah, they're going to love us). After a reasonable but non-trivial amount of time, it'll be available at your favorite bookstore or online bookshelf.

This was my second book (the first is about QuickTime for Java), and my first with a co-author. Important lessons learned from both experiences:

• Subversion rocks -Granted, Joshy covered this earlier. For sharing and revisioning code, it's truly awesome. We used subversion both for the code examples and for the book contents (the write-ups and screenshots). This was an easy way to share and manage stuff between the two of us, far more practical than e-mail could ever have been (imagine juggling source, images, write-ups, etc. for each of 100 hacks). When two contributors (hi Jonathan; hi Romain) came on the project part-way through, it was easy to get them caught up by having them do a svn checkout. In fact, when I dropped into VirtualPC to test my hacks on Windows and make any adjustments, I would just use a separately checked-out repository there, committing back any changes coded on that side. The only thing I can truly say I dislike about subversion is that setting a property to ignore class files is nowhere near as easy or as memorable as just doing cvs ignore *.class
• I should have made an ant build file sooner - On the QTJ book, I set up an ant build file early in the process, and every example in the book can be compiled and run with a single ant call like ant run-ch03-undoableqteditor. It's a hassle up front, but it pays off. We didn't do that work up front on Swing Hacks, so I have to go back and do it now so the downloadable code will be ready for everyone to use when the book's ready.
• Find just the right amount of experimentation to do, and budget accordingly - Generally, inventors of a technology don't write the defining book on it, though there are exceptions (such as O'Reilly's perl, Ruby, and Subversion books). So, the author usually has some amount of learning to do in the writing process, and even a little experimentation. You want to get some things in that nobody's done before (or at least nobody's posted in a place where Google can find it). Thing is, if your whole book is like this, it will take 10 years to write. And if you never do this, the book can have a "nothing new" feel to it. So on Swing Hacks, Joshy and I did put some stuff in the proposal where we said "we haven't done this before, but we really think it's doable". And those were good to figure out and get in there. But they can't all be like that.
• Read your co-authors work early - Joshy and I have different code-quoting styles, and we only realized late in the game that they looked strange when they were in the same chapter. I ended up changing a lot of my long listings to a more broken-into-pieces style to match. By the way, do people still type in programs from books by hand? We've really tried hard to make sure this is still possible, though it is obviously far preferable to download a book's code from its supporting website.

Hopefully, you'll like the book. We learned a lot from each other and from our contributors as we were writing it.

This work is licensed under a Creative Commons License.