Skip to main content

Java Media without Mediocrity

Posted by editor on April 21, 2005 at 5:52 AM PDT

In JMF, wherefor art thou?, Mason Glaves wrote of his frustration with the status of Java Media Framework. Hung up on unfixed bugs and the lack of rival implementations of the spec, he writes:

At this point Sun really needs to consider taking action. Either they need to begin active development on the RI again, or they need to come out, officially, and tell the world that they have dropped support for JMF's Reference Implementation. They could sit on the code, or present it to the Open Source community for further development.

I admire the open-mindedness of Mason's blog — the simple reaction would be to say "start working on it again", but Mason realizes that there would also be some upside to an explicit abandonment of JMF. But please oblige me as I take another step back to try to get some perspective.

Why are we even doing media in Java?

There are two ways to attack this question:

  1. We're already doing Java, so why do we need media?
  2. We're already doing media, so why do we need to do it in Java?

Everyone in these kinds of discussions seems to be a Java programmer, so they look at it from POV 1: they assume that the choice of Java has been made, and then media ends up being something you typically want sooner or later. Heck, we've done this in Swing Hacks: we assume that if you're delivering a polished and/or cool desktop application, that you may need to incorporate sound for various reasons, and we do some stuff with JavaSound, JMF, and QuickTime for Java.

But I think this POV is a trap. How can you really assume Java when, almost 10 years in, very few people are writing and delivering J2SE applications to end-users? (and don't you dare jump to the talkbacks and type "of course Java on the desktop matters - I use Eclipse every day"; that just proves my point of J2SE irrelevance and developer myopia.) Maybe that was the right mindset back in 1997, when all sorts of desktop apps were being furiously ported to Java, and a "build it and they will come" mentality made sense. But it's not true today.

Interesting illustration of this: JMF'ers have been clamoring for an upgrade of JMF's Flash support, since it only handles Flash 2. A Sun response from 2002 is atypically aboveboard in spelling out that:

  • Macromedia wrote the original JMF Flash support
  • They aren't inclined to do commit further resources to update it
  • Sun is not inclined to update it either, and is unsure of its legal ability to do so anyways

In other words, this is a direct repudiation of "build it and they will come," from the maker of one of the most critical internet media formats.

And if Sun won't support it either, then we might as well just give up. This is the road that Mason, and everyone else who's used JMF, has travelled.

But what about Plan B? Assume media, and then get to Java? How do we do that?

By being cross-platform. That's the Java value-proposition that's always mattered. That's why Sun, to their credit, is so determined to keep Java from forking and to enforce compatibility.

But then again, there's always been a cross-platform version of JMF... if cross-platform media matters, then why hasn't that taken off? Well, there are some technical reasons. For one thing, its list of supported formats sucks. As user zander said in a talkback to Mason: right now you really have to look for a move to actually play on the darn thing.

The funny thing is, JMF's very open, very extensible architecture should make it easy to add support for more formats and more codecs. It doesn't have a "favorite son" format, and treats all its Players and Processors equally

And I think that may be the key problem. Mediocrity through equality.

QuickTime has a favorite format, and so does Windows Media. These API's see the world through the eyes of their favorite son formats. For example, when you open an MP3 in QuickTime, it ceases to be an MP3, and becomes a Movie with an audio track of MPEG audio. The ID3 tags are stripped from the front of the stream and become user data items.

Is that a bad thing? No, it's a good thing. It's the only way QuickTime can have an editing API — something utterly missing from JMF (despite the fact that JMF has a capture API... and what's the point of that if I can't edit what I capture?!). Import from popular formats, export to popular formats, but have a preferred format, something rich, robust, and widely supported.

There's one really good choice for such a format today: MPEG-4. It's an open, if patent- and license-encumbered, standard, and it's fabulously capable, with support for world-class audio, video, and even interactivity. The latter has been a focus of IBM, whose MPEG-4 Toolkit offers not only an all-Java MPEG-4 video player, but also supports 2D graphics and interactivity via the SMIL-like XMT-Ω markup, and has tools to compile the markup into its binary form. This is very useful to MPEG-4 developers, is frequently mentioned on the MP4-Tech list and would be great if there weren't a big ol' license fee associated with it.

So, is this worth doing? I try not to spend other people's money — I'm a libertarian after all — so let's consider the possible self-interest to Sun or other parties:

  • Great MPEG-4 support might help legitimize desktop Java. Many of the things that desktop java was built to do have been done with web applications instead. But I wouldn't want to try to manage my media collection or do any serious media work with a web app, even with flavor-of-the-month AJAX or equivalent DHTML cheese. Done right in Java, this works on Windows, Mac, and Linux from day one (plus the emerging J2SE-on-the-device space, as recently noted by Ted Kosan).
  • If you're going to support anything new, support MPEG-4. iPods play MP3 and AAC, both products of MPEG specs. The Sony PSP plays MPEG-4 videos off its memory card. DVD and direct broadcast satellite are moving to the compelling H.264 codec. Media is still fragmented, but MPEG-4 makes most people happy. Java-based (and Java-branded!) tools help people use this media.

A well-thought out approach to MPEG-4 might mean leaving JMF behind: in JMF, there's no object to represent the thing being played or processed (like a Movie in QuickTime for Java), just a DataSource that represents where its media comes from (or a DataSink that represents where it goes). That's probably not a good metaphor for supporting editing, interacting with the variables of an MPEG-4 scene, manipulating metadata, etc.

This isn't necessarily a magic bullet: such an entry would be coming very late to the MPEG-4 party, would compete with QuickTime's embrace of MPEG-4 and Windows Media's attempts to make MPEG-4 irrelevant, and I still can't say with any certainty that there's enough value in cross-platform MPEG-4 support to pay for all the MPEG-4 license fees, to say nothing of the engineering. Maybe it's something that could be explored by say, doing an open-source port of MPEG-4 reference implementations to Java and seeing how it performs and if it merits further work.

But I do think that JMF is a dead-end, and that if there's a future to Java Media, it's because media people will want something that only Java can provide, not because "Java people" want media.

Related Topics >>