Chris Adamson's Blog

Java Media without Mediocrity

In JMF, wherefor art thou?, Mason Glaves wrote of his frustration with the status of Java Media Framework. Hung up on unfixed bugs and the lack of rival implementations of the spec, he writes:

At this point Sun really needs to consider taking action. Either they need to begin active development on the RI again, or they need to come out, officially, and tell the world that they have dropped support for JMF's Reference Implementation. They could sit on the code, or present it to the Open Source community for further development.

I admire the open-mindedness of Mason's blog — the simple reaction would be to say "start working on it again", but Mason realizes that there would also be some upside to an explicit abandonment of JMF. But please oblige me as I take another step back to try to get some perspective.

Why are we even doing media in Java?

There are two ways to attack this question:

1. We're already doing Java, so why do we need media?
2. We're already doing media, so why do we need to do it in Java?

Everyone in these kinds of discussions seems to be a Java programmer, so they look at it from POV 1: they assume that the choice of Java has been made, and then media ends up being something you typically want sooner or later. Heck, we've done this in Swing Hacks: we assume that if you're delivering a polished and/or cool desktop application, that you may need to incorporate sound for various reasons, and we do some stuff with JavaSound, JMF, and QuickTime for Java.

But I think this POV is a trap. How can you really assume Java when, almost 10 years in, very few people are writing and delivering J2SE applications to end-users? (and don't you dare jump to the talkbacks and type "of course Java on the desktop matters - I use Eclipse every day"; that just proves my point of J2SE irrelevance and developer myopia.) Maybe that was the right mindset back in 1997, when all sorts of desktop apps were being furiously ported to Java, and a "build it and they will come" mentality made sense. But it's not true today.

Interesting illustration of this: JMF'ers have been clamoring for an upgrade of JMF's Flash support, since it only handles Flash 2. A Sun response from 2002 is atypically aboveboard in spelling out that:

• Macromedia wrote the original JMF Flash support
• They aren't inclined to do commit further resources to update it
• Sun is not inclined to update it either, and is unsure of its legal ability to do so anyways

And if Sun won't support it either, then we might as well just give up. This is the road that Mason, and everyone else who's used JMF, has travelled.

But what about Plan B? Assume media, and then get to Java? How do we do that?

By being cross-platform. That's the Java value-proposition that's always mattered. That's why Sun, to their credit, is so determined to keep Java from forking and to enforce compatibility.

But then again, there's always been a cross-platform version of JMF... if cross-platform media matters, then why hasn't that taken off? Well, there are some technical reasons. For one thing, its list of supported formats sucks. As user zander said in a talkback to Mason: right now you really have to look for a move to actually play on the darn thing.

The funny thing is, JMF's very open, very extensible architecture should make it easy to add support for more formats and more codecs. It doesn't have a "favorite son" format, and treats all its Players and Processors equally

And I think that may be the key problem. Mediocrity through equality.

QuickTime has a favorite format, and so does Windows Media. These API's see the world through the eyes of their favorite son formats. For example, when you open an MP3 in QuickTime, it ceases to be an MP3, and becomes a Movie with an audio track of MPEG audio. The ID3 tags are stripped from the front of the stream and become user data items.

Is that a bad thing? No, it's a good thing. It's the only way QuickTime can have an editing API — something utterly missing from JMF (despite the fact that JMF has a capture API... and what's the point of that if I can't edit what I capture?!). Import from popular formats, export to popular formats, but have a preferred format, something rich, robust, and widely supported.

There's one really good choice for such a format today: MPEG-4. It's an open, if patent- and license-encumbered, standard, and it's fabulously capable, with support for world-class audio, video, and even interactivity. The latter has been a focus of IBM, whose MPEG-4 Toolkit offers not only an all-Java MPEG-4 video player, but also supports 2D graphics and interactivity via the SMIL-like XMT-Ω markup, and has tools to compile the markup into its binary form. This is very useful to MPEG-4 developers, is frequently mentioned on the MP4-Tech list and would be great if there weren't a big ol' license fee associated with it.

So, is this worth doing? I try not to spend other people's money — I'm a libertarian after all — so let's consider the possible self-interest to Sun or other parties:

• Great MPEG-4 support might help legitimize desktop Java. Many of the things that desktop java was built to do have been done with web applications instead. But I wouldn't want to try to manage my media collection or do any serious media work with a web app, even with flavor-of-the-month AJAX or equivalent DHTML cheese. Done right in Java, this works on Windows, Mac, and Linux from day one (plus the emerging J2SE-on-the-device space, as recently noted by Ted Kosan).
• If you're going to support anything new, support MPEG-4. iPods play MP3 and AAC, both products of MPEG specs. The Sony PSP plays MPEG-4 videos off its memory card. DVD and direct broadcast satellite are moving to the compelling H.264 codec. Media is still fragmented, but MPEG-4 makes most people happy. Java-based (and Java-branded!) tools help people use this media.

A well-thought out approach to MPEG-4 might mean leaving JMF behind: in JMF, there's no object to represent the thing being played or processed (like a Movie in QuickTime for Java), just a DataSource that represents where its media comes from (or a DataSink that represents where it goes). That's probably not a good metaphor for supporting editing, interacting with the variables of an MPEG-4 scene, manipulating metadata, etc.

This isn't necessarily a magic bullet: such an entry would be coming very late to the MPEG-4 party, would compete with QuickTime's embrace of MPEG-4 and Windows Media's attempts to make MPEG-4 irrelevant, and I still can't say with any certainty that there's enough value in cross-platform MPEG-4 support to pay for all the MPEG-4 license fees, to say nothing of the engineering. Maybe it's something that could be explored by say, doing an open-source port of MPEG-4 reference implementations to Java and seeing how it performs and if it merits further work.

But I do think that JMF is a dead-end, and that if there's a future to Java Media, it's because media people will want something that only Java can provide, not because "Java people" want media.

Comments

• How about supporting Ogg Theroa ( http://www.theora.org/ ).

  The quality of the video/compression ratio is acceptable.
  The codec is royalty free.
  The video format and codec is open source.

  The world does not revolve around MPEG4. Many developers need the ability to do high quality video playback without the drawback of having to pay royalties.

  Posted by: peterenglish on April 22, 2005 at 07:25 AM

• And what exactly do "media people" want? JMF and QuickTime for Java are dead and have been dead for the past 2+ years.

  Posted by: matthewkrone on April 22, 2005 at 02:04 PM

• We tried to use JMF for a portal app a couple years ago and it was promising. We killed it because of lack of ongoing dev and support for JMF. Another problem was that it required client intervention to install.

  The main reason for us using Java/JMF to do multimedia is because of Java's promise of multi-platform compatability: we could write an applet or JNLP app that virtually any client OS could run and it would allow us to, at the client, record/compress/upload audio/video. That's _extremely_ powerful when you consider the full range of Java APIs available for use in addition to the media APIs. No other _free_ framework exists to do that with such power and flexibility.

  If there was a free, open, supported and hopefully light-weight Java multimedia capability, I think you'd see Java on the desktop really take off. After all, most users are doing multimedia to some degree: digital photo editing, playing MP3s, video editing, etc. Being able to download a free app that does all sorts of wonderful multimedia tricks in addition to having networking features, etc could be just the reason that compells millions of users to download and install a Java runtime. And imagine being able to assume that your clients have a recent runtime installed... :-)

  Just my $.02

  Posted by: bsavard on April 24, 2005 at 11:29 AM

• What about FOBS? Its a great project on Sourceforge that has taken up some of the slack. It is based on FFMpeg. Check it out.

  http://sourceforge.net/projects/fobs

  Posted by: setaja on April 26, 2005 at 02:54 PM

• This article is picked up at javalobby.org and
  my views are on this thread:

  Should JMF be revived?

  Posted by: rexguo on April 26, 2005 at 08:26 PM

• I am looking at using the Fobs4jmf plugin to JMF for an upcoming project. The major appeal to me is the cross-platform compatibility as I work in a shop with a mix of *nix & Win32 servers & clients. It would be really nice to have more development on JMF or open source and see where it goes from there. A common format (the golden children format) would be a nice benefit as long as it worked across platforms. I like Quicktime for Java, but I can't use it on Linux. I don't see that changing. If Quicktime for Java was ever ported to Linux, I would use that in a heartbeat.

  Posted by: dbnettle on May 03, 2005 at 09:37 AM

• I had a nice comment with supporting statements and all before IE and this preview button didnt get along.

  Go Vorbis. Royalty free - already developed - Audio, Video and Streaming to boot. Vorbis needs a Java implementation and a home to help spread its cause.

  Java needs a fav media format and it needed it yesterday, but MPEG4 has already been taken. (...and a favorite son cannot have 2 fathers ... just ask W)

  Posted by: benco420 on May 22, 2005 at 02:21 AM

• www.vorbis.com

  Posted by: benco420 on May 22, 2005 at 02:25 AM

• FYI, there has been other MPEG-4 solutions for JMF. And when it comes to interactivity, nothing beats real-time video streaming thanks to a native - yet multiplatform - codec.
  Please check JFFMPEG:
  http://sourceforge.net/projects/jffmpeg
  Other projects, including FOBS, have taken it as the basis of their developments.
  And for commercial solution, please check Sjmf:
  http://www.salyens.com/

  Posted by: gtardy on July 10, 2005 at 06:26 PM

• I invite anyone interested to check out FMJ, a new open-source project to implement/replace JMF, at http://fmj.sourceforge.net/.

  
  We are still heavy in the development stage, and are looking for contributors.

  Posted by: kenlars99 on May 24, 2006 at 03:11 AM

This work is licensed under a Creative Commons License.