Skip to main content

Safety Last!

Posted by editor on February 27, 2007 at 12:40 PM PST

I think I've been talking up QuickTime and Java in ways that seem to have confused people. Just the other week, the Java Posse talked about "that stuff that Chris Adamson did", and I'm like "oh crap, did they just say I wrote QuickTime for Java?"

The Freshman

Soooo... let's reset. Let's go way back. Way way back. We're talking "first season of The Simpsons" back. 1990, when Apple released QuickTime 1.0, a System Extension providing dynamic media capabilities to Macs running System 7 (and, to a lesser degree, to those on 6.0.8). Like most stuff at the time, this framework was, and continues to be, written in and called by procedural-C.

The term "QuickTime" is also associated with the file format and with provided applications such as the "QuickTime Player", but for the purposes of this blog, we're only interested in the multimedia framework.

During the 90's, Apple launched a couple of companies in collaboration with IBM. One of them was Kaleida Labs, which created an object-oriented, garbage-collected computing platform. Unfortunately, instead of creating Java, they created ScriptX. And I can tell by your puzzled reactions that you've never heard of it. In fact, according to Owen Linzmayer's Apple Confidential 2.0, the Kaleida misadventure resulted in only two pieces of software that could be used in the Classic Mac OS: some Unicode text classes, and QuickTime for Java.

The General

QuickTime for Java provided something that the original, ever-expanding QuickTime library didn't: an object-oriented interface. This made it fairly appealing in the late 1990's, when for a time it seemed like everybody was going to rewrite their applications in Java. Apple was apparently worried that existing QuickTime developers would not only change languages, but possibly media frameworks as well. There's a "Summary of QuickTime for Java" -- I cited it in my QTJ book, but it has disappeared from Apple's site in the intervening years -- that basically spells out Apple's motivation as wanting to continue to provide access to QuickTime, even if developers didn't use C anymore. It even had gentle introductions to Java concepts like object-orientation and garbage collection, while assuming prior knowledge of hard-core QuickTime concepts like reference movies and updating resources. In other words "we'll help you with Java, if you must go that way, but please keep using QuickTime".

The result was that Java developers had, on Mac and Windows anyways, a powerful (if hard-to-use and poorly-documented) media framework, the only one that ever gave Java developers significant creative capabilities like editing.

Problem is, QTJ was a pretty massive effort -- by class count, it is about as large as Java 1.1 -- and that made maintenance difficult. Furthermore, the mass defection to Java as a desktop platform didn't happen (or happened and was quickly reversed), so application developers ended up continuing to write native Mac and Windows apps and use the native libraries. In time, Apple lost interest in QTJ. They managed to break it in 2003 when they moved their Mac OS X Java implementation from Carbon to Cocoa, and fixed it with a radically new version six months later, and that's about the last time they put in a lot of work on QTJ.

Our Hospitality

A lot of people still like QTJ, and have wanted to do more with it. In 2006, I started the Lloyd project. The plan was to do what Apple wasn't doing: continue QTJ development, by writing open-source Java wrappers to procedural-C API's that hadn't been exposed as QTJ classes. I started working on some metadata stuff that was introduced in QuickTime 7. I also wanted to provide an on-screen preview component for video cameras, a feature that had been in earlier versions of QTJ but not after the big breakage of 2003, particularly given that an all-Java method of doing so was posted to the quicktime-java list. I showed off both of these at JavaOne 2006. And I really didn't do much else with it -- I never even attempted to get the native code working as a Windows DLL -- because I was really slammed editing both and ONJava, and trying to do some programming on the side.

You see, at that time, I was also working on a QTJ application for Veldhiezen Group, who were looking for a simple "capture and go" type application. Their chief interest was an interesting feature, the ability to compress at capture time, rather than capturing uncompressed and then do a separate compression step. This necessitated working on the capture preview component and working through a lot of fascinating bugs, including handling changes to the camera settings (if you're expecting pixels in one format, and the format changes, you'll either render garbage or blow out a pixel buffer), dealing with some undocumented threading craziness on windows that was causing deadlocks, catching a bug that mangled the captured audio, etc. My plan was to put what I'd learned about capture previews back into Lloyd.

But in getting help from Apple, they weren't crazy about the idea. At first I thought it was that they thought I was drawing too heavily on not-for-public-consumption code, but it was more than that. They were concerned about leading too many people to base their apps on QTJ because if you hadn't noticed the lack of interest in QTJ over the previous three years, or its heavy dependence on deprecated native libraries like QuickDraw, the point was obvious: QTJ is no longer being actively developed, and depending on a heading-towards-obsolescence library is a Bad Idea.

Sherlock, Jr.

"OK," I'm thinking, "QTJ is breathing its last, and the existing alternatives all suck, so now what are we supposed to do?"

It turns out that I had taken a look at QTKit, the new Cocoa-friendly, object-oriented API for media programming on the Mac. Initially, it seemed at most a pleasant diversion, but Apple has been talking it up more and more in the last year or so. For example, their Leopard Overview notes that QTKit will be the only way to write 64-bit QuickTime applications... the straight-C API will remain 32-bit only.

Well, that's interesting, I thought. And it got me thinking... there's almost 20 years of legacy code in QuickTime, some of which surely supports features that are either deprecated (the old Image Compression Manager that can't handle bi-directional codecs like H.264) or just quietly elided in Apple marketing (anyone remember wired sprites?). Someday, Apple will want to get rid of all that stuff, but without breaking anybody's apps, or QuickTime itself. So what if they created a simpler wrapper framework, got most people to move over to that, and then changed the framework so instead of wrapping QuickTime, it would be a concrete implementation of its features? Total conjecture, but I think QTKit might be part of a massive, decade-ish-long plan to migrate QuickTime over to Cocoa, eventually abandoning the legacy code instead of modernizing it. Obviously, if that's the case, then there are interesting implications (what becomes of QuickTime for Windows or QTJ? how much of the existing functionality will developers demand be migrated to QTKit?), and it'll be worth watching.

Feet First

But at any rate, there's this modern, object-oriented media API, the apparent future of Apple's media API's... and you can't call it from Java. Well, that sounds like a straightforward to-do. How hard could it be to write some JNI code to map Java objects to Objective-C objects?

As it turns out, pretty hard if you don't have a comfort level with Objective-C. But at least I've got the ball rolling... enough to put some code out there in the Keaton project. Keaton is a Java wrapper to QTKit, in pretty much the same way and for the same reasons that QTJ existed to allow Java apps to use QuickTime.

Don't get your hopes up yet; I've only barely started. I've made it far enough to know that it works and is practical. Specifically, I have two classes with one or two meaningful methods each, just enough to allow you to create a QTMovie object from a and render it with a QTMovieViewComponent AWT object. This is what the TestQTMovieViewComponent class does:

    System.out.println ("Creating movie for URL: " + u);
    QTMovie movie = new QTMovie (u);
    System.out.println ("Created movie");

    // set movie on the view
    qtmvc.setMovie (movie);
    System.out.println ("Set movie on view");

Here's what the test app looks like, with movies loaded into both its windows:


The code is extremely primitive at this point, with lots of logging messages and no Obj-C memory management whatsoever. I also crashed trying to open some multi-GB DV files. But the idea works, the whole thing builds with Ant (the .jnilib is even a "universal binary"), and I'm pretty confident that the rest of the QTKit API can be wrapped pretty much with the basic ideas in these classes. Now it's just a matter of taking the time to do so.

Go West

But what does it accomplish? After all, Keaton will only run on Mac, since that's the only platform QTKit exists on. Well, for me, it's an opportunity to get deeper into QTKit and broaden my understanding of media frameworks in general. It might also be useful for developers putting together media apps who need a Mac solution... lots of times, we're forced to use an "isolation layer" approach, in which you put up an API with interfaces for the functionality you need on all platforms, and then back that up with platform-specific implementations you load at runtime (I think ION does this, using both JMF and QTJ and whatever else it can find). It's not an ideal solution, but people have been waiting 10 years for an ideal solution to Java multimedia, and it's obviously not coming, so we need to do what we can with what we've got.

OK, so there's my brain-dump about what I've been playing with media-wise. Thanks for indulging me. Check back in a few months and I might have something worth trying out.