Skip to main content

Never Let Me Down Again

Posted by editor on May 2, 2007 at 8:00 AM PDT

How Java and pointers cracked the Mac

So in a brief moment online Sunday, I got an IM from Dick Wall of the Java Posse podcast, fact checking what was known at that point about the Mac security crack that won the CanSecWest security challenge by gaining user-level access to the machine through use of a malicious web page. Since the initial reports indicated that QuickTime was the cause, and turning off Java in the browser was a defense, it seemed to imply that the cause was QuickTime for Java. I mentioned to Dick that it might not just be QuickTime, but any number of associated technologies that QTJ needs to import, and also reminded him of an earlier QTJ security hole that allowed an attacker to use a Mac's webcam and access its images (which is different from my much-cited blog that turns on your webcam but can't access the pixels).

Well, in less than two weeks since the discovery of the hole, Apple has plugged it with QuickTime 7.1.6 for Mac OS X and Windows. And now that a fix is out, security researchers are able to share Details on Dino's QuickTime Advisory. How the attack works is pretty interesting, and actually vindicates what we've said about Java security all these years.

Java's own actions are stringently checked by a security manager, of course, and the fine-grained security model makes it possible (if not particularly easy) to craft security policies that allow or restrict disk I/O, network access, etc. But of course, those guarantees can only be enforced within the realm of Java. You can't, for example, use the security manager to put careful limits on the use of Runtime.exec(), because the command passed to it could be fairly benign (pwd) or ruinous (rm -rf ~). Similarly, native code is resistent to the Java security model, because there's no way to know what's on the other end of your JNI call.

And that's where the problem lies in this case. One of the interesting (and scary) historical traits of QuickTime is the QuickTime Media Layer, which is to some degree a port of much of the classic Mac OS, representing all the parts of that OS that QuickTime depends on (or needed, circa QuickTime 3): Mac-style file and resource API's, QuickDraw, and, scarily, memory management. QTML is how QuickTime could be brought to Windows with all its functionality intact, but it ends up being why this attack works on Windows too.

Because some QuickTime API's require pointers to blocks of memory, such as when you're passing in an image to be compressed and added as a sample to a Movie's video media, there are classes to represent pointers and classic-Mac-style handlers (ie, pointers to relocatable pointers). And there's the rub. The attack uses the QTHandleRef.toQTPointer() method with an integer overflow to copy arbitrary byte arrays to memory. Apple's new fix apparently bounds-checks the arguments to this method so that passing huge negative values to the offset argument can't be used as a means of addressing arbitrary points in memory.

There doesn't seem to be anyone claiming that this crack invalidates Java's security concepts, and Apple has been praised for getting out a fix (with credit to the researchers who found the crack) in fairly short order. If anything, it does remind us that calls from Java to native code can never be assumed to be safe, and if you're using JNI, you may want to take a moment to think if there are opportunities in your code for overflows or other unintended uses of your native calls to go awry.

Also in Java Today,

the second Mobile & Embedded Community Podcast, Report from Brazil, has been posted. In it, Community Leader Roger Brinkley and Tech Evangelist Terrence Barr highlight the latest community news and report on the April events in Brazil at Sun Tech Days and the FISL conference. Don't miss Roger's interview with Bruno and Lucas, project owners of the Marge Project, a Java Bluetooth Framework that shows how to create Bluetooth-enabled applications in a simple way.

Elliotte Rusty Harold's recent presentation to the NYC Study Group, Java 7 and Beyond, offers one of the most concise and thorough tours yet compiled of possible JDK 7 features. The HTML version of his slides covers proposed language changes like closures, property literals, and type inference, and possible library additions like a new date and time API, the Swing Application Framework, and long-awaited filesystem API's to expose and preserve file metadata.