Fabrizio Giudici's Blog

Amazingly good APIs

Paul Buchheit criticized Java imaging APIs as they require "closer to 100 lines of Java" just to do a simple image resizing, that in Python can be done in three lines:

i = Image.open("/tmp/c.jpg")
i.thumbnail([220, 133], Image.ANTIALIAS)
i.save('/tmp/c-thumb.jpg', quality=90)

He also criticized the over-use of the factory pattern in Java since it would be part of the stuff that is overly complex in the platform.

Well, there could be a lot to say about Java imaging: there are several APIs, partly they overlap, it's clear that developers often don't know how to start, or use the wrong one, etc... There has been clearly a lack of documentation here - it's a fault.

But I don't think it's a fault that the existing APIs are low-level. I've always thought of the APIs in the Java runtime as basic constructor blocks: they are very powerful, but you need time to assemble them. Think for instance of JDBC: to insert a record in a database you have to write dozens of lines of code.

This to me is a good thing. The fact that there are basic APIs doesn't prevent people from using higher level APIs, which perhaps are a bit less powerful, but simple to use. And it's good if there are many of them. For instance, you could decide to insert that record by means of JPA, JDO, Hibernate, iBATIS, etc. All these libraries do the job in a few lines of code.

For the persistence thing, actually there are a lot of ready-to-use higher level products; for imaging I'm not aware of any. So I started writing one (after all James Gosling just said "Java is a community, not a product"), Mistral, which is such a higher level imaging API which sits on top of basic APIs . With Mistral you would do the task in something like:

EditableImage i = EditableImage.create(new ReadOp("/tmp/c.jpg"));
i.execute(new ResizeOp(220, 133), Quality.BEST);
i.execute(new WriteOp("JPEG", "/tmp/c-thumb.jpg", new ImageWriteParam() {{ setCompressionQuality(90); }}));

These are three lines of code as the Python example. If people are scared by the long third line, I'll be glad to provide some subclass of ImageWriteParam that takes the quality as a constructor parameter.

Now, within Mistral there are lots of factories and other patterns. They are initialized by default, so no code is needed in most cases, and the default behavior is to use Java2D. With a single line, you can decide to use JAI - or perhaps ImageJ or another existing library. If you really like your own way to do the resizing, you can define your own MyResizeOp and use with just a line of code as above.

Now, with just a few more lines of code, you can wrap your statements into an ImagingTask:

ImageProcessor.getDefault().post(new ImagingTask() 
  {
    protected void run()
      {
        ...
      }
  });

If you have a lot of images to process in a batch, this will eventually load balance them on a multi-core system; with an alternate configuration, it can distribute the tasks to a local cluster; with other few changes in the configuration it can run on the Sun Grid (for doing all of this, you are kindly requested to wait for some weeks as we finish the implementation of the distributed image cache which is necessary for the performance).

Now, all of this with the same few lines of code, working in different scenarios. I don't know Python, but I don't think it is able to do all of this. That's why I like Java a lot more. :-)

Disclaimer: in spite of the title, I'm not asserting that Mistral is made by amazingly good APIs - even if I thought, I wouldn't tell it publicly until the product is complete, tested and released. The title was just chosen after Pauls' one...

Comments

• 
  The number of lines required to accomplish something is one measure of an API's "ease of use". Another would be the ability to quickly learn how to do something in the API. Sometimes the number of primitives can make it hard to see some of the useful ways to combine them. A monolithic interface lets the programmer hit the completion key in their IDE and quickly see what functionality is available.

  
  So these ideals of composability and discoverability (if they were real words) are basically opposed to each other, but that can be resolved by providing convenience classes that use the primitives. There's no reason the Python example couldn't be done equivalently in Java with a layer that hides the construction of the operation and parameter objects:

  
  EditableImage i = EditableImage.read("/tmp/c.jpg");
  i.resize(220, 133, EditableImage.QUALITY_BEST);
  i.writeJpeg("/tmp/c-thumb.jpg", 90);
  

  Posted by: dnoble on May 22, 2007 at 01:32 PM

• Right. The point with a slighly more verbose syntax like in my example is because of design. Using the "Command" pattern, thus encapsulating every operation in a separate class, makes it possible to extend the functionality by the user with new operations that are first-citizens in the framework, behaving just like the preset ones (e.g. taking advantage of the distributed computing stuff and other things).

  Of course nothing prevents us from adding some ad hoc methods for the most common operations as in your example, that are just shortcuts over the command pattern (but let's focus on the path we've followed: first requirements, then design, up to the implementation. It's the right way to measure a system, rather than starting right from the code).

  It's also a matter of taste: an OO purist would probably make a bad face ouf of those ad-hoc methods. ;-) For this reason I'm saying that it makes perfectly sense to have alternate frameworks, with different approaches, and one should choose the most liked. And they should be hardly part of the JRE; while it's important that the JRE + extensions such as JAI provide affordable basic blocks.

  Posted by: fabriziogiudici on May 22, 2007 at 01:47 PM

• We often tend to overcomplicated our APIs, but you have to be careful, because you may sacrifice flexibility.

  In your EditableImage example. Look at your "writeJpeg" method. Do you also have methods for writePNG, writeBMP, writeGIF, etc? What happens when there's a new image format?

  In the Python example, they have a thumbnail operation. Do you have to have a method in the Image object for all operations? Or do you pass operations to the image like in the other examples?

  When you oversimplify an API you sometimes make it more complicated too, because you might end up with thousands of simple little methods to cover all cases.

  It's a though balance to strike, of course.

  Posted by: augusto on May 22, 2007 at 02:06 PM

• we are living this era where less code is supposed to be less complex.. :)) this is funny.. it is the same to assume translation this comment in 3 Chinese characters would be more simple for me and you :))

  The most simple language of the universe is the one you already mastered. That is for me the only truth.
  

  Posted by: felipegaucho on May 23, 2007 at 04:05 AM

• i perceived that the ruby on rails campaign against java "complexity" (http://www.youtube.com/watch?v=PQbuyKUaKFo this funny video is a testimonial) is spreading most of what i can imagine... It's possible that now also Python would position itself using a similar approach?

  Posted by: adragoni on May 23, 2007 at 07:47 AM

• I had just begun work that required a rescale for a thumbnail. Paul was correct, finding the setQuality method was a little work.

  Without his discussion, I would not have been aware of Mistral.

  Thanks to both Paul and to you.
  

  Posted by: pwc on May 23, 2007 at 09:29 AM

• Read this too: : http://today.java.net/pub/a/today/2007/04/03/perils-of-image-getscaledinstance.html

  Posted by: fabriziogiudici on May 23, 2007 at 09:31 AM

• Let's be honest, yes?

  The Java SE image APIs are horrible. They aren't even low-level APIs, they are products of some very diseased minds.

  A low-level API would make low-level operations simple. A very simple, low-level image operation would be to wrap a simple 2D array of integers (containing ARGB values) or a 2D array of bytes (containing color indices) in an image object. With wrapping I mean to avoid copying, and instead reuse the existing 2D array as the storage for the image data. The 2D arrays are supposed to be arranged in a simple way, no holes, no gaps, simple row/column organisation etc. Just what someone would allocate and fill when e.g. calculating some image contents with some more or less complex algorithm or reading the data from some device.

  Welcome to the wonderful land of DataBuffer subclasses, SampleModel subclasses, WritableRaster, transfer types (which aren't types at all), ColorModel subclasses, and BufferedImage. Hello?

  A good API makes simple things simple and complex things possible.The Java 2D image API fails miserable on both accounts. Doing simple things is horribly complex and doing complex things is impossible, because one just can't figure out how all the things are supposed to work together.

  And to add insult to injury, Sun just couldn't be bothered to document things properly. No, as long as the developers of that API pretend to understood things it was fine. Has Sun ever heard of something called quality control and documentation rules?

  Oh, and Sun just didn't do it once. Oh no. First the AWT image API, based on the horrible concept of producers and consumers, then the Java 2D image API based on an over-abstraction and nested data structures, and then JAI.

  They never learn, they just never learn.

  Posted by: ewin on May 23, 2007 at 03:09 PM

• Well, JAI has a complex but coherent API and it allows to do a lot of complex stuff. With "complex" I mean that creating the set of parameters for each operation requires some lines of code. Given this, there's an extremely rich set of primitives - if one knows well imaging, it shouldn't be a problem to put them together (well, I'm not a deep expert of imaging, but the guy that put up the first test case for Mistral - something moderately complex, involving DFTs, IDFTs, computing cross-correlations etc - made things working with JAI in a very short time (and Java is not his first language). Sure, he screwed out loud in the first days, but once the basic mechanism was understood he went on, maybe with the support of the mailing list.

  Given this, since JAI targets manipulation of very large images by means of tiling, I see Rasters and DataBuffers unavoidable: you need them to implement a transparent tiling engine that would be impossible to do with a plain array.

  Maybe these things could have been pushed out Java2D and used only by JAI, keeping Java2D simpler. But - again, I'm not expert, maybe some Java2D guy will be commenting on this - I think that again SampleModels are important here. The point is that different pixel layouts are important for speed, for instance to have the hardware blitter working. For instance, you load an image with SampleModel #1, but would like to convert it into a VolatileImage which has a different SampleModel.

  Indeed, my point about the imaging APIs are two: bugs, some of them have been there for years, mainly related to performance; and the consistent lack of documentation. Sun didn't pay attention enough to imaging programmers until recently, when Romain, Chet and others started publishing some code examples.

  Nevertheless, I think we need a comprehensive community where to share information, covering anything about Java imaging, starting from Java2D, up to JAI and others. I'll be back very soon with a concrete proposal.

  Posted by: fabriziogiudici on May 23, 2007 at 03:34 PM

• Most people do not work with very large images. Most program's largest images are 48x48 or 64x64 - large icons. Or people draw some 320x200 graph, but nothing complex or large. So again, why are simple things not simple?

  I know why SampleModel is there - to describe non-trivial, as well as trivial image data organizations, and I have used it for satellite image data. I am not complaining about the need to have such a thing - when dealing with complex data layouts. I am complaining about the fact that I am exposed to it and the whole shebang of WritableRaster, ColorModel, DataBuffer and whatnot when dealing with some very trivial image data layout. The kind of layout every programmer would naturally chose when having to calculate a simple image.

  Why are there no simple BufferedImage(int [][] data) and BufferedImage(byte[][] data, int[] colorMap) constructors which take care about all the crazy details?

  Of course these simple constructors should prescribe a particular data layout, the simple layout people would start with, ARGB or 8bit indexed.

  
  int[][] imageData = new int[64][64];
  for(int y = 0; y < 64; y++) {
  for(int x = 0; x < 64; x++) {
  imageData[x][y] = some_funy_way_to_calculate_a_pixel_in_ARGB();
  }
  }
  BufferedImage image = new BufferedImage(imageData);

  But that would be to simple for Sun. Instead they give us BufferedImage(ColorModel cm, WritableRaster raster, boolean isRasterPremultiplied, Hashtable properties)

  as the only way to create a BufferedImage backed up by preexisting data.
  Thank you very much.

  I really don't know where the programmers who did this learned their trade. There is definitely room for improvement when it comes to their API design skills.

  Posted by: ewin on May 24, 2007 at 03:39 PM

This work is licensed under a Creative Commons License.