Skip to main content

ASM incompatible changes

Posted by kohsuke on February 12, 2010 at 10:56 AM PST

ObjectWeb ASM is a great library that's used to parse Java class files. It's used in all kinds of projects, such as Hibernate, Corba, JAX-WS, Jersey, Spring, Hudson, to name a few.

But I have a pet peeve to this otherwise great library, namely its insistence on small size (which by itself isn't a bad thing), and its consequences.

One of the choices that made to achieve this was to omit the debug information entirely from the class files, including the line number tables. This is unlike every other OSS Java projects. So you can't step through the ASM code from the debugger, your IDE won't offer any assistance while editing, and it generally makes it hard to use.

Another choice they made to achieve the small size is to ignore the backward compatibility. Whenever ASM changes, and it has to change every so often (at least every time a class file format changes), it doesn't do anything to keep the existing applications working. For example, from 2.x to 3.x, the ClassReader.accept method, which parses the class file, has changed in an incompatible way. It used to take ClassVisitor and boolean, which was a flag, but in 3.x it needed to take more flags, so they opted for taking an integer as a bit mask.

It was entirely possible to retain the backward compatibility by defining the following simple one line method, but no, keeping asm.jar smaller was more important for ASM, so they just removed the method instead.

public void accept(ClassVisitor cv, boolean skipDebug) {
  accept(cv, skipDebug?SKIP_DEBUG:0);
}

The end result is that if your application is using one framework that uses one version of ASM underneath, and if you have another library that uses another version of ASM underneath, then there just isn't any way to make them work together — using the latest version of ASM breaks libraries/frameworks/apps that are built with earlier versions. Google can tell you how widespread this problem is.

In the face of this, framework/library developers works around the problem by doing package renaming. So everyone ends up getting their own private copy of ASM. and the result is far from optimal. For example, if I remember correctly, GlassFish now has 4 copies of ASM in it, one used by Corba, one used by EclipseLink, one used by JAX-WS, and another one by Jersey. The CDI implementation might have another one, too. Needless to say, size increase by this far outweigh the minimal size decrease obtained by not keeping the backward compatibility.

I think ASM developers should step back and really think hard if reducing the jar file size by a few 10KBs is really worth this much pain throughout the food chain. More concretely, I humbly propose:

  • ASM ships their jars with full debug info, like everyone else does. If small footprint is important for some users, a minimized jar can be delivered separately.
  • ASM retains backward compatibility in their API for minor releases, and for every API-breaking major release, ASM should deliver an official package renamed jar. This allows me to write a library/framework that depends on ASM 2.x, and my library can peacefully coexist with someone else's library that uses ASM 3.x. Official package renamed jars cut the # of duplicates.
Related Topics >>

Comments

ASM, debug info and backward compat

Hi Kohsuke, I am the smallest contributor of the ASM team and very happy that you use ASM. About debug info, latest release o f ASM contains a jar named asm3.2-debug-all.jar containing all debugging info. About backward compatibility, I think the major problem is that we don't provide a clean story for upgrading from 2.x to 3.x, so each user bang his head on the same wall again and again. I am not very relunctant to use package renaming because there are some hidden costs. By example, for ASM, lot of good tutorials was written using ASM 2.x and are still valid with version 3.x. We simply don't have the resources to upgrade all them on each major release. Yes, ASM is popular and is a good candidate to explain the jar hell. I think that the only real answer that is to wait JSR 294 modules.

Rémi

Thanks for the pointer to

Thanks for the pointer to asm-debug-all. I see it in http://repo1.maven.org/maven2/asm/asm-debug-all/3.2/, so it looks like Maven ready, too.

You are right that the module system in Java would address this, but my point is that ASM is a particularly bad-behaving library. For example, I can't think of any Apache commons projects that makes regular backward incompatible changes like ASM does.

Re Thanks for the pointer to

Those Apache projects don't rely on the visitor pattern as ASM does. Updating an interface is just not backward compatible.

You can come up with something along the line of what JSR 199 does but this require to create an object for each bytecode you parse.

ASM is very powerful because it relies on the visitor pattern but this is also its curse.

Rémi

I think it's useful to

I think it's useful to separate out binary compatibility from source compatibility. I'm mainly interested in the former in this context.

With that said, I think there's a lot of ways to mitigate the binary incompatibility problems (and some of them very easy, like the ClassReader.accept() method I talked), even if we cannot eliminate them completely.

For example, adding a new callback for new byte code is less of a problem --- I can still use a new visitor implementation with older ASM fine, plus it's only going to choke if we actually parse a class file that uses a new instruction. It's also possible to provide a default base implementation of a visitor as a layer to absorb incompatibility (IIRC, JSR-199 did something like that at one point in its early stage.)

 

 

Official repackaged jars would already be a huge help

Remi,

Per Koshuke's suggestion, would it at least be possible for the ASM team to provide for each major (i.e. backwards incompatible with the previous) version of ASM a canonical version of ASM that has been repackaged based on the major version number, say to something like org.objectweb.asm-2.x? Then multiple projects could safely rely on a shared distributed version of asm without requiring each project to package it's own version. Tutorials need not be updated; it would suffice to mention the existence of the alternately packaged jars. If the repackaged jars were additionally distributed with source attachments, it would even be easy to do IDE development against a specific version.

I write this because I think ASM is a wonderful tool, and I would love to see it more usable in projects with multiple dependencies.

I partially disagree

Breaking backwards compatibility for major releases (2.x versus 3.x) is perfectly acceptable in my book so long as it improves the API usability (and I believe in this case it does). Removing debug symbols on the other hand... They should ship at least two versions of the library, one for debugging and another for production. My personal pet peeve is ease-of-use in all its colors whether it comes from API usability or debug symbols. Fix that and I'm happy.

Maybe it's because where I

Maybe it's because where I work, but I take backward compatibility very seriously.

If JavaSE or JavaEE broke a backward compatibility for every major release, I suspect a lot of people would be very unhappy.

I don't think so

If Sun broke backwards compatibility every 5 years I hardly think most people would care (there would be a vocal minority but...). Obviously you'd have to justify every time you broke compatibility (to improve usability, to fix an architectural problem, etc) but in the long run we'd be better off. Can anyone honesty say that most people care that Java 1.7 can run Java 1.1 software? No one in their right mind expects a company to support 15 year-old software. And if some company *does* want it, let them pay for it using support contracts. I am willing to bet most people will not care enough. 5 years 'ought to be enough for anybody :)

Re:

But isn't it sensible to try to keep an interface if it is not that hard to to? I mean if you have to write a one-liner to keep an interface why don't you write it?

Amen!!

I emailed the ASM team at one point asking them to place backwards-incompatible changes in a separate package; the response was basically "We see no problem here, since jar-jar links is so easy to use". It's a shame to see such a wonderful open source project marred by such poor judgement.

+1 - I completely agree.

+1 - I completely agree. They did the same thing to the ClassWriter constructor. CXF has resorted to some reflection stuff to be able to work with both 2.x and 3.x, but that's SUCH a crappy solution. Then again, all the package renaming stuff that all the projects do is also a crappy solution.