Kohsuke Kawaguchi's Blog

Compiling with JDK6 and running on JDK5

It's common for a Java project to compile with later versions of JDK than it minimally requires. For example, when Hudson runs on Java6 it takes advantages of those features, but it can also run on Java5 without those advanced features.

The technique to do this is well understood. Here's one such code fragment taken from Hudson:

try {
    for (ThreadInfo ti : Functions.getThreadInfos())
        r.put(ti.getThreadName(),Functions.dumpThreadInfo(ti));
} catch (LinkageError _) {
    // not in JDK6. fall back to JDK5
    ...
}

This is desirable, since you can take advantages of the latest JavaSE features without forcing users to upgrade.

The problem is that you now need to compile with JDK6, so when other parts of code accidentally depends on new additions in Java6 and breaks the minimum Java5 requirement, your build process won't complain. If you are lucky, your tests will catch it, but no test attain 100% of code coverage, so there's still a good chance that such a problem slips into the production code. (For example, I've been bitten a few times by using IOException(String,Throwable) constructor that was added in 1.6, since I typically use it only in handling errors that tend not to be tested well.)

Hudson had a fair share of those incidents, and it just happened one too many times for me.

So I decided to bite the bullet and write a tool for it.

The idea of the tool is simple. I first run a "signature builder" with JDK5, capturing all the method and field signatures from a JRE.

I then wrote a separate tool called "signature checker", which uses this signature file and inspect your classes. If your classes depend on things that don't exist in the signature list, you get an error message. This tool is packaged up as a Maven plugin, so to use this, you just add the following snippet inside your <build> element in pom.xml:

<plugin>
  <!-- make sure our code doesn't have 1.6 dependencies except where we know it -->
  <groupId>org.jvnet</groupId>
  <artifactId>animal-sniffer</artifactId>
  <version>1.2</version>
  <executions>
    <execution>
      <goals>
        <goal>check</goal>
      </goals>
      <configuration>
        <signature>
          <groupId>org.jvnet.animal-sniffer</groupId>
          <artifactId>java1.5</artifactId>
          <version>1.0</version>
        </signature>
      </configuration>
    </execution>
  </executions>
</plugin>

The nested <signature> element specifies the signature list to use. In addition to java1.5, I've got java1.3, java1.4, and java1.6 available.

If you don't want to do this for every Maven build, you can instead have the following snippet:

<plugin>
  <!-- make sure our code doesn't have 1.6 dependencies except where we know it -->
  <groupId>org.jvnet</groupId>
  <artifactId>animal-sniffer</artifactId>
  <version>1.2</version>
  <configuration>
    <signature>
      <groupId>org.jvnet.animal-sniffer</groupId>
      <artifactId>java1.5</artifactId>
      <version>1.0</version>
    </signature>
  </configuration>
</plugin>

And then you can run mvn compile animal-sniffer:check to check the dependency, or you can further add the following POM snippet so that the check is performed automatically during a release:

<plugin>
  <artifactId>maven-release-plugin</artifactId>
  <configuration>
    <goals>install animal-sniffer:check deploy</goals>
  </configuration>
</plugin>

The tool uses ASM and statically analyze the code, so it doesn't miss any reference, unlike test based approach.

Now, in the places where you knowingly use features that go beyond the minimum requirement, you put the @IgnoreJRERequirement annotation on a method. This is a signal from you to the checker that you're aware of the dependency there and you know what you are doing.

For this code to compile, you need to add animal-sniffer.jar to the dependency list. This annotation is configured as @Retention(CLASS), so you don't need this jar to be at runtime. To tell Maven not to put it in the runtime, this fragment includes <optional>true</optional>:

<dependency>
  <!-- for JRE requirement check annotation -->
  <groupId>org.jvnet</groupId>
  <artifactId>animal-sniffer-annotation</artifactId>
  <version>1.0</version>
  <optional>true</optional>
</dependency>

There are certain edge cases that this tool doesn't handle correctly (like the case when a visibility of a method changes from 'protected' to 'public' between Java5 to Java6 — not that I know such a case exists), but I think it runs pretty well, at least on Hudson, and the added peace of mind is priceless.

As usual, I'm always looking for more people to work on any of my projects, so if you are interested, please send me an e-mail, and you'll be a committer right away.

Finally, the reason the tool is called animal-sniffer is because JavaSE code names are traditionally named after animals, like Mantis, Tiger, Mustang, Dolphin, and so on.

Comments

• The technique introduced in the first part of this blog looks like it might suffer from the same problem as this:
  http://blogs.sun.com/ritzmann/entry/is_catching_noclassdeffounderror_more_efficient

  The problem is that the JVM is giving no guarantee when classes are linked and it may happen well in advance before your try/catch block gets a chance to be executed.

  Posted by: ritzmann on November 15, 2008 at 12:32 AM

• Ergh.

  Back when I was doing multi JDV version support, I found it easier to have different source roots for code that used later versions of Java. Just a small interface layer. Have that implement an interface from the main source base, so that only a single getConstructor-newInstance was necessary. Then build with the appropriate -source -target -bootclasspath.

  The worst example of this sort of bug I've seen is an extra StringBuffer.append added in 1.5. That doesn't require a change in source to make it incopmatible. -bootclasspath ftw.

  Posted by: tackline on November 15, 2008 at 02:36 AM

• 
  Fabian — thanks for the pointer to your blog. I'll experiment this with Apple JVM and see what happens. I knew that JVM spec wouldn't require such lazy loading behavior (and GCJ is one such example I suppose), but I thought in practice all the modern JVMs (Sun's, BEA's and IBM's) do the lazy loading, and so far reports from Hudson users seem to indicate that that's the case.

  
  It would be really unfortunate if we have to go back to reflection just because of apple JVM...

  Posted by: kohsuke on November 15, 2008 at 10:55 AM

• 
  tackline — I think your approach is sound, but note that using -source and -target doesn't help you when it comes to library dependencies. So you'd still need a tool like this to make sure that you are not depending on new library features introduced in later versions of JDK.

  
  I suppose another downside is that in a plugin enabled system like Hudson, everyone would end up defining their little JDK6 jail class, which is somewhat awkward.

  Posted by: kohsuke on November 15, 2008 at 10:58 AM

• In what maven repo are those artifacts located?

  Posted by: francisdb on November 16, 2008 at 04:40 AM

• These are all in the java.net Maven2 repository. See more details about this repository at maven2-repository.dev.java.net

  Posted by: kohsuke on November 17, 2008 at 08:57 AM

• thanks, great tool

  Posted by: francisdb on November 17, 2008 at 10:08 AM

• FYI: http://retroweaver.sourceforge.net/ will already do this for you.

  Posted by: cowwoc on November 17, 2008 at 01:00 PM

• cowwoc — I thought retroweaver only does bytecode transformation. Does it check for missing features in JRE?

  Posted by: kohsuke on November 17, 2008 at 05:01 PM

• retroweaver also replaces at least some missing features with alternatives.

  Posted by: mthornton on November 18, 2008 at 01:22 AM

• Since you're doing bytecode analysis anyway, why not auto-ignore usages that are in a try {} catch (LinkageError e) block, rather that forcing the annotation? (I know that the try... catch may not always work as pointed out by ritzmann

  Posted by: stephenconnolly on November 18, 2008 at 02:36 AM

• kohsuke - -bootclasspath allows javac from one JDK use the rt.jar of another older version. So if you use JDK6 javac with a JDK5/JRE5 rt.jar, you wont accidentally link to 6-only features.

  Posted by: tackline on November 18, 2008 at 03:15 AM

• The problem is that until Maven provides toolchain support, you don't know where JDK1.5 is hiding on the system.

  Posted by: stephenconnolly on November 18, 2008 at 04:32 AM

• kohsuke, yes it does. I'm not sure how extensively it checks, but it definitely tries to do so.

  Posted by: cowwoc on November 18, 2008 at 05:50 AM

• 
  mthornton — yeah, but it's for certain selected features, like replacing StringBuilder by a StringBuffer. I don't think it's trying to replace all new JDK6 functionalities. For example, replacing System.console() is just plain impossible.

  
  stephenconnolly — Yes, I think it's quite conceivable for the tool to make sure that the fall back works correctly. That's an exercise for a future contributer :-)

  Posted by: kohsuke on November 18, 2008 at 08:20 AM

• cowwoc &madsh; To do this, one needs to have a signature list of JRE, and neither retroweaver Ant task (http://retroweaver.sourceforge.net/ant-task.html) or Maven plugin (http://mojo.codehaus.org/retroweaver-maven-plugin/) has a parameter for such things. The download bundle doesn't contain signature lists either. So at this point I doubt if it does the job.

  
  I've also learned from another person that IntelliJ IDEA can do the same thing (by looking at @since annotation in the source code during editing.) So IntelliJ user can get the same benefit that way.

  Posted by: kohsuke on November 18, 2008 at 08:24 AM

• Great work! And work perfectly with another animal...

  Posted by: jfarcand on November 18, 2008 at 08:13 PM

• Did you ever consider using Buildr to build glassfish? It's a ruby based build tool where the build files are ruby code and just call API provided by Buildr, so all information is reachable without needing mojos and with proper data structures. It works with Maven dependency/repository mechanism, and was just voted to become a top-level apache project
  

  Posted by: ittayd on November 18, 2008 at 08:25 PM

• Interesting tool, indeed.
  We have a library that is used
  - on JDK6 with included JAXB
  - and on JDK5 + external JAXB libs

  When we use your tool, all JAXB API references are listed as error, to much to flag each occurence with the provided @Ignore annotation.
  Perhaps it makes sense to be able to configure the plugin to ignore some packages like javax.xml.bind and javax.activation.
  Thanks,
  Holger

  Posted by: brands on November 19, 2008 at 01:21 AM

• Here is another idea around the same problem. http://www.jroller.com/eu/entry/dealing_with_api_compatibility

  BTW, Eclipse also has API tools that can do even more then just check if method was there. There is also Clirr http://clirr.sourceforge.net/

  Posted by: euxx on November 23, 2008 at 03:45 AM

• ritzmann wrote:
  > The problem is that the JVM is giving no guarantee when classes are linked and it may happen well in advance before your try/catch block gets a chance to be executed.

  Linkage of a class may certainly occur in advance, but the following sources seem to suggest that a JVM must not throw a linkage error before a class is actually used. Therefore I'm not yet convinced that the technique presented in this blog entry is unsound. Maybe it's more an issue of JVM implementations not conforming to the specification.

  From: http://java.sun.com/docs/books/jvms/second_edition/html/Concepts.doc.html#19175
  The Java programming language allows an implementation flexibility as to when linking activities (and, because of recursion, loading) take place, provided that the semantics of the language are respected, that a class or interface is completely verified and prepared before it is initialized, and that errors detected during linkage are thrown at a point in the program where some action is taken by the program that might require linkage to the class or interface involved in the error.

  From: http://www.developer.com/java/other/print.php/2248831
  Implementations typically delay the loading of a type as long as possible. They could potentially, however, load classes much earlier. Class loaders (see below) can opt to load a type early in anticipation of eventual use. If this strategy is chosen, the class loader must not report any problem (by throwing a subclass of java.lang.LinkageError) encountered during loading until the type's first active use. In other words, a type must appear to be loaded only when needed.
  

  Posted by: pniederw on November 28, 2008 at 12:37 PM

• jfarcand — Thanks. And for others reading this comment section, JeanFrancois is referring to Grizzly (https://grizzly.dev.java.net/)

  
  ittayd — As a matter of fact, yes, the word "Buildr" show up in our hallway conversation. Either me or Roberto brought that up, half seriously. The problem is that for the project of the size GlassFish, it's just too risky. We need to "test" it with a smaller project first.

  
  brands — if you have JAXB API and activation jars in your classpath, it shouldn't report errors for them. So my question is, why don't you have them in your classpath? That said, I agree that it's useful to be able to ignore some packages all together.

  
  euxx — thanks for the pointers. Clirr seems to be designed for a different purpose and cannot be used for the purpose of animal sniffer, but it sounds like an useful tool on its own. Regarding your technique of using dynamic proxy, an enhancement to support parameter types that are themselves possibiliy missing would be useful (that is, calling a method with some parameters whose type isn't necessary available in the current JDK.)

  Posted by: kohsuke on December 10, 2008 at 11:38 AM

• Is there any simple way to edit the signature file ?

  I'm working on a Google App Engine maven plugin, and would like to check code for compatibility with the GAE white list (subset of Java6). As your animal-checker allready does the job I'd like to create a GAE sig file, but the binary format is not as simple as I expected to edit.

  Posted by: ndeloof on April 19, 2009 at 01:25 AM

• Hi,

  I'm trying to use animal-sniffer to validate an API, not Java.

  So I downloaded the sources and used the SignatureBuilder to generate the signature for this API that I wanna check compatibility with previous versions.

  But, when I try to check using my signature I got the following error:
  [ERROR] FATAL ERROR
  [INFO] ------------------------------------------------------------------------
  [INFO] 48188
  [INFO] ------------------------------------------------------------------------
  [INFO] Trace
  java.lang.ArrayIndexOutOfBoundsException: 48188
  at org.objectweb.asm.ClassReader.readClass(Unknown Source)
  at org.objectweb.asm.ClassReader.accept(Unknown Source)
  at org.objectweb.asm.ClassReader.accept(Unknown Source)
  at org.jvnet.animal_sniffer.PackageListBuilder.process(PackageListBuilder.java:29)
  at org.jvnet.animal_sniffer.ClassFileVisitor.processJarFile(ClassFileVisitor.java:59)
  at org.jvnet.animal_sniffer.ClassFileVisitor.process(ClassFileVisitor.java:39)
  at org.jvnet.animal_sniffer.maven.CheckSignatureMojo.apply(CheckSignatureMojo.java:128)
  at org.jvnet.animal_sniffer.maven.CheckSignatureMojo.buildPackageList(CheckSignatureMojo.java:120)
  at org.jvnet.animal_sniffer.maven.CheckSignatureMojo.execute(CheckSignatureMojo.java:94)
  at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:483)
  at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:678)
  at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:540)
  at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:519)
  at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:371)
  at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:332)
  at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:181)
  at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:356)
  at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:137)
  at org.apache.maven.cli.MavenCli.main(MavenCli.java:356)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)
  at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
  at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
  at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
  at org.codehaus.classworlds.Launcher.main(Launcher.java:375)

  Any thoughts?

  
  VELO

  Posted by: velobr on June 18, 2009 at 05:05 AM

• My best guess is that your class files are broken. Or are you using some really cutting-edge class file versions? Like the one for JDK7?

  Posted by: kohsuke on June 19, 2009 at 12:46 PM

• The jar is valid... I use it on a regular basis w/o any troubles...

  What I did, I just changed main method on SignatureBuilder to be like this:
  SignatureBuilder builder = new SignatureBuilder( new FileOutputStream( "flex_sdk_3.3.0.4852_signature" ) );
  builder.process( new File( "C:/flex/flex_sdk_3.3.0.4852_mpl/lib/flex-compiler-oem.jar" ) );
  builder.close();

  It did produce this signature file:
  http://rapidshare.com/files/247871352/flex_sdk_3.3.0.4852_signature.html

  But when I try to run animal-sniffer maven plugin I got this error.

  
  VELO

  Posted by: velobr on June 23, 2009 at 01:37 PM

This work is licensed under a Creative Commons License.