Skip to main content

Using Java compiler in your Web Start application

Posted by kirillcool on May 27, 2005 at 5:56 AM PDT

In JAXB Workshop project, I was faced with the following problem: after the XJC generator produces a set of Java source files, I needed to compile them and load them into the running JVM in order to collect cross-reference information:






The standard technique for compiling Java source files in regular standalone application is to use the tools.jar that resides under jdk/lib directory, and use com.sun.tools.javac.Main class and its compile function to do the job. Another, more sophisticated technique that doesn't require explicitly specifying the location of the tools.jar is to use java.home system property and deduce the location of tools.jar from it:

private static Method javac = null;

static {
   try {
      File jreHome = new File(System.getProperty("java.home"));
      System.out.println("java.home = " + jreHome.getAbsolutePath());
      File toolsJar = new File(jreHome.getParent(), "lib/tools.jar");
      System.out.println("tools.jar = " + toolsJar.getAbsolutePath());
      ClassLoader toolsJarLoader = new URLClassLoader(
            new URL[] { toolsJar.toURL() });
      javac = toolsJarLoader.loadClass("com.sun.tools.javac.Main")
            .getMethod("compile", new Class[] { String[].class, PrintWriter.class })
;
      hasToolsJar = true;
   } catch (Exception exc) {
      javac = null;
      hasToolsJar = false;
   }
}  

In the code above lies our first problem when the code will be executed in Web Start environment - no JDK installation is needed for such environment. Moreover, all default installations of JDK 6.0 early builds (that install JRE also), point the javaws.exe to work with the JRE and not the JDK. In this case, the value of java.home will point to the JRE installation directory, that doesn't have tools.jar under lib.

At this point it becomes obvious that the Java compiler must be bundled with your application. And now the bad news - the size of the tools.jar is staggering 6,590 KB! The reason - it contains a lot of tools, such as apt, javah and so on. A simple solution would be to leave only the classes necessary for javac, resulting in about 1.5 MB, but that would not sit good with the Java license, especially for commercial applications.

Currently, the only fully-TCK compliant alternative (for Java 5.0 features) is the JDT compiler that comes with Eclipse 3.1M7. The relevant jar is org.eclipse.jdt.core_3.1.0.jar and its size is 3,430 KB, almost half that of tools.jar. The list of all available compiler options is available here.

At this point, it appeared that all was left to do - replace the package name and the function parameters. However, the problem that laid ahead was unexpected - missing classes in the classpath. The files generated by JAXB's XJC generator, reference classes in jsr173_api.jar and jaxb-api.jar (both parts of JAXB 2.0 Early Access Reference Implementation). These files are, of course, properly referenced in the JNLP descriptor (along with the JDT compiler jar):

<resources>
  <j2se version="1.5+" />
  ...
  <jar href="jaxb-api.jar"/>
  <jar href="jsr173_api.jar"/>
  <jar href="org.eclipse.jdt.core_3.1.0.jar"/>
  ...
</resources>

Nonetheless, the compiler was failing with error messages indicating that the corresponding Java classes couldn't be found in the classpath. A quick look at the compiler parameters show that we can specify -cp followed by the list of jar files. The first attempt would be to simply provide the following value:

-cp jaxb-api.jar;jar173_api.jar

This approach fails, because the Web Start cache (even if you manage to find its location on the remote computer) stores the file under very cryptic names. For example, on my machine the cache is under C:\Documents and Settings\Owner\Application Data\Sun\Java\Deployment\cache\6.0, and sample file name is 3595ab4e-24c448c4.

The second approach attempts to retrieve the actual URL of these two jars in the current JVM. Here we are once again faced with a problem specific to Web Start - the remote jar files do not appear in the java.class.path system property. The only way to retrieve them is via manifest files.

A manifest file is, in fact, the most prominent component that makes jar file different from zip file. It is located under META-INF directory, and its name is MANIFEST.MF. So, how can we obtain a list of all available jars? Very simple - ask the class loader to enumerate all META-INF/MANIFEST.MF resources:

try {
   for (Enumeration<?> e = org.jvnet.jaxbw.Workspace.class
         .getClassLoader().getResources("META-INF/MANIFEST.MF");
         e.hasMoreElements();) {
      URL url = (URL) e.nextElement();
      _logger.info("Scanning " + url.getFile());
      ...
} catch (IOException exc) {
   exc.printStackTrace();
}

Sample logger output from the above code in Web Start environment is

INFO: Scanning https://jaxb-workshop.dev.java.net/webstart/jaxb-api.jar!/META-INF/MANIFEST.MF

From the above URL, we can obtain the URL of the jar itself (marked in red), and then give it to the compiler in the classpath:

-cp https://jaxb-workshop.dev.java.net/webstart/jaxb-api.jar;https://jaxb-workshop.dev.java.net/webstart/jar173_api.jar

However, the compiler still complains about the missing files. Obviously, the classpath should reference local files only. It becomes clear now that we must have local copies of the remote jars and provide their (local) filenames to the compiler. One way to do this is: download the relevant jar files based on their URLs and save them as temporary files. The files should be marked to be deleted on application exit. The code now becomes:

   private static synchronized String getLocalClassPath(Logger _logger) {
      if (isJarsLoadedLocal) {
         return localJarsClassPath;
      }

      Set classpathJars = new HashSet();

      try {
         for (Enumeration<?> e = org.jvnet.jaxbw.Workspace.class
               .getClassLoader().getResources("META-INF/MANIFEST.MF"); e
               .hasMoreElements();) {
            URL url = (URL) e.nextElement();
            _logger.info("Scanning " + url.getFile());
            if (url.getFile().contains("jaxb-api.jar")) {
               String jaxbApiLocal = getLocalJarFilename(url, _logger);
               if (jaxbApiLocal != null) {
                  classpathJars.add(jaxbApiLocal);
               }
            }
            if (url.getFile().contains("jsr173_api.jar")) {
               String jsr173ApiLocal = getLocalJarFilename(url, _logger);
               if (jsr173ApiLocal != null) {
                  classpathJars.add(jsr173ApiLocal);
               }
            }
         }
      } catch (IOException exc) {
         exc.printStackTrace();
      }

      String classpath = "";
      if (classpathJars.size() > 0) {
         classpath = " -cp "";
         String separator = "";
         for (String localJarName : classpathJars) {
            classpath += (separator + localJarName);
            separator = File.pathSeparator;
         }
         classpath += """;
      }

      isJarsLoadedLocal = true;
      localJarsClassPath = classpath;

      return localJarsClassPath;
   }

Here, we use a static synchronized function. This function checks whether the classpath was already computed, and if so - returns the classpah immediately. Otherwise - it enumerates all available jars, and calls getLocalJarFilename function that creates a local temporary copy of the parameter jar URL (function shown below). Then, after all the jars that we were interested in have been downloaded, we create the final classpath (using File.pathSeparator). The code for getLocalJarFilename is:

   private static String getLocalJarFilename(URL remoteManifestFileName,
         Logger _logger) {
      _logger.info("Trying to create local version of "
            + remoteManifestFileName);
      // remove trailing
      String urlStrManifest = remoteManifestFileName.getFile();
      String urlStrJar = urlStrManifest.substring(0, urlStrManifest.length()
            - MANIFEST.length() - 2);
      _logger.info("Remote : " + urlStrJar);
      InputStream inputStreamJar = null;
      File tempJar;
      FileOutputStream fosJar = null;
      try {
         URL urlJar = new URL(urlStrJar);
         if (urlJar == null) {
            _logger
                  .info("No remote version (probably not running in JNLP)");
            return null;
         }

         inputStreamJar = urlJar.openStream();
         String strippedName = urlStrJar;
         int dotIndex = strippedName.lastIndexOf('.');
         if (dotIndex >= 0) {
            strippedName = strippedName.substring(0, dotIndex);
            strippedName = strippedName.replace("/", File.separator);
            strippedName = strippedName.replace("\", File.separator);
            int slashIndex = strippedName.lastIndexOf(File.separator);
            if (slashIndex >= 0) {
               strippedName = strippedName.substring(slashIndex + 1);
            }
         }
         tempJar = File.createTempFile(strippedName, ".jar");
         _logger.info("Created temporary file '" + tempJar.getAbsolutePath()
               + "'");
         tempJar.deleteOnExit();
         fosJar = new FileOutputStream(tempJar);
         byte[] ba = new byte[1024];
         int bytesWritten = 0;
         while (true) {
            int bytesRead = inputStreamJar.read(ba);
            if (bytesRead < 0) {
               break;
            }
            fosJar.write(ba, 0, bytesRead);
            bytesWritten += bytesRead;
         }
         _logger.info("Written " + bytesWritten + " bytes to local file");
         return tempJar.getAbsolutePath();
      } catch (Exception ioe) {
         System.out.println(ioe.getMessage());
         ioe.printStackTrace();
      } finally {
         try {
            if (inputStreamJar != null) {
               inputStreamJar.close();
            }
         } catch (IOException ioe) {
         }
         try {
            if (fosJar != null) {
               fosJar.close();
            }
         } catch (IOException ioe) {
         }
      }
      return null;
   }

Important points - we create a temporary file using File.createTempFile (no need to guess the remote computer OS etc.), mark it with deleteOnExit() (can then reuse it as long as the application is running) and finally return its absolute path.

The code for compiling a set of Java source files now becomes:

ByteArrayOutputStream baosMessages = new ByteArrayOutputStream();
ByteArrayOutputStream baosWarnings = new ByteArrayOutputStream();
long time0 = System.currentTimeMillis();
StringBuffer bigAllNames = new StringBuffer();
for (SingleClassInfo currClassInfo : allClasses) {
   String currFilename = currClassInfo.getRootDirName()
+ File.separator + currClassInfo.getRelJavaFilename();
   this._logger.info("Adding " + currFilename + " to compilation");
   bigAllNames.append(""" + currFilename + "" ");
}
String warningSetting = new String("allDeprecation,"
      + "allJavadoc," + "assertIdentifier," + "charConcat,"
      + "conditionAssign," + "constructorName," + "deprecation,"
      + "emptyBlock," + "fieldHiding," + "finalBound,"
      + "finally," + "indirectStatic," + "intfNonInherited,"
      + "javadoc," + "localHiding," + "maskedCatchBlocks,"
      + "noEffectAssign," + "pkgDefaultMethod," + "serial,"
      + "semicolon," + "specialParamHiding," + "staticReceiver,"
      + "syntheticAccess," + "unqualifiedField,"
      + "unnecessaryElse," + "uselessTypeCheck," + "unsafe,"
      + "unusedArgument," + "unusedImport," + "unusedLocal,"
      + "unusedPrivate," + "unusedThrown");


// Create classpath. For compiling XJC-generated files we need two
// jar files, jaxb-api.jar and jsr173_api.jar. When Milano is run
// from the local computer, these jars should be on classpath
// (explicitly or on PATH varibable). However, when our application
// runs from Java Web Start, these jars are not available by their
// names. Here, we will try to fetch them as resources and create
// temporary files on the local machine.
String classpath = Workspace.getLocalClassPath(this._logger);
this._logger.info("Classpath is " + classpath);

// Start compilation using jdtcore
boolean compilationStatus = org.eclipse.jdt.internal.compiler.batch.Main
      .compile("-1.5 -source 1.5 -warn:" + warningSetting + " "
   + classpath + " " + bigAllNames.toString(),
   new PrintWriter(baosMessages), new PrintWriter(
baosWarnings));


// Parse the messages and the warnings.
long time1 = System.currentTimeMillis();
BufferedReader br = new BufferedReader(new InputStreamReader(
      new ByteArrayInputStream(baosMessages.toByteArray())));
while (true) {
   String str = br.readLine();
   if (str == null) {
      break;
   }
   this._logger.info("/javac/ " + str);
   compilerMessages.addLast(str);
}
BufferedReader brWarnings = new BufferedReader(
      new InputStreamReader(new ByteArrayInputStream(baosWarnings
   .toByteArray())));
LinkedList<String> warnings = new LinkedList<String>();
while (true) {
   String str = brWarnings.readLine();
   if (str == null) {
      break;
   }
   this._logger.warning("/javac/ " + str);
   compilerMessages.addLast(str);
   warnings.addLast(str);
}
this._logger.info((time1 - time0) + " milliseconds, "
      + (compilationStatus ? "success" : "failure"));
return compilationStatus;
Related Topics >>