Hope these tidbits are helpful to somone.

-kto

JVM TI Agents Article

Just a plug (and additional reference) on my December 2006 article on JVM TI at http://java.sun.com/developer/technicalArticles/J2SE/jvm_ti.

I really had not expected this article to be very popular, but I was assuming that only people writing JVM TI agents would be interested. It appears there is quite a bit of general curiosity on VM agents. Of course the Java Lobby and Sun System News links helped too. :^)

Big thanks to Janice for all her help with this article! It would never have read as well without her help.

Just for reference, there is also a JVMPI to JVM TI conversion article at: http://java.sun.com/developer/technicalArticles/Programming/jvmpitransition/.

-kto

My Ant Adventure (Updated 1/23/2007)

Update for 1/23/2007, just a very short note on windows.

The findbugs target needs to add vmlauncher="false", so the line:

<exec executable="findbugs" failonerror="true">

changes to

<exec executable="findbugs" failonerror="true" vmlauncher="false">

It's not exactly clear why this is necessary, but this allows the findbugs target to work on windows, and also works everywhere else. The findbugs -version exec was also removed because this will trigger the GUI to start up if it runs findbugs.bat, and findbugs.bat doesn't seem to have a -version option.

In addition, the ant 1.7.0 install bits seem to have a windows bin/ant.cmd file that does not have execute permissions, by adding execute permissions to this file, ant can be run from a Makefile. Again, it's not exactly clear why this is needed, but it makes sense for it to have execute permissions, so this seems harmless.

Updated 1/22/2007, tried to mark changes in bold underline. Also added the actual build.xml to download.

Now I have never liked ants (see my ants blog), but this story is about my adventure with the Apache Ant build system. I can safely say that I still hate ants, all ants, but even ants have a place in the world, ant build scripts included.

So I was crawling along with my little NetBeans Java project just happy as a clam using NetBeans 5.5 with the findbugs plugin, and the JUnit tests that NetBeans help me to create and run so easily. I never had to write an Ant script before, I just pointed NetBeans at my sources, set a few properties and let NetBeans handle it. Overall it was pretty easy, and simple.

But when it came time for a batch product build, I had used a Makefile and a separate build process. So the romantic picnic at the park was over, it was time for the ants to take over. :^) The Makefile worked, but wasn't ideal, and it was always a temporary thing, but it did create the bin scripts and run some bin script tests that I hadn't managed to get the NetBeans building to do.

First off, it's never good to have two different ways to build a product, you never know for sure if the two build mechanisms really built the same thing. So in general, when it comes to a build process, everyone should follow the Highlander rule of "There can be only one". Granted, a "development build" will always be slightly different, probably a subset, but it should be using the same basic mechanisms as a more formal and complete product build.

Second, I was missing some of the things I wanted built into the batch build process, namely running findbugs and running all my JUnit tests. So I wasn't happy with my existing batch build process.

Ideally I want a batch build process to:

1. compile with full javac error checking and treat all errors as fatal (javac -Xlint:all -Werror)
2. construct the 'dist' area (the jar file and including any necessary bin scripts)
3. run all the JUnit tests
4. run the bin script tests
5. run javadoc and treat all errors as fatal
6. run findbugs with full checking and treat all errors as fatal

Lucky for me I've got a very clean set of source files, and I want to keep them that way, otherwise I'd have to back off on some of my "treat all errors as fatal" requirements. ;^) The first two items above represent the typical development build or "full build" while inside NetBeans, but it was important to me that all the steps should be manually runnable inside NetBeans too.

So I started my adventure to make this happen.

My first thought was about having the Makefile run findbugs and JUnit directly, but that seemed wrong and didn't solve my first problem above. So the answer was fairly obvious, if I needed a Makefile, it would be trivial to have it run the ant script, right? So I just needed to create an ant script. Of course it's never that easy. I did manage to get this to work, and I did learn enough about ant to create this standalone ant script, one that did everything I wanted, but it was NOT simple and easy, and the end result was not very satisfactory.

It turns out that when you create a NetBeans project with an existing ant script, some of the NetBeans features (like debugging a JUnit test) will not work. :^( So some NetBeans features (or what I perceive to be features) are tied to the specific ant build scripts that NetBeans creates.

Then there were the ant problems, which involved having a version of ant that would work with the latest findbugs ant task and the junit ant task. I finally settled on using ant 1.7 but since I allow for my project to build anywhere, I needed ant 1.7 everywhere, and that meant I had to manage my own version of ant (I could not trust all systems to have an ant that would work for me). I think the ant inside NetBeans 5.5 was 1.6 something, and it didn't work with the findbugs ant task for some reason, but that was ok because while inside NetBeans you really want to use the findbugs plugin for the best interaction with your sources. So I finally gave up on the findbugs ant task and just used:

   <condition property="findbugs.home" value="/Applications/findbugs">
        <os family="mac"/>
    </condition>
    
    <target name="findbugs-batch" depends="init"
            description="Run findbugs in batch mode">
        <exec 
            executable="${findbugs.home}/bin/findbugs"
            failonerror="true">
            <arg value="-textui"/>
            <arg value="-effort:max"/>
            <arg value="-low"/>
            <arg path="${dist.jar}"/>
        </exec>
    </target>

An UPDATE on this. Turns out that Windows is giving me no end of grief regarding pathnames, so this was changed to assume findbugs was in the PATH setting, overall this seems to make life easier. In general avoiding having ant deal with full paths is ideal, that also means avoiding the ant property ${basedir} which WILL be a full path. The Makefile changed too, see below. I want this ant script to work on MacOSX, Solaris, Linux, and Windows, others may not have this requirement. I also wanted it to work no matter where the source was moved to, so again, avoid fullpaths. I also added the findbugs -exitcode option so that errors trigger a non-zero process exit code and also the -maxHeap 512 option to give findbugs lots of heap space (it seems to need it, and runs a bit faster this way). So now I use something more like:

    <target name="myfindbugs" depends="init"
            description="Run findbugs in batch mode">
        <exec 
            executable="findbugs"
            failonerror="true">
            <arg value="-maxHeap"/>
            <arg value="512"/>
            <arg value="-textui"/>
            <arg value="-effort:max"/>
            <arg value="-low"/>
            <arg value="-exitcode"/>
            <arg path="${dist.jar}"/>
        </exec>
    </target>

As it turns out, because using the findbugs ant task causes the ant VM to run out of memory, so you had to restart ant with more memory, what a pain. Using my above batch target the findbugs process uses it's own VM. This really doesn't change the performance much since findbugs takes a few minutes to run anyway, and just using the bin findbugs script must set the max memory up higher or something.

So now I had to deal with this loss of NetBeans Junit debug capability. I did like how NetBeans automatically created the menu for the targets in my build.xml file, but I could not live without the ability to quickly debug any JUnit test. So I needed to either configure my build.xml file better, or try another approach.

So I went back to letting NetBeans create it's own ant scripts and when I found where NetBeans placed these files (build.xml plus the nbproject directory), I just copied over all the nbproject directory and the build.xml file to my project's Mercurial repository and reopened the repository as a pre-configured NetBeans project. That way I didn't have to worry about matching the NetBeans ant conventions to get the JUnit debug feature to work.

An update on this, first, do NOT include the nbproject/private directory in your repository. And second, if you tell NetBeans to use anything other than the default Java platform, things don't work very well. This was easy for me to avoid, but the way the Java platforms are defined in the ant scripts didn't make the files transportable. Maybe this was a Mac specific thing?

Then I edited the NetBeans generated build.xml file (which is now my primary product build.xml file) and added to it.

• Some properties and hooks into the NetBeans ant scripts I needed to set for some reason. Some of these properties should not need to be set since they are set for you already in the nbproject files, so I'm puzzled why you have to set property values sometimes and not others. You will also notice the echo targets I have created, which is just my style, I like to know when targets are being run, and the ant verbose option is too verbose, so I added appropriate echo commands.
  

  
      <!-- project name (determines jar and script name) -->
      <property name="project.name"           value="jprt"/>
      
      <!-- top level package name -->
      <property name="package.name"           value="jprt"/>
      
      <!-- top level paths -->
      <property name="src.dir"                value="src"/>
      <property name="test.src.dir"           value="test"/>
      <property name="lib.dir"                value="lib"/>
      
      <!-- source paths -->
      <property name="bin.src"                value="${src.dir}/bin"/>
      <property name="sbin.src"               value="${src.dir}/sbin"/>
      <property name="doc.files.src"          value="${src.dir}/${package.name}/doc-files"/>
      
      <!-- build paths -->
      <property name="build.dir"              value="build"/>
      <property name="manifest.file"          value="${build.dir}/mainfest.mf"/>
      <property name="build.classes.dir"      value="${build.dir}/classes"/>
      <property name="build.test.classes.dir" value="${build.dir}/test/classes"/>
      
      <!-- dist paths -->
      <property name="dist.dir"               value="dist"/>
      <property name="bin.dir"                value="${dist.dir}/bin"/>
      <property name="sbin.dir"               value="${dist.dir}/sbin"/>
      <property name="dist.jar"               value="${dist.dir}/${project.name}.jar"/>
      
      <!-- path to test scripts  -->
      <property name="test.script"            value="${test.src.dir}/test.sh"/>
      <property name="test.system.script"     value="${test.src.dir}/test_system.sh"/>
      
      <!-- javadoc paths -->
      <property name="dist.javadoc.dir"       value="${dist.dir}/javadoc"/>
      <property name="doc.files"              value="${dist.javadoc.dir}/${package.name}/doc-files"/>
      
      <!-- classpath settings -->
      <property name="javac.classpath"        value=""/>
      <property name="run.classpath"          value="${build.classes.dir}"/>
      <property name="javac.test.classpath"   value="${build.classes.dir}:${lib.dir}/junit.jar"/>
      <property name="run.test.classpath"     value="${build.test.classes.dir}:${javac.test.classpath}"/>
      
      <!-- default system options -->
      <condition property="system.instance" value="testsystem-ant">
          <not> <isset property="system.instance"/> </not>
      </condition>
      
      <!-- always provide debugging information -->
      <property name="javac.debug"            value="true"/>
      
      <!-- the main class for the project -->
      <property name="main.class"             value="${package.name}.tools.Main"/>
      
      <!-- make sure netbeans knows we have a manifest file -->
      <property name="manifest.available"     value="true"/>
  
      <!-- hooks into netbeans ant files -->
      <target name="-pre-compile"             depends="mycompilestart"/>
      <target name="-post-compile"            depends="mycompileend"/>
      <target name="-do-jar-with-manifest"    depends="mymanifest"/>
      <target name="-pre-jar"                 depends="myjarstart,mymanifest"/>
      <target name="-post-jar"                depends="myjarend,mybinfiles,mysbinfiles,mypropfiles,myjartests"/>
      <target name="javadoc"                  depends="myjavadoc"/>
  
  

• The special targets of my own:
  

  
      <!-- just used to add messages at certain points -->
      <target name="mycompilestart"> <echo message="COMPILING"/>           </target>
      <target name="mycompileend">   <echo message="COMPILING COMPLETED"/> </target>
      <target name="myjarstart">     <echo message="BUILDING JAR"/>        </target>
      <target name="myjarend">       <echo message="JAR COMPLETED"/>       </target>
      
      <!-- create my own manifest file -->
      <target name="mymanifest" description="Create manifest file">
          <echo message="Creating manifest file: ${manifest.file}"/>
          <manifest file="${manifest.file}">
              <attribute name="Built-By"   value="${user.name}"/>
              <attribute name="Main-Class" value="${main.class}"/>
          </manifest>
      </target>
      
      <!-- copy the bin files -->
      <target name="mybinfiles" description="Create bin files">
          <echo message="Populating bin files"/>
          <mkdir dir="${bin.dir}"/>
          <copy todir="${bin.dir}">
              <fileset dir="${bin.src}" casesensitive="yes">
                  <include name="*"/>
              </fileset>
          </copy>
          <chmod dir="${bin.dir}" perm="ugo+rx" includes="*"/>
      </target>
      
      <!-- copy the sbin files -->
      <target name="mysbinfiles" description="Create sbin files">
          <echo message="Populating sbin files"/>
          <mkdir dir="${sbin.dir}"/>
          <copy todir="${sbin.dir}">
              <fileset dir="${sbin.src}" casesensitive="yes">
                  <include name="*"/>
              </fileset>
          </copy>
          <chmod dir="${sbin.dir}" perm="ugo+rx" includes="*"/>
      </target>
      
      <!-- copy the property files into the sbin directory -->
      <target name="mypropfiles" description="Create sbin property files">
          <echo message="Populating ${dist.dir} with prop files"/>
          <mkdir dir="${dist.dir}"/>
          <copy todir="${dist.dir}">
              <fileset dir="${src.dir}" casesensitive="yes">
                  <include name="*.properties"/>
              </fileset>
          </copy>
          <echo message="System instance: ${system.instance}"/>
          <!-- Ant echo is Broken: <echo file="${dist.dir}/config-default.properties"
                message="jprt.system.instance=${system.instance}"/> -->
          <exec executable="echo" failonerror="true"
                output="${dist.dir}/config-default.properties">
              <arg value="jprt.system.instance=${system.instance}"/>
          </exec>
      </target>
      
      <!-- test the jar file -->
      <target name="myjartests" description="Test jar.">
          <echo message="Testing: java -jar ${dist.jar} "/>
          <java jar="${dist.jar}" fork="true" failonerror="true">
              <assertions> <enable/> </assertions>
          </java>
          <echo message="Testing: java -jar ${dist.jar} help"/>
          <java jar="${dist.jar}" fork="true" failonerror="true">
              <arg value="help"/>
              <assertions> <enable/> </assertions>
          </java>
          <echo message="Testing: java -jar ${dist.jar} usage"/>
          <java jar="${dist.jar}" fork="true" failonerror="true">
              <arg value="usage"/>
              <assertions> <enable/> </assertions>
          </java>
          <echo message="Testing: java -jar ${dist.jar} version"/>
          <java jar="${dist.jar}" fork="true" failonerror="true">
              <arg value="version"/>
              <assertions> <enable/> </assertions>
          </java>
          <echo message="TESTING JAR COMPLETED."/>
      </target>
      
      <!-- test the bin script -->
      <target name="mybintests" description="Test bin.">
          <chmod file="${test.script}" perm="ugo+rx"/>
          <echo message="Testing: sh -c ${test.script} ${dist.dir}"/>
          <exec executable="sh" failonerror="true">
              <arg value="-c"/>
              <arg value="${test.script} ${dist.dir}"/>
          </exec>
          <echo message="TESTING BIN SCRIPT COMPLETED."/>
      </target>
      
      <!-- test the system script -->
      <target name="mysystemtest" description="Test system.">
          <chmod file="${test.system.script}" perm="ugo+rx"/>
          <echo message="Testing: sh -c ${test.system.script} ${dist.dir} ."/>
          <exec executable="sh" failonerror="true">
              <arg value="-c"/>
              <arg value="${test.system.script} ${dist.dir} ."/>
          </exec>
          <echo message="TESTING SYSTEM SCRIPT COMPLETED."/>
      </target>
      
      <!-- findbugs target -->
      <target name="findbugs" depends="myfindbugs"
              description="Run findbugs in batch mode"/>
      
      <!-- my target that runs findbugs directly (in separate VM with 512M heap) -->
      <target name="myfindbugs" description="Run findbugs in batch mode">
          <echo message="findbugs -maxHeap 512 -textui -effort:max -low -exitcode ${dist.jar}"/>
          <exec executable="findbugs" failonerror="true" vmlauncher="false">
              <arg value="-maxHeap"/>
              <arg value="512"/>
              <arg value="-textui"/>
              <arg value="-effort:max"/>
              <arg value="-low"/>
              <arg value="-exitcode"/>
              <arg path="${dist.jar}"/>
          </exec>
          <echo message="FINDBUGS COMPLETED."/>
      </target>
      
      <!-- my own javadoc target because doc-files don't seem to get copied over -->
      <target name="myjavadoc" depends="init,-javadoc-build,-javadoc-browse" 
              description="Build Javadoc.">
          <mkdir dir="${doc.files}"/>
          <echo message="Populating doc-files directory: ${doc.files}"/>
          <copy todir="${doc.files}">
              <fileset dir="${doc.files.src}" casesensitive="yes">
                  <include name="*"/>
              </fileset>
          </copy>
          <echo message="JAVADOC COMPLETED."/>
      </target>
      
      <!-- all target -->
      <target name="all" depends="jar,test,mybintests,javadoc,findbugs"
              description="Do everything"/>
      
  
  

So all the above is new and updated. I could not get the echo to add a newline to the file, even following all the documentation on echo, so I just used the exec target and the echo command of the system. I also had problems with javadoc populating the doc-files, so I had to add my own doc-files copy. I'm not sure what I did to break this. The javadoc command seemed to be sensitive to relative vs. full paths, or the current directory setting.

My recommendation again is to avoid the use of full paths everywhere you can, it just makes life easier. Also, any need to do simple shell commands like:
domainname | cut -d'.' -f2 | tr '[:upper:]' '[:lower:]'
had better be done in a shell script or in the Makefile. In my real Makefile I do some system specific shell commands to determine the value of an ant property that I pass into ant. This works well for me because when running ant directly I don't need this setting. The other way to do this would be to exec a shell script and capture it's output. Running shell scripts in ant seems to work best when you run them with sh command. Windows often will not recognize a shell script.

The findbugs task was not connected into the NetBeans build/test system, and is just used by the Makefile or directly from the ant script.

Well, it turns out that this works very nicely.

The following Makefile was updated to avoid the use of full paths and simplify how the ant targets are used.

The Makefile was trivial, and looks like:

# Makefile to simulate a NetBeans build

# This Makefile is located one directory below the ant basedir
TOPDIR  = ..

# Value of ant property that needed to be created OS specific
system_instance=testsystem

# How to run ant
ANT_OPTIONS += -Djprt.system.instance=${system_instance}
ANT = ant $(ANT_OPTIONS)

# All ant targets of interest
ANT_TARGETS = all jar test javadoc findbugs clean init compile

# Create a make target for each
$(ANT_TARGETS):
        ( cd $(TOPDIR) && $(ANT) $@ )

# Declare these phony (not filenames)
.PHONY: $(ANT_TARGETS)

Update, now the path to ant and findbugs just need to be put in your PATH.

I'm sure there are better ways to do some of this, but I did get the above working, and I am just an ant beginner. I still don't see any easy way to loop over a set of names with ant, and the 'condition' task is really confusing. (No wonder there are 600+ page books on how to write ant scripts.) I suppose people say the same thing about Makefiles. ;^)

Hope someone gets something out of this, and yes I've learned to live with ants. ;^)

-kto

JDK6 Build Cheat Sheet

JDK6 Build Cheat Sheet

Just thought I'd list a few ways that the JDK can be built. These apply to JDK6 and JDK7, JDK5 building is a little different but has some of the same settings. The gnumake used here is the GNU make 3.78.1 or newer.

• On Solaris systems with the Companion CD installed, it can usually be found with the name gmake in /usr/sfw/bin/ or /opt/sfw/bin/.
• On Linux just use make from /usr/bin/
• On Windows and using CYGWIN, just use make from CYGWIN's /usr/bin/ (but make sure it's not 3.81, which seems to have taken away the ability to use paths with the ':' character, like C:/anything.)
• On Windows with MKS, you will need a GNU make built for MKS.

See the JDK build instructions for more information.
NOTE: Any variables described below that start with ALT_ can be set on the gnumake command line, or be set in your environment. None of the Makefiles should be setting these variables internally, so no internal make variable setting should change your environment variable setting of them. The command line variable settings will override any setting inside the Makefiles.

Control Builds

From the control/make/Makefile:

• Build everything for a developer:

  gnumake dev
  OR
  gnumake DEV_ONLY=true

• Skip the fastdebug build (just builds the product bits):

  gnumake SKIP_FASTDEBUG_BUILD=true DEV_ONLY=true

• Don't build the fastdebug bits, and skip the hotspot and deploy areas:

  gnumake BUILD_HOTSPOT=false BUILD_DEPLOY=false ALT_JDK_IMPORT=/jdk1.6.0 SKIP_FASTDEBUG_BUILD=true DEV_ONLY=true

• Build just the j2se debug image:

  gnumake BUILD_HOTSPOT=false BUILD_DEPLOY=false ALT_JDK_IMPORT=/jdk1.6.0 SKIP_FASTDEBUG_BUILD=true SKIP_DEBUG_BUILD=false DEV_ONLY=true debug_build

Here is a short list of the make variables that can be set on the gnumake command line:

• BUILD_HOTSPOT

  Set to false to avoid building the hotspot VM, but you will need to also set ALT_JDK_IMPORT_PATH to a built jdk product of the same version you are building. I recommend setting this to false all the time, unless you are changing the hotspot VM.

• BUILD_DEPLOY

  Set to false to avoid building javaws or the plugins. I recommend setting this to false all the time.

• BUILD_INSTALL

  Set to false to avoid building install bundles (you should never need to do this). I recommend setting this to false all the time.

• BUILD_J2SE

  Set to false to avoid the core j2se, which is unlikely you'd want to do this. I'd leave this one alone, unless you are only working on the hotspot VM.

• BUILD_MOTIF

  Set to false to avoid building the Motif libraries, but you would need to then set ALT_MOTIF_DIR to where to get the built Motif libraries. This doesn't take long to build so I have never played with this, and I'm not even sure it's necessary

• NO_DOCS

  Set to true to avoid any javadoc generation.

• NO_IMAGES

  Set to true to avoid any images generation. This means the creation of the more formal JDK install-like image with the rt.jar and tools.jar files. When the images are not built, the JDK is left in the OUTPUTDIR area as a simple set of bin, lib, include, and classes directories. Which can be run as-is, but do not represent the form that a JDK install image will take. The creation of the images is mostly a disk movement activity, and on some machines this is very fast, and with others it can be slow.

• SKIP_FASTDEBUG_BUILD

  Set to true to skip building any fastdebug bits (a set of -g -O built images with full runtime assert checking enabled).

• SKIP_DEBUG_BUILD

  Set to false to build the debug images (-g with full runtime assert checking).

• DEV_ONLY

  Set to true to get the settings: SKIP_COMPARE_IMAGES=true BUILD_INSTALL=false NO_DOCS=true.

• SKIP_COMPARE_IMAGES

  Set to true to avoid a comparison of previous release build bundles with the ones you have created. You normally won't want this, so always set this to true.

• ALT_OUTPUTDIR

  Alternative output directory rather than control/build. I recommend that the sources and this output directory be on your build machines local disk, for best build time performance.

• ALT_BOOTDIR

  Alternative location for a bootstrap JDK, or install image of the previously released JDK (e.g. jdk1.5.0 for building jdk1.6.0). You really should set this all the time, just to make sure you get a local disk copy, for best build time performance. In theory, this doesn't have to be jak1.5.0, but could be a newer JDK, even the same version being built. But normally it's the previously release JDK.

• ALT_JDK_IMPORT_PATH

  Alternative location for a JDK build of the same release as what you are building. If you skip building some components, the Makefiles will try and copy over the pre-built bits from this location, e.g. if you use BUILD_HOTSPOT=false (or don't have a hotspot directory), then the hotspot built images will be copied from this location.

Also try running gnumake help for more help on building options.

J2SE Builds

If you only want to work with the j2se and not the VM, you might consider just going into the j2se/make directory and running gnumake from there. Just run gnumake help for some help.

I'd recommend setting ALT_BOOTDIR and ALT_JDK_IMPORT_DIR (as described above).

Hotspot Builds

If you only want to work with the hotspot and not the VM, you might consider just going into the hotspot/make directory and running gnumake from there. Just run gnumake help for some help.

I'd recommend setting ALT_BOOTDIR and ALT_JDK_IMPORT_DIR (as described above).

Traditionally the hotspot build procedure have differed greatly from the rest of the JDK. This is mostly due to the fact that for the most part, it's more like building a large C++ library rather than anything else. Building each of the platforms was slightly different, for example on Windows the tool NMAKE.EXE was used instead of gnumake. It's the Makefiles and build scripts in the hotspot/build directory that contain the "real" hotspot build procedures. In JDK6 a gnumake Makefile was added at hotspot/make/Makefile which will use the Makefiles in the hotspot/build area, providing you a more generic hotspot building experience. More experienced hotspot builders may prefer to dive into the hotspot/build area.

Summary

So as you can see, some similarities, some differences between these build rules.

Hope this was helpful, if you have more questions please post comments.

-kto

My Dog is now Open Source

After a long debate it's finally happened, I've open sourced my dog. It just seemed like the right time, and all these concerns about people sticking forks in my dog, or taking it and actually teaching it new tricks, what was I thinking? So here she is:

Oh by the way, here are a list of issues I need fixed:

9103456 Keeps jumping up on people and licking them
9104567 Occasionally eats something dead, barfs in the front room
9105678 Runs around the house like a crazy dog when new people visit
9106789 Constantly barks at cats, then runs away from them

-kto

Build Machines and Test Machines

Just thought I'd touch on a subject that some people forget about when having to build products that will run on many different machines. Building and testing on a single machine is easy, building bits that need to run on dozens of different machines is the hard part.

Test machines and Build machines are different. Ideally you WANT to test on any and all machines you can get your hands on, but when you build, you need build machines that match specifications and have pre-installed products, anything else and you run the risk of creating faulty build bits. These build machines can also be so old that some products might not be available, or might have problems running on it. Or some product dependency that a product depends on won't be available. So anything used in the build process has to be carefully considered and carefully tracked.

Once the platforms a release will officially support is decided, then we can decide what the official build machines and configurations will be. But we also have to consider the timing of making these changes during the release, so expect these changes to be gradual over time and not all at once.

I'm not a Release Engineer, but I do appreciate the job they have to do. Once a product is released, the Release Engineering team has to preserve those build machines as long as the product might need to be updated, patched, or re-built for any reason. This can be a very long period of time, anywhere from 5-7 years. Any change to a build system can create a difference in the built product, and sometimes those differences are not readily apparent, sometimes not until a bug is found in the field. If only we could all deliver pure Java products, our lives would be so much easier.

So when it comes to build systems and changes to build systems, expect it to be slow and careful.

-kto

Teamware, Mercurial, and SCCS revs that go bump in the night

As seen in Mark's recent blog on the JDK and Source Code Management (SCM) solutions at http://blogs.sun.com/mr/entry/openjdk_scm, you can see that we are finally looking at converting to a new SCM.

Inside Sun the historic and most common SCM used has been Teamware. The Sun Workshop product had delivered Teamware with it's compilers and tools for many releases, but the newer Sun Studio product dropped Teamware from it's feature list many years ago. Of course this didn't stop the developers from continuing to use it. The best high-level description of Teamware I could find is at http://docs.sun.com/source/806-3573/intro.html.

Besides all the basic SCM features you'd want, Teamware provided two extremely valuable features in my view. The primary feature was the ability to have any number of workspaces (repositories) with a parent/child relationship, there is no single repository. The second feature was it's extremely powerful merging tool called filemerge. Luckily filemerge can work with any SCM, and over the years many source merging tools have come into the picture, so as much as I loved filemerge, it's not that critical a SCM feature anymore. But that first feature of having essentially a tree of repositories allowed us to do a sort of 'divide & conquer' or distributed approach to development. Any group of developers could spin off a team workspace and a child workspace per team member, completely independent from the master workspace. This distributed development model allowed teams to work in complete isolation, yet have all the SCM tools available to them at any number of levels. Well, bottom line is, we just can't give this up very easily.

At the core of Teamware was SCCS which managed source file changes on a one by one basis. A Teamware workspace is essentially a set of SCCS controlled source files, with a bit of frosting. If you really want the nitty gritty details, take a look at http://docs.sun.com/source/806-3573/underhood.html It's these SCCS files that contain the various revisions of source files, along with comments and details from the developer when a revision of a source file was created.

As Mark stated in his blog, we are strongly leaning toward Mercurial, but we need to study the impact of the conversion. Martin Englund has been looking into the SCCS migration into Mecurial so we have a feeling of the size and performance differences. Just copying files into a new SCM isn't the issue here, it's the conversion of all the file revisions and the information that goes along with each revision. As an example, the Hotspot workspace alone contains roughly 3,300 files, but has a total of 140,000 individual file revisions. And Hotspot is one of the younger workspaces in our JDK workspace corral. We know (thanks to Martin) that Mercurial can handle the 3,300 files with amazing speed, and Martin even tried 40,000 files. It was very fast. Now we need to see what happens when it gets all the revisions too.

Teamware did not provide a very good 'change set' model, the workspaces had individual history of putbacks or commits, but the history of these actions didn't propagate up the tree of repositories. So we lack any good Teamware history data to help us group the individual SCCS file revisions together into logical 'change sets'. So for the most part we are left with the individual SCCS revision data for thousands of files. So we are brainstorming how we can group individual file changes together into change sets with just the:

• SCCS comments (in theory the files of a change set might have the same comment)
• possible bug ids (7 digit numbers) in the comments (in theory the files of a change set might have the same bug id)
• user id's (in theory a change set would have the same user id)
• times with no time zone :^( (maybe in theory, the files of a change set would have been changed within 24 hours of each other?)

Why not just do a blind 'one file revision' to 'one change set'? I guess it's part of being an engineer, we can do better than that. :^) And it looks like a fun graph project too, humm, maybe I can get some use out of that Mathematics degree I have?

Of course, if we figure this out, and I don't get fired for doing a horrible job of it (just kidding), this SCCS revision conversion work could map to any of SCM's with a change set model, but we will be focusing on Mercurial for the time being.

The Open Solaris team did a fairly detailed study on SCMs which is available at http://opensolaris.org/os/community/tools/scm/.

Mercurial is an open source project and details about Mercurial can be found at http://www.selenic.com/mercurial/wiki/index.cgi. Martin Englund has also been nice enough to provide a Teamware to Mercurial mapping for the existing Teamware users at http://blogs.sun.com/martin/entry/mercurial_for_teamware_users.

No transition will be easy, and we know that some of you might be upset that the choice wasn't SubVersion, but for what we need to do in the JDK, with our JDK developers all over the world, our answer wasn't SubVersion.

JPRT: Build/Test System for the JDK

I did a little blogging on JPRT at http://blogs.sun.com/kto/entry/jprt_sun_hardware_is_so but that was mostly to talk about the COOL rack of Sun hardware that I used. Now I want to talk a little more about why we need something like JPRT, and what it does for us. I've been working on this JPRT project for quite some time now, so I've kind of lost touch with the real world lately. Ronald Reagan is still the President, isn't he? ;^) Anyway....

JPRT ("JDK Putback Reliablity Testing", but ignore what the letters stand for, I change what they mean every day, just to annoy people :^) is a build and test system for the JDK, or any source base that has been configured for JPRT. As I mentioned in the above blog, JPRT is a major modification to a system called PRT that the HotSpot VM development team has been using for many years, very successfully I might add. Keeping the source base always buildable and reliable is the first step in the 12 steps of dealing with your product quality... or was the 12 steps from Alcoholics Anonymous... oh well, anyway, it's the first of many steps. ;^)

Internally when we make changes to any part of the JDK, there are certain procedures we are required to perform prior to any putback or commit of the changes. The procedures often vary from team to team, depending on many factors, such as whether native code is changed, or if the change could impact other areas of the JDK. But a common requirement is a verification that the source base with the changes (and merged with the very latest source base) will build on many of not all 8 platforms, and a full 'from scratch' build, not an incremental build, which can hide full build problems. The testing needed varies, depending on what has been changed.

Anyone that was worked on a project where multiple engineers or groups are submitting changes to a shared source base knows how disruptive a 'bad commit' can be on everyone. How many times have you heard:
"So And So made a bunch of changes and now I can't build!".
But multiply the number of platforms by 8, and make all the platforms old and antiquated OS versions with bizarre system setup requirements and you have a pretty complicated situation (see http://download.java.net/jdk6/docs/build/README-builds.html).

We don't tolerate bad commits, but our enforcement is somewhat lacking, usually it's an 'after the fact' correction. Luckily the Source Code Management system we use (another antique called TeamWare) allows for a tree of repositories and 'bad commits' are usually isolated to a small team. Punishment to date has been pretty drastic, the Queen of Hearts in 'Alice in Wonderland' said 'Off With Their Heads', well trust me, you don't want to be the engineer doing a 'bad commit' to the JDK. With JPRT, hopefully this will become a thing of the past, not that we have had many 'bad commits' to the master source base, in general the teams doing the integrations know how important their jobs are and they rarely make 'bad commits'. So for these JDK integrators, maybe what JPRT does is keep them from chewing their finger nails at night. ;^)

Over the years each of the teams have accumulated sets of machines they use for building, or they use some of the shared machines available to all of us. But the hunt for build machines is just part of the job, or has been. And although the issues with consistency of the build machines hasn't been a horrible problem, often you never know if the Solaris build machine you are using has all the right patches, or if the Linux machine has the right service pack, or if the Windows machine has it's latest updates. Hopefully the JPRT system can solve this problem. When we ship the binary JDK bits, it is SO very important that the build machines are correct, and we know how difficult it is to get them setup. Sure, if you need to debug a JDK problem that only shows up on Windows XP or Solaris 9, you'll still need to hunt down a machine, but not as a regular everyday occurance.

I'm a big fan of a regular nightly build and test system, constantly verifying that a source base builds and tests out. There are many examples of automated build/tests, some that trigger on any change to the source base, some that just run every night. Some provide a protection gateway to the 'golden' source base which only gets changes that the nightly process has verified are good. The JPRT (and PRT) system is meant to guard the source base before anything is sent to it, guarding all source bases from the evil developer, well maybe 'evil' isn't the right word, I haven't met many 'evil' developers, more like 'error prone' developers. ;^) Humm, come to think about it, I may be one from time to time. :^{ But the point is that by spreading the build up over a set of machines, and getting the turnaround down to under an hour, it becomes realistic to completely build on all platforms and test it, on every putback. We have the technology, we can build and rebuild and rebuild, and it will be better than it was before, ha ha... Anybody remember the Six Million Dollar Man? Man, I gotta get out more often.. Anyway, now the nightly build and test can become a 'fetch the latest JPRT build bits' and start extensive testing (the testing not done by JPRT, or the platforms not tested by JPRT).

Is it Open Source? No, not yet. Would you like to be? Let me know. Or is it more important that you have the ability to use such a system for JDK changes?

So enough blabbering on about this JPRT system, tell me what you think.
And let me know if you want to hear more about it or not.

Stay tuned for the next episode, same Bloody Bat time, same Bloody Bat channel. ;^)

-kto

New JDK Build Documentation

The latest JDK 6 build machine setup and build instructions are now available at: http://download.java.net/jdk6/docs/build/README-builds.html. Do NOT go to http://download.java.net/README-JRL.html, those are old pages, and I'm trying to get them updated or removed.

We have re-structured the build instructions and am trying to add in more information and details on the various build dependencies. This is a work in progress and we will be adding more information to this file as we move forward.

If you have any input or comments, please share it with everyone by posting a forum note at http://forums.java.net/jive/forum.jspa?forumID=25.

-kto

Late JavaOne 2006 Trip Report

Started this right after JavaOne, then got distracted... hopefully someone gets something useful out of it. Go to http://developers.sun.com/channel/ for the complete scoop on JavaOne 2006.

What a week, I'm tired, my feet hurt, and I think I have a headache from the after dark speaker system. I must be getting too old for this, but I'll be back next year, what am I? Nuts? Why do people inflict so much pain on themselves, complain about it, and then do the same thing the next year? Because it's a blast maybe? Because it's JavaOne? Who knows?
I suppose living in the Bay Area and needing to commute 90min each way everyday makes it a little harder on these old bones. Next year I'll get a Hotel room, maybe bring the spouse and stay the weekends before and after. Make a little vacation out of it and avoid the 12 hours of commuting. Maybe get one of those rooms with a big jacuzzi tub to soak in after those 10:30pm BOFs and the late night parties. :^)

So enough complaining, I'll give a little trip report on what I saw and did at JavaOne this year, with my somewhat twisted humor (I read too many National Lampoon magazines in college). There is NO way I could cover even a fraction of all that was happening, so this is just a smattering of what was at JavaOne 2006.

• Monday:

  I had fully intended to go to NetBeans Day, but plans were changed and I had to attend another event. So I don't have any report on NetBeans Day, so I'm looking forward to what people have to say about it. I've always heard good things about NetBeans Day in the past.

• Tuesday:

  Left home at 8:15am (like every morning) to get to the Bart Station for my 60min Bart ride. The wife was generous enough to drive me to and from the Bart station, so I could avoid the parking hassles at the Bart Station.

  Commentary: They created Bart so that people wouldn't drive their cars and then they didn't provide enough parking at the Bart stations, again so we wouldn't drive our cars? Kind of like not providing any roads to the Bart Stations?

  Missed most of the General Sessions this year, I tried to concentrate on seeing as many technical sessions as my mind could absorb, or at least that was the plan, I severely over estimated my ability absorb though. I had events and meetings all week that I had to attend, so I figured I'd just fill in the week with technical sessions and BOFs, boy was I naive. I even used that silly schedule builder and signed up for everything, I figured it would be smooth sailing, just like the fastpass tickets at Disneyland. Ah, but Disneyland knows a bit more about the management of lines or as some people call them "queues", I wasn't impressed with the way the enrolled vs. not enrolled lines were handled. Some of these rooms were huge, and would never overflow, yet they insisted on completely evacuating the room prior to each talk, creating havoc in the hallways. Then they give you an evaluation form from the 1960's where you have to use a number 2 pencil to fill in the little dots for your badge number, which I could never remember, and the writing was so small and the rooms so dark that you couldn't read the damn thing. Sure seemed like a waste of trees to me. They should just create an online survey, send it out to the registered attendees after the show and ask them to fill out evaluations on the talks they attended, paperless, fast, modern.

  Anyway, my first Tuesday event was Lab-1205 at 11am on Java Performance Analysis, a hands on lab taught by Tom Marble and Andrew Johnson. They used the visualgc tool to view state of the Java heap and explained via real demonstration the impact of various good and bad heap settings on a Java application. More material than could fit in an hour, but well worth attending, it was well done. Go to http://java.sun.com/performance/jvmstat/ for more information on jvmstat and visualgc.

  After the cattle drive to lunch, I tried to attend TS-3439 on Mustang, it's always a good idea to attend talks that might cover anything you may have worked on, but alas, I was lured to the Pavillion by the free Peets coffee provided by Siemens. Had a nice latte and chatted with the Siemens people about their GPS AV monitoring system, very interesting. Talk about Big Brother is watching.

  I did manage to attend the afternoon TS-5033 IntelliJ IDEA: Integrated Team Environment. This was an interesting talk to me, mostly because I've been currently working very hard the last 6 months on something very similar for the JDK development teams, something I'll blog about later called JPRT. The IntelliJ folks have done some good work here, recognizing the need to improve the total turnaround time for developers. They have built a system where putbacks to the source base are automatically detected and trigger builds/testing, they called it a continuous system that always kept their source repository "clean". I got some good ideas from this talk.

  My plan was to attend TS-3097 on JUnit, which I have started using in NetBeans lately. But apparently anything having to do with testing or JUnit appears to be very popular. The line was huge! And this was 20 minutes before the talk was to start! So I went to TS-3313 on IBM's VM. I wasn't very impressed, the talk seemed to be missing something and seemed to have been written by the marketing department.

  Now supposedly there was an unBOF at the Thirsty Bear, a place which I had never been to (yes I live in the area, but I avoid downtown San Francisco). I heard later that it was great, and I'm pissed I missed it. But for some silly reason I got some of the worst directions from people this year. I'd be told something was on 2nd and Mission, when it was on 3rd and Mission, or on 3rd across from the W when it was on Howard across from the W. I sure got my walking in. The bad directions reminded me of a trip I took to Ireland years ago, where we learned that you never never ever ask a drunk Irishman (or Irish-woman) for directions, they won't let you go until they have given you directions, and usually the directions are wrong. (My last name is O'Hair, so I can make fun of Irish people. ;^)

  In the evening there was the Pavillion Reception, which was great, lots of beer, wine, and MEAT! They were carving up big huge hunks of turkey and big old cow legs, good stuff. (My wife doesn't eat meat, so consequently I have to go out hunting for my own occasionally).

  Then my last event of the evening was the JDK Community party at the Argent Hotel, in a big fancy tent in the courtyard. Lots of beer, wine, food, and merriment. Rich Sands didn't get up and dance on the tables, but it was fun anyway. ;^) Talked to some people from Amazon, who had some great things to say about the quality of the binary snapshots, which they regularly download and test. To find out more about the JDK community go to http://community.java.net/jdk/. This lasted until around 10pm and I hitched a ride with a friend back to Livermore, saved myself $4 in bart fares. ;^)

• Wednesday:

  Attended the Java puzzlers talk (TS-1188), the first time I attended this talk. As expected, Joshua and Neal did a good job, fooled me with those puzzlers. The latest Java Puzzler book is at http://www.javapuzzlers.com/

  Attended Mark's Integrating XML into the Java Language talk (TS-3441). Having recently dealt with some code that used JDOM and attempting to work with that code, I appreciated his points. Not sure what people think about changing the language for this, the audience seemed to have a mixed reaction.

  At the Sun PODS I ran into some old co-horts that are now in the Sun Studio team working on a C/C++ NetBeans plugin. They gave me a demo of it and I plan on trying to use it to build and browse the JDK at some point. I've become a NetBeans convert lately. :^) For more information on this NetBeans C/C++ module, go to http://cnd.netbeans.org/.

  Also while wandering the Pavillion floor, ran into a past manager friend of mine and he was interested in having us try a new set of Sun Studio compilers for Linux, we currently use gcc/g++ with our JDK Linux builds. I'll probably give these a try when I get a chance. For more information on these Sun Studio compilers for Linux go to http://developers.sun.com/prodtech/cc/linux_index.html. Why would we change our Linux builds to a non-gcc compiler? I don't know if we would, but if the performance on Linux was significantly better, I suspect we would seriously consider it. I'm not suggesting we abandon gcc builds, but allow for both, and decide which set of binaries we would ship based on performance and quality issues.

  Managed to get an invite to the SDN party at Jillian's. Played some pool with a few Sun co-horts, and had an interesting chat with an employee of the U.N. While there I had a chat with another former manager of mine (yes, I'm still on speaking terms with all my former managers). And he voiced his concern that sometimes the Java capabilities of the Sun Studio tools (for Solaris AND Linux) are often overlooked, especially the performance analyzer. I agreed, but haven't done much in the way of performance analysis in a while. In most cases, performance issues found in Java apps on Solaris or Linux, are pretty generic issues, so using platform specific tools to diagnose generic Java performance issues is perfectly valid in my book. The Sun Studio 11 tools (including the performance analyzer) is free and available at http://developers.sun.com/prodtech/cc/index.jsp. Don't be annoyed too much by the 'buy this' and 'buy that', become an SDN member and go to http://developers.sun.com/prodtech/cc/downloads/index.jsp. With Solaris I've heard it's a Solaris SVR4 package install, which may require root access, so be prepared for a formal Solaris package install. I assume on Linux it's just a tar ball installation, I'll find out soon enough.

  Ok so now you are asking "Is this the only time Sun employees get to talk to each other? At the JavaOne Pavillion?". Well, no, not the only way, but it is great way for the various Sun teams working on Java projects to demonstrate their work, and it's really nice for everyone to take the time and see what others have done. I always look forward to touching bases with past co-workers and talking about the current hot projects going on, in Sun and with the other companies too. The JavaOne Pavillion is definitely a highlight to me.

  At 10:30 we had the JDK Community BOF (BOF-0178). It was not well attended, but a huge attendance wasn't expected, especially this late. The number of developers really interested in working in the JDK is far less than the number of developers wanting to use the latest JDK builds. Several people expressed to me their great satisfaction on having the weekly Mustang builds available to them, and one person was particularly impressed with the quality of these builds. Although I'm sure there are a few people who might disagree about the high quality of the Mustang weekly builds, it was nice to hear at least voices saying they really liked having the weekly binary snapshots available to them. Once again, to find out more about the JDK community go to http://community.java.net/jdk/.

  Met up with a fellow Livermore Bart commuter and we wandered off to catch one of the last Bart trains.

• Thursday:

  Attended the Jackpot talk (TS-1278), and am looking forward to checking out the Jackpot NetBeans module at http://jackpot.netbeans.org. Creating your own source transformers seems like a cool idea that could come in handy when managing large source projects.

  Sat in on the Harmony talk (TS-3752). They had some interesting component separations but overall it seems like it will be many years before thay have a fully functioning and well performing JDK. The J9 VM they were using didn't seem to give them much performance, or not as much as I expected it would, a bit surprising, but they are just getting started.

  Had to run off to some Sun meetings for the bulk of the afternoon. They should have a Meeting Quality Metric of some kind. Once a Meeting is over, it should be measured how worthwhile the meeting was, and if the metric is low, you sacrifice the meeting in a formal ceremony, declaring to never have the meeting again.

  Attended part of the After Dark Bash, waited in line for 20 minutes to get my two free beers, then gave one away and left shortly after that so I missed the band and most of the party. Maybe I'm getting too old for these kind of parties, the flashing lights and loud music gave me a bit of a headache. But everyone was having fun and there were lots of games and people to talk to, if you could hear them. ;^)

  At 9:30pm we had the JDK Tools BOF (BOF-0351), which we did in a pretty informal way, but got lots of good questions from an audience of about 200, a GREAT turnout for being so late. Sundar demonstrated jmap and jhat, and Alan talked to a few slides, and I just stood there and injected comments now and then. I hope people were happy with the give and take, we did plan on it being informal, and it could have been better prepared, but I thought it went fine. It was just a BOF after all. Next year maybe we can wear funny hats or do a few rehearsed skits about failing Java apps and trouble-shooting tools coming to the rescue or something. ;^)

  The JConsole BOF immediately followed (BOF-0442), but I was so tired I decided to get on Bart and go home. I needed some rest. I heard it went well, and most of the Tools people stayed for this BOF which was in the same room. I'm sorry I missed it.

• Friday:

  After my last 60 minute Bart ride in, managed to catch the first 15 minutes of Sundar's talk on Scripting TS-1382. Unfortunately I had to go to another Sun meeting and was unable to hear the rest of this talk, basically an update on Java Scripting in Mustang. The room was pretty full, maybe 400-500 people maybe? So Java Scripting continues to be a popular topic, or it was Friday and a bunch of people didn't have anything else to do. For more on Java scripting go to https://scripting.dev.java.net/.

  Attended the DTrace Hands on Lab (LAB-9010) and had a nice time just playing with DTrace. Even though I provided the DVM agent at https://solaris10-dtrace-vm-agents.dev.java.net/ getting dedicated time to play with DTrace is a different story. This was a good class, it provided some intro to DTrace and then took you right into using it on JDK 6 (Mustang) with the new DTrace probes provided there. The room was pretty full, so it seemed to be a popular topic. DTrace isn't for the beginner necessarily, but it is certainly something you want to know about if you venture into Solaris 10. There is nothing like it anywhere else. For more on DTrace try going to http://www.opensolaris.org/os/project/dtrace-chime/java_dtrace_api/.

  Immediately after the DTrace class I volunteered to help out with the Mustang Hands on Lab (LAB-1506), which is a course for people interested in working on or building the Mustang (JDK 6) source. The class when better than last year, but it's a difficult class to organize. Two incidents in the class are worth mentioning. Several people unfamiliar with X or GNOME managed to kill their window manager with a few well placed fingers on the keyboard. I was at a bit of a loss as to what had happened, never having used those key strokes before (Control Shift Backspace maybe?). Luckily Peter Kessler (the fount of all knowledge :^) was around to explain things, now I know why I'm not a GUI person. ;^) The second incident involved what appeared to be a crash of the system somehow, and the immediate reaction of the student in the lab was just 're-boot', like it was just a common thing to happen, a simple Blue screen that you fix with a re-boot. But this was Solaris 10, and I said "No, let's figure this out first, this is Solaris and this should never happen"! (is this what happens when people work with Windows too long, they learn to accept system crashes and just 're-boot'? Sigh, how sad.) Anyway, multiple people helping in the Lab were aghast that Solaris 10 would do such a thing, and there was no end of Sun employees looking at the machine and trying to diagnose the problem. I'm not sure what the final resolution was, but the reaction by my fellow Sun employees was the same as mine, "Capture the core file or crash data and make sure we tell someone". This is one of the reasons I like working at Sun, and working with the people I work with, they really do care about quality, don't just re-boot, scream bloody murder and report it. :^0 For more information on building Mustang and getting the latest JDK 6 source or binary snapshots go to https://mustang.dev.java.net/

In Summary, the conference seemed more crowded and the Sun PODs in particular seemed really busy to me, more so than in years past, but that's just an impression. Everyone seemed upbeat and friendly, and it seemed like overall a good conference.

Hope you enjoyed the long winded trip report, Looking forward to next year.

Oh yeah, I almost forgot, I normally skip the "Picture with Duke", but this time there was no line, so click on this to see my picture with Duke:

Picture With Duke

Ok, one last thing... I'm sure you are asking, "Why the Aloha Shirt?". Well, I'm not sure, once upon a time, a long long time ago, a fellow named Bob J. used to wear them, and Sun also used to have "Friday Aloha Shirt" days, but as to why I continue to wear them? To the point that it's all I wear? I guess it's because they aren't boring, kind of the complete opposite of a business suit. And they look better than a printed T-Shirt. :^)

The Bloody Bat and Fixing Regressions

First I'm going to tell a little story about the bloody bat, then I'll explain what it has to do with Regressions in the JDK (if you haven't heard about the Mustang JDK Regression Challenge, go to https://mustang.dev.java.net/regchal/). For those of you with children, and especially female children this story may strike a cord. If your female children aren't quite to the dating stage, boy are you in for it. ;^) I don't know why I do these things, I suspect it had to do with being raised in a house with just brothers and no sisters. But I became concerned about some of the young men that were being brought to our home. I figured that it would be important for them to know that the girl's father could be a little deranged, and maybe, just maybe, that fact would 'keep them in line'. So I took my Louisville slugger bat and applied some drippings of red paint. I also did this to a metal stake that I use on Halloween, but that's another story. Anyway, when we would have young male visitors, or even large groups of young female visitors at the home (they would spread the word about the lunatic father), I would come in from the back door with the bloody bat saying things like "That cat won't come in our back yard again.", or "Oh no, you have MORE guests? I just got clean up.", or "I didn't know I could hit a squirrel that far.". You get the idea. Sometimes they would laugh, think it was funny, but sometimes their eyes would get big and they'd really think I was crazy. Now I have no idea if this worked or not, but so far all the boyfriends have been polite and my kids are turning out to be fantastic kids, even if their father is bit bonko. Of course their mother probably had the most to do with my kids being so fantastic, I probably just ruined their young dating years (and that's a bad thing? :^). SO. What does this have to do with finding JDK regressions? Nothing really, it's not like I can influence you to find and file JDK regressions, I mean, I don't even know where you live. Do I? :^) In all seriousness, we really want to beat all the regressions out of this Mustang release, and we need your help. Nothing is too small to tell us about, and everyone is taking this very seriously. And what the heck, you might win a pretty slick machine! By the way, most of us Sun employees are pretty bummed that we aren't in the running to win this machine. I may need to go make a visit to the person that made up those rules, humm where's my bat? ;^)

JNI How To

Compilation of JNI Code

I've added a few clarifying comments in italics below.

Using the Java^TM Native Interface (JNI) is not something many Java programmers have to deal with, but when you do, you need to know something about native applications. Whether it's Windows, linux, or Solaris, each native platform and sometimes native compiler or even the release of the native compiler has slightly different issues, so I thought I would write a little on each of these issues.

With JNI you are actually building a native shared library (.so or .dll file) that the Java^TM Virtual Machine (JVM) will dynamically load into the Java native process with dlopen() on Solaris and Linux, or LoadLibrary() on Windows. This "dynamically loaded shared library" fact is the basis for many of the compilation issues, not so much the use of JNI, dynamic loading of native libraries can be tricky. Some of the compilation issues relate to the library being loaded into a Java process and causing conflicts in the way the JDK was compiled itself, or the way the JDK assumes code was compiled (sometimes this is considered a JDK bug, but it depends on a great number of factors).

All libraries loaded into java are assumed to be MT-safe (Multi-thread safe). This means that multiple threads could be executing the code at the same time, and static or global data may need to be placed in critical sections. If you don't know what MT-safe means, you should do some research and make sure you understand what this means, it is a critical concept to understand when creating JNI libraries and writing JNI code. So if there is a compiler option to select MT-safe generated code, you will need it.

Compiler optimizations are always an issue and you need to be careful with high optimization levels. The safest route is to stick with just -g compilations, then use lowest optimization level you need, increasing it only when your native code and the overall application really benefits from increased optimizations.

Tips

So here a a few tips on the different platforms and the common compiler options to watch out for:

Solaris

• On Solaris, using the Sun Studio 11 C compiler, the typical compile and link command lines might look something like:
  For 32bit SPARC:
  
  cc -xO2 -v -mt -xc99=%none -xCC -Xa -xregs=no%appl -xmemalign=4s -xstrconst -xarch=v8 -KPIC -DTRIMMED -DNDEBUG -Dsparc -D__solaris__ -D_REENTRANT -DTRACING -DMACRO_MEMSYS_OPS -DBREAKPTS -c *.c
  
  
  cc -mt -xarch=v8 -Mmapfile -z defs -ztext -G -o libXXX.so *.o -lc
  
  
  For 64bit SPARC:
  
  cc -xO2 -v -mt -xc99=%none -xCC -Xa -xregs=no%appl -xstrconst -xarch=v9 -KPIC -DTRIMMED -DNDEBUG -Dsparc -D__solaris__ -D_REENTRANT -DTRACING -DMACRO_MEMSYS_OPS -DBREAKPTS -c *.c
  
  
  cc -mt -xarch=v9 -Mmapfile -z defs -ztext -G -o libXXX.so *.o -lc
  
  
  For X86:
  
  cc -xO2 -mt -v -xc99=%none -xCC -Xa -xregs=no%frameptr -xstrconst -KPIC -DTRIMMED -DNDEBUG -Di586 -D__solaris__ -D_REENTRANT -DcpuIntel -D_LITTLE_ENDIAN= -Di386 -DTRACING -DMACRO_MEMSYS_OPS -DBREAKPTS -c *.c
  
  
  cc -mt -Mmapfile -z defs -ztext -G -o libXXX.so *.o -lc
  
  For AMD64:
  
  cc -xO2 -mt -v -xc99=%none -xCC -Xa -xregs=no%frameptr -xstrconst -xarch=amd64 -KPIC -DTRIMMED -DNDEBUG -Di586 -D__solaris__ -D_REENTRANT -DcpuIntel -D_LITTLE_ENDIAN= -Damd64 -DTRACING -DMACRO_MEMSYS_OPS -DBREAKPTS -c *.c
  
  
  cc -mt -xarch=amd64 -Mmapfile -z defs -ztext -G -o libXXX.so *.o -lc
  
  
• Architecture/File Format: For SPARC 32bit use -xarch=v8, for SPARC 64bit use -xarch=v9, for X86 (32-bit) leave the option off or use -xarch=generic , and for AMD64 (64bit) use -xarch=amd64 with both C and C++.
  This is to be specific as to the architecture and the file format of the .o files (and ultimately of the .so). Other -xarch values may work, but many won't mix well with the native code of the JDK, so be careful. Also be careful when using -xarch and some of the other compiler optimization options that target specific processors (e.g. -fast should not be used unless you follow it with an overriding -xarch option).
• MT-Safe, Position Independent: Use -KPIC -mt with both C and C++.
  The -mt should be used for all compiles and links, it makes sure the library is built properly to be a MT-safe library. The -KPIC option makes sure your code is built in a position-independent way to maximum use in a shared library.
• Register usage: For SPARC (both 32bit and 64bit) use -xregs=no%appl and for X86 and AMD64 use -xregs=no%frameptr with both C and C++.
  On SPARC, certain global registers should not be used by system libraries and on X86, you will be better off if the frame pointer register is NOT used as a general purpose register. Not using -xregs=no%frameptr can cause problems in getting accurate stack traces in native code.
• Alignment: For SPARC 64bit do NOT use -xmemalign=4 with both C and C++.
  Usually the defaults for memory alignment is acceptable, but the general theory is that a shared library compiled with great alignment is probably ok, with less alignment is a SIGBUS disaster waiting to happen. We build the SPARC 32bit JDK with -xmemalign=4s to be compatible with any user libraries built by older compilers that had this as the default. Newer Sun compilers have a default of -xmemalign=8s, so we explicitly ask for -xmemalign=4s. We build the SPARC 64bit JDK with -xmemalign=8s explicitly (which has always been the default), if the -xmemalign=4s option is used to build a SPARC 64bit library, it's likely the JDK will crash with a SIGBUS (alignment error).
• Optimization: The higher the optimization level, the greater the risk of running into some kind of mix problem, much of the JDK is built with -xO4, and the rest with -xO2, and sometimes the JVM can make assumptions about native code optimizations that can cause problems.
  For example, on X86 it is important that the code in the native libraries have frame pointer registers, which is why we use -xregs=no%frameptr, and prevent this optimization. Sometimes optimizations above -xO2 can remove the use of frame pointer registers (wanting the extra general purpose register for performance sake) and this can cause problems when doing stack frame walking. The -xO2 level has proven to be relatively safe.
• C++ Exceptions: Use -features=no%except -DCC_NOEX if your C++ code does not use C++ exceptions.
  Unless you are well aware how to handle C++ exceptions in shared libraries. Very little of the JDK code uses C++ exceptions.
  The problem, as I understand it, is when the C++ exception escapes your shared library, you really don't want this to happen. So if you do use C++ exceptions, you just need to keep it 'contained' so to speak, inside your own library, handling them internally or turning them into Java exceptions with JNI. There also can be some code performance loss when the C++ compiler is generating code that needs to handle C++ exceptions. So if your code is not using C++ exceptions at all, it's best to tell the compiler that, it really can't know at compile time.
• Quality: Use -Xa -v -xstrconst -xc99=%none -errwarn=%all with the C compiler, and -errwarn=%all with the C++ compiler.
  Although not required, I have found these options to be helpful in tracking down problems in C code. The -Xa allows both ISO standard C and K&R C but favors ISO Standard C when they conflict. The -v asks for more warning messages. The -xstrconst forces all string literals in a strictly read-only area of memory. The -xc99=%none makes sure C99 language features are not used (assuming you want highly portable C code). The -errwarn=%all option turns all warning errors into fatal, and you might not want this, but if you have spent some time in getting all your code error free, it might be worthwhile.
• Library for java_g: The library must be named with a _g.so suffix.
  The Mustang (JDK 6) release does not have a 'java_g' version so this applies to Tiger (JDK 5) and older releases only.
• Library: Use -G -z defs -z text when building your library.
  Although not required, ideally you want your shared library to have a read-only and 100% shared text or code segment. These options will help guarantee that.
• Mapfiles: Use -M mapfile.
  Although this isn't technically required, it is highly recommended. A mapfile (called a version script in Linux) allows you to control various aspects of the shared library that is built. The feature of mapfiles that I recommend people should use here is the ability to limit the extern symbols accessable from your library. For example, if you had a native method called "func" in Java class "bar" which was in the Java package "foo", ultimately with the use of javah you would end up with an external symbol name of "Java_foo_bar_func", and your JNI shared library would have an extern symbol with that name. But there could be lots of extern functions in your native library, and odds are, all you want to expose is this one name. This mapfile would do the trick:
  

  Interface_1.1 {
          global:
              Java_foo_bar_func;
          local:
              *;
  };
  

  
  Creating a shared library with only the one symbol you wanted to make available.
• Static Linking: Do NOT use static linking of the C++ runtime.
  Static links mean that the code from a library is actually copied into your own library, e.g. you inherit extern symbols The C++ runtime library is /usr/lib/libCrun.so (and also /usr/lib/libCstd.so). Although it's possible to do this, it just isn't worth it.
• C++ Library Runpath and Dependencies: Use -norunpath -xnolib.
  The -xnolib makes it so you are explicit with every library you want loaded, so if you indeed want all the C++ runtime support libraries, you would need to add -lCstd -lCrun -lm -lc. The -norunpath has to do with the runpath or runtime path directories built into the library you have built. In general you do not want to create a library that refers to any directories in your C++ compiler installation area. The libCrun.so and libCstd.so libraries will be (and should be) found in /usr/lib and in general you don't need to do anything special for those libraries. But if you have other private libraries that you are dependent on you need to deal with the runpath issue (see the -R option in the man pages).
• Dependencies: Use ldd -r LibraryName.
  After the shared library has been built, the utility ldd can be used to verify that all dependent libraries have been satisfied, and all externs can be found. If ldd says anything is missing, it is very likely that the JVM will also be unable to load this library. This usually means that you missed some -lname options when building the library, or perhaps forgot a -R path option that tells the library where to look for libraries at runtime.
  Also, inspect the list of libraries that your library needs, does it make sense?

Linux

• On Linux, using the gcc version 3.2, the typical compile and link command lines might look something like:
  For X86:
  
  gcc -O2 -fno-strict-aliasing -fPIC -pthread -W -Wall -Wno-unused -Wno-parentheses -DNDEBUG -Di586 -DARCH='"i586"' -DLINUX -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -D_REENTRANT -D_LITTLE_ENDIAN -c *.c
  
  
  gcc -Wl,-O1 -Wl,-version-script=mapfile -z defs -Wl,-soname=libXXX.so -static-libgcc -shared -mimpure-text -o libXXX.so *.o -lc
  
  For AMD64:
  
  gcc -O2 -fno-strict-aliasing -fPIC -pthread -W -Wall -Wno-unused -Wno-parentheses -pipe -DNDEBUG -Damd64 -DARCH='"amd64"' -DLINUX -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -D_REENTRANT -D_LITTLE_ENDIAN -D_LP64=1 -c *.c
  
  
  gcc -Wl,-O1 -Wl,-version-script=mapfile -z defs -Wl,-soname=libXXX.so -static-libgcc -shared -mimpure-text -o libXXX.so *.o -lc
  
  
• MT-Safe, Position Independent: Use -fPIC -pthread -D_REENTRANT.
  The -D_REENTRANT should be used for all compiles and links, I don't know all the details on the use of this macro, I just know we use it. I suspect that the use of -pthread means that you also get -D_REENTRANT, but I'm not 100% sure of that on all versions of Linux or GNU compilers, so you should probably check the compiler you are using. In addition we use -Di586 -DARCH='"i586"' -DLINUX -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -D_LITTLE_ENDIAN on X86 and -DAMD64 -DARCH='"AMD64"' -D_LP64=1 on AMD64. The -KPIC option makes sure your code is built in a position-independent way to maximum use in a shared library.
• Register Usage: Use -fno-omit-frame-pointer.
  It is important that these libraries have frame pointer register usage, see the above comments on the Solaris -xregs=no%frameptr option.
• Pointer aliasing: Use -fno-strict-aliasing.
  Avoid this pointer optimization, on Solaris compilers this optimization kicks in at -xO5, and you'd want to avoid it there too. If you want this optimization you need to make sure that all pointer usage in the native code follows the rules specified by this option.
• Optimizations: Much of the JDK is built with -O3, and the rest with -O2.
  The same general comments on optimizations apply to this compiler as to the above Sun compiler. Be careful as you increase the optimization level. The -O2 level has proven to be relatively safe.
• Quality: Use -W -Wall -Wno-unused -Wno-parentheses -Werror.
  Although not required, I have found these options to be helpful in tracking down problems in C code with gcc by issuing more warning errors. The -Werror turns all warning errors into fatal ones.
• Library: Use -shared -z defs -Wl,-O1 -Wl,-soname=LibraryName.
  When building the shared library, this -soname option makes sure the library name is stored inside the library. The -O1 linker option will help reduce the size of the resulting shared library.
• Version scripts (mapfiles): Use -Xlinker -version-script=mapfile.
  
• Library for java_g: The library must be named with a _g.so suffix