Kelly O'Hair's Blog

September 2007 Archives

OpenJDK Mercurial Transition Update 3

Update 3

Build 20 now contains a separate "langtools" (javac, javah, javap, apt, and javadoc) directory in the Build 20 source bundles.

Build 21 (could be 22) will have separate corba, jaxp, and jaxws directories. Build 20 had some duplicate javac tests between j2se and langtools, this should get corrected in Build 21, which should be out soon.

Build 22 continues to be our target for the last JDK7 promotion built via the TeamWare workspaces, the Build after this one would be done via Mercurial repositories. Then my co-workers might want to burn me at the stake, so I may go into hiding for a while. ;^)

We have started to look at the various conventions we want to put in place for things like changeset comments, etc. The general feeling is that the changeset comment should always contain a bugid and synopsis, and not much else, pushing the bulk of the data about a bug and it's fix into the bug database. The Mercurial changeset itself serves as the exact changes for a bug, so there is no need to save diffs or webrevs, just the changeset global ID.

It has been agreed that the release engineering team will be creating global tags for the various promotions, something like "jdk7-ea-b23", so re-creating the sources as of a promotion should be easy. (The fact that Mercurial makes it so easy to re-create accurate source trees for any changeset or tag name may change the way we deal with saving source bundles, or even creating source bundles. Time will tell.

There may still be a few SCCS keywords in the sources at first, but these should be harmless. We'll continue to clean up the source as we go.

The current plan is to apply a source normalization script to the Java and Native C/C++ sources as they transition from TeamWare to Mercurial. The normalization procedure will attempt to clean up the whitespace uses in the sources, expanding TABS, converting ^M characters, removing trailing blanks on lines, and making sure the file ends with a newline character. Hopefully we can put some hooks in place to prevent this kind of whitespace clutter from coming back.

(WHAT? NO TABS? ... yes, no tabs ... ARE YOU NUTS? ... at times ... WHY? ... because tabs create a source display problem ... BUT IT WORKS FINE FOR ME IN VI/EMACS! ... yes, but what about everybody else?)

-kto

OpenJDK Mercurial Transition Update 2

Update 2

The work to create OpenJDK/JDK7 Mercurial repositories is still progressing, we had a hickup getting the langtools split off from the j2se and it did not make Build 19 as you probably well know. We integrated the split into Build 20 and have spent the last 2 weeks adjusting to the change. So Build 20 will contain a separate "langtools" (javac, javah, javap, apt, and javadoc) directory in the Build 20 source bundles, which will become a standalone Mercurial repository soon.

Build 20 also includes a great deal of Makefile changes and minor changes to many files to remove SCCS keywords in the sources and test files. We haven't fixed everything, but we are making quite a bit of progress.

In Build 21 (or Build 22 as a backup) we hope to split off corba, jaxp, and jaxws as separate openjdk repositories. (At some later date, we may be able to better interface with the corba, jaxp, and jaxws teams in terms of how they integrate and how we accept changes from these areas into the jdk product, so this is just step one). Other changes in Build 21 and 22 will be around SCCS keyword changes, and of course all the various teams are also integrating changes all around us, which is to some degree why this is taking a bit longer. Build changes often impact everyone, and we are dealing with a moving target.

We continue to target Build 22 (Build 23 as a backup) be the last promotion built via the TeamWare workspaces, the Build after this last TeamWare one would be done via Mercurial repositories. Then at some point after that, we can start working on providing those same OpenJDK repositories on the openjdk website.

In other areas, we are starting to look at the various conventions we want to put in place for things like changeset comments, etc. and how we will use Mercurial hooks to protect us from mistakes and verify the conventions are followed.

Build 20 Issues with regards to the langtools split

• You need 'ant' to build langtools, and you need to make sure the environment variable ANT_HOME is set (especially on Windows).
• When building from the control/make/Makefile, langtools is built first, then the resulting 'dist' directory is supplied to the j2se build via the variable ALT_LANGTOOLS_DIST. The langtools dist/bootstrap/javac.jar is then run with the BOOT jdk (ALT_BOOTDIR) to create all the resulting jdk classes for the build.
  IF langtools is not built or not found, then the javac launcher from the ALT_JDK_IMPORT_PATH is run (a partial jdk build). This ALT_JDK_IMPORT_PATH is assumed to be a recent jdk7 build, or at least recent enough to handle building the jdk7 itself.
  It's acceptable to set ALT_BOOTDIR and ALT_JDK_IMPORT_PATH to the same jdk7, and might even work if they both refer to the same jdk6, but no guarantees.
  ALT_JDK_IMPORT_PATH continues to be the place to import parts of a jdk that a partial build has chosen to not build, but it is also now a backup place to get a javac when the langtools dist area is not available.
• Prior to the langtools split, the jdk was built in a somewhat bootstrap process, primarily because the javac sources were part of the rest of the jdk. Now that javac is separate, and can be run with a BOOT jdk, the thought was that the entire jdk could be built without actually being run during the build process, a great simplification and a benefit to anyone doing cross-platform builds. Unfortunately, it turns out that the build procedure relied on a few tools like rmic, idlj, native2ascii, and javazic be run from the freshly built classes of these tools in the jdk source. So for the time being, these tools will be run from the BOOT jdk or run via the BOOT jdk until we can resolve these issues. The actual jdk install image (e.g. rt.jar tools.jar etc.) is now created before it is run now, although the image is massaged a little after it's initial image is created.

-kto

This work is licensed under a Creative Commons License.