Skip to main content

HotSpot development on Linux with NetBeans - Part 1

Posted by simonis on January 24, 2008 at 4:31 PM PST

Here comes yet another step-by-step tutorial which explains how to fetch the OpenJDK sources, compile them and work with them inside the NetBeans IDE. It focuses on building and running the different flavors (opt/debug, client/server JIT compiler, template/C++ interpreter) of the HotSpot VM on Linux/x86 and concludes with a short evaluation of NetBeans 6.0 as an development environment for HotSpot hacking.

Because it was two big for a single blog entry, it is split up into two parts: this first part explains how to build and run the HotSpot while the NetBeans integration is described in the second part.

If you're interested in building on Windows you can consult Ted Nedwards blog. A description of how to build different Java-only parts of the JDK with NetBeans 6 can be found here (note that the current NetBeans "world" project under jdk/make/netbeans which is supposed to build the entire JDK (including the HotSpot) doesn't work anymore after the Mercurial switch and the resulting directory restructuring).

Prerequisites - Mercurial, Forest extension, Freetype, Findbugs, CUPS

I started to develop on a Suse Enterprise Linux 9.3 server with 4 Intel Xeon CPUs at 3GHz with 4GB of memory. It had gcc 3.3.3 and gnumake 3.80 installed by default which both suffice for OpenJDK development.

The following subsections will detail how I installed various required software packages. I usually compile and install new software into /share/software and link the resulting executables to /usr/local/bin which comes first in my PATH environment variable.

Python

Also my box had Python 2.4 installed, I decided to install a fresh, 2.5 version of Python. You probably don't have to repeat this step because Mercurial should work perfectly fine with Python 2.4.

> cd /share/software
> tar -xzf Python-2.5.1.tgz

> cd Python-2.5.1/
> mkdir /share/software/Python-2.5.1_bin
> configure --prefix=/share/software/Python-2.5.1_bin
> make
> make install
> ln -s /share/software/Python-2.5.1_bin/bin/* /usr/local/bin/

 

Mercurial

After this step I downloaded, compiled and installed Mercurial. As mentioned above, you'll be probably fine if you use your default Python installation for this step:

> cd /share/software
> tar -xzf mercurial-0.9.5.tar.gz
> cd mercurial-0.9.5/
> make install-bin PYTHON=/share/software/Python-2.5.1_bin/bin/python PREFIX=/share/software/hg

> ln -s /share/software/hg/lib/python2.5/site-packages/* /share/software/Python-2.5.1_bin/lib/python2.5/site-packages/
> ln -s /share/software/hg/bin/hg/* /usr/local/bin/
> cat > ~/.hgrc
[ui]
username = Volker H. Simonis
^D
> hg debuginstall

The last command hg debuginstall, should complete without error and should produce the following output:

Checking encoding (UTF-8)...
Checking extensions...
Checking templates...
Checking patch...
Checking merge helper...
Checking commit editor...
Checking username...
No problems detected

 

The Forest Extension

After I had a working Mercurial, I could use it to get the Forest extension which isn't strictly needed but which will simplify the download of the OpenJDK sources. Note that you'll have to set the http_proxy environment variable to point to your http proxy server if you're behind a firewall (e.g. http_proxy=http://proxy:8080).

cd /share/software
hg clone http://www.terminus.org/hg/hgforest
ln -s /share/software/hgforest/forest.py /share/software/hg/lib/python2.5/site-packages/hgext/

 

Cloning the OpenJDK sources

Now comes the big moment. I fired up a hg fclone command to clone the OpenJDK sources. If everything works fine, this should download about 28.000 files to your local machine. The output should look as follows (stripped-down..):

> cd /share/software
> mkdir OpenJDK
> cd OpenJDK

> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
[.]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 26 changes to 26 files
26 files updated, 0 files merged, 0 files removed, 0 files unresolved

[corba]

... ... ...

[langtools]
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2974 changes to 2974 files
2974 files updated, 0 files merged, 0 files removed, 0 files unresolved

There may still be a problem here, if you're behind a firewall. Instead of getting the sources, you may get the following error:

> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
[.]
abort: error: Name or service not known

This is because of a bug in fclone which doesn't honor the setting of the http_proxy environment variable (although hg alone does use it). Fortunately, this can be fixed easily by setting the http proxy in the ~/.hgrc configuration file like this:

[http_proxy]
host=proxy:8080

That's it! Now we should have the sources. In order to build them, we still need to install some third party libraries like Freetype, Findbugs and Cups and of course the binary encumbrances which are bundled in the appropriate binary plugs file. You may skip any of the following installation steps if your Linux distribution already has the development packages installed for the corresponding library.

Freetype

Let's start with Freetype:

> cd /share/software/OpenJDK
> tar -xzf freetype-2.3.5.tar.gz

> cd freetype-2.3.5/
> vi include/freetype/config/ftoption.h
uncomment /* #define TT_CONFIG_OPTION_BYTECODE_INTERPRETER */
> mkdir ../freetype-2.3.5_bin
> ./configure --prefix=/share/software/OpenJDK/freetype-2.3.5_bin/
> make
> make install

In order to make the Freetype library accessible for the OpenJDK build, we can set the following environment variables:

> export ALT_FREETYPE_LIB_PATH=/share/software/OpenJDK/freetype-2.3.5_bin/lib
> export ALT_FREETYPE_HEADERS_PATH=/share/software/OpenJDK/freetype-2.3.5_bin/include

Another possibility to make a library available during the OpenJDK build is to set the corresponding variable on the make command line. This is the approach that I'll use later on in the build section. Nevertheless, I'll also list the export statements after the installation of each library for the sake of completeness.

Findbugs

The same thing for Findbugs ..

> cd /share/software/OpenJDK
> tar -xzf findbugs-1.3.1.tar.gz
> export FINDBUGS_HOME=/share/software/OpenJDK/findbugs-1.3.1

 

Cups

.. and Cups:

> cd /share/software/OpenJDK
> tar -xzf cups-1.3.5-source.tar.gz

> cd cups-1.3.5/
> mkdir ../cups-1.3.5_bin
> ./configure --prefix=/share/software/OpenJDK/cups-1.3.5_bin
> make
> make install
> export ALT_CUPS_HEADERS_PATH=/share/software/OpenJDK/cups-1.3.5_bin/include

 

Binary plugs, Boot JDK and Ant

Finally we need to get the Binary plugs and a boot JDK (at least Java 6). I already had Java 6 and Ant installed in /share/software/Java/1.6.0 and /share/software/Ant/1.6.4 respectively, so I only had to download and install the Binary plugs and make all of them known to the OpenJDK build:

> cd /share/software/OpenJDK
> java -jar jdk-7-ea-plug-b24-linux-i586-04_dec_2007.jar
> export ALT_BINARY_PLUGS_PATH=/share/software/OpenJDK/jdk-7-ea-plug-b24-linux-i586-04_dec_2007/openjdk-binary-plugs
> export ALT_BOOTDIR=/share/software/Java/1.6.0
> export ANT_HOME=/share/software/Ant/1.6.4/

With this step we ultimately finished the necessary preparations and can now happily proceed to build the OpenJDK!

Building the OpenJDK

Before we start the build, we should first run the sanity check to see if our settings and the available tools and libraries are sufficient for the build. Because I don't like to clutter my environment, I'll set all the needed environment variables on the command line, such that they only affect the current command as follows:

> cd /share/software/OpenJDK/jdk7
> LANG=C \
  FINDBUGS_HOME=/share/software/OpenJDK/findbugs-1.3.1 \
  ANT_HOME=/share/software/Ant/1.6.4/ \
  ALT_CUPS_HEADERS_PATH=/share/software/OpenJDK/cups-1.3.5_bin/include \
  ALT_BOOTDIR=/share/software/Java/1.6.0 \
  ALT_BINARY_PLUGS_PATH=/share/software/OpenJDK/jdk-7-ea-plug-b24-linux-i586-04_dec_2007/openjdk-binary-plugs \
  ALT_FREETYPE_LIB_PATH=/share/software/OpenJDK/freetype-2.3.5_bin/lib \
  ALT_FREETYPE_HEADERS_PATH=/share/software/OpenJDK/freetype-2.3.5_bin/include \
  make sanity

You shouldn't proceed forward, until the sanity check completes without any errors or warnings. The checks performed by the sanity check are still quite weak and a passed sanity check is no guarantee for a successful build. The sanity check for example just verifies that ALT_BINARY_PLUGS_PATH points to a valid directory, but not if that directory really contains the binary plugins!

Internally, SUN apparently still uses gcc 3.2.2 to build the JDK and with gcc 3.2.2 there seem to be no warnings during the build so they decided to use the -Werror option on Linux which instructs gcc to treat every compiler warning as error. If you however want to build with a gcc version higher than 3.2.2 (and you'll probably want to do this on a newer Linux distribution) you'll either have to use precompiled headers (USE_PRECOMPILED_HEADER=true) or comment the line WARNINGS_ARE_ERRORS = -Werror in the file hotspot/build/linux/makefiles/gcc.make (see Bug 6469784).

Using precompiled headers is probably the easier way to go for first time users and it has the additional benefit of speeding up the build considerably. But it only helps with warnings related to inlining and it has the disadvantage of hiding problems with the includeDB (see this mail thread for a discussion of the topic). If you want to use gcc 4.3 or higher, you'll probably have to disable the treatment of warnings as errors for a successful build (see for example here).

Building the corba subdirectory will fail if you haven't set ALT_JDK_IMPORT_PATH. This is because of a known bug in corba/make/common/shared/Defs.gmk (see this mail thread). You can fix the problem by inserting the following lines into corba/make/common/shared/Defs.gmk, just before the line that includes Compiler.gmk:

ifdef ALT_LANGTOOLS_DIST
  LANGTOOLS_DIST :=$(call FullPath,$(ALT_LANGTOOLS_DIST))
else
  LANGTOOLS_DIST =
endif

After this last patch, we can finally start the build. I'll focus here on debug builds because you're probably a developer if you read this and as a developer you're probably interested in a debug build (after all you could download a product build, so it would not be worth the work).

> cd /share/software/OpenJDK/jdk7
> LANG=C \
  FINDBUGS_HOME=/share/software/OpenJDK/findbugs-1.3.1 \
  ANT_HOME=/share/software/Ant/1.6.4/ \
  ALT_CUPS_HEADERS_PATH=/share/software/OpenJDK/cups-1.3.5_bin/include \
  ALT_BOOTDIR=/share/software/Java/1.6.0/ \
  ALT_BINARY_PLUGS_PATH=/share/software/OpenJDK/jdk-7-ea-plug-b24-linux-i586-04_dec_2007/openjdk-binary-plugs \
  ALT_FREETYPE_LIB_PATH=/share/software/OpenJDK/freetype-2.3.5_bin/lib \
  ALT_FREETYPE_HEADERS_PATH=/share/software/OpenJDK/freetype-2.3.5_bin/include \
  HOTSPOT_BUILD_JOBS=5 \
  ALT_PARALLEL_COMPILE_JOBS=5 \
  USE_PRECOMPILED_HEADER=true \
  SKIP_DEBUG_BUILD=false \
  SKIP_FASTDEBUG_BUILD=true \
  DEBUG_NAME=debug \
  ALT_OUTPUTDIR=/share/software/OpenJDK/jdk7/build/openjdk_full_debug \
  make 2>&1 | tee /share/software/OpenJDK/jdk7/build/openjdk_full_debug.log

Note that we don't use the make_debug target because there's a bug in the top-level Makefile that ignores the ALT_OUTPUTDIR if that target will be used (see this mail thread). You should be also aware of the fact, that the build will always create an empty directory named -fastdebug which can be ignored and removed.

It is also advisable to save the build output in a file. This can be achieved by redirecting the whole output to tee as shown in the call to make above. tee is a utility that duplicates its input to the standard output and to an additional file. This file can be consulted later on if there have been build problems or if we just want to know how a file has been built and where the resulting object files have been placed to.

Following the above pattern it is also possible to build a product or a fastdebug build. You just have to set SKIP_DEBUG_BUILD=true SKIP_FASTDEBUG_BUILD=false DEBUG_NAME=fastdebug for a fastdebug build and SKIP_DEBUG_BUILD=true SKIP_FASTDEBUG_BUILD=true for a product build.

Notice that ALT_PARALLEL_COMPILE_JOBS is currently only honored by the corba and the jdk subprojects while the hotspot subproject uses HOTSPOT_BUILD_JOBS as indicator to do a parallel build. Unfortunately, due to another bug, neither ALT_PARALLEL_COMPILE_JOBS nor HOTSPOT_BUILD_JOBS is handed over from the top-level makefile to the hotspot makefile. However, this can be easily fixed by adding the following lines to /make/hotspot-rules.gmk, just before the hotspot-build target:

ifneq ($(PARALLEL_COMPILE_JOBS), 1)
  HOTSPOT_BUILD_ARGUMENTS += HOTSPOT_BUILD_JOBS=$(PARALLEL_COMPILE_JOBS)
endif

With this change, setting ALT_PARALLEL_COMPILE_JOBS on the make command line will be enough to trigger a parallel hotspot build, which should be considerable faster on a multi-processor machine (a good default setting for ALT_PARALLEL_COMPILE_JOBS is hard to predict, but 1.5xNrOfCPUs should be a good starting point).

Sooner or later (depending on your machine and the right setting of ALT_PARALLEL_COMPILE_JOBS:) the build should finish (hopefully without any error). Among others, this will create the following subdirectories in the build location that was specified with ALT_OUTPUTDIR:

bin
corba
j2sdk-image
lib
hotspot
include
j2re-image
langtools

corba, hotspot and langtools contains the build results of the corresponding subprojects. bin, include and lib contain the binaries, include files and libraries that are contained in the corresponding directories of a Java SDK or RE distribution. Finally, j2sdk-image and j2re-image contain a complete image of a Java SDK or RE distribution, assembled from the subdirectories of the build subdirectory. If everything went fine, we can now call bin/java (or j2sdk-image/bin/java or j2re-image/bin/java which is all the same) to verify our build:

> /share/software/OpenJDK/jdk7/build/openjdk_full_debug/bin/java -version
openjdk version "1.7.0-internal-debug"
OpenJDK Runtime Environment (build 1.7.0-internal-debug-dXXXXXX_04_jan_2008_11_27-b00)
OpenJDK Server VM (build 12.0-b01-jvmg, mixed mode)

That looks really nice, isn't it!!! We managed to built a complete debug version of the OpenJDK from scratch!

Building the HotSpot

Now that we've successfully built the OpenJDK and started hacking the HotSpot VM, we probably don't want to go through all this hassle just to verify that a small VM change compiles and works correctly. Luckily, the HotSpot developers at SUN didn't wanted either, so they provided an elegant way to rebuild the HotSpot part of the VM and test it.

And here is how it works. Go to the hotspot/make directory and execute the following make command:

> LANG=C \
  ALT_BOOTDIR=/share/software/Java/1.6.0/ \
  HOTSPOT_BUILD_JOBS=5 \
  ALT_OUTPUTDIR=../../build/hotspot_debug \
  make jvmg jvmg1 2>&1 | tee ../../build/hotspot_debug.log

As you can see, there are considerably fewer variables needed to build the VM (in fact the only real dependency is the boot JDK specified with ALT_BOOTDIR). By selecting the corresponding build target it is possible to build debug builds (jvmg and jvmg1 targets), fastdebug builds (fastdebug and fastdebug1 targets), optimized builds (optimized and optimized1 targets) and product builds (product and product1 targets). A "1"-suffix in the target name indicates that the client version (the one with the C1 JIT compiler) will be build while a target name without suffix will build the server version (the one with the C2 JIT compiler) of the corresponding VM. A fastdebug build is an optimized build with assertions (C/C++ style asserts in the VM code) enabled. An optimized build has no assertions, while a product build is an optimized build without assertions and -DPRODUCT defined (this may for example disable non-product switches in the resulting VM).

With this information in mind, you'll probably easily guess that the last make command builds the debug version of the client and the server VM. The results will be placed in the corresponding __compiler1/ and __compiler2/ subdirectories of the output directory with the little anomaly that debug build will be placed in the jvmg subdirectory (i.e. linux_i486_compiler1/jvmg/ and linux_i486_compiler2/jvmg/ in this example).

Running the HotSpot

These directories not only contain the HotSpot VM as a shared library (libjvm.so) but also a small executable called gamma. I don't really know the origin of this name (perhaps someone of the geeks can comment on this?), but it is a really convenient possibility to test the newly created VM:

> build/hotspot_debug/linux_i486_compiler1/jvmg/gamma -version
build/hotspot_debug/linux_i486_compiler1/jvmg/gamma: \
error while loading shared libraries: libjvm.so: \
cannot open shared object file: No such file or directory

> LD_LIBRARY_PATH=build/hotspot_debug/linux_i486_compiler1/jvmg \
  build/hotspot_debug/linux_i486_compiler1/jvmg/gamma -version
JAVA_HOME must point to a valid JDK/JRE to run gamma
Error: could not find libjava.so
Error: could not find Java 2 Runtime Environment.

> LD_LIBRARY_PATH=build/hotspot_debug/linux_i486_compiler1/jvmg \
  JAVA_HOME=build/openjdk_full_debug/j2sdk-image \
  build/hotspot_debug/linux_i486_compiler1/jvmg/gamma -version
openjdk version "1.7.0-internal-debug"
OpenJDK Runtime Environment (build 1.7.0-internal-debug-dXXXXXX_04_jan_2008_11_27-b00)
OpenJDK Client VM (build 12.0-b01-internal-jvmg, mixed mode)

As you can see, we just have to put the directory which contains the desired libjvm.so (client or server) into the LD_LIBRARY_PATH and define JAVA_HOME to point to a valid JDK or JRE (e.g. the one we built in the first step). gamma is a simple launcher intended for internal engineering test. It is build from hotspot/src/os/linux/launcher/java.c and hotspot/src/os/linux/launcher/java_md.c (search for launcher.c in the build log to see the details). hotspot/src/os/linux/launcher/java.c and hotspot/src/os/linux/launcher/java_md.c are stripped down versions of the real Java launcher sources jdk/src/share/bin/java.c and jdk/src/solaris/bin/java_md.c from the jdk workspace. The gamma launcher misses some of the logic of the default Java launcher which finds the corresponding VM and JDK automatically. Therefore it is necessary to signal their location by setting the LD_LIBRAY_PATH and JAVA_HOME environment variables. The big advantage however is the fact that it builds within the hotspot project without any dependency on the jdk workspace and comes in quite handy for fast development and testing.

Just as a side note: you probably wondered why I wrote jdk/src/solaris/bin/java_md.c in the previous paragraph, for the location of java_md.c in the jdk workspace. The solaris part was not a typo. The jdk workspace isn't that well structured like the hotspot workspace which divides platform and architecture dependant files into the corresponding os, cpu and os_cpu subdirectories. Instead, in the jdk workspace there's just a windows directory for the Windows native code and a solaris directory which contains all the Unix native code (separated by ifdefs if necessary). (In fact there is a linux subdirectory, but it only contains the linux man pages and no code). This may identify as serious problem for the various porting projects which attempt to port the OpenJDK to other Unix-like operation systems.

Currently, the gamma launcher is somewhat outdated (hotspot/src/os/linux/launcher/java.c is based on the 1.6.0-b28 JDK version of jdk/src/share/bin/java.c as stated in the file) but it is still sufficient to test the HotSpot VM. Hopefully it will be updated, as the JDK7 development moves forward.

Building a HotSpot with C++-Interpreter

If you want to build the C++-Interpreter instead of the default template interpreter, you have to additionally set CC_INTERP=true on the build command line. Currently the C++-Interpreter only works for the 32-bit x86 debug build and for the 32-bit opt and debug builds on SPARC (see my previous blog entry for how to get the 32-bit x86 opt and the 64-bit SPARC versions running). Notice that you'll also have to disable the the treatment of warnings as errors if you built the C++-Interpreter, because its sources generate some warnings.

For your convenience I created a patch file (in fact I run hg diff > linux32.pach in the hotspot directory) which fixes all the problems mentioned so far. To apply it, just download it and call patch as follows:

> cd hotspot/
> patch -p1 < linux32.patch
patching file build/linux/makefiles/gcc.make
patching file src/cpu/x86/vm/cppInterpreter_x86.cpp
patching file src/share/vm/interpreter/bytecodeInterpreter.cpp

You should now be able to build a debug version of the C++-Interpreter enabled HotSpot with the following command:

> LANG=C \
  ALT_BOOTDIR=/share/software/Java/1.6.0/ \
  HOTSPOT_BUILD_JOBS=5 \
  ALT_OUTPUTDIR=../../build/hotspot_CC_INTERP_debug \
  CC_INTERP=true \
  make jvmg jvmg1 2>&1 | tee ../../build/hotspot_CC_INTERP_debug.log

Notice that building an interpreter-only VM isn't currently supported out of the box with the current top-level HotSpot makefiles. Previously, the CORE targets (debugcore, jvmgcore, fastdebugcore, optimizedcore, profiledcore and productcore) could be used for this purpose and building the HotSpot that way, gave you a pure, interpreter-only VM. However now, as Tom Rodriguez explains "..core is just a system without a compiler but still including the classes needed by the compiler like nmethod so it's slightly less minimal ... and the core makefile targets should still work". These core targets are defined in the platform-dependant makefiles hotspot/build//Makefile only, and not in hotspot/make/Makefile so you'll probably have to hack hotspot/make/Makefile to build them.

Developing with NetBeans

See HotSpot development on Linux with NetBeans - Part 2

 

References

[1] Kelly O'Hair's Build Cheat Sheet
[2] Kelly O'Hair's Glossary for JDK Builds
[3] OpenJDK Build Readme
[4] NetBeans C/C++ Support Quick Start Tutorial
[5] Interview with Gordon Prieur about Sun Studio 12

Related Topics >>

Comments

Dear Volker Simonis It is nice to read your Blog, this blog is excelllent for the beginners like me. I am a beginner in building the Openjdk and have faced some problems that would need your help. I have followed part of your procedures in building debug mode Openjdk. Here are some problems that I encountered: 1) The forest extension of hg (i.e. fclone) didn't work even though I had download, compiled and installed it. So that's why I download the Openjdk from the image rather than using hg. (I think this problem is not critical) 2 (critical) ) I have followed your following steps to build the debug version of openjdk: HOTSPOT_BUILD_JOBS=5 \ ALT_PARALLEL_COMPILE_JOBS=5 \ USE_PRECOMPILED_HEADER=true \ SKIP_DEBUG_BUILD=false \ SKIP_FASTDEBUG_BUILD=true \ DEBUG_NAME=debug \ ALT_OUTPUTDIR=/share/software/OpenJDK/jdk7/build/openjdk_full_debug \ make 2>&1 | tee /share/software/OpenJDK/jdk7/build/openjdk_full_debug.log After setting those environment variables, it started compiling and ended successfully but the build location is still under the product version's location which is under /usr/download/openjdk/openjdk/build/linux-i586. I wonder that if it is still compiled into product version which is shown as follow: [root@localhost bin]# ./java -version openjdk version "1.7.0-internal" OpenJDK Runtime Environment (build 1.7.0-internal-root_2008_12_04_16_04-b00) OpenJDK Client VM (build 14.0-b07, mixed mode) I tried to add some println code in hotspot and compiled it again in order to see whether the change was effective but seems the changes did not show any output ~ Any build process I had missed out? I hope you could help me, thanks~

All the files in the "incls/" subdirectory are generated files. You can find them in the output directory (i.e. the directory you build into) under "<os>_<arch>_compiler<1|2>/generated/incls/" depending on your platform (e.g. "linux_i486_compiler2/generated/incls/_precompiled.incl"). Of course you have to build the HotSpot first in order to find these files. The "DT_RETURN_MARK" macros are defined just at the top of "hotspot/src/share/vm/prims/jni.cpp". Regards, Volker