Skip to main content

Getting started with Mercurial and OpenJDK on Solaris

Posted by lancea on April 24, 2009 at 7:34 AM PDT

A gentle guide to getting started with OpenJDK and Mercurial on Solaris


Technorati Tags: ,

As I am about ready to start checking in some bug fixes and enhancements to the JDBC and RowSet code in Java SE, i figured i should finally spend some time configuring my Solaris 10 box for Mercurial. It was a bit of an adventure, so I hope the following will help you avoid a few of the pitfalls that I encountered.

I decided that I wanted to avoid building Mercurial myself, figuring using pre-built version would be the way to go. The Mercurial web site suggests to download the binary from sunfreeware.com or blastwave.org for Solaris. I decided to get the software from sunfreeware.com. As I am running on Solaris 10, i need to go to the Solaris 10 section on sunfreeware.com. In order to install Mercurial, you will need to get the following packages (assuming you did not have them installed):

After downloading the packages, su to root and execute the following commands:

gunzip mercurial-1.1.2-sol10-sparc-local.gz 
gunzip libgcc-3.4.6-sol10-sparc-local.gz
gnuzip python-2.5.1-sol10-sparc-local.gz
pkgadd -d mercurial-1.1.2-sol10-sparc-local
pkgadd -d libgcc-3.4.6-sol10-sparc-local
pkgadd -d python-2.5.1-sol10-sparc-local
pkgadd -d openssl-0.9.8k-sol10-sparc-local

Note: Please make sure you grab the correct version of openssl as the build differs based on your platform.

Everything installed cleanly and an hg version seemed to work fine. So next i was ready to install the Forest Extension which is required by OpenJDK(Note: I would create a directory to host your Mercurial extensions and workspaces and cd to it prior to running the following hg command):

However, when I went to run the above command, i encountered the following error:

ace2 183 =>/usr/local/bin/hg clone
http://bitbucket.org/pmezard/hgforest-crew
real URL is http://bitbucket.org/pmezard/hgforest-crew/
destination
directory: hgforest-crew
Traceback (most recent call last):
File "/usr/local/bin/hg", line 20, in
  mercurial.dispatch.run()
File "/usr/local/lib/python2.5/site-packages/mercurial/dispatch.py",
line 20, in run
  sys.exit(dispatch(sys.argv[1:]))
File "/usr/local/lib/python2.5/site-packages/mercurial/dispatch.py",
line 29, in dispatch
  return _runcatch(u, args)
File "/usr/local/lib/python2.5/site-packages/mercurial/dispatch.py",
line 79, in _runcatch
  except revlog.RevlogError, inst:
File
"/usr/local/lib/python2.5/site-packages/mercurial/demandimport.py", line
74, in __getattribute__
  self._load()
File
"/usr/local/lib/python2.5/site-packages/mercurial/demandimport.py", line
46, in _load
  mod = _origimport(head, globals, locals)
File "/usr/local/lib/python2.5/site-packages/mercurial/revlog.py",
line 20, in
  _compress = zlib.compress
File
"/usr/local/lib/python2.5/site-packages/mercurial/demandimport.py", line
74, in __getattribute__self._load()
File
"/usr/local/lib/python2.5/site-packages/mercurial/demandimport.py", line
46, in _loadmod = _origimport(head, globals, locals)
ImportError: ld.so.1: python: fatal: relocation error: file
/usr/local/lib/python2.5/lib-dynload/zlib.so: symbol inflateCopy:
referenced symbol not found

I would also encounter the same error above if I did the following:

/opt/csw/bin/python -c "import zlib"

Googling did not shed a whole lot of light outside of others have seen this before. I was able to get past the error above by putting /usr/local/bin at the front of my PATH variable and unsetting my LD_LIBRARY_PATH variable. However, i then encountered the following error when I tried again to get the Forest Extension:



abort: could not import module _md5!

You will encounter this error if you have not installed openssl.

As I did not find out at the time that I needed openssl, I decided to give up and try installing using the binaries from blastwave.org. The installation instructions can be found here. However, here are the basic steps(Note: you must be logged in as root):

First you must get pkgutil installed:

  • pkgadd -d http://blastwave.network.com/csw/pkgutil_`/sbin/uname -p`.pkg (select default )
  • mkdir /etc/opt/csw
  • cp -p /opt/csw/etc/pkgutil.conf.CSW /etc/opt/csw/pkgutil.conf
  • /opt/csw/bin/pkgutil --catalog
  • /opt/csw/bin/pkgutil --install gnupg textutils (select Y)
  • /opt/csw/bin/gpg --keyserver pgp.mit.edu --recv-keys A1999E90
  • /opt/csw/bin/gpg --list-keys
  • /opt/csw/bin/gpg --edit-key A1999E90 (select Trust and then option 5)
  • vi /etc/opt/csw/pkgutil.conf and uncomment out use_gpg and use_md5

Now we are ready to install the packages:

  • pkgutil --install common
  • pkgutil --install python
  • pkgutil --install mercurial

Set your PATH so that /opt/csw/bin is at the front of your PATH:

setenv PATH /opt/csw/bin:${PATH}

or

PATH=/opt/csw/bin:${PATH};export PATH

Let us see if we can get hg going after our install:

# /opt/csw/bin/hg
Traceback (most recent call last):
File "/opt/csw/bin/hg", line 11, in
  from mercurial import demandimport; demandimport.enable()
ImportError: No module named mercurial

Googling indicated that I would need to possibly set PYTHONPATH to where my site-packages directory was:

PYTHONPATH=/opt/csw/lib/python/site-packages/

OK, let us try again:

hg version
Mercurial Distributed SCM (version 1.1.2)

Copyright (C) 2005-2008 Matt Mackall  and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Success!

Now, one thing that I noticed was that when I did the install of mercurial and python, I ended up with a python and python2.5 directory in /opt/csw/lib. The sole contents of python was the site-packages directory and this directory was for some reason not in the python2.5 directory. I decided to create a symbolic link so that the site-packages was accessible via the python2.5 directory and then unset the PYTHONPATH variable and see if I had better luck with hg and I did. I had a fully working mercurial environment!

Next it was time to try to bring over the latest Forest Extension again via (remember to cd to where you want to install the forest extension):

hg clone http://bitbucket.org/pmezard/hgforest-crew/

This worked fine. Next, in my home directory, I created a .hgrc file which looks like:

[ui]
username=lancea
[extensions]
forest=/ace2_apps/openjdk/hgforest-crew/forest.py
fetch=

The username property is set to what I want my checkin comments to use as my name. The forest property is set to where I installed the Forest Extension.

Next I want to make sure that hg can see the Forest Extension that I just configured in my .hg file:

ace2 193 =>hg
Mercurial Distributed SCM

basic commands:

add        add the specified files on the next commit
annotate   show changeset information per file line
clone      make a copy of an existing repository
commit     commit the specified files or all outstanding changes
diff       diff repository (or selected files)
export     dump the header and diffs for one or more changesets
fclone     make a clone of an existing forest of repositories
fpull      pull changes from the specified forest
fpush      push changes to the specified forest.
fstatus    show changed files in the working forest
fupdate    update working forest
init       create a new repository in the given directory
log        show revision history of entire repository or files
merge      merge working directory with another revision
parents    show the parents of the working dir or revision
pull       pull changes from the specified source
push       push changes to the specified destination
remove     remove the specified files on the next commit
serve      export the repository via HTTP
status     show changed files in the working directory
update     update working directory

use "hg help" for the full list of commands or "hg -v" for details

If your .hgrc is set up correctly you should see commands such as fclone and fupdate in the list of basic hg commands.

Now you are ready to create a clone an OpenJDK 6 or OpenJDK 7 project. For OpenJDK 7, I am going to clone the tl project forest as this is where the JDBC code lives using the fclone command:

If you want to clone the OpenJDK 6 project, you would do the following:

There is also a sandbox play area within the OpenJDK project are so that you can get familiar with mercurial and you can clone a copy using the following commands:

This is a great way to get familiar with hg commands.

In order to push back changesets you will need to generate an ssh key and email it to keys@openjdk.java.net. Use the ssh-keygen command and make sure to specify a passphrase:

  • ssh-keygen -t rsa -b 4096

This will generate a ~/.ssh/id_rsa.pub text file containing the public ssh key which you will email to keys@openjdk.java.net. In order to be able to push changesets back to OpenJDK projects, you must be granted permission. Please see Producing a Changset and How to Contribute for details.

I hope you found this blog useful and hopefully it will help you overcome a few of the the speed-bumps that I encountered. I also want to thank Kelly O'Hair who helped keep me sane and made sure that I was not missing the obvious.

May I also suggest that you check out Project Kenai, a new Sun sponsored web site for hosting your Open Source projects that also supports Mercurial. This is where the JavaMail RI and the Corba Glassfish V3 RI projects reside.

Additional resources on Mercurial and OpenJDK can be found at:

  1. Mercurial: The Definitive Guide
  2. OpenJDK Developer's Guide

Comments

Thanks for the comments. Perhaps the forest URLs are now the same, they were not originally and the mercurial site still points to the other, but good to know. Are you sure your remote site has the forest extension configured?

The Forest extension version from http://hg.akoha.org/hgforest works fine for me for Mercurial 0.9.5, 1.1, 1.2.1. Actually, it's the same repository as you mentioned in http://bitbucket.org/pmezard/hgforest-crew/ By the way, I have an issue how to set up an access to my forest of repositories over http. I set up multiple repositories on my local web server, and has access to them over http. I can successfully do hg clone http:///hg hg clone http:///hg/my_repo hg clone http:///hg/my_repo1 However, I always get the following error when I do fclone: % hg --debug fclone http:///hg using http:///hg sending between command sending capabilities command capabilities: unbundle=HG10GZ,HG10BZ,HG10UN lookup changegroupsubset abort: Remote forests cannot be cloned because the other repository doesn't support the forest extension.

Yep, grabbing openssl-0.9.8k-sol10-sparc-local cured my problem with the code from sunfreeware.com though i did not hit this with the software from blastwave

Hi Misha, If you are using Mercurial 1.x you will want the newer version of the forest extension which is what i gave in the URL You installed the earlier version. I will try downloading the openssl package and then update the blog...thanks for the tip!

Lance, I also had problem like abort: could not import module _md5! but it's gone after I installed openssl package on Solaris. Also I got the forest extension from hg.akoha.org/hgforest which works with all the versions of Mercurial.