Kohsuke Kawaguchi's Blog

Kohsuke Kawaguchi is a staff engineer at Sun Microsystems. He has been working on XML and XML schema languages since 2001, in particular RELAX NG, W3C XML Schema, JAXB, and JAXP. He also hosts many projects on java.net. More information on him can be found at http://www.kohsuke.org/.

Growth of Hudson plugin ecosystem

A Hudson committer Seiji Sogabe put together a chart that shows the growth of Hudson plugins. He wrote a little program that scrapes the download section of the Hudson java.net project, and determined when the first version of each plugins are released.

The blue bar shows the new plugins released in the given month, and the red bar shows the total number of plugins available to that date. While there are definitely ups and downs in the the pace of releases, he noted a substantial increase in the rate in this year — 2008 saw 55 new plugins, but in this year, we already have 44 new plugins.

We need to revise the update center so that people can find the plugins they want more easily. We also need to revise the documentation that explains how to develop a plugin, since the extensibility mechanism and the form design mechanism are both substantially simplified and modernized. I'm also hoping that I'll find some way to list up what plugins extend what extension points, which would help new plugin developers by allowing them to see examples.

Hudson adoption in the Eclipse community survey

Eclipse community did a survey on Java developers, asking various questions.

One of the questions they had was what build tool they use, and here you can see the adoption of various CI tools:

Among CI tools, Hudson is #1 with 128 votes, followed by Cruise Control with 67 votes. Bamboo is in the 3rd with 19 votes, and the rest is insignificant. So this is another validation that Hudson's adoption is going very strong.

(It's bit strange that they put Ant/Maven and Hudson/CruiseControl all in the same question, but I think we can still use this for the comparison among CI tools.)

After-JavaOne project

JavaOne is always such a big week for me (and many of us) that I need a bit of time to unwind before I go back to my regular routine. So this year, I took on a little hobby project.

My daughter is 4 year old now, and so I play with her with LEGO all the time. While building random things, I discovered that I enjoy building spheres, mainly because they are easy :-)

Then one day while I was playing with her, I thought that I could make a sphere much more smooth if I could make the studs pointing outward in all the directions (instead of just pointing upward like you normally do.)

I was thinking about how one could do this, but eventually figured out that I can basically do it the way a volleyball is assembled --- by combining 6 pieces of equal shapes around a cube.

So I first build a very small sphere in this way, by using pieces that I already have. I liked the result, so I wanted to make a bigger version. Then I thought, building it out of a single color is kind of boring, so, why don't make it a globe?

And since I'm a programmer, I hacked together a quick Java program to do this. First, a bit of geometric computation to figure out the shape of the 1/6-sphere. Then I've expanded the program to compute the Miller cylindrical projection of the resulting LEGO sphere (3 pairs of 1/6-sphere are separately color coded.)

I can then overlay this into the Miller cylindrical projection of the earth to determine what color to put where.

I've also used a LEGO CAD software to virtually build this 1/6-sphere, so that I can figure out (roughly speaking) how many pieces of what kind I need all in all.

After all that preparation work, finally on Saturday after JavaOne, I started assembling it. My daughter pitched in a little, but this is still too small for her, so I built the most of it, and here's the result.

Unlike my early prototype, this globe is fully filled inside without any empty space. I built 8 pieces of 6x6x5 brick (because of the dimensions of a lego brick, this is the perfect cube), with 3 1x4 brick with studs on the side, so that when assembled together, the resulting cubic core has studs in all the directions. I'm actually not quite happy with the result, as many of the distinctive northern hemisphere coastal shapes are no longer visible, but Africa and South America are still quite recognizable.

I'm already thinking what if I build an even bigger globe — x1.5 or even x2 on every dimension. Hmm, maybe after next JavaOne...

Starting Hudson slave from Live USB media

Ingredients

Hardware

• PC that supports USB boot
• USB stick (at least 512MB, but depends on the projects to build. I used 8GB)
• Functioning DCHP server on the network

Software

• Slax ISO image.
• Slax JDK module jdk-6u11-i586-1pst.lzm.
• Hudson swarm plugin swarm-client-1.0-jar-with-dependencies.jar

Formatting USB stick

Format a USB stick with VFAT. The drive needs to have enough space for performing builds.

Installing Slax

Slax is a modern, portable, small and fast Linux operating system with a modular approach and outstanding design. It can be directly booted from a CD or an USB drive. It's based on Slackware, and uses unionfs so that the resulting system is writable (backed by memory, thus without touching HDDs in the system.)

Use of USB stick allows us to boot a Hudson slave without modifying the HDDs in the system, and configurations can be persisted.

First, loop-mount an ISO file and copies files to a USB disk.

nanaka # mount -o loop slax-6.1.1-2.iso /mnt/floppy/
nanaka # cd /mnt/floppy
nanaka floppy # ls -la
drwxr-xr-x 4 root root 2048 2009-04-29 14:59 .
drwxr-xr-x 5 root root 4096 2009-06-07 09:29 ..
drwxr-xr-x 6 root root 4096 2009-04-18 16:39 boot
drwxr-xr-x 7 root root 4096 2009-04-18 17:10 slax
nanaka floppy # cp -a boot slax /media/disk (/media/disk is tge USB stick)

Run boot/bootinst.sh to set up MBR on the USB stick.

nanaka disk # cd /media/disk/boot        
nanaka boot # ./bootinst.sh 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
                        Welcome to Slax boot installer                         
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

This installer will setup disk /dev/sdb1 to boot only Slax.

Warning! Master boot record (MBR) of /dev/sdb will be overwritten.
If you use /dev/sdb to boot any existing operating system, it will not work
anymore. Only Slax will boot from this device. Be careful!                 

Press any key to continue, or Ctrl+C to abort...

Flushing filesystem buffers, this may take a while...
Setting up MBR on /dev/sdb...
The Master Boot Record of  /dev/sdb  has been updated.
Activating partition /dev/sdb1...
No partition table modifications are needed.
Updating MBR on /dev/sdb...
Setting up boot record for /dev/sdb1...
Disk /dev/sdb1 should be bootable now. Installation finished.

Read the information above and then press any key to exit...

This completes a USB-bootable Slax.

Installing JDK

To start a Hudson swarm plugin, we need a JDK. Slax already has a module jdk-6u11-i586-1pst.lzm, so we'll install this.

With Slax, one only needs to place modules to slax/modules directory, and they'll be automatically made available upon boot.

nanaka #cp jdk-6u11-i586-1pst.lzm /media/disk/slax/modules/

If 6u11 is too old for you, non-official package of 6u14 is available from here for your convenience.

At this point, boot from USB and continue configuration from Slax. A successful boot should result in a KDE environment:

User Account

Create an user account that runs the Hudson swarm client. We could just use root for this, but we'll create the 'hudson' user in the 'hudson' group:

root@slax:~# addgroup hudson
root@slax:~# useradd -g hudson -d /home/hudson -s /bin/bash hudson
root@slax:~# mkdir /home/hudson
root@slax:~# chown hudson:hudson /home/hudson

Installing Hudson Swarm Slave

Put the swarm client in /home/hudson and create fsrot:

root@slax:~# su - hudson
hudson@slax:~$ mkdir fsroot
hudson@slax:~$ ls 
fsroot/  swarm-client-1.0-jar-with-dependencies.jar

Auto-starting Swarm Client

Modify /etc/rc.d/rc.local to start the swarm client automatically upon boot:

root@slax:~# cat /etc/rc.d/rc.local
#!/bin/sh
#
# /etc/rc.d/rc.local:  Local system initialization script.
#
# Put any local startup commands in here.  Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.

USER=hudson
USER_HOME=/home/${USER}
CLIENT=${USER_HOME}/swarm-client-1.0-jar-with-dependencies.jar
LOG=${USER_HOME}/hudson.log

JAVA_HOME=/usr/lib/java

# Swarm client option
DESCRIPTION="SLAX Live-USB"
EXECUTORS=2
FSROOT=${USER_HOME}/fsroot
LABELS="Swarm"

hudson_start() {
if [ -e ${CLIENT} ]; then
        /bin/su - "${USER}" \
        -c \
        "${JAVA_HOME}/bin/java -jar \"${CLIENT}\" \
        -description \"${DESCRIPTION}\" \
        -executors \"${EXECUTORS}\" \
        -fsroot \"${FSROOT}\" \
        -labels \"${LABELS}\" \
        > \"${LOG}\" 2>&1  &"
fi
}

hudson_start

Similarly, tweak /etc/rc.d/rc.local_shutdown to terminate the swam client and clean up log files, etc.

root@slax:~# cat /etc/rc.d/rc.lo
rc.local           rc.local_shutdown
root@slax:~# cat /etc/rc.d/rc.local_shutdown
#!/bin/sh
#
USER=hudson
USER_HOME=/home/${USER}
LOG=${USER_HOME}/hudson.log
FSROOT=${USER_HOME}/fsroot

hudson_stop() {
        PID=`ps ax | grep swarm-client | grep -v grep | awk '{ print $1 }'`
        if [ -n "${PID}" ]; then
                /bin/kill "${PID}" > /dev/null 2>&1
        fi
        if [ -e "${LOG}" ]; then
                /bin/rm "${LOG}" > /dev/null 2>&1
        fi
        for jar in "${FSROOT}"/maven*.jar ; do
                /bin/rm "${jar}" > /dev/null 2>&1
        done
}

hudson_stop

Verify That Everything Works

Start the swarm client and make sure that everything works as expected.

root@slax:~# /etc/rc.d/rc.local
root@slax:~# ps aux | grep swarm
hudson   14414  9.4  1.2 662988 25576 pts/1    Sl   19:47   0:00 /usr/lib/java/bin/java -jar /home/hudson/swarm-client-1.0-jar-with-dependencies.jar -description SLAX Live-USB -executors 2 -fsroot /home/hudson/fsroot -labels Swarm
root     14465  0.0  0.0   2972   824 pts/1    S+   19:48   0:00 grep swarm

Verify that the client is connected by looking at the Hudson node management scren:

And its configuration screen:

Now shut down the system. Don't remove the USB stick until the system is fully halted.

Inspecting the USB Stick

Bring the USB back to another system to take a look at what's inside.

sogabe@nanaka ~/Desktop $ cd /media/disk/
sogabe@nanaka /media/disk $ ls
boot  slax
sogabe@nanaka /media/disk $ cd slax/
sogabe@nanaka /media/disk/slax $ ls
GNU_GPL  base     cheatcodes.txt  livecd.sgn    make_iso.sh  optional          rootcopy      tools
LICENSE  changes  images          make_iso.bat  modules      requirements.txt  slaxsave.zip  xino
sogabe@nanaka /media/disk/slax $ cd changes/
sogabe@nanaka /media/disk/slax/changes $ ls
dev  etc  home  lib  mnt  opt  root  tmp  usr  var
sogabe@nanaka /media/disk/slax/changes $ cd home/hudson
sogabe@nanaka /media/disk/slax/changes/home/hudson $ ls
fsroot  swarm-client-1.0-jar-with-dependencies.jar

You can see that slax/changes contains files that we just created above. If further changes are necessary, just modify these files. Slax also copies all the files in slax/rootcopy into the real '/' upon boot, so for example if you want /home/hudson/fsroot after booth, you can create slax/rootcopy/home/hudson/fsroot in the USB stick.

This completes the Live USB version of the Hudson swarm client. Slax makes it really easy to do this.

Slides for the Hudson technical session

JavaOne activities around Hudson

JavaOne is just a few days ahead now, so let me recap what's happening around Hudson during JavaOne.

On Sunday, we have Unconference (RSVP) and a party at the Thirsty Bear (RSVP to "RSVP-ThirstyBear2009 at sun dot com".) The great thing about Unconference is that we have a big block of time (6 hours, in fact, if I'm reading it right), where enough people gather. So we have enough time to really talk about where the project should go, what we need to do, and hear what others are doing. If you have a plugin coding questions, I think this would be also a good time to bring them.

On Monday, Eduardo, my boss, will kick off what I think as "the GlassFish track" in CommunityOne at 10:50am. He'll cover the entire GlassFish Portofolio, of which Hudson is a part. I believe he'll touch Hudson, and where Sun wants to take Hudson at the higher level. The next session in the same room is from Lukas Hasik and Tomas Musil titled "Test Your Product on Multiple Machines in Parallel with Hudson". This part of CommunityOne is a free event (register.)

On Wednesday, I'll be presenting a technical session "TS-5301 Continuous Integration in the Cloud with Hudson" with Jesse from 9:45am. In the evening, there's "BOF-5105 Hudson Community Meet-up" from 7:45pm, where we'll see a lot more demo from multiple folks.

Throughout CommunityOne and JavaOne, I'll be mostly at the Hudson booth in the pavilion floor. This is a great opportunity to have face-to-face time, and so especially if you want me to fix some problems on the spot, or talk about the ideas/problems/questions, please come by! This part of JavaOne is free as long as you have registration.

Finally since JavaOne attracts people from all over the world, we'll be doing "Hudson Localization Festa" by using the new Hudson translation assistance plugin. You can come to the booth, look at a few pages, choose your locale, then type the translations in there, and we'll integrate that into Hudson.

Hope to see you soon next week!

Hudson swarm slave plugin

I run a budget-less Hudson cluster, just like many of you do, and one of the challenges is to have enough computing resources in a cluster. I rely on recycled computers as the main workhorse, so I constantly look for unused computers under people's desks, and when people give them to me, I reinstall them as Hudson slaves.

In this process, I learned a lesson; people don't like to be separated from their computers, even long after they stopped using them, because they think they might occasionally need them in the future, or they may later discover that they forgot to copy some data to their new computers. That is, what's really happening is that the idea of your computer getting reformatted makes them nervous, and that is entirely understandable — if you'll never see it again, you better darn sure that you won't never ever need it, right?

Therefore, a better approach is to let them keep the computers, and instead ask them to run a slave as a virtual machine. Even better, such a set up would even let me ask them to run the VM on their newer computers, too, because it's very easy to see that a VM won't jeopardize their systems. With a little bit of VirtualBox Python API programming, I can make it so that the VM only runs when no one else is logging in, by using the memory and virtual disk that are proportional to the host system.

Or you can take this even further and just login to all the machines on the network, and just start a VM everywhere you can.

Toward that goal, I released the Hudson swarm slave plugin. This plugin comes with a little CLI agent, which uses a UDP broadcast to discover a nearby Hudson master, that's equipped with the Hudson swarm plugin. It then hooks up the slave on the fly, and joins the cluster, then Hudson will start running builds and tests on this swarm CLI agent.

With the recent improvements in Hudson to automatically install JDKs, Ants, and Mavens from the network, this instantly turns a plain-vanilla PC into a working Hudson slave, without administrators manually registering the new slave. I think this opens up a whole new way of creating a Hudson cluster, in a somewhat self-organizing fashion.

So there, Mahesh, that's how I plan to recruit your 3 beautiful Sun workstations with Core 2 Extreme CPUs. I know you are not really using them... :-)

Project of the day: SSH daemon for EC2 Windows AMIs

In my attempt to make Hudson EC2 plugin (which I blogged earlier) work with Windows AMIs, I wrote a little SSH daemon.

Here's the problem statement. Windows AMIs on EC2 do not have any built-in programmable remote access technology. This is a really poor packaging job of Amazon — their documentation talks about doing the remote desktop protocol (RDP) to talk to the server. This obviously prevents programs like Hudson to talk to the launched Windows instances.

The program I wrote is an SSH daemon, in the sense that it speaks SSH protocol, but at the same time, it's not a real SSH daemon. It's packaged specifically for EC2 use cases.

For example, it's not capable of launching a process in a different user account. Regardless of the user name that you provided to the authentication, the daemon will always create a new process with the same user that the daemon itself runs. It also only accepts a public key authentication, and you must provide a key you used to launch the EC2 instance. This is the most critical part of this daemon, which makes sure that only the authorized users can access the instance.

The daemon implements SCP and process execution part of SSH, so you can copy files both ways, and start arbitrary proceses on the server (and connect stdin/out/err to SSH client.) OTOH, Windows don't have the terminal in the same way Unix does, so launching a shell wouldn't really work.

For more about the project, see the project website.

Hudson EC2 plugin

Continuous Integration often requires a heterogeneous environments; for example, the GlassFish build requires Linux, Solaris, and Windows, and the JDK build requires something like 10 different environments, each carefully created so that we can test what we need to test. Unfortunately, heterogeneous environments reduce the resource utilization — you can easily have some Windows slaves idle while all the Solaris slaves are busy, just because you happen to have more jobs that require Solaris. So doing some kind of virtualization makes a lot of sense.

Similarly, CI jobs tend to be spiky. When you need computring resources, you need them a lot (think about a month before a release, where all kinds of tests run like crazy for almost every commit), but when you don't need them, you don't. These charateristics of CI makes it a great fit with the cloud computing technology. You can have a good resource utilization and faster turn-around time (if your tests can run in parallel.)

Toward this end, I just released the Hudson EC2 plugin. This plugin enables Hudson to automatically provision new instances on EC2, based on the system demand. That is, if Hudson notices that your system is overloaded, it will provision new slaves on EC2, and when those instances go unused for a certain time period, it will shut them down. You can run all your slaves on EC2 if you want, or you can maintain your local build cluster and use EC2 as a reserve capacity.

And here's how to use the plugin. First, go to EC2 and sign up for the service. Then enter the username/password information. Because of the way EC2 works, you also need to have an RSA private key. If you have already been using EC2 and have your own key, you can paste it here. Otherwise, you can have Hudson generate one. If you let Hudson generate one, you should save this private key in your file system as well, as you'll need this to interactively logon to EC2 instances. Use "Test Connection" button to verify that Hudson can successfully talk to EC2.

Next, configure AMIs that you want to launch. For this, you need to find the AMI IDs for the OS of your choice. ElasticFox is a good tool for doing that, but there are a number of other ways to do it. Hudson can work with any Unix AMIs. Windows is currently unsupported.

Configuring labels allow Hudson to pick the right AMI to start. For example, if all your existing slaves labeled "solaris" are fully busy and you have more builds that are tied to the "solaris" label, Hudson will start the AMIs that have the "solaris" label.

Init script is the shell script to be run on the newly launched EC2 instance, before Hudson starts launching a slave agent. If the AMI doesn't have Java pre-installed, you need to do so in this script. This is also a good place to install additional packages that you need for your builds and tests.

The load monitoring part is in the core, so the EC2 plugin really just contains some glue code that talks to EC2 to provision a new system. This means we can easily write simialar plugins for different clouds and virtualization technologies. For example, someone in the community is writing a plugin that do this with VMWare labmanager. VirtualBox lets us do something similar, with its web services (for which I wrote a JAX-WS client library some time ago, so might be fun to eat my own dogfood, so to speak.) Vivek and I also installed Eucalyptus on our spare machines, so I hope to be able to extend this plugin to work with that, too.

I'm going to talk more about this (and other things) in my JavaOne talk, so if you are coming to JavaOne, please consider dropping by!

Hudson Selenium Grid Plugin

I've released the Hudson Selenium plugin, which instantly lets you deploy Selenium Grid on top of your existing Hudson cluster. By using this plugin, you can start using Selenium Grid without installing it on individual machines in the cluster manually. This is the latest addition to the family of plugins that let you reuse you Hudson cluster infrastructure for additional purposes.

Once you install this plugin from the update center, Hudson master will become the Selenium Grid hub, the center of the Selenium Grid system. Similarly, each slave that you have in your cluster will now run Selenium RCs. The whole process happens completely automatically, without any additional configuration.

To connect to this grid from your Selenium tests, create a DefaultSelenium instance by specifying the host name of your Hudson master, like this:

new DefaultSelenium("hudson.mydomain", 4444, "*firefox", "http://site.that.iam.testing/");

By default, Selenium Grid randomly chooses a Selenium RC to carry out the execution, but this behavior makes it difficult to reliably start a platform dependent browser (like IE and Safari), that needs to be run on specific computers. As such, it's convenient to be able to specify the slave that runs the browser. To do this, Hudson Selenium Grid plugin extends the stock Selenium Grid by extending the browser identifier. Specifically, you can specify "label1&label2&...:browser" to tell Selenium Grid to start the specified browser on a slave that has all the given labels. For example, if your Windows slaves have the "windows" label, the browser identifier of "windows:*iexplore" would cause a Selenium RC on some Windows slave to start the IE.

We are increasingly seeing tools like Hadoop and Selenium, which requires a cluster of computers to operate. Given the rise of multi-core systems, cloud computing, and other general shift for increased parallelism, I expect this trend to continue. Now, the problem of those tools is that it's harder to set them up and keep them going. One of the things I want to do with Hudson is to enable their deployments on the Hudson cluster. I think this is a convincing value proposition, given that Hudson byitself is pretty easy to install and maintain (even on a cluster), and that it already has an update center built-in.

I intend to talk more about this in my JavaOne session, so please drop by!

Hudson PXE plugin 1.0 is released

Whether you recycle old PCs or use new ones as Hudson slaves, you have to first install an OS on a system. As the Hudson cluster I managed gets bigger, I find this more and more painful. You have to have a CD of the OS, and you have to have a working CD-ROM drive (somehow on recycled PCs, CD-ROM drives are often unreliable.)

Then the installation itself is rather boring, yet it's interactive. You have to answer multiple questions, for which your answers are always identical. After the OS gets installed, you have to go through a series of initial set up, like installing packages, pointing the package manager to a mirror, creating an user, etc.

On most Unix, these steps can be totally automated, by configuring installers to run with canned answers, plus a post-installation script. You can also eliminate the need of CD by network-booting your PC (AKA PXE boot.)

There are many documentats that show you how to do this, but they have a problem. Aside from the fact that none of them really talk about how you set up one PXE boot server to support multiple different OSes, it's all rather tedious, and I hate tedious things.

So I did what I normally do. Just push the job to Hudson! And that's how I wrote a plugin that makes this whole thing much simpler, and it is available now as Hudson PXE plugin. This is one of my many attempts to bring distributed builds closer and more affordable to people.

The plugin can be installed from your Update center, as usual. Currently it supports OpenSolaris, Ubuntu, Fedora, and CentOS, and OS support is extensible, so your contributions are appreciated. You first download the ISO images of those OSes (see Wiki for exactly which variation you need to download), then tell Hudson where those ISO files are. And that's all you need to do.

Now, put a PC on the same network as Hudson, start it, and press F12 to initiate a network boot. You'll see a menu of all the OSes that Hudson has, like this:

You can then choose to silent-install this OS, or interactively install it (convenient if you are installing your own PC, not Hudson slaves.)

Also, I should point out that this plugin works nicely with existing DHCP servers on the network. This plugin doesn't touch them at all. There's also a separate tool that you can use to install PCs on different networks from where Hudson master is running.

With a bit more work done on Hudson CLI, we should be able to automate the entire thing, up to the registration of a slave to Hudson. Wouldn't that be really nice? — power on a new PC, hit F12, let it run for an hour and you have a new Hudson slave, fully configured and ready for action!

Automatic Continuous Integration for Grails projects on Google Code

Japan is in the middle of a week long holiday this week, so our Japanese Hudson committers are cranking out a lot of cool stuff. This one is from Kiyotaka, who wrote a Hudson plugin called GCrawler.

GCrawler (1) searchs Google Code and discovers all Grails projects, (2) reads Subversion repository to figure out metadata, and (3) automatically add them to Hudson and builds them. The net result is a free CI service for Grails projects on Google Code. It even creates views for different versions of Grails.

You can see the live instance and the list of the whole thing from here. He also wrote "It'd be nice to support Griffon as well, and hope G2one folks would get to do it" — so I guess here's a gauntlet on the ground for G2one folks! :-)

On top of all this, this Hudson runs on a cloud — in Morph eXchange. My hat off to Kiyotaka.

Hudson's growth chart

This is from Seiji Sogabe, who is a Hudson committer. He put together a chart of the hudson.war size from 1.100 to 1.300. This is another way to see the progress in Hudson.

Off the top of my head, a big jump in 1.160 or so is probably the native Maven2 support, but I don't know what's the jump around 1.280 — maybe we picked up extra unnecessary dependencies? With Maven, it's so easy to get more than what you need accidentally. I should take a look when I can...

Hudson CLI and Groovy shell

"You can do everything from GUI" has always been one of Hudson's strengths, and we also have the REST API, but at the same time, CLI is also very useful for improving automation around administration, builds, and so on. So starting Hudson 1.302, I added a CLI agent to Hudson. CLI can be downloaded from your Hudson from http://server/hudson/cli, which also shows you the basic usage.

Internally, the interesting implementation detail of this CLI is that CLI is "dumb" — it doesn't have any hard-coded knowlege of any command, and instead it downloads the necessary code on-demand from the master via the same remoting technology that Hudson uses between the master and slaves, which is rather transparent to the program running on it.

This enables a single CLI jar to morph into whatever command that plugins on the master has provided. And once the necessary code is obtained, the CLI command can choose what to execute locally in CLI agent and what to execute on the master, making the optimal use of network.

As one of the first commands, I implemented a Groovy shell, which can be started like this:

$ java -jar hudson-cli.jar -s http://yourserver/hudson groovysh
Groovy Shell (1.6.0, JVM: 1.6.0_02-fastdebug)
Type 'help' or '\h' for help.
-------------------------------------------------------------------------------
groovy:000>;

The underlying remoting technology enabled us to write this command in a single Java class of 45 lines (including empty lines and comments), so you can see how easy it is to write a new command in Hudson.

Since this shell is connected to the live Hudson instance, you can introspect the state of the running Hudson instance, or even changes them on the fly.

groovy:000> import hudson.model.*;
===> [import hudson.model.*;]
groovy:000> Hudson.instance
===> hudson.model.Hudson@4b09558d

For example, you can define a new builder class in Groovy, and add it to your project, all on a live system (not that it's recommended, but it's a very useful tool for inspecting problems and applying a temporary bandaid.) This is also very useful if you are writing plugins and try out Hudson's behaviors. The interactivity of Groovy shell allows you to quickly figure out what API has what effects, which you can in turn bake into your code.

All in all, I really like Groovy. And if you have ideas about the CLI commands you'd like to see, please let us know.

Hudson hits 1.300

Hudson has reached version 1.300 last Friday. While it's not like this release is fundamentally different from any other past releases, it does feel like it is some kind of a milestone.

The community continues to grow. We have around 1500 human-generated e-mails per month nowadays, with 7500/wk or so downloads of hudson.war. Access to the update center is increasing steadily last time I checked (although I don't have the numbers right now.) There are 137 plugins right now and 120 committers, and according to my JavaOne 2008 slides, that's up from 50+ and 50+ each in June 2008. We've added many user visible features, such as update center, better Windows integrations, load average monitoring on a cluster, preventive failed node detection in a cluster, and so on. There are improvements in areas that's not so visible, too, such as a new modern mechanism for plugins to define/implement extension points, or more convention-over-configuration to reduce the amount of Jelly coding you'd have to do. IDE plugins for authoring Hudson plugins have improved, too, and relevant Maven mojos that we write contains a lot more error diagnostics to assist the plugin developers. Oh, and last but certainly not least, much better test coverage.

So thank you for all the contributers (and if you'd like to become one you just need to tell us so — our commit access is open to almost anyone), and thank you for all the users who continue to guide us and help us make the project better.

Hudson has recently adopted a new release model, which involves reduced release frequency (of once a week, so it's still not too infrequent) and longer soaking period via a release branching. So in this pace, 1.400 will be about 2 years from now.

BTW, this somewhat unusual linear version numbering scheme reflects the train model of releases — each release contains fixes and features that are available at that time, and the trunk is kept usuable almost all the time. If I call it 1.2.99 instead of 1.299, I'd create a false impression that somehow 1.2.99 -> 1.3.0 is a far more significant update than 1.2.98 to 1.2.99.

June 2009

Search this blog:

Categories

Archives

June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
January 2005

Recent Entries

Growth of Hudson plugin ecosystem

Hudson adoption in the Eclipse community survey

After-JavaOne project

This work is licensed under a Creative Commons License.