Skip to main content

Hudson EC2 plugin

Posted by kohsuke on May 18, 2009 at 4:27 PM PDT

Continuous Integration often requires a heterogeneous environments; for example, the GlassFish build requires Linux, Solaris, and Windows, and the JDK build requires something like 10 different environments, each carefully created so that we can test what we need to test.

Unfortunately, heterogeneous environments reduce the resource utilization — you can easily have some Windows slaves idle while all the Solaris slaves are busy, just because you happen to have more jobs that require Solaris. So doing some kind of virtualization makes a lot of sense.

Similarly, CI jobs tend to be spiky. When you need computring resources, you need them a lot (think about a month before a release, where all kinds of tests run like crazy for almost every commit), but when you don't need them, you don't. These charateristics of CI makes it a great fit with the cloud computing technology. You can have a good resource utilization and faster turn-around time (if your tests can run in parallel.)

Toward this end, I just released the Hudson EC2 plugin. This plugin enables Hudson to automatically provision new instances on EC2, based on the system demand. That is, if Hudson notices that your system is overloaded, it will provision new slaves on EC2, and when those instances go unused for a certain time period, it will shut them down. You can run all your slaves on EC2 if you want, or you can maintain your local build cluster and use EC2 as a reserve capacity.

And here's how to use the plugin. First, go to EC2 and sign up for the service. Then enter the username/password information. Because of the way EC2 works, you also need to have an RSA private key. If you have already been using EC2 and have your own key, you can paste it here. Otherwise, you can have Hudson generate one. If you let Hudson generate one, you should save this private key in your file system as well, as you'll need this to interactively logon to EC2 instances. Use "Test Connection" button to verify that Hudson can successfully talk to EC2.

Next, configure AMIs that you want to launch. For this, you need to find the AMI IDs for the OS of your choice. ElasticFox is a good tool for doing that, but there are a number of other ways to do it. Hudson can work with any Unix AMIs. Windows is currently unsupported.

Configuring labels allow Hudson to pick the right AMI to start. For example, if all your existing slaves labeled "solaris" are fully busy and you have more builds that are tied to the "solaris" label, Hudson will start the AMIs that have the "solaris" label.

Init script is the shell script to be run on the newly launched EC2 instance, before Hudson starts launching a slave agent. If the AMI doesn't have Java pre-installed, you need to do so in this script. This is also a good place to install additional packages that you need for your builds and tests.

The load monitoring part is in the core, so the EC2 plugin really just contains some glue code that talks to EC2 to provision a new system. This means we can easily write simialar plugins for different clouds and virtualization technologies. For example, someone in the community is writing a plugin that do this with VMWare labmanager. VirtualBox lets us do something similar, with its web services (for which I wrote a JAX-WS client library some time ago, so might be fun to eat my own dogfood, so to speak.) Vivek and I also installed Eucalyptus on our spare machines, so I hope to be able to extend this plugin to work with that, too.

I'm going to talk more about this (and other things) in my JavaOne talk, so if you are coming to JavaOne, please consider dropping by!

Related Topics >>

Comments

Just tried to configure the

Just tried to configure the EC2 plugin. The EC2 connection test works fine, but I get an error when checking AMIs that work fine with the EC2 command line tools. It says "Endpoint URL is not a valid URL".
Could you give me a hint about the underlying problem?

Java install on the EC2 slave

Hi

Got a question:

I extracted java jdk on the EC2 slave, set the JAVA_HOME and the added the JAVA_HOME/bin to the .bash_profile. I then bundle this up (along with a few other changes) as an AMI. I added the following line to the init script: . ~/.bash_profile

When the slave is started by hudson, I see:
-----------------------
Installing Java

2009-09-06 12:48:45 URL:http://s3.amazonaws.com/hudson-ci/jdk/linux-i586/java1.6.0_12.tgz?AWSAccessKeyId=0Q5GJF508WPT2B17MFG2&Expires=1252259302&Signature=Tzar2FQhYSf2nBaWs3YdeOF1Us0%3D [44508775/44508775] -> "/usr/java1.6.0_12.tgz" [1]
-----------------------

What's really going on here ? Does this affect the build ?
Thanks Baljeet

Small question

Firstly, thanks this is cool. Q. When does the plugin decide to shut the EC2 instance down. Is this configurable (maybe by tinkering with the plugin) ? Thanks Baljeet

It shuts down when the

It shuts down when the instance is idle for 30 minutes. Given the price model of EC2, I suppose an enhancement would be to bring it down at the one-hour boundary. This is not currently configurable, but you can file an RFE --- I'm curious though, why do you want to make it longer or shorter?

The one hour boundary

The one hour boundary boundary would be ideal from a billing perspective. 30 mins idle time before shutdown works too, since my build would take a few minutes. I was trying to figure out what sort of shutdown schedule would work best from a pricing perspective.

Kohsuke - this plug-in is magic and perfectly timed for me. Really pleased to see it. Congratulations on the very cool work.

Installation was a breeze. Great job! I thought I'd add that the packages are named SUNWcvs and SUNWsvn on OpenSolaris, so the Init Script should be written as: pkg install SUNWcvs SUNWsvn