Skip to main content

Hudson supports distributed builds

Posted by kohsuke on March 27, 2006 at 6:07 PM PST

One of the benefits of Hudson is to have a single place to pick up your builds, a single place to configure. In the web technologies & standards group that I work in, our Hudson installation has now some 80 or so projects on it.
On top of this somewhat loaded Hudson installation, the Tango project wants to add even more tests and builds.

When you have a large number of projects on Hudson, the performance becomes an issue. It especially limits our ability to run continuous testing, as tests tend to take much longer than builds (for example, the JAXB RI just takes a few minutes to build, but a test takes 30 minutes to run by using 8 threads on our 4 CPU server machine.)

Also, tests are a kind of things where you can use as much resources as you can get. For example, the JAX-WS RI currently runs one set of continuous testing, but if the resource permits, we can add one more set of continuous testing, which will test the latest JAX-WS RI with the latest JAXB RI. This will allow us to find an integration problem earlier (just like Apache Gump.) Today this is only run nightly, but hey, why not just do it continuously?

So, in this new version of Hudson 1.19, I implemented the "master/slave" mode. You can register slave computers to Hudson, and Hudson will distribute build workload to those machines. The master will continue to maintain the website, so for everyone except admin, things will look no different. Neither users nor project owners won't notice that things are done remotely.

My first slave is a dual Opteron Linux box, (thanks Jeff!) and I'll be adding a SunBlade 2500 shortly (thanks Rama!). Since a slave doesn't have to be a dedicated node, I'm talking to some of my colleagues to have a part of their machines donated as slaves.

This allows more builds and tests to be run concurrently. As I mentioned above, I think it will be very easy to utilize added resources. (Besides, I always wanted to run my own computing cluster!)

Finally, this version does have other improvements beside the master/slave mode.

Related Topics >>