Skip to main content

Do you distribute your builds?

Posted by johnsmart on December 30, 2008 at 1:23 PM PST

When organisations first set up a Continuous Integration environment, distributed builds are often fairly low down on the list - more in the "nice-to-have" category, or considered too advanced to look at initially. However, distributed builds are actually more than just a "nice-to-have" - once you start using them, they become quite addictive!

There are two main scenarios where distributed builds can come in handy: to speed up your build process, or to do environment-specific build jobs. In the first case, you break your build process down into stages, and spread the work over several build agents running in parallel. For example, you might be running functional tests using Selenium on several different browsers. There is no reason you can't run these tasks in parallel, on different machines - this can potentially speed up your build process substantially!

The other main reason for distributed builds is when you need to run a particular build job on a particular environment. For example, you can only run Selenium web tests using Internet Explorer on a Windows machine. And despite the dropping use of IE in favour of real web browsers ;-), this is still something you might want to test. If your build server is running on a Linux box, you have little choice but to run the Selenium web tests separately on a Windows build agent.

Support for distributed builds is becoming more prevalent. On the open source front, for example, Hudson provides pretty good support for distributed builds, which is very easy to set up. All of the main commercial build servers provide good distributed build features, usually with more sophisticated support for advanced configurations. Both Bamboo and TeamCity, for example, let you send certain build jobs to particular "compatible" build agents. This way, you can send Windows-based tests to build agents on Windows machines, or send a build job that deploys an application on the integration server to a build agent running locally on this server.

When you set up distributed builds, you will probably need to fine-tune your build process for best results. For example, to run web functional tests without having to rebuild and retest the whole application, I often set up a special Maven project for this purpose. The basic build job compiles the code, runs unit and integration tests, and deploys a WAR file to the snapshot repository. The functional test Maven project (or module, to be precise) uses different profiles to run Selenium using different browsers. Separate build jobs run on different distributed build agents - some can run on any available build agent, whereas the IE jobs can only run on Windows-compatible build agents.

These are just a few examples of how distributed builds are used - I'd love to hear about others!

"Best development course I have been on in a very long time...Greatly enjoyed the course...A 'must' course for serious Java developers..." - Check out the 2009 series of Java Power Tools Bootcamps.