John Ferguson Smart's Blog

A bird's-eye survey of the world of Continuous Integration Tools in 2008

The original CI tool (if you don't count ye old cron job) came in first with a wopping 35% for CruiseControl. Hudson and Continuum where neck-and-neck, with 14% for Hudson and 13% for Continuum. IntelliJ's TeamCity performed well for a commercial product, with a score of around 9%.

But times change, and technologies evolve. A year is a long time in the Java world. So lets take a look at the lay of the land today. And, while you're at it, why not vote on the new 2008 Continuous Integration Poll.

I haven't used CruiseControl for a wee while, so I can't vouch for it's latest features. In my experience, it's powerful, and flexible, but a right pain in the nether regions to configure and maintain. The reporting and web interfaces are pretty so-so as well. But, still, lots of people out there are using it.

Hudson, the new kid on the block, is constantly evolving, and adding feature after feature. The latest releases have added role-based security, plus an every-increasing list of very cool plugins. And the user interface is still as groovy as ever! Of all the open source CI tools, Hudson is what I generally use, given the choice.

Continuum 1.1 has arrived at last, with some cool new features like project groups and role-based security. Unfortunatly, though, the online documentation still leaves a lot to be desired.

On the commercial front, JetBrains is now offering a free "Professional Edition" of TeamCity, the Continuous Integration tool from the makers of IntelliJ. This edition is designed for small organisations with a limited number of user accounts, and no support for complex user authentication schemes such as project-based roles and LDAP support.

Indeed, there are also a number of commercial Continuous Build servers. There are plenty of good open source Continuous Build tools out there, so why might you choose a CI commercial tool over an open source one? Let's play the devil's advocate for a moment, and consider the options.

If your shop works with both Java and Microsoft technologies, you might appreciate TeamCity's support of both Java and .NET builds.

TeamCity also has a feature called a "personal build", which is basically a sandbox on the build server where code is automatically compiled and tested before being committed to version control. This is basically equivalent to the developer compiling and running the full set of unit tests before each commit, but with the advantage of being automated. Parabuild seems to proposes something simmilar with their "unbreakable builds".

Both TeamCity and Parabuild provide innovative features around distributed builds, though Hudson also provides some basic support for this as well.

Atlassian also do a nice Continuous Build server, called Bamboo, which, unsuprisingly, integrates very smoothly with JIRA.

Documentation might be another factor in your decision. Documentation for a commercial product is often of higher quality than the open source equivalent. Continuum is a flagrant example of this. The Hudson online documentation is better, though it still has a rather Wiki-ish feel to it. TeamCity documentation is excellent.

However, introducing Continuous Integration into an organisation is as much, if not more, about changing mindsets than it is about choosing a particular tool. And there is little risk of vendor lock-in, as it is fairly easy to replace one CI server with another. So, if your new to CI, you might just want to download an easy-to-use open source CI tool (Hudson comes to mind) and start out with that. Once you get familiar with what you can do with a CI server, you can always rethink your requirements later on.

To get a better idea of what tools people are using, I've published a new Continuous Integration Poll - come along and vote for your favorite tool!

Comments

• Hi John. I tried to cast a vote (most notably for Hudson), but I get an "Invalid token" error message. Tried both Safari and Camino browser.

  Posted by: fabriziogiudici on February 26, 2008 at 05:01 AM

• Same attempt (Hudson) and result (invalid token) here - eduard/o

  Posted by: pelegri on February 26, 2008 at 06:37 AM

• Personally, I think Hudson kicks azz. Also, Hudson's author, Kohsuke Kawaguchi, is featured as a rockstar programmer in my new book: Secrets of the Rockstar Programmers .

  Ed

  Posted by: edburns on February 26, 2008 at 06:40 AM

• Hi guys, there seems to have been a hitch with the poll module - should be all fixed now, hopefully!

  Posted by: johnsmart on February 26, 2008 at 01:16 PM

• Hi John,

  I'm pretty sure TeamCity supports LDAP, although only in the enterprise version.

  Posted by: dlinsin on February 26, 2008 at 10:22 PM

• It works now, thanks. Yes, Kohsuke rocks! :-)

  Posted by: fabriziogiudici on February 26, 2008 at 11:33 PM

This work is licensed under a Creative Commons License.