Kohsuke Kawaguchi's Blog

Make your builds always available

click to enlarge

When you are working on a software project, often you have other people who need to access the builds. For example, here at Sun when I work with the JAXB RI, our builds need to be handed to the quality assurance team, the TCK team, the JAX-RPC RI team, and other miscellaneous people who use the JAXB RI internally.

For many of them, this hand-off of a build is not just one time thing. It is a continuous on-going process. For example, the QA team needs a new build once we fix a bug they reported. During the lifetime of a project, this cycle happens a lot.

For this reason, it is often convenient to have a standard distribution channel of your builds, where people can always get the latest build. It is even better if there’s a permanent link to the latest build, because often other teams have their own procedure of integrating a build (such as putting that into their CVS repository), and a static URL allows automation of this process.

It is also better to have fresher builds. Our time is always more valuable than machines’ time, so it’s better to spend CPUs to start a new build now than to ask for the QA team to wait another day for tonight’s nightly build.

An added benefit of doing a constant build is that you can get notified when the workspace doesn’t build. I often forget to put back a file or two, and as a result the build breaks, and I know it’s not just me.

Hence comes the notion of “continuous build”, where a program automatically builds a new version of your build every time someone changes something in the repository. I found an tool called Damage Control that makes this easy, but unfortunately I found that it doesn’t scale to the size of the projects that I work with.

Therefore, I decided to re-implement it on my own, by using Java. I’m calling it Hudson, because in a sense this system is like a build master in my project, and Hudson sounded like just that kind of a name.

Anyway, Hudson is a web application, and the installation is as simple as dropping a war file into a web container, such as Tomcat. Everything else can be configured through the web interface.

A typical set up in Hudson is to set up a CVS changelog trigger, so that change notification e-mail from CVS will automatically start a new build. Hudson then starts a new build, assigning a unique build number, recording the build log, and archiving the results. It has an RSS feed so that people can be notified of new builds or build failures. It has a log rotation so that the history information won’t get too big.

click to enlarge

Hudson can also be used to monitor jobs that run externally. I have many cron jobs, and I hook them all up to Hudson, Hudson records the output from a job, keeps history of past runs, and produce RSS feeds. Once I see a failure, I can go to Hudson, browse all the outputs I need to see, and find out when it started to fail. I found this to be a lot easier than going through e-mails from cron.

So, try Hudson! I hope you’ll like it.

(A part of Hudson, in particular the art work, is covered by the license of Damage Control)

Comments

• Have you seen CruiseControl?

  Posted by: ljnelson on February 07, 2005 at 11:20 AM

• 
  I think Hudson/DamageControl approaches the problem in slightly different way. They got more pleasant interfaces to work with, and generally focused more on actually publishing the builds.

  I also like the index page where I can see all the projects at once.

  Posted by: kohsuke on February 07, 2005 at 11:34 AM

• It looks great but on my Windows box it's failing to read the CVS login info from .cvspass even when I explicitely tell it where the .cvspass file is. Not sure what's going on there.

  On my OS X box, it tries to execute the command
  cvs -d :pserver:adrian@bigcow:/D//cvs log -d '>=2005-02-08' elj
  which fails. On the command line the error comes out as:
  cvs log: cannot open CVS/Entries for reading: No such file or directory
  cvs [log aborted]: no repository

  or from within the checked out copy:
  cvs server: nothing known about elj

  I suspect it should really be leaving off the final parameter (the name of the project: elj) and executing the command from in the checked out copy.

  It looks promising though - if I could get past the CVS problems it would be perfect.

  Posted by: ajsutton on February 07, 2005 at 03:56 PM

• Hi Koshuke,

  Unless you have purchased a licence for the icons you took from DamageControl, you are in violation of the INCORS licence:

  http://www.iconexperience.com/license.php

  Codehaus (where DamageControl is hosted) has purchased an INCORS licence that allows DamageControl to incorporate these icons in its distribution and SCM. However, this licence is non-transferable. A copy of the INCORS licence is included in DamageControl.

  Please stop using these icons immediately and update the Hudson code and any published screenshots.

  DamageControl's own licence (BSD) allows you to rip off as much as you want of the code, features, etc. I would still encourage you to modify Hudson's L&F to avoid that users get confused about the two products.

  Kind regards,
  Aslak Hellesoy (DamageControl project manager).

  Posted by: rinkrank on February 07, 2005 at 04:09 PM

• I'd like to look into the issues, so please file it as an issue from http://hudson.dev.java.net/

  Posted by: kohsuke on February 07, 2005 at 04:22 PM

• 
  Thanks, Aslak. I will take off the bundle and screen shots immediately as you suggested. Because the BSD license was the only license I saw, I assumed all the artworks are covered under the same license.

  
  My aplogies.
  

  Posted by: kohsuke on February 07, 2005 at 04:24 PM

• 
  I removed all the screen shots, the files from the source code, and the release bundles. Once again, I'm deeply sorry for this.

  I'd like this kind of things to never happen in the future. For this reason, I have one question to Aslak.

  If I create a derived work of DamageControl (which to my mind Hudson is), and distribute it, it seems to me that it would violate the copyright of INCORS. If so, is it correct to say that DamageControl is not an open-source project according to OSI's definition of "open-source"?

  Specifically, in the "derived works" section, it requires that an open-source project must allow modifications and derived works.

  Posted by: kohsuke on February 07, 2005 at 04:55 PM

• Hi Koshuke,

  Thanks for responding so quickly to this. The developers of DamageControl have always intended DamageControl to be open source.

  You're right that the current state of DamageControl doesn't allow derived works when the INCORS icons are part of the derived works, since these are not OSI compliant.

  In order to make DamageControl be 100% OSI compliant we may remove the icons from the standard DamageControl distribution and instead allow them to be downloaded as a separate bundle and dropped into DamageControl. I will contact the INCORS folks and my lawyer to verify that this is possible while still being in accordance with all licences.

  Don't worry, no hard feelings on my part - I assumed it was an oversight. Whatever solution we come up with, I'll make sure the documentation for DamageControl makes it clearer what the terms of the various licences are.

  Please keep an eye on DamageControl's dev list. I'll make some announcements there when I have spoken to the lawyers.

  Cheers,
  Aslak

  Posted by: rinkrank on February 07, 2005 at 05:12 PM

• I posted a new version with different artworks to resolve the issues.

  Posted by: kohsuke on February 08, 2005 at 08:31 AM

• Sounds interesting Koshuke.
  Will definitely try it out, DamageControl did not fit into my scenario with project dependenies last time I had a look at it.

  P.S. every software-engineer should have his own lawyer nowadays ;) (..and his own documentation officer and masseuse and....)

  Posted by: mreasy on February 08, 2005 at 11:34 AM

• Do you know Anthill OS ?
  http://www.urbancode.com/projects/anthill/
  This is an open source projet as easy to install as your hubson with the major advantage to not rely on an external mechanism for automated builds.
  

  Posted by: leberre on February 10, 2005 at 02:33 AM

• Thanks for the pointer to Anthill. I didn't know about it, and it sure sounds like everyone needs to roll their own build manager (including myself!)

  For others, Anthill is not an open-source project as far as I can tell, and it is not free-as-in-beer either for commercial users.
  

  Posted by: kohsuke on February 10, 2005 at 07:42 AM

• Anthill OS is the Open Source version of the build framework on which is based Anthill. So Anthill OS uses a Mozilla - like Licence so I is a real open source project (free-as-in-beer for everybody: I haven't seen any restriction about commercial users in the licence).
  

  Posted by: leberre on February 11, 2005 at 01:00 AM

• CruiseControl and Gump are two other projects that do this.

  Posted by: cdj4real on February 25, 2005 at 10:13 AM

• Just to clarify, Anthill (OS) is a free, open source tool. There is also Anthill Pro which is a commercial product. Open source projects are granted free licenses for Pro though.

  Posted by: derkec on June 21, 2005 at 10:41 AM

This work is licensed under a Creative Commons License.