CI Adoption Stories (5/8): Optimizing Continuous Integration to push build artifacts
This case study is the fifth of an 8-part blog series about why so many developers adopt continuous integration, and originally published on the Atlassian blogs. It naturally has a bit of a Bamboo slant to it (if you look close enough, you can even see the odd panda), but the issues discussed apply to any team trying to adopt modern development practices, no matter what CI tool is used.
Optimizing Continuous Integration to push build artifacts
Simon Smith and Steve Peters are Java developers working on a large web application alongside 5 other developers, spread over two geographical sites. The application is designed using a clean modular architecture. The project uses Maven 2, which helps define and enforce the modular architecture. It consists of ten distinct Maven modules representing different parts of the system, with well-defined dependencies between the module. For example, there is a Maven module for the core domain logic, and another for the web layer. Steve has also written a specialized library encapsulating some customized localization features. Source code is stored in a Subversion repository. Build artifacts (JAR and WAR files) are deployed to a Nexus Maven repository, to be made available to other developers.
The Maven repository distinguishes between snapshot versions, which typically correspond to a nightly build or to the latest code changes made to the version control system, and release versions, which are more stable, numbered releases. Developers in the same team will typically use snapshots reflecting the latest code changes, whereas release versions are usually for external use where more stability is required. This approach does an excellent job of enforcing modularity and keeping dependencies clean.
Life before Continuous Integration
However, before CI was introduced, their approach to build artifacts was a source of frequent integration headaches. For example, if Steve corrected a bug in the localization module, he needed to first commit his code changes to Subversion, and then to manually deploy a new snapshot onto the Maven repository. Only then would his corrections be available to Simon. The deployment phase was time-consuming, as it involved executing a complete build and test cycle, and then transferring the file to the Nexus server on the other geographical site - depending on the size of the module, it could take between 5 and 20 minutes, several times a day, for the whole process to complete. Steve would occasionally forget this second step, which resulted in his corrections not being visible to Simon, even though they had been committed to Subversion. This also happened occasionally in the nine other modules.
The main impact of this sort of incident was lost time. The problem was relatively insidious, as in theory at least there was a simple work-around - Simon just needed to check out Steve's module and do the deployment himself. However this would interrupt Simon in his thought-stream, and oblige him to work on something else for 15 to 30 minutes. The problem was also hard to isolate: one time, it took 4 hours of debugging before Simon finally tracked a problem down to an undeployed snapshot. And if something went wrong, Simon was hard-put to trouble-shoot the issue.
This sort of wasted time prompted the team to look for a more automated solution. So, to make the overall integration process smoother, the team decided to install a Bamboo server. There is now a dedicated set of build plans for each Maven module, that compile and test the code whenever changes are made, run code quality and code coverage metrics, and notifies developers if something went wrong. In other words, from a CI perspective, each module can be considered as a separate CI project.
Since the development efforts are now closely integrated, modules use snapshot dependencies to ensure that they always have the latest code changes. Each module also has a second build plan, which automatically deploys the build artifact to the enterprise repository whenever the first build succeeds. Since the initial build runs a battery of unit and integration tests, a minimum of stability is ensured. For Steve and Simon, the result has been a massive time saver. Now, every developer can now access the latest stable versions of the libraries with no manual intervention required. The deployment still takes 5 to 20 minutes, several times a day, but it happens automatically, with no human intervention, and with no interruption in the developer's concentration.
"Probably the best training course I've been on."..."Not just how to write Java code but the 'business end' - how to build, test, deploy, manage and monitor"..."One of the best and most useful courses I have attended. And they didn't even try to sell me anything!" - Coming soon to London! Get up to scratch with the latest in Java tools and best practices! Sign up on the 2009 season of the Java Power Tools Bootcamps.