Skip to main content

Building in the cloud

Posted by carcassi on August 10, 2012 at 11:40 AM PDT

This week I moved the continuous build of 5 projects to the cloud instance of Jenkins provided by Cloudbees. Here's a brief summary of my experience.

Backstory (for those interested)

A few years ago, I setup a Jenkins/Hudson instance. It started off to build my project, but it became the host for a few other projects within the EPICS collaboration. With the growth, I had the following problems:

  • It's a pain to maintain. The system was supported using breadcrumbs of time from myself and the network administrator who, with the commissioning of the facility at the door, has been increasingly busy. So even things that should be simple, like add Java 7 support, would effectively never get done.
  • I can't really give full permission to outside collaborator. To be able to give it following the cybersecurity policy would have been a hassle (and rightly so).
  • The special status of BNL (the facility I work for). Other collaborator may feel funny telling their management that the software they are writing is being build somewhere else... like admitting in some way that they can't do it themselves.

So, for these and other reasons, I started to look at having a Jenkins instance running somewhere else. So I started to look at Cloudbees. After a few false starts, this week I sat down and did it. It took, in the end, only a day and a half. Here's the result: https://openepics.ci.cloudbees.com/.

Building a Maven project hosted SourceForge project

All 5 projects are:

  • Hosted on SourceForge
  • Maven projects
  • Have a maven site to be deployed
  • Deploy their artifact in a maven repository also hosted on SourceForge

The first thing to do was to create a user account for the build, in my case epics-jenkins. This way, when the build contacts the forge, it is clear who it is and we can give appropriate permissions. The next thing is to authorize the Jenkins/Cloudbees build to authenticate as epics-jenkins: when we create a job, we can see the public key that will be used (it's the same one for a whole account); we can grab that and add it to the keys used by the epics-jenkins account on SourceForge. After that, we should add epics-jenkins to all our projects, making sure that we add enough permission so that it can access the sourceforge shell/scp/rsync services.

We also need to setup the credentials for when we are going to deploy the artifacts that are the output of the build. We need to give maven a settings.xml with the specifics. There is already a tutorial on how to do that. This was improved since the last time a looked, and it now presents a good option for Windows too.

The next thing to do is to setup our project. Given that these were maven projects, that was very straight forward (there is a maven specific job in Jenkins/Hudson). What makes it easier is that they have added a custom pre-step and post-step to the maven task, which is essential. To do a maven deploy, you need to pre-create a shell on sourceforge for epics-jenkins, or the scp will fail. So, in your pre-step you'll need to execute the command:

ssh epics-jenkins,epics@shell.sourceforge.net create

after that, the deploy will actually work. I also hadto add the scp wagon (by default in maven 2, needs to be explicitly added in maven 3). For deploying the site, I actually prefer to let it build locally and then use a post-step:

rsync -r -e ssh --delete target/site/* epics-jenkins,epics-util@web.sourceforge.net:htdocs/

As the final step, we should not forget to add:

The build has now been working well for this week. I started to give access to other people in the collaboration. I still need to move a number of other projects (not all with a build as simple as the first 5 projects), but so far so good!

Related Topics >>