Skip to main content

Starting Hudson slave from Live USB media

Posted by kohsuke on June 7, 2009 at 11:49 AM PDT

(Translated from the original article written in Japanese)
This article shows how to prepare a Live USB stick that becomes a Hudson slave automatically, by using the Hudson swarm plugin.

Ingredients

Hardware

  • PC that supports USB boot
  • USB stick (at least 512MB, but depends on the projects to build. I used 8GB)
  • Functioning DCHP server on the network

Software

Formatting USB stick

Format a USB stick with VFAT. The drive needs to have enough space for performing builds.

Installing Slax

Slax is a modern, portable, small and fast Linux operating system with a modular approach and outstanding design. It can be directly booted from a CD or an USB drive. It's based on Slackware, and uses unionfs so that the resulting system is writable (backed by memory, thus without touching HDDs in the system.)

Use of USB stick allows us to boot a Hudson slave without modifying the HDDs in the system, and configurations can be persisted.

First, loop-mount an ISO file and copies files to a USB disk.

nanaka # mount -o loop slax-6.1.1-2.iso /mnt/floppy/
nanaka # cd /mnt/floppy
nanaka floppy # ls -la
drwxr-xr-x 4 root root 2048 2009-04-29 14:59 .
drwxr-xr-x 5 root root 4096 2009-06-07 09:29 ..
drwxr-xr-x 6 root root 4096 2009-04-18 16:39 boot
drwxr-xr-x 7 root root 4096 2009-04-18 17:10 slax
nanaka floppy # cp -a boot slax /media/disk (/media/disk is tge USB stick)

Run boot/bootinst.sh to set up MBR on the USB stick.

nanaka disk # cd /media/disk/boot        
nanaka boot # ./bootinst.sh
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
                        Welcome to Slax boot installer                        
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

This installer will setup disk /dev/sdb1 to boot only Slax.

Warning! Master boot record (MBR) of /dev/sdb will be overwritten.
If you use /dev/sdb to boot any existing operating system, it will not work
anymore. Only Slax will boot from this device. Be careful!                

Press any key to continue, or Ctrl+C to abort...

Flushing filesystem buffers, this may take a while...
Setting up MBR on /dev/sdb...
The Master Boot Record of  /dev/sdb  has been updated.
Activating partition /dev/sdb1...
No partition table modifications are needed.
Updating MBR on /dev/sdb...
Setting up boot record for /dev/sdb1...
Disk /dev/sdb1 should be bootable now. Installation finished.

Read the information above and then press any key to exit...

This completes a USB-bootable Slax.

Installing JDK

To start a Hudson swarm plugin, we need a JDK. Slax already has a module jdk-6u11-i586-1pst.lzm, so we'll install this.

With Slax, one only needs to place modules to slax/modules directory, and they'll be automatically made available upon boot.

nanaka #cp jdk-6u11-i586-1pst.lzm /media/disk/slax/modules/

If 6u11 is too old for you, non-official package of 6u14 is available from here for your convenience.

At this point, boot from USB and continue configuration from Slax. A successful boot should result in a KDE environment:

User Account

Create an user account that runs the Hudson swarm client. We could just use root for this, but we'll create the 'hudson' user in the 'hudson' group:

root@slax:~# addgroup hudson
root@slax:~# useradd -g hudson -d /home/hudson -s /bin/bash hudson
root@slax:~# mkdir /home/hudson
root@slax:~# chown hudson:hudson /home/hudson

Installing Hudson Swarm Slave

Put the swarm client in /home/hudson and create fsrot:

root@slax:~# su - hudson
hudson@slax:~$ mkdir fsroot
hudson@slax:~$ ls
fsroot/  swarm-client-1.0-jar-with-dependencies.jar

Auto-starting Swarm Client

Modify /etc/rc.d/rc.local to start the swarm client automatically upon boot:

root@slax:~# cat /etc/rc.d/rc.local
#!/bin/sh
#
# /etc/rc.d/rc.local:  Local system initialization script.
#
# Put any local startup commands in here.  Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.

USER=hudson
USER_HOME=/home/${USER}
CLIENT=${USER_HOME}/swarm-client-1.0-jar-with-dependencies.jar
LOG=${USER_HOME}/hudson.log

JAVA_HOME=/usr/lib/java

# Swarm client option
DESCRIPTION="SLAX Live-USB"
EXECUTORS=2
FSROOT=${USER_HOME}/fsroot
LABELS="Swarm"

hudson_start() {
if [ -e ${CLIENT} ]; then
        /bin/su - "${USER}" \
        -c \
        "${JAVA_HOME}/bin/java -jar "${CLIENT}" \
        -description "${DESCRIPTION}" \
        -executors "${EXECUTORS}" \
        -fsroot "${FSROOT}" \
        -labels "${LABELS}" \
        > "${LOG}" 2>&1  &"
fi
}

hudson_start

Similarly, tweak /etc/rc.d/rc.local_shutdown to terminate the swam client and clean up log files, etc.

root@slax:~# cat /etc/rc.d/rc.lo
rc.local           rc.local_shutdown
root@slax:~# cat /etc/rc.d/rc.local_shutdown
#!/bin/sh
#
USER=hudson
USER_HOME=/home/${USER}
LOG=${USER_HOME}/hudson.log
FSROOT=${USER_HOME}/fsroot

hudson_stop() {
        PID=`ps ax | grep swarm-client | grep -v grep | awk '{ print $1 }'`
        if [ -n "${PID}" ]; then
                /bin/kill "${PID}" > /dev/null 2>&1
        fi
        if [ -e "${LOG}" ]; then
                /bin/rm "${LOG}" > /dev/null 2>&1
        fi
        for jar in "${FSROOT}"/maven*.jar ; do
                /bin/rm "${jar}" > /dev/null 2>&1
        done
}

hudson_stop

Verify That Everything Works

Start the swarm client and make sure that everything works as expected.

root@slax:~# /etc/rc.d/rc.local
root@slax:~# ps aux | grep swarm
hudson   14414  9.4  1.2 662988 25576 pts/1    Sl   19:47   0:00 /usr/lib/java/bin/java -jar /home/hudson/swarm-client-1.0-jar-with-dependencies.jar -description SLAX Live-USB -executors 2 -fsroot /home/hudson/fsroot -labels Swarm
root     14465  0.0  0.0   2972   824 pts/1    S+   19:48   0:00 grep swarm

Verify that the client is connected by looking at the Hudson node management scren:

And its configuration screen:

Now shut down the system. Don't remove the USB stick until the system is fully halted.

Inspecting the USB Stick

Bring the USB back to another system to take a look at what's inside.

sogabe@nanaka ~/Desktop $ cd /media/disk/
sogabe@nanaka /media/disk $ ls
boot  slax
sogabe@nanaka /media/disk $ cd slax/
sogabe@nanaka /media/disk/slax $ ls
GNU_GPL  base     cheatcodes.txt  livecd.sgn    make_iso.sh  optional          rootcopy      tools
LICENSE  changes  images          make_iso.bat  modules      requirements.txt  slaxsave.zip  xino
sogabe@nanaka /media/disk/slax $ cd changes/
sogabe@nanaka /media/disk/slax/changes $ ls
dev  etc  home  lib  mnt  opt  root  tmp  usr  var
sogabe@nanaka /media/disk/slax/changes $ cd home/hudson
sogabe@nanaka /media/disk/slax/changes/home/hudson $ ls
fsroot  swarm-client-1.0-jar-with-dependencies.jar

You can see that slax/changes contains files that we just created above. If further changes are necessary, just modify these files. Slax also copies all the files in slax/rootcopy into the real '/' upon boot, so for example if you want /home/hudson/fsroot after booth, you can create slax/rootcopy/home/hudson/fsroot in the USB stick.

This completes the Live USB version of the Hudson swarm client. Slax makes it really easy to do this.

Related Topics >>

Comments

Notes for some minor corrections: 1) In section User Account change "addgroup hudson" to "groupadd hudson" 2) In section Installing Hudson Swarm Slave change "fsrot" to "fsroot" It might also be useful in this section to specify copying the swarm-client-1.0-jar-with-dependencies.jar to /home/hudson before the ls command is executed, just to make things a little more clear.