Skip to main content

Automating test coverage reports with NetBeans and Cobertura

Posted by fabriziogiudici on February 4, 2007 at 4:57 PM PST

Whether you're an agilist or not, you should be aware that automated testing is one of the key factors for the success of your projects. While test runners such as JUnit are widely known, there are other powerful tools that can help: those that produce reports about your test coverage.

The count of tests for class, in fact, is not enough to understand whether your code is well covered; this depends on their structure and complexity. A basic rule says that tests should exercise every single line of your classes: but how can you check this in an easy way?

Tools such as Cobertura are useful in this scenario: by instrumenting the byte code of your classes, they can precisely report which percentage of your code is covered by test and even produce a pretty printed listing of your code where you can easily spot the uncovered parts in red (look here for an example).

Today coverage tools come also with plugins for the most popular IDE such as NetBeans, Eclipse or IDEA, so they can be launched with a button or a menu. I like this approach, but anyway I'm a fan of the old batch mode ;-) also because it can be easily integrated into a continuous integration server (in this way, automated testing and coverage report run without consuming computing resources on my laptop). So I want to have coverage tests both available in my NetBeans GUI and from the command line.

To make it possible is easy, thanks to the fact that Cobertura comes with a specific Ant target and the Ant scripts used by NetBeans can be customized.First of all, we must import the Cobertura tasks in our script:

    <path id="cobertura.classpath">
        <fileset dir="${basedir}">
            <include name="lib/cobertura-1.8/cobertura.jar" />
            <include name="lib/cobertura-1.8/lib/**/*.jar" />
        </fileset>
    </path>

<taskdef classpathref="cobertura.classpath" resource="tasks.properties"/>

The first thing to do is to instrument code. This means that Cobertura will process your bytecode and add some specific calls in each basic block to keep some counters up-to-date with the number of invocations per block. This is the aim of the cobertura-instrument target:

    <target name="cobertura-instrument" depends="init,compile-test,-pre-test-run">
        <cobertura-instrument todir="${build.test.cobertura.classes.dir}">
            <fileset dir="${build.classes.dir}">
                <include name="**/*.class"/>
            </fileset>
        </cobertura-instrument>
    </target>

Adding a dependency on init, compile-test,-pre-test-run means that we are enforcing the code compilation, including the test code. We don't want to overwrite our bytecode because the instrumented code is a bit slower, so we are telling Cobertura to store the generated code into the path specified by the "${build.test.cobertura.classes.dir} variable. We can define it editing the file nbproject/project.properties (this is the file where all the properties for the Ant script are stored):

build.test.cobertura.classes.dir=${build.dir}/cobertura-instrumented-classes

Now we can define the target that collects data for the coverage:

    <target name="test-coverage" 
                     depends="init,compile-test,-pre-test-run,cobertura-instrument,
                    -do-test-run,test-report,-post-test-run,-test-browse"/>

The idea is that launching ant test-coverage instead of ant test will execute the test with coverage report. The sequence in the depends clause has just been copied from the test target that you can find in nbproject/build-impl.xml, the Ant script automatically generated by NetBeans for your projects; we have only added a dependency to make sure that the instrumentation is performed just before running the tests.

To have things working, we must also arrange some stuff so the instrumented classes are found in the classpath before the original ones. This can be accomplished by editing the run.test.classpath property in nbproject/project.properties:

run.test.classpath=\
    lib/cobertura-1.8/cobertura.jar:\
    ${build.test.cobertura.classes.dir}:\
    ${javac.test.classpath}:\
    ${build.test.classes.dir}:\
    ... other project library references ...

Note that, of course, we have also to include Cobertura itself in the classpath.

The last target generates the HTML report out of the collected data and cleans up the temporary files generated by Cobertura:

    <target name="cobertura-coverage-report" depends="init">
        <cobertura-report srcdir="${src.dir}" destdir="${cobertura.report.dir}"/>
       <delete file="Serialized" failonerror="false"/>
       <delete file="cobertura.ser" failonerror="false"/>
    </target>

Again, we can place the property:

cobertura.report.dir=${build.dir}/cobertura-report

in nbproject/project.properties.

That's all. Now I can run Cobertura from inside NetBeans, just using the context menu over the node representing build.xml and selecting the test-coverage target. Or I can run it with the command line, and eventually having it remotely executed by my favourite continuous integration tool.

Technorati tags: , ,

Related Topics >>