John Ferguson Smart's Blog

John is a freelance consultant specialising in Enterprise Java, Web Development, and Open Source technologies, currently based in Wellington, New Zealand. Well known in the Java community for his many published articles, John helps organisations to optimize their Java development processes and infrastructures and provides training and mentoring in open source technologies, SDLC tools, and agile development processes. John is principal consultant at Wakaleo Consulting, a company that provides consulting, training and mentoring services in Enterprise Java and Agile Development.

Hudson project-based matrix security is out!

One feature that I've been waiting for a long time to see in Hudson is project-level security. To be able to say that certain projects can only be built by certain users. This comes in very handy if certain builds jobs should only be executed by certain people, for security or auditing purposes, for example. A release into QA (or even production) might come under this category.

TeamCity has this feature. Contiuum has some pretty refined user management as well. But, up until recently, Hudson only had project-level user rights. Now, however, you can setup project-specific rights in Hudson as well. And I have to say, this is one of the more exiting Hudson features to have come out in a while (and that's saying something!).

You can check this new feature out in the latest Hudson build. Let's take a quick tour of this new feature.

First, you need to activate security. You do this in the Configure System screen (in the "Manage Hudson" section). For more advanced setups, you can hook into an LDAP server or use the underlying servlet container's security, but to start off, it's easier to use Hudson's own user database.

One tip that you should remember: make sure you tick the "Anyone can do anything" checkbox before you proceed, as otherwise you might end up locking yourself out! (If you do lock yourself out, just edit the config.xml file in your .hudson directory and change the "useSecurity" entry to false).

Once you're done, save this screen. You will see a "sign up" link in the top right-hand corner of the screen. This goes to a screen where users can sign up. Use this screen to add a few users, starting with "admin" (it is always a good idea to create an "admin" user first).

Now go back to the Configure System screen and pick "Project-based Matrix Authorization Strategy". At first glance, this bares a striking resemblance to the standard "Matrix-based security". Do not be troubled! This section lets you define application-wide rights, such as for adminitrations and anonymous users. You can add users to the matrix by typing a user name in the "User/group to add". Here, don't forget to add a row for the administrator with everything ticked. Also, if you want a user to be able to build a project anywhere, give them build authorisations here. In fact, the same applies for all of the job-level authorisations.

For more sophisticated environments, the same approach works fine with LDAP integration as well.

Now the fun starts. Go to one of your projects, and click on "Enable project-based security". Now you can assign project-level authorisations for this project to your users. In the screenshot below, Jane can do anything on this project, whereas Bob can only build.

One thing that is important to remember is that project-level rights overrule application-level rights. If bob has application-wide build authorisation, but does not have project-level build authorisation for a particular project, then he won't be able to build that project. Hudson also seems to get confused if you assign . So, in a nutshell, give your users full rights at an application level by default, then limit their access to particular projects in the project-level build configuration.

This feature is still fairly new, and there are still a few bugs to be ironed out. For example, I've come across the odd bug in the last few releases during the sign-on process - you occasionally get an error when you try to log on with a new user. For testing purposes, just go back to the previous page - in this case, you are in fact logged on. I'm sure Kosuke will sort that one out soon ;-). Still, nice work Kosuke!

"Best development course I have been on in a very long time...Greatly enjoyed the course...A 'must' course for serious Java developers..." - Read what people are saying about the Java Power Tools Bootcamps.

Using the Maven Release Plugin

It is an excellent and widely used best practice to tag each of your stable releases in your version control system for future reference. However, this sort of bookkeeping is tedious and error-prone at the best of times. Like many tedious, error-prone tasks, it is one of those things that could do with a bit of automation.

Fortunately, Maven can help. The Maven Release Plugin helps you automate the whole process of upgrading your POM version number and tagging a release version in your version control system. Let's see how it works. Here is an extract from a POM file, showing the version number that uniquely identifies this version:

<project...>
    ...
    <groupId>com.wakaleo.myapp</groupId>
    <artifactId>myapp-core</artifactId>
    <packaging>jar</packaging>
    <version>1.0.1-SNAPSHOT</version>
    ...

The SNAPSHOT suffix means that each time I deploy this version, a new snapshot will be deployed into my Maven repository. Anyone who wants to use the latest, bleeding-edge SNAPSHOT version can add a SNAPSHOT dependency in their project. This would usually be myself, or other members of the development team. Snapshots, by definition, tend to be fairly unstable beasts.

        <dependency>
            <groupId>com.wakaleo.myapp</groupId>
            <artifactId>myapp-core</artifactId>
            <version>1.0.1-SNAPSHOT</version>
        </dependency>   

As a side-note, the fearless and the reckless can take this a step further by always using the LATEST version, regardless of its actual version number, and regardless of whether it happens to be an official release or just a snapshot:

        <dependency>
            <groupId>com.wakaleo.myapp</groupId>
            <artifactId>myapp-core</artifactId>
            <version>LATEST</version>
        </dependency>   

When the version 1.0.1 is ready, we need to update the POM file, commit the new POM file to version control, tag this version as a release, and then move on to work on version 1.0.2. The Maven Release plugin can automate much of this process. However, before the Maven Release plugin can do its magic, you need to make sure you have everything it needs set up in your POM file.

First of all, you need to be working with a SNAPSHOT release. However, when you are ready to release your new version, you should remove any references to snapshots in your dependencies. This is because a release needs to be stable, and a build using snapshots is, by definition, not always reproducible.

The next thing you need is an block, so that it can find where to create a new release tag and commit the changes. Here is a real-world example:

    <scm>
        <connection>scm:svn:https://wakaleo.devguard.com/svn/maven-plugins/maven-schemaspy-plugin/trunk</connection>
        <developerConnection>scm:svn:https://wakaleo.devguard.com/svn/maven-plugins/maven-schemaspy-plugin/trunk</developerConnection>
        <url>https://wakaleo.devguard.com/svn/maven-plugins/maven-schemaspy-plugin/trunk</url>
    </scm>  

Next, you need to configure the Release plugin itself. This mainly involves telling Maven where your release tags go, via the "tagBase" configuration element. If you are using the Subversion trunk/tags/branches convention, Maven will automatically put release tags in the "tags" directory. In the following example, we use a slight variation on the normal convention, and place releases in the "tags/releases" directory:

<project>
  ...
  <build>
    ...
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-release-plugin</artifactId>
        <configuration>
          <tagBase>https://wakaleo.devguard.com/svn/maven-plugins/maven-schemaspy-plugin/tags/releases</tagBase>
        </configuration>
      </plugin>
    </plugins>
    ...
  </build>
  ...
</project>

Now you can get down to business, and try out a (semi-)automated release. The first thing you need to do is to make sure all your latest changes have been committed to (in our case) Subversion. If there are any outstanding changes, Maven won't let you do a release. First of all, you need to prepare the release, using the "prepare" goal:

$ mvn release:prepare

If you want to know exactly what Maven will do to your POM file and your SCM ahead of time (generally a good idea), you can run the operation in the "dry-run" mode, as shown here:

$ mvn release:prepare -DdryRun=true

$ mvn release:clean release:prepare

The "prepare" goal actually does quite a lot. Indeed, it will:

• Make sure that there are no uncommitted changes or SNAPSHOT dependencies (see above)
• Update the SNAPSHOT version number to a release version (e.g. going from "1.0.1-SNAPSHOT" to "1.0.1")
• Update the SCM section of the POM file to point to the release tag rather than the trunk in the Subversion repository
• Run all the application tests to make sure everything still works
• Commit the changes made to the POM file
• Create a new tag in Subversion for this release
• Update the SNAPSHOT version number to a new SNAPSHOT version (e.g. going from "1.0.1" to "1.0.2-SNAPSHOT")
• Commit the changes made to the POM file

But wait, a minute, you might say. Didn't we forget to deploy our release somewhere? Well, that is why the goal is called "prepare". We only set everything up in preparation for the release, we haven't actually released anything yet. But dont worry, performing the release is pretty straight-forward, too. Just use "mvn release:perform":

$ mvn release:perform

• Check out the release we just tagged
• Build the application (compiling, testing and packaging)
• Deploy the release version to local and remote repositories

All in all, a very convenient way to automate your release process.

"Best development course I have been on in a very long time...Greatly enjoyed the course...A 'must' course for serious Java developers..." - Read what people are saying about the Java Power Tools Bootcamps.

Replication in Subversion 1.5

One common requirement when you set up a Subversion repository concerns how it will be backed up. Another involves what happens if the main repository server goes down for some reason. Yet another is about how to deal with teams distributed across large distances or connected with limited networks. Subversion has some interesting solutions to all of these concerns, notably via the svnsync tool, and more recently with a new feature called Write-Through Proxying, that makes setting up distributed repositories much easier.

Earlier on this month I did a short talk at the Wellington JUG about using svnsync and Write-Through Proxying to set up distributed repositories with Subversion. The talk gives an overview of what's involved in setting up repository replication with svnsync, and shows how Write-Through proxying allows you to make life easier for remote developers. Although it doesn't go into too much detail (the Subversion documentation is pretty good), it does give a general idea of how it works. I've posted the slides for this presentation below.

New article on FindBugs

Findbugs is one of my favorite static analysis tools. It's goal in life is to enables you to isolate and correct dangerous Java bugs in your code, and it does this particularly well. I find that the issues raise by FindBugs are very often worthy of attention - indeed, the issues it finds can be particularly knarly ones, and the "false-positive" rate is suprisingly low for this type of tool.

If you want to find out a little more about FindBugs, and especially how it compares to other static analysis tools on the market, check out this article on DevX.

Announcing the new Maven Schemaspy Plugin

Schemaspy is a little known but very useful database analysis tool that generates an interactive graphical representation of your database structure, in terms of tables and relationships. This is a very cool tool that works wonders when you need to understand a new database structure. Indeed Schemaspy gets a chapter in the Java Power Tools book. A sample Schemaspy report can be found here.

If you have a Maven project, this is a great thing to have in your Maven reports. Unfortunately, there is no official Maven plugin for Schemaspy (at least, not for Maven 2). So, to get around this, I wrote my own. This plugin lets you generate SchemaSpy reports from within your pom.xml file - they appear in the Maven reports just like ordinary Javadoc documentation.

For the moment, this plugin is deployed in the Wakaleo Maven repository, so you need to add a reference to this repository in your POM file or your settings.xml file, as shown here:

    <pluginRepositories>
        ...
        <pluginRepository>
            <id>Wakaleo Repository</id>
            <url>http://maven.wakaleo.com/repos/</url>
        </pluginRepository>
        ...             
    </pluginRepositories>

The plugin is pretty easy to use: you just provide the details you would expect to access your database (JDBC URL, username, password,...). The full JDBC URL works fine. Or you can just provide the database name, type, and host address, which makes the configuration a bit lighter. The following listing illustrates how to use the plugin:

<project>
  ...
  <reporting>
    <plugins>
        <plugin>
            <groupId>com.wakaleo.schemaspy</groupId>
            <artifactId>maven-schemaspy-plugin</artifactId>
            <version>1.0</version>
            <configuration>
                <databaseType>mysql</databaseType>
                <database>testdb</database>
                <host>localhost</host>
                <user>scott</user>
                <password>tiger</password>
            </configuration>
        </plugin> 
    </plugins>
  </reporting>
</project>

The plugin documentation can be found here. In particular, look at the usage page for more details on how to use the plugin.

Using easyb with Maven

Easyb is a very cool way to test your Java application in BDD-style with Java. But wouldn't it be nice to be able to integrate your BDD stories into your Maven build process? The good news is, you can!

Writing a BDD-style test story with easyb is, well, easy. Groovy is a great way for Java developers to write tests quickly and efficiently, even if they don't know the finer points of the Groovy scripting language. For example, the following test story (in a file called "AccountDepositsStory.groovy") explores what should happen when you open a new bank account:

This is easy enough, and there are many ways to execute it. You can run the script from the command line, or using the Ant task. One of my favorite techniques is to use the IntelliJ plugin, which is, indeed, very cool. However, as a Maven user by choice, what I really want to do is to integrate EasyB scripts into the Maven build process.

As it turns out, this, also, is pretty easy, thanks to the Maven Easyb Plugin. Just put all of your EasyB story scripts under the src/test/easyb directory and add a reference to the EasyB Maven plugin in your build plugins, as shown here:

<project...>
    ...
    <build>
        <plugins>
            ...
            <plugin>
                <groupId>org.easyb</groupId>
                <artifactId>maven-easyb-plugin</artifactId>
                <version>0.9</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>test</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

In this example, we set up Easyb to run during the test lifecycle phase (in other words, when you run "mvn test"). When you run your tests in Maven, EasyB will kick in and run all of the stories it can find in the src/test/easyb directory. By convention, it considers any file ending in "*Story.groovy", "*.story", "*Specification.groovy" or "*.specification" to be an easyb story. An example of the easyb plugin in action is shown here:

C:\Users\john\projects\onlinebank\onlinebank-core>mvn test
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building onlinebank-core
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
...
[INFO] [easyb:test {execution: default}]
[INFO] Using easyb dependency org.easyb:easyb:jar:0.9:compile
[INFO] Using easyb dependency commons-cli:commons-cli:jar:1.1:compile
[INFO] Using easyb dependency org.codehaus.groovy:groovy-all-minimal:jar:1.5.0:compile
     [java] Running account deposits story (AccountDepositsStory.groovy)
     [java] Scenarios run: 2, Failures: 0, Pending: 0, Time Elapsed: 0.368 sec
     [java] Running account withdrawls story (AccountWithdrawlsStory.groovy)
     [java] Scenarios run: 2, Failures: 0, Pending: 0, Time Elapsed: 0.123 sec
     [java] Running open account specification (openAccount.specification)
     [java] Specifications run: 2, Failures: 0, Pending: 0, Time Elapsed: 0.038 sec
     [java] 6 total behaviors run with no failures
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Mon Aug 11 10:09:24 NZST 2008
[INFO] Final Memory: 8M/14M
[INFO] ------------------------------------------------------------------------

On the downside, the plugin is still a little green - it doesn't integrate with the Maven site generation as yet, for example, the reporting is pretty basic, and you need to bind the plugin to the 'test' lifecycle phase yourself. However, the project is a very dynamic one, and these details should be ironed-out in no time - so stay tuned for the latest BDD developments!

You can check out a simple sample Maven 2 project using easyb here.

If you want to learn more about EasyB, I'll be giving a talk on the subject at the Java Emerging Technologies Conference 2008 (JET 2008) in Auckland on September 17, as well as talks for the Melbourne and Sydney Java Users Groups while I'm over that way for the Melbourne and Sydney Java Power Tool bootcamps. So, if you are interested, and in the area, come along!

Creating Maven projects from the command line - the easy way

Maven archetypes are great. But remember how painful it is to create a new Maven project from the command line, with all those command-line options to remember? Enough to put off even the most hardened Maven enthusiast!

Sure, you can always use your trusty IDE. But what about those times when you really need to create a Maven project from the command line. Maybe your using a machine without all the right plugins for your favorite IDE. Maybe the machine doesn't have your favorite IDE. But, there you are, still needing to create a Maven project, and not wanting to do it by hand.

Fortunately, there is a better way. You can still use the Maven archetype plugin, but if you use the archetype:generate goal, Maven will step you through the options using a command-line wizard. The first thing this plugin does is to propose a somewhat imposing list of available archetypes (44 at last count), each with a unique number: you'll never have to remember the exact name of an archetype again!

To get started, just open a command line window in your project directory and type archetype:generate:

C:\Users\john\projects>mvn archetype:generate
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'archetype'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Pet Database Core Module
[INFO]    task-segment: [archetype:generate] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] Preparing archetype:generate
[INFO] No goals needed for project - skipping
[INFO] Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextCla
ssLoaderResourceLoader'.
[INFO] Setting property: velocimacro.messages.on => 'false'.
[INFO] Setting property: resource.loader => 'classpath'.
[INFO] Setting property: resource.manager.logwhenfound => 'false'.
[INFO] [archetype:generate]
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-arc
hetype-quickstart:1.0)
Choose archetype:
1: internal -> appfuse-basic-jsf (AppFuse archetype for creating a web application with Hibernate, S
pring and JSF)
2: internal -> appfuse-basic-spring (AppFuse archetype for creating a web application with Hibernate
, Spring and Spring MVC)
3: internal -> appfuse-basic-struts (AppFuse archetype for creating a web application with Hibernate
, Spring and Struts 2)
4: internal -> appfuse-basic-tapestry (AppFuse archetype for creating a web application with Hiberna
te, Spring and Tapestry 4)
5: internal -> appfuse-core (AppFuse archetype for creating a jar application with Hibernate and Spr
ing and XFire)
6: internal -> appfuse-modular-jsf (AppFuse archetype for creating a modular application with Hibern
ate, Spring and JSF)
7: internal -> appfuse-modular-spring (AppFuse archetype for creating a modular application with Hib
ernate, Spring and Spring MVC)
8: internal -> appfuse-modular-struts (AppFuse archetype for creating a modular application with Hib
ernate, Spring and Struts 2)
9: internal -> appfuse-modular-tapestry (AppFuse archetype for creating a modular application with H
ibernate, Spring and Tapestry 4)
10: internal -> maven-archetype-j2ee-simple (A simple J2EE Java application)
11: internal -> maven-archetype-marmalade-mojo (A Maven plugin development project using marmalade)
12: internal -> maven-archetype-mojo (A Maven Java plugin development project)
13: internal -> maven-archetype-portlet (A simple portlet application)
14: internal -> maven-archetype-profiles ()
15: internal -> maven-archetype-quickstart ()
16: internal -> maven-archetype-site-simple (A simple site generation project)
17: internal -> maven-archetype-site (A more complex site project)
18: internal -> maven-archetype-webapp (A simple Java web application)
19: internal -> struts2-archetype-starter (A starter Struts 2 application with Sitemesh, DWR, and Sp
ring)
20: internal -> struts2-archetype-blank (A minimal Struts 2 application)
21: internal -> struts2-archetype-portlet (A minimal Struts 2 application that can be deployed as a
portlet)
22: internal -> struts2-archetype-dbportlet (A starter Struts 2 portlet that demonstrates a simple C
RUD interface with db backing)
23: internal -> struts2-archetype-plugin (A Struts 2 plugin)
24: internal -> shale-archetype-blank (A blank Shale web application with JSF)
25: internal -> maven-adf-archetype (Archetype to ease the burden of creating a new application base
d with ADF)
26: internal -> data-app (A new Databinder application with sources and resources.)
27: internal -> jini-service-archetype (Archetype for Jini service project creation)
28: internal -> softeu-archetype-seam (JSF+Facelets+Seam Archetype)
29: internal -> softeu-archetype-seam-simple (JSF+Facelets+Seam (no persistence) Archetype)
30: internal -> softeu-archetype-jsf (JSF+Facelets Archetype)
31: internal -> jpa-maven-archetype (JPA application)
32: internal -> spring-osgi-bundle-archetype (Spring-OSGi archetype)
33: internal -> confluence-plugin-archetype (Atlassian Confluence plugin archetype)
34: internal -> jira-plugin-archetype (Atlassian JIRA plugin archetype)
35: internal -> maven-archetype-har (Hibernate Archive)
36: internal -> maven-archetype-sar (JBoss Service Archive)
37: internal -> wicket-archetype-quickstart (A simple Apache Wicket project)
38: internal -> quickstart (A simple Apache Tapestry 5 Project)
39: internal -> scala-archetype-simple (A simple scala project)
40: internal -> lift-archetype-blank (A blank/empty liftweb project)
41: internal -> lift-archetype-basic (The basic (liftweb) project)
42: internal -> cocoon-22-archetype-block-plain ([http://cocoon.apache.org/2.2/maven-plugins/])
43: internal -> cocoon-22-archetype-block ([http://cocoon.apache.org/2.2/maven-plugins/])
44: internal -> cocoon-22-archetype-webapp ([http://cocoon.apache.org/2.2/maven-plugins/])
Choose a number:  (1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/
31/32/33/34/35/36/37/38/39/40/41/42/43/44) 15: :18

The default archetype is the "maven-archetype-quickstart", the simple Maven project structure that we are all familiar with. But there are plenty of others to choose from, including all of the appfuse archetypes, and many other technology stacks.

Next, you are prompted for the groupId, artifactId, version and package:

[INFO] artifact org.apache.maven.archetypes:maven-archetype-webapp: checking for updates from maven-
archetype-webapp-repo
[INFO] artifact org.apache.maven.archetypes:maven-archetype-webapp: checking for updates from centra
l
Downloading: http://localhost:8081/nexus/content/groups/public/org/apache/maven/archetypes/maven-arc
hetype-webapp/1.0/maven-archetype-webapp-1.0.jar
3K downloaded
Define value for groupId: : com.wakaleo.onlinebank
Define value for artifactId: : onlinebank-core
Define value for version:  1.0-SNAPSHOT: :
Define value for package: : com.wakaleo.onlinebank.core

Finally, you can review your choices, and, if you are happy, go ahead and create the project.

Confirm properties configuration:
groupId: com.wakaleo.onlinebank
artifactId: onlinebank-core
version: 1.0-SNAPSHOT
package: com.wakaleo.onlinebank.core
 Y: :
  Y: :
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating OldArchetype: maven-archetype-quickstart:RELEASE
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.wakaleo.onlinebank
[INFO] Parameter: packageName, Value: com.wakaleo.onlinebank.core
[INFO] Parameter: package, Value: com.wakaleo.onlinebank.core
[INFO] Parameter: artifactId, Value: onlinebank-core
[INFO] Parameter: basedir, Value: C:\Users\john\projects\onlinkbank
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] ********************* End of debug info from resources from generated POM *******************
****
[INFO] OldArchetype created in dir: C:\Users\john\projects\onlinkbank\onlinebank-core
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27 seconds
[INFO] Finished at: Thu Aug 07 19:51:37 NZST 2008
[INFO] Final Memory: 8M/15M
[INFO] ------------------------------------------------------------------------

All in all, a very convenient way to create a Maven project structure with a minimum of fuss.

Melbourne talk on easyb

After the Java Emerging Technologies Conference 2008 (JET 2008) in Auckland, I will be giving a talk on Behavior Driven Development (BDD) in Java with easyb, a new and very hip behavior driven development framework for Java for the Victorian Java User Group in Melbourne, Australia.

This talk takes place at the same time as the Melbourne Java Power Tools Bootcamp - this is the first Australian bootcamp, and is looking like it will be a lot of fun!

Custom schema generation with Hibernate annotations

Hibernate 3 annotations are a great way to persist your domain model in a database backend. It's quite easy to have Hibernate generate or update the database schema as required, and not have to hand-code SQL scripts at all. However, this approach often hits a few hurdles when it reaches the DBA teams - they usually insist on obscure and archaic naming conventions, and sometimes even want to see the SQL script before modifying the producton database!

Luckly, you don't have to go back to the bad old ways of hand-coded SQL and convoluted XML mappings just yet - Hibernate annotations can cope with all this. For those interested, I've written up some details on how to do, including a few pit-traps to avoid, in new article on Javaworld.

Poll: What sort of unit testing does your team do?

Unit testing is generally considered to be a key part of software development. But how is unit testing really practiced in the industry at the moment? Do you test your code at all? Do you write your tests before or at the same time as your code (as in using a TDD or BDD-style approach), or do you wait until the application is written before coming back to the unit tests (e.g. "Yes boss, it's all done, I just have to write the unit tests")? Do you use test coverage metrics to see how much of your code is being executed during tests, and to isolate untested code?

Note, in this survey, the question is about practices in your team as a whole, not just you as an individual. This is to take into account the fact that people who hang around web sites like this are probably amongst the more IT-literate members of their teams, and the idea is to get a general picture of industry practices. Still, as usual, it's not a scientific survey by any means ;-).

Anyway, you can check out the pole here.

EasyB in action at the JET 2008 conference in Auckland

On September 17th I will be giving a talk on easyb, a new and very hip behavior driven development framework for Java, at the Java Emerging Technologies Conference 2008 (JET 2008), taking place in Auckland.

This talk will go through the basic principles of Behavior Driven Development, and look at how it builds on and differs from "traditional" Test-Driven Development. Then we move on to look at easyb, a very cool DSL-based behavior driven development framework for Java that uses Groovy to let you pretty much write tests that document themselves.

The talk will be very practical. We will look at how to write basic easyb test stories, how to use fixtures in easyb and of course how to integrate easyb tests into your build process. There will also be some live demos just to prove that it really is as easy as it looks! No knowledge of TDD, BDD or Groovy is required!

The conference is free to attend, so if you are in Auckland, and want to learn about behavior driven development in Java, come along!

Don't draw UML diagrams - let the machine do it for you!

My friend Paul Duvall, principal author of that great Continuous Integration reference "Continuous Integration", has written an excellent article on automating the process of generating technical documentation in "Automation for the people: Pushbutton documentation". For most developers, technical documentation is a pain to do, and is rarely complete or up to date, if it is ever done at all. However, much of what needs to be documented - class diagrams and database DDLs, for example - can be extracted directly from the source code.

In this article, he discusses how to generate Javadocs embedded with UML diagrams using UMLGraph, how to document your database with entity-relationship diagrams using that brilliant but little known tool SchemaSpy, and how to graphically document your Ant script using Grand. A picture tells a thousand words, as they say, and this is very true in the IT industry, where a few diagrams can do wonders for your understanding of a system.

Of course, tools will never be able to explain why you did things in a particular way, or what the intention behind it all is - so they are no excuse not to write good comments. Indeed, they rely on good comments to be really useful. Paul also talks about how to add steroids to your comments with DOxygen, and also how to write user documentation using DocBook.

All great stuff!

BDD with ease with Easyb

Behaviour-Driven Development, or BDD, is considered by many to be a natural extention of Test-Driven Development (or TDD). Test-Driven Development is about designing software with the tests in mind.This tends to make the detailed design of your classes cleaner, more modular and more flexible. In practice, it involves writing your tests at the same time as your code - before, simultaneously or slightly after. As a bonus, if you do it well, you get a high degree of test coverage and a good set of regression tests into the bargin.

But one of the biggest problems that people come up against is knowing what tests to write. It's very easy to write a set of superficial unit tests that execute virtually all of your code but actually test very little in semantic terms. Andy Glover covers this issue well in his article on Test Coverage.

BDD proposes an interesting solution to this problem - instead of naming your tests "testThis" or "testThat", you call them "shouldDoThis" and "shouldDoThat". Immediately, you are thinking about what your class "should" do - in other words, you are thinking specifications. The difference is subtle, but this turns out to be a much more intuitive way of driving development than thinking in terms of raw tests. Rather than saying "what do I test", you are saying "what should my class actually do", or "how should my class behave". Which is very much the spirit behind TDD.

Enter Easyb. Easyb is a BDD framework for Java. Basically, you describe your class's intended behaviour in a simple domain-specific language, using terms like "given", "when" and "then". The test cases are readable, and serve both as design documentation and as unit tests. Suppose for example you want to test your new bank account class. You need to ensure that, if you overdraw, the code throws an exception and that the account balance remains unchanged (we haven't implemented bank fees yet ;-) ). A simple Easyb test case to do this might look like this:

      given "a bank account containing $100", {
        bankAccount = new BankAccount(100)
      }
      when "$150 is withdrawn", {
		overdraw = {
			bankAccount.withdraw(150)
        }
	  }
	  then "an InsufficientFundsException exception should be thrown", {
         ensureThrows(InsufficientFundsException.class){
             overdraw()
         }
      }
      and "then the account should still contain $100", {
          bankAccount.balance().shouldEqual 100
      } 
	

EasyB uses a very readable Domain Specific Language (DSL), that you use to write very readable (pretty much self-documenting, in fact) test cases. Rather than reasoning in terms of asserts, or even assertThats, you describe your test case in terms of simple structured user stories. Instead of assert statements, you have expressions such as "shouldEqual", "shouldBeLessThan" and "shouldHave" (for collections), which have the same natural feel as Hamcrest expressions, and are just as readable. The code itself is in Groovy, which is close enough to Java to be very easy to pick up even if you have no prior experience with Groovy scripting.

So, if you're interested in Behavior-Driven Development, go check out the EasyB website to learn the groovy moves and disco danse steps of BDD with EasyB in the inimitable style of my good friend Andy Glover.

More praise for Java Power Tools

Positive reviews are still proliferating on the web about Java Power Tools book. The TechBookReport website gives the book an overall verdict of "Highly Recommended":

"Overall this is a great resource. It brings together all the major pieces one would want when building a development and build environment for Java. Each of the tools that is covered is worth looking at...each chapter can be read standalone and provides a complete tutorial for that tool...In short, this is deserving of a place on your bookshelf. Highly recommended."

The book also gets 5 stars on the O'Reilly web site, with one reviewer saying "Consider this book as doing the grunt work and giving you the ability to make a decision on which tools to invest your valuable time resource."

My good friend Andy Glover also talks about Java Power Tools on his blog saying, among other things, "it’s a vade mecum for anyone serious about producing working software effectively."

Architecture-Oriented or Feature-Oriented - how do you organise your development teams?

Traditionnally, there are two fundamental approaches when it comes to organising your development teams: the Architecture-Oriented approach and the Feature-Oriented approach. The first priviledges teams that focus on the different architectural layers or components, whereas the second prefers to organise teams around deliverable application features.

Architecture-Oriented teams are organised according to the system architecture of the application. For example, in a typical e-commerce application, you might have a web-tiers layer, a services layer and a database access layer. Using an Architecture-Oriented approach, you would have a team (or team-member, for very small projects) dedicated to each layer.

An Architecture-Oriented approach is good if you are a control-freak. Projects that use this approach tend to have a cleaner and more rigourous system architecture, and it is indeed easier to enforce a consistant coding and architecture style using Architecture-Oriented teams. In fact, it becomes natural - each team becomes very good at what they do, and the quality of the code in each layer is often very high. Interfaces between the layers are well-defined, clean and consistent, and it is easier to design and implement reusable components this way.

However, this approach does have some drawbacks. Building a software system this way requires a very high degree of upfront design and planning, which can lack in flexibility. On the one hand, integration is simpler because the interfaces between the layers are well-defined early on in the piece. But on the other hand, this integration is often untestable until very late on in the piece, because all the layers need to be finished before any feature can be properly tested. This makes practices such as Continuous Integration difficult to implement effectively.

An Architecture-Oriented approach also tends to lead to Taylor-like specialisation, and to a certain degree of knowledge-siloing. For example, only the database team really knows how the database layer works, and for everyone else it's a bit of a black box. This may or may not be considered to be a good thing. The job gets done well, but the high degree of specialisation may put the organisation at a risk if and when team members leave the organisation.

Feature-Oriented teams are organised by feature, or use case, or user story, or some other tangible and deliverable unit of functionality. For this iteration, Team A will take care of the Shopping Cart Checkout feature, whereas Team B will implement the full-text catalog search. Each team implements their feature from A to Z, so they usually have to touch all the layers of the application architecture.

Feature-Oriented teams are more dynamic - in effect, they are reorganised at the start of every iteration. This tends to encourage a broad knowledge of the system. In a Feature-Oriented approach, each team delivers tangible, fully-working and fully-testable chucks of functionality. Integration can be more complex, as the demarcation between individual teams is more fuzzy in terms of the actual source code, but Continuous Integration can be used much more efficiently to provide quick feedback about integration issues. And since each team is the master of all of the application layers, as far as their feature is concerned, change requests can be catered for much more easily. This makes a Feature-Oriented approach much more flexible.

However, as this is the real world, and nothing is black or white, a Feature-Oriented approach also has some drawbacks. This approach may foster a "Jack of all trades, but master of none" culture, where developers have a superficial knowledge of all system layers rather than an in-depth knowledge of one. This can also lead to architectural compromises, and the risk of poor or inconsistant code slipping into an area in which a developer is not fully familiar. In general, architectural integrity is harder to guarantee with a Feature-Oriented approach.

Personally, I prefer the flexibility of the Feature-Oriented approach, but keep a healthy respect for the clean architecture that is fostered by an Architecture-Oriented approach. One possible compromise that I have used on some of my projects goes along the following lines:

First, a crack team of experienced developers designs and documents a set of high-level architectural design guidelines along with a clean reference implementation that illustrates the target architecture in practice. This reference implementation is designed to provide examples to other team members as to how things are meant to be done. Eliminate the guesswork and arbitrary design decisions wherever possible.

Once the reference implementation is completed, the real project work can start, and the rest of the project team members can come on board. From here-on it's pretty much a Feature-Oriented approach, using a SCRUM-like agile development methodology. Continuous Integration and thorough unit testing help to smooth out the integration process. However, architectural integrity is still a potential issue. So informal code reviews and automatic code quality metrics (in particular, dependency analysis) help to keep an eye on the code quality in each layer. The crack team also stays on to act as recognised "gurus" in various domains, and to mentor the newer developers on the tricks of the trade.

And hopefully, this leads to a little of the best of both worlds.

Java Power Tools Bootcamps coming up in New Zealand

The next Java Power Tools Bootcamp is planned for Auckland, between the 11th and the 14th of August. This is the first Auckland session, and I'm certainly looking forward to it. Places are limited, so don't miss out!

And for those who missed out on the first Java Power Tools Bootcamp in Wellington, another bootcamp has been scheduled in Wellington for the 25th-28th of August.

The Java Power Tools Bootcamp is a comprehensive, innovative and hands-on workshop covering best-of-breed open source tools and techniques for Agile Development in Java. Learn how to optimize your development process, hone your programming skills and know-how, and ultimately produce better software. It covers key areas of software development and best practices, including:

• Agile Development Principles and Practices and Maven 2
• Unit and Integration testing with JUnit 4, Hamcrest, Selenium and so on.
• Code quality and documentation with tools like Checkstyle, PMD, FindBugs and Crap4J
• SCM and Continuous Integration with Subversion and Hudson

Performance Testing is not Scalability Testing

People often confuse performance and scalability testing, but they are actually quite different activities. Performance testing involves ensuring that your application responds to requests within an acceptable timeframe. Of course, defining what "acceptable" is is a fine art, and perceived performance (what the user actually sees) is often more important than real performance.

Performance testing may aim at testing the overall performance of the system as a whole. Or it can be a safeguard activity, designed to catch sub-optimal queries before they reach production, for example. In both cases, well-designed integration tests, that simulate typical user interactions, can serve as the basis of these performance tests. For simple uses, simple timers, or the JUnit 4.4 @Timeout annotation, can suffice. However, the integration tests should be designed well enough to accept variable data, to avoid database caching and other such issues. Tests using JUnit 4.4 Theories can be a good candidate for this sort of reuse.

Load testing, on the other hand, is about concurrent access - how does your application repond when lots of users start to use the system at the same time. Does it still respond to each user within a reasonable time? Does it degrade gracefully or come to a grinding halt? Load testing is harder and more technical than pure performance testing, as a lot more variables are involved.

Tools like JMeter, the Grinder and TestMaker are quite capable of executing JUnit tests in parallel, to simulate load. Now, if these tests are integration tests that reproduce a typical user interaction, you can obtain a quite reasonable set of load tests, at least for the Java layers of your application. These tests tend to be more stable than web-based tests, and can serve both as performance and regression tests.

Tools like Selenium can test your application from the web layer down. This is great for testing the whole application, including the web pages, and for regression testing. You can even embed Selenium tests in a JUnit test case and use them for load testing as well. This is fine, but the user interface of a web application is typically less stable than the service and DAO layers, and so the test scripts will probably require more maintenance.

Just my 2¢.

Unit tests are not integration tests

The classic difference between integration and unit testing is that unit tests run in isolation, or near-isolation. Using an in-memory database for DAO unit tests. Using mocked-out components for the other layers. Integration tests, on the other hand, test the whole stack. If you're using Spring, you inject the whole Spring context and try to flush out any configuration issues, or integration issues between the various layers.

Integration tests tend to run slowly. Unit tests run fast. You don't want to slow down your quick coding turn-around time by running the integration tests every time - run them less frequently than the unit tests (there's usually no need to retest the Spring configuration files if you've only changed a couple of lines of code - you'll do this before you commit, anyway).

Integration tests often have more of a BDD feel to them. They simulate, to some extent, expected user behavior. Unit tests can do this too, but integration tests do so at a higher level. This means they can be candidates for reuse later on for performance and scalability tests. This is even better if you add some parameters to your integration tests (e.g. by using JUnit 4.4 Theories) - this makes them much easier to reuse for more heavy-weight testing. I'll discuss this in more detail in the next installment.

Personally, I materialize this distinction using naming conventions (e.g. ClientTest for a unit test, ClientIntegrationTest for an integration test). Then it's pretty easy to get each category of test to run at different times in the build cycle. In maven, it's as simple as 20 or so lines of XML :-), as shown here:

            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <skip>true</skip>
                </configuration>
                <executions>
                    <execution>
                        <id>unit-tests</id>
                        <phase>test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <skip>false</skip>
                            <excludes>
                                <exclude>**/*IntegrationTest.java</exclude>
                            </excludes>
                        </configuration>
                    </execution>
                    <execution>
                        <id>integration-tests</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <skip>false</skip>
                            <includes>
                                <include>**/*IntegrationTest.java</include>
                            </includes>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

A bit cumbersome, I admit. But, with this, you can use the standard Maven test command to run uniquely the unit tests:

mvn test

Then, to run just the integration tests, you run:

mvn integration-test

Naturally, you can do the same thing at a lower level in Ant with the JUnit Ant task.

In TestNG, you can use test groups - this is one of the nice features of TestNG compared to JUnit 4.4.

Next time: Performance Testing is not Scalability Testing

New features in Hudson

I've just been playing with the latest release (1.222). One of the nicest new features is the Plugin Manager. It looks and feels very much like a web version of the NetBeans plugin manager (maybe this isn't really suprising ;-) ). If you aren't familiar with the Netbeans plugin manager, you get four tabs:

• An Updates tab, where you can see which of your installed plugins are out of date, and upgrade them;
• An Available tab, where you can browse the plugins available on the Hudson web site, and install them if you want. If you choose to install them, it will download and install them for you - this is so much easier than doing it by hand!
• An Installed tab, where you can see what plugins are currently installed on your server;
• And an Advanced tab, where you can configure your proxy if necessary, or upload a plugin by hand.

Actually, the Hudson project is a good example of the application of agile delivery principles. A new stable version is released at least once a week. Even so, in my experience, bugs are rare and upgrades have always gone without a hitch. Indeed, with such a frantic release schedule, the stability of Hudson is truely impressive. You're the man, Kohsuke!

September 2008

Search this blog:

Categories

Archives

September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
October 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
November 2006

Recent Entries

Hudson project-based matrix security is out!

Using the Maven Release Plugin

Replication in Subversion 1.5

Articles

Integrating Maps into Your Java Web Application with Google Maps and Ajax
Want to provide maps in your web application? The Google Maps API is straightforward to call from Java, and with an Ajax-ian approach, you can make it extra user-friendly. John Ferguson Smart shows you how to combine these approaches. Oct. 26, 2006

Instant Messaging in Java Made Easy: The Smack API
Jabber is a popular and widely supported XML-based API for exchanging instant messages. You could compose the messages by hand, but there's an alternative. John Ferguson Smart introduces the Smack API, which makes it easy to use Jabber services from Java. Oct. 5, 2006

Web Services Made Easy with JAX-WS 2.0
Standards are so much easier to adhere to when your tools do it for you. Thanks to JAX-WS and its implementation in application servers like GlassFish, you can write web services as plain ol' Java objects, just by adding a few annotations. John Ferguson Smart shows how it's done. Jun. 13, 2006

All articles by John Ferguson Smart »

This work is licensed under a Creative Commons License.