Tom White's Blog

Testing Archives

jMock 2 and my Java Unit Testing Toolkit

The long-awaited final version of jMock 2 was released today. There are some big changes since version one. For example, you can now write

Cat cat = mock(Cat.class);

and then set expectations on the returned cat object itself:

checking(new Expectations() {{
    one(cat).miaow();
}});

(This means that the miaow method is expected to be called exactly once on the cat object.)

This change means that you can refactor your method names without breaking your tests, unlike jMock 1 where the method names were strings:

Mock mockCat = mock(Cat.class);
mockCat.expects(once()).method("miaow")

You may have noticed that the new syntax is a little weird (check out the double braces, and how e.g. will modifiers need new statements), and it may take a little getting used to, but I think it's worth it for the IDE completion and refactoring support. For example, in test driven mode I can write the name of the method that doesn't yet exist then get my IDE to create the method for me.

The other nice thing is the integration with Hamcrest (a library of matchers that I've written about before). My toolkit for writing new unit tests now includes:

1. Hamcrest for making flexible assertions.
2. jMock 2 for defining expected behaviour.
3. Responsible Naming for understanding what the test does.

Testing for errant network connections

We kept breaking our XML catalog resolution in the course of developing an application. We would refactor the parser code, or we would upgrade a schema and forget to upgrade the catalog. The application wouldn't break, but it took longer to run since resources were being retrieved over the network rather than using the local catalog. Because we didn't time our test runs, and because we had lots of non-network dependent tests in the suite, this regression would go unnoticed for a while. When it was noticed we'd fix the symptom, then move on. Until it happened again...

After the sixth or so occurrence in few years, I wrote a class to detect the problem. It's a simple implementation of SecurityManager that throws an exception when an attempt is made to connect to a site that is not on the list of approved hosts. The code appears below.

To use it we set the -Djava.security.manager command line argument (to the fully-qualified classname of RestrictedNetworkAccessSecurityManager) when running our test suites. Tests that access hosts that we aren't expecting will then fail with an error.

It's not just for applications that use XML catalogs. It's useful for almost any code, as a way to audit - and regression test - the network resources your application depends on. (Like a database you thought wasn't being used any more.) It's also a simple way to discover when third-party libraries connect to the internet unannounced.

The code is rough and ready - it's good enough for testing, but not really suitable for anything else, since it is totally permissive except for the checks on outbound connections. Feel free to use it, and suggest improvements or ways you've tackled the same problem in the comments section.

import java.net.SocketPermission;
import java.security.AccessControlException;
import java.security.Permission;
import java.util.Arrays;
import java.util.List;

public class RestrictedNetworkAccessSecurityManager extends SecurityManager {
  private static final List<String> ALLOWED_HOSTNAMES = Arrays.asList(new String[]{
    "localhost", "127.0.0.1",
  });

  @Override
  public void checkConnect(String host, int port, Object context) {
    checkConnect(host, port);
  }

  @Override
  public void checkConnect(String host, int port) {
    if (host == null) {
      throw new NullPointerException("host can't be null");
    }
    if (!host.startsWith("[") && host.indexOf(':') != -1) {
      host = "[" + host + "]";
    }
    if (ALLOWED_HOSTNAMES.contains(host)) {
      return;
    }
    String hostPort;
    if (port == -1) {
      hostPort = host;
    } else {
      hostPort = host + ":" + port;
    }
    String message = "Opening a socket connection to " + hostPort + " is restricted.";
    throw new AccessControlException(message, new SocketPermission(hostPort, "connect"));
  }

  @Override
  public void checkPermission(Permission perm) {
    // Allow all other actions
  }

  @Override
  public void checkPermission(Permission perm, Object context) {
    // Allow all other actions
  }
}

Hamcrest

In Literate Programming with jMock I enthused about jMock's idea of constraints and flexible assertions. Now the jMock team has released version 1.0 of Hamcrest, the constraints part of jMock.

Hamcrest matchers (what were called constraints in jMock) are actually useful for more than just writing unit tests, but it is their application in writing assertions where they really shine and will probably see most use.

So now I can write

assertThat(a, equalTo(b))

or even

assertThat(a, is(equalTo(b)))

rather than JUnit's

assertEquals(b, a)

Apart from being more readable, assertThat takes any matcher as the second argument, so I can combine matchers or even write my own rather than creating an ever growing listof assertXxx methods. For example, I can say such things as

assertThat(collection, hasItem(anyOf(is(item1), is(item2))));

And since the matchers (and the assertThat method) are accessed using static imports, I can use them in any test framework I like.

So why not read the tutorial and give Hamcrest a go?

Lift Off

In a previous blog entry I mentioned a literate functional testing framework that we had developed at our company, Kizoom. The framework was initially developed by my colleague Robert Chatley for testing a rather obscure digital TV XML application. After that project we produced a new version of the framework for testing HTML applications, which we used internally for a number of projects with great success. Last month we spend a bit of time cleaning up the API and released LiFT, short for Literate Functional Testing, as an open source project on java.net.

Here I'd like to give a flavour of LiFT and why you might want to use it.

When you set out to test your web application you need to think about the mechanics of how you interact with the application. Typically this involves fetching pages, finding elements on the page that you want to read (so you can make an assertion about them) and elements you want to change (so you can simulate user input). Often, the code required to handle these aspects obscures the intent of the test. Consider this HttpUnit test to do a Google search for an image of our ill-fated company summer party of two years ago:

import junit.framework.TestCase;

import com.meterware.httpunit.WebConversation;
import com.meterware.httpunit.WebForm;
import com.meterware.httpunit.WebImage;
import com.meterware.httpunit.WebResponse;

public class HttpUnitGoogleTest extends TestCase {

    public void testGoogleImageSearch() throws Exception {
        WebConversation conversation = new WebConversation();
        WebResponse page = conversation.getResponse("http://www.google.com/");
        assertEquals(page.getTitle(), "Google");
        page = page.getLinkWith("Images").click();
        WebForm form = page.getForms()[0];
        form.setParameter("q", "kizoom");
        page = form.submit();
        assertTrue(page.getText().contains("<b>Kizoom</b> summer party"));
        boolean foundImage = false;
        for (WebImage i : page.getImages()) {
            if (i.getSource().endsWith("summer04.jpg")) {
                foundImage = true;
                break;
            }
        }
        assertTrue(foundImage);
    }

}

Hmm, not too clear what it's testing. I'm not trying to single out HttpUnit here - it's a great tool for interacting with web applications, but it's pretty low-level. Of course, I could abstract some of the operations into helper methods to make the test clearer. For instance, the bit at the end where I'm looking for an image is ripe for extracting as a method. In fact, this is the approach we took for years - building libraries of helper functions that made using HttpUnit a bit less of an effort. However, it's not a structured approach. We can do better.

import com.kizoom.lift.NavigatingLiftTestCase;

import static com.kizoom.lift.constraints.Constraints.*;
import static com.kizoom.lift.matcher.Matchers.*;

public class GoogleTest extends NavigatingLiftTestCase {

    public void testGoogleImageSearch() throws Exception {
        goTo("http://www.google.com/");
        assertThat(page, has(title("Google")));
        clickOn(link("Images"));
        enter("kizoom", into(textField()));
        clickOn(button("Search Images"));
        assertThat(page, has(text("Kizoom summer party")));
        assertThat(it, has(image().withUrlThat(endsWith("summer04.jpg"))));
    }
}

This is much more concise. But more importantly it is much more readable. Each line can be read as a sentence, either as a jMock-style flexible assertion starting with "assert that", or as a command, such as "click on".

Page navigation is handled for us - we are provided with a page variable which always refers to the current page we are on.

Perhaps the most powerful idea is in the use of matchers. Matchers allow us to refer to bits of the page - so we can assert something about them or do something to them. For example, title("Google") creates a matcher that matches HTML title elements with text content "Google". We then use the has constraint to assert that there is (at least) one such match. Or take the next line: link("Images") matches a link with text "Images" which we use to navigate to the next page by means of the clickOn action.

Matchers allow for extensibility - it is easy to write your own if you need to - and help avoid the creation of ad hoc helper method libraries. Often you don't even need to go this far, you can refine an existing matcher using any jMock constraint. This is what is done in the last line. The withUrlThat refinement on the image matcher limits the matches to those that meet the given constraint. In this case we use the jMock endsWith constraint to match the image filename.

LiFT still has some rough edges - for example, its error messages are not always as clear as they could be - but it is definitely ready for testing real web applications. There's much more I could say, but I won't here. If you want to learn more try LiFT out by following the introduction, or read the slides from the talk Robert and I gave at the Google London Test Automation Conference in September, or even watch the video.

More Literate Programming with jMock: Anaphora

According to the dictionary, an anaphor is a word used to avoid repetition. It refers back to something in the conversation. The word "it" in the previous sentence refers back to the word "anaphor" in the first sentence, so "it" is an anaphor for "anaphor". Natural language is often ambiguous, and one reason for this is that it may not be clear which word an anaphor such as "it" is referring to.

But ambiguity and programming languages don't go very well together - so why would anyone want to mix the two? Actually, there are circumstances in programming languages where there is no real ambiguity, and an anaphor can have a use in eliminating repetition. Think of it as applying the DRY (Don't Repeat Yourself) principle at the syntax level.

Continue Reading...

Literate Programming with jMock

We've been using jMock at our company for some time now. We've found it great for test driven development and isolating our unit tests from the rest of the system more effectively. One aspect of jMock that stands out for me is its idea of constraints. In fact, we've found this idea so useful that we always use the org.jmock.MockObjectTestCase base class rather than junit.framework.TestCase, even when we aren't mocking anything out. This seems to have registered with the jMock development team, as they are planning to extract the constraints into a separate project, codenamed Hamcrest (it's an anagram of "matchers").

Continue Reading...

This work is licensed under a Creative Commons License.