David Walend's Blog

David Walend started learning Java with the alpha 3 release in 1994 after a kind computer science professor at Tufts University overheard his tantrum on distributed simulations, memory management, multithreaded code and meteorologists of questionable parentage. His active open source projects include Somnifugi JMS, JDigraph, Cricket Cage, and JEnum. At Alphatech David researches systems for coordinating aircraft with unreliable communications. He is an adviser to Galactic Village Games. David should finally finish his masters project (combining genetic programming and reinforcement learning) at Tufts in 2004.

Good-bye, Alpha

I released SomnifugiJMS v22 a few weeks back. Not alpha-0-22. This subtlety is a big change in approach for a small change in code. I make fewer changes in each release, and the extremely stabile JMS specification is only so big. I've added most of the features I intend to implement. After eight years of work, six years in production systems, and 21 alpha releases it's starting to feel done. I'd like to claim that I came to this conclusion on my own, but others had a big influence.

I hadn't thought to do it at all until I read Kirill Grouchnikov's series of articles on his approach to open source projects. Kirill says point blank not to do alpha releases: "...in a virtual world this means stripping the alpha / beta / 0.* status from your project. You have very short time window to impress the potential users, and a solid release number with no greek letters is a must-have. This is the least you can do after six years of development." Kirill's approach is not like mine, but his words got me thinking.

Bruno Ghisi pointed out a remarkable survey on why we do open source work. I suspect Kirill is much closer to the "professionals" and "believers" groupings than I am. I straddle the "skill enhancers" and "fun seekers" groupings. I write most of my code after hours, even if it is for work instead of hobby. I pick projects that teach me about new technologies and keep my skills sharp. Further, I enjoy the meditative nature of coding the way other people enjoy watching TV. It's something I do to relax, clear my head and settle down at the end of the day. My only nod to the "professionals" is that I wait to register new open source projects until I have a first release ready to go. However, for me to stay focused on a project without pay, that project has to entertain me or at least teach me something.

Alpha labels on my projects are a warning to everyone: This code isn't ready for prime time. I'll change the API if I need to, and I think I'll need to. And I won't apologize for it.

SomnifugiJMS is easy for me to recognize as mature. The JMS specification has been stabile for six years. SomnifugiJMS supports most of it. I have no plans to support the missing parts. SomnifugiJMS is done except for maintenance releases.

JDigraph is a very different story. The project is, among other things, an experiment on how to create good, clear API. Splitting mutators off from accessors worked amazingly well. Adding type specifiers for nodes and edges taught me how to use generics well. That part of the code is now very stabile. Building generic graph minimization algorithms exposed some big weaknesses of the Java language change that no one was talking about. I want to fix that, although it really needs operator overloading to do well. (I'll probably use that project to teach myself Scala.) Eventually I'll snap a line, cut out the failed experiments, and change over to maintenance releases. But not just yet.

SalutafugiJMS is in between. I finished its second release a few weeks ago. I've covered about a quarter of the big challenges in JMS. It's JMS again, so others determined the API six or more years ago. However, I know I want to switch over to an open, pure Java ZeroConfig library underneath -- a big code quake. I marked the second release as alpha. Maybe release four will have enough of the JMS specification to drop the alpha label.

JMX and Test-Driven Development

At JavaOne this year I did a short talk on using JMX in test-driven development. The talk was based on what I discovered working with JMX on another project, and delved deeper into while adding JMX support to SomnifugiJMS as part of release 21. Test-driven development worked extremely well when combined with JMX. JMX should help testing systems with some defined life cycle.

Test Driven Development

Test driven development maximizes the benefits of writing tests by demanding that we write the tests before the code that will attempt to pass those tests. It ensures some level of quality because all work completed will pass at least one test. Developers establish a rapid, highly iterative development loop. The tests provide an easy way to measure progress. Managers enjoy having the detailed insight that a list of tests gives them. When I've got a little time away from the day job to do open source work, I can write a small test, then get back to making the test pass later.

Test code and JMX monitoring code have a shared programatic problem. We promise ourselves that we'll go back and write test code later, and will add JMX monitoring code before we need it. Deadlines loom large, we get things working, and move on to the next project without doing either job well. Test driven development breaks that vicious cycle for tests. Using JMX in the tests breaks the cycle for both. (Thanks to Chris for pointing this out after my talk.)

The test driven development loop is pretty simple. First, decide what feature to implement next. Design the API for that feature. Next, write a test for that feature and observe that all the other tests still pass. Finally write code to make the new test pass. Repeat until tests of all the features pass. Like John says, you can write the tests just after the code so long as you create tests in the same flow and are done before your work escapes your head and your desktop. Your code will have tests before it leaves the cradle.

Test driven development presents some challenges. You have to have some clue about what features to develop before starting the loop. You need to have some idea about the classes and methods that will form your API. Test driven development works best for black-box testing, so you'll have to expose enough of your system to test your work. In Java, that often means adding public access to classes and methods, hacking in via reflection, or excising test code out of packages and classes after you run your tests. JMX can help with these challenges somewhat, by providing an alternative way to get information out.

JMX

JMX is the Java Management eXtension API (from the late 1990s -- the X dates it). JMX provides a standard way for Java systems to report their state and health to the outside world, and to allow themselves to be adjusted and tuned via standard tools. It provides two main styles of interacting with the running system: MBeans expose select getters, setters and methods to outside control. Notifications push information out from the Java system to listeners.

MBeans expose select methods on a class to the JMX system. Java classes follow a special design pattern -- the MBean -- and registering instances of the class with an MBean server. An MBean interface defines the exposed methods. You could use this interface to expose useful attributes to make them observable in tests, and to set attributes and call methods in order to drive tests. For testing, the MBean interface doesn't provide any great advantage over just using public methods. However, I found I could use the MBean's existence in the MBean server to provide a simple test of an object's lifecycle.

JMX also specifies using Notifications to signal lifecycle and state changes from inside objects via the speaker/listener pattern. Add NotificationListeners to objects that implement the NotificationEmitter interface. When something of note happens within the MBean object, have it send a Notification to the listeners. This speaker/listener pattern works well in testing. It provides a good alternative to exposing state because the exposure is controlled by your code sending the Notifications, not via public method calls to your object. Further, it lets you write tests of lifecycle changes instead of tests based on methods. Lifecycle events are easier to get right early in development, generally easier to change, and are less likely to change over time than method signatures. Your tests will be less brittle and you'll spend less time rewriting them.

JMX-Based Tests

I found a few key features to make the code easy to test, and a few key features to test. MBeans will be easier to test if each MBean knows its ObjectName. Tests should check that expected MBeans exist. Because the MBean server will hold on to stray references, tests should check that MBeans are cleaned up at the end of their useful life. Notifications need to be machine-understandable; I subclassed Notification and added an enum field. Define the MBean interface and the Notification help in the same file to make it easier for others to find the Notifications. Tests should check that the code sent the expected Notifications exercised during the test run. These code examples are from SomniMessageConsumer.java, SomniMessageConsumerMBean.java and MonitorTest.java.

Each MBean Knows its ObjectName

To make each MBean know its object name, put the method in the MBean interface:

public interface SomniMessageConsumerMBean
{
    public ObjectName getObjectName();
    ...
}

and fill it in:

public abstract class SomniMessageConsumer 
    implements MessageConsumer, SomniMessageConsumerMBean, NotificationEmitter
{

...
    private final ObjectName objectName;

    protected SomniMessageConsumer(String name,
                                             Takable feed,
                                             SomniExceptionListener somniExceptionListener,
                                             String destinationName,
                                             SomniSession session)
    {

...
        try
        {
            objectName = new ObjectName("net.walend.somnifugi:type="
                                                       +this.getClass().getName()+",
                                                       name="+name);
            registerWithJMX();
        }
        catch(MalformedObjectNameException mone)
        {
            throw new SomniRuntimeException("JMX problem with "+this.getName(),mone);
        }
    }

Using the MBean in your test code becomes easy; you aren't constantly writing queries to find the right MBean.

        expectedMBeans.add(subscriber.getObjectName());

Tests Check for Expected MBeans

I check what MBeans were registered in the server at least twice per test, so I carved out a method to make sure that everything was there, with nothing extra, using an MBean query.

    @SuppressWarnings("unchecked")
    void checkExpectedMBeanNames(Set<ObjectName> expected)
        throws MalformedObjectNameException,
                   MBeanException,
                   AttributeNotFoundException,
                   InstanceNotFoundException,
                   ReflectionException
    {
        MBeanServer server = ManagementFactory.getPlatformMBeanServer();

        ObjectName query = new ObjectName("net.walend.somnifugi:*");
        Set<ObjectName> result = (Set<ObjectName>)server.queryNames(query,null);
        
        Set<ObjectName> remainder = new HashSet<ObjectName>(result);
        remainder.removeAll(expected);
        
        assertTrue("Remainder is not empty: "+remainder,remainder.isEmpty());
        
        assertEquals(expected,result);
    }

Then I fell into a regular pattern in the tests -- The code makes all the JMS parts, then makes sure they are all in the server.

        SomniTopicConnection connection = (SomniTopicConnection)SomniJNDIBypass.IT.getTopicConnectionFactory().createTopicConnection();
        SomniTopicSession session = (SomniTopicSession)connection.createTopicSession(false,Session.AUTO_ACKNOWLEDGE);
        String topicName = "testNotifyTopicSubscribers";
        SomniTopic topic = SomniJNDIBypass.IT.getTopic(topicName);
        SomniTopicPublisher publisher = (SomniTopicPublisher)session.createPublisher(topic);
        SomniTopicSubscriber subscriber = (SomniTopicSubscriber)session.createSubscriber(topic);

        connection.start();
        
        MBeanServer server = ManagementFactory.getPlatformMBeanServer();

        Set expectedMBeans = new HashSet();
        expectedMBeans.add(connection.getObjectName());
        expectedMBeans.add(topic.getObjectName());
        expectedMBeans.add(session.getObjectName());
        expectedMBeans.add(publisher.getObjectName());
        expectedMBeans.add(subscriber.getObjectName());
        checkExpectedMBeanNames(expectedMBeans);

Tests Check that MBeans are Cleaned Up

The second call to checkExpectedMBeanNames() makes sure that the test cleans out all of the MBeans at the end of the test. If the code doesn't clean them out, the server holds onto a reference, which prevents the garbage collector from reclaiming the memory. Some of these objects hold many references, so a leak can use a lot of memory.

        connection.close();
        SomniJNDIBypass.IT.removeTopic(topicName);
        
        expectedMBeans.clear();
        checkExpectedMBeanNames(expectedMBeans);
    }

Machine-Understandable Notifications

For a finer lifecycle test, I created machine-understandable Notification subclasses. Notification comes with a String type, Object source, and a long sequenceNumber. The source and sequence number are pretty straightforward to use. The type String offers very little. I made a subclass to carry a machine-understandable enum and to help with consistency in the type string. This enum has to implement SomniNotificationType. There's no way to enforce that in Java, but it's at least javadoced. getSomniNotificationType() provides a way to check the enum in code.

@SuppressWarnings("serial")
public class SomniNotification
    extends Notification
    implements Serializable
{
    private final SomniNotificationType noteType;
    private final String partName;
    
    public SomniNotification(SomniNotificationType noteType,
                                         Object source,long sequenceNumber,
                                         String partName)
    {
        super(noteType.getTypeString(),source,sequenceNumber,partName + noteType.getVerbPhrase());
        
        this.noteType = noteType;
        this.partName = partName;
    }

    public SomniNotificationType getSomniNotificationType()
    {
        return noteType;
    }
    
    public String getPartName()
    {
        return partName;
    }

    
    
    /**
    To be implemented by an enum defined in the MBean interface.
    */
    public static interface SomniNotificationType
    {
        public String getTypeString();

        public String getVerbPhrase();
    }
    
}

Collocate the MBean interface and Notification Enum

I put the notification enum in the same file, as a static inner class of the MBean interface. This grouping means there's only one place to look in source code to understand how the MBean interacts with a JMX monitor.

public interface SomniMessageConsumerMBean
{
...
    @SuppressWarnings("serial")
    public static enum SomniMessageConsumerNotificationType
        implements SomniNotification.SomniNotificationType, Serializable
    {
        CONSUMERCLOSED("net.walend.somnifugi.ConsumerClosed"," closed"),
        MESSAGECONSUMED("net.walend.somnifugi.MessageConsumed"," consumed"),
        MESSAGELISTENERCHANGED("net.walend.somnifugi.MessageListenerChanged"," changed"),
        MESSAGETIMEDOUT("net.walend.somnifugi.MessageTimedOut"," timed out");
        
        private final String typeString;
        private final String verbPhrase;
        
        private SomniMessageConsumerNotificationType(String typeString,String verbPhrase)
        {
            this.typeString = typeString;
            this.verbPhrase = verbPhrase;
        }
        
        public String getTypeString()
        {
            return typeString;
        }
        
        public String getVerbPhrase()
        {
            return verbPhrase;
        }
    }

Tests Check Notifications Sent During the Run

I created TestNotificationListener to test that the system was producing the right notifications. It takes a list of expected notifications in the constructor, and verifies that all the notifications came in the right order inside a check() method.

    private static class TestNotificationListener
        implements NotificationListener
    {
        private List expected;
        private List received = new ArrayList();
        
        TestNotificationListener(List expected)
        {
            this.expected = expected;
        }
        
        public void handleNotification(Notification notification, Object handback)
        {
            SomniNotification somniNotification = (SomniNotification)notification;
            received.add(somniNotification.getSomniNotificationType());
        }
        
        void check()
        {
            assertEquals(expected,received);
        }
    }

In the test, I give the TestNotificationListener a list of notifications, do things during the test, and make sure the right notifications were sent. This approach lets me define the expected lifecycle steps within the class, and observe them from outside, without using public methods.

        List expected = new ArrayList();
        expected.add(SomniMessageConsumerMBean.SomniMessageConsumerNotificationType.MESSAGECONSUMED);
        expected.add(SomniMessageConsumerMBean.SomniMessageConsumerNotificationType.MESSAGECONSUMED);
        expected.add(SomniMessageConsumerMBean.SomniMessageConsumerNotificationType.CONSUMERCLOSED);
        TestNotificationListener listener = new TestNotificationListener(expected);

        Object handback = null;
        server.addNotificationListener(name,listener,null,handback);

        ...//Lots of code for the test
        
        listener.check();

Where This Technique Will Work

Using JMX for test-driven development will work very well in some cases, but doesn't fit all problems. If you want to support JMX MBeans and Notifications, using this technique is an easy right decision. If objects in your system have nontrivial lifecycles, it is a very good fit. If you aren't sure about the methods your system should support, but know more about the lifecycle of the objects, it should work well. SomnifugiJMS was a particularly good fit for JMX-based tests. I wanted to add JMX support. The Java Message Service specification defines a horde of parts used to ship message around. Each part has its own lifecycle. Plus I wanted to remove non-JMS public methods, and to use JMX to support more complex tests.

Some systems may have trouble using this technique. JMX has some overhead; if you are concerned about how much CPU and memory goes into creating and shuffling strings for logging, JMX may be too heavyweight. It's not a very good fit for data structure or algorithm code, where the public API is the main focus. Systems with no lifecycle won't get much benefit. JMX-based tests written for systems with extremely complex, changing lifecycle may be too brittle.

References

Wikipedia has a non-volatile description of test-first development. I found this when I was looking for something with minimal hype and political agenda. The JMX reference from Sun contains examples of how to use JMX in your source code. The bias in the document is more toward monitoring existing systems -- it does a great job of telling how to hook up monitors via different protocols, but a skilled programmer see how to use stock JMX classes in an existing system via delegation. Eamonn McManus' Eamonn McManus' blog is the best source for what's coming next in JMX.

July 2008

Search this blog:

Categories

Archives

July 2008
June 2008
November 2007
June 2007
May 2007
March 2007
December 2006
November 2006
September 2006
July 2006
June 2006
May 2006
January 2006
August 2005
April 2005
March 2005
January 2005
December 2004
November 2004
August 2004
July 2004
June 2004
May 2004
January 2004
October 2003
September 2003

Recent Entries

Good-bye, Alpha

JMX and Test-Driven Development

How big is that boolean[]?

Articles

j1-2k8-mtW07: JMX for Unit Tests in Test-Driven Development
Using the Java Management Extensions -- JMX -- to observe internal state provides an elegant alternative to reflection and compiler hacks. As a byproduct it provides a JMX interface for the completed system. This talk will cover how to benefit from unit testing with JMX, and the code and overhead required to use the technique. May. 8, 2008

Understanding Service Oriented Architecture
Do you feel like software vendors and conference speakers have stretched the meaning of "service oriented architecture" to the point where anything is an SOA? Do you even know what the term should mean any more? In this re-introductory article, David Walend offers an overview of what true SOAs are, how they work, and what they can do for you. Apr. 4, 2006

All articles by David Walend »

This work is licensed under a Creative Commons License.