Eamonn McManus's Blog

J2SE Archives

Defining MBeans with annotations

The number one question I get about the JMX API at conferences and other public events is whether there will be support for defining MBeans using annotations. People see that they can make EJBs or Web Services just by adding annotations to a POJO, and they ask why they can't make MBeans the same way. In version 2.0 of the JMX API, being defined by JSR 255, this will be possible.

The exact details are still subject to change as a result of discussions within the JSR 255 Expert Group, but here's a snapshot of where we are now. I think the final version will be fairly close to this.

In addition to defining MBeans with annotations, there are some new proposed annotations that will also apply to MBeans defined in the existing ways.

Prior art

Several projects already exist that provide this functionality, but the most developed is probably Spring. So our starting point was Spring's MBean annotations (see also API documentation).

Defining an MBean

In the proposed design, an MBean can be defined through annotations to achieve the same effect as a Standard MBean. In this and other examples, I'll show what you write with today's API, and what you'll be able to write with the new annotations.

public interface CacheMBean {
    public int getSize();
    public void setSize(int size);

    public int getUsed();

    public int dropOldest(int n);
}

public class Cache implements CacheMBean {

    public int getSize() {...}

    public void setSize(int size) {...}


    public int getUsed() {...}


    public int dropOldest(int n) {...}
}

 







@MBean
public class Cache {
    @ManagedAttribute
    public int getSize() {...}
    @ManagedAttribute
    public void setSize(int size) {...}

    @ManagedAttribute
    public int getUsed() {...}

    @ManagedOperation
    public int dropOldest(int n) {...}
}

This defines an MBean with read-only attribute Used, read-write attribute Size, and operation dropOldest.

I'll call an MBean defined this way an @MBean.

One way to look at this is that with the existing Standard MBeans, public methods from the class are picked out as being management methods by virtue of being in the Standard MBean interface that the class implements. So in this example the CacheMBean interface defines which methods in Cache are the management methods. In the new form, the methods are picked out by being annotated, and there is no need to define an interface.

Pros and cons of @MBeans

The new style appears considerably more convenient than Standard MBeans. You only have to maintain one source file, rather than managing a class and an interface.

There is a downside, however, which may show up in bigger projects. The advantage of the Standard MBean approach is that the MBean interface tells you exactly what the attributes and operations of the MBean are. There is no extraneous information in the MBean interface: every method defines an attribute or an operation.

On the other hand, with @MBeans the management attributes and operations are potentially mixed in with many other methods, public or private. So it is not immediately obvious what the management interface of the MBean is.

This disadvantage applies both when reading the source code and when looking at the Javadoc output.

A second disadvantage is that it is no longer possible to construct a proxy. Proxies simplify client code by allowing it to access attributes and operations as compiler-checked method calls. They don't matter if you are only going to interact with your MBeans through a graphical interface like JConsole, but they are a big help if you are writing an application that will interact with your MBeans specifically.

For smallish projects, these disadvantages are likely to be minor. Furthermore, it should be possible to define an annotation processor that extracts a Standard MBean interface from an @MBean, so it can be used for documentation and proxying. In the example above, the annotation processor could create the CacheMBean interface every time you compile your program, based on the @ManagedAttribute and @ManagedOperation annotations in the Cache class.

Defining an MXBean

The existing @MXBean annotation can be used instead of @MBean to define an MXBean rather than a Standard MBean.

@MXBean
public class Cache {
    ...remainder as above...
}

Descriptions

Although the JMX API allows for textual descriptions to be associated with attributes, operations, and parameters, when you use a Standard MBean today these descriptions have meaningless default values. I've written before about how you can add meaningful descriptions, but it isn't easy. This is a really obvious use for annotations.

The proposed new @Description annotation can be used with Standard MBeans, MXBeans, and @MBeans. (Notice that both columns use the new API here!)

@Description("some sort of cache")
public interface CacheMBean {

    @Description("number of cache slots in use")
    public int getUsed();
    ...
}

public class Cache implements CacheMBean {

    public int getUsed() {...}
    ...
}

@Description("some sort of cache")
public class Cache {
    @ManagedAttribute
    @Description("number of cache slots in use")
    public int getUsed() {...}
    ...
}

We international types will of course be thinking about internationalization, and I'll have more to say about that below.

Finding the MBeanServer and/or ObjectName

Often an MBean needs to know what MBean Server it is registered in, or what its name is in that MBean Server. To do this it currently needs to implement the MBeanRegistration callback interface. The required values are passed to that interface's preRegister method. But the interface contains three other methods, which the MBean must implement even if it has nothing interesting to do in them.

In the new proposal, the @Resource annotation from javax.annotation can be used instead of implementing MBeanRegistration when all that's needed is to discover what the MBeanServer or ObjectName is:

 
public class Cache
    	implements CacheMBean, MBeanRegistration {

    private volatile MBeanServer mbs;

    private volatile ObjectName myName;

    public ObjectName preRegister(
    	    MBeanServer mbs, ObjectName name) {
    	this.mbs = mbs;
    	this.myName = name;
    	return name;
    }
    public void postRegister(Boolean done) {}
    public void preDeregister() {}
    public void postDeregister() {}

    ...
}

@MBean
public class Cache {

    @Resource
    private volatile MBeanServer mbs;
    @Resource
    private volatile ObjectName myName;











    ...
}

When the MBean is registered, the MBean Server will inject the appropriate values into these fields.

This possibility is open to all types of MBeans, not just @MBeans. You could continue to have a Standard MBean as today, but stop implementing MBeanRegistration in favour of @Resource annotations.

Simplified notification handling

Today, if an MBean emits notifications then it must implement the NotificationBroadcaster or NotificationEmitter interface. This means it must keep track of the set of listeners, as listeners are added and removed. It must also define the list of notification types that it can emit, by implementing getNotificationInfo().

In practice, everybody uses the NotificationBroadcasterSupport class instead of doing all this work themselves. In the simplest case, you just inherit from that class, and pass the list of notification types to the superclass constructor. If you already have a superclass, then you need to have a private NotificationBroadcasterSupport instance and delegate the NotificationBroadcaster methods to it.

New annotations allow you to define the list of notification types more simply, and to emit notifications without having to keep track of listeners.

 


public class Cache
    	extends NotificationBroadcasterSupport
    	implements CacheMBean {
    public Cache() {
    	super(new MBeanNotificationInfo[] {
    	    new MBeanNotificationInfo(
    	    	new String[] {"my.notif.type"},
    	    	Notification.class.getName(),
    	    	"my notification"
    	    )}
    	);
    }

    ...
    void somethingHappened() {
    	Notification n = new Notification(...);
    	super.sendNotification(n);
    }
    ...
}

@MBean
@NotificationInfo(types={"my.notif.type"},
    description=@Description("my notification"))
public class Cache {


    @Resource
    private volatile SendNotification send;








    ...
    void somethingHappened() {
    	Notification n = new Notification(...);
        send.sendNotification(n);
    }
    ...
}

The @NotificationInfo annotation is what allows you to avoid constructing an MBeanNotificationInfo as in the messy "Today" code.

The new SendNotification interface contains just the method sendNotification. When you register this MBean in the MBean Server, it will inject an object into the send field which allows the MBean to send notifications. The MBean no longer has to be concerned with managing listeners, which happens somewhere behind the scenes.

Resource injection of SendNotification is available to all types of MBeans. Defining the notification types with @NotificationInfo is not available to Dynamic MBeans, which are expected to provide a complete MBeanInfo, including the MBeanNotificationInfo[] array.

More detail than you want to know about @NotificationInfo appears below.

You can stop reading now

If your eyes are already glazing over with all this code, you can safely stop here, and you'll have seen the main ideas. The remainder of this entry is about secondary items, and further details about the main ones.

Descriptor contents

In the JMX API included in the Java SE 6 platform, we introduced a way to define your own annotations to specify Descriptor contents. So you might define @Units like this:

@Documented @Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Units {
    @DescriptorKey("units")
    String value();
}

The new API accepts such annotations on classes or methods that also have the @ManagedX annotations. For example:

@MBean
public class Cache {
    ...
    @Units("bytes")
    @ManagedAttribute
    public int getUsed() {...}
    ...
}

We've also added a new @DesriptorFields annotation, since the use of @DescriptorKey is somewhat non-obvious, and overkill for occasional use. So you can achieve the same effect like this:

@MBean
public class Cache {
    ...
    @DescriptorFields("units=bytes")
    @ManagedAttribute
    public int getUsed() {...}
    ...
}

This annotation can be used in Standard MBeans and MXBeans as well as @MBeans and @MXBeans.

Operation impact

The @ManagedOperation annotation has an optional element impact of type Impact. This is a new enum with values {INFO, ACTION, ACTION_INFO, UNKNOWN} corresponding to the integer codes defined by MBeanOperationInfo. So you can do this:

@MBean
public class Cache {
    ...
    @ManagedOperation(impact = Impact.ACTION)
    public int dropOldest(int n) {...}
}

You can also apply @ManagedOperation to a method in a Standard MBean interface or an MXBean interface in order to specify the impact.

MBean constructors

Each public constructor in an @MBean or @MXBean is converted into an MBeanConstructorInfo inside the MBean's MBeanInfo. This is exactly the same as for existing Standard MBeans and MXBeans.

StandardMBean class

The class StandardMBean can be used to customize an @MBean or an @MXBean in the same way as for a Standard MBean or MXBean today. You simply supply null for the mbeanInterface parameter in the constructor.

Details on Resource injection

The @Resource annotation can be used to inject an ObjectName, MBeanServer, or SendNotification. The annotation can be applied to a field or to a void method with a single parameter. For example:

    @Resource  // field injection
    private volatile ObjectName name;

    private MBeanServer mbs;
    @Resource  // method injection
    private synchronized void setMBeanServer(MBeanServer mbs) {
        this.mbs = mbs;
    }

The MBean Server determines what to inject based on the type. The type is either the declared type of the field or parameter, or it is specified explicitly in the @Resource annotation. For example, the following annotations have the same effect:

    @Resource
    private volatile ObjectName name;

    @Resource(type = ObjectName.class)
    private volatile Object name;

I don't think the second form will be used very often, but it might be used to inject the MBeanServer into a field of type MBeanServerConnection, for example.

The ObjectName (etc) will be injected as many times as there are appropriate @Resource annotations, including in parent classes.

@Resource annotations that don't match one of the given types are ignored. (Perhaps they are for some other API.) But even if the type is not recognized, @Resource fields and methods must be instance (not static), and @Resource methods must have exactly one parameter and return void.

I've used volatile in all these examples because the Java Memory Model would not otherwise guarantee that the MBean would actually see the injected values. For method-based injection, synchronized is an alternative, provided the MBean also uses synchronized to access the injected value. Notice that the same considerations apply to the existing MBeanRegistration technique.

Resource injection happens after the MBean's preRegister method (if any) is called, but before the MBean is registered in the MBean Server. If an injection method throws an exception, then postRegister(false) will be called and the exception will be thrown in the same way as for preRegister.

More on descriptions

In addition to the description text, the @Description annotation can specify the values of the descriptionResourceBundleBaseName and descriptionResourceKey fields in the corresponding Descriptor. This is enough to allow for internationalization:

@Description(value="some sort of cache",
             key="cache.mbean.description",
             bundleBaseName="MyResources")

To complete the story here, we need to have something that is able to apply these Descriptor fields to localize the MBeanInfo. We have some ideas on what that something might look like, but they are not yet fully formed.

More on @NotificationInfo

As I threatened, here is more information than you wanted to know about @NotificationInfo.

If an MBean has a @NotificationInfo annotation, then that annotation is translated into an MBeanNotificationInfo in the MBean's MBeanInfo. MBeanNotificationInfo includes a name which is the name of the notification class. It is usually "javax.management.Notification", but it might be a subclass. So @NotificationInfo has an optional notificationClass element which is a Class<? extends Notification>. For example:

@NotificationInfo(types = {AttributeChangeNotification.ATTRIBUTE_CHANGE},
                  notificationClass = AttributeChangeNotification.class)
    

If the MBean can emit more than one class of MBean, then it can use @NotificationInfos:

@NotificationInfos(
    @NotificationInfo(types = {"my.first.notif", "my.second.notif"})
    @NotificationInfo(types = {AttributeChangeNotification.ATTRIBUTE_CHANGE},
                      notificationClass = AttributeChangeNotification.class)
)

The @NotificationInfo is applied to an MBean class, but a @Description on that class applies to the MBean, not its notifications. The existence of @NotificationInfos is another reason why we cannot use @Description straightforwardly.

This is why there is an optional element of type @Description inside @NotificationInfo, so you would write:

@NotificationInfo(types={"my.notif.type"},
      description=@Description(value="my notification", key="my.notif.descr"))

You cannot use @DescriptorFields, for the same reason as @Description, so there's another optional element that allows you to write:

@NotificationInfo(types={"my.notif.type"},
      descriptorFields={"foo=bar"})

Ideas still in progress

We're studying the possibility of providing a way to cause a notification that is sent every time a given operation is completed.

We're looking at ways in which an MBean could say what its ObjectName is. Probably the MBean would only provide a subset of the information needed to construct the name. It's still unclear exactly what this might look like.

What next?

This is still work in progress, as you'll have gathered. I'm very much interested in comments and suggestions, either here or at jmx-spec-comments@sun.com. Thanks!

[Tags: jmx jsr jsr255 annotations]

When can JMX notifications be lost?

The JMX Best Practices guide says notifications can sometimes be lost. Why is that? When might it happen? Read on.

Here's the relevant text from the Best Practices guide:

It is important to be aware of the semantics of notification delivery when defining how notifications are used in a model. Remote clients cannot assume that they will receive all notifications for which they are listening. The JMX Remote API only guarantees a weaker condition:

A client either receives all notifications for which it is listening, or can discover that notifications may have been lost.

This text might seem somewhat alarming. First of all, notice that it only applies to remote clients. A local client (within the same Java VM) will reliably get all notifications it asks for.

Secondly, the text is describing something that will only happen in unusual circumstances. Notifications will only be lost when they arrive so fast that they cannot be delivered to the remote client quickly enough, or if there is a long network outage during which enough notifications arrive to overflow the notification buffer on the server. If you're sure that the rate of notifications is always low then you probably don't need to worry. Long network outages will probably trigger other problems in your client, so you'll need to deal with them more generally than just worrying about lost notifications.

Careful clients

But if you have many notifications, you probably want to follow the advice in the subsequent paragraphs of the Best Practices guide:

Notifications should never be used to deliver information that is not also available in another way. The typical client observes the initial state of the information model, then reacts to changes in the model signalled by notifications. If it sees that notifications may have been lost, it goes back and observes the state of the model again using the same logic as it used initially. The information model must be designed so that this is always possible. Losing a notification must not mean losing information irretrievably.

When a notification signals an event that might require intervention from the client, the client should be able to retrieve the information needed to react. This might be an attribute in an MBean that contains the same information as was included in the notification. If the information is just that a certain event occurred, it is often enough just to have a counter of how many times it occurred. Then a client can detect that the event occurred just by seeing that the counter has changed.

Stateless servers

The design of the existing standard connectors is such that notification loss can happen when there are many notifications coming from the MBeans in the MBean Server. This is true even for clients that are only listening for a small subset of those notifications. In the extreme case, a client that is listening for a very rare notification might not see it, because other MBeans are generating frequent notifications that nobody is listening to. Once again, the client can tell that this has happened (via JMXConnector.addConnectionNotificationListener).

The existing connectors behave like this because they have been designed to have no non-transient state on the server. A consequence is that the server has no non-transient record of which clients are interested in which notifications. Therefore it has to store all notifications in its buffer, in case some client it doesn't remember is interested in them.

The servers were designed to have no non-transient state for better scalability. In retrospect, this was probably a design mistake. In many client/server systems, you have one server, or just a few servers, and a large number of clients. So limiting state in the server is an excellent idea, because it allows the server to handle many more clients. But in management systems, the situation is usually the opposite: you typically have one client (a management program such as JConsole) that may connect to and manage many servers. There are no common use cases where a server might have a large number of JMX clients.

In version 2.0 of the JMX API, being defined by JSR 255, we are adding an Event Service. Among other things, this will fix the problem where a client might lose notifications that it is interested in because there are many other notifications that it is not interested in.

Notification loss is inevitable

Even with the new Event Service, notification loss will still be possible, however. Can't we get rid of it?

To answer this question, consider what happens when notifications are produced faster than they can be handled. This might be because of network delays, or because the client needs to do some work for each notification. Suppose this situation persists. What should the system do?

There are basically three possibilities:

1. Some notifications are eventually dropped. This is what the JMX Remote API does, and it is also what the new Event Service will do.
2. Notification senders are slowed down. This is what usually happens in the local case. An MBean sends a notification to a local listener by invoking the listener's handleNotification method. Unless it has multiple threads, the MBean will wait for that method to complete before doing anything else, including sending any more notifications.
3. Notifications accumulate in an unbounded buffer. This is actually the worst solution. In the real world there is no such thing as an unbounded buffer. And even if you save the notifications in a giant disk, which is effectively unbounded, you still haven't fixed the problem that the client is getting further and further behind the server. When the client finally gets a notification that was sent yesterday, is that still any use?

When we were designing the JMX Remote API, we assumed that most MBeans that send notifications were not expecting sending to be slow. In the local case, sending is just invoking a method, and that method is usually punctual. If we had wanted to apply solution 2, slowing down senders, that could have broken the assumptions of existing MBeans. Coding MBeans so that they can cope with a blocked send would also be considerably more difficult. So, even though this solution (flow control) is arguably better, we were reluctant to impose it.

The future: JMX Event Service

As I mentioned, in version 2.0 of the JMX API we are designing a new Event Service. This will be part of the JDK 7 platform. Though it will not eliminate notification loss, it will significantly reduce the likelihood of such loss. And it will also allow you to plug in your own transport for notifications. In particular you could plug in the Java Message Service to use an existing message bus.

[Tags: jmx jms.]

"Top threads" plugin for JConsole

Peter Doornbosch has created a much improved version of the JTop sample plugin for JConsole. JTop only shows thread name, cumulative CPU time, and thread state. The "top threads" plugin also shows, per thread, current CPU percentage, average CPU percentage, and the evolution of CPU percentage over time. Very nice!

There does seem to be a bug whereby the TextArea that shows the stack trace of the selected thread is editable. Furthermore this TextArea is "always on top" so it can obscure part of the "new connection" dialogue. I found that I could shrink the TextArea down to nothing by grabbing the control at its top, so this problem isn't very severe.

Thanks to Daniel Fuchs for the pointer.

[Tags: jmx jconsole jtop.]

Custom types for MXBeans

MXBeans map between arbitrary Java types and a fixed set of types in javax.management.openmbean called the Open Types. This allows clients to interact with MXBeans, without needing to know the original Java types (which might require putting extra jars in their classpath and so on).

Up until now the mapping rules were fixed. Certain types can not be mapped by these rules, for example self-referential types or types such as Object or Number. In the Java 7 platform, we're planning to allow customization of the rules, as part of JSR 255 which is defining version 2.0 of the JMX API. This is a summary of the proposed changes.

What problem are we solving?

There are three sorts of use cases here:

1. You have some types that you control and that are used in your MXBean interfaces. For example, java.lang.management.MemoryUsage which is referenced by java.lang.management.MemoryMXBean. An update to java.lang.management could update MemoryUsage, for example to add an annotation to it.
2. You have some types that you do not control that are used in MXBean interfaces that you do control. For example, you are an end-user and you define an MXBean interface MyMemoryMXBean that references java.lang.management.MemoryUsage. You can't change MemoryUsage but you can change MyMemoryMXBean.
3. You have some types that you do not control that are used in MXBean interfaces that you do not control either. For example, you want to define an MXBean-ified version of some legacy MBeans defined by somebody else.

The proposal is to add one class, one interface, and two annotations to address these cases. The existing class javax.management.StandardMBean acquires two new constructors and the existing class javax.management.JMX acquires two new overloads of existing methods.

MXBeanMapping and @MXBeanMappingClass

The most important change is the new class javax.management.openmbean.MXBeanMapping:

public abstract class MXBeanMapping {
    protected MXBeanMapping(Type javaType, OpenType<?> openType);
    public final Type getJavaType();
    public final OpenType<?> getOpenType();
    public final Class<?> getOpenClass();
    public abstract Object fromOpenValue(Object openValue) throws InvalidObjectException;
    public abstract Object toOpenValue(Object javaValue) throws OpenDataException;
    public void checkReconstructible() throws InvalidObjectException;
}

Suppose we want to define a mapping for the class MyLinkedList, which looks like this:

public class MyLinkedList {
    public MyLinkedList(String name, MyLinkedList next) {...}
    public String getName() {...}
    public MyLinkedList getNext() {...}
}

This is not a valid type for MXBeans, because it contains a self-referential property "next" defined by the getNext() method. (This example comes from Simone Bordet.) So we would like to specify a mapping for it explicitly. When an MXBean interface contains MyLinkedList, that will be mapped into a String[], which is a valid Open Type.

To define this mapping, we first subclass MXBeanMapping:

public class MyLinkedListMapping extends MXBeanMapping {
    public MyLinkedListMapping(Type type) throws OpenDataException {
        super(MyLinkedList.class, ArrayType.getArrayType(SimpleType.STRING));
        if (type != MyLinkedList.class)
            throw new OpenDataException("Mapping only valid for MyLinkedList");
    }

    @Override
    public Object fromOpenValue(Object openValue) throws InvalidObjectException {
        String[] array = (String[]) openValue;
        MyLinkedList list = null;
        for (int i = array.length - 1; i >= 0; i--)
            list = new MyLinkedList(array[i], list);
        return list;
    }

    @Override
    public Object toOpenValue(Object javaValue) throws OpenDataException {
        ArrayList<String> array = new ArrayList<String>();
        for (MyLinkedList list = (MyLinkedList) javaValue; list != null; list = list.getNext())
            array.add(list.getName());
        return array.toArray(new String[0]);
    }
}

The call to the superclass constructor specifies what the original Java type is (MyLinkedList.class) and what Open Type it is mapped to (ArrayType.getArrayType(SimpleType.STRING)). The fromOpenValue method says how we go from the Java type to the Open Type, and the toOpenValue method says how we go from the Open Type to the Java type.

With this mapping defined, we can annotate MyLinkedList with the new annotation @javax.management.openmbean.MXBeanMappingClass:

@MXBeanMappingClass(MyLinkedListMapping.class)
public class MyLinkedList {...}

Now we can use MyLinkedList in an MXBean interface and it will work. This satisfies use case 1 above.

MXBeanMappingFactory and @MXBeanMappingFactoryClass

If we are unable to annotate individual classes, then we can define a mapping factory that is consulted every time a type needs to be mapped. This is also useful if we would like to apply the same set of rules across a whole set of classes (for example, any class that implements List<E> is mapped in the same way as List<E>).

A mapping factory is a subclass of javax.management.openmbean.MXBeanMappingFactory:

public abstract class MXBeanMappingFactory {
    public static final MXBeanMappingFactory DEFAULT;
    protected MXBeanMappingFactory();
    public abstract MXBeanMapping mappingForType(Type t, MXBeanMappingFactory f)
    throws OpenDataException;
}

For example, suppose we can't change MyLinkedList, so we can't add the @MXBeanMappingClass annotation to it. We can achieve the same effect by defining a mapping factory as follows:

public class MyLinkedListMappingFactory extends MXBeanMappingFactory {
    public MyLinkedListMappingFactory() {}

    @Override
    public MXBeanMapping mappingForType(Type t, MXBeanMappingFactory f)
    throws OpenDataException {
        if (t == MyLinkedList.class)
            return new MyLinkedListMapping(t);
        else
            return MXBeanMappingFactory.DEFAULT.mappingForType(t, f);
    }
}

Now we can add the new annotation javax.management.openmbean.MXBeanMappingFactoryClass to any MXBean interface that references MyLinkedList:

@MXBeanMappingFactoryClass(MyLinkedListMappingFactory.class)
public interface SomethingMXBean {
    public MyLinkedList getSomething();
}

This satisfies use case 2 above.

New StandardMBean constructors and JMX.newMXBeanProxy method

The existing class StandardMBean is used to make instances of Standard MBeans and MXBeans when you need to control some aspects of their behaviour, such as the descriptions in the MBeanInfo. We can extend the set of constructors as follows:

public class StandardMBean implements DynamicMBean {
    ...existing constructors...
    public <T> StandardMBean(T impl, Class<T> intf, MBeanOptions options);
    protected StandardMBean(Class<?> intf, MBeanOptions options);
}

The options parameter allows us to specify a number of potentially interesting things:

• an MXBeanMappingFactory (this is the case we're looking at here)
• whether this is an MXBean or not
• perhaps the contents of the MBeanInfo, as an alternative to subclassing and overriding e.g. getDescription(MBeanAttributeInfo)
• whether the methods of MBeanRegistration should be forwarded to the implementation object if it implements that interface

The question of what the MBeanOptions type is is still open. It could be a Map<String, ?> where each option is represented by a string constant. This is the approach taken by the JMX Remote API, for example. Or, it could be a special-purpose class called MBeanOptions with methods to set each of the options. I plan to write more on this later; for now I'll assume it's the MBeanOptions option.

If you need to create an MXBean that implements the SomethingMXBean interface above and uses the MyLinkedListMappingFactory, but you can't add an annotation to SomethingMXBean, then you can do so using either of the two existing ways to use StandardMBean, subclassing or delegation, but supplying an option:

MBeanOptions options = MBeanOptions.DEFAULT.withMXBeanMappingFactory(
    new MyLinkedListMappingFactory);
// either use delegation:
SomethingMXBean impl = new SomethingImpl();
StandardMBean mbean = new StandardMBean(impl, SomethingMXBean.class, options);
// or use subclassing:
public class MySomething extends StandardMBean implements SomethingMXBean {
    public MySomething() {
        super(SomethingMXBean.class, options);
    }
}

If you have a custom mapping in your MBean server, then you need the same custom mapping in a client if the client is making a proxy. So we add another overloading of JMX.newMXBeanProxy:

public class JMX {
    ...
    public static <T> T newMXBeanProxy(
        MBeanServerConnection mbsc,
        ObjectName objectName,
        Class<T> intf,
        MBeanOptions options);
    ...
}

Using this, you can create a proxy like this:

SomethingMXBean proxy = JMX.newMXBeanProxy(
    mbsc, objectName, SomethingMXBean.class, options);

Here the options object can be the same as before.

This satisfies use case 3 above.

Interoperation when there are custom types

MXBeans map arbitrary Java types to Open Types. The addition of custom mappings doesn't change this - the result still has to be Open Types. So for generic clients like JConsole, nothing changes when custom types are added into the mix.

A client that is aware of the MXBean interfaces in use (like SomethingMXBean) can construct a proxy. To do that, it must have the interface available. If the interface has a @MXBeanMappingFactoryClass annotation, or if it contains a type that has a @MXBeanMappingClass annotation, then the classes referenced by those annotations must be present in the client too. It usually isn't any more difficult to arrange for the mapping classes to be present than to arrange for the original MXBean interface to be present.

If the mapping has been specified on the server using an explicit MXBeanMappingFactory, then the same or an equivalent factory must be used on the client. This is the case where there is the most risk of inconsistency. To help check that client and server are using the same MXBeanMappingFactory, the Descriptor of an MXBean using an MXBeanMappingFactory will contain a field naming the factory class, MyLinkedListMappingFactory in the examples above.

Some details

Here are some details I omitted above for clarity.

The methods toOpenValue and fromOpenValue have inconsistent throws clauses. This reflects a similar inconsistency in the MXBean spec, itself due to historical reasons.

The Java type and Open Type in MXBeanMapping must be supplied to the constructor by the subclass, and cannot be changed thereafter. This could be a limitation but I cannot currently see any cases where you would not be able to supply values to the super-constructor call. MXBeanMapping is a class and not an interface for a number of reasons, notably that it allows us to have final methods (getOpenType, getOpenClass, getJavaType) and methods with default implementations (checkReconstructible). MXBeanMappingFactory is a class not an interface so that we can add methods to it in later versions of the API if necessary.

The method MXBeanMapping.checkReconstructible() is used to determine if it is possible to map back from a value of the OpenType to a value of the original Java type. The default implementation does nothing. A subclass can override this method to throw OpenDataException if this mapping is not possible. See the MXBean specification for a discussion of reconstructible types.

MXBeanMappingClass can be defined like this:

@Documented @Retention(RUNTIME) @Target(TYPE) @Inherited
public @interface MXBeanMappingClass {
    Class<? extends MXBeanMapping> value();
}

The use of Class<? extends MXBeanMapping> means that if you write @MXBeanMappingClass(Something.class) and Something is not a subclass of MXBeanMapping, then you will get a compile error.

The class mentioned in the annotation must have a public constructor with a single parameter of type java.lang.reflect.Type. Unfortunately we can't get the compiler to check that!

The @Inherited annotation implies that if you subclass MyLinkedList then the subclass will inherit the same MXBeanMapping, which might not be what you want. But not inheriting the mapping seems more likely to surprise users.

The signature of MXBeanMappingFactory.mappingForType is this:

public MXBeanMapping mappingForType(Type t, MXBeanMappingFactory f)
    throws OpenDataException

The MXBeanMappingFactory parameter ensures that the same mappings are applied recursively. So if the type to be mapped is MyLinkedList[] in the example above, the mapping will proceed as follows:

MyLinkedListMappingFactory.mappingForType(MyLinkedList[].class, MyLinkedListMappingFactory) →
    MXBeanMappingFactory.DEFAULT.mappingForType(MyLinkedList[].class, MyLinkedListMappingFactory) →
        MyLinkedListMappingFactory.mappingForType(MyLinkedList.class, MyLinkedListMappingFactory)
        ← MyLinkedListMapping
    ← ArrayMapping(MyLinkedListMapping)
← ArrayMapping(MyLinkedListMapping)

where ArrayMapping is the private class that the default mapping factory uses to map arrays.

The key point here is that, even though MyLinkedListMappingFactory forwards any type it doesn't handle to the default mapping factory, the default factory will rerun any contained type through the user-specified mapping factory. So MyLinkedListMappingFactory doesn't have to recognize MyLinkedList[] or List<MyLinkedList> or whatever in addition to plain MyLinkedList. The general rules for SomeType[] or List<SomeType> in the default factory will apply, and then MyLinkedListMappingFactory will be applied to the contained SomeType, and do the right thing if that is MyLinkedList.

Open questions

This is still a draft proposal, and some questions remain.

As I mentioned above, the exact type of the MBeanOptions parameter to various methods is yet to be determined.

Should the @MXBeanMappingFactory option be inherited? For example, if you define

public interface SubSomethingMXBean extends SomethingMXBean {...}

should you inherit the @MXBeanMappingFactory annotation from SomethingMXBean? I think the answer must be yes, but what if SubSomethingMXBean inherits from more than one MXBean interface, and they have different @MXBeanMappingFactory annotations?

Should the @MXBeanMappingFactory option be applicable to packages? A reminder that you can annotate packages by creating a file called package-info.java in the package with contents like this:

@javax.management.openmbean.MXBeanMappingFactory(MyLinkedListMappingFactory.class)
package com.example.mbeans;

If @MXBeanMappingFactory can apply to packages and is also inherited from superinterfaces then we may have some complicated rules for precedence between the two.

Conclusion

In conclusion, we're specifying something quite simple: how to extend the standard MXBean mappings with custom mappings. But the details turn out not to be so simple!

Acknowledgements

This specification has been discussed in the JSR 255 Expert Group. Mandy Chung also had some very helpful comments.

[Tags: jmx, mxbeans.]

Reimplementing the RMI protocol

In my last entry, I mentioned that I had reimplemented the RMI registry portably, before discovering that there was a much simpler solution to the security problem I was addressing. Here's the reimplementation for what it's worth. It allows you to go further than the socket factory hack. And if you ever need to understand gory details of the RMI protocol, this could come in useful. Because it does much less than the full-blown RMI implementation in the JDK, it's much easier to understand.

Here are some of things you can do with a reimplemented RMI registry that you can't do with just socket factories:

• Show different subsets of the registry contents to different clients, for example depending on where they are connecting from or whether they have authenticated with SSL.
• Only allow a bind or unbind operation to succeed if it adds a hidden password to the name. So if you want to bind "foo", the string you give would actually be "foosecretpassword". Anything else will fail.
• Trigger additional actions when an object is bound in the registry, for example persisting the object or replicating it in another registry.

By the way, most of these advantages would evaporate if the registry API were augmented to allow you to supply your own Registry object to LocateRegistry.createRegistry.

My reimplementation separates out the registry functionality and the RMI reimplementation. The registry functionality is in the class RegistryImpl, which is almost completely uninteresting, except possibly for the use of ConcurrentHashMap to avoid having to synchronize explicitly across modifications.

The class RegistryServer is where the real action happens. Its public API consists of just a constructor that takes a Registry parameter (typically an instance of RegistryImpl or a subclass) and a port number. It could reasonably have a close method as well, which is left as the proverbial exercise for the reader.

If you're interested in the details of RMI's wire protocol, you could do worse than to study this class in addition to the official specification, especially since the latter has some known errors.

At a high level, an RMI request looks like (object-id, opnum, hash). The object-id is the ObjId of the remote object being invoked. For the RMI registry, this must be the distinguished value ObjID(REGISTRY_ID) because that is the id that LocateRegistry.getRegistry will send. Since we are not exporting any other RMI objects on our port, we don't actually need to check the id, but we do anyway.

The RMI protocol exists in two variants, the 1.1 variant that was used in JDK 1.1, and the 1.2 variant that was added in JDK 1.2. In the 1.1 variant, the opnum is an index within the methods of the given Remote interface, and the hash is a checksum or digest to make sure that both ends have the same understanding of what these methods are and what order they appear in. In the 1.2 variant, the opnum is -1 and the hash is a digest of the method signature.

For compatibility reasons, LocateRegistry.getRegistry only uses the 1.1 variant, but I have coded the server to recognize both, even though I can't think of any reason you would need the 1.2 variant in this case.

In addition to these variants, there are three communication styles. The "single op protocol" is used by RMI-over-HTTP and sends a single operation over a connection before closing it. The "stream protocol" can send any number of operations over a connection, but only one at a time. The "multiplex protocol" can send any number of operations, and can send more than one operation in parallel.

We're only interested in the stream protocol here because that's what LocateRegistry.getRegistry uses. In fact, in the JDK no RMI client will ever use the multiplex protocol, which is a pity because it would allow a more efficient use of TCP connections. The JDK can handle the multiplex protocol if it receives a client request that uses it, but will never originate such a request. That's my understanding from reading the code, at least.

One thing that is not immediately clear from the protocol specification is exactly when RMI switches over to using object serialization. You can see this in the reimplementation by looking at when we switch over from using DataInputStream and DataOutputStream to using ObjectInputStream and ObjectOutputStream.

The RMI protocol is not self-describing, in that you have to know the signature of the method being invoked in order to read its parameters and write its return value correctly. I think this is a flaw, by the way.

Another thing that is not very obvious from reading the specification is that you must include "class annotations" in the serialized data you send back, even if they are null. This is why we subclass ObjectOutputStream and override annotateClass.

I've done some basic testing of this code, but it is obviously not production-quality. Some performance tuning could be done (though performance of the registry is not usually critical). I am not sure that protocol errors will produce the same exceptions as "real" RMI, though I am not sure I care either.

The source files are RegistryImpl.java (the boring implementation of the Registry interface) and RegistryServer.java (the exciting implementation of a subset of the RMI protocol).

Securing the RMI registry

If you've had occasion to use the RMI registry seriously, you may have encountered some of its shortcomings. Chief of these is that anybody on the local machine can modify the registry. There are only a few things you can do about that, of which the craziest is to reimplement enough of RMI to code your own compatible version of the registry. I did that, and write about it in a later entry, but then I discovered a simpler alternative, due to Peter Jones.

We encountered this problem when defining the JMX Remote API. The standard connector defined by this API is the RMI connector, and the obvious way for it to work would have been to interpret a URL like service:jmx:rmi://somehost:8888/bar as meaning "an RMIServer object called bar in the RMI registry at port 8888 on host somehost. We wouldn't have needed to support any other address types. The same address would work on the client and on the server.

Well, if you have a look at the documentation, you'll see that things are quite a bit more complicated than that. Addresses come in two forms, of which the first is this rather ugly one:

service:jmx:rmi:///jndi/rmi://somehost:8888/bar

That is as nothing to the ugliness of the second form, though:

service:jmx:rmi://somehost/stub/rO0ABXNyAC5qYXZheC5tYW5hZ2VtZW50LnJlbW90ZS5ybWk
uUk1JU2VydmVySW1wbF9TdHViAAAAAAAAAAICAAB4cgAaamF2YS5ybWkuc2VydmVyLlJlbW90ZVN0dW
Lp/tzJi+FlGgIAAHhyABxqYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN002G0kQxhMx4DAAB4cHc5A
AtVbmljYXN0UmVmMgAADjEyOS4xNTcuMjAzLjM5AACauPDsdTz1J8m+BsXsHAAAAQ/Ow/uqgAEAeA==

Blechh.

As Spec Lead for JSR 160, I bear considerable responsibility for this ugliness, and you can believe that I didn't accept it gladly.

Why we perpetrated these address forms

There were two problems that led us to exclude the simple approach with the RMI registry. The first was that a longstanding bug in the JDK meant that you couldn't have more than one RMI registry in the same VM. So there was great potential for interference between different modules each needing to create a registry. This bug was fixed in JDK 1.5 but JSR 160 was supported all the way back to 1.3.

The second problem was the one I mentioned above, namely that the only access check on changes to the RMI registry is that the client making the change must be connecting from the same machine that the registry is running on. This means that if you store your JMX Connector Server in an RMI registry on a multiuser machine, some other unscrupulous user can come along and replace it with their own RMI object. Users would then connect to the rogue object without realising it, and send their password or other credentials to authenticate.

Securing the RMI registry

We've addressed this problem in a couple of ways. First of all, you can protect your registry using SSL client authentication. Luis-Miguel Alventosa has explained this in detail in his blog. The big advantage of using SSL is that clients can be sure they are connecting to the server they are expecting, and not a server being run by Dr Evil on the same port. But that's server authentication. Using SSL client authentication seems redundant given that the client can authenticate to the JMX Connector Server with a password or other credentials. You only really need the SSL part to protect the registry from rogue modifications. And setting up the necessary SSL client certificates is a non-trivial amount of work.

The second possibility is to access implementation-specific sun.* classes to define a custom registry. We did this in the JDK code that handles the JMX command-line properties, -Dcom.sun.management.jmxremote etc. When you specify those properties, we create a special read-only RMI registry with a single entry "jmxrmi". See sun.management.jmxremote.SingleEntryRegistry in the JDK sources, which does its work by subclassing sun.rmi.registry.RegistryImpl.

We can get away with this because this code is part of the same JDK as the implementation-dependent classes it is accessing. I strongly discourage anyone from doing this sort of thing outside the JDK. The sun.* classes are subject to change without notice, and indeed as of JDK 6 the compiler will produce warnings for code that references them.

A portable (if hacky) solution

A viable alternative is described in the Work Around section of RFE 5029435.

The access check to see if the caller is on the same machine can be faked so that it always fails, even if the caller is on the same machine. The idea is that the caller's address comes from Socket.getInetAddress(). By imaginative use of RMI socket factories and Socket subclasses, we can arrange for this method never to return a local address.

The access check is only made when the registry is accessed via RMI. It is not made if the Registry object returned by LocateRegistry.createRegistry is accessed directly from within the VM where it was created. So you can create the registry and set up its contents within the same VM, but others, even on the same machine, cannot modify it. Here's what it looks like in code:

    Registry r = LocateRegistry.createRegistry(
    	    8888, null, new NoLocalAddressServerSocketFactory());
    Foo fooObject = new Foo();
    Remote fooStub = UnicastRemoteObject.exportObject(fooObject, 0);
    r.bind("foo", fooStub);
    Bar barObject = new Bar();
    Remote barStub = UnicastRemoteObject.exportObject(barObject, 0);
    r.bind("bar", barObject);

The bind operations here will succeed because they are accessing the registry object locally, not via RMI. But any attempt to modify it via RMI, even from the same machine, will fail, if we define NoLocalAddressServerSocketFactory as follows. (Thanks to Tom Hawtin for the idea behind this version, simpler than the one I originally posted.)

public class NoLocalAddressServerSocketFactory
        implements RMIServerSocketFactory {

    public ServerSocket createServerSocket(int port) throws IOException {
        return new NoLocalAddressServerSocket(port);
    }
    
    private static class NoLocalAddressServerSocket extends ServerSocket {
        NoLocalAddressServerSocket(int port) throws IOException {
            super(port);
        }
        
        @Override
        public Socket accept() throws IOException {
            Socket s = new NoLocalAddressSocket();
            super.implAccept(s);
            return s;
        }
    }
    
    private static class NoLocalAddressSocket extends Socket {
        @Override
        public InetAddress getInetAddress() {
            return null;
        }
    }
}

It's a bit hacky, but it does the trick!

You're not limited to rejecting all modifications. For example, suppose you wanted to allow all modifications, whether or not the caller is remote, and security be damned. Then you could simply return InetAddress.getLocalHost() instead of null. Or, if you've exported the registry through SSL (see below), then you might allow modifications only from clients that have been authenticated through SSL, but you would allow lookups from all clients. So you would return InetAddress.getLocalHost() if the client is authenticated and null otherwise.

If you are already using a socket factory

Things are a bit more complicated if you are already using an RMIServerSocketFactory, such as SslRMIServerSocketFactory. In that case, you'll need to combine subclassing with delegation, as outlined here:

public class NoLocalAddressServerSocketFactory2
        implements RMIServerSocketFactory {

    private final RMIServerSocketFactory factory;
    
    public NoLocalAddressServerSocketFactory2(RMIServerSocketFactory f) {
        if (f == null) {
            f = RMISocketFactory.getSocketFactory();
            if (f == null)
                f = RMISocketFactory.getDefaultSocketFactory();
        }
        this.factory = f;
    }

    public ServerSocket createServerSocket(int port) throws IOException {
        ServerSocket ss = factory.createServerSocket(port);
        return new NoLocalAddressServerSocket(ss);
    }
    
    private static class NoLocalAddressServerSocket extends ServerSocket {
        private final ServerSocket serverSocket;
        
        NoLocalAddressServerSocket(ServerSocket ss) throws IOException {
            this.serverSocket = ss;
        }
        
        @Override
        public Socket accept() throws IOException {
            return new NoLocalAddressSocket(serverSocket.accept());
        }

	@Override
        public void close() throws IOException {
            serverSocket.close();
        }

        ...override 13 other ServerSocket methods in the same way as close()...
    }
    
    private static class NoLocalAddressSocket extends Socket {
        private final Socket socket;
        
        NoLocalAddressSocket(Socket s) {
            this.socket = s;
        }
        
        @Override
        public InetAddress getInetAddress() {
            return null;
        }

	@Override
        public void close() throws IOException {
            socket.close();
        }
        
	...override 38 more Socket methods in the same way as close()...
    }
}

The main drawback that I can see is that if a future version of java.net.Socket adds more methods, they won't be overridden by NoLocalAddressSocket, and that might cause problems if the RMI internals use the new methods. (If you're really worried about that, you might want to use cglib to produce the equivalent of NoLocalAddressSocket dynamically.)

Credit where it's due

The public bug database is anonymized but I can reveal that this idea, like so many RMI ideas, is the work of the discreet genius Peter Jones.

Multihomed Computers and RMI

A multihomed computer is one that has more than one network interface. Problems arise when you export an RMI object from such a computer. Here's why, and some ways you can work around the problem.

A typical example of a multihomed computer is a laptop with both Ethernet and WiFi interfaces. If you look closely at such a computer, you'll see that each interface has its own IP address. Use ifconfig -a on Unix or ipconfig/a on MS Windows to see these addresses. The picture here shows my laptop, which is connected through an Ethernet cable to the company intranet and through WiFi to a wireless access point.

There's an important point here, which is that an IP address is the address of a network adaptor, not the address of a computer.

Now let's look at some details of how RMI works. To interact with a remote object, you need to get a stub for it. Typically you connect to an RMI registry to get a remote object to start from. Methods on that object may return references to other remote objects, which are also stubs.

For example, one way to use the JMX Remote API is to pick up a stub for a remote RMIServer object. Then you call newClient() on that object, which returns you a stub for a new RMIConnection object. The picture here shows the first step, where you pick up the stub from the RMI registry.

What do these stubs look like? The stub object implements the appropriate Remote interface, for example RMIServer or RMIConnection or Spume. The implementation of these methods in the stub forwards the method to the remote object. So when you call stub.newClient(), for example, the call is forwarded over the network to the real RMIServer object on the remote machine. This is just basic RMI and should be no surprise.

Stubs are serializable. This is how you can get them from a remote RMI registry, of course. But it also means that you can send the stub to someone else. A stub doesn't care where you get it from - it just connects to its remote object and does its stuff.

So what's inside a stub?

• The host where the remote object lives. This is a string, and we will have much more to say about it later.
• The port on that host where RMI is listening for requests for that object and possibly other objects.
• The object id that allows RMI to know which object you are talking to.
• The socket factory, an instance of RMIClientSocketFactory that controls how the connection to the given host and port is made.

The first item we're interested in here is the host. This is a string, and by default it is the result of InetAddress.getLocalHost(). getHostAddress(). In other words it is a numeric IP address like 10.0.10.58. We can immediately see why this default behaviour is going to be a problem for my multihomed laptop. This address is the WiFi address. If a client from within the other network (the company intranet) wants to connect using this stub, it can't.

The simple solution that RMI provides for this situation is a system property, "java.rmi.server.hostname". If I only ever want connections from clients within the intranet, I just set this to the intranet address of my laptop, 129.157.209.250, and I'm done.

But what if I want connections both from the intranet and from the wireless network? That's where things get a bit more complicated.

Domain Name Service

First, if my laptop has a DNS name such as eamonnslaptop.france.sun.com then I might be able to arrange for that name to resolve to more than one IP address. (Look at the DNS lookup for google.com for example.) So I could set java.rmi.server.hostname to the DNS name and wait for RFE 5052134 to be implemented.

Apart from the fact that I might not be able to wait, this solution is unsatisfactory because it's highly unlikely that such a DNS name exists in my case. The two IP addresses come from two different DHCP servers, one for the local intranet and one for the local wireless network. They will probably be different if I come back tomorrow and connect up my laptop again. There are plenty of other scenarios, for example involving "floating IP addresses", that will also fail.

Client socket factory solutions

All is not lost, though. Remember that, in addition to a host and port, the stub also contains a client socket factory which controls exactly how a connection is made to the host and port. By supplying our own socket factory, we can try to do the right thing for all clients, regardless of where they are connecting from.

Before we see what that might look like, a note about what it implies. The client socket factory is part of the RMI stub, which means, paradoxically, that it is defined by the server. So if I want my clients to be able to do some magic to choose the right address to connect to, I'll need to export my remote object appropriately. Clients will not be able to choose to use the magic factory on their own, or indeed not to use it if I have defined one. (Well, using reflection or serialization games, they might, but it will almost certainly not be portable.)

Another implication is that my clients will need to have the client socket factory class in their classpath. (Or they could set up code downloading if they are very brave.)

Given that, what might the client socket factory look like?

ThreadLocal client socket factory

One idea (due to Laurent Farcy) is to have a ThreadLocal variable that the client sets explicitly around code that might use a stub. The outline might be something like this...

            String oldHostName = ThreadLocalRMIClientSocketFactory.getHostName();
            try {
                ThreadLocalRMIClientSocketFactory.setHostName("129.157.209.250");
                ...operations using the stub...
            } finally {
                ThreadLocalRMIClientSocketFactory.setHostName(oldHostName);
            }
        

...with a socket factory that looks something like this (untested code)...

            public class ThreadLocalRMIClientSocketFactory
                    implements RMIClientSocketFactory {
                private static final ThreadLocal<String> hostName =
                    new ThreadLocal<String>();
                
                public Socket createSocket(String host, int port)
                        throws IOException {
                    String hostOverride = getHostName();
                    if (hostOverride != null)
                        host = hostOverride;
                    return new Socket(host, port);
                }

                public static String getHostName() {
                    return hostName.get();
                }

                public static void setHostName(String name) {
                    hostName.set(name);
                }
            }
        

Basically, the socket factory ignores the address contained in the stub and forces the address to be the one you set with the ThreadLocal instead. This is workable if (a) you know the address that your stub is supposed to connect to, and (b) you can be sure that the host name is set appropriately around every use of the stub. (RMI can close idle connections at any time and will create a new one the next time you use the stub.) These conditions can be satisfied for certain uses of the JMX Remote API and you could encapsulate them in a custom JMXConnectorProvider. For example, you could arrange for connections to the JMXServiceURL service:jmx:mrmi://129.157.209.250:8888/jmxrmi to pick up an RMIServer from the RMI registry at that address and to set the ThreadLocal to 129.157.209.250 around the call to RMIServer.newClient() and around all calls to the resultant RMIConnection. It's heavy going, but you can do it.

Choice of IP addresses

Another possibility is to set the java.rmi.server.hostname property to a list of all local IP addresses. Then you can define a client socket factory that picks the right IP address for the client.

The tricky part is in that last sentence. How do we know which IP address is appropriate? If I have the list [129.157.209.250, 10.0.10.58], how do I know which address is the one for me?

One answer is that I could simply try to connect to all of them and pick whichever one works. I can make all the connection attempts in parallel using a NIO Selector. Or, if I'm on at least version 5.0 of the Java platform, I could use isReachable() on each address, perhaps in parallel in different threads. (The remarks in the isReachable() documentation might discourage me from doing that, however.)

Either way, though, I have to face up to an uncomfortable truth, which is that not all IP addresses are unique. The 10.0.10.58 address of my WiFi interface is an example. The 10.* IP addresses are specifically reserved for private networks. A client on the same wireless network as my laptop can contact it using that address. A client on the company intranet can't, but it's possible that the same address is being used for something else on that network. Such a client might end up connecting to some totally different machine thinking it was my laptop.

For any given network configuration, you can probably find appropriate logic to pick the right IP address out of the list. But you probably can't write logic that will work everywhere.

With that caveat, let's look at the details. First, we need to set the java.rmi.server.hostname property appropriately. In my example, it will be set to "129.157.209.250!10.0.10.58". Notice that if there is only one network interface, the property will be set to the same value that RMI would have used anyway. But if there is more than one, it will be set to a string that only our magic client socket factory can understand. Because system properties are global to the Java Virtual Machine, this means that all RMI objects in the JVM had better be exported with the socket factory. Given that default RMI doesn't work splendidly with multihomed machines this is not necessarily a big drawback. You could also try to set the property just at the point where you export your objects, though it will be hard to coordinate with other possible RMI exporters in the JVM.

So here's the code to set the property.

        System.setProperty("java.rmi.server.hostname",
            addressString(localAddresses()));
        ...
            
    private static Set<InetAddress> localAddresses() throws SocketException {
        Set<InetAddress> localAddrs = new HashSet<InetAddress>();
        Enumeration<NetworkInterface> ifaces =
                NetworkInterface.getNetworkInterfaces();
        while (ifaces.hasMoreElements()) {
            NetworkInterface iface = ifaces.nextElement();
            Enumeration<InetAddress> addrs = iface.getInetAddresses();
            while (addrs.hasMoreElements())
                localAddrs.add(addrs.nextElement());
        }
        return localAddrs;
    }

    private static String addressString(Collection<InetAddress> addrs) {
        String s = "";
        for (InetAddress addr : addrs) {
            if (addr.isLoopbackAddress())
                continue;
            if (s.length() > 0)
                s += "!";
            s += addr.getHostAddress();
        }
        return s;
    }
        

(This code, and the remainder of the code, should work unchanged on versions 5.0 or 6 of the Java platform, and should work on 1.4 if you remove the generics.)

Now here's the magic client socket factory. As I described, it uses NIO to connect to all the addresses in parallel, and picks whichever address works first. Again, be aware that this may not work if there are network-private IP addresses in the picture. You may want to modify the logic to work appropriately for your network environment.

import java.io.IOException;
import java.io.Serializable;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
import java.nio.channels.Channel;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.rmi.server.RMIClientSocketFactory;
import java.rmi.server.RMISocketFactory;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

public class MultihomeRMIClientSocketFactory
        implements RMIClientSocketFactory, Serializable {
    private static final long serialVersionUID = 7033753601964541325L;
    
    private final RMIClientSocketFactory factory;
    
    public MultihomeRMIClientSocketFactory(RMIClientSocketFactory wrapped) {
        this.factory = wrapped;
    }
    
    public Socket createSocket(String hostString, int port) throws IOException {
        final String[] hosts = hostString.split("!");
        final int nhosts = hosts.length;
        if (hosts.length < 2)
            return factory().createSocket(hostString, port);

        List<IOException> exceptions = new ArrayList<IOException>();
        Selector selector = Selector.open();
        for (String host : hosts) {
            SocketChannel channel = SocketChannel.open();
            channel.configureBlocking(false);
            channel.register(selector, SelectionKey.OP_CONNECT);
            SocketAddress addr = new InetSocketAddress(host, port);
            channel.connect(addr);
        }
        SocketChannel connectedChannel = null;
        
        connect:
        while (true) {
            if (selector.keys().isEmpty()) {
                throw new IOException("Connection failed for " + hostString +
                        ": " + exceptions);
            }
            selector.select();  // you can add a timeout parameter in millseconds
            Set<SelectionKey> keys = selector.selectedKeys();
            if (keys.isEmpty()) {
                throw new IOException("Selection keys unexpectedly empty for " +
                        hostString + "[exceptions: " + exceptions + "]");
            }
            for (SelectionKey key : keys) {
                SocketChannel channel = (SocketChannel) key.channel();
                key.cancel();
                try {
                    channel.configureBlocking(true);
                    channel.finishConnect();
                    connectedChannel = channel;
                    break connect;
                } catch (IOException e) {
                    exceptions.add(e);
                }
            }
        }
        
        assert connectedChannel != null;
        
        // Close the channels that didn't connect
        for (SelectionKey key : selector.keys()) {
            Channel channel = key.channel();
            if (channel != connectedChannel)
                channel.close();
        }
        
        final Socket socket = connectedChannel.socket();
        if (factory == null && RMISocketFactory.getSocketFactory() == null)
            return socket;
        
        // We've determined that we can connect to this host but we didn't use
        // the right factory so we have to reconnect with the factory.
        String host = socket.getInetAddress().getHostAddress();
        socket.close();
        return factory().createSocket(host, port);
    }
    
    private RMIClientSocketFactory factory() {
        if (factory != null)
            return factory;
        RMIClientSocketFactory f = RMISocketFactory.getSocketFactory();
        if (f != null)
            return f;
        return RMISocketFactory.getDefaultSocketFactory();
    }

    // Thanks to "km" for the reminder that I need these:
    public boolean equals(Object x) {
        if (x.getClass() != this.getClass())
            return false;
        MultihomeRMIClientSocketFactory f = (MultihomeRMIClientSocketFactory) x;
        return ((factory == null) ?
                (f.factory == null) :
                (factory.equals(f.factory)));
    }

    public int hashCode() {
        int h = getClass().hashCode();
        if (factory != null)
            h += factory.hashCode();
        return h;
    }
}
    

The MultihomeRMIClientSocketFactory constructor takes an RMIClientSocketFactory parameter which can be null or another factory to be used once the address to connect to has been determined. For example, it could be an SslRMIClientSocketFactory.

It would be nice if RMI came with something like this by default, but I'm not sure given the design of RMI stubs that there is a good general solution. At a minimum, it would be good if RMI allowed you to specify the java.rmi.server.hostname locally for a given export, and if it allowed you to inspect and change the socket factory inside a stub.

JSR-262 and WS-Management

JSR-262 is defining "A Web-Services Connector for JMX Agents", and I'm the Spec Lead. The Expert Group recently decided to replace the custom protocol that was defined in the Early Draft document with a protocol that builds on the WS-Management standard. Here's the background to that decision.

WS-Management and WSDM

Ideally a Web Services Connector for the JMX world would build on some existing and standard Web Services protocol. The problem we faced early on in JSR-262 is that there were two such protocols, WS-Management and WSDM. Without getting into the details, the protocols were incompatible and support for them was about evenly split between the major industry players. So rather than choosing one of them arbitrarily, which would have been politically very difficult, we defined a custom protocol.

A custom protocol

The custom protocol is defined in the Early Draft document. It is conceptually very simple, and very specific to JMX technology. Basically we defined two XML document schemas, one representing an MBeanServer request, such as getAttribute or queryNames, and the other representing the corresponding reply. Then the protocol basically consisted of a trivial WSDL that defined an operation taking an MBeanServerRequest parameter and returning an MBeanServerResponse.

(This is simple only conceptually, because it becomes massively complicated once you start getting into the details of how Java objects are serialized into XML inside the requests and responses.)

Convergence between WS-Management and WSDM

The picture changed with the publication of the white paper Toward Converging Web Service Standards for Resources, Events, and Management, co-signed by key players in each of the two camps. With this paper, an approach based on an existing standard rather than a custom protocol became possible. Furthermore, we had begun getting feedback that defining a new and arguably rival standard was leading to confusion.

From a practical perspective, it is not expected that the converged standard will be finalized before about 2008. JSR-262 is targeted for Java SE 7, which is also scheduled for 2008, but given the necessary lead time to integrate into Java SE, we cannot reasonably wait for the converged protocol to be ready. However, studying the convergence proposal, we can see that its bottom layers are the existing standards WS-Transfer, WS-Enumeration, and WS-Eventing. WS-Management is basically just a definition of how to use these three specifications for management.

So if we include WS-Management in version 1.0 of JSR-262, we can plan to support the converged protocol in a later version, and that should be an incremental and compatible change. It will add the new specifications discussed in the convergence document: WS-ResourceTransfer, WS-Transfer Addendum, WS-MetadataExchange 1.1, and WS-EventingNotification.

A further advantage of including WS-Management is that we have a much better story for integration into heterogeneous network environments. Previous JMX standards defined a sort of "JMX island". They were very useful provided everything was based around JMX technology. But if parts of your network didn't use JMX technology, then they were cast out of the island.

Client use cases for JSR-262

Rather than launching straight into a deeply technical exploration of what this change implies for the JSR-262 protocol, I think it is useful to look at what we see as being the client use cases driving our choices.

Client use case: JMX Connector Client

This is a Java client that is using the JMX Remote API to access the server. Ideally, this client should look exactly like a client using another protocol (such as the RMI connector defined by JSR-160). The only difference should be the URL it specifies. So in particular there should not need to be any configuration of XML schemas or the like on the client or server for this use case.

The switch to WS-Management will be completely invisible to this sort of client. It still establishes its connection with JMXConnectorFactory.connect, using code something like this:

JMXServiceURL url = new JMXServiceURL("service:jmx:ws://blah:8888/blah");
JMXConnector cc = JMXConnectorFactory.connect(url);
MBeanServerConnection mbsc = cc.getMBeanServerConnection();
String osName = mbsc.getAttribute(
    new ObjectName("java.lang:type=OperatingSystem"), "Name");

This code is exactly the same as if we were using the RMI connector that is already part of the Java SE platform, except that we have replaced :rmi: in the JMXServiceURL by :ws:. We could deploy both an RMI connector and a WS connector for the same MBean Server, and then clients could connect with either.

The details of exactly what gets sent over the wire do change compared to the Early Draft, of course, since the protocol is now based on WS-Management. But both the client and the server will be updated to reflect the change.

Client use case: JMX-aware client

This is a client, probably not running on the Java platform, that is interacting with JMX resources. For example, suppose you have an app server that exposes the JSR-77 MBeans, and you deploy a JSR-262 connector server on top of it. Then you could write a perl or ruby or C# client that is able to display the various statistics exposed by those MBeans.

It might be more accurate to call this client "JMX-model-aware". It probably doesn't know what the MBeanServer interface looks like, but it does know the details of one or more JMX models. In the example, it knows what the names of the MBeans in the JSR-77 model are, even if it calls them "Resource Instances" rather than MBeans. It knows what attributes it can expect to find in those MBeans, even if it calls them "properties".

Client use case: Specific WS-Management client

This is a client that is expecting a particular schema or set of schemas for the WS-Management resources it is interacting with. For example, a client that knows about a CIM model and that accesses that model using the standard WS-CIM mapping. In this use case, JMX is just an implementation technology and the client interaction is exactly the same as if it were interacting with a C# agent implementing the same model.

Not a client use case: using the JMX API to access WS-Management

We do not intend to address the use case of a JMX Connector Client that is using the JMX Remote API to interact with arbitrary WS-Management instrumentation. The connector client we define expects the WS-Management agent it interacts with to implement the exact schemas we define. If you need to interact with WS-Management, then you should use a WS-Management client API, such as the one defined by the Wiseman project. The JMX Remote API is not a good fit for this case.

XML schemas for WS-Management content

To understand the WS-Management client use case, you need to know that WS-Management does not specify the XML schema to be used for "Management Resources". Suppose you want to GET the properties of a certain physicalDisk resource, equivalent to getting the attributes of the physicalDisk MBean in JMX parlance. WS-Management says what the SOAP envelope for the request and reply must look like, but it doesn't specify what XML schema will be used for the contents -- the actual attributes. In this example from the WS-Management spec, the SOAP body of the reply might look like this:

<s:Body>
  <PhysicalDisk xmlns="http://schemas.acme.com/2005/02/samples/physDisk">
    <Manufacturer> Acme, Inc. </Manufacturer>
    <Model> 123-SCSI 42 GB Drive </Model>
    <LUN> 2 </LUN>
    <Cylinders> 16384 </Cylinders>
    <Heads> 80 </Heads>
    <Sectors> 63 </Sectors>
    <OctetsPerSector> 512 </OctetsPerSector>
    <BootPartition> 0 </BootPartition>
  </PhysicalDisk>
</s:Body>

The client had better be prepared to deal with the http://schemas.acme.com/2005/02/samples/physDisk namespace, <PhysicalDisk>, <Manufacturer>, and so on. Conversely, the server has to know what schema the client is expecting.

So to be useful in a general WS-Management context, JSR-262 will have to provide an adaptor in the JMX sense of the word. An adaptor takes a standard protocol such as SNMP or WS-Management and converts it into operations on the JMX MBean Server.

In the case of a WS-Management adaptor, it will need to take SOAP headers like these (excerpted)...

<wsman:ResourceURI>http://samples.org/2005/02/physicalDisk</wsman:ResourceURI>
<wsa:Action>
  http://schemas.xmlsoap.org/ws/2004/09/transfer/Get
</wsa:Action>
<wsman:SelectorSet>
  <wsman:Selector Name="LUN"> 2 </wsman:Selector>
</wsman:SelectorSet>

...and derive a JMX ObjectName like org.samples.2005.02:type=physicalDisk,LUN=2. And it will need to convert the AttributeList that it gets back from MBeanServer.getAttributes into an XML fragment like the one above.

We will define some JMX-specific rules that say how an arbitrary ObjectName is represented in the SOAP headers of a WS-Management request, and how the contents of an arbitrary MBean are represented in the SOAP body of a request or a reply. This will allow the connector use case to work without any explicit customization of the client or the server.

We will also define a plug-in mechanism that will allow you to customize these rules for a particular environment, in the WS-Management use case. So you could could tell the adaptor how to convert <wsman:ResourceURI>http://samples.org/2005/02/physicalDisk</wsman:ResourceURI> etc into org.samples.2005.02:type=physicalDisk,LUN=2, and how to convert between an AttributeList and <PhysicalDisk xmlns="http://schemas.acme.com/2005/02/samples/physDisk"> etc.

A particular application of this plug-in mechanism would apply to the WS-CIM mapping, as illustrated above. However, to be complete we would also need to specify how CIM classes are translated into JMX MBeans, which is beyond the scope of the JSR. It's definitely something we'll investigate in order to validate the proposed plug-in mechanism.

Another way to look at this is that the first two use cases correspond to a situation where you are starting from an existing JMX model such as the JSR-77 MBeans. The third use case corresponds to a situation where you are starting from an existing WS-Management model such as one you might get from applying WS-CIM to a CIM model.

So what happens now?

We need to rework the specification and the Reference Implementation to take these changes into account. Luckily the work of implementing the WS-Management protocol in the Java language has already been done in the Wiseman project on java.net, which we intend to build on.

Once the changes have been made to the specification, we plan to release another Early Draft. That's probably going to take a few months, though. In the meantime, comments on this change are welcome either in public on this blog or in private to jsr-262-comments@jcp.org. (And now I'll start getting spam at that address!)

Adding Descriptors to MBeans in Tiger

Mustang (Java SE 6) includes the ability to give additional information about MBeans to management clients via Descriptors, as I described previously. But what if you are not yet able to migrate to the Mustang platform? As I hinted in that previous entry, all is not lost. You can still use Descriptors, though it's more work.

We often hear from developers that they'd dearly love to move to the latest and greatest Java SE platform, but they can't. Their app server isn't yet supported on that platform; or their giant mission-critical financial application would require an enormous test campaign on the new platform, that the QA department doesn't have the resources to do; or their pointy-haired boss fears the unknown or doesn't see what the fuss is all about. So if there's some way to make the shiny new features available on older platforms, these developers are all for it!

Descriptors

Descriptors allow you to give additional information about MBeans to management clients. For example, a Descriptor on an MBean attribute might say what units it is measured in, or what its minimum and maximum possible values are. As of Mustang, Descriptors are a basic part of the JMX API and are available in all types of MBeans. In particular, the Mustang version of the jconsole tool will display an MBean's descriptors, as the following screenshot illustrates:

The descriptor here includes some standard descriptor fields, namely immutableInfo, interfaceClassName, and mxbean; as well as a custom field author.

You can perfectly well use Mustang jconsole to connect to a JMX agent running on Tiger (J2SE 5.0). (You can even connect to an earlier J2SE version such as 1.4, though in that case you will need to have the JMX and JMX Remote API classes in the classpath of your application.) So even if you can't migrate your application to Mustang, it's perfectly feasible to install a second JDK, the Mustang one, and just use its jconsole to connect to your application running on Tiger. Likewise you could write your own management client running on Mustang, even though your application is still running on Tiger.

So if you could somehow arrange for your MBean to contain descriptors in Tiger, then you could show them with your Mustang jconsole or access them in your Mustang management client. And this is in fact possible.

Descriptors on Tiger

The trick is that descriptors have always existed in the JMX API, but only in conjunction with Model MBeans. Model MBeans are an advanced (not to say obscure) part of the API, so you probably haven't paid much attention to them. But the key point here is that the Model MBean package defines a set of subclasses of the standard metadata classes such as MBeanInfo and MBeanAttributeInfo. For example, ModelMBeanAttributeInfo is just like MBeanAttributeInfo except that it adds a descriptor. Prior to Mustang, MBeanAttributeInfo did not have a descriptor, but ModelMBeanAttributeInfo did. Since ModelMBeanAttributeInfo is a subclass of MBeanAttributeInfo, you can use a ModelMBeanAttributeInfo wherever an MBeanAttributeInfo is expected. So in particular if you have a plain MBean, for example a Standard MBean, you can change its MBeanInfo so that it contains an array of ModelMBeanAttributeInfo rather than just MBeanAttributeInfo.

Suppose we want to create a Standard MBean, but with descriptor fields as in the snapshot above. Here's how we might go about it.

First, we create a subclass of StandardMBean called DescriptorStanda