Eamonn McManus's Blog

Community: Java Tools Archives

VisualVM - All-in-One Java Troubleshooting Tool

VisualVM is a new graphical troubleshooting tool that is being developed in a project on java.net. It gives access to the functionality that is available through existing JDK tools such as JConsole, jinfo, and jstack, and adds to that support for lightweight profiling of CPU and memory usage.

These screenshots from the project page give an idea of what the tool can do.

The tool is extensible via downloadable plugins, and the latest milestone publishes an API for developing your own plugins. Also, if you have developed plugins using the JConsole Plugin API it's possible to use them unchanged in VisualVM.

I'm not personally involved in this project but I'm certainly following it with great interest.

[Tags: visualvm.]

How much does it cost to monitor an app with jconsole?

Recently I've seen several people ask what the cost of enabling JMX monitoring on an application is. If I run with -Dcom.sun.management.jmxremote and connect jconsole, how much will that affect the performance of my app? Here are the results of some highly unscientific experiments.

Here's the quick answer for people who look at the answers in the back of the book. Running with -Dcom.sun.management.jmxremote has no appreciable impact. Connecting jconsole has an impact of 3--4%.

The answer to this question is likely to depend very much on the nature of the app and what else is going on in the machine where it is running. My measurements here are on a toy application which computes digits of the mathematical constant e using BigInteger arithmetic. This isn't intended to be an illustration of appropriate mathematical techniques. It's just a simple way to create an app that does a lot of computation and a lot of allocation of large short-lived objects. So my measurements here are likely to be interesting for apps like that, and probably not wildly irrelevant for other sorts of apps.

Here for reference is the method that the app spends most of its time in:

    private static void computeE(int digits) {
        BigInteger one = BigInteger.TEN.pow(digits);
        BigInteger e = BigInteger.ZERO;
        BigInteger invfact = one;
        BigInteger n = BigInteger.ONE;
        while (invfact.compareTo(BigInteger.ZERO) > 0) {
            e = e.add(invfact);
            invfact = invfact.divide(n);
            n = n.add(BigInteger.ONE);
        }
        System.out.println(e);
    }

Note by the way that the last few digits computed by this method are inaccurate due to truncation of the division results.

I ran this method with the digits parameter equal to 30000 on an elderly SPARC machine. I was logged in to the machine in an X Windows session too, so there was some background activity. I ran each measurement two or three times for each configuration and took the smallest time. (I did say this was unscientific. What are these repeatable test conditions and standard deviations of which you speak?)

I created one thread per CPU to make sure that the machine was CPU-bound. Otherwise we might see zero overhead for enabling monitoring because it might be using an otherwise-idle CPU.

Startup time is slightly greater when you enable monitoring because it needs to do some extra work such as creating an RMI connector. I assumed that the figure of interest, though, is the cost while the app is running. So I measured the time between the creation of the computeE threads and their completion.

I used two Java platforms for the measurements: Sun's JDK 1.5.0_07 and a Mustang snapshot (1.6.0-rc-b92). The time for the computation was 37% better on Mustang, presumably because of improvements to the JIT compiler and/or heap management. Also the time is better still if you run with -server. But what we're interested is the relative difference between monitoring and not monitoring.

The first measurement was of the application running without any special command-line parameters. So monitoring was not enabled. This is the baseline for the other measurements.

Then I ran the application with -Dcom.sun.management.jmxremote, which has two effects:

• It creates the Platform MBean Server and populates it with the standard set of MBeans containing the JVM's instrumentation.
• It creates and starts an RMI Connector Server and puts its address where jconsole can find it.

So with this option we could connect jconsole from the local machine. But I didn't connect jconsole for this measurement. The effect on performance was less than 1% in this case, and in fact any observed difference was less than the margin of error. So there was no significant difference in the time to do the computation. In other words, so long as you don't connect jconsole, enabling monitoring should have no effect on your app's performance.

I got the same result when I ran with
-Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.remote.ssl=false
so allowing jconsole to connect remotely but without security also has no effect on performance. Note that we strongly discourage running apps this way in production environments.

Running with
-Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.password.file=jmxremote.password
(meaning there is security including the creation of an SSL socket) also has an impact of less than 1%.

Then I ran the insecure options again but this time connected Tiger's jconsole from another machine. Here are the figures I got:

So here there's a fairly big impact. Some of this is due to jconsole polling the various JVM MBeans so it can graph the changes in various values like heap size and number of threads over time. There is no way to stop jconsole from doing that, even if you're not interested in that information, but what you can do is change the polling interval. If you change the interval from the default 4 seconds to 1000 seconds, say, then you won't see much impact from polling. (The interval also applies to the graphing of your own attributes, though, so if you want that then you'll have to pay for the JVM polling.)

Running Tiger's jconsole with -interval=1000 changes the figures to this:

Jconsole isn't polling, but there's still some overhead, presumably due to background activity from RMI and/or to increased memory usage because of the open connection.

Mustang's jconsole, besides being generally nicer, has an optimization that means it obtains the various JVM attributes much more efficiently, so the polling impact is less. Even if you can't migrate your apps to Mustang right now, you might want to download the Mustang JDK just so you can run its jconsole!

Here are the figures running the app with the insecure options and connecting with Mustang's jconsole from another machine:

So you get better behaviour from jconsole by default here. And of course you can still use the -interval=1000 trick, to get this:

Finally, I tried a few configurations with security enabled (-Dcom.sun.management.jmxremote.ssl=true -Dcom.sun.management.jmxremote.authenticate=true, which are the default settings) and did not see any signficant difference. Even with jconsole polling every four seconds, there was no observable extra overhead due to SSL.

Reminder

These results are not conclusive! They're merely indicative of one set of measurements on one type of app. The impact could be much greater if the addition of the JMX connection pushed the app into a different mode of operation. For example, the additional memory usage could be enough to change the behaviour of the garbage collector significantly and probably negatively. But you would have to be operating fairly close to the edge of such a possible change for that to happen, meaning it could happen even without the JMX connection if for example the app needed to handle a few more objects.

Using annotation processors to save method parameter names

The Java compiler doesn't save parameter names in the class files it generates. This is a problem for Standard MBeans, because we'd like to show those names in management clients. I talked about this in an earlier blog entry, where I suggested using a @PName annotation on each parameter to specify its name redundantly. Here's another approach, using annotation processors, which will work without adding any annotations at all.

[Updated 4 August 2006 to incorporate recent API changes.]

Annotation processors were introduced in the Tiger JDK, via the apt tool. apt is a command that you can use instead of javac. It does all the same things as javac, but in addition you can give it one or more annotation processors. An annotation processor is a sort of compiler plug-in that you can use to get the compiler to execute your own code during compilation.

Annotation processors cannot change the code that the compiler will generate for a given source file. (This is consistent with the idea that annotations cannot change Java language semantics.) However, they can generate new files based on what they find in the source files being compiled. In particular, they can generate new Java source files, and those new files will also be compiled in the current run. The annotation processors will be run on these new files too, potentially resulting in several "rounds" of source file generation.

Another interesting thing that annotation processors can do is to make checks. For example, the @Description annotation that I defined in the earlier blog entry only makes sense in an MBean interface. If you put it anywhere else, it will have no effect, and you will have no indication that whatever you were trying to achieve with it isn't working. You could use an annotation processor to detect stray @Description annotations and emit warnings or errors for them at compile time. Bruce Chapman gave an interesting BOF at the latest JavaOne on the subject of user-defined compile-time checking using annotation processors (slides here)

A processor for MBean operation parameter names

The name annotation processor is a bit of a misnomer. It's actually a general-purpose compiler plugin mechanism. You can arrange for it to analyze all source files that are being compiled, whether or not they contain annotations.

We can use this ability to define a compiler plugin that extracts the parameter names out of MBean operations defined in Standard MBeans. Here's the idea. Suppose we have the following Standard MBean interface (from the earlier blog entry):

package com.example.myapp;

public interface CacheControllerMBean {
    /** Drop the n oldest entries whose size matches the given constraints. */
    public int dropOldest(int n, int minSize, int maxSize);
}

We would like to generate another Java interface defined like this:

package com.example.myapp;

public interface CacheControllerMBeanPNames {
    public static final String[] dropOldestPNames = {
    	"n", "minSize", "maxSize",
    };
}

Using this, we will be able to modify the AnnotatedStandardMBean class that we defined before, so that it can pick up the names for the parameters in CacheControllerMBean by using reflection on CacheControllerMBeanPNames.

Before looking at how that can be done, let's look at some of the other ways we could achieve the same thing. One of them is to use some sort of script that picks out the relevant information and generates the needed files. The problem with this sort of script is that it's very difficult to do the required pattern matching correctly. Will we be able to write a regular expression that recognizes a Java interface no matter how it is defined? Even if it is defined like this for example?

public // for now
interface
/* Name may change in later version */ CacheControllerMBean
    {

If so, then we'll basically be reinventing the parser from the Java compiler. Why not just use that parser, as annotation processors allow us to do?

Another possibility is to define a doclet that you can plug into the javadoc tool. This can certainly work, but it is not as straightforward. You'll have to figure out how to invoke javadoc with your doclet, and how to arrange for the generated source files to be compiled and included in your jar file. All of this "just works" with annotation processors. Furthermore, doclets are not a standard Java feature, so you have to use a com.sun API to code them.

Assuming we do use an annotation processor, we can choose between generating Java source code and generating plain text (or binary) files. We could generate an XML file with the parameter names, for example. There are several advantages to generating a source file, however. One is that the compiler will do some basic sanity checking on what our processor generates. Another is that we don't have to do anything special to arrange for the generated class to be packaged up in our jar file; it just gets put there along with all the other compiled classes. A third advantage is that we can retrieve the information at runtime using the same Reflection API that we are already using to introspect the MBean interface.

Writing the annotation processor

Up until now I've been talking about the apt tool from JDK 5.0. That's a non-standard tool, as you can see from the interfaces you use to define a processor. These are all com.sun.* interfaces.

A standard annotation processor facility is being defined by JSR 269, "Pluggable Annotation Processing API". This will be part of the Mustang (Java SE 6) platform, which means that a Java compiler from any Mustang implementation can support it. Processing is no longer handled by a separate non-standard tool like apt, but by specifying the processor(s) directly to javac. The interfaces for defining a processor are all in the javax.* namespace.

The Mustang documentation for annotation processors is still a bit raw, and I found the slides from the BOF on the subject at the latest JavaOne to be immensely helpful.

The Java classes that are of interest when writing a processor are in javax.annotation.processing and in the various javax.lang.model.* packages.

A processor is a Java class that implements the Processor interface, usually by subclassing AbstractProcessor. We're going to define a processor in the class com.example.processors.MBeanPNameProcessor, and we'll be able to invoke it like this:

javac -processor com.example.processors.MBeanPNameProcessor source-files

To shorten the text, I'll use wildcard imports in what follows. In practice, you're always much better off getting your IDE to spell out the imports explicitly.

Here's the beginning of the processor:

package com.example.processors;

import java.io.*;
import java.util.*;
import javax.annotation.processing.*;
import javax.lang.model.*;
import javax.lang.model.element.*;
import javax.lang.model.util.*;
import javax.tools.*;

@SupportedAnnotationTypes("*")
@SupportedSourceVersion(SourceVersion.RELEASE_6)
public class MBeanPNameProcessor extends AbstractProcessor {
    public MBeanPNameProcessor() {
    }

Most processors will subclass AbstractProcessor as this one does. That means that they only have to implement the process method. You do have to supply some information with annotations on the processor, however.

The @SupportedSourceVersion annotation says what version of the Java programming language your processor understands. Here, we say we understand version 6, i.e. Mustang. Obviously that means that we understand all earlier versions too. But when the Dolphin (Java SE 7) platform arrives, and somebody compiles with javac -source 7, our processor won't run. We can't guarantee that our processor will be able to handle new language features like superpackages and XML literals that might show up in Dolphin. So this annotation guarantees that we won't make it try.

The @SupportedAnnotationTypes annotation is somewhat more complicated. Different processors can "claim" different annotations. Without getting into the details of this, let's just say that a processor such as ours that is not directly concerned with annotations should do the following:

• specify @SupportedAnnotationTypes("*");
• return false from its process method;
• be specified in the list of processors before any processor that also specifies @SupportedAnnotationTypes("*") and that returns true from its process method.

Every processor must have a public no-arg constructor, as here.

The next thing is a handy method for debugging the processor:

    private static final boolean silent = false;

    private void note(String msg) {
        if (!silent)
            processingEnv.getMessager().printMessage(Diagnostic.Kind.NOTE, msg);
    }

This is the equivalent of System.out.println, except its output shows up as a "note" from the compiler, something like this:
Note: Found com.example.myapp.CacheControllerMBean

We can set silent to true when we're confident the processor works.

Here, processingEnv is a protected field defined in the parent class AbstractProcessor and defined when the processor is initialized. Purists like me might have preferred a processingEnv() method, though you could see processingEnv as being like System.out.

Here's the process method, which defers the interesting work to another method we will define later:

    public boolean process(Set<? extends TypeElement> annotations,
                           RoundEnvironment roundEnv) {
        note("process: annotations=" + annotations + ", roundEnv=" + roundEnv);
        checkForMBeanInterfaces(roundEnv.getRootElements());
        return false;
    }

The annotations parameter is not interesting for us because we're not defining an actual annotation processor. The RoundEnvironment contains information about the current processing "round". Of this information, we're only interested here in the classes being compiled, as returned by getRootElements.

On the first round, process will be invoked with the set of classes that were named on the command line or that need to be recompiled because they're referenced by those classes. Then, if we generate any new source files, there'll be a second round where the classes will be the ones contained in those source files. This continues until a round generates no new source files; then there'll be a final round with no input classes that can be used to do any final processing.

We're not concerned with the details of rounds for this processor. Whatever classes we see, we'll analyze. If there are no classes, we won't analyze anything.

This processor will handle MBean interfaces that are top-level members of a package, but also that are defined within other classes. Defining an MBean interface as a nested class isn't recommended in general but I often do it for tests so that the whole test fits into one source file. The processor could be simplified slightly if we only supported top-level MBean interfaces.

The following method will initially be given the list of classes being compiled. Then for each class in the list it will be invoked recursively with all of that class's members, namely constructors, fields, methods, and nested classes. Of these, only nested classes interest us, so we'll filter out all the others. Finally we'll call another method on every class we see, including nested classes, in order to pick out the MBean interfaces.

    private void checkForMBeanInterfaces(Collection<? extends Element> elements) {
        note("checkForMBeanInterfaces: " + elements);
        Collection<? extends TypeElement> typeElements =
                ElementFilter.typesIn(elements);

        for (TypeElement type : typeElements) {
            checkForMBeanInterfaces(type.getEnclosedElements());
            checkForMBeanInterface(type);
        }
    }

Here's the method that detects an MBean interface. An MBean interface must match the following criteria:

• the name of the type ends with ...MBean
• the type is an interface (not a class or annotation or enum)
• the interface is public
• the interface has no type parameters (like MyMBean<T>)

    private void checkForMBeanInterface(TypeElement type) {
        // name must end in MBean
        if (!type.getQualifiedName().toString().endsWith("MBean"))
            return;

        // must be an interface
        if (type.getKind() != ElementKind.INTERFACE)
            return;

        // must be public
        if (!type.getModifiers().contains(Modifier.PUBLIC))
            return;

        // must not have type parameters
        if (!type.getTypeParameters().isEmpty())
            return;

        writePNamesInterface(type);
    }

If the type met all those conditions, then we're ready to analyze it and write out the generated ...MBeanPNames interface with the method parameter names:

    private void writePNamesInterface(TypeElement type) {
        note("Found " + type);
        String pnamesInterfaceName =
                processingEnv.getElementUtils().getBinaryName(type) + "PNames";
        try {
            writePNamesInterface(type, pnamesInterfaceName);
        } catch (IOException e) {
            processingEnv.getMessager().printMessage(Diagnostic.Kind.ERROR,
                    "Could not create source file for " + pnamesInterfaceName +
                    ": " + e);
        }
    }

If we get an exception while we're trying to write the new file, then we convert that into a compiler error as shown above. Generally speaking, processors should not throw exceptions.

We use getBinaryName here just because we're handling nested classes. The upshot is that if the interface is com.example.Main.TestMBean then we will generate com/example/Main$TestMBean.java rather than com/example/Main/TestMBean.java which would fail.

Here's the code that actually generates the new interface.

    private void writePNamesInterface(TypeElement type, String pnamesInterfaceName)
    throws IOException {
        Filer filer = processingEnv.getFiler();
        OutputStream os =
            filer.createSourceFile(pnamesInterfaceName).openOutputStream();
        PrintWriter pw = new PrintWriter(os);
        int lastDot = pnamesInterfaceName.lastIndexOf('.');
        String baseName = pnamesInterfaceName.substring(lastDot + 1);
        pw.println("// " + baseName + ".java - generated by " +
                MBeanPNameProcessor.class.getName());
        pw.println();
        if (lastDot > 0) {
            pw.println("package " + pnamesInterfaceName.substring(0, lastDot) + ";");
            pw.println();
        }
        pw.println("public interface " + baseName + " {");
        for (ExecutableElement method :
             ElementFilter.methodsIn(type.getEnclosedElements())) {
                writeMethodPNames(pw, method);
        }
        pw.println("}");
        pw.close();
        os.close();
    }

    private void writeMethodPNames(PrintWriter pw, ExecutableElement method) {
        pw.println("    public static final String[] " + method.getSimpleName() +
                   "PNames = {");
        pw.print("        ");
        for (VariableElement param : method.getParameters())
            pw.print("\"" + param.getSimpleName() + "\", ");
        pw.println();
        pw.println("    };");
    }
}

That's it! This is a complete processor that can generate a CacheControllerMBeanPNames interface like this:

package com.example.myapp;

public interface CacheControllerMBeanPNames {
    public static final String[] dropOldestPNames = {
    	"n", "minSize", "maxSize",
    };
}

(It's worth noting that the generated interface will be incorrect if the MBean interface contains overloaded methods, i.e. more than one method with the same name. We strongly recommend against including overloaded methods in MBeans so "this could be construed as a feature.")

Using the generated interface

All we need to do now is to modify AnnotatedStandardMBean from the earlier blog entry so that it picks up the parameter names from the ...MBeanPNames interface if it exists. Compared to writing an annotation processor, that is a piece of cake. The new code is in bold below:

    @Override
    protected String getParameterName(MBeanOperationInfo op,
                                      MBeanParameterInfo param,
                                      int paramNo) {
        String name = param.getName();
        Method m = methodFor(getMBeanInterface(), op);
        if (m != null) {
            PName pname = getParameterAnnotation(m, paramNo, PName.class);
            if (pname != null)
                name = pname.value();
	    else {
                String name1 = getNameFromPNames(op, name, paramNo);
                if (name1 != null)
                    name = name1;
            }
 
        }
        return name;
    }

    private String getNameFromPNames(MBeanOperationInfo op, String name, int paramNo) {
        try {
            Class<?> pnamesClass =
    	    	    Class.forName(getMBeanInterface().getName() + "PNames");
            Field namesField = pnamesClass.getField(op.getName() + "PNames");
            String[] names = (String[]) namesField.get(null);
            return names[paramNo];
        } catch (Exception e) {
            // no PNames class, or malformed
            return null;
        }
    }

Phew! I hope that's been of interest. My thanks to Joe Darcy, who took the trouble to give me some very detailed explanations.

This work is licensed under a Creative Commons License.