Scott Oaks's Blog: December 2007 Archive

Posted by sdo on December 19, 2007 at 01:01 PM | Permalink | Comments (5)

[NOTE: The code in this blog was revised 2/11/08 due to some errors on my part the first time, and some changes as it was ingtegrated into grizzly. And thanks to Erik Svensson for pointing out a few errors, it has been revised again on 2/13/08.]
I'm quite interested these days in parsing performance: much of what a Java appserver does is take bytes from a network stream (usually, but not always, in some 8-bit encoding) and convert them into Java strings (based on 16-bit characters). Because servlet and JSP APIs are written in terms of strings, much of that conversion is unavoidable, but parsing network protocols at the byte level is appropriate in some circumstances.

As I prepared to prototype some tests around that, I realized I needed a good framework to test my changes, and of course that framework is grizzly. In fact, the newly-released grizzly 1.7 has a new protocol parser that exactly fit my needs (partly because I joined the grizzly project so that I could modify the parser as I needed; such are the joys of open source!).

I'll talk about some of my performance tests with network parsing in later blogs; for now, I wanted to write a quick entry on how to use grizzly 1.7's new protocol parser. In grizzly 1.7, the ProtocolParser interface was reimplemented to make it much easier to deal with the messages that the parser is expected to produce. This means that it is now possible to use standard grizzly filters to handle the data produced by a ProtocolParser, simply like this:

controller.setProtocolChainInstanceHandler(new DefaultProtocolChainInstanceHandler() {
      public ProtocolChain poll() {
          ProtocolChain protocolChain = protocolChains.poll();

          if (protocolChain == null) {
              protocolChain = new DefaultProtocolChain();
              ((DefaultProtocolChain) protocolChain).setContinuousExecution(true);
              protocolChain.addFilter(new MyParserProtocolFilter());
              protocolChain.addFilter(new MyProcessorFilter());
          }

          return protocolChain;
      }
}

The nice thing about this is that additional filters (like a debugging log filter) can be inserted anywhere along the chain; the protocol use is completely integrated into the standard grizzly design. Note that call to setContinuousExecution -- it should be the default for protocol parsers (and will be eventually), but version 1.7 of grizzly will need that call. [Note that the standard LogFilter in grizzly is not appropriate in this case, since it tries to read directly from the socket as well; it's trivial to write your own if you like.]

Now it's a matter of implementing the two filters and the parser itself. The ParserProtocolFilter class will handle reading the requests and calling the parser, but in order for it to know which parser to use, you must extend it and override the newProtocolParser method:

public class MyParserProtocolFilter() {
    public ProtocolParser newProtocolParser() {
         return new MyProtocolParser());
    }
}

What about the parser itself? That's the meat of the issue. The new protocol parser interface expects a basic flow like this: start processing a buffer, enumerate the message in the buffer, and end processing the buffer. The buffer can contain 0 or more complete messages, and it's up to the protocol parser to make sense of that. Here's the outline of a simple protocol parser that parses a protocol where the first byte is a number of bytes in string, followed by the remaining bytes:

public class MyProtocolParser implements ProtocolParser {
    byte[] data;
    int position;
    ByteBuffer savedBuffer;
    int origLimit;
    public void startBuffer(ByteBuffer bb) {
        // We begin with a buffer containing data. Save the initial buffer
        // state information. The best thing here is to get the backing store
        // so that the bytes can be parsed directly. We also need to save the
        // original limit so that we can place the buffer in the correct state at the
        // end of parsing
            savedBuffer = bb;
            savedBuffer.flip();
            partial = false;
            origLimit = savedBuffer.limit();
            if (savedBuffer.hasArray()) {
                data = savedBuffer.array();
                position = savedBuffer.position() + savedBuffer.arrayOffset();
                limit = savedBuffer.limit() + savedBuffer.arrayOffset();
            } else ...maybe copy out the data, or use put/get when parsing...
    }

    public boolean hasMoreBytesToParse() {
        // Indicate if there is unparsed data in the buffer
        return position < limit;
    }

    public boolean isExpectingMoreData() {
        // If there is a partial message remaining in the buffer, return true
        return partial;
    }

    public String getNextMessage() {
        // We already know this, but other protocols might parse here
        return savedString;
    }

    public boolean hasNextMessage() {
        // In our case, it's easier to parse here
        int length = data[position];
        if (data.length < position + 1 + length) {
            savedString = new String(data, position + 1, length);
            position += length + 1;
            savedBuffer.limit(length + position + 1);
            savedBuffer.position(position + 1);
            partial = false;
        }
        else partial = true;
        return !partial;
    }

    public boolean releaseBuffer() {
        // If there's a partial message return true; else false
            if (!hasMoreBytesToParse())
                savedBuffer.clear();
            else {
                // You could compact the buffer here if you're
                // concerned that there isn't enough space for
                // further messages, but compacting comes at a
                // performance price -- whether to compact or not
                // depends on your protocol.
                savedBuffer.position(position);
                savedBuffer.limit(origLimit);
            }
            return partial;
    }
}

The point of this is that the ParserProtocolFilter will repeatedly call hasNextMessage/getNextMessage to retrieve messages (Strings in this case) to pass to the next filter. When it's done, it will call releaseBuffer, which is responsible for setting the position and limit in the buffer to reflect the data consumed by the (possibly multiple) messages returned.

So what about the downstream filters? You probably noticed that when we parsed the data, we also set the limit/position in the ByteBuffer to reflect the message boundaries. That's because not all grizzly filters will understand that the data is protocol based and has been seperated into types. For instance, you could write a LogFilter that just prints out the data received; it doesn't know about the messages (and we wouldn't want it to -- we'd want it to print the raw data anyway, rather than information in the message).

But downstream filters can also understand what a message is and hence they can work like this:

public class MyProcessorFilter implements ProtocolFilter {
    public boolean execute(Context ctx) {
        String s = (String) ctx.getAttribute(ProtocolParser.MESSAGE);
        if (s == null) {
            // no message; just use the bytes in the buffer like a
            // normal filter
            s = getStringFromBuffer(ctx);
        }
        .. do something with s ..
    }
}

So, apart from writing the protocol parser (which could be quite complex, depending on the actual protocol and how it breaks into messages), using the new grizzly framework for protocol parsing is quite simple: you just set up the parser class, and then have a filter that processes the messages from the parser. And long the way, you can use any other grizzly filter or framework feature you need.

A Glassfish Tuning Primer

Posted by sdo on December 03, 2007 at 11:25 AM | Permalink | Comments (1)

When I reported our recent excellent SPECjAppServer 2004 scores, one glassfish user responded:

I sure wish you guys were able to come up with a thorough write up
about the SPEC Benchmark architecture, and the techniques you guys
used to get the numbers you get and, more importantly, how those
techniques might apply to every day applications we run in the wild.

While we do have a full performance-tuning chapter in the glassfish/SJSAS docset, I can understand the appeal of a quick cheat-sheet for getting the most out of glassfish in production. Most of this information has appeared in various blogs, particularly by Jeanfrancois, who is so expertly focused on making sure that grizzly and our http path is as fast as possible. Still, I hope that gathering this quick list together will be a good single-source summary.

One thing to note about these guidlines: a lot of glassfish configurations (particularly when you start with a developer profile) are optimized for developers. In development, performance is different: you'll trade off a few seconds here and there to make starting the appserver faster, or deploying something faster. In production, you'll make opposite trade-offs. So if you wonder why some of the things in this list aren't necessarily the default setting, that's probably why.

Tune your JVM

The first step is to tune the JVM, which is of course different for every deployment. These are the options set via the jvm-option tag in your domain.xml (or the JVM options page in the admin console). As a general rule, I like to use the throughput collector with large heaps and a moderate-sized young generations: that makes young GCs quite fast. That will lead to a periodic full GC, but the impact of that on total throughput is usually quite minimal. If you absolutely cannot tolerate a pause of a few seconds, you can look at the concurrent collector, but be aware that this will impact your total throughput. So a good set of JVM arguments to start with are:

-server -Xmx3500m -Xms3500m -Xmn1500m -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+AggressiveOpts

On a CMT machine like the SunFire T5220 server, you'll want to use large pages of 256m, and a heap that is a multiple of that:

-server -XX:LargePageSizeInBytes=256m -Xmx2560m -Xms2560m -Xmn1024m -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:ParallelGCThreads=16 -XX:+AggressiveOpts

More details of the impact of a CMT machine are available at Sun's Cool Threads website.

Make sure to remove the -client option from your jvm options, to include the -Dcom.sun.enterprise.server.ss.ASQuickStartup=false flag, and -- if you are using CMP 2.1 entity beans -- to include -DAllowMediatedWriteInDefaultFetchGroup=true.

Tune the default-web.xml

Settings in the default-web.xml file are overridden by an application's web.xml, but I find it easier to set production-ready values in the default-web.xml file so that all applications will get them. In particular, under the JspServlet definition, add these two parameters:

<init-param>
  <param-name>development</param-name>
  <param-value>false</param-value>
</init-param>
<init-param>
  <param-name>genStrAsCharArray</param-name>
  <param-value>true</param-value>
</init-param>

That will mean you cannot change JSP pages on your production server without redeploying the application, but that's generally what you want anyway.

On note about this: this file is only consulted when an application is deployed. So make sure you change the file and then deploy your application, or you won't see any benefit from this change.

Tune the HTTP threads

As you know, there are two parameters here: the HTTP acceptor threads, and the request-processing threads. These value have unfortunately had different meanings in a few of our releases, and some confusion about them remains. The acceptor threads are used to both to accept new connections to the server and to schedule existing connections when a new request comes over them. In general, you'll need 1 of these for every 1-4 cores on your machine; no more than that (unlike, say SJSAS 8.1 where this had a completely different meaning). The request threads run HTTP requests. You want "just enough" of those: enough to keep the machine busy, but not so many that they compete for CPU resources -- if they compete for CPU resources, then your throughput will suffer greatly. Too many request processing threads is often a big performance problem I see on many machines.

How many is "just enough"? It depends, of course -- in a case where HTTP requests don't use any external resource and are hence CPU bound, you want only as many HTTP request processing threads as you have CPUs on the machine. But if the HTTP request makes a database call (even indirectly, like by using a JPA entity), the request will block while waiting for the database, and you could profitably run another thread. So this takes some trial and error, but start with the same number of threads as you have CPU and increase them until you no longer see an improvement in throughput.

Tune your JDBC drivers

Speaking of databases, it's quite important in glassfish to use JDBC drivers that perform statement caching; this allows the appserver to reuse prepared statements and is a huge performance win. The JDBC drivers that come bundled with the Sun Java Systems Application Server provide such caching; Oracle's standard JDBC drivers do as well, as do recent drivers for Postgres and MySQL. Whichever driver you use, make sure to configure the properties to use statement caching when you set up the JDBC connection pool -- e.g., for Oracle's JDBC drivers, include the properties

ImplicitCachingEnabled=true
MaxStatements=200

Use the HTTP file cache

If you serve a lot of static content, make sure to enable the HTTP file cache.

Have I piqued your interest? As I mentioned, there are hundreds of pages of tuning guidelines in our docset. But here at least you have some important first steps.

Powered by
Movable Type 3.01D

java.net RSS Feeds