Kohsuke Kawaguchi's Blog

JAX-WS RI goes truly asynchronous!

As I've talked about in the past, I've been involved with making the JAX-WS RI better architectured. The first wave of that work is now stable enough to be made available as "milestone 1" and also in Glassfish v2 milestone1. Also see Vivek's blog for more about this.

But we aren't done yet. We recently learned that our JBI folks are interested in using the JAX-WS RI and WSIT as a lower level communication layer ("binding component" is their jargon, I think.) And they are very big on asynchrony — they speak of 100,000s of concurrent connections!

So Jitu and I have been working on a prototype of the evolution of the pipeline architecture that supports highly concurrent and asynchronous processing, tentatively dubbed as "valve." The basic idea is the emulation of the continuation passing style in Java. I've done this more seriously in the dalma project and in a way more transparent to Java applications, but the emulation approach we take for valves is different. It's not transparent, and it's custom-designed for the pipeline processing, so it has much smaller overhead. CPS allows one thread to juggle multiple concurrent requests, instead of blocking on I/O. Combined with truly asynchronous transports (like Grizzly), it would allow us to achieve a great scalability that otherwise is simply impossible.

The good news is that on the client side, the JAX-WS API already has a programming model to take advantage of this. So when this change is ready, your existing application just gets faster, without changing one line. On the server side, however, we needed a new kind of Provider API that allows middlewares and applications to process requests asynchronously. This is likely go into the JAX-WS RI's private package for a while.

interface AsyncProvider<T> {
  void invoke(T reqeuest, Callback<T> callback, WebServiceContext);
}
interface Callback<T> {
  void sendReply(T response);
  void sendFault(WebServiceException fault);
}

If you are interested in more details, see the slides here and here. So, have fun with the JAX-WS RI 2.0.1 m1, and stay tuned for more improvements to come in the future!

Comments

• Awesome! We also have a plan to add asynchronous processing (in addition to asynchronous I/O already possible) to the Restlet API, and your approach definitely makes sense.

  
  OTOH, it seems to me that this optimization should be fully transparent and handled at the JVM level. The JVM could indeed manage thousands of lightweight threads (fibers) that would share a limited pool of heavy threads. The JVM could swap between fibers at a regular interval and immediately if the fibers blocks on an I/O event. Maybe it would be too expensive?
  

  Posted by: jlouvel on October 19, 2006 at 08:16 AM

• I agree that the continuation would make a great addition to JVM, but not because it's a "lightweight thread" (because threads are as cheap as it gets with JVM), but because you can do interesting things with continuation.
  

  Posted by: kohsuke on October 19, 2006 at 08:38 AM

• But I thought that you introduced Valves because "We can't afford to tie a thread with one request/response processing" with very high loads.

  It seems to me that continuations are not a "natural" way of programming in a procedural languages like Java. They are indeed powerful, but more as a way to workaround the cost of creating new threads.

  Did I miss something?
  

  Posted by: jlouvel on October 19, 2006 at 09:22 AM

• 
  Yes, when we emulate this the "user land" in JAX-WS, we know exactly what state needs to be captured and restored, so I think that's why we can do it better than using a thread (which each reserves a fair amount of space, AFAIK, for the stack.)

  
  But when you ask JVM to do it, I think they essentially has to do the same with what the OS does. It would have to allocate a callstack space, etc. So isn't that going to take as much space as the OS native thread?

  
  Continuations have other uses than just a simple replacement of the thread. For example, you can hibernate the computation, move it to other machines, or to some limited extent goes back the computation.

  
  I share with your frustation that JVM seems to have enough knowledge about capturing the complete execution state of a thread (after all it knows enough about creating all the methods in the stack frame and its local variables), and so it just seems oh so easy for it to just "expose" that! Then we all benefit.

  Posted by: kohsuke on October 19, 2006 at 09:39 AM

This work is licensed under a Creative Commons License.