Skip to main content

Pushing Jersey to the limit

Posted by ljnelson on April 28, 2010 at 1:57 PM PDT

I wanted to describe an interesting problem I recently faced that involved the most excellent JAX-RS specification and its Jersey implementation.  The quick overview is that we wanted the output of our JAX-RS resource classes to be client-agnostic, but we also wanted the ability to insert a decorator of sorts in between the objects we were returning and the normal JAX-RS serialization/writing process, all without relying on any vendor-supplied extensions.  This turned out to push the capabilities of the JAX-RS specification quite hard, and I thought the experience was worth writing up.  Thanks to Paul Sandoz and Marc Hadley for making appropriate noises along the way.

The Client

Many of you may be familiar with SmartGWT.  This is a truly wonderful AJAX toolkit that provides a GWT wrapper around a robust, battle-hardened Javascript library called SmartClient.  One of its many strengths is its comprehensive support for data binding by way of DataSources.

One such DataSource is the RestDataSource, which is an extremely convenient way to talk to a preexisting back end service.  Its convenience comes at a minor price: to the extent that you use it out of the box, you are bound by its request and response formats.  These are not onerous formats to support, but they do smell like SmartClient, and to the extent you want to not couple your stuff to a given vendor's requirements you might not want to hardcode your output to meet these requirements.  (There are many ways to tweak SmartGWT so that it sends and expects data in ways that you control, but I won't be covering those here.)

The Server

In a project I'm working on, we had the requirement to offer up our API via JAX-RS-compliant web services (I don't want to say REST, because I really really really really don't want to go down that rathole; that's what rest-discuss is for).  We also wanted to not get into the serialization format wars--we wanted to define our resources such that they could be returned via XML or JSON, and according to whatever whims a given client might expect.

That meant (for us) that we wanted our resource class to return simple collections of objects, and leave both the XML (or JSON) serialization to JAX-RS, and--more interestingly--any wrapping or reorganizing behavior to JAX-RS as well.  That is, a given resource would not know that it was being called from a SmartGWT client, even though the SmartGWT client expects XML data in a particular format.

As it turns out, as I mentioned, the requirements imposed by the RestDataSource are pretty mild.  A SmartGWT RestDataSource can indicate, for a retrieval operation, whether it wants paging or not, and it can also specify the start index and the (exclusive) end index of the slice that it wants you to return.  Then it expects that you return a very particular XML payload; consult the RestDataSource documentation for all the details.

The Problems

So we had several problems:

  • How were we going to turn a request like foo/bar?_startRow=0&_endRow=3&_operationType=fetch into something not so SmartGWT-RestDataSource-ish, but preserve the semantics?
  • How were we going to take a Collection, as returned by one of our resource classes, and get it ultimately spat out in RestDataSource-friendly XML?

Handling the Request

 First, we focused on the "incoming" problem.

RestDataSource, if not otherwise configured, sends its "fetch" requests with at least three well-known request parameters:

  • _operationType=fetch
  • _startRow=some number
  • _endRow=some number

The concepts here are good, but the "row"-ness doesn't really fit with the way we think about things, so we decided to normalize this into a URL format that we defined.  A request like:

http://your.company.com/api/gorp?_operationType=fetch&_startRow=0&_endRow=10

would be canonically represented to our resource classes instead as:

http://your.company.com/api/gorp/paged/0,10

To do this, we employed a simple javax.servlet.Filter that used a RequestDispatcher to forward the incoming request to the new URL.  (Note that we did not eliminate the query parameters--they are still there, but our classes never look for them, just as our classes might not look for other query parameters that other client toolkits might define.)

Describing this Filter is probably a subject for another blog entry--suffice it to say this was not hard to write, and I'm not going to cover its implementation here.

Handling the Response

Next, we focused on the "outgoing" problem.

Assuming we have a resource class that returns something, like this:

@Path("gorp")
public class GorpResource {

  @Path("paged/{pagingControl}")
  public Collection<? extends Gorp> getGorpPaged(@PathParam("pagingControl") PagingControl control) {
    // we'll fill this in in a bit
  }
}

...how do we turn a Collection into RestDataSource-compliant XML?

RestDataSource XML, it turns out, requires a very specific format.  It looks like this:

<response>
  <status>0</status>         <!-- your response status; haven't explored the legal values -->
  <startRow>0</startRow>     <!-- the zero based index that was handed to you -->
  <endRow>10</endRow>        <!-- the exclusive index of the data you were able to return
  <totalRows>400</totalRows> <!-- an estimate of the total size of the domain that your query COULD have returned -->
  <data>
    <!-- your data goes here in any format you want -->
  </data>
</response>

(It also turns out this isn't exhaustively documented anywhere; it now is in the SmartGWT forums.)

The answer lies with the MessageBodyWriter class.

MessageBodyWriter

A MessageBodyWriter is a part of JAX-RS whose implementations are responsible for serializing a resource class's return value into a format that the client can deal with.  Normally a MessageBodyWriter is supposed to be tied closely to its output format.  It's expected, in other words, that if you want to write one of these things, you're doing so because you want to stream your payload in a particular text format, say, or you want to wrestle control away from the JAXB-driven default that JAX-RS ships with.

In our case, we actually didn't want to do this.  We simply wanted to wrap the Collection up inside a wrapper object that would define the RestDataSource "header" and "footer" elements that I hope are evident from the sample text above.  Then in a perfect world we would take that wrapper object, our Collection safe inside it, and hand it back to JAX-RS as though it had been returned directly from the resource class in the first place.  JAX-RS would serialize this wrapper object as it normally would, regardless of how else it had been configured, and everything would come up roses.

It takes a lot of manure to bring up roses.  So I began shoveling.

First, I defined a RestDataSource container object that I labeled--inspiringly--RestDataSourceResponse.  I've reproduced the interesting parts below:

@XmlRootElement(name = "response")
@XmlType(propOrder = { "status", "startRow", "endRow", "totalRows", "data" })
class RestDataSourceResponse implements Serializable {
  private static int defaultPageSize;
  private final int status;
  private final long startRow;
  private final long endRow;
  private final long totalRows;
  @XmlAnyElement
  @XmlElementWrapper(name = "data")
  private final Collection<?> data;
}

If I could manage to set this class's data element to our Collection, then serializing this thing would cause SmartGWT XML to go out over the wire.

Next up, I defined a MessageBodyWriter that made use of just about every JAX-RS way of injecting things under the sun.

A bit of background.  When JAX-RS was born, dependency injection was not a new concept but had not been baked into Java EE 6 yet (see JSR 299 for all those gory details).  So JAX-RS has its own way of doing a limited degree of dependency injection.  Oversimplifying on purpose, in general you can put @Context on an instance variable somewhere, and provided it's The Right Sort Of Thing it will be injected for you.

One such Thing that I was going to need in my MessageBodyWriter was an instance of Providers, because that's how you get existing MessageBodyWriters that JAX-RS already knows about.  So high up in my MBW is an instance variable like this:

@Context
private Providers providers;

So, now armed with a way to root around in JAX-RS's innards, I had to figure out exactly what an MBW is supposed to do.

Implementing the MessageBodyWriter Contract

The first thing to note is that a MessageBodyWriter takes a type parameter.  The idea here is that you're usually going to create a MessageBodyWriter that knows how to serialize a Gorp instance, for example, and nothing else, so you're going to create a subclass of it called GorpWriter, let's say, and it's going to extends MessageBodyWriter:

@Provider
public class GorpWriter extends MessageBodyWriter<Gorp> {
  //...
}

But that's not what we had to do, because we wanted to handle everything.  So our MessageBodyWriter implementation had to extend MessageBodyWriter:

@Provider
public class RestDataSourceAdapter extends MessageBodyWriter<Object> {
  //...
}

(What's the @Provider annotation?  This is JAX-RS's way of indicating that a given class implements an interface.  By marking a class as a @Provider, you're saying that it is eligible for dependency injection, or, in this case, return from Providers#getMessageBodyWriter().)

isWriteable

The first thing a MessageBodyWriter has to do is to see if it can write a particular thing that is handed to it.  This is handled by its implementation of the isWriteable method.  Briefly, you're handed a return value from some resource class somewhere, along with some--but not all--of the context ultimately available to you from the request, and you get to figure out whether you can handle the thing or not.

In our case, we had to translate.  We had to translate the given object into a RestDataSourceResponse (by wrapping it), and then delegate the question to the innards of JAX-RS:

@Override
public boolean isWriteable(final Class<?> type, final Type genericType, final Annotation[] annotations, final MediaType mediaType) {
  boolean returnValue = false;
  if (type != null && !(RestDataSourceResponse.class.isAssignableFrom(type))) {
    final MessageBodyWriter<RestDataSourceResponse> delegate = this.providers.getMessageBodyWriter(RestDataSourceResponse.class, RestDataSourceResponse.class, annotations, mediaType);
    returnValue = delegate != null && delegate.isWriteable(RestDataSourceResponse.class, RestDataSourceResponse.class, annotations, mediaType);
  }
  return returnValue;
}

The signature here is worth investigating.  The type argument looks to my eyes (after experimentation) like it is the actual class of the returned object from the resource class.  The genericType argument looks to be the declared return type.  Anyway, both are available to you to help you make your decision about whether you want to write instances of these types or not.

The first thing to notice here is that we check to make sure that the type in question is not RestDataSourceResponse.class.  We want to let JAX-RS's normal innards handle the writing of such things itself.

In the case that we are talking about any other class, we ask JAX-RS for a MessageBodyWriter that knows how to deal with RestDataSourceResponses.  If we haven't done anything else fancy with JAX-RS, this will be some default Jersey-supplied JAXB writer of some kind; we just care that it's non-null.  When we get such a delegate, we ask him if he is capable of writing instances of RestDataSourceResponse, which we'll be producing in our writeTo() method (detailed later).

getSize

Next up is the getSize() method.  Here a MessageBodyWriter is being asked to compute the size, in bytes, of the payload headed back to the browser.  The specification implies, but does not state directly, that by the time this method is called you can assume that your isWriteable() method has been called and has returned true.

You can punt on this method if you like, and return -1L, but I wanted to be accurate.  To be accurate, I had to take the incoming object whose size is to be calculated, surreptitiously wrap it in a RestDataSourceResponse object, and then hand that back into the innards of JAX-RS so that it can do the size calculation:

@Override
public long getSize(final Object object, final Class<?> type, final Type genericType, final Annotation[] annotations, final MediaType mediaType) {
  long returnValue = -1L;
  final MessageBodyWriter<RestDataSourceResponse> delegate = this.providers.getMessageBodyWriter(RestDataSourceResponse.class, RestDataSourceResponse.class, annotations, mediaType);
  if (delegate != null) {
    // CreateResponse basically wraps the incoming object in a
    // new RestDataSourceResponse
    final RestDataSourceResponse response = this.createResponse(object);
    assert response != null;
    returnValue = delegate.getSize(response, RestDataSourceResponse.class, RestDataSourceResponse.class, annotations, mediaType);
  }
  return returnValue;
}

This delegation pattern should be quite familiar.  We are, once again, asking JAX-RS to do the dirty work, pretending that the resource has actually returned a RestDataSourceResponse instead of whatever it actually returned.

Now for the fun part.

writeTo

A MessageBodyWriter has a writeTo() method that actually does the writing of the object.  As you might expect, we're going to implement this by following the same delegation strategy we did for the prior two methods.  But there's an added twist, which is that we need to somehow augment the JAXBContext that's going to tell JAXB how to serialize our response...but I'm getting ahead of myself.

Here's the writeTo method, implemented naively, which, at its heart looks similar to the other two.  I've omitted error checking and logging to keep this short; you would be nuts to do the same:

@Override
public void writeTo(final Object object,
                    final Class<?> type,
                    final Type genericType,
                    final Annotation[] annotations,
                    final MediaType mediaType,
                    final MultivaluedMap<String, Object> httpHeaders,
                    final OutputStream entityStream)
  throws IOException, WebApplicationException {
  assert type.isInstance(object);
  final MessageBodyWriter<RestDataSourceResponse> realWriter = this.providers.getMessageBodyWriter(RestDataSourceResponse.class, RestDataSourceResponse.class, annotations, mediaType);
  final RestDataSourceResponse response = this.createResponse(object);
  realWriter.writeTo(response, RestDataSourceResponse.class, RestDataSourceResponse.class, annotations, mediaType, httpHeaders, entityStream);
}

Now I say "naively", because although this skeleton is correct, JAX-RS is not going to be able to write our response.  Why not?

Welcome to the JAXB Jungle

We are going to detour out of JAX-RS land and into the thorny, tangled jungle of JAXB, which I fear might be thoroughly understood by only the mighty Kohsuke Kawaguchi.

As it turns out, in the project I was working on that drove this blog entry, we make liberal use of interfaces for a whole host of reasons--some good, some bad.  JAXB really really really really really really doesn't like interfaces, and so when you start wanting to have them turned into XML, you run into a whole heap of trouble.  The aforementioned Mr. Kawaguchi has produced the Unofficial JAXB Guide, which seems to be the only source of information on how to get JAXB to deal with interfaces, and, as you can see, it's not trivial.

Additionally, in our writeTo() method above, if we forward all control off to the "real" JAX-RS-provided MessageBodyWriter--the one that will use JAXB by default to do XML serialization--then the JAXB environment within which it runs will contain only the RestDataSourceResponse class (as well as some other boilerplate ones, like java.util.Date and the like).  Notably, it will know nothing about classes referred to by our RestDataSourceResponse, such as, for example, the class of its wrapped object (our Collection).  Gee, it sure would be nice if the "real" MessageBodyWriter would use a JAXBContext built by us that knows all about the type(s) we just wrapped.

What to do?

Getting a JAXBContext: Implementing ContextResolver

It turns out that Jersey's implementation of the JAX-RS specification allows us to supply our own JAXBContext that knows about all sorts of types.  To offer it up, we need to provide a ContextResolver implementation, known as a "context provider", and that context provider had best be stuffed with every last concrete type we can think of at the time that we go to write our response.

When such a ContextResolver exists, Jersey's innards will grab it and use it to provide a JAXBContext for its internal operations.  Perfect!

So first, we amend our MessageBodyWriter signature so that it also implements ContextResolver:

@Provider
public class RestDataSourceAdapter extends MessageBodyWriter<Object> implements ContextResolver<JAXBContext> {
  //...
}

Conveniently enough, we're already a @Provider, so apart from implementing the getContext() method, we're all set.

Let's think for a moment what we would have to do to get a JAXBContext out of this thing.

We know we'll need to invoke the newInstance(Class...) method.  And we know we're going to need RestDataSourceResponse.class, and the classes reachable from its payload, and--

Of course, we don't have any of that inside the getContext() method.  So before we even get started we're going to have to go back to writeTo() and somehow make that information available outside of it in a thread-safe manner.

Supplying JAXBContext With Classes: Enter Instance-level ThreadLocals

What we want is a container that we can use to grab type information from writeTo() that is also available to our getContext() implementation.  But many threads may be in play.  So we need a thread-safe construct for storing all of these classes that we're going to need to build a new JAXBContext instance.

Near the top of our MBW, therefore, we do this:

private final ThreadLocal<Set<Class<?>>> types;

(It occurs to me, actually, as I write this, that this probably should be a ThreadLocal>>>.  But that's just too many angle brackets, don't you think?    Stay tuned to this blog in case I get burned by this.)

And then, in our writeTo method, we do this:

final Set<Class<?>> typeSet = new HashSet<Class<?>>();
this.processTypes(typeSet, mediaType, type, RestDataSourceResponse.class, genericType, object.getClass());
final RestDataSourceResponse response = this.createResponse(object);
assert response != null;
final Collection<?> data = response.getData();
if (data != null) {
  this.processTypes(typeSet, mediaType, data.getClass());
}
this.types.set(typeSet);

We'll look at the processTypes() method in a moment.  The gist is that we're grabbing all the type information we possibly can (without doing something silly like introspecting classes), and we're setting it into our instance-level ThreadLocal type set.  Then, later, our getContext() method can use that variable to discover all the types it's going to need.

(The ThreadLocal has to be an instance variable and not a static field because we're interested in the thread that is working on this particular instance of MessageBodyWriter-and-ContextResolver, not any old thread that happens to be working with any old instance.)

Now let's look at processTypes to see exactly what kind of concrete type information we can suck out of the arguments to the writeTo() method without getting too ridiculous.  The gist here is that we take in a Set of Class instances, which we use to keep track of types we've already gathered up, a couple of other boilerplate parameters, and then a variable list of Types that need to be mined for more Class instances.  We recursively plow through this pile until we're all done, and at the end of it all the Set that was passed in will contain all the class information we can possibly infer about the stuff that came back from our resource class's return value:

private final void processTypes(final Set<Class<?>> typeSet, final MediaType mediaType, final Type... types) {
  if (typeSet == null || types == null || types.length <= 0) {
    return;
  }
  for (final Type type : types) {
    if (type == null || typeSet.contains(type)) {
      continue;

    } else if (type instanceof ParameterizedType) {
      final ParameterizedType ptype = (ParameterizedType)type;
      // Recursive calls
      this.processTypes(typeSet, mediaType, ptype.getRawType());
      this.processTypes(typeSet, mediaType, ptype.getOwnerType());
      this.processTypes(typeSet, mediaType, ptype.getActualTypeArguments());

    } else if (type instanceof GenericArrayType) {
      final GenericArrayType gat = (GenericArrayType)type;
      // Recursive call
      this.processTypes(typeSet, mediaType, gat.getGenericComponentType());

    } else if (type instanceof WildcardType) {
      final WildcardType wt = (WildcardType)type;
      // Recursive calls
      this.processTypes(typeSet, mediaType, wt.getLowerBounds());
      this.processTypes(typeSet, mediaType, wt.getUpperBounds());

    } else if (type instanceof TypeVariable) {
      final TypeVariable<?> tv = (TypeVariable<?>)type;
      // Recursive call
      this.processTypes(typeSet, mediaType, tv.getBounds());
      final GenericDeclaration gd = tv.getGenericDeclaration();
      if (gd != null) {
        if (gd instanceof Class) {
          // Recursive call
          this.processTypes(typeSet, mediaType, (Class<?>)gd);
        }
        // Recursive call
        this.processTypes(typeSet, mediaType, gd.getTypeParameters());
      }

    } else if (type instanceof Class) {
      final Class<?> c = (Class<?>)type;
      if (Modifier.isAbstract(c.getModifiers()) || c.isInterface()) {
       /*                  * We'll talk about this in a moment.         */       } else {
        typeSet.add(c);
      }
    } else {
      throw new UnsupportedOperationException("I don't know how to work with a Type of type " + type.getClass());
    }
  }
}

I probably overdid this method, since I'm not sure--and none of the Javadoc says one way or the other--what kinds of Types can be supplied to writeTo.

So again, this method recursively calls itself until it finds a concrete Class type, whereupon it adds it to the set.

What about abstract types and interfaces?  Recall that these types are going to be used to build a JAXBContext, and JAXB doesn't like interfaces or abstract classes.

As it turns out, we can use the same ContextResolver magic here to allow other registered ContextResolvers to provide us with concrete classes when they are supplied with interfaces.  This is sort of like dynamically implementing an @XmlSeeAlso annotation.  So the "we'll talk about this" comment above can be replaced with:

final ContextResolver<Class> concreteClassProvider = this.providers.getContextResolver(Class.class, mediaType);
if (concreteClassProvider != null) {
  final Class<?> concreteClass = concreteClassProvider.getContext(c);
  if (concreteClass != null && !Modifier.isAbstract(concreteClass.getModifiers()) && !concreteClass.isInterface()) {
    typeSet.add(concreteClass);
  }
}

...and now if there are any ContextResolver implementations lying around on the classpath, they'll be called and will have the opportunity to offer up a concrete class that implements the supplied abstract class or interface.

Putting it All Together

Back to our own ContextResolver implementation.  OK, so we've got a place where a bunch of type information is stashed away.  Now all we have to do is use it.  Here's the naive implementation:

@Override
public JAXBContext getContext(final Class<?> ignored) {
  JAXBContext returnValue = null;
  final Set<Class<?>> types = this.types.get();
  if (types != null && !types.isEmpty()) {
    try {
      returnValue = JAXBContext.newInstance(types.toArray(new Class<?>[types.size()]));
    } catch (final JAXBException jaxbException) {
      // Here's where you'd log the error.  You WILL log the error, right?
    } finally {
      this.types.remove();
    }
  }
  return returnValue;
}

JAX-RS's innards will call this method during the JAX-RS-supplied-and-implemented rendering of our RestDataSourceResponse object.  It will probably pass in something like RestDataSourceResponse.class, but, to be frank, by this point we know more about types than even JAX-RS itself does, so we essentially just ignore the incoming parameter in favor of getting our Set of Classes instead.  Once we get our paws on these Classes, we use them to create a new JAXBContext instance, which we then return into the bowels of JAX-RS somewhere, blissfully unaware of how it's going to be used.

If anything goes wrong here, we return null, which is permitted (and encouraged) by the contract to indicate that we're just not a very useful ContextResolver after all.

Conclusion

Here's the five second version.  This blog entry has shown you how to:

  • take the return value of a resource class
  • stash away JAXB-related type information from it
  • wrap it in a reasonably arbitrary wrapper
  • ask JAX-RS to serialize the wrapper object
  • offer up a new JAXBContext customized for the particular kind of wrapper object

...all without relying on anything other than the JAX-RS specification.

I hope you found this useful.  Thanks for reading, and thanks to Paul Szndoz and Marc Hadley for a great, flexible specification and implementation.

Best,
Laird

Comments

Could you please post the sources?

Hi Laird,
thank you for sharing your experience. It's so useful!
I really like your idea to decorate the client agnostic JAX-RS resource output with smartclient sugar: it leads at the same time to a clean, simple and so smart :-) solution.
I was just ruminating on this kind of solution, but I am not a JAX-RS/JAXB expert and from your example it doesn't seem so easy to implement.

I'm starting to develop some code following your article, but I have a somewhat strict schedule and I would avoid coming to a standstill after days of trials...
so, in case you could not post the sources, could you please just give some feedback about the overall feasibility/effectiveness?

One last thing: if the entire machinery works you could really propose it as a smartgwt extension.

Thank you in any case
Davide

The dot at the end of the sentence.

Hi Laird First of all, thanks for this well-written elaboration on how to make Jersey meet SmartGWT in the middle. I'm on my second day using the REST architecture and found your article while trying to figure out how to make Jersey speak SmartGWT's language. Needless to say, I feel like a very little fish in a very big ocean - but at least I understand the concept behind your solution. There is however one little thorn in my side - the RestDataSourceAdapter.createResponse implementation. What may seem trivial to you is my only obstacle to be authentically enlightened by this post. Would you be so kind as to provide the implemetation?

Have you succeeded?

Hi ninjav,
have you succeeded in implementing the entire solution?
I am struggling to understand why my RestDataSourceAdapter is never called...

Thank you
Davide

Alternative solution?

Hi, I've been working on a similar solution, which might be a bit easier (even though it doesn't handle to transformation between vendor specific formats etc). However, it works really great for letting SmartGWT talk to a Jersey-backed web service. Have a look at http://technowobble.blogspot.com/2010/08/using-smartgwt-with-jersey-rest....

RestDataSource

Hi Laird, Thanks for the praise for SmartGWT and thanks for making use of our technology for this article.

On the RestDataSource, you spent quite a lot of time talking about the particular format it expects - you may not have realized how absolutely trivial it is to make the superclass (DataSource) consume whatever format you want. See this sample; there's no built-in support for RSS because only a single setting is required:

http://www.smartclient.com/smartgwtee/showcase/#data_integration_server_rss

So, if you wanted a different format, you're just a few (declarative) settings away.. although it seems like in this instance the RestDataSource format was pretty much what was wanted anyway.

[NOTE: the RSS sample is in the Pro/EE showcase solely because it uses a server-side proxy to access an RSS feed on a remote host. Identical code could access a local RSS feed with the free LGPL version]

Also, on your lengthy conversation with Support in the forums - everything discussed is actually documented, it's covered in the ResultSet rather than RestDataSource docs because it's a general (and extremely powerful) behavior across all DataSources:

http://www.smartclient.com/smartgwtee/javadoc/com/smartgwt/client/data/ResultSet.html
http://www.smartclient.com/smartgwtee/javadoc/com/smartgwt/client/data/DSRequest.html#getStartRow%28%29

Hi, Charles; thanks for

Hi, Charles; thanks for chiming in.

Regarding the configurability of the format, are you referring to the XPath magic?  That is definitely something we make heavy use of.  I was remarking more on the apparent-to-this-newbie required <response>, <status>, <startRow>, <endRow>, <totalRows> and <data> elements.  All else being equal I wanted full control over what my returned payloads look like with a minimum of fuss.  Obviously for toolkits such as yours that support paging and caching elegantly, some version of these semantic elements will have to be present--no doubt--but we might refer to them by different names.

Cheers,
Laird

They aren't required

Those fields - status, startRow, etc - are required only if you explicitly choose to use RestDataSource. If you use just DataSource, no such fields are required. Further, SmartGWT can be trivially adapted to whatever terminology you want to use.

Best way to understand is to compare these two samples (sorry, they're SmartClient samples, but the equivalence should be obvious):

DataSource, no metadata fields (startRow et al) required in the responses:

http://www.smartclient.com/index.jsp#xmlEditSave

RestDataSource, metadata fields present:

http://www.smartclient.com/index.jsp#restEditSave

Then finally, a sample of using code to populate SmartClient's DSResponse based on an arbitrary wire format not dictated by SmartClient/Smart GWT:

http://www.smartclient.com/index.jsp#xmlServerValidationErrors

This last capability is how SmartClient can interact with arbitrary services with a completely different set of terminology. For example, if in a given service the equivalent of startRow and endRow is "offset" and "count", and they don't even mean the same thing as startRow and endRow (perhaps they are one-based, and "count" is the number of records returned, not the index of the last record returned) and this information is inside a SOAP header for a WSDL web service, that's not a problem, a simple XPath can extract the values and populate the DSResponse.startRow/endRow fields.

In a nutshell, if a given service has any support for paging, sorting, etc, with any set of terminology, Smart GWT can be quickly adapted to use it. Then all of the data binding behaviors built into Smart GWT's components instantly work, because they depend on the DataSource abstraction only.

XML mapper for GWT

Very interesting post. In my current project we also use a RESTful architecture and a GWT based frontend. Though the XML representations of our resources had a distinct format. Against this background I wrote a little XML mapper for GWT. Using this mapper you can annotate your GWT model and map distinct data from the XML by using XPath expressions.

Feel free to take a look at it under http://code.google.com/p/piriti/