Skip to main content

Screencast about a RESTful Comet application

Posted by caroljmcdonald on September 20, 2008 at 5:53 AM PDT

Here is a screencast about a Comet application which allows users to watch and chat about
a slide show. The Application loads the slide URLs from a RESTful Web Service and then uses
dojo bayeux with Grizzly on Glassfish to publish the slides and/or text to all the slideshow
subscribers.



You can read more about this application at
RESTful Web Services and Comet

Comments

So the main question I can't help think about is.. scalability. If I have potentially millions of users all hitting my server farm, they are all keeping long-lived connections open. As far as I know, this is a very resource intensive process.. something on the order of a couple 1000 connections tops, per server.. if that. So using something like Comet would mean, unless things have changed, I would probably need hundreds of servers to handle a large load of clients hitting my site because of the limited threads each server could handle to keep the connections open. The project I am working on has been considering a "push" model, but our thought was having the consumer of our REST api pass in one (or more) methods for which the REST API could push back data. For example, since most consumers of our API will be from web servers that would render the UI using the REST API responses, they could pass us there URL to there server, and when a long task was ready to push more data, using the URL supplied we would simply send the appropriate HTTP "request" to the supplied URL. As part of our API doc, we'd explain each protocol we supprot (initially http, but smtp, email, twitter, etc could be implemented) and what we would send back based. So I am curious how you handle large loads with regards to server farms and the usual scalability/clustering stuff.

Its probably not a good idea for some web applications, but there are scenarios that warrant it. For example we already do webex over the web to share presentations, this is useful in my opinion. This would be a good discussion to have at http://cometdaily.com/

Yea, but arguably, Comet is by definition an "abuse" of HTTP, which really isn't designed for long lived connections (regardless of how that connection is maintained on the server). One of the pretenses of the REST architecture (specifically REST using HTTP) is the embracing of the HTTP protocol. Which mean not just leveraging HTTP method verbs, headers and result codes, but also the scalability of a connectionless, stateless protocol augmented by characteristics of the requests that make them suitable for things like caching (e.g. GET being idempotent). While, in essence, HTTP uses sockets, and a socket is a socket is a socket, so why not have long lived connections, the truth is clearly that vast majority of HTTP traffic isn't like that and, more specifically, the interleaving infrastructure isn't designed to handle massive amounts of long term connections. All of a sudden, those high speed, interleaving proxies are saturated with connections, and useless. So, while, perhaps, you can do "Comet over REST", I don't really if it's in fact actually a good idea.

Grizzly is an HTTP framework that uses the Java NIO API to provide fast HTTP processing. Grizzly provides long-lived streaming HTTP connections that can be used by Comet, with support built on top of Grizzly's Asynchronous Request Processing (ARP). With Grizzly ARP, each Comet request doesn't monopolize a thread, and the availability of threads permits scalability. you can ask more questions about this in the Glassfish forum http://forums.java.net/jive/forum.jspa?forumID=56&start=0 Jean-Francois implemented comet in grizzly so his blog is also a good place to look for more info http://weblogs.java.net/blog/jfarcand/ also you can read the grizzly comet doc here http://docs.sun.com/app/docs/doc/820-4496/ggrgt?a=view Jean-Francois and Paul Sandoz said something about wanting to get comet support working with Jersey, but I don't know where that is on the priority list.

I think you both make valid points. My thought on this was the discussion I read about some months ago regarding using ajax to keep connections open, and have the server push updates so that the web client could update the UI, rather than have a polling method on the web side constantly hitting the server.