Jean-Francois Arcand's Blog

New Adventures in Comet: Throttling the Server push

Web Applications that uses Comet Request processing are more an more used with asynchronous clients (AJAX, SOA, GWT, etc.). At AjaxWorld this week, I was surprised to see how many developers attended the two scheduled Comet talks and how many companies already have Comet based application in production. But like with every new technologies, no formal design recommendations have been widely described and discussed (unless I'm missing something :-)). Hence two Comet applications doing essentially the same thing might significantly differ in their design. Well, let me share what I've learned over the last year, having see multiples real Comet based applications. Oh no! Another series of blogs from me!

So far, the biggest problem I've seen is related to the frequency data are pushed to clients. When a server push data back to the client, it can easily flood the subscribed clients by sending too many data in a short period of times. To demonstrate the problem, let use the most famous Comet application after the Chat which is the real time stock option application. As an example, take a look at the Lighstreamer demo. In that demo, stock options are updated real time. I don't know the technical details about the application, but let's assume it use Comet. Depending on the source you use, stock options value can be updated very frequently, mostly every second. Thus if the server have to push data every seconds, it is easy to understand how bad your application will scale if 10 000 Comet clients are connected, waiting for a server push. In that situation, you need a way to throttle the stock options updates so the subscribed Comet clients aren't getting flooded by server pushes. There is two solutions:

• On the client side, you decide to read ant update the page, let's say every 5 seconds, and discard all other pushes from the server or
• throttle the pushes on the server side.

Most of the time, I would recommend the latter approach as pushing data over a connection is not free, not to say that if 10 000 users are connected, updating them with data they will not use might eat all both client and server CPU. I know this is trivial, but still scalability problems might be visible only when your application goes into production, which is unfortunately late in the development cycle. Now the big question is how to do that, because this is not as simple as it looks. I cannot speak for other Comet implementation, but in Grizzly Comet, a database (assuming the stock options data are stored/updated there) will invoke the CometContext.notify() method, passing the updated data. If we want to throttle the data to avoid flooding the Comet clients, the logic will have to happens on the database side. Well, I don't like that approach for two reasons:

• First, if your application uses several sources (a database, a web service, etc.) to push data, it will means all those sources will have to implement the 'throttling logic'.
• Second, if one of your Comet client start flooding your server with data, the 'throttling logic' will have to be implemented either on the client or server side.

Naaa...I don't like that because in my opinion, the 'throttling logic' should be part of the Comet implementation API itself. Fortunately for GlassFish users using Grizzly Comet, there is an API you can use to customize the 'throttling logic':

public interface NotificationHandler {

    /**
     * Return true if the invoker of notify() should block when
     * notifying Comet Handlers.
     */
    public boolean isBlockingNotification();

    /**
     * Set to true if the invoker of notify() should block when
     * notifying Comet Handlers.
     */
    public void setBlockingNotification(boolean blockingNotification);


    /**
     * Notify all CometHandler.
     * @param cometEvent the CometEvent used to notify CometHandler
     * @param iteratorHandlers An iterator over a list of CometHandler
     */
    public void notify(CometEvent cometEvent,Iterator<CometHandler> iteratorHandlers)
        throws IOException;


    /**
     * Notify a single CometHandler
     * @param cometEvent the CometEvent used to notify CometHandler
     * @param iteratorHandlers An iterator over a list of CometHandler
     */
    public void notify(CometEvent cometEvent,CometHandler cometHandler) 

An implementation of this interface will most of the time contains the logic to reduce server pushes, which resources to use when updating clients, etc. Your application just have to call CometContext.setNotificationHandler(myHandler)...that's it! Thus, when writing (or porting) a Comet application, you can always customize (or re-use the available ones) NotificationHandler and make sure that when under load, neither clients nor the server flood the subscribed clients when invoking CometContext.notify(). Better, you might develop with an un-throttled NotificationHandler and latter change with one that only push data every 10 seconds, only once you are production ready.

To recap, when writing (or porting) a Comet application, you must make sure a server push doesn't bring down the server because the server is pushing data too frequently, flooding clients and the server itself. Being able to customize the way data are pushed back to client is something I consider extremely important. Under load, your customized NotificationHandler can make a huge difference and make browser experience much better.

Next time I will discuss another commons problem, which is when a client start sending tons of data to the server. Should the data be throttled on the client or server side?

technorati: grizzly comet cometd ajax/a> glassfish

Comments

• I downloaded Glassfish build 40 yesterday, and the cometd-example.war -- it just doesn't work. I sometimes get weird messages like the following:

  DEBUG: [SyntaxError: syntax error, file: http://www.sheetbase.com/cometd/dojo/src/bootstrap1.js, line: 274]
  DEBUG:

  and at other times, I do not get an error message but the chat application does not show any messages.

  Could you please retest it from a clean build and see if the install still works? Somehow, there is no other documentation other than the snippets in your blog so it is difficult for me to find out what is wrong. I would very much appreciate a reverification that it's all working.

  And, actually, how about a simpler getting-started example -- where the server simply pushes the current time? I would be very happy to write it if you could guide me. The chat application is a bit difficult to understand for someone new to comet because the protocol has three different types of messages and their handling using JSON is not intuitive to somebody not well-versed in Dojo.

  Any help is much appreciated.

  Posted by: anjul on March 25, 2007 at 12:42 PM

• sorry, all my line breaks in the previous post disappeared...

  Posted by: anjul on March 25, 2007 at 12:43 PM

• Hi Jean-Francois,
  It was a pleasure meeting you at AjaxWorld.
  Data throttling is certainly a very important aspect for Comet solutions, especially in financial applications. For example Lightstreamer implements several mechanism to throttle data:
  - Bandwidth Control: a maximum bandwidth can be allocated for each user.
  - Frequency Control: each single subscription can request a maximum update frequency.
  - Multi-Stage Filtering: A pre-filter is available to preliminarily sample the data on a global basis.
  - Adaptive Streaming: The server automatically detects Internet congestions and heuristically throttles the data flow based on the available bandwidth.
  Furthermore it is important to aggregate the data efficiently within TCP packets, to manage the trade-off between latency and overhead reduction.

  Cheers
  -Alessandro Alinone
  

  Posted by: alinone on March 29, 2007 at 01:47 AM

• Hi Alessandro! Thanks for the information! For people that doesn't know what is Lightstreamer, just click here. A very nice Comet server!

  Posted by: jfarcand on March 30, 2007 at 08:06 AM

• Hi Anjul. Just look at my other blogs on Comet. I'm always adding links to examples. For your current problem, which browser are you using? For me the example works fine. Thanks

  Posted by: jfarcand on March 30, 2007 at 08:07 AM

• google鲜花鲜花园林公司花卉租摆园艺绿化工程股票花木租摆礼仪公司交通园林公司园林工程luhua18分类目录绿化工程绿化施工花卉知识

  Posted by: goodscape on April 29, 2007 at 05:34 PM

• 那家[url=http://www.21gctt.com][b]翻译公司[/b][/url]的[url=http://www.21gctt.com][b]翻译

  [/b][/url]比较好啊[url=http://www.jituo.net]？[/url]谢谢

  Posted by: erxiao on May 08, 2007 at 01:50 AM

• 那家翻译公司的翻译比较好啊?谢

  谢

  Posted by: erxiao on May 08, 2007 at 01:51 AM

This work is licensed under a Creative Commons License.