Jean-Francois Arcand's Blog

gCometd: Introducing the Cometd framework and its Bayeux protocol support in Grizzly

The Cometd framework and its Bayeux protocol is now supported starting with Grizzly 1.0.11 and GlassFish 9.1 b35. In this first blog, I will talk about the gCometd implementation, how to enable it and describe a DOJO cometd enabled example.

The Cometd framework is defined as:

Cometd is a scalable HTTP-based event routing bus that uses a push technology pattern
known as Comet. The term 'Comet' was coined by Alex Russell in his post 
'Comet: Low Latency Data for the Browser'. The letter d is for 'Daemon', as described 
by Answers.com, Daemon is a program or process that sits idly in the background 
until it is invoked to perform its task. Cometd consists of a protocol spec called 
Bayeux,javacript libraries (dojo toolkit), and an event server.

I don't want to go over the details of what is the Bayeux protocol, as they already exists excellent descriptions of it here, here and here. But in short (from Alex Russel description):

Bayeux is a JSON-based protocol for clients to register interest in events 
and for servers to deliver them in a more timely fashion than Ajax-based polling
 allows. The goals of the Bayeux spec so far have been to:

    * make event delivery fast
    * keep it simple
    * provide extension points in the protocol

What I find interesting with Cometd and Bayeux is the fact that if an application is build using only DOJO (or Ajax component), the application can be deployed in any server that supports the Bayeux protocol. This looks like very promising!. Since Grizzly already supports Comet via its Asynchronous Request Processing(ARP) extension, I've decided to read the spec and try to see if it was possible to implements it. Well, I know it was possible since Jonas Jacobi and John Fallows did an extremely popular talk at the last Javapolis using Grizzly ARP. I didn't see their implementation and decided to design one I can use with GlassFish allowing JSF, JSP and Servlet to use cometd, and without by using the Grizzly tiny WebServer allowing only AJAX/DOJO component to use cometd. I've called the implementation gCometd.

Before starting, I've shared the idea of adding native support for the Bayeux protocol with Takai Naoto (the man behind JRuby on Grizzly (newly called gRuby ;-)). By the time I told him, he also comes with an implementation. Wow! kudo for the Bayeux spec folks because it seems not too difficult to implement! The final result is a merge of the two implementations. Can't wait to see gCometd in action? The next couple of paragraphs will describe how to enable it.

Enabling gCometd in Grizzly

First, download Grizzly 1.0.11 (and up) from here. Next, just do:

 % java \
-Dcom.sun.grizzly.adapterClass=com.sun.grizzly.cometd.standalone.CometdAdapter \
-Dcom.sun.enterprise.web.connector.grizzly.enableCometSupport=true \
-jar grizzly-1.0.11.jar <port> <cometd folder>

port: The port on which Grizzly will listen.
comed folder: The folder from which file will be serviced.

That's it. This is really useful when developing DOJO cometd application as it starts extremely fast. If you aren't using JSF/JSP/Servlet, this is the implementation you want to use!. Event better, I will soon show how to debug a DOJO/cometd application using the Netbeans Phobos module. Et oui, you read it correctly: using the Phobos module (which under the hood uses this Grizzly), you will be able to debug your DOJO/JavaScript application with the Netbeans debugger!

Enabling gCometd in GlassFish

First, download GlassFish b35 from here. Install GlassFish and then follow the complete instruction described here.

You just need to add the bold line in domain.xml:

        <http-listener acceptor-threads="1" address="0.0.0.0" 
           blocking-enabled="false" default-virtual-server="server"
           enabled="true" family="inet" id="http-listener-1" port="8080"
           security-enabled="false" server-name="" xpowered-by="true">
                <property name="cometSupport" value="true"/>
        </http-listener>

Next, add to your application web.xml the following:

    <servlet>
        <servlet-name>Grizzly Cometd Servlet</servlet-name>
        <servlet-class>
            com.sun.grizzly.cometd.servlet.CometdServlet
        </servlet-class>
        <init-param>
            <description>
                expirationDelay is the long delay before a long polled request 
                is resumed. Default is 60000.
            </description>
            <param-name>expirationDelay</param-name>
            <param-value>60000</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>Grizzly Cometd Servlet</servlet-name>
        <url-pattern>/cometd/*</url-pattern>
    </servlet-mapping>

Package your war and deploy it. Now every requests sent to your war's context-path/cometd will be serviced by the gCometd runtime.

DOJO cometd example.

To see the gCometd in action, you can download a sample here. This sample is an adapted Chat application available in Jetty, written by Greg Wilkins. Greg Wilkins is a member of the Cometd specs and the creator of Jetty. Jetty was the first cometd implementation and I've decided to adapt its cometd example for the purpose of this blog.

For Grizzly, just unzip the war file under the cometd folder. For GlassFish, just deploy the application. Then start your browser and point to the context-root of the sample ("/index.html" for Grizzly, "cometd/" for GlassFish). An excellent tool to use to see what DOJO's Cometd client is sending to gCometd is Firebug. I strongly recommend you install it. It really help understanding what is Bayeux! Under the hood, the DOJO cometd component does (a brief overview):

• When index.html is loaded, DOJO does:

  <script type="text/javascript"> cometd.init({}, "/cometd-example/cometd"); </script>

  This initiate a handshake with the gCometd runtime (using Bayeux)
• Next, when you enter a user name, DOJO client will exchange message with gCometd. The interesting part here is one of the DOJO connection will be polled, waiting for web event (push from the server). This is achieved by doing in a JavaScript function:

         cometd.subscribe("/chat/demo", false, room, "_chat");
         cometd.publish("/chat/demo", { user: room._username, join: true, chat : room._username+" has joined"});

• Every time a message is typed, the browser will execute:

  cometd.publish("/chat/demo", { user: room._username, chat: text});

  On the gCometd side, the message will be pushed back to all cometd clients that have subscribed to the chat channel using a long polled connection.
• Finally, to leave the chat, the browser will send:

  cometd.unsubscribe("/chat/demo", false, room, "_chat");

OK enough wording...just try it :-). In the upcoming weeks, I will start a series of blogs trying to give more details on how to write a cometd application. I'm also investigating how to make the gCometd implementation container agnostic, so it can be used in other Java container that support Comet/Continuation/Long polling. This way I can write my gCometd application and use it in Jezzly. But until them, just try it and give feedbacksssss!

technorati: grizzly comet cometd ajax glassfish

Comments

• Hi,

  We are very happy and excited to know about this... We tried the example and it worked reasonably.

  As a Company, we are in the process of developing a high end solution for the Financial Market (in the Portfolio Management sector).

  Can you give us some leads to references / examples with source codes about how cometd can be used... I have downloaded and installed the dojo toolkit and also configured glassfish build 35...

  Any working sample programs will go a long way in cutting short the time that we would need to create and implement something interesting.

  Thanks in advance... We look forward to your response.

  Regards,
  Sajit

  Posted by: sd_sdtpl on February 09, 2007 at 01:23 AM

• Hi - 1.0.11 doesn't exist on https://maven-repository.dev.java.net/repository/grizzly/jars/ (it only goes up to 1.0.10)

  It'll be good to read your progress on cometd - keep it up!

  Posted by: ribbon on February 09, 2007 at 06:15 PM

• I am looking out for the source code of working examples... Any advice / directions / leads to such sources will be highly appreciated.

  Regards,
  Sajit

  Posted by: sd_sdtpl on February 11, 2007 at 02:48 AM

• Hi, it seems we have sync problem with java.net maven repository. I will ping th admin. Stay tuned!

  Posted by: jfarcand on February 12, 2007 at 06:36 AM

• Hi, we are working on a set of examples. Should blog about it soon. Stay tuned!

  Posted by: jfarcand on February 12, 2007 at 07:32 AM

• Still working with folks internally to figure out why the site isn't updated. I've uploaded version 1.0.11 here

  Posted by: jfarcand on February 12, 2007 at 07:45 AM

• OK version 1.0.11 is now available on the Maven repo.

  Posted by: jfarcand on February 12, 2007 at 07:59 AM

• Hi Jean,

  Thanks for your response... I have downloaded 1.0.11 but have not been able to make much progress, since I am not clear about how to proceed using grizzly.

  Can you give me some documentation or examples which will help me push some dynamic and custom content from the server-side. Most of the examples talk about client forwarding some content to the server which in turn forwards the same content to all the clients subscribing to the same channel (like how it is in a typical chat environment).

  I am trying to simulate where clients subscribe to a specific channel (using dojo.io.cometd) being provided by the server and than the server publishing (using the grizzly server side implementation of cometd) information to all the subscribing clients.

  Without source examples or documentation, I am getting stuck... Please help.

  I look forward to the examples that you have mentioned in your post above. Meanwhile any pointers / leads will help me greatly.

  Thanks again... and we highly appreciate your work and blogs... they are highly informative and helpful... keep it coming... :-).

  Regards, Sajit

  Posted by: sd_sdtpl on February 12, 2007 at 11:09 PM

• Hi, in order to achieve what you want, you need to implement at least one server side component that will push data to the client. This is doable by writing a CometHandler[1] registered on the same CometContext as the Comed context. In GlassFish, it means /cometd. I'm giving a talk on the topic next AjaxWorld 2007 in March and will try to push an example here. Unfortunately, I can't guarantee when I will have the time to do it.Feel free to email me for specific questions[1] http://weblogs.java.net/blog/jfarcand/archive/2006/10/writting_a_come.html

  Posted by: jfarcand on February 14, 2007 at 12:35 PM

• Hi Jean,

  i'm in the process of evaluating the Comet/Cometd implementation of grizzly.
  I successfully wrote me a "server" that publishes data to the grizzly cometd implementation.
  After that i connected some clients that get invoked when new data is available. All done with HTTP and the Bayeux Protocol.

  It seems to be regardless to which channel a client is subscribed. He always get invoked when new
  data is available. E.g. the client subscribes with the following bayeux message:
  [{"channel":"/meta/subscribe","subscription":"/test/one","connectionId":"/meta/connections/75593851d23664e6","clientId":"75593851d23664e6"}]

  The server pushes the following data:
  [{"data":{"user":"Marco","chat":"Hello all"},"channel":"/sample/demo","connectionId":"/meta/connections/54debf6fb9750fad","clientId":"54debf6fb9750fad"}]

  The client that subscribed to the channel "test/one" gets the data for the channel "sample/demo".
  Any ideas about that?
  Thanks Marco

  Posted by: marco_balzter on April 10, 2007 at 07:01 AM

• FYI: The answer and discussion can be followed on users@grizzly.dev.java.net.

  Posted by: jfarcand on April 12, 2007 at 04:48 PM

• hello, I am very glad to find an example using cometd (with the complete source). does it work also with tomcat 6 ?

  Posted by: bernada on June 14, 2007 at 05:03 PM

• Hi, Tomcat 6 doesn't support the bayeux protocol. I'm sure they will eventually supports it, but for now only Jetty, Grizlzy and GlassFish supports it :-)

  Posted by: jfarcand on June 14, 2007 at 05:11 PM

• Hi, thanks for your answer about tomcat 6.
  I've tested your chat sample on grizzly it works fine, but on glassfish b41d I didn't succeed; as soon as I put the
  the http connector doesn't work any more, (no response written). what is wrong ???

  Posted by: bernada on June 28, 2007 at 02:45 PM

• as soon as I put the property CometSupport...

  Posted by: bernada on June 28, 2007 at 02:46 PM

• Hi, edit glassfish.home/domains/domain1/config/domain.xml and remove line <property ="proxiedProtocols" value=:ws/tcp"/> as b41d seems to be broken. If you can, use the latest build (b52) to use the latest bayeux implementation.

  Posted by: jfarcand on June 29, 2007 at 07:13 AM

• Hi, I am using Glassfish v2-b50g. I would have sent this in an email but I cannot find your email address.

  The greatest shortcoming of the Glassfish cometd implementation is
  that we cannot detect when users disconnect: when a client
  disconnects, BayeuxCometHandler#onDisconnect is never called. This is against one of the recommendataions from the Bayeux spec, in 4.4.1:

  > A Bayeux server SHOULD not rely solely on the client sending a disconnect message to remove client state information because a disconnect message might not be sent from the client or the disconnect request might not reach the server.

  According to 4.3.1 from the Bayeux spec, the server should have a
  timeout configured such that disconnections that occur without a
  reconnection within the timeout should be treated as disconnections.
  This is totally not implemented in Glassfish cometd at all.The comment in BayeuxCometHandler#onInterrupt in b50g is misleading, because in CometdServlet, the BayeuxCometHandler is added to the CometContext with completeExecution = true, and instances of DataHandler (rather than BayeuxCometHandler) are associated with SelectionKeys for client connections in CometContext. In other words, the BayeuxCometHandler#onInterrupt method will never be called on client disconnect in any Cometd servlets that follow any reasonable implementation encouraged by Glassfish; only DataHandler#onInterrupt will be called, and this method in b50g contains zero lines of code.

  This is a huge problem, since users can just close their browsers and we will never know. I have only studied the code for a total of 4 hours, but it seems not too hard to fix this in the Glassfish implementation, and helping Glassfish with their Cometd implementation might be easier than rolling our own framework on top of Comet (on the other hand, it might not be; this isn't completely clear to me). But, at least I should ask the question: how can I contribute to the Glassfish Cometd implementation?

  According to section 4.2.2. of the spec, "Once successfullyconnected, a client is implicitly subscribed to the channel "/meta/connections/CLIENTID"." I was looking forward to using these
  client-specific channels to send messages to particular clients. Yet,
  looking at the CometdNotificationHandler, it seems this isn't
  implemented. This makes it very difficult for us to send messages to
  specific clients, at least without breaking out of cometd. It is really a pity, because this problem is super-easy to fix; DataHandler#clientID would have to be set and the loop in CometdNotificationHandler#notify0 could just check against the
  clientID variable in the DataHandlers it is looping through.

  Another problem that is not quite as serious: the Glassfish cometd
  implementation provides no easy means for client-to-server or
  server-to-client communication. Any Cometd servlet implementation
  written with the current EventRouterImpl and BayeuxCometHandler
  classes will only be able to have client-to-client communication (via
  the server). I had to subclass BayeuxCometHandler to insert
  application-specific behavior for onData, which is not an elegant
  solution. We should be able to easily process incoming messages and generate new outgoing ones; BayeuxCometHandler should allow this to be done through delegation so users don't have to resort to inheritance.

  So, please contact me and let me know how I can help to improve the cometd implementation in Glassfish.

  Thank you,
  Adam

  Posted by: aduston on July 18, 2007 at 03:39 PM

• Hi Adam, the gCometd implementation is still not uptodate with the spec, but I've made some improvement after b50g. But your points are still valid...drop me an email at: jeanfrancois.arcand at sun.com or on the users@glassfisg.dev.java.net so we can discuss the current issues and see where you can help. Many thanks for the feedback! -- Jeanfrancois

  Posted by: jfarcand on July 19, 2007 at 06:02 AM

• Hi, using this example with Glassfish v2-b58g I get a NPE (at com.sun.grizzly.cometd.bayeux.Handshake.toJSON(Handshake.java:198)). The problem is that when the Handshake object was created (method VerbUtils.newHandshake()), its ext field is null, so an exception y thrown because handshake.toJSON uses that field. The map passed to the method newHandshake is : [{"version":0.1,"minimumVersion":0.1,"channel":"/meta/handshake"}]

  Posted by: lacayo on September 22, 2007 at 06:06 AM

• ... ¿Or may the problem be the message [{"version":0.1,"minimumVersion":0.1,"channel":"/meta/handshake"}], which should include more information so VerbUtils.configureExt() create an ext field? If that is the case, ¿why doesn't the message contains that information? I tried to post an issue in grizzly's project issue tracker, but I don't know the version included in Glassfish v2-b58g, as grizzly's (and cometd) classes are packed in appser-rt.jar. Thanks.

  Posted by: lacayo on September 22, 2007 at 07:27 AM

• Unfortunately, the version shipped with GlassFish v2 is not up to date with the latest Bayeux spec. I'm working to have the implementation ready for v2 ur1, which is coming end of October. Which client are you using? Please file a bug under Glassfish.dev.java.net, version 9.1 ur1. Any patches accepted :-) I've a hard time right now to follow the cometd stuff and evolve Grizzly Comet (see grizzlet. The bayeux implementation in Grizzly workspace is the same as GlassFish BTW, except it is based on Grizzly 1.6 instead of 1.0. Thanks for the feedback! - Jeanfrancois

  Posted by: jfarcand on September 22, 2007 at 01:06 PM

• Hello Jeanfrancois,

  i have some problems with the chat example and grizzly.

  I always get a NullPointerExc after a browser access on the chat:

  04.11.2007 21:20:48 com.sun.grizzly.http.DefaultProcessorTask invokeAdapter
  SCHWERWIEGEND: processorTask.serviceError
  java.lang.NullPointerException
  at com.sun.grizzly.cometd.bayeux.Handshake.toJSON(Handshake.java:202)
  at com.sun.grizzly.cometd.BayeuxCometHandler.onHandshake(BayeuxCometHandler.java:85)
  at com.sun.grizzly.cometd.BayeuxCometHandlerBase.onEvent(BayeuxCometHandlerBase.java:46)
  at com.sun.grizzly.comet.DefaultNotificationHandler.notify0(DefaultNotificationHandler.java:183)
  at com.sun.grizzly.comet.DefaultNotificationHandler.notify(DefaultNotificationHandler.java:136)
  at com.sun.grizzly.comet.CometContext.notify(CometContext.java:378)
  at com.sun.grizzly.cometd.EventRouterImpl.route(EventRouterImpl.java:80)
  at com.sun.grizzly.cometd.standalone.CometdAdapter.service(CometdAdapter.java:229)
  at com.sun.grizzly.http.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:599)
  at com.sun.grizzly.comet.CometEngine.executeServlet(CometEngine.java:494)
  at com.sun.grizzly.comet.CometEngine.handle(CometEngine.java:286)
  at com.sun.grizzly.comet.CometAsyncFilter.doFilter(CometAsyncFilter.java:74)
  at com.sun.grizzly.arp.DefaultAsyncExecutor.invokeFilters(DefaultAsyncExecutor.java:162)
  at com.sun.grizzly.arp.DefaultAsyncExecutor.interrupt(DefaultAsyncExecutor.java:140)
  at com.sun.grizzly.arp.AsyncProcessorTask.doTask(AsyncProcessorTask.java:79)
  at com.sun.grizzly.http.TaskBase.call(TaskBase.java:346)
  at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:179)

  
  Do you know this error or do you have a solution ?

  Thanks Christian

  Posted by: keineahnung on November 04, 2007 at 12:26 PM

• Which version of GlassFish are you using? The current bayeux implementation is behind the spec in GlassFish. I will take a look. Many thanks!

  Posted by: jfarcand on November 05, 2007 at 07:55 PM

• OK GlassFish 9.1ur1 has an updated Bayeux 1.0 implementation :-) I will blog about it soon with an example. Thanks!!!

  Posted by: jfarcand on November 06, 2007 at 07:24 PM

• I updated my grizzly trunk to the current version.
  It works very fine now. No exceptions on serverside and clientside.

  Next i want to create a class, which streams data to specified channels.
  This class should not implement a own bayeux client. How can use Grizzly methods, to send modified datas only from a java-client to the eventrouter (server)?

  I am testing with BayeuxCometHandler class at the moment . There are methods onData or onSubscribtion, which I can use. But this doesn't work very well.

  Thank you

  Posted by: keineahnung on November 12, 2007 at 11:26 AM

• Great. Sorry I'm at ApacheConf with limited time to read my emails :-) Do you mind if we discuss using the users@grizzly.dev.java.net? Wll be easier. Can you elaborate more on what you want to do on the client side? Thanks!!

  Posted by: jfarcand on November 13, 2007 at 12:40 PM

This work is licensed under a Creative Commons License.