Skip to main content

Distributed (JMS) messaging applications using GlassFish and MantaRay

Posted by rampsarathy on January 8, 2007 at 3:37 AM PST



Distributed (JMS) messaging applications<br /> using GlassFish and MantaRay


name="mozTocId236768">Distributed (JMS) messaging applications
using GlassFish and MantaRay





   

Generic JMS RA (1.7) that is bundled with GlassFish V2 enables
applications deployed on GlassFish to use almost any JMS 1.1 compliant
message broker. This article talks about how applications (typically
Message Driven Beans) deployed in a GlassFish  cluster can use
MantaRay peer to peer architecture for 
their JMS messaging needs using
Generic resource adapter for JMS.

MantaRay is an open source enterprise-grade messaging middleware
solution that is based on a peer-to-peer architecture. With its
peer-to-peer topology, MantaRay eliminates the need for costly central
messaging brokers.





picture1.png



               
              
           

style="font-weight: bold;">      
              
              
            Figure 1: GlassFish
and MantaRay



The steps/configuration mentioned below are based on the following
versions :



Project GlassFish V2 Build 30

https://glassfish.dev.java.net/downloads/v2-b30.html



MantaRay Java 2.0.1



Generic JMS RA 1.7

(Bundled with GlassFish, under
GLASSFISH_HOME/lib/addons/resourceadapters/genericjmsra/genericra.rar)
OR

href="https://genericjmsra.dev.java.net/files/documents/3308/43334/genericra.rar">https://genericjmsra.dev.java.net/files/documents/3308/43334/genericra.rar

Initial
Setup

  • Refer to MantaRay documentation for installing MantaRay.
    Typically it involves extracting the zip file into a folder, which
    would be denoted  as MANTA_HOME in this document. 

    •           No changes are required to
    the configuration files for a basic setup.

             

           For the purpose of this
    document MantaRay  is installed under MANTA_HOME, GlassFish V2 is
    installed under GLASSFISH_HOME.   


      Install GlassFish application server and create a cluster, the
      domain that is used for this cluster creation should have been created
      using the cluster profile, the
      following steps briefly describe the tasks for 3 instance cluster
      creation.

             

            # NODE-AGENT CREATION

           

            GLASSFISH_HOME/bin/asadmin
    create-node-agent --user <adminUser>  --passwordfile
    <passwordFileLocation> --savemasterpassword=true agent1



            #START NODE-AGENT

            GLASSFISH_HOME/bin/asadmin
    start-node-agent --user  <adminUser> --passwordfile
    <passwordFileLocation> agent1

       

            # BEGIN CLUSTER CREATION

            GLASSFISH_HOME/bin/asadmin
    create-cluster --port <domainAdminPort> --user <adminUser>
    --passwordfile <passwordFileLocation> --autohadb=false
               
        cluster1



            # CREATE INSTANCE1



            GLASSFISH_HOME/bin/asadmin
    create-instance --port <domainAdminPort> --user <adminUser>
    --passwordfile <passwordFileLocation> --nodeagent ra-agent
            --cluster cluster1
    --systemproperties HTTP_LISTENER_PORT=1111:IIOP_LISTENER_PORT=33700

    instance1



            # CREATE INSTANCE2

           GLASSFISH_HOME/bin/asadmin
    create-instance --user admin --port <domainAdminPort>
    --passwordfile <passwordFileLocation> --nodeagent agent1
    --cluster         cluster1
    --systemproperties HTTP_LISTENER_PORT=1112:IIOP_LISTENER_PORT=33701
    instance2



            # CREATE INSTANCE3

            GLASSFISH_HOME/bin/asadmin
    create-instance --user admin --port <domainAdminPort>
    --passwordfile <passwordFileLocation> --nodeagent agent1
    --cluster         cluster1
    --systemproperties HTTP_LISTENER_PORT=1113:IIOP_LISTENER_PORT=33702
    instance3




    • Modify the cluster1's classpath to add MantaRay  jars
      located in MANTA_HOME folder.  The asadmin GUI could
      be used to
      modify a domain's classpath. Open a browser and type the url of the
      application server admin GUI - https://hostname:adminport. Go to
      Configuration -> cluster1-config -> JVM Settings -> Path
      Settings . Add
      an
      entry for the jar files shown below in
      the classpath suffix. 
    MANTA_HOME/ext/antlr.jar   


    MANTA_HOME/manta.jar

    MANTA_HOME/ext/clink-1.5.0.jar

    MANTA_HOME/ext/commons-logging.jar

    MANTA_HOME/ext/log4j-1.2.8.jar


    • Modify the cluster1's JVM option to add MantaRay configuration
      file as a  JVM propert. The asadmin GUI could
      be used to do this. Open a browser and type the url of the
      application server admin GUI - https://hostname:adminport. Go to
      Configuration -> cluster1-config -> JVM Settings -> JVM
      options .

           
    -DmantaConfig=MANTA_HOME/config/default_config.xml   
      

    • Edit the server.policy file in the
      [GLASSFISH_HOME/domains/<domain>/config/] directory using your
      favourite text editor and add the following line to the default grant
      block.

       
           
    permission java.util.logging.LoggingPermission "control";

           
        permission
    java.util.PropertyPermission "*",
    "read,write";              




    Should you use an application client in your application,
    edit
    the application client's client.policy file in the
    [GLASSFISH_HOME/lib/appclient/client.policy] directory and
    add the following line to it.

                   
    permission javax.security.auth.PrivateCredentialPermission
    "javax.resource.spi.security.PasswordCredential * \"*\"","read";



    • Create a File system JNDI object store to bind ManataRay JMS
      administered objects.  The following code snippet creates a FS
      object store and binds the required MantaRay objects to the jndi tree.


    import javax.naming.*;

    import java.util.Properties;



    public class Main {

       

        public Main() {

        }

       

        public static void main(String[] args) {

        try {

               
    Properties props = new Properties();

               
    props.put("java.naming.factory.initial",
    "com.sun.jndi.fscontext.RefFSContextFactory");

               
    props.put("java.naming.provider.url",
    "file:/<objectStoreFolderName>");

               
    InitialContext jndiContext = new InitialContext(props);

               
    MantaQueue mq = new MantaQueue("SampleQueue1");

               
    MantaQueue outmq = new
    MantaQueue("SampleQueue2");          


               
    MantaTopic top = new MantaTopic("SampleTopic");

               
    MantaXAConnectionFactory confac = new
    MantaXAConnectionFactory();           


               
    jndiContext.rebind("SampleQueue1", mq);

               
    jndiContext.rebind("SampleQueue2",
    outmq);                       


               
    jndiContext.rebind("SampleTopic",
    top);           


               
    jndiContext.rebind("mantaxaconnectionfactory", confac);

               

            } catch (Exception e) {

               
    System.out.println("Could not create JNDI " + "context: " +
    e.toString());

               
    System.exit(1);

            }

                }

       

                }





    Configuring GlassFish
    cluster and  MantaRay



    picture.png




           
             
    Figure 2 : Configuration



       

        The above configuration describes a simple MDB
    application that is configured to listen to a MantaRay topic
    SampleTopic, and when it receives a message from the topic  it
    sends the message to a queue SampleQueue2.

      MantaRay is a messaging middleware solution
    based on peer-peer architecture. MantaRay's support for JMS1.1 API has
    been exploited to achieve the above configuration. Applications
    deployed on GlassFish clusters (or instances) can use MantaRay's
    peer-peer messaging capabilities through standard JMS apis using the
    Generic JMS Resource Adapter. It supports both publish/subscribe and
    point-point messaging models. Please refer to MantaRay documentation
    for more details.



    JNDI object store : MantaRay
    administered objects are bound to a file system object store and
    generic JMS ra is configured to use this object store. Other JNDI
    stores (LDAP..) can also be used but their configuration is beyond the
    scope of this document. A simple standalone program (mentioned in the
    Initial Setup section) can be used to bind MantaRay administered
    objects to the JNDI tree.



    MantaRay configuration file :
    The default configuration file -MANTA_HOME/config/default_config.xml is
    used by all the MantaRay peers, since the default MantaRay
    configuration file is designed to be used by multiple peers there are
    no inherent port conflicts. The MantaRay peers would use ports in the
    range of 6600 to 6700 (default_config.xml). If you feel the need for
    configuring specific ports , then a separate configuration file has to
    be created for every peer. A typical use case for this would arise when
    you want to use the MantaRay management console, each peer would then
    require a unique RMI port.



    GlassFish Configuration :A
    GlassFish cluster is created with 3 (or N ) instances. The cluster
    configuration is modified to include MantaRay client jars in the
    classpath and also a JVM property is added for mantaConfig file. When
    an application (MDB) is deployed on the cluster it is deployed on all
    instances that belong to the cluster and also presents a homogeneous
    view to the user.  Every instance of the application in the
    cluster creates one MantaRay peer to consume or produce messages. The
    MDB uses the standard JMS 1.1 APIs to send/receive messages.  It
    also uses generic JMS resource adapter which provides connection
    pooling for the application.





    Note: Restart the cluster after completing all the changes.



    # Stopping the node agent stops all the instances in the cluster also.

    GLASSFISH_HOME/bin/asadmin
    stop-node-agent --user  <adminUser> --passwordfile
    <passwordFileLocation> agent1



    # Start the node agent with sync instances set to true, this would
    start the cluster with the updated configuration.

    GLASSFISH_HOME/bin/asadmin
    start-node-agent --user  <adminUser> --passwordfile
    <passwordFileLocation> --syncinstances=true agent1

    Configuring
    the Resource Adapter

    • Add ${appserver-install-dir}/bin to your PATH. The asadmin
      CLI
      command can be found at ${appserver-install-dir}/bin. In GlassFish a
      resource adapter configuration is used to specify the configuration of
      a resource adapter. Use the following command to create a resource
      adapter configuration for genericra, to configure it to work with
      MantaRay Java 2.0.1.
    asadmin
    create-resource-adapter-config --user <adminname>
    --password
    <admin password> --property
    SupportsXA=true:ProviderIntegrationMode=jndi:RMPolicy=OnePerPhysicalConnection:

    JndiProperties=java.naming.factory.initial\=com.sun.jndi.fscontext.RefFSContextFactory

    java.naming.provider.url\=file://space/mantarayobjects:LogLevel=FINEST
    genericra


    A brief description of the
    various properties used in the above command is explained below:

    SupportsXA

    Set the supports distributed transactions attribute to true. The level
    of transactional support the adapter provides -- none, local, or XA --
    depends on the capabilities of the Enterprise Information System [EIS]
    being adapted. If an adapter supports XA transactions and this
    attribute is XA, the application can use distributed transactions to
    coordinate the EIS resource with JDBC and JMS resources.



    ProviderIntegrationMode

    Set the integration mode as JNDI. Two integration modes exist in the
    Generic Resource Adapter for JMS. The JNDI mode allows the resource
    adapter to use the administered objects published in the message
    provider's JNDI provider to integrate with the message provider.



    JndiProperties

    This property [comma-separated list of name-value pairs] specifies JNDI
    provider properties to be used for
    connecting to the JMS provider's JNDI. In our case, we set it to the
    JNDI configuration specified earlier.



    RMPolicy

    Some XAResource implementations such as IBM MQ Series, relies
    on a
    Resource Manager per Physical Connection and this causes issues when
    there is inbound
    and
    outbound communication to the same queue manager in a single
    transaction (For example, an MDB sends  a response to a
    destination). When RMPolicy is set to OnePerPhysicalConnection, the
    XAResource
    wrapper implementation's isSameRM in Generic JMS RA would check if both
    the XAResources use the same physical connection, before delegating to
    the wrapped objects. Ensure that this attribute is set to
    "OnePerPhysicalConnection" if the application uses XA.

    Deploying
    the Resource adapter

    • Download the Generic RA bits from the project
      site. With Glassfish V2, Generic
      RA is available out-of-the-box with the application server and you
      could choose to use the bundle resource adapter as well in the step
      below.
    • Deploy the resource adapter using the asadmin deploy
      command, as shown below. In the image above, see Generic JMS
      RA deployed in the application server.

       
       
        $ asadmin deploy --user admin --password
    adminadmin
    <location of the generic resource adapter rar file>

          

    Creating
    Connection Factories and Administered Objects in GlassFish.

    In order to configure a JMS Connection Factory, using the Generic
    Resource Adapter for JMS, a Connector connection pool and resource
    needs to be created in the application server, as shown below. In the
    image above, see inpool [pointing to Generic JMS RA and QCF]
    and jms/XAConFac [for inpool] created in the application server.


    Connector
    connection pool creation

    #Creates a Connection Pool called inpool and points to
    mantaxaconnectionfactory

    asadmin create-connector-connection-pool -- raname genericra
    connectiondefinition javax.jms.QueueConnectionFactory
    --transactionsupport  XATransaction --property
    ConnectionFactoryJndiName=mantaxaconnectionfactory inpool


    Connector
    resource creation

    #Creates a connector resource named jms/XAConFac and
    binds this resource to JNDI for applications to use.

    asadmin create-connector-resource --poolname inpool jms/XAConFac


    Admin
    Objects

    For JMS Destination Resources, an administered object needs to be
    created. In the image above, see jms/SampleQueue2 [pointing to Generic
    JMS
    RA and SampleQueue2] created in the application server.

    #Creates a javax.jms.Queue Administered Object and binds it to
    application server's JNDI tree at
    jms/SampleQueue2 and points to SampleQueue2

    asadmin create-admin-object --raname genericra --restype
    javax.jms.Queue --property DestinationJndiName=SampleQueue2
    jms/SampleQueue2


    Component
    Deployment descriptors

    The deployment descriptors need to take into account the resource
    adapter and the connection resources that have been created. A sample
    sun-ejb-jar.xml for a Message Driven Bean that listens to a destination
    called SampleTopic in MantaRay, and publishes back reply messages
    to a
    destination resource named jms/SampleQueue2 is shown below.





    <?xml version="1.0" encoding="UTF-8"?>

    <!DOCTYPE sun-ejb-jar PUBLIC "-//Sun Microsystems, Inc.//DTD
    Application Server 8.1 EJB 2.1//EN"
    "http://www.sun.com/software/appserver/dtds/sun-ejb-jar_2_1-1.dtd">

    <sun-ejb-jar>

      <enterprise-beans>

        <ejb>

          <ejb-name>MyBean</ejb-name>

          <jndi-name>sunMDB</jndi-name>

          <resource-ref>

           
    <res-ref-name>jms/MyQueueConnectionFactory</res-ref-name>

            <jndi-name> style="color: rgb(51, 204, 0);">jms/XAConFac</jndi-name>

          </resource-ref>

          <resource-env-ref>

       
    <resource-env-ref-name>jms/OutQueue</resource-env-ref-name>

        <jndi-name> style="color: rgb(51, 204, 0);">jms/SampleQueue2</jndi-name>

          </resource-env-ref>

          <bean-pool>

           
    <steady-pool-size>10</steady-pool-size>

           
    <resize-quantity>2</resize-quantity>

           
    <max-pool-size>30</max-pool-size>

           
    <pool-idle-timeout-in-seconds>60</pool-idle-timeout-in-seconds>

          </bean-pool>

          <mdb-resource-adapter>

           
    <resource-adapter-mid>genericra</resource-adapter-mid>

            <activation-config>

             
    <activation-config-property>

               
    <activation-config-property-name>DestinationType</activation-config-property-name>

               
    <activation-config-property-value>javax.jms.Topic</activation-config-property-value>

             
    </activation-config-property>

             
    <activation-config-property>

               
    <activation-config-property-name>MaxPoolSize</activation-config-property-name>

               
    <activation-config-property-value>30</activation-config-property-value>

             
    </activation-config-property>

             
    <activation-config-property>

               
    <activation-config-property-name>RedeliveryAttempts</activation-config-property-name>

               
    <activation-config-property-value>3</activation-config-property-value>

             
    </activation-config-property>

             
    <activation-config-property>

               
    <activation-config-property-name>RedeliveryInterval</activation-config-property-name>

               
    <activation-config-property-value>1</activation-config-property-value>

             
    </activation-config-property>

             
    <activation-config-property>

               
    <activation-config-property-name>ReconnectAttempts</activation-config-property-name>

               
    <activation-config-property-value>1000</activation-config-property-value>

             
    </activation-config-property>

             
    <activation-config-property>

               
    <activation-config-property-name>ReconnectInterval</activation-config-property-name>

               
    <activation-config-property-value>1</activation-config-property-value>

             
    </activation-config-property>

             
    <activation-config-property>

               
    <activation-config-property-name>DestinationJndiName</activation-config-property-name>

               
    <activation-config-property-value>SampleTopic style="color: rgb(51, 204, 0);"></activation-config-property-value>           


             
    </activation-config-property>         


             
    <activation-config-property>

               
    <activation-config-property-name>ConnectionFactoryJndiName</activation-config-property-name>

               
    <activation-config-property-value>mantaxaconnectionfactory</activation-config-property-value>           


             
    </activation-config-property>    

            </activation-config>

          </mdb-resource-adapter>

        </ejb>

      </enterprise-beans>

    </sun-ejb-jar>



    The business logic encoded in Message Driven Bean could then lookup the
    configured QueueConnectionFactory/Destination resource

    to create a connection as shown below (in the onMessage(...) method).


    Context context = null;

    ConnectionFactory connectionFactory = null;

    logger.info("In PublisherBean.ejbCreate()");

    try {

        context = new InitialContext();

        queue = (javax.jms.Queue) context.lookup
    ("java:comp/env/jms/SampleQueue2");

        connectionFactory = (ConnectionFactory)
    context.lookup("java:comp/env/jms/MyQueueConnectionFactory");

        connection =
    connectionFactory.createConnection();

        QueueSession qss =
    connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);

        QueueSender sender = qss.createSender(queue);

        TextMessage msg = qss.createTextMessage();

        msg.setText(txtmsg.getText());

        sender.send(msg);

        connection.close();



    } catch (Throwable t) {

        logger.severe("PublisherBean.ejbCreate:"
    + "Exception: " +

        t.toString());

    }





    Client

    The setup can be tested using a simple client sample that comes along
    with MantaRay





    We will use the MANTA_HOME/samples/src/sample/jms/topics/ReliableChat
    and MANTA_HOME/samples/src/sample/jms/queues/Talk as producer and
    consumer application respectively

    Complie the samples by executing the Compile.sh script that is provided
    under the above folders, this should generate the classes for these
    sample applications.



    To see it in action, modify the
    MANTA_HOME/samples/src/sample/jms/topics/ReliableChat/RunSample1.sh and
    ensure that we publish to the correct topic, "-t SampleTopic".

    Modify the MANTA_HOME/samples/src/sample/jms/queues/Talk/RunSample1.sh
    and ensure that we receive from the correct queue, "-qr SampleQueue2"



    Output of
    MANTA_HOME/samples/src/sample/jms/topics/ReliableChat/RunSample1.sh



    > ./RunSample1.sh style="background-color: rgb(204, 204, 204);">
    *** MantaRay Layer
    Name: m10.12.171.1226603 ***     style="background-color: rgb(204, 204, 204);">
    MantaRay 2.0.1
    initialization completed.     style="background-color: rgb(204, 204, 204);">


    Enter text messages
    to clients that subscribe to the SampleTopic topic.     style="background-color: rgb(204, 204, 204);">
    Press Enter to
    publish each message.     style="background-color: rgb(204, 204, 204);">
    Typing 'exit' will
    stop the program.     style="background-color: rgb(204, 204, 204);">


    Hi, this is a test
    message

    TOPICSENDER: Hi,
    this is a test message





    Output of MANTA_HOME/samples/src/sample/jms/queues/Talk/RunSample1.sh

     

    > ./RunSample1.sh style="background-color: rgb(204, 204, 204);">
    *** MantaRay Layer
    Name: m10.12.171.1226604 ***     style="background-color: rgb(204, 204, 204);">
    MantaRay 2.0.1
    initialization completed.     style="background-color: rgb(204, 204, 204);">


    Start receiving
    messages on queue "SampleQueue2".     style="background-color: rgb(204, 204, 204);">


    Enter text to send
    to queue "SampleQueue1".     style="background-color: rgb(204, 204, 204);">
    Press Enter to send
    each message.

    Empty messages will
    not be sent.

    Typing 'exit' will
    stop the program.     style="background-color: rgb(204, 204, 204);">
    QUUERECEIVER>TOPICSENDER:
    Hi, this is a test message     style="background-color: rgb(204, 204, 204);">
    TOPICSENDER: Hi,
    this is a test message     style="background-color: rgb(204, 204, 204);">
    TOPICSENDER: Hi,
    this is a test message





    We have received the same message 3 times from the queue SampleQueue2.
    This is because the MDB application in all the 3 instances of the
    cluster is a subscriber to the topic and has received the message and
    processed it. This results in 3 messages in the SampleQueue2. 
    This is an undesirable effect of deploying the application in a
    cluster. In any enterprise scenario, this would be unacceptable.



    In my next blog i will describe how mutually exclusive message
    processing can be achieved. With few changes to the configuration and
    MDB deployment descriptors and absoultely NO changes to the application
    code, Generic JMS RA is capable of guranteeing that only one MDB
    instance will process the message.


    Known Issues and Limitations



        All the MantaRay peers share the same configuration
    file, so management of these peers may not be possible from the
    MantaRay console because the RMI port will be the same. To use the
    management console, please provide each instance (GlassFish instance)
    with a separate MantaRay configuration file (mantaConfig JVM property
    should point to the respective configuation file). Another drawback of
    sharing the same configuration file is that the file based persistance
    folder is also shared and this may not be desirable.


    Resources

    • Generic Resource Adapter for JMS community page -
      download/documentation
    • Generic Resource
      Adapter for JMS user guide
      • href="https://genericjmsra.dev.java.net/docs/userguide/userguide.html">https://genericjmsra.dev.java.net/docs/userguide/userguide.html
    • Project GlassFish
    • MantaRay

      • href="http://www.coridan.com/Products.asp?ppid=2146&pid=2149&iID=10015">http://www.coridan.com/Products.asp?ppid=2146&pid=2149&iID=10015
    • For additional
      information on the '      RMPolicy' class="pagesubtitle">resource
      adapter configuration attribute refer issue#5 in the Generic RA issue
      tracker page. For documentation on usage of this attribute refer the
      Generic Resource Adapter for JMS user guide.
      • href="https://genericjmsra.dev.java.net/issues/show_bug.cgi?id=5">https://genericjmsra.dev.java.net/issues/show_bug.cgi?id=5




    Related Topics >>
    Blog Links >>