Skip to main content

Shoal Dynamic Clustering

Posted by hamada on November 9, 2006 at 11:57 AM PST

The href="http://shoal.dev.java.net">Shoal Framework provides a
cluster communication and event mechanisms. Shoal's  core service
(GMS), enables an application to dynamically become a member of  a
predefined cluster, and as such, is subscribed to cluster events, such
as :



- href="https://shoal.dev.java.net/ShoalGroupEventNotifications.html">Member
join, planned shutdown, failures

- Recovery member selection

- Automated delegated recovery initiation



A cluster member is also able to broadcast messages to an individual
member or all members of the cluster(see href="https://shoal.dev.java.net/ShoalMessaging.html">Messaging).
In addition to messaging, members can share data using GMS's Shared
Cache (see href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/ee/cms/core/DistributedStateCache.java?rev=1.2&view=markup">DistributedStateCache).



The main goal for the shoal project to provide a clustering framework
for the GlassFish project, however it does not preclude it from being
used for purposes, such as the foundation for a grid deployment, or a
deployment of a highly available service, etc (I would be interested in
hearing about other use cases).



Shoal/GMS utilizes the JxtaManagement component (a href="http://www.jxta.org">JXTA based group service provider) for
dynamic cluster configuration, formation, and monitoring.  This
component is broken out as follows :

  • href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/jxtamgmt/NetworkManager.java?rev=1.2&view=auto&content-type=text/vnd.viewcvs-markup">NetworkManager. 
    Given a instance and group name, NetworkManager uses a SHA-1 hash to
    encode the cluster GroupID, and NodeID, in addition it also defines a
    set of predefined communication identifiers which are used for
    formation, monitoring and messaging.  In addition, an application
    may pass additional configuration parameters, such as bootstrapping
    addresses to facilitate cross sub-net and firewall communication.
  • href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/jxtamgmt/SystemAdvertisement.java?rev=1.1.1.1&view=auto&content-type=text/vnd.viewcvs-markup">SystemAdvertisement.
    An extensible XML document describing system characteristics (HW/SW
    configuration. CPU load would be a nice extension). This information is
    exchanged during cluster formation, and monitoring.  It is
    envisioned that such information would serve at the foundation of a
    Grid  framework.
  • href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/jxtamgmt/MasterNode.java?rev=1.2&view=auto&content-type=text/vnd.viewcvs-markup">MasterNode.
    Is a lightweight protocol allowing a set of nodes to discover one
    another, and autonomously elect a master for the cluster.  The
    protocol is resilient to multi node collisions and employs an
    autonomous mechanism to avoid network chatter to resolve collisions.
    • href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/jxtamgmt/ClusterView.java?rev=1.1.1.1&view=auto&content-type=text/vnd.viewcvs-markup">ClusterView.
      Maintains an ordered view of the Cluster
  • href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/jxtamgmt/HealthMonitor.java?rev=1.1.1.1&view=auto&content-type=text/vnd.viewcvs-markup">HealthMonitor. 
    Is a  lightweight protocol allowing a set of nodes to monitor the
    health of a cluster. The HealthMonitor relies on a tunable heart beat,
    which is acted upon by the MasterNode to notify the group of failures,
    and by other members to elect a new master if the master node fails.
  • href="https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/jxtamgmt/ClusterManager.java?rev=1.1.1.1&view=auto&content-type=text/vnd.viewcvs-markup">ClusterManger.
    Manages lifecycle of this SPI
href="https://shoal.dev.java.net/source/browse/*checkout*/shoal/www/ClusterManager.png"> alt="ClusterManager Stack" title="ClusterManager Stack"
src="https://shoal.dev.java.net/source/browse/*checkout*/shoal/www/ClusterManager.png"
style="border: 0px solid ; width: 396px; height: 267px;">
Figure 1. ClusterManager Software Stack



By using JXTA as the foundation for the ClusterManager, shoal is able
to :

  • Use logical name cluster and node addressing (requires unique
    naming)
  • Achieve near Zero configuration for cross sub-net and firewall
    connectivity
  • Inherit dynamic transport selection without application
    intervention (multicast vs. unicast)
  • Inherit dynamic route repair
    • Enables mobility, same name different physical addresses
    • Automatic rerouting on/to available interfaces on multi-homed
      nodes
  • Inherit traffic scoping to cluster members
  • Establish secure end-to-end channels



for additional blogs on the project, see blog entries by Shreedhar, Bernard, Traversat and Masood Mortazvi

Related Topics >>