Mohamed Abdelaziz's Blog: Community: Java Enterprise Archives

Posted by hamada on October 06, 2007 at 03:00 PM | Permalink | Comments (3)

In today's information technology world, fault tolerance has become an expected system characteristic, as demands on such systems, not only requires the availability of data, but also the efficiency of such systems.

By clustering a set of servers, with minimal, or no configuration, through a dynamic discovery protocol, a compute cluster can be formed to increase compute power, availability, security, and geographical distribution.

As a fault tolerance strategy, the Shoal clustering framework devises a self organization protocol, allowing nodes to autonomously elect a master for the cluster (based on discovery views), as well as candidates in the case of master failure. This allows for dynamic deployment of clusters as well as expanding/shrinking an existing cluster.

Shoal, the cluster management framework, provides the foundation for network configuration and dynamic and autonomous cluster formation. The goals of the framework are:

Near Zero-configuration
Multi cluster support

Application specific clustering, eg. load balancer/HA cluster
Cluster traffic isolation
Added security
improved resource utilization

Dynamic clustering

Hot pluggable

Simplified addressing

By cluster, node, and channel name

Adaptive communication channels

Fault tolerance

Transparent multi protocol support (tcp, udp, http)

Shoal 1.0 was release August 30th, and is utilized in GlassFish V2, which was released September 17th, for clustering and http session HA.

Currently the following features are being worked on :

Enabling multi cluster support
Enabling cross sub-net and cross firewall deployment
Solidifying lightweight cluster multicast

We encourage community participation in enabling and identifying missing features by joining and participating in the shoal community.

Shoal Dynamic Clustering

Posted by hamada on November 09, 2006 at 11:57 AM | Permalink | Comments (2)

The Shoal Framework provides a cluster communication and event mechanisms. Shoal's core service (GMS), enables an application to dynamically become a member of a predefined cluster, and as such, is subscribed to cluster events, such as :

- Member join, planned shutdown, failures
- Recovery member selection
- Automated delegated recovery initiation

A cluster member is also able to broadcast messages to an individual member or all members of the cluster(see Messaging). In addition to messaging, members can share data using GMS's Shared Cache (see DistributedStateCache).

The main goal for the shoal project to provide a clustering framework for the GlassFish project, however it does not preclude it from being used for purposes, such as the foundation for a grid deployment, or a deployment of a highly available service, etc (I would be interested in hearing about other use cases).

Shoal/GMS utilizes the JxtaManagement component (a JXTA based group service provider) for dynamic cluster configuration, formation, and monitoring. This component is broken out as follows :

NetworkManager. Given a instance and group name, NetworkManager uses a SHA-1 hash to encode the cluster GroupID, and NodeID, in addition it also defines a set of predefined communication identifiers which are used for formation, monitoring and messaging. In addition, an application may pass additional configuration parameters, such as bootstrapping addresses to facilitate cross sub-net and firewall communication.
SystemAdvertisement. An extensible XML document describing system characteristics (HW/SW configuration. CPU load would be a nice extension). This information is exchanged during cluster formation, and monitoring. It is envisioned that such information would serve at the foundation of a Grid framework.
MasterNode. Is a lightweight protocol allowing a set of nodes to discover one another, and autonomously elect a master for the cluster. The protocol is resilient to multi node collisions and employs an autonomous mechanism to avoid network chatter to resolve collisions.

ClusterView. Maintains an ordered view of the Cluster

HealthMonitor. Is a lightweight protocol allowing a set of nodes to monitor the health of a cluster. The HealthMonitor relies on a tunable heart beat, which is acted upon by the MasterNode to notify the group of failures, and by other members to elect a new master if the master node fails.
ClusterManger. Manages lifecycle of this SPI

Figure 1. ClusterManager Software Stack

By using JXTA as the foundation for the ClusterManager, shoal is able to :

Use logical name cluster and node addressing (requires unique naming)
Achieve near Zero configuration for cross sub-net and firewall connectivity
Inherit dynamic transport selection without application intervention (multicast vs. unicast)
Inherit dynamic route repair

Enables mobility, same name different physical addresses
Automatic rerouting on/to available interfaces on multi-homed nodes

Inherit traffic scoping to cluster members
Establish secure end-to-end channels

for additional blogs on the project, see blog entries by Shreedhar, Bernard, Traversat and Masood Mortazvi

Powered by
Movable Type 3.01D

java.net RSS Feeds