Mohamed Abdelaziz's Blog

Community: JXTA Archives

JXTA 2.5 FCS is just around the corner

• An NIO based Scalable TCP/IP message transport
• A per PeerGroup ThreadPool, and retiring of the service message quota logic
• Introduced an complement to the NetworkConfigurator, which abstracts node operating modes, and manages the platform lifecycle
• Much improved Tutorials and programmers guide
• Optimized route healing, beneficial to mobile nodes
• Improved connectivity (rendezvous, relay, JxtaSocket, JxtaBiDiPipe, pipes
• Improved JxtaServerPipe concurrent connection handling
• Expose and make use of direct messengers whenever possible, thus achieving 10x performance.
• Improved startup time to sub-second from a previous 5 second startup time
• Optimized directed propagated pipes to utilize direct messengers, and avoid two layers of overhead, resulting in higher reliability and improved performance
• Improved SRDI GC which caused CPU spikes at times
• Improved endpoint messenger concurrency which lead to improved message delivery latency
• Countless bug fixes throughout the code base, which improves the overall reliability of the platform

Shoal Dynamic Clustering

• NetworkManager.  Given a instance and group name, NetworkManager uses a SHA-1 hash to encode the cluster GroupID, and NodeID, in addition it also defines a set of predefined communication identifiers which are used for formation, monitoring and messaging.  In addition, an application may pass additional configuration parameters, such as bootstrapping addresses to facilitate cross sub-net and firewall communication.
• SystemAdvertisement. An extensible XML document describing system characteristics (HW/SW configuration. CPU load would be a nice extension). This information is exchanged during cluster formation, and monitoring.  It is envisioned that such information would serve at the foundation of a Grid  framework.
• MasterNode. Is a lightweight protocol allowing a set of nodes to discover one another, and autonomously elect a master for the cluster.  The protocol is resilient to multi node collisions and employs an autonomous mechanism to avoid network chatter to resolve collisions.
• ClusterView. Maintains an ordered view of the Cluster
  
• HealthMonitor.  Is a  lightweight protocol allowing a set of nodes to monitor the health of a cluster. The HealthMonitor relies on a tunable heart beat, which is acted upon by the MasterNode to notify the group of failures, and by other members to elect a new master if the master node fails.
• ClusterManger. Manages lifecycle of this SPI
  

• Use logical name cluster and node addressing (requires unique naming)
  
• Achieve near Zero configuration for cross sub-net and firewall connectivity
• Inherit dynamic transport selection without application intervention (multicast vs. unicast)
• Inherit dynamic route repair
  
• Enables mobility, same name different physical addresses
• Automatic rerouting on/to available interfaces on multi-homed nodes
  
• Inherit traffic scoping to cluster members
• Establish secure end-to-end channels

Demystifying Pipes, JxtaSockets, JxtaMulticastSocket, and JxtaBiDiPipes

Lately there has been several inquiries about JXTA's PipeService, and companion utilities (JxtaSocket, JxtaMulticastSocket, and JxtaBiDiPipe) on JXTA's discussion lists, hence this blog to shed more light on the PipeService and utilities provided, and their inherit features.

The initial goal of the PipeService was to provide primitive (unidirectional, and unreliable) message based communication channels upon which complex and sophisticated communication channels can be built. The PipeService provides three types of pipes described as follows:

• Unidirectional
• Unreliable
• Many to one
• Limited to 64KB messages

• Unidirectional
• Limited reliability (no direct notification of failures other than channel failure)
• Many to one
• Limited to 64KB messages

• Multidirectional (many to many)
• Unreliable
• Limited to 64KB messages in infrastructure mode, and limited by physical transport limitation where applicable

JXTA JSE platform software stack

Unicast pipes (clear, and secure) are bound by the PipeResolver, which provides two modes of bindings, a static (where a set of peers is specified), and a dynamic (no peers specified) binding. Static binding is beneficial in constraining communication to a limited set of peers, while dynamic is beneficial in implementing unconstrained mobile communication channels. Both modes can be utilized in implementing fault tolerant communication channels.

Propagated pipes employ a mechanism similar to that of IGMP, where a creation of an input pipe results in propagated pipe channel join message (a pipe SRDI message to a the peer's rendezvous). In infrastructure mode propagated pipe messages are sent to the rendezvous for propagation to the subscribers of the propagated pipe channel, while in ad-hoc mode messages are propagated over the endpoint (currently only the TCP transport supports such function over multicast). A pipe closure results in a channel resign message (a pipe SRDI message to the rendezvous expiring the previous join). It's worth noting that the PipeResolver plays no part in binding of such pipe type

An InputPipe creation results in the creation of an endpoint incoming messenger, and an SRDI message with the pipe ID with an infinite expiration time. Once the InputPipe is closed another SRDI message is emitted with the pipe ID, and expiration time of 0, invalidating any reference for the pipe and peer. Note: This behavior applies to all pipe types.

An output pipe creation results in a PipeResolver query message emitted querying for instances of the pipe. A match results in a PipeResolver response to the querying peer, where an endpoint messenger to responding peer is created. Note that such messenger may not be fully resolved until the EndpointRouter has determined a valid route. During such event few messages maybe queued (3 to be exact) until the messenger has been resolved. In addition, as part of the pipe resolver protocol the pipe resolver periodically reattempts pipe resolutions in the background to validate resolved pipe endpoints. It is through this mechanisms pipes can migrate from one peer to another. note: this behavior is limited to unicast pipe types.

SRDI message path within the JXTA network

A pipe endpoint is composed of a peer, group, and pipe ID, through this definition there are two inherit modes of pipe endpoint migration:

1. Peer endpoint migration, an inherit feature of the Endpoint-Router, where a peer may change physical, or virtual addresses.
2. Pipe endpoint migration, where a pipe endpoint ceases to exists on a peer, where the PipeResolver, attempts to rebind to the pipe (which I always refer to as a virtual channel)

The first mode is transparent the application, while the second may result in an IOException.

A standalone message and stream based reliability layer over non reliable messengers, is provided in the platform to facilitate reliable communication channels.

• Ensures message sequencing
• Interfaces to pipes or messengers
• Ensures delivery
• Exposes message, and stream interfaces
• Spawns a single thread to deal with retransmission
• Does not ensure message integrity

JxtaServerSocket, and JxtaServerPipe expose a input pipe to process connection requests and negotiate communication parameters, whereby a JxtaSocket, or JxtaBiDipipe bind to respectively to establish private dedicated pipes independent of the connection request pipe. JxtaSocket, and JxtaBiDiPipe utilize a messenger instead of an output pipe for the back channel to reduce connection latency (avoids 4 messages), therefore only peer endpoint migration is inherited, and one should always expect an IOException if a PipeEndpoint migrates, or ceases to exist.

• Subclass java.net.Socket, and java.net.ServerSocket respectively
• Built on top of pipes, endpoint messengers, and the reliability library
• Provides bidirectional and reliable communication channels
• Exposes stream based interface a la Socket
• Provides configurable internal buffering, and message chunking
• Does not implement the Nagels algorithm, therefore streams must be flushed as needed (a common oversight when wrapping object and data streams over the socket stream)

• Built on top of pipes, endpoint messengers, and the reliability library
• Provides bidirectional and reliable communication channels
• Exposes message based interface
• Requires no heart beat to maintain an open connection
• Provides no message chunking

JxtaMulticastSocket extends java.net.MutlicastSocket and provides mutlicast functionality over JXTA propagated pipes.

• Subclass java.net.MulticastSocket
• Built on top of propagated pipes
• Exposes datagram based interface a la MulticastSocket
• Provides unicasting through datagram addressing

1. The JXTA platform imposes a soft MTU of 64KB (which becomes a hard limit under load), messages exceeding MTU are simply dropped. While nodes maybe able to transmit and receive messages larger than 64KB, it is strongly advised this limit is not exceeded in reliable mode, as it will result in failed or broken channels.
2. When utilizing relay peers, such limit must not be exceeded in either mode (reliable or otherwise) as the relay nodes are likely to be loaded and/or the fairness message queue size quotas are enforced, thus leading to message loss and consequently failed or broken channels.

• Pipe advertisement mismatch. Typically occurs when dynamically creating and discovering pipe advertisements using non unique identifiers, or deterministic method. Normally overcome through hardcoded pipe advertisements (or pipe ID's), encoded pipe ID's.
• Un-synchronized pipe binding. Typically caused by network islands, which are joined at a later point in time. Easily overcome through rendezvous event listener prior to any binding attempt.
• Failed Secure pipe connections. Typically due to failure of establishing NetPeerGroup credentials (TLS resides in the NetPeerGroup, and inherited by all sub-groups), or missing a trusted certificate or certificate chain in PSE. This is overcome by establishing NetPeerGroup credentials, and ensuring the perspective trusted certificate or certificate chain is stored into PSE (See the shell command pse.importcert).
  
  

  Distributed Collections, and Maps

  Posted by hamada on June 14, 2005 at 11:56 PM | Permalink | Comments (3)
  
  

  There's no reason to wonder any longer. The JXTA platform is well suited for the task, and provides several mechanisms which allow a variety of features which can be offered by such applications. i.e.

  Imagine if members of a JXTA virtual multicast group could dynamically self organize, then they can easily self organize into replica and consumer nodes, whereby objects are transparently replicated within a given JXTA virtual mutlicast group. In addition through the use of these virtual channels, nodes can detect node failures and dynamically self adapt their designation within the group. Also, mobile nodes can continue to participate within the group without the need to reconfigure. It is this dynamic nature of the JXTA platform, with which, one can achieve scalable and fault tolerant applications with a bit of code.

  I have been in discussion about this with my Sun colleagues and Some of the JXTA community members and so far we don't see any inhibitors to implementing simple distributed Maps, or Collections, which can serve as the foundation for more complex incarnations.

  We invite you to join the discussion of such features on dev@platform.jxta.org, or dev@jxta.org

  It's time for alternatives.

  
  
  

  Have you heard of Google's Summer of Code?

  Posted by hamada on June 06, 2005 at 10:28 AM | Permalink | Comments (0)
  
  If you have answered yes to the above, we are very happy for you!. Google is sponsoring the Summer of Code, and Project JXTA is one of the participating organizations. We have created a list of projects in the areas where we seek contribution. If you are a student and it is of interest to you, you can read about all the details and the list of projects at :
  http://wiki.java.net/bin/view/Jxta/HowToParticipate
  We look forward to a fruitful summer.
  
  
  

  JXME on CDC? maybe closer than you think

  Posted by hamada on May 15, 2005 at 11:53 PM | Permalink | Comments (0)
  
  The JXME project has provided proxy based support of a subset of the JXTA protocols. The proxy provided much of the interaction between the device and other nodes on the network. Recently there has been quite a bit of interest of supporting a richer set of protocols for the "Connected Device Configuration", and as a result most of the JXTA protocols have been ported to run on JSR 218, and the foundation profile this includes :

  - All JXTA advertisement bindings, including support for application defined advertisements.

  - All protocol messages in support of edge functionality

  - LiteXML support

  - All JXTA Core service (minus PeerInfo, and (of course) Proxy)

  - TCP/IP transport

  - Client Relay transport

  - JXTA ID's

  There remains a handful of classes to be implemented to provide an in memory cache manager, and PlatformConfigurationFactory. So far the stack sitting at about 581K, and it is expected not to deviated that much. There still remains plenty of work to be done, and I can all the help I can get. Please pay the project a visit and feel free volunteer for any of the unassigned tasks. It will definitely be appreciated.

  I keep a diary of my progress on my sun blog, and under JXME.
  
  
  

  This work is licensed under a Creative Commons License.
  
  
  
  Powered by
  Movable Type 3.01D