FlashGridding with Java Technology (TS-3714)
I'm sitting here watching Van and his ComputeCycles.org team describe the benefits of Java and Jini in the construction of grid technologies that bridge heterogeneous networks (including firewalls), systems / operating environments and deal with Deutsch's Fallacies of Networking to assist in massively parallel processing (master-worker pattern) - and cannot help but say, pretty cool...
In a nutshell
ComputeCycle is looking to provide a solution to massively parallel tasks that can handle the complexities of a fully distributed grid. Fully distributed, in my mind is not relying on a single farm of processors, such as Sun Grid provides today, but instead looking to allow farms like Sun Grid, in house cycle stealing, Internet cycle stealing to deliver a coherent computing platform. Whereas the ComputeServer project provides a master-worker framework for processing against the Sun Grid, the ComputeCycle team is looking to expand to support workload management across a federation of providers, using Jini and true distributed service semantics.
Using GlassFish for integration with NetBeans... Jini gives them JERI, Reggie and JavaSpaces, and then their own frameworks.... this pattern seems to repeat itself frequently ;). And interestingly Groovy... Michael Henderson "Groovy killed the XML File". To help with respect to the configuration across the environment... it's syntax is remarkably close to what Jini Services need.
"KeyRing" based authentication - authorized/verified access to keypairs using keytool files (SSL Encryption of all communications... intra and extra grid network)
Enforce the use of a SecurityManager
Master Session has a session space... delivers isolation to valid key holders, provides a separate protection domain
Dynamic Configuration, Activation and Provisioning
Groovy enabled ConfigurationProvider (Jini provider)
2 phase configuration... 1) generic worker service, 2) flavors applied(provisioned) during grid session activation
Distributed Resource Management
Jini leasing model & session model
sessions have expirations and are managed by a "GridManager", but the sessions reside in JavaSpaces (Outrigger) w/ their enforceable isolation.
Federate structure supported w/ a GlobalSesison as the root... this enables very dynamic coupling of sessions to the resources, and session master to the workers
Lookup Service(Reggie).... to find resources
Grid Manager.... service controls - activate, grow/shring, recover, checkpoint, shutdown, all global signals, knowledgeable of all participating workers through a Session TupleSpace (which the Grid Manager instantiates and then gives a reference back to the Session Master.
The workers monitor the space for work to be done, and does a transactional take.
workers are naturally fault tolerant, but how do we maintain the 2n/n+1 availability .. this starts to put the requirements for mahalo (a distributed transaction management technology) to allow for transactional updates made for the replication needed to establish high-availability.
One typical question is one of dynamic work binding - how do we let non Java/Jini tools to submit work? They use GlassFish (heavyweight/permanent services) and JNDI to enable the vending of lookup proxies through web services/https in order to allow for firewall penetration.
I think that the approach is very promising, and would encourage anyone interested to take a look!