GlassFish 3.1 Shoal Group Management Service (GMS) for runtime clustering services

Scope of the Project

Shoal GMS is a clustering framework that provides infrastructure to build fault tolerance, reliability and availability
The clustering framework provides the following functionalities to Glassfish services and user applications.

  • GMS event notification
    • register callback for GMS event notification
    • GMS runtime notifies when changes in group membership occur.
  • Cluster-wide Messaging
    • send a message to one, sublist or broadcast to all members of cluster
    • register message callback to process messages sent by other group members.
  • GMS status methods
    • list of all members or just list of CORE members
    • request a member status on any member in cluster

Recent enhancement to GMS is to have a pluggable transport and a Grizzly implementation of that transport.
GF HA subsystem is building on top of GMS clustering framework and its messaging.

TBD: Show minimal that a GF module or a app to be deployed in GF needs to do to access GMS API.

%%sortable

Feature-ID Desired Improvement Priority Comments Issue Link Eng Response
GMS-1 Shoal GMS over Grizzly implementation P1 Support GMS to work with Grizzly as the transport provider   YES
GMS-2 Test GMS over Grizzly based on evaluation criteria P1 Run 6-8 critical scenarios of all scenarios over several iterations to establish feasibility of Grizzly transport based implementation and stability levels equivalent to the JXTA based implementation   YES
GMS-3 Make OSGi compliant GMS Libraries P1 GMS needs to become an OSGi module to be integrated into GlassFish 3.1 workspace   YES
GMS-4 Monitoring Stat Providers P2     TBD
GMS-5 Create a GMS Service in GlassFish that will act as a delegate to start and stop GMS module in each GF 3.1 instance P2     TBD
GMS-6 Address any requirements specific to in-memory replication component, EJB components such as timer, transaction service, etc. P1     YES
GMS-7 Identify Requirements for other Glassfish components needed for GMS within v3 P1     YES

Feature Overview

GMS-1 Shoal GMS over Grizzly implementation

  • Description

Replace JXTA as the transport provider for GMS and provide an implementation over Grizzly using Grizzly as the transport provider for TCP and UDP messages. The basic implementation is being provided by Bongjae Chang, Shoal community member, but needs further enhancements to make it production quality.

  • Sub-tasks
    • Tune default Grizzly NetworkManager parameters. These are settable via following GMS Property parameters. <b>MAX_POOLSIZE, CORE_POOLSIZE, KEEP_ALIVE_TIME, POOL_QUEUE_SIZE, HIGH_WATER_MARK, MAX_PARALLEL, WRITE_TIMEOUT, MAX_WRITE_SELECTOR_POOL_SIZE</b>. These parameters manage resources available to process incoming Grizzly processing.
  • Owners

Joe Fialli

  • Dependencies
    • Grizzly framework and util jars version 1.9.19
  • Estimate

GMS-2 Test GMS over Grizzly based on evaluation criteria

  • Description

The GMS over Grizzly implementation will be tested over several of the critical test scenarios to establish its viability and stability for usage in a production quality GlassFish deployment. Initial stability tests will be based on module level distributed tests.

  • Sub-tasks
    • Requires QE to port over earlier GMS tests that were based on Appserver and depended on Appserver asadmin commands et al and revert to a module level test as we did with initial GlassFish v2 testing.
    • Fix P1 and P2 type bugs to establish basic stability.
    • Add GMS messaging load testing since HA relying directly on GMS over Grizzly messaging. (in v2.1, HA used jxta messaging directly)
  • Owners

Joe Fialli

  • Dependencies

QE tests being ported over
Ported tests setup on a hudonsized automated distributed dev testing setup.

  • Estimate TBD

GMS-3 Make GMS OSGi compliant

  • Description

GMS was not an OSGi compliant module. Changes being incorporated to mavenize GMS to publish to maven repository, and use GF v3 maven commands to build an OSGi compliant module

  • Sub-tasks

Sheetal to fill in

  • Owner

Sheetal

  • Dependencies

GMS for JoinedAndReady (instance start and restart), failure detection notifications and all Messaging pertaining to session replication.
DOL (possibly for deployment framework support to correctly handle automatic timer creation) <br>

  • Estimate TBD

GMS-4 Monitoring Stat Providers

  • Description

Provide monitoring stats that will help with diagnosing GMS membership related issues.

  • Sub-tasks

TBD

  • Dependencies
  • Owner

Sheetal, Joe

  • Estimate TBD

GMS-5 Create a GMS Service in GlassFish that will act as a delegate to start and stop GMS module in each GF 3.1 instance

  • Description

Create a GMS Service in GlassFish that will delegate start and stop operations on GMS module in each GF 3.1 instance. This is an @Startup class that will be part of the startup services in a clustered GF instance. Need to close on requirements with respect to when GMS module will be started - need architects to consider impact of introducing instability and unpredictability in group memberships with lazy startup guidelines.

GMS-6 Address any requirements specific to in-memory replication component, EJB components such as timer, transaction service, etc.

  • Description

...

GMS-7 Identify Dependencies on other Glassfish components needed for GMS within v3

  • Description

Identify functionalities that GMS depends on from other Glassfish component in v2.1 that are needed in v3 also.

    • GMS group property configuration parameters from domain.xml. (gmsconfig document being written)
    • Ability to register a "createCluster" handler on DAS (when "asadmin create-cluster" is executed on a DAS) to enable DAS to dynamically join the newly created cluster.
    • DAS should be started 5 seconds before any of its clusters are started. asadmin start-cluster only worked when DAS was running in v2.1.
    • <others to be identified>

One-pager / Functional Specification

  • Coming soon

Dev Tests

  • Coming Soon - Information on GMS dev tests will be posted on Dev Tests page
  • Introducing junit tests.
  • 2 to 3 developer level tests that run multiple instances of a gms group on one machine. (not completely automated testing)
  • Developer level test are basis of shoal test scenarios run on 10 node cluster.

Quality

  • Link to Test Plans

Documentation

  • Link to Documentation

Schedule

  • Project Schedule / Task List / Current Status

References / Links

Email Alias

  • Please contact us at [^mailto:dev@shoal.java.net]