GlassFish Server Open Source Edition 3.1 - Clustering Infrastructure One Pager

1. Introduction

1.1. Project/Component Working Name:

GlassFish Server Open Source Edition 3.1 Clustering Infrastructure

1.2. Name(s) and e-mail address of Document Author(s)/Supplier:

Tom Mueller: TMueller@java.net

1.3. Date of This Document:

May 14, 2010

2. Project Summary

2.1. Project Description:

The Clustering Infrastructure project for GlassFish Server Open Source Edition 3.1 consists of basic administrative functionality that is needed to support clusters including support for cluster and instance creation, read, update, and delete (CRUD), startup with synchronization with the domain administration server (DAS), and dynamic reconfiguration as the system is running.

See below for a longer, more detailed technical description.

2.2. Risks and Assumptions:

Risks:

1. Hardcoded "server" and "server-config": In GlassFish 3.0, the configuration elements for "server" and "server-config" were hardcoded into the software in many places. To run instances that are not the DAS, it is necessary to find all of these and change the code to use an instance name and configuration that is determined at run-time. The risk is that some may not be found.

1. Performance: In GlassFish v2, there were some scalability problems with clustering. The intent is to remove some of those bottlenecks in this release to make clustering scale better. The risk is that the system still may not perform as desired. This risk is being addressed by using performance testing earlier in the design and development of the feature.

1. Compatibility issues related to lack of a node-agent: One of the big differences between cluster support in 3.1 and GlassFish v2 is that 3.1 will lack support for a node-agent. For GlassFish v2 installations that are using a node agent, this will mean that an upgrade to 3.1 will require modifying scripts that make use of the node agent, especially in the area of instance startup and monitoring.

Assumptions:

1. V2-based Design: A basic assumption for this feature is that the design for clustering that was used for V2 will work for this release, except for a few specific changes that are being made to address performance issues.

3. Problem Summary

3.1. Problem Area:

This project provides the basic infrastructure for clustering that is needed to allow the rest of the GlassFish team to complete the overall clustering feature for GlassFish Server Open Source Edition 3.1. Once this project is completed, developers will be able to:

• Create and manage (start/stop) clusters using asadmin
• Create and manage instances (start/stop) which may be within clusters
• Enhance management interfaces (CLI and console) to support clustering related features

3.2. Justification:

Clustering is one of the drivers of the 3.1 release and the clustering infrastructure is necessary to complete the clustering feature.

4. Technical Description:

4.1. Details:

Design details are provided in the design specification.

The basic idea is to implement the same clustering design that was used for GlassFish 2, except for modifications in a few areas to improve scalability. These modifications include:

• Operation without a node agent. The 3.1 release will not include a node agent. However, it will have and SSH capability (provided by another project) that will allow centralized management of a cluster.
• A new algorithm for synchronization with the DAS during server start-up.
• A new algorithm for dynamic configuration (reexecute commands rather than replicating results)

4.2. Bug/RFE Number(s):

The following bugs and RFEs will be addressed by the project. These issues are all marked with the clusteradmin status whiteboard value (query).

4.3. In Scope:

This project includes the following new asadmin subcommands:

• create-instance, delete-instance, list-instances, start-instance, stop-instance
• create-local-instance, start-local-instance, stop-local-instance, delete-local-instance
• create-cluster, delete-cluster, list-clusters, start-cluster, stop-cluster
• copy-config, delete-config, list-configs

This project includes modifications to the following asadmin subcommands:

• create-domain

This project includes modifications to annotations and new annotations that allow other commands to apply to clusters. In some cases, no additional modification will be needed for a CLI. But for other commands, cluster-specific functionality will have to be implemented by the team providing the command.

4.4. Out of Scope:

The following clustering administration functionality is being provided by other projects:

1. Implementation of cluster-specific functionality for asadmin commands not listed above
2. Clustered application deployment
3. The use of SSH to replicate functionality provided by a node agent
4. Support for clustered connectors, pooling, resources and JDBC
5. Security with a cluster
6. Transaction recovery
7. Modifications to the administration console that are needed to support clustering.

The following clustering-related administration commands from v2 will not be supported:

4.5. Interfaces:

4.5.1 Public Interfaces

• Interface: asadmin subcommands for clustering
• Comment: See the "in scope" section. The details of these commands are in the design specification

4.5.2 Private Interfaces

List private interfaces which are externally observable.

• Interface: Inter-node Protocols
• Comment: The protocols that are used to synchronize information between the DAS and instances, either at instance startup time (synchronization) or while the server is running (dynamic reconfiguration) are private and are subject to change as needed to improve the implementation.

• Interface: Node-agent file system
• Comment: The data for an instance that is part of a cluster is stored in a file system within the GlassFish installation directory that is parallel to the "domains" file system that already exists. The "create-node-agent" and other command create and modified files under this file system. The arrangement of files within this file system is a private interface and may be changed at any time.

• Interface: das.properties file
• Comment: The das.properties file contains information that is needed by an instance to contact the DAS that it should synchronize with. Here is an example of what the file looks like:

#Domain Administration Server Connection Properties
#Wed Jun 02 09:50:26 CDT 2010
agent.das.protocol=http
agent.das.port=4848
agent.das.host=das.example.com
agent.das.isSecure=false

4.5.3 Deprecated/Removed Interfaces:

None.

4.6. Doc Impact:

The documentation impact includes manual pages for all of new subcommands and modifications to the manual pages for the changed commands. See the "in scope" section.

4.7. Admin/Config Impact:

This project has a significant impact on the administration of GlassFish, as it introduces the clustering concept to the project.

4.8. HA Impact:

This project has a significant impact on the high availability and clustering aspects of GlassFish as a whole (that is the point of the project).

4.9. I18N/L10N Impact:

The i18n/l10n impact consists of making sure that the output from the new subcommands follows the patterns that are already established for administrative commands.

4.10. Packaging, Delivery & Upgrade:

4.10.1. Packaging

There is a separate project for packaging in GlassFish 3.1 which defines how the software for this project is delivered in packages.

4.10.2. Delivery

All initial distributions will include the clustering administration infrastructure software.

4.10.3. Upgrade and Migration:

Since GlassFish 3.0 and 3.0.1 did not support clustering, there are no upgrades or migrations of clusters to this release. However, it is expected that a GlassFish 2.1.1-based system with clusters can be upgraded to GlassFish 3.1. There is a separate project for upgrading to GlassFish 3.1 which handles the details of upgrade and migration with a dashboard that tracks progress on this.

The general philosophy for upgrade in GlassFish 3 is that there is no special upgrade program. If software encounters a configuration file that is in an older format, it automatically updates that file to the new format. However, there are some upgrade issues that are specific to this project:

• Transition to the new protocol that is used for synchronization of instances. Some information is needed for the das.properties file on hosts that run instances that isn't available in 2.1 configuration files.
• GlassFish 2.x hosts that run a node agent will need to do something else to get the instances running on the host.

4.11. Security Impact:

The security issues for GlassFish 3.1 administration, including those related to clustering, are being handled by a separate security project for GlassFish 3.1. See that project for details about security issues.

4.12. Compatibility Impact

Generally, the intent is to maintain compatibility with the clustering commands from GlassFish 2. However, there are several areas where there will be incompatibilities:

• Anything dealing with node agent will not be compatible because the implementation of clustering for 3.1 will not support a node agent.
• Instructions for making DAS/instance protocol secure will be different because protocol is different.
• The das.properties file will contain different information about the protocol between the DAS and an instance.
• 2.x had an exclude list for synchronization. This will not be supported for 3.1. Rather, any directories that are not explicitly synchronized can be used for storing application-specific files which will not be synchronized. 
• When synchronizing instances, 2.x looked at every file. This means that if an arbitrary file within an application is modified on the DAS, the change would be synchronized to the instances. However, in 3.1, synchronization is based on the timestamps of higher level directories. So this will result in an operational difference between the two.

4.13. Dependencies:

<!-An internal dependency is a dependency on a project, module, component or product that is within the GlassFish project. An external dependency is a dependency on a project, component or product that resides outside of the GlassFish project.->
4.13.1 Internal Dependencies
This project does not introduce any new internal dependencies on other software.

4.13.2 External Dependencies
This project does not introduce any new external dependencies on other software.

4.14. Testing Impact:

The features provided by this project can be tested through the use of the asadmin command tool, either on a single system or using multiple systems. It is expected that many of the tests from the GlassFish 2.1.1 release that are related to clustering can be reused for this release. Tests that involve the node agent will have to be modified to be reused.

There is nothing in this feature what would prevent tests from being automated.

5. Reference Documents:

Design specification document:

• GlassFish Server Open Source Edition 3.1 - Clustering Design Spec

Sub-project planning documents:

• GlassFish Server Open Source Edition 3.1 - Basic Clustering
• GlassFish Server Open Source Edition 3.1 - Configuration
• GlassFish Server Open Source Edition 3.1 - Synchronization
• GlassFish Server Open Source Edition 3.1 - Dynamic Reconfiguration

Other related projects:

• GlassFish 3.1 Packaging
• GlassFish 3.1 Upgrade
• GlassFish 3.1 Security

6. Schedule:

6.1. Projected Availability:

Integration of this feature will begin with Milestone 1 and continue through Milestone 4. The features will be complete at Milestone 4 and production quality level by the hard code freeze milestone.