GlassFish - World's first Java EE 7 Application Server


Production Redeployment One Pager

1. Introduction

1.1. Project/Component Working Name:

Production Redeployment

1.2. Name(s) and e-mail address of Document Author(s)/Supplier:

Marian Muller : marian.muller@serli.com
Pierre-Henry Soulaneix : pierre-henry.soulaneix@serli.com

1.3. Date of This Document:

01/09/2013

2. Project Summary

2.1. Project Description:

Support production redeployment.

2.2. Risks and Assumptions:

This feature is done by external contributor SERLI and there is certain risk associated with external contributors.

3. Problem Summary

3.1. Problem Area:

To allow users to switch between versions with no loss of service.
To allow users to deploy two active versions of the same application.

The production redeployment described in this document may interest users:

  • to deploy application versions without loss of service.
  • when deploying application versions which break session compatibility (so active redeployment cannot be used).
  • to rollback to the previous version of an application.

As application versioning does not keep an order relation between versions, production redeployment can be used for both upgrading and rolling back application versions.

3.2. Justification:

To provide high availability.

4. Technical Description:

4.1. Details:

4.1.1. Expected behaviour

With production redeployment feature, the user will be able to deploy simultaneously two active versions of the same application. This feature aims at two targets: allow the user to switch from one version to the other without loss of service, and allow clients of the said application to experience a smooth upgrade.

When two versions of the same application are enabled concurrently, we distinguish :

  • The 'active' version, which is the most recently enabled. It serves all the new clients.
  • The 'retired' version, which is the previously deployed (and active) version. It only serves the clients who were already using it.

When the two versions are enabled concurrently, the incoming requests will have to be routed to the appropriate version, depending on their attached session. The routing will follow a simple set of rules:

  • If the request has no session information, route to the active version.
  • If the request has session information and a matching session is found in the session manager of one version, use that version.
  • If the request has session information but no matching session can be found, use the active version.

The retirement process makes sure that the retired version :

  • only serves the previously existing clients.
  • is disabled after a given timeout, or when all the attached sessions have expired.
  • can be rolled back without losing its clients' sessions.

Production redeployment will be available in the CLI through a set of options for the deployment commands. It will also have to be made available through the GUI.
Options will be available to configure the production redeployment behavior, especially for specifying the retirement time.

The production redeployment will be an optional feature of the deployment commands (CLI options will be optional). Omitting the production redeployment options will result in experiencing the previous existing behavior.

Please see this page for more technical details.

4.1.1.1. Cluster mode

Production redeployment supports running on a cluster. Deployment commands are replicated to each instance, and each instance manages the two concurrent enabled versions (requests routing, version switching, disabling at the end of retirement). The retirement end time is added to domain.xml, which is available to all instances in the cluster.

When an instance is started (e.g. added to the cluster or restarted), the retired version is enabled in the retired state, while the active version is enabled as usual. If this endtime has already been reached when the instance starts, the retired version is simply not enabled.

Regarding the end of the retirement period, three options are being considered:

a. The DAS must be up during the whole process, failing what, expired retired applications would remain enabled and served until DAS comes back up.

c. The DAS does not need to be up during the whole process. Thanks to the retirement time in domain.xml, instances know when the retired version should be disabled. Instances can start routing all requests to the active version when the time expires, and wait for the DAS to be back up and to send the disable command, before actually disabling the retired version.

d. The DAS does not need to be up during the whole process. Thanks to the retirement time in domain.xml, instances know when the retired version should be disabled. Instances can disable the retired version on their own when the time expires (by running the disable command locally). When the DAS is back up, it sends the disable command to all instances. Instances that have already disabled the version simply ignore the command.

4.1.2. Restricted features

  • Any restriction of the application versioning applies to production redeployment (as it is based on versioning).
  • Production redeployment assumes that both active versions use the same context-root. Otherwise, ensuring continuity of service would not make sense, since clients would have to be aware of the context-root modification. If production redeployment is used with different context-roots, an error occurs.

4.1.3. Unsupported features

  • Allowing concurrent activation of a version for test purpose (that is, a version enabled but only available from a specific IP address or using a test-specific context-root).
  • While an active and a retired versions are enabled concurrently, it is not possible to deploy another version using production redeployment (which would make two retired versions). A user has to explicitely disable the retired version before.

4.1.4. Assumptions on applications

For production redeployment to be fully fonctional, it is assumed that:

  • The application does not use Message-Driven Beans. The incoming messages could not be routed to a specific version's MDB. Nevertheless, the application may use JMS to send messages, granted that the recipient supports being called by two concurrent versions of the application.
  • The application does not declare any global-scope JNDI resource. Two concurrent versions could not declare the same global resource.
  • Any external resource (databases, filesystems, sockets, ...) accessed by the application must support being used by two versions concurrently.

N.B.: As external resources can not be handled by production redeployment, application authors should make sure that two enabled versions are either compatible regarding the external resources (e.g. same database schema), or use separate resources.

N.B.: The application may rely on any technology based on HTTP protocol (e.g. WebSockets, JAX-WS, ...) and benefit from production redeployment's request routing, granted that the technology relies on HTTP sessions.

4.1.5. Use case examples

4.1.5.1. Context of examples

For production redeployment to apply, there must be an active version of the application. For instance an application version "foo:1.0" can be deployed using the regular command with versioning:

$ asadmin deploy --name foo:1.0 foo1.war

This version serves all the incoming requests as usual.

If production redeployment is used when there is no active version of the application, an error occurs.

4.1.5.2. Upgrade an application without loss of service

A user wants to upgrade the application "foo:1.0" to "foo:2.0", without loss of service, and to keep "foo:1.0" to serve clients with existing sessions during 5 minutes:

$ asadmin deploy --name foo:2.0 --retiretimeout=300 foo2.war

Both "foo:1.0" and "foo:2.0" are enabled. Although, "foo:1.0" is only available to clients with existing sessions created by "foo:1.0" (it is "retired"). The default version for other clients is "foo:2.0" (the "active" version).
After all the attached sessions have expired, or after the given retirement time (e.g. 300 seconds), whichever comes first, "foo:1.0" is automatically disabled.

4.1.5.3. Disable a retired version

A user wants to explicitely disable the version "foo:1.0" (this can happen before the end of the retirement time):

$ asadmin disable foo:1.0

Only "foo:1.0" is disabled. "foo:2.0" is now used for all the clients, even those with an active session created by "foo:1.0". The routing mechanism is disabled as only one version remains available.

4.1.5.4. Roll back an application without loss of service

A user wants to roll the application "foo" back to the disabled "foo:1.0", without loss of service, and to keep "foo:2.0" to serve clients with existing sessions during 30 seconds:

$ asadmin enable --retiretimeout=30 foo:1.0

Version "foo:2.0" stays retired to serve existing clients during 30 seconds, while "foo:1.0" becomes the active version again.
After all the attached sessions have expired, or after the given retirement time (e.g. 30 seconds), whichever comes first, "foo:2.0" is automatically disabled.

Note: the "foo:1.0" version has never been completely disabled during the process of retiring and rolling back, so no sessions are lost.

4.1.5.5. Disable an active version

A user wants to explicitely disable the version "foo:2.0" before the end of the retirement time of "foo:1.0":

$ asadmin disable foo:2.0

Three options are being considered:

a. The operation is not permitted while two versions are enabled: an error message shows up.

b. The active version (e.g. "foo:2.0") is actually disabled, but the retired version (e.g. "foo:1.0") stays on to serve its clients until all sessions complete, unless a positive retiretimeout value is specified, in which case this timeout would be honored.

c. Both versions are disabled, OR, the retired version becomes active again.

4.1.5.6. Undeploy an application

A user wants to explicitely disable and undeploy the retired version "foo:2.0" (this can happen before the end of the retirement time):

$ asadmin undeploy foo:2.0

Version "foo:2.0" is disabled and removed, while "foo:1.0" stays the only active version. The routing mechanism is disabled as only one version remains available.

4.1.5.7. List applications with their current state

A user wants to list all the deployed applications, with their current production redeployment related state.

$ asadmin list-applications --long

NAME      TYPE        STATUS    EXTENDED_STATUS   RETIRES_ON
foo:rc1  <web>       disabled
foo:1.0  <web>       enabled    active
foo:2.0  <web>       enabled    retired          2012-12-19 03:33:54

4.1.6. Rejected scenarios

Production redeployment relies on application versioning. Therefore, any scenario rejected by application versioning applies here.

4.1.6.1. Multiple retired versions

Production redeployment does not support multiple retired versions. This scenario is rejected.

  • A user has deployed the application "foo:1.0" using the regular command with versioning: 
    $ asadmin deploy --name foo:1.0 foo1.war
  • A user wants to upgrade the application "foo:1.0" to "foo:2.0", without loss of service, and to keep "foo:1.0" to serve clients with existing sessions during 10 minutes: 
    $ asadmin deploy --name foo:2.0 --retiretimeout=600 foo2.war
  • While "foo:1.0" and "foo:2.0" are already enabled, a user wants to upgrade to "foo:3.0", without loss of service, and to keep "foo:2.0" to serve clients with existing sessions during 15 minutes: 
    $ asadmin deploy --name foo:3.0 --retiretimeout=900 foo3.war

This raises an error, and deployment fails. Production redeployment does not support multiple retired versions.

To deploy "foo:3.0" and keep "foo:2.0" as retired, the user has to explicitely disable the already retired version (foo:1.0):

$ asadmin disable foo:1.0
$ asadmin deploy --name foo:3.0 --retiretimeout=900 foo3.war

After this command, "foo:3.0" is the active version, and "foo:2.0" is retired. After the given retirement time (e.g. 15 minutes), "foo:2.0" is automatically disabled.

4.1.6.2. Enabling a version with different context-root

Production redeployment cannot be used if the version to be deployed uses a different context-root than the current active version.

$ asadmin deploy --name foo:1.0 --contextroot foo foo1.war
$ asadmin deploy --name foo:2.0 --contextroot newfoo --retiretimeout=60 foo2.war

ERROR: active version of application "foo" uses a different context root : foo.

4.1.6.3. Enabling a version while no version is active

Production redeployment cannot be used if there is not already an active version.

$ asadmin list-applications --long

NAME      TYPE        STATUS    EXTENDED_STATUS   RETIRES_ON
foo:1.0  <web>       disabled
foo:2.0  <web>       disabled
$ asadmin enable --retiretimeout=60 foo:2.0

ERROR: there is no active version of application "foo".

4.2 Domain.xml

An attribute will be added to the application-ref element : retirement-time. This attribute will hold the date and time when the retirement period of a version ends.

An example of application version stored with its retirement period end time in domain.xml:

<application-ref ref="foo:1.0" virtual-servers="server" retirement-time="2012-12-19 03:40:29"></application-ref>

The date and time of the retirement period end are computed using the retirement timeout and the current date/time when enabling a version.

4.3. Bug/RFE Number(s):

None.

4.4. In Scope:

Support concurrent deployment of two active version.

Routing of the incoming requests to the appropriate version.

Only standalone web applications (.war) are supported by production redeployment.

4.5. Out of Scope:

The application availability is only guaranteed on the HTTP layer, that is for incoming HTTP requests only.

Enterprise applications (.ear) are not supported by production redeployment.

4.6. Interfaces:

4.6.1 Public Interfaces

The production redeployment is performed by the deploy or the enable command. Instead of disabling the currently enabled version and enabling the new one, both versions are enabled at the same time.

Only one option is added to the deploy and enable commands to trigger production redeployment:

  • The --retiretimeout=n option tells GlassFish to use production redeployment instead of the usual deployment. This means the current active version will NOT be disabled right now, and the version being deployed/enabled will be enabled concurrently. The version being deployed will be the new "active" version. The previous version will be the "retired" version, and will only be available to clients with an active session created by the said version, for the given retirement time (in seconds), or until there are no longer any session attached to this version (when n equals -1).
  • Interface: deploy
  • Comment Option
  • Comment: Add the --retiretimeout option to trigger production redeployment instead of the usual redeployment, and to configure the retirement time.
  • Interface: enable
  • Comment: Add the --retiretimeout option to trigger production redeployment instead of the usual redeployment, and to configure the retirement time.
  • Commands undeploy and disable will have to be modified to behave correctly regarding production redeployment. This means they should handle the cases of disabling/undeploying a version whether it is active or a retired.
  • Interface: list-applications
  • Comment: When used with --long option, display the production redeployment related informations : the version state (active or retired) and the retirement end time.
  • When the server is restarted, if there were retired versions which did not reach their retirement end time, they are loaded and enabled in the retired state until the retirement end time is reached. The active versions are loaded and enabled as usual.

4.6.2 Private Interfaces

None.

4.6.3 Deprecated/Removed Interfaces:

None.

4.7. Doc Impact:

Update man pages for the new/modified CLI commands

Application Deployment Guide/Developer Guide for new features

4.8. Admin/Config Impact:

CLI:

  • the retiretimeout option to be added to deploy and enable commands.
  • the production redeployment related informations, displayed when using the --long option.
    GUI:
  • provide a way to trigger production redeployment through GUI.
  • provide a way to view production redeployment related informations.

4.9. HA Impact:

No impact.

4.10. I18N/L10N Impact:

No impact.

4.11. Packaging, Delivery & Upgrade:

4.11.1. Packaging

No impact.

4.11.2. Delivery

No impact.

4.11.3. Upgrade and Migration:

No impact.

4.12. Security Impact:

No impact.

4.13. Compatibility Impact

No impact. The new feature will stay optional, so the previous behavior is kept as is.

4.14. Dependencies:

4.14.1 Internal Dependencies

None.

4.14.2 External Dependencies

None.

4.15. Testing Impact:

Add new tests (automated) to test production redeployment feature.

5. Reference Documents:

Production Redeployment Design Spec

6. Schedule:

6.1. Projected Availability:

  • Initially integrated (may not be feature complete) - 12/19/2012 (EE7 MS3)
  • Feature complete (ready for handoff to QA) - 1/16/2013 (EE7 MS4)
  • At production quality level - 3/13/2013 (EE7 MS6)