GlassFish Server Open Source Edition 3.1 - How to make CLIs cluster aware This wiki page details how a CLI owner / developer can make use of the command replication framework.
- Contact tmueller@java.net for more information (original author was vijaysr@dev.java.net)
Introduction This document describes what a Glassfish CLI owner / developer should do to make his/her CLI cluster-aware using the command replication framework in 3.1. Overview The command replication framework has been designed to ensure that the effort involved to make existing CLIs cluster-aware is minimum. All existing CLIs are automatically replicated on all target instances unless otherwise specified (through the use of a simple annotation). Refer to this document for more information on the feature and the implementation details. Listed below are some steps a CLI owner may want to follow to ensure that their CLIs behave as expected in a clustered environment. Support for
option The first step is to ensure that the CLI implementation supports the
parameter that was specified by the user. The targets can be of the following types :
- "domain"
- "server" <- the default which represents the DAS
- config
- stand-alone-server instance
- cluster
- clustered instance
The details of what types of targets should be supported by each command for its
option has been documented in detail here. For example, the create-custom-resource command, supports "server" / "domain" / cluster / instance types as part of its
option. This information is for v2 and you have to support the same for v3. The CommandRunner framework will inject this parameter into CLI implementation through the
facility and the CLI implementation is expected to do its job as per the target type. To avoid code duplication, the CLi can use the following : This is a service, present in the internal-api module, org.glassfish.internal.api package, which a command implementation can get by using
@Inject Target targetUtil;
The CLI developer can use this to avoid "boiler plate code" (like getting list of server instances for a cluster, checking if a target is cluster or not, getting the cluster element given a name etc). The method names should be self explanatory and we are still in the process of adding more "services" that will help others avoid repeating same code. Here is an example of using the utility to obtain the Config for a target:
Config c = targetUtil.getConfig(target);
Elements for Cluster, Server etc have updated with some DuckTyped method which will also be useful for developers to avoid boiler plate code. If you have a specific requirement, please let us know and we will take care of it ASAP Enforcing target types allowed for commands Different commands allow different target types. As already mentioned, the create-custom-resource command, supports "server" / "domain" / cluster / instance types as part of its
option. But the create-http-lb command supports only a server instance or a cluster type as part of its
option. To ensure that the the user has specified only the valid target types for a command and to help avoid all CLI implementation do this check everywhere, the framework will enforce the target type rule specified by the CLI through
annotation that was specified in the CLI implementation. You can specify any combination of the following as part of the
annotation.
-
<- represents the target "domain"
-
<- represents the target "server"
-
CommandTarget.STANDALONE_SERVER
<- represents a stand alone server instance type
-
<- represents a cluster type
-
<- represents a config type
-
CommandTarget.CLUSTERED_INSTANCE
<- represents an instance that is part of a cluster
Since a vast majority of the commands support config type, "server", stand alone server and cluster types, by default the framework will check to ensure that the target specified by the user is one of
,
CommandTarget.STANDALONE_SERVER
,
,
. If not, the command fails and is not executed at all. So in effect, the absence of
annotation in the CLI implementation is equivalent to
@TargetType(CommandTarget.DAS,CommandTarget.STANDALONE_SERVER,CommandTarget.CLUSTER,CommandTarget.CONFIG
. For those commands that support other combinations of target will have to specify the list of types they support using the
annotation. For example, the create-http-lb command which supports only stand alone server or cluster as type will have to look like :
...
...
@TargetType(CommandTarget.STANDALONE_SERVER, CommandTarget.CLUSTER)
@ExecuteOn(.....)
@Service(..)
public class ....... {
}
Enabling operations on instance(s) that are part of cluster By default, all commands on a server instance that is already part of a cluster will be disallowed. For example, if you have a cluster with instances in1, in2 and a stand-alone-server in3 and you want to deploy an application. The framework will allow
command with
c1 or in3 but not in1 or in2 (as shown below). This behavior is to ensure that all instances that are part of a cluster will remain in sync.
$ asadmin deploy --target c1 ~/servletonly.war
Application deployed successfully with name servletonly.
Command _deploy executed successfully on server instance in1
Command _deploy executed successfully on server instance in2
Command deploy executed successfully on server instance in1
Command deploy executed successfully on server instance in2
Command deploy executed successfully.
$ asadmin deploy --target in1 ~/servletonly.war
remote failure: The deploy command is not allowed on target in1 because it is part of cluster c1
Command deploy failed.
The above mentioned default behavior will work for a vast majority of the commands. But there are a handful of commands which require operations to be allowed on instance(s) that are already part of a cluster. These few commands that plan to allow operations on a clustered instance will have to ensure that the command implementation code has
with
CommandTarget.CLUSTERED_INSTANCE
along with other supported types. One example will be the enable command which should allow user to enable an application that is already on a clustered instance. To let such commands override the default behavior of framework, the implementation of
command has to specify
annotation as listed below :
@Service(name="enable")
@I18n("enable.command")
@TargetType(CommandTarget.DAS, CommandTarget.STANDALONE_INSTANCE, CommandTarget.CLUSTER, CommandTarget.CLUSTERED_INSTANCE)
@ExecuteOn(value={RuntimeType.DAS, RuntimeType.INSTANCE})
@Scoped(PerLookup.class)
public class EnableCommand extends StateCommandParameters implements AdminCommand {
...
...
}
CLI implementors responsibilities As discussed above, the framework takes care of most of the common code that are required for all commands. So when the CLI implementation's
method is called, the implementor can assume that
- the parameters, including the
, have been validated (and help message displayed in case of wrong / missing operands)
- the target type enforcements have been done
- all parameters have been injected into the CLI implementation as long as the implementation uses the
annotation facility appropriately. (Note that the target name specified in the command line is also injected into the CLI implementation if there is
present)
The rest of the work required to effect the required changes (like changing config objects or password files or LB config) should be done by the CLi implementation. The
service may be useful to get access to various config elements but the actual updates to be done is the implementor's responsibility. Because the same command gets replicated on DAS and instances, if the CLI effects different changes on DAS and instance (this is rare), then it is the implementor's responsibility to take care of that also. The implementations for create-http-listener and create-virtual-server may be good references. Replicating commands Annotation To ensure that your CLI implementation is replicated on all target instances, the CLI implementation of
should be annotated with
. Here is an example :
@ExecuteOn({RuntimeType.DAS, RuntimeType.INSTANCE})
@Service("create-jdbc-resource")
public class CreateJdbcResource implements AdminCommand {
}
The presence of the
annotation in the CLI implementation class will indicate to the command replication framework that this CLI has to be replicated on all applicable instances as specified in the
option. Existing CLI Since a majority of existing CLIs need replication, the command replication framework will replicate the command on all instances by default (i.e. if there is no
specified in the CLI implementation class). For example, look at source code for CreateJdbcConnectionPool and DeleteJdbcConnectionPool commands. You will not see any @ExecuteOn annotation and these commands get replicated without any other changes. New CLI If a new CLI is being added to 3.1 and that CLI needs replication, then the developer can choose either to use the default behavior or specify the
annotation as shown in the sample code above. Avoiding replication Since the default behavior is to replicate the command on DAS and all applicable instances, if you want your CLI to be executed only on DAS or only on Instances, then you would use
with the
set as required. For example, the
command should be run only on DAS, not on the instances. Another example will be
commands which can get all information from DAS itself and hence need not be replicated. The code for CLI that implements
, the StopDomainCommand class, will look like this :
@Service(name="stop-domain")
@Async
@I18n("stop.domain.command")
@ExecuteOn(RuntimeType.DAS)
public class StopDomainCommand implements AdminCommand {
}
This fisheye link shows that actual change that has been done to make the command
run on DAS only. Note that specifying
@ExecuteOn(RuntimeType.Instance)
, will result in the execution of the command only on the instances. Specifying failure options In addition to RuntimeType, you can specify the following additional attributes with the @ExecuteOn annotation ifFailure By default, if there are errors during the execution of a command on an instance, the whole command execution is deemed to have failed and further execution is stopped. However, the CLI developer can change this default behavior and choose to ignore the failure or warn the user rather than stopping further command execution by specifying the ifFailure attribute and setting it to FailurePolicy.Ignore or FailurePolicy.Warn. For example:
@ExecuteOn(value={RuntimeType.DAS}, ifFailure=FailurePolicy.Warn)
ifOffline By default, if an instance is found to be offline during the command replication process, the whole command execution is deemed to have failed. However, the CLI developer can change this default behavior and choose to ignore the failure or warn the user rather than stopping further command execution by specifying ifFailure attribute and setting it to FailurePolicy.Ignore or FailurePolicy.Warn. For example:
@ExecuteOn(value={RuntimeType.DAS}, ifOffline=FailurePolicy.Ignore)
Doing pre/post processing for a command Some commands may require some pre/post processing. For example, when an application gets deployed, references have to be created in all applicable instances and the instances have to be forced to sync up with the DAS. Another example, will be that MQ or LB settings/properties may have to be reconfigured whenever an instance is added to a cluster. For such use cases, the command replication framework has introduced the @Supplemental annotation. An implementation must use the value() attribute of the @Supplemental annotation to express the supplemented command. Its value is the name of the command as defined by the supplemented command's @Service annotation. Take for example, the post-processing required for deployment command. The deployment command implementation looks like this :
@Service(name="deploy")
@ExecuteOn(RuntimeType.DAS)
public DeployCommand implements AdminCommand {
//Do Actual Deployment
}
Now a supplemental command, that wants to be executed everytime after a successful deployment will look like this :
@Service(name="DeploymentSupplementalCommand")
@Supplemental("deploy")
@ExecuteOn(RuntimeType.INSTANCE)
public DeploymentSupplementalCommand implements AdminCommand {
//Do logic that happens after deployment has been done
}
Note that the supplemental command implementation implements
. Hence it can use @Param facility and expect the corresponding parameters specified in CLI to be injected at runtime. The parameter values available for injection are the same ones that were provided for the main, original command with which the supplemental command is associated. So, for example, the DeploymentSupplementalCommand will have access to the parameter values that were available to the DeployCommand invocation. Let us look at one more example, that of changing MQ/LB properties when an instances is added to a cluster. The source for CreateLocalInstanceCommand class may look like this :
@Service(name = "create-local-instance")
@Scoped(PerLookup.class)
public final class CreateLocalInstanceCommand extends CreateLocalInstanceFilesystemCommand {
//Do local instance creation
}
And a supplemental command that wants to do some action after the successful completion of a local instance creation will look like this:
@Service(name="CreateLocalInstanceSupplementalCommand")
@Supplemental("create-local-instance")
public CreateLocalInstanceSupplementalCommand implements AdminCommand {
//Change MQ/LB properties here
}
Note that a CLI command can be supplemented with multiple supplemental commands. In such a case, all supplemental commands will be executed after the completion of the main command but without any guarantee on the order in which the supplemental commands are run. Specifying failure options for Supplemental commands ifFailure By default, if there are errors during the execution of a supplemental command, the whole command execution is deemed to have failed. However, the CLI developer can change this default behavior and choose to ignore / warn the user rather than stopping further command execution by specifying
attribute and setting it to
or
Timing the execution of SupplementalCommand By default, a supplemental command that is registered for a "main" command, will be called for execution after the "main" command execution is completed. However, the CLI developer can force the supplemental command to be executed before the execution of the main command by using the attribute setting
attribute of
to
Supplemental.Timing.Before
. Specifying where to execute the SupplementalCommand By default, the supplemental commands are executed on DAS and all applicable instances. There may be scenarios (like the deployment supplemental command mentioned above) where a CLI developer wants the supplemental command to be executed only on DAS or only on instances. For this the developer will annotate the supplemental command implementation class with
@ExecuteOn(RuntimeType.{DAS|INSTANCE})
as the case may be. A command or supplemental command might need to invoke another command (for example, code running on the DAS might need to trigger a different command on one or more instances). Such invocations might use the
which accepts a ParameterMap to pass parameters and their values to the command. The
creates a new ParameterMap populated using the parameters and values of a given other
which has already been injected. You can pass an optional
listing the parameter names from the specified command you want excluded from the new
. Contact For more help, suggestion for improvements etc contact [] FAQs Is there a description of the semantics of using --target? The semantics is the same as v2. Basically you can specify one target - that target can be a cluster/stand alone instance/domain/config. Different commands accept all or few of these 4 values. Which command can take what "type" of target is documentedhere. For example, if you take, create-custom-resource command, the doc says that the --target option can take server / domain / cluster / instance types. How do I pass information between main and supplemental command implementations There may be scenarios where you want to exchange some information between supplemental and main commands. For example, deploy command has to pass information on generated directory, application name etc to its supplemental command. This is done by :
|
