GlassFish Wiki : 3.1SSHProvisioningConfig

0.5

SSH Provisioning and domain.xml

GF v2 has node agents. GF 3.1 will have ssh nodes (systems that can be accessed via ssh). Beyond 3.1 we may once again have node agent support.

SSH node configuration information must be saved in domain.xml and we should do so in a fashion that can handle node agent nodes in the future. For reference here is a domain.xml snippet for a node agent from v2:

v2 node-agent snippet

 
        <server . . . name="myinstance1"  node-agent-ref="my_node_agent">
. . .
</server>

<node-agents>
  <node-agent name="my_node_agent" start-servers-in-startup="true" system-jmx-connector-name="system">
    <jmx-connector accept-all="false" address="0.0.0.0" auth-realm-name="admin-realm" enabled="true" name="system" port="42852" protocol="rmi_jrmp" security-enabled="true">
      <ssl cert-nickname="s1as" client-auth-enabled="false" ssl2-enabled="false" ssl3-enabled="true" tls-enabled="true" tls-rollback-enabled="true"/>
      <property name="client-hostname" value="nwc.Red.IPlanet.COM"/>
    </jmx-connector>
    <auth-realm classname="com.sun.enterprise.security.auth.realm.file.FileRealm" name="admin-realm">
      <property name="file" value="${com.sun.aas.instanceRoot}/config/admin-keyfile"/>
      <property name="jaas-context" value="fileRealm"/>
    </auth-realm>
    <log-service alarms="false" file="${com.sun.aas.instanceRoot}/logs/server.log" log-rotation-limit-in-bytes="500000" log-rotation-timelimit-in-minutes="0" log-to-console="false" retain-error-statistics-for-hours="5" use-system-logging="false">
      <module-log-levels admin="INFO" classloader="INFO" cmp="INFO" cmp-container="INFO" configuration="INFO" connector="INFO" corba="INFO" deployment="INFO" ejb-container="INFO" group-management-service="INFO" javamail="INFO" jaxr="INFO" jaxrpc="INFO" jdo="INFO" jms="INFO" jta="INFO" jts="INFO" management-event="INFO" mdb-container="INFO" naming="INFO" node-agent="INFO" resource-adapter="INFO" root="INFO" saaj="INFO" security="INFO" self-management="INFO" server="INFO" synchronization="INFO" util="INFO" verifier="INFO" web-container="INFO"/>
    </log-service>
    <property name="rendezvousOccurred" value="true"/>
  </node-agent>
</node-agents> 
       

For 3.1 we must introduce the concept of an ssh node. Below are two proposals for doing this. Option 1 refactors node-agent into a generic "node" config element with specialization for ssh and the traditional node agent. Option 2 keeps node-agent and just adds a different variation of it to support ssh nodes.

Option 1: refactor with generic element "node"

In this option we create a new element, the generic "node". A node can have zero or more connectors. In 3.1 we would support zero connectors or an ssh connector. In the future other connectors (like one to support legacy style node agents) could be supported.

The benefit to this option is it cleanly reflects the architecture of node and connectors. The negative is it is more work to implement since it refactors domain.xml.

 
        <server . . . name="myinstance1"  node-ref="ssh_node">
. . .
</server>

<nodes>

  <!-- An SSH node -->
  <node name="ssh_node" host="gf1.sfbay.sun.com" glassfish-home="/export/gf">
    <ssh-connector port="22">
      <ssh-auth type="key" username="dipol" keyfile="~/.ssh/id_dsa" />
    </ssh-connector>
  </node>

  <!-- A localhost node. No connector needed. -->
  <node name="mynode" host="localhost">
  </node>

  <!-- A localhost node from a different install location. -->
  <node name="mynode" host="localhost" glassfish-home="/export/apps/gf">
  </node>

  <!-- Placeholder node after create-node-config -->
  <node name="yournode">
  </node>

  <!-- No connector node after create-local-instance -->
  <node name="yournode" host="gf1.sfbay.sun.com" >
  </node>

  <!-- A node-agent node (post 3.1 if we do node agents) -->
  <node name="mynodeagent" host="gf2.sfbay.sun.com">
    <jmx-connector accept-all="false" address="0.0.0.0" auth-realm-name="admin-realm" enabled="true" name="system" port="42852" protocol="rmi_jrmp" security-enabled="true">
      <ssl cert-nickname="s1as" client-auth-enabled="false" ssl2-enabled="false" ssl3-enabled="true" tls-enabled="true" tls-rollback-enabled="true"/>
    </jmx-connector>
    <auth-realm classname="com.sun.enterprise.security.auth.realm.file.FileRealm" name="admin-realm">
      <property name="file" value="${com.sun.aas.instanceRoot}/config/admin-keyfile"/>
      <property name="jaas-context" value="fileRealm"/>
    </auth-realm>
    <agent start-servers-in-startup="true" system-jmx-connector-name="system">
      <log-service alarms="false" file="${com.sun.aas.instanceRoot}/logs/server.log" log-rotation-limit-in-bytes="500000" log-rotation-timelimit-in-minutes="0" log-to-console="false" retain-error-statistics-for-hours="5" use-system-logging="false">
        <module-log-levels admin="INFO" classloader="INFO" cmp="INFO" cmp-container="INFO" configuration="INFO" connector="INFO" corba="INFO" deployment="INFO" ejb-container="INFO" group-management-service="INFO" javamail="INFO" jaxr="INFO" jaxrpc="INFO" jdo="INFO" jms="INFO" jta="INFO" jts="INFO" management-event="INFO" mdb-container="INFO" naming="INFO" node-agent="INFO" resource-adapter="INFO" root="INFO" saaj="INFO" security="INFO" self-management="INFO" server="INFO" synchronization="INFO" util="INFO" verifier="INFO" web-container="INFO"/>
      </log-service>
    <agent>
    <property name="rendezvousOccurred" value="true"/>
  </node>
</nodes> 
       

Some things to note:

A generic node element "node" is now used to refer to a node of any type.
A new node connector element "ssh-connector" holds the ssh specific connection info
hostname is not in the connector, it is an attribute of "node".
A new element "agent" is introduced to support the other attributes and elements specific to node-agent
A node may have no connector information. This can be used for local nodes (i.e. hostname is localhost relative to the DAS) or for nodes where SSH won't be used, or for stubs (ala create-node-agent-config). A node stub can be populated by a subsequent run of "update-node" or via the GF console. By creating the stub we can support subsequent calls to create-instance which reference the node by name.
We don't need to nail down legacy node-agent configuration for 3.1 since that will likely change if/when legacy style node-agents are added in the future (for example they would not use JMX if re-indroduced).

Option 2: Tweak node-agent to support an SSH connector

In this design we make the smallest amount of change to the v2 node-agent config needed to support an SSH connector. The benefit to this design is it is a smaller, more incremental change.

 
        <server . . . name="myinstance1"  node-agent-ref="node_nwc">
. . .
</server>

<node-agents>

  <!-- An SSH node -->
  <node-agent name="node_nwc" host="nwc.red.iplanet.com" glassfish-home="/export/glassfish">
    <ssh-connector port="22">
      <ssh-auth type="key" username="dipol" keyfile="~/.ssh/id_dsa" />
    </ssh-connector>
    <property name="rendezvousOccurred" value="true"/>
  </node-agent>


  <!-- A node placeholder after create-node-agent-config (in v3.1) -->
  <node-agent name="mynode">
    <property name="rendezvousOccurred" value="false"/>
  </node>

  <!-- A node-agent node (post 3.1 if we do node agents) -->
  <node-agent name="my_node_agent" start-servers-in-startup="true" system-jmx-connector-name="system">
    <jmx-connector . . .>
      . . .
    </jmx-connector>
    . . .
    <property name="rendezvousOccurred" value="true"/>
   </node-agent>
</node-agents> 
       

Discarded design

Keep node-agents and ssh-nodes completely separate

In this design we keep ssh nodes and node agents completely separate.

 
        <server . . . name="myinstance1"  ssh-node-ref="node_nwc">
. . .
</server>

<server . . . name="myinstance2"  node-agent-ref="my_node_agent">
. . .
</server>

<ssh-nodes>
  <ssh-node name="node_nwc" host="nwc.red.iplanet.com" port="20" glassfish-home="/export/glassfish">
    <ssh-auth type="key" username="dipol" keyfile="~/.ssh/id_dsa" />
  </ssh-node>
</ssh-nodes>

<node-agents>
  <node-agent name="my_node_agent" . . . >
  . . .
  </node-agent>
</node-agents> 
       