GlassFish Wiki : 3.2Dcom

MS-RPC Microsoft Remote Procedure Call
OSI - Open Systems Interconnection
COM - Component Object Model
DCOM Distributed COM
CORBA - Common Object Request Broker Architecture
DCE/RPC - Distributed Computing Environment / Remote Procedure Calls
MSRPC - Microsoft Remote Procedure Call (Microsoft's enhancement of DCE/RPC)
NT - Windows Platform – New Technology (not so new now!)
NTLM - NT Lan Manager
NTLMSSP - NT LAN Manager Security Support Provider

2. Project Summary

What we are proposing is using DCOM for communicating with Windows directly. It is a replacement for SSH on Windows.

2.1. Project Description:

Change how we do inter-host communications in a Windows environment.
This is a change to just the distributed communication that is currently done with SSH and SCP. This doesn't change anything related to GMS or asadmin command replication via HTTP.

2.2. Risks and Assumptions:

DCOM is a very complex part of Windows. It will likely take a surprisingly large amount of effort to produce a production-quality solution.

Assumption:

I will have access to the appropriate hardware so that I can develop and test on multiple platforms
I will be able to purchase reference materials and have time to do the necessary research about the nitty-gritty details of DCOM on Windows.
I will be given access to Microsoft's IDE for developing DCOM apps in different languages. We will not put any such native code applications into our installations but it will be essential for developing the GlassFish changes to support DCOM directly on Java.
There will be no legal or other problems with using JInterop, an open source utility in GlassFish.

3. Problem Summary

Make configuring a Distributed GlassFish installation on Windows servers just as easy as on UNIX or Linux.

3.1. Problem Area:

We currently support using SSH on Windows in order to run a distributed GlassFish system of multiple instances and/or clusters. SSH is not native to Windows. Through bitter experience we have learned that getting SSH working smoothly and perfectly on Windows is extremely difficult. Moreover it is the user himself whom is forced to do a lot of complicated configuring and software installation.

3.2. Justification:

It is too difficult for Windows users to configure and use GlassFish 3.1 for Windows environments. We need to provide an easier all-Java solution for Windows that leverages the software built into Windows. Windows has a rich architecture and history for providing tools for inter-host communications. We need to start using these tools.

4. Technical Description:

DCOM: Distributed Component Object Model
DCOM is used for communications between Windows computers. It is the Windows equivalent of CORBA or RMI. It is used with TCP/IP and HTTP. DCOM is available on all supported Windows platforms. It is pre-installed and ready to go unlike SSH.

4.1. Details:

How do I know that DCOM will work?
Hudson uses DCOM to provision Windows slaves and then uses it to communicate back and forth with them.
Hudson uses Jinterop, an open source Java-DCOM library for communicating with DCOM directly from Java.

So in other words Hudson was the guinea pig and has provided the proof of concept and guarantees that all of the technology pieces will work together correctly.

DAS contacts the DCOM server on the remote machine. The DCOM server is built-in to Windows and is normally running by default. It is in fact running as a Windows Service.
DAS orders the DCOM server to fire-up a GlassFish program (most likely it will simply be the asadmin script) that will in turn start the instance.

As you can see it is conceptually a literal plug-in replacement for SSH.

4.1.1 Sample Usage

Here is a simple example to give you the flavor of the j-interop API. This example will write to the Windows Registry.

 
        // The 3 args used below are Strings that you supply for auth
            IJIAuthInfo authInfo = new JIDefaultAuthInfoImpl(domainName, userName, password);

            IJIWinReg registry = JIWinRegFactory.getSingleTon().getWinreg(authInfo,args[0],true);
            //Open HKLM
            JIPolicyHandle policyHandle = registry.winreg_OpenHKLM();
            //Open a key here
            JIPolicyHandle policyHandle2 = registry.winreg_OpenKey(policyHandle,"Software\\Classes",IJIWinReg.KEY_ALL_ACCESS);

            System.out.println("Printing first 1000 entries under \"Software\\Classes\"...");
            for (int i = 0;i < 1000;i++)
            {
                String[] values = registry.winreg_EnumKey(policyHandle2,i);
                System.out.println(values[0] + " , " + values[1]);
            }
            System.out.println("****************************************************");
            String key = "MyNewRegistryKey";            System.out.println("\nCreating Key " + key + " under \"Software\\Classes\"...");
            JIPolicyHandle policyHandle3 = registry.winreg_CreateKey(policyHandle2,
                   key,IJIWinReg.REG_OPTION_NON_VOLATILE,IJIWinReg.KEY_ALL_ACCESS);

            System.out.println("Setting values to " + key);
            registry.winreg_SetValue(policyHandle3,"j-Interop_None");
            registry.winreg_SetValue(policyHandle3,"j-Interop_String",".".getBytes(),false,false);
            Object[] values1 = registry.winreg_QueryValue(policyHandle3,"j-Interop_String",1024);
            registry.winreg_SetValue(policyHandle3,"j-Interop_String_Ex","%PATH%\\Test12345".getBytes(),false,true);
            registry.winreg_SetValue(policyHandle3,"j-Interop_Bin","123456789".getBytes(),true,false);
            registry.winreg_SetValue(policyHandle3,"j-Interop_Dword",100);

            String[] strings = {"123", "456", "6789", "10","11"};
            byte[][] data = new byte[strings.length][];
            for (int i = 0; i < strings.length;i++)
            {
                data[i] = strings[i].getBytes();
            }

            registry.winreg_SetValue(policyHandle3,"j-Interop_Multi_sz",data);

            for (int i = 0; i < 6;i++)
            {
                Object[] values = registry.winreg_EnumValue(policyHandle3,i);
                System.out.println(values[0] + " , " + values[1]);
            }

            System.out.println("Retrieving j-Interop_String_Ex value " + key);
            Object[] values = registry.winreg_QueryValue(policyHandle3,"j-Interop_String_Ex",1024);
            System.out.println(new String((byte[])values[1]));

            System.out.println("Deleting j-Interop_Bin value");
            registry.winreg_DeleteKeyOrValue(policyHandle3,"j-Interop_Bin",false);

            System.out.println("Saving the " + key + " in a file to local server location as c:\\temp\\j-Interop");
            registry.winreg_SaveFile(policyHandle3,"c:\\temp\\j-Interop");

            registry.winreg_CloseKey(policyHandle3);
            registry.winreg_CloseKey(policyHandle2);
            registry.winreg_CloseKey(policyHandle);
            registry.closeConnection();

// 
       

4.2. Bug/RFE Number:

RFE

4.3. In Scope:

here are the 6 SSH related asadmin commands in GlassFish 3.1

create-node-ssh
delete-node-ssh
list-nodes-ssh
ping-node-ssh
setup-ssh
update-node-ssh

Note the unfortunate wiring in of the name of the actual implementation instead of using something more generic. Then we could have simply slipped in DCOM. In order to avoid confusion we'll have to implement 6 new commands. It would be too confusing to setup DCOM, for instance, by calling setup-ssh. The plan is to introduce these new commands:

create-node-dcom
delete-node-dcom
list-nodes-dcom
ping-node-dcom
update-node-dcom
setup-dcom

These existing commands' API shall be modified to support DCOM:

install-node
uninstall-node

Lifecycle commands will work using DCOM as appropriate with no API change. E.g.

create-instance
start-instance
stop-instance
delete-instance

4.4. Out of Scope:

N/A

4.5. Interfaces:

4.5.1 Public Interfaces

As mentioned before the new public interfaces are the new asadmin commands discussed above.
Note that commands that need a password get them precisely the same way as the corresponding ssh commands get them. Namely by either a password file passed in as the -W parameter to asadmin. Or by the user typing it in at the command line when requested interactively. The password will be saved in domain.xml if appropriate exactly like it is done for ssh commands.
The variable in the password file is:
AS_ADMIN_DCOMPASSWORD

4.5.1.1 create-node-dcom

The create-node-dcom subcommand creates a node that is enabled for communication over DCOM.

By default, the subcommand fails and the node is not created if the DAS cannot contact the node's host through DCOM. To force the node to be created in the DAS configuration even if the host cannot be contacted through DCOM, set the --force option to true.

Options:

--nodehost required The name of the remote host
--installdir optional The remote GlassFish product installation dir. Note this is the parent directory of the GlassFish directory.
--nodedir optional The name of the parent directory for instance directories belonging to this node
--dcomuser optional The name of the remote Windows user default=the current username
--force optional Force the creation of this node even if there are connection errors
--install optional Specifies whether or not to also install GlassFish on the remote host
--archive optional Specifies the GlassFish installation archive file.
--windowsdomain optional Specifies the Windows Domain Name default is the hostname
name the node name

4.5.1.2 delete-node-dcom

The delete-node-dcom subcommand deletes a node that is enabled for communication over DCOM.

Options:

--force optional Forces the deletion of this node
--uninstall optional Specifies whether or not to uninstall GlassFish on the remote host
name the node name

4.5.1.3 list-nodes-dcom

The list-nodes-dcom subcommand List the nodes that are enabled for communication over DCOM in the domain.

Options:

-l|--long optional Provide more information for each node

4.5.1.4 ping-node-dcom

The ping-node-dcom command verifies end-to-end communications using DCOM with a remote host.

Options:

--validate -v optional Verifies that GlassFish is installed and operational on the remote host
name the node name

4.5.1.5 update-node-dcom

The update-node-dcom command updates the configuration data of a DCOM node.

Options:

--nodehost optional The name of the remote host
--installdir optional The remote GlassFish (parent) installation dir
--nodedir optional The name of the parent directory for instance directories belonging to this node
--dcomuser optional The name of the remote Windows user default=the current username
--windowsdomain optional The name of the Windows Domain the user is a member of. default=the nodehost
--force optional Force the modification of this node
name the node name

4.5.1.6 setup-dcom

No setup per se is required. The user has to make sure that the DCOM ports (135, 445) are available and that the Server Service is enabled. What this command does is test the connection with the remote host by copying a file to the remote host and running a script. The name is re-used to make it look similar to setup-ssh.

Note: It could be renamed to test-dcom if desired?

Options:

--dcomuser optional The name of the remote Windows user default=the current username
--windowsdomain optional Specifies the Windows Domain Name default is the hostname
--remotetestdir _optional Specifies a writable and accessible folder on the remote host default is C:_

4.5.1.7 install-node

I'm not going to duplicate existing documentation here. Instead here are the changes:
Note that it will use DCOM to copy files and run a script on the remote Windows node.

Added Options:

-d|--dcom optional Make the new node a DCOM node
-w|--windowsdomain optional Specifies the Windows Domain Name default is the hostname
--dcomuser optional The name of the remote Windows user default=the current username (Note: this is an alias. It does not show up in automatically generated help displays. --sshuser wil also work)

Different Meanings for Existing Options

--sshport disallowed
--sshkeyfile disallowed
--sshuser accepted as an alias for --dcomuser

4.5.1.8 uninstall-node

I'm not going to duplicate existing documentation here. Instead here are the changes:

Added Options:

--dcomuser optional The name of the remote Windows user default=the current username (Note: this is an alias. It does not show up in automatically generated help displays. --sshuser wil also work)

Different Meanings for Existing Options

--type Here you must enter the value: "DCOM" as the type. (This is strange-looking but backwards-compatible)
--sshport disallowed
--sshkeyfile disallowed
--sshuser accepted as an alias for --dcomuser

NOTE FOR doc team – the "type " parameter is not documented in the help pages.

4.5.2 Private Interfaces

No new private interfaces that are observable externally. Here is a snippet from domain.xml showing the actual generated element from creating a node:

   <node node-host="wnevins-lnr" name="lnr" windows-domain="wnevins-lnr" type="DCOM"
            install-dir="${com.sun.aas.productRoot}">
      <ssh-connector ssh-port="135">
        <ssh-auth password="REAL_PASSWORD_HERE"></ssh-auth>
      </ssh-connector>
    </node>

4.5.3 Deprecated/Removed Interfaces:

None.

4.6. Doc Impact:

Rather extensive new documentation will be required for this feature.

4.7. Admin/Config Impact:

The Admin Console will need to support the new DCOM configuration and commands.
The GlassFish Configuration will require new elements.

4.7.1 Admin Console Support:

Admin Console will be enhanced to support the new commands related to DCOM. User can list, create, edit and delete DCOM nodes.

The table that lists out existing Nodes will include DCOM nodes and user can differentiate different types of nodes by the Type Column. Listing Table screenshot . An action column will be added, with Ping being the action available. This allows user to ping the existing Node.

Another button which brings user to the 'validate dcom' page will be added to the listing table.

This allows user to test if a remote Windows machine is setup correctly for DCOM connectivity.

The Node creation screen will add the option of "DCOM" to the Types dropdown so user can select to create a DCOM node. The 2 optional fields related to DCOM will be available as a text box for user to fill in if desired. Create DCOM Node screenshot

Currently user can switch from CONFIG node to SSH node and vise versa after the node has been created. With the addition of DCOM nodes, user can switch between CONFIG, SSH and DCOM, and the appropriate fields will be displayed.

In 3.1.1, install-node and uninstall-node is not supported in the console because it is local command. If these commands can be changed to remote command in 3.1.2, console may add in support to that. If thats the case, install-node-dcom and uninstall-node-dcom will be supported too.

4.8. HA Impact:

None.

4.9. I18N/L10N Impact:

No impact other than the usual for any new code.

4.10. Packaging, Delivery & Upgrade:

4.10.1. Packaging

A new external package is used to OSGi-ify j-interop, jinterop-deps and jcifs.
The name of the jar is: j-interop-repackaged.jar

The location in the source code is:
https://svn.java.net/svn/glassfish~svn/trunk/main/nucleus/packager/external/j-interop

https://svn.java.net/svn/glassfish~svn/branches/3.1.2/packager/external/j-interop_

4.10.2. Delivery

No impact.

4.10.3. Upgrade and Migration:

No backward compatibility issues. There will be no upgrade issues since this is a new feature.

4.11. Security Impact:

Of course we will have to be careful to maintain good security since it involves communications between hosts.

4.12. Compatibility Impact

Old interfaces remain the same. We are adding new commands and functionality - not modifying the existing behavior.

4.13. Dependencies:

4.13.1 Internal Dependencies

Common utilities.

4.13.2 External Dependencies

 <dependency>
            <groupId>org.jinterop</groupId>
            <artifactId>j-interop</artifactId>
            <version>2.0.4</version>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.jinterop</groupId>
            <artifactId>j-interopdeps</artifactId>
            <version>2.0.4</version>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.samba.jcifs</groupId>
            <artifactId>jcifs</artifactId>
            <version>1.2.19</version>
            <optional>true</optional>
        </dependency>

The Oracle Legal Department is working on the licensing of the open source projects that we require. Progress is available at an Oracle internal wiki page.

4.14. Testing Impact:

A lot of testing will be required. This will be challenging to create Dev Tests. It would definitely require several dedicated Windows machines to make that happen.

Oracle Wide Area Network configured machines are a worst-case scenario. Security on Windows machines is extremely high. DCOM access is disallowed by Oracle IT. I think that QA will need to go off of OWAN completely and run machines on private local LANs.

4.15. JIRA Issues

A new component was created and it should be used for filing all issues: distributed management

5. Reference Documents:

JInterop
JInterop JavaDoc
Subversion URL for j-interop trunk
Subversion URL for j-interopdeps trunk

5.1 Building External Projects from Source

5.1.1 JCIFS (SAMBA)

The source is not available in a version control system. The version that j-interop uses is 1.2.19 from May 22, 2008. Below are the URLs to get the source zipped into a jar and the javadoc.

6. Schedule:

October 14, 2011 - Complete with no known P1 bugs. One-Pager updated. DevTests created. Handoff One-Pager to QA and DOC teams. Asarch approved.

7. Reviewer Comments

Attachments:

create-dcom-node.png (image/png)

nodes.png (image/png)