GlassFish Platform as a Service Functional Specification

Contributors:
  Jerome Dochez
  Tom Mueller
  Sanjeeb Sahoo
  Sivakumar Thyagarajan

Introduction

(slides)

The goal of this document is to provide high level understanding of GlassFish Platform as a Service (PaaS) feature design. This feature will provide a Java EE centric solution to cloud computing that enables a user to deploy a Java EE application to the cloud, with the system allocating and provisioning virtual machines within the cloud to run the application, run the services required by the application, provide load balancing, and dynamically scale the application deployment as needed. 

This document is a master plan that includes descriptions of features that may not be included in the first release of this feature. 

This document is specific to the GlassFish application server; It is not a goal to provide a extensible facility to any type of cloud enabled application server. It is also not a goal to provide extensible management of the cloud infrastructure, it's basically allowing GlassFish to be plugged into an existing virtualization infrastructure (the feature will support many) and offer a Java EE centric way of deploying and managing applications without deep understanding of cloud infrastructure.

PaaS Feature Architecture

The PaaS feature is set of services that will be available within a single process or as a set or processes running on different machines or virtual machines communicating with a well defined set of interfaces. At the core, the PaaS can be seen as an extended DAS where virtualization functions and features are added.

The feature set for the PaaS feature includes:

• Simplify application deployment by
  • allowing services that are required by an application to be inferred from the application,
  • allowing services to be used by an application to be specified as part of the application deployment plan,
  • providing the ability to provision the services for an application based on virtual machines allocated from a cloud,
  • deploying an application to a cluster that is created for the application,
  • configuring the application to use the provisioned services.
• Simplify application management by
  • providing application auto-scaling by monitoring application performance and adjusting the resources that are available to the application automatically,
  • providing the ability to customize the decision making process regarding auto-scaling,
  • providing complete application services such as backup, backing up not just the domain data, but the service data too.
  • extending cloud-based application management to the complete application lifecycle (list, enable, disable, undeploy). 
• Simplify domain creation by
  • allocating a virtual machine and deploying a DAS to it on behalf of a user,
  • support the ability to run the PaaS feature itself in the cloud.

In each case, the user is obviated of the need to manage the system at the infrastructure (machines, networking, software, storage) level.  The Java EE platform is presented as a service, allowing the user to be concerned with running a Java EE application rather than provisioning nodes, clusters, instances, resources, services, etc. The Java EE application is the primary concern of the user, making these other concerns secondary. 

The feature must support:

• the ability to make use of various underlying infrastructure as a service (IaaS) or virtualization providers,
  • OVM
  • Amazon Web Services
  • VirtualBox
  • Xen
  • KVM
  • etc.
• the ability to support various services that are used by Java EE applications:
  • Java EE container (GlassFish only)
  • JMS (OpenMQ only)
  • load balancer (various vendors)
  • database (various vendors)
  • etc.

This list of which IaaS providers and which services are supported in a particular release will be part of the release plan. The architecture must be independent of which specific ones are selected.

Considering that many of the tasks above are already implemented by the DAS, the PaaS architecture is implemented on top of DAS with the following added functionalities.

• offer a management API (presence of higher level software like Enterprise Manager might allocate/deallocate virtual machines based on other requirements that neither the DAS or Applications can express).
• enhance deployment backend to support deployment of cloud applications 
• defines APIs and Plugins to talk to the virtualization infrastructure.

Definitions

Node = Operating system instance running on either a virtual image or on bare metal.
Instance = GlassFish instance running on a node.

Architectural Components 

The PaaS feature consists of the architectural components described in this section.  These components are services that reside within an instance of GlassFish server, but depending on the configured topology, they may or may not be within the same server.

To provide this flexibility, the architecture needs a way to define a module that can be accessed either locally or remotely without significantly changing the client software. There are several technologies available in GlassFish to do this, including RMI, MBeans/JMX, AdminCommands/__asadmin via HTTP and SSH where necessary, and REST. There are several advantages to using AdminCommands as the mechanism for this feature: they are the foundation for building the GlassFish clustering feature, the command replication framework provides the ability to run an asadmin command using the same API whether the command is local or remote, and the framework has the ability to support secure communication via SSL client certificates.  The implementer of the function just has to extend AdminCommand, while the user of the function just has use one of the classes in the RemoteAdminCommand hierarchy to run the command. 

For this feature, we could implement a class that provides better support for running a command independently from where it is located (either local or remote). We might also need to enhance the security layer to deal with multiple certificates, for the case where the DAS from multiple domains is communicating with a centralized IaaS management service.

This design makes the unit of development (the module) for the feature AdminCommand. There will be lower level commands (possibly hidden) for creating VMs, provisioning services, etc. and higher level  commands such as "deploy" would ultimately be running these lower level commands.  Any PaaS feature interface that potentially spans JVM boundaries would be expressed as an AdminCommand.

The diagram at the end of this sections shows how these components fit together. 

Orchestrator

The orchestrator is the top-level services that provides the ability to deploy an application to the cloud. It is responsible for:

• Service Dependency Discovery - Determining what services an application requires when it is deployed, either through implicit dependencies or through explicit+ dependencies specified through a clould.xml file. 
• Service Provisioning - Requesting the necessary VMs or servers from the virtualization management layer and provisioning the needed services on those VMs.
• Service Association - Associating the provisioned service with the application in GlassFish by defining resources.
• Deployment - Deploying the application within GlassFish so that it makes use of the provisioned services.

The orchestrator uses a set of service plugins to allow applications to be deployed with various services such as a database, JMS, etc. 

Elasticity Manager

The elasticity manager  is responsible for providing elasticity to the deployed application.  It is responsible for:

• Metric collection - Gathering data about application use and machine resources that is used as input to the decision making process.
• Scaling decisions - Make decisions about increasing or decreasing the number of instances used to run the application, or other methods of scaling such as informing users. 
• Action execution - Once a decision is made, the elasticity manager has to implement that decision.

IaaS Management Service (IMS)

The Infrastructure as a Service (IaaS) management service provides an abstraction of the IaaS cloud to the GlassFish software.  This abstraction encapsulates various APIs that are available for managing cloulds. 

The IaaS management service is responsible for carrying out the decisions made by the elasticity manager and the orchestrator. If the IaaS provides VM-level monitoring services, then the IaaS management service communicates information about IaaS decisions (such as moving a VM) to the elasticity manager, which then includes that information into its elasticity algorithms. It is capable of allocating and de-allocating virtual machines across all groups defined in its configuration. Virtual machines are allocated across groups (virtualization management decision tbd) but the IaaS management service is responsible for deciding on which bare metal hardware such virtual machines should run.

Groups

Machines are divided into groups. Groups can be geographically determined, or can be based on network boundaries. Such groups are composed of physical machines and share the same virtualization technology (libvirt, jclouds..). It is unclear how well such groups will map to public clouds offerings (at worst I suppose you have a single group).

The group will be managed by an IaaS management service which has the following APIs.

• group management (add, remove machines).
• activate/suspend machines.
• have pre-defined set of templates that can be used to virtual machines instantiation
• allocate/deallocate virtual machines
• monitor and expose monitoring data of allocated virtual machines

The IaaS management service will not duplicate the DAS responsibilities of maintaining a list of clusters or list of deployed applications. It's main responsibility is to provide bare metal and run VMs on it. I consider the IaaS management service stateless artifacts. The IaaS management service for libvirt should be implemented on top of GlassFish nucleus distribution.

Domains Manager

The Domains Manager is responsible for automating the process of creating a domain within the cloud.  The domains manager may be a simple command line interface that is used by the PaaS admin to create a domain on behalf of a GlassFish user, or it could provide a GUI console that would provide a single landing point for users to create a domain (or domains), and it could even provide a simplified interface for application deployment.

The domains manager is an optional part of the architecture, as the PaaS admin could manually allocate a VM from the cloud, create a GlassFish domain on it, and configure that domain to use the IMS. 

The domains manager could have the following functions:

For GlassFish Users:

• create account and/or login to the a console
• choose which cloud to use, collect cloud credentials and store them in the IaaS management service
• create a Java EE domain by using the IaaS management service to create a VM, put GlassFish there, create a domain, pass IaaS management service location to the domain. (This includes potentially choosing from different VM options (OS type, web/full distribution, etc.)
• direct the user to the console for the domain
• show a list of domains that the user has created
• delete a domain
• provide an integrated console to deploy applications directly to a domain, possibly hiding the existence of domains in the interface
• job scheduling for scheduled tasks with respect to the PaaS infrastructure - an example is backing up domains and configurations.
• reporting on monitoring of physical resources and providing a listener based notification for exhaustion of physical resources as cloud demand nears exceeding supply.

For the PaaS Administrator

• view the GlassFish users (and do everything a user can do)
• configure which IaaS management service is used
• provide a UI for the IaaS management service (this is TBD - the IaaS management service could have its own UI)

The domains manager acts as an interface point for users of the PaaS. It has the ability to create a VM for a DAS. But it doesn't analyze applications, create VMs for clustered instances, create VMs for other services. It doesn't have a pluggable interface for service providers.

The domains manager runs on a GlassFish DAS.

The domains manager lifecycle begins with a manual installation of GlassFish with the PaaS feature. A separate GlassFish domain is needed to run the domains manager itself, i.e., separate from the domains that are being managed by users of the domains manager. If this GlassFish domain is to be run within the cloud itself, a VM for the domain would need to be manually created using tools from the cloud infrastructure.

Domains Manager Data Store

The domains manager stores the following information

• user store configuration (how to authenticate and authorize users)
• IaaS management service configuration information (where is the IaaS Management Service to be used by this domains manager)
• per-user domain configuration information
  • where is the DAS (console URL, VM location)
  • what IaaS management service it is using

PaaS Feature Operation

The description of PaaS operation will refer to a set of services: Orchestrator, Elasticity Manager, IaaS Management and a Domains Manager.

Let's have a look on how these services fit together. In the diagram below, the boxes do not represent process boundaries but service boundaries.

DAS Creation

How the GlassFish cloud environment bootstraps itself depends on the deployment environment.

1. Domains Manager present. The user will register itself to domains manager and request for a domain creation. Domains manager will allocate a virtual machine and setup a DAS on that machine. It should be possible to run the DAS on bare metal. It should also be possible to attach an existing DAS to domains manager (tbd). Domains manager will need to configure the DAS with the right IaaS Management configuration so the DAS can request virtual machines.
2. Domains Manager not present. The user will need to install manually a DAS on bare metal or in a Virtual Machine and configure it appropriately with the right IaaS definitions so the DAS can use the IMS to allocate/deallocate virtual machines.
   Unknown macro: {gliffy}

Deployment

The most important difference with the PaaS Feature versus 3.1 is the deployment use case. In 3.1 the users were most of the time performing the following tasks when deploying an application to a multi-instance environments :

• create a cluster
• for each physical boxes, create a node
• for each node, create one or more instances in that cluster
• provision services needed by the application (databases, JMS, etc.)
• define resources within the application server to associate those services with application identifiers
• deploy applications to the cluster.

As you can see, the physical layout of the deployment is set before the deployment itself and the elasticity of the cluster can be achieved though manually adding instances to an existing cluster.

With the PaaS feature, the dynamics will be reversed where the hardware is mapped to a deployment request and the elasticity is entirely managed by the PaaS architecture (of course manual modification could still be possible, tbd).The following steps is how the system could handle a cloud deployment request.

• deploy an application
  • orchestrator pre-deploy (read DDs and annotations) + explicit cloud metadata to determine the required services and deployment environment (number of vms, types of vms).
  • orchestrator creates a cluster (similar to a 3.1 cluster, maybe invisible to users).
  • orchestrator requests to IaaS management service for allocation of Virtual Machine (type, number). By default, a round robin within the IaaS management service list might suffice but more elaborated algorithm (probably with metadata input) could be used.
  • IaaS management service will start hardware machines
  • IaaS management service will install the requested virtual machines types (in the started machines or underused ones, at his discretion).
  • IaaS management service will return the VMs information to the orchestrator
  • Orchestrator will use the new virtual machine information to provision services (via service plugins) and configure the cluster
  • Orchestrator will associate services with resources
  • Normal deployment follows.

It is still unclear at this point where the virtual machines templates will reside. It is entirely possible that we choose to associate template management at the IaaS management service level as it is usually a primary function of the same IT department (the IaaS admin responsibilities).

Elasticity

Elasticity will be mostly managed by the PaaS architecture within the DAS where instances can be provisioned or decommissioned following events, monitoring triggers. Let's have a look at the work the PaaS architecture will have to do to add a new GlassFish instance to a deployment environment.

• The Elasticity manager receives notification that application X needs a new instance (maybe through a deamon running, tbd)
• The Elasticity manager requests through the IaaS Management Service for a new Virtual Machine.
• The Virtual Machine is added to the Application X's domain as a GlassFish node and an clustered instance on that node
• Normal 3.1 replication to the new GlassFish instance follows.

Removing an instance is nothing more than removing a node from the cluster.

Role of Cluster

More than one application can be deployed to a set of virtual machines as they have dependencies on each other (or form a complete feature set). The current notion of cluster guarantees that all applications deployed to it will be mapped to the same GlassFish instances (deployment environment) and adding the elasticity characteristics to cluster will provide the ability to grow or shrink set of applications.

By default, application deployed without a cluster will be provided one (naming tbd).

The life-cycle of the cluster will be entirely managed though the life-cycle of an application space. When the last application of a cluster is undeployed, the DAS through the orchestrator will release all the virtual machines. High level steps of application space destruction :

• DAS undeploys the application from the domain (domain use tbd).
• DAS through the IMS: destroys the virtual machines, reclaim the disk space used by the VM.
• DAS deletes the cluster from its configuration.

There are two cases that require special processing:

1. Redeployment of the application - the data certainly needs to be preserved in this case, and there should be a way to reuse the provisioned services without recreating them, i.e., a redeploy is not an undeploy followed by a deploy. 
2. There must be a way to preserve the data from provisioned services. whether for backup or because one wants to put the application to "sleep" for a while and bring it back later. This would also imply the ability to some how restore data to a provisioned service.

Upgrade

There are several aspects of upgrade and update related to this architecture. 

For upgrade from a previous release of GlassFish that doesn't support this feature, users would not automatically be able to make use of any of the PaaS support features for applications that are already deployed to a domain.  Rather, the various services that are part of this feature would need to be configured in either a new system or an upgraded system, and then an application would need to be deployed again into the new cloud-based infrastructure. 

For update from one release of GlassFish to another where both support the Paas Support feature, an in-place update would be supported in the same way for a non-cloud-enabled system.  However, an improvement in the way that updates to GlassFish instances are accomplished is recommended, i.e., make it possible to update the GlassFish software on an instance from the DAS console. 

An upgrade of a Java EE application that is deployed to this infrastructure is handled by the Orchestrator component. The upgrade cannot just be an undeploy followed by a deploy, because this would result in the loss of service data in provisioned services.  The Orchestrator component will provide special handling for the application redeploy case. 

Divide the Responsibilities/World

As we need to offer an Open Source implementation of the PaaS feature, there is a need to define at least one plugin implementation (probably based on libvirt for private, on ? for public clouds). There is also a need to define a segmentation and representation of the physical assets (machines) so that a successful mapping between the (to be determined by Java EE spec) metadata is properly mapped to hardware boxes.

Roles

The following roles are important in the cloud space:

The IaaS Administrator

The IaaS admin is responsible for configuring bare metal and configure the machines that are managed by the IaaS management service. He will most likely be providing the virtual machines templates used to create GlassFish instances. Finally, he manages the access control on both the IaaS management service as well as the virtual machines. It is assumed that only the IaaS admin will have access to the target machines used to run the virtual machines. None of the other roles members will be able to connect to the bare metal or install software on it.

The admin tools used by the IaaS admin must support

• configuration for pointing the IMS at a particular cloud, either public or private, including setting up the API that is used to access that cloud,
• configuring the available VM templates, and
• configuring access control for the IMS.  If there is a DM, then the DM and the IMS have to trust each other. There also might be a user management layer involved, such the IMS needs to trust that user management layer. 

The PaaS Administrator

The PaaS admin will configure GlassFish and the domains manager following the allocations of the IaaS from the IaaS admin. His main responsibilities are :

• defines users and mapping from users to departmental DAS machines
• configure domains manager and/or DAS to connect to IMS.
• defines the elasticity requirements (tbd).

The GlassFish User

The GlassFish user is the administrator as we know it today in 3.1. He has complete access to domain configuration and can obviously deploy/undeploy applications.

In addition, if there is a Domains Manager console, a GlassFish user can create, list, and delete a domain from the Domains Manager, and depending on the capabilities of that console, may be able to deploy an application directly from that console. 

VM Creation

• Based on a template
• Templates are built with an external tool (Assembly Builder, etc...)
• Template content:
  • GlassFish installation
  • Users credentials so the DAS can connect to it (optional)
• Template characteristic:
  • Service provided (Java EE Service, Database Service, etc...)
  • Memory (can be altered, tbd)
  • Number of Processors (can be altered, tbd)
  • Disk space

TBD, do we need more sophisticated settings like number of disks attached to the template, size of disks, etc... ?

Topology examples

Simple

In this deployment, the DAS is compounding several roles :

• DAS functions (users management, deployment)
• Orchestrator (ability to deploy applications with provisioned services)
• Elasticity (responsible for managing Virtual Machines, creation, deletion, elasticity)
• IaaS management service (ability to create / delete virtual machines through a libvirt plugin
• Interface through the libvirt plugin to multiple machines compatible with libvirt (Kvm, Xen, VirtualBox ?, OVM ?)

All in the cloud

In this deployment, all the components of our PaaS infrastructure are allocated in virtual machines running on a public or private cloud. This represents significant challenges at implementation since the DAS or instances can be reassigned different IP addresses at each restart (reliance on a smart DNS, static IPs might be a solution). The All in the cloud solution is significantly different from the other that I repeated some of the actions happening on domain creation/deployment.

Hybrid

In the this hybrid deployment, virtual machines are allocated both on a pool of privately owned machines as well as some public virtual machines purchased from external providers.

Inter-departmental cloud

In this complicated deployment, there are several DAS instances serving different business units with the Domains Manager fronting them. Again the exact location of the IaaS management service is still not defined.

services.png (image/png)
services.png (image/png)
simpleCloud.png (image/png)
hybridCloud.png (image/png)
InterDeptCloud.png (image/png)
allInCloud.png (image/png)
deployment.png (image/png)
deployment.png (image/png)
deployment.png (image/png)
DAS Creation.png (image/png)
services.png (image/png)
deployment.png (image/png)
elasticity.png (image/png)
InterDeptCloud.png (image/png)
InterDeptCloud.png (image/png)
hybridCloud.png (image/png)
simpleCloud.png (image/png)
elasticity.png (image/png)
elasticity.png (image/png)
allInCloud.png (image/png)
DAS Creation.png (image/png)
services.png (image/png)
deployment.png (image/png)
services.png (image/png)
InterDeptCloud.png (image/png)
deployment.png (image/png)
services.png (image/png)
services.png (image/png)
services.png (image/png)
services.png (image/png)
services.png (image/png)
DAS Creation.png (image/png)
deployment.png (image/png)
elasticity.png (image/png)
elasticity.png (image/png)
simpleCloud.png (image/png)
allInCloud.png (image/png)
allInCloud.png (image/png)
hybridCloud.png (image/png)
InterDeptCloud.png (image/png)
services.png (image/png)
services.png (image/png)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
services.png (image/png)
DAS Creation.png (image/png)
DAS Creation.png (image/png)
DAS Creation.png (image/png)
deployment.png (image/png)
elasticity.png (image/png)
simpleCloud.png (image/png)
allInCloud.png (image/png)
hybridCloud.png (image/png)
InterDeptCloud.png (image/png)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
GlassFish PaaS Overview.odp (application/vnd.oasis.opendocument.presentation)
services.png (image/png)
allInCloud.png (image/png)
deployment.png (image/png)
deployment.png (image/png)
elasticity.png (image/png)

Is the Domains Manager now called Group Manager? (Or is Group Manager something else?) Posted by edbratt at Mar 23, 2011 12:22

No. The IMS used to be called a Group Management Service (GMS), but to avoid confusion with the Shoal-based GMS that is already in GlassFish, the name for that has been changed to IaaS Management Service (IMS).  The IMS is also sometimes referred to as the virtualization layer.  Posted by trmueller at Apr 11, 2011 09:07