Live status of manual upgrade I'm keeping my notes here while attempting to run upgrade and manually make changes in domain.xml as needed for implementation service code that isn't functional yet. The goal is to discover the steps that are needed to run v3 with an empty v2 domain. Then the next goal is to make sure that list goes to zero steps as everything is implemented. For background, see issue 9127 Am running v3 with the following command where v2domain1 is a domain copied into the glassfish v3 domains directory:
java -jar modules/glassfish.jar -upgrade true -domain v2domain1
Revision 30674 Iteration 0 No changes to v2 domain
hostname% java -jar modules/glassfish.jar -upgrade true -domain v2domain1
Aug 21, 2009 11:17:01 AM com.sun.enterprise.glassfish.bootstrap.ASMain main
INFO: Launching GlassFish on Felix platform
Welcome to Felix.
=================
Aug 21, 2009 11:17:13 AM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-mbeanserver [152]
Aug 21, 2009 11:17:13 AM OSGiModuleImpl start
INFO: Started bundle org.glassfish.core.kernel [54]
Aug 21, 2009 11:17:13 AM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.util [67]
Aug 21, 2009 11:17:13 AM OSGiModuleImpl start
INFO: Started bundle com.sun.enterprise.config [168]
ERROR: Error starting reference:file:/Users/bobby/servers/glassfishv3/glassfish/modules/osgi-main.jar (org.osgi.framework.BundleException: Activator start error in bundle com.sun.enterprise.osgi-adapter [129].)
java.lang.RuntimeException: Failed to parse file:/Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/config/domain.xml
at org.glassfish.config.support.DomainXml.parseDomainXml(DomainXml.java:155)
at org.glassfish.config.support.DomainXml.run(DomainXml.java:100)
at com.sun.enterprise.module.common_impl.AbstractModulesRegistryImpl.populateConfig(AbstractModulesRegistryImpl.java:157)
at com.sun.enterprise.module.common_impl.AbstractModulesRegistryImpl.createHabitat(AbstractModulesRegistryImpl.java:142)
at com.sun.enterprise.module.bootstrap.Main.createHabitat(Main.java:421)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:118)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
at org.jvnet.hk2.osgimain.Main.start(Main.java:117)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:984)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:263)
at java.lang.Thread.run(Thread.java:637)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[3,1]
Message: found: DTD, expected START_ELEMENT or END_ELEMENT
at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.nextTag(XMLStreamReaderImpl.java:1231)
at javax.xml.stream.util.StreamReaderDelegate.nextTag(StreamReaderDelegate.java:66)
at org.glassfish.config.support.XMLStreamReaderFilter.nextTag(XMLStreamReaderFilter.java:66)
at org.jvnet.hk2.config.ConfigParser.parse(ConfigParser.java:85)
at org.glassfish.config.support.DomainXml.parseDomainXml(DomainXml.java:149)
... 14 more
Note: process hangs at this point. Killed server with ctrl-c.
Iteration 1 Removed DTD
hostname% java -jar modules/glassfish.jar -upgrade true -domain v2domain1
Aug 21, 2009 11:20:08 AM com.sun.enterprise.glassfish.bootstrap.ASMain main
INFO: Launching GlassFish on Felix platform
Welcome to Felix.
=================
Aug 21, 2009 11:20:11 AM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-mbeanserver [152]
Aug 21, 2009 11:20:11 AM OSGiModuleImpl start
INFO: Started bundle org.glassfish.core.kernel [54]
Aug 21, 2009 11:20:11 AM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.util [67]
Aug 21, 2009 11:20:11 AM OSGiModuleImpl start
INFO: Started bundle com.sun.enterprise.config [168]
Aug 21, 2009 11:20:11 AM org.jvnet.hk2.config.ConfigParser handleElement
SEVERE: Ignoring unrecognized element j2ee-application at Line number = 10
Column number = 271
System Id = file:/Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/config/domain.xml
Public Id = null
Location Uri= file:/Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/config/domain.xml
CharacterOffset = 688
ignoring
Aug 21, 2009 11:20:11 AM org.jvnet.hk2.config.ConfigParser handleElement
SEVERE: Ignoring unrecognized element j2ee-application at Line number = 12
Column number = 235
System Id = file:/Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/config/domain.xml
Public Id = null
Location Uri= file:/Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/config/domain.xml
CharacterOffset = 1174
ignoring
ERROR: Error starting reference:file:/Users/bobby/servers/glassfishv3/glassfish/modules/osgi-main.jar (org.osgi.framework.BundleException: Activator start error in bundle com.sun.enterprise.osgi-adapter [129].)
java.lang.RuntimeException: No <config> seen for name=null
at org.glassfish.config.support.DomainXml.parseDomainXml(DomainXml.java:152)
at org.glassfish.config.support.DomainXml.run(DomainXml.java:100)
at com.sun.enterprise.module.common_impl.AbstractModulesRegistryImpl.populateConfig(AbstractModulesRegistryImpl.java:157)
at com.sun.enterprise.module.common_impl.AbstractModulesRegistryImpl.createHabitat(AbstractModulesRegistryImpl.java:142)
at com.sun.enterprise.module.bootstrap.Main.createHabitat(Main.java:421)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:118)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
at org.jvnet.hk2.osgimain.Main.start(Main.java:117)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:984)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:263)
at java.lang.Thread.run(Thread.java:637)
Will look into the first error and compare v2/v3 domain.xml files:
SEVERE: Ignoring unrecognized element j2ee-application at Line number = 10
Column number = 271
A v3 domain.xml starts off with no internal applications deployed so the domain/applications element of domain.xml is empty. Edited domain.xml to remove these elements. Iteration 2
- Removed domain/applications/**
This resulted in some class not found errors. Top of stack is here (causes omitted since the cause is obvious):
SEVERE: java.lang.ExceptionInInitializerError
Aug 21, 2009 12:10:22 PM
SEVERE: at org.glassfish.admin.monitor.MonitoringBootstrap.setStatsProviderManagerDelegate(MonitoringBootstrap.java:116)
Aug 21, 2009 12:10:22 PM
SEVERE: at org.glassfish.admin.monitor.MonitoringBootstrap.postConstruct(MonitoringBootstrap.java:101)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:89)
Aug 21, 2009 12:10:22 PM
SEVERE: at java.security.AccessController.doPrivileged(Native Method)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:86)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:206)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:129)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.enterprise.v3.server.UpgradeStartup.start(UpgradeStartup.java:143)
Aug 21, 2009 12:10:22 PM
SEVERE: at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:460)
...
Class not found was com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder from this JVM option:
<jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder</jvm-options>
Will compare the jvm options between v2 and v3. Elements are in /domain/configs/config[@name='server-config']/java-config. Will also check that the attributes of the element make sense.
Iteration 3 Hacking java-config element Summary: remove the following jvm-option element.
<jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder</jvm-options>
More detailed information now. The following elements are common to v2 and v3 domain:
<jvm-options>-XX:MaxPermSize=192m</jvm-options>
<jvm-options>-client</jvm-options>
<jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
<jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
<jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
<jvm-options>-Xmx512m</jvm-options>
<jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
<jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
<jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext${path.separator}${com.sun.aas.derbyRoot}/lib</jvm-options>
<jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
<jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
<jvm-options>-XX:NewRatio=2</jvm-options>
These elements are not present in v3, but may not be causing any problems. Will email dev@gf and give them a look.
<jvm-options>-Dsun.rmi.dgc.server.gcInterval=3600000</jvm-options>
<jvm-options>-Dsun.rmi.dgc.client.gcInterval=3600000</jvm-options>
<jvm-options>-Dcom.sun.enterprise.taglibs=appserv-jstl.jar,jsf-impl.jar</jvm-options>
<jvm-options>-Dcom.sun.enterprise.taglisteners=jsf-impl.jar</jvm-options>
Re-running the server, we fail again, but we're making progress. At this point the upgrade service can start, see that there is a failure, and shut itself down properly.
hostname% java -jar modules/glassfish.jar -upgrade true -domain v2domain1
Aug 21, 2009 1:39:10 PM com.sun.enterprise.glassfish.bootstrap.ASMain main
INFO: Launching GlassFish on Felix platform
Welcome to Felix.
=================
Aug 21, 2009 1:39:12 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-mbeanserver [152]
Aug 21, 2009 1:39:12 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.core.kernel [54]
Aug 21, 2009 1:39:12 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.util [67]
Aug 21, 2009 1:39:12 PM OSGiModuleImpl start
INFO: Started bundle com.sun.enterprise.config [168]
Aug 21, 2009 1:39:12 PM org.glassfish.config.support.DomainXml$DomainXmlReader parse2ndTime
INFO: Forced to parse file:/Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/config/domain.xml twice because we didn't see <server> before <config>
Aug 21, 2009 1:39:12 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.transaction.jta [65]
Aug 21, 2009 1:39:12 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.config-api [94]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.orb.connector [141]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.connectors.admin [178]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.jdbc.admin [181]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.ejb.ejb-container [43]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.jms.admin [179]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.monitoring-core [155]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.web.cli [96]
Aug 21, 2009 1:39:13 PM org.hibernate.validation.util.Version <clinit>
INFO: Hibernate Validator null
Aug 21, 2009 1:39:13 PM org.hibernate.validation.engine.resolver.DefaultTraversableResolver detectJPA
INFO: Instantiated an instance of org.hibernate.validation.engine.resolver.JPATraversableResolver.
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle com.sun.grizzly.grizzly-config [136]
Aug 21, 2009 1:39:13 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.server-mgmt [191]
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.branding [180]
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.deployment.common [50]
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app adminapp is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app admingui is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app JBIFramework is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app WSTXServices is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app WSTCPConnectorLCModule is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app __ejb_container_timer_app is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app __JWSappclients is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app MEjbApp is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app VaadinGFTest is false resulting in false
Aug 21, 2009 1:39:14 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: app VaadinTest is false resulting in false
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.internal-api [198]
Aug 21, 2009 1:39:14 PM
INFO: com.sun.enterprise.naming.util.NamingActivator@76f56093 called
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-naming [87]
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-api [79]
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.flashlight.framework [176]
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.registration.glassfish-registration [166]
Aug 21, 2009 1:39:14 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.jms.core [15]
Aug 21, 2009 1:39:15 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.container-common [17]
Aug 21, 2009 1:39:15 PM org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread run
INFO: JMXStartupService: JMX authenticator is com.sun.enterprise.container.common.GenericAdminAuthenticator@5846b1c9
Aug 21, 2009 1:39:15 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.web.osgi-web-container [159]
Aug 21, 2009 1:39:15 PM WebExtender run
INFO: Waiting for Server to start
Aug 21, 2009 1:39:15 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.rest-service [186]
Aug 21, 2009 1:39:15 PM com.sun.enterprise.v3.services.impl.GrizzlyService postConstruct
SEVERE: Unable to start v3. Closing all ports
java.lang.NullPointerException
at com.sun.grizzly.config.dom.NetworkListener$Duck.findThreadPool(NetworkListener.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jvnet.hk2.config.Dom.invokeDuckMethod(Dom.java:825)
at org.jvnet.hk2.config.Dom.invoke(Dom.java:778)
at org.glassfish.config.support.TranslatedConfigView.invoke(TranslatedConfigView.java:83)
at $Proxy54.findThreadPool(Unknown Source)
at com.sun.grizzly.config.GrizzlyEmbeddedHttp.configure(GrizzlyEmbeddedHttp.java:248)
at com.sun.grizzly.config.GrizzlyServiceListener.initializeListener(GrizzlyServiceListener.java:88)
at com.sun.grizzly.config.GrizzlyServiceListener.configure(GrizzlyServiceListener.java:77)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.configure(GrizzlyListener.java:65)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy.configureGrizzly(GrizzlyProxy.java:120)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy.<init>(GrizzlyProxy.java:111)
at com.sun.enterprise.v3.services.impl.GrizzlyService.createNetworkProxy(GrizzlyService.java:349)
at com.sun.enterprise.v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:271)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:89)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:86)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:129)
at com.sun.enterprise.v3.server.UpgradeStartup.start(UpgradeStartup.java:143)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:460)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:404)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:121)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
at org.jvnet.hk2.osgimain.Main.start(Main.java:117)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:984)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:263)
at java.lang.Thread.run(Thread.java:637)
Aug 21, 2009 1:39:15 PM
SEVERE: java.lang.NullPointerException
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.grizzly.config.dom.NetworkListener$Duck.findThreadPool(NetworkListener.java:191)
Aug 21, 2009 1:39:15 PM
SEVERE: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Aug 21, 2009 1:39:15 PM
SEVERE: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
Aug 21, 2009 1:39:15 PM
SEVERE: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
Aug 21, 2009 1:39:15 PM
SEVERE: at java.lang.reflect.Method.invoke(Method.java:597)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.jvnet.hk2.config.Dom.invokeDuckMethod(Dom.java:825)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.jvnet.hk2.config.Dom.invoke(Dom.java:778)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.glassfish.config.support.TranslatedConfigView.invoke(TranslatedConfigView.java:83)
Aug 21, 2009 1:39:15 PM
SEVERE: at $Proxy54.findThreadPool(Unknown Source)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.grizzly.config.GrizzlyEmbeddedHttp.configure(GrizzlyEmbeddedHttp.java:248)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.grizzly.config.GrizzlyServiceListener.initializeListener(GrizzlyServiceListener.java:88)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.grizzly.config.GrizzlyServiceListener.configure(GrizzlyServiceListener.java:77)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.services.impl.GrizzlyListener.configure(GrizzlyListener.java:65)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.services.impl.GrizzlyProxy.configureGrizzly(GrizzlyProxy.java:120)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.services.impl.GrizzlyProxy.<init>(GrizzlyProxy.java:111)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.services.impl.GrizzlyService.createNetworkProxy(GrizzlyService.java:349)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:271)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:89)
Aug 21, 2009 1:39:15 PM
SEVERE: at java.security.AccessController.doPrivileged(Native Method)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:86)
Aug 21, 2009 1:39:15 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.deployment.autodeploy [134]
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:129)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.v3.server.UpgradeStartup.start(UpgradeStartup.java:143)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:460)
Aug 21, 2009 1:39:15 PM
SEVERE: at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:404)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:121)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.jvnet.hk2.osgimain.Main.start(Main.java:117)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:984)
Aug 21, 2009 1:39:15 PM
SEVERE: at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:263)
Aug 21, 2009 1:39:15 PM
SEVERE: at java.lang.Thread.run(Thread.java:637)
Aug 21, 2009 1:39:15 PM com.sun.enterprise.v3.server.AppServerStartup run
INFO: Startup service failed to start : null
Aug 21, 2009 1:39:15 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.transaction.jts [72]
Aug 21, 2009 1:39:15 PM org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread startConnector
INFO: JMXStartupService: Started JMXConnector, JMXService URL = service:jmx:rmi://dhcp-ubur02-71-236.East.Sun.COM:49474/jndi/rmi://dhcp-ubur02-71-236.East.Sun.COM:49474/jmxrmi
Aug 21, 2009 1:39:15 PM com.sun.enterprise.v3.server.AppServerStartup run
INFO: GlassFish v3 startup time : Felix(3880ms) startup services(1791ms) total(5671ms)
Aug 21, 2009 1:39:15 PM WebExtender event
INFO: Received Server Started Event
Aug 21, 2009 1:39:15 PM AppServerStartup run
INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
Aug 21, 2009 1:39:15 PM com.sun.enterprise.v3.server.UpgradeStartup start
INFO: Exiting after upgrade
Aug 21, 2009 1:39:18 PM com.sun.enterprise.v3.admin.StopDomainCommand execute
INFO: Server shutdown initiated
Aug 21, 2009 1:39:18 PM org.glassfish.admin.mbeanserver.JMXStartupService shutdown
INFO: JMXStartupService: shutting down AMX and JMX
Aug 21, 2009 1:39:19 PM org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread shutdown
INFO: JMXStartupService: Stopped JMXConnectorServer: service:jmx:rmi://dhcp-ubur02-71-236.East.Sun.COM:49474/jndi/rmi://dhcp-ubur02-71-236.East.Sun.COM:49474/jmxrmi
Aug 21, 2009 1:39:19 PM AppServerStartup run
INFO: [Thread[GlassFish Kernel Main Thread,5,main]] exiting
Aug 21, 2009 1:39:19 PM
INFO: felix.fileinstall.poll (ms) 5000
Aug 21, 2009 1:39:19 PM
INFO: felix.fileinstall.dir /Users/bobby/servers/glassfishv3/glassfish/domains/v2domain1/autodeploy-bundles
Aug 21, 2009 1:39:19 PM OSGiModuleImpl detach
INFO: Stopped bundle = [removed all the stopped bundle messages]
Will investigate the NPE in grizzly code and will also check to make sure those 4 exta jvm-option elements aren't causing an issue. I've removed them and get the same result. Will go that route for now.
Note: At this point we have enough of a workaround so that the upgrade service is actually being called. I'm going to summarize these steps on the main upgrade dashboard page and send a link out to the module owners. Then we can start with individual modules that are hitting trouble, starting with the grizzly npe above.
Followup: Justin may have a simple fix for getting around the NPE. Could just be a thread pool missing in domain.xml, so he's giving it a look and we may have one more item for the manual workaround list.
While Justin is looking at it, I'm looking at other differences between the v2 and v3 domains. There are some jvm-options in v3 that are not present in v2. Am capturing them here for reference and will test a v2 upgrade with/without them.
<jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
<jvm-options>-XX:+LogVMOutput</jvm-options>
<jvm-options>-XX:LogFile=${com.sun.aas.instanceRoot}/logs/jvm.log</jvm-options>
<jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
Now at revision 30710 Same results. Just checking.
Revision 30744 Am removing the following elements from /domain/servers/server[@name='server']
<application-ref ref="adminapp" virtual-servers="__asadmin" />
<application-ref ref="admingui" virtual-servers="__asadmin" />
<application-ref ref="JBIFramework" />
<application-ref ref="WSTXServices" />
<application-ref ref="WSTCPConnectorLCModule" />
<application-ref ref="__ejb_container_timer_app" virtual-servers="server" />
<application-ref ref="__JWSappclients" virtual-servers="server" />
<application-ref ref="MEjbApp" virtual-servers="server" />
<resource-ref ref="jdbc/__CallFlowPool" />
Leaving only these two resource references, matching the v3 default:
<resource-ref ref="jdbc/__TimerPool" />
<resource-ref ref="jdbc/__default" />
Since running the server overwrites domain.xml and I lose comments, here's more scratch pad. Am changing this:
<thread-pools>
<thread-pool max-thread-pool-size="200" min-thread-pool-size="0" />
<thread-pool name="http-thread-pool" max-queue-size="4096" />
</thread-pools>
to this:
<thread-pools>
<thread-pool name="http-thread-pool" max-queue-size="4096"></thread-pool>
<thread-pool name="thread-pool-1" max-thread-pool-size="200"/>
</thread-pools>
Update: This at least gets to a new error. Getting closer:
Aug 24, 2009 5:18:47 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.web.osgi-web-container [160]
Aug 24, 2009 5:18:47 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.rest-service [186]
Aug 24, 2009 5:18:47 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.deployment.autodeploy [135]
Aug 24, 2009 5:18:47 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.transaction.jts [73]
Aug 24, 2009 5:18:47 PM org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread startConnector
INFO: JMXStartupService: Started JMXConnector, JMXService URL = service:jmx:rmi://dhcp-ubur02-71-236.East.Sun.COM:8686/jndi/rmi://dhcp-ubur02-71-236.East.Sun.COM:8686/jmxrmi
Aug 24, 2009 5:18:48 PM com.sun.enterprise.v3.services.impl.GrizzlyProxy start
INFO: Listening on port 8080
Aug 24, 2009 5:18:48 PM com.sun.grizzly.config.GrizzlyEmbeddedHttps configureSSL
WARNING: pewebcontainer.all_ssl_ciphers_disabled
Aug 24, 2009 5:18:48 PM com.sun.grizzly.Controller start
INFO: Starting Grizzly Framework 1.9.18-M1 - Mon Aug 24 17:18:48 EDT 2009
Aug 24, 2009 5:18:49 PM com.sun.enterprise.v3.services.impl.GrizzlyProxy start
INFO: Listening on port 8181
Aug 24, 2009 5:18:49 PM com.sun.grizzly.Controller start
INFO: Starting Grizzly Framework 1.9.18-M1 - Mon Aug 24 17:18:49 EDT 2009
Aug 24, 2009 5:18:49 PM com.sun.enterprise.v3.services.impl.GrizzlyProxy start
INFO: Listening on port 4848
Aug 24, 2009 5:18:49 PM com.sun.grizzly.Controller start
INFO: Starting Grizzly Framework 1.9.18-M1 - Mon Aug 24 17:18:49 EDT 2009
Aug 24, 2009 5:18:50 PM com.sun.enterprise.v3.services.impl.GrizzlyService postConstruct
SEVERE: Unable to start v3. Closing all ports
java.lang.NullPointerException
at com.sun.enterprise.v3.admin.adapter.AdminConsoleAdapter.init(AdminConsoleAdapter.java:425)
at com.sun.enterprise.v3.admin.adapter.AdminConsoleAdapter.postConstruct(AdminConsoleAdapter.java:386)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:89)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:86)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at org.jvnet.hk2.component.Habitat$1.get(Habitat.java:275)
at java.util.AbstractList$Itr.next(AbstractList.java:345)
at com.sun.enterprise.v3.services.impl.GrizzlyService.registerNetworkProxy(GrizzlyService.java:399)
at com.sun.enterprise.v3.services.impl.GrizzlyService.registerNetworkProxy(GrizzlyService.java:391)
at com.sun.enterprise.v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:310)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:89)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:86)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:129)
at com.sun.enterprise.v3.server.UpgradeStartup.start(UpgradeStartup.java:143)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:460)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:404)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:121)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
at org.jvnet.hk2.osgimain.Main.start(Main.java:117)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:984)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:263)
at java.lang.Thread.run(Thread.java:637)
Aug 24, 2009 5:18:50 PM
Have sent updated stack trace and domain.xml to Justin. Will also look through it in the debugger tomorrow.
Revision 30785 Have updated main page's instructions to add the thread pool information. Am attaching the domain.xml I have at this point to avoid making the changes again. domain.xml_rev_30785 Earlier today I was seeing an error in logging code when starting up: logging.properties not found and the server exited. Am no longer seeing that, so it must have been something I did to clobber the file even though I don't think I touched that dir. Need to start with a fresh domain again and check.... Yep, one is being created for me. Am hitting the thread pool failure again, so I have the thread-pool info incorrect. Fixed thread pool notes on dashboard page and fixed attachment. Am now getting a new error, so some work must be going on in the grizzly module:
[...]
Aug 25, 2009 3:42:48 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.container-common [18]
Aug 25, 2009 3:42:48 PM org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread run
INFO: JMXStartupService: JMX authenticator is com.sun.enterprise.container.common.GenericAdminAuthenticator@39662993
Aug 25, 2009 3:42:48 PM WebExtender run
INFO: Waiting for Server to start
Aug 25, 2009 3:42:48 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.web.osgi-web-container [161]
Aug 25, 2009 3:42:48 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.rest-service [188]
Aug 25, 2009 3:42:48 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.deployment.autodeploy [136]
Aug 25, 2009 3:42:49 PM com.sun.enterprise.v3.services.impl.GrizzlyProxy start
INFO: Listening on port 8080
Aug 25, 2009 3:42:49 PM com.sun.enterprise.v3.services.impl.GrizzlyService postConstruct
SEVERE: Unable to start v3. Closing all ports
java.lang.ClassCastException: $Proxy32 cannot be cast to com.sun.grizzly.config.dom.Protocol
at com.sun.grizzly.config.GrizzlyEmbeddedHttps.configureSSL(GrizzlyEmbeddedHttps.java:180)
at com.sun.grizzly.config.GrizzlyEmbeddedHttps.configureProtocol(GrizzlyEmbeddedHttps.java:95)
at com.sun.grizzly.config.GrizzlyEmbeddedHttp.configure(GrizzlyEmbeddedHttp.java:266)
at com.sun.grizzly.config.GrizzlyServiceListener.initializeListener(GrizzlyServiceListener.java:88)
at com.sun.grizzly.config.GrizzlyServiceListener.configure(GrizzlyServiceListener.java:77)
at com.sun.enterprise.v3.services.impl.GrizzlyListener.configure(GrizzlyListener.java:65)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy.configureGrizzly(GrizzlyProxy.java:120)
at com.sun.enterprise.v3.services.impl.GrizzlyProxy.<init>(GrizzlyProxy.java:111)
at com.sun.enterprise.v3.services.impl.GrizzlyService.createNetworkProxy(GrizzlyService.java:349)
at com.sun.enterprise.v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:271)
at com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:174)
at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:89)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:86)
at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:58)
at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:236)
at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:129)
at com.sun.enterprise.v3.server.UpgradeStartup.start(UpgradeStartup.java:144)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:460)
at com.sun.enterprise.module.bootstrap.Main.launch(Main.java:404)
at org.jvnet.hk2.osgiadapter.HK2Main.start(HK2Main.java:121)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
at org.jvnet.hk2.osgimain.Main.start(Main.java:117)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1461)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:984)
at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:263)
at java.lang.Thread.run(Thread.java:637)
Will check with Justin.
Now on revision 30836 No longer getting the ClassCast exception, so something is changing. Now back to null pointer exception in init() method. Haven't head any email back yet. Trying to debug. Debugging, I see there are properties missing in domain.xml that the upgrade service is counting on (though in some cases a default could be implied). Am adding under /domain/configs/config[@name='server-config']/admin-service (still a work in progress):
<property name="adminConsoleDownloadLocation" value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war" />
<property name="ipsRoot" value="${com.sun.aas.installRoot}/.." />
Hey, it works. Time for a beer. The current domain.xml file with all the changes is domain.xml_rev_30836. I'll add information to the dashboard page and send an email out to the module leads and maybe dev@gf too?
Rev 30873 Am rebuilding to test. Two questions that need to get resolved about issue 9127
- Where will the domain.xml transforming code actually live. Ideally, some part of the upgrade service inside GlassFish since the upgrade tool's job is just gathering information and the server does the rest.
- Part of the workaround is to remove the elements under /domain/applications – for existing apps, the elements should be transformed so that domain.xml can still be parsed. Will get some more information about this and update the issue.
Was seeing unexpected hang of server with last revision. Trying 30881 now. Building...building...building...
Fixing the workaround Here is the original workaround for issue 9127 and more. Am putting it here for easy reference as we resolve the issues on the main Upgrade wiki page. More information can be found with issue 9127, but if you are attempting to do an upgrade starting with a v2 domain, here are changes you need to make manually in
before proceeding. Or use the one I'm currently trying. The rev number is the revision of the code I've tested with.
- Remove the DTD from the beginning of the file.
- Remove all the elements under /domain/applications (except any applications you have explicitly deployed). Also remove the corresponding resource-ref elements under /domain/servers/server[@name='server']. The resource-ref elements that should remain by default are:
<resource-ref ref="jdbc/__TimerPool" />
<resource-ref ref="jdbc/__default" />
- In the //domain/configs/config[@name='server-config']/java-config element, remove the following <jvm-options> elements:
- -Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder
- -Dsun.rmi.dgc.server.gcInterval=3600000
- -Dsun.rmi.dgc.client.gcInterval=3600000
- -Dcom.sun.enterprise.taglibs=appserv-jstl.jar,jsf-impl.jar
- -Dcom.sun.enterprise.taglisteners=jsf-impl.jar
- The following jvm-options should also be added:
<jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
<jvm-options>-XX:+LogVMOutput</jvm-options>
<jvm-options>-XX:LogFile=${com.sun.aas.instanceRoot}/logs/jvm.log</jvm-options>
<jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
- Remove the elements under thread-pools (unless you have created some explicitly) and add:
<thread-pool name="http-thread-pool" max-queue-size="4096"></thread-pool>
<thread-pool name="thread-pool-1" max-thread-pool-size="200"/>
- Add the following under /domain/configs/config[@name='server-config']/admin-service:
<property name="adminConsoleDownloadLocation" value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war" />
<property name="ipsRoot" value="${com.sun.aas.installRoot}/.." />
- At this point you can run the service. The command I'm currently using is here. If your v2 domain isn't in the v3 domains dir, copy it there or use the -domaindir option with either of the following:
java -jar modules/glassfish.jar -upgrade true -domain <v2domain>
asadmin start-domain -v --upgrade <v2domain>
|
