ActiveMatrix Administrator Server creation failing with "Error determining if action is finished"

ActiveMatrix Administrator Server creation failing with "Error determining if action is finished"

book

Article ID: KB0072804

calendar_today

Updated On:

Products Versions
TIBCO ActiveMatrix Service Grid 3.3.x, 3.4.0

Description

When trying to create an AMX Admin server using TCT, it may fail with the following error "TIBCO-AMX-CLI-000044: Error determining if action is finished" or similar errors in the TCT console window or TCT logs. 

  <tibco_home>\tct\<version>\admincmdline.log

=========================
23 Jul 2020 10:04:51  INFO - TIBCO-AMX-HPA-050114: Deploying resource instances...
23 Jul 2020 10:04:52  INFO - .
23 Jul 2020 10:05:02  INFO - .
23 Jul 2020 10:05:06  INFO - TIBCO-AMX-HPA-050115: Verifying resource instance deployment...
23 Jul 2020 10:05:06  INFO - TIBCO-AMX-HPA-050116: Deploying JAAS realm...
23 Jul 2020 10:05:07  INFO - .
23 Jul 2020 10:05:07  INFO - TIBCO-AMX-HPA-050117: Deploying admin (this may take a few minutes)...
23 Jul 2020 10:05:12  INFO - .
23 Jul 2020 10:05:17  INFO - .
23 Ju..........

23 Jul 2020 10:07:31 ERROR - TIBCO-AMX-CLI-000044: Error determining if action is finished:
com.tibco.amf.admin.api.servicefactory.remote.axis2.AdminHttpException

at com.tibco.amf.admin.api.servicefactory.remote.axis2.ClientSession.isServiceAvailable(ClientSession.java:685)
at com.tibco.amf.admin.api.servicefactory.remote.axis2.ClientSession.isServerRunning(ClientSession.java:613)
at com.tibco.amf.admin.cmdline.bootstrap.AdminBootstrapUtil.checkAdminPluginsLoaded(AdminBootstrapUtil.java:218)
at com.tibco.amf.admin.cmdline.bootstrap.AdminBootstrapUtil.checkAdminPluginsLoaded(AdminBootstrapUtil.java:209)
at com.tibco.amf.admin.cmdline.bootstrap.AdminBootstrapUtil.checkAdminPluginsLoaded(AdminBootstrapUtil.java:199)
at com.tibco.amf.admin.cmdline.bootstrap.DeployProgressIndicator.isFinished(DeployProgressIndicator.java:37)
at com.tibco.amf.admin.cmdline.AsyncActionExecutor.run(AsyncActionExecutor.java:153)
23 Jul 2020 10:07:31 ERROR - TIBCO-AMX-CLI-000338: Action failed at 7/23/20 10:07 AM in 214.811 seconds

BUILD FAILED
/opt/TIBCO/persistentdata/BPMQA_ISS_MZTI_CUST_TAPBPM_bpmadmin/configHome/tct/bpm/2020-07-23-10-03-31/scripts/build-parent.xml:12: The following error occurred while executing this line:
/opt/TIBCO/persistentdata/BPMQA_ISS_MZTI_CUST_TAPBPM_bpmadmin/home/administrator/3.4/scripts/instance-mgmt.xml:292: TIBCO-AMX-CLI-000042: Failed on error : '[]'

Total time: 12 minutes 17 seconds
=========================

Sometimes it fails with an error like below:

--------------------
summary = Admin experienced server side error: 
com.tibco.amf.admin.api.core.exception.AdminException
summary = Timed out. Waited for 10 minutes to allow platform application to complete its deployment on the Admin node, but it is still not finished yet. This usually indicates a problem has occurred.
    at com.tibco.amf.admin.server.AdminInitServlet.waitUntilTaskFinish(AdminInitServlet.java:347)
    at com.tibco.amf.admin.server.AdminInitServlet.init(AdminInitServlet.java:127)
    at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:431)
    at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:681)
    at com.tibco.amf.hpa.tibcohost.jetty.deployer.support.EMFSupportServletHandler.initialize(EMFSupportServletHandler.java:123)
    at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
    at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
    at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
    at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
    at com.tibco.amf.hpa.tibcohost.jetty.deployer.WarDeployer$4.doStart(WarDeployer.java:313)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at com.tibco.amf.hpa.tibcohost.jetty.ConnectorInfo.addContext(ConnectorInfo.java:824)
    at com.tibco.amf.hpa.tibcohost.jetty.internal.JettyManager.addContext(JettyManager.java:783)
    at com.tibco.amf.hpa.tibcohost.jetty.deployer.WarDeployer.startDeployment(WarDeployer.java:428)
    at com.tibco.amf.hpa.tibcohost.jetty.deployer.WarDeployer.doDeploy(WarDeployer.java:237)
    at com.tibco.amf.hpa.tibcohost.jetty.deployer.WarDeployer.access$2(WarDeployer.java:230)
    at com.tibco.amf.hpa.tibcohost.jetty.deployer.WarDeployer$3.run(WarDeployer.java:189)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
    at java.lang.Thread.run(Thread.java:736)

BUILD FAILED
/opt/tibco/amx-312/data/Inst32Bit/tct/admin/2011-03-30-12-43-47/scripts/build.xml:12: The following error occurred while executing this line:
/opt/tibco/amx-312/administrator/3.1/scripts/instance-mgmt.xml:237: Failed on error

Total time: 14 minutes 12 seconds
--------------------

Environment

All Supported OS Platforms

Resolution

There are multiple possible causes of the above issues:

A) Root cause: The Admin DB schema has not been fully cleaned up (i.e., to remove everything within the DB schema, or just delete and recreate the DB schema) since last unsuccessful attempt of Admin instance creation.

Solution: 

1. Fully clean up the Admin DB schema, file systems (<config_home>), and EMS server configs, data stores (but make sure '>' is in files queues.conf, topics.conf under folder "<ems_config_home>/ems/data")
2. Recreate the AMX Admin server using TCT


B) Root cause: The Admin DB schema (user) does not have the correct privileges.

For example, for Oracle 11gR2 database the DB user *cannot* have dba privileges, must have the 'connect' and 'resource' privileges, and be assigned to the default schema.

Solution: 

1. Grant appropriate permissions/priviledges to the Admin DB schema.
2. Clean up the DB schema, file systems (<config_home>), and EMS server
3. Recreate the AMX Admin server using TCT


C) Root cause: Microsoft SQL Server as the Admin DB does not have the correct configuration.

Solution:

SQL Server DB needs a prerequisite configuration - set read_commited_snapshot on

After creating the DB, from SQL run below query:

  alter database <DB_Name> set read_committed_snapshot on

That should fix this issue.

D) Root cause: The issue occurs on 32-bit AIX platform (see TAP-7618).

Please note that the 32-bit AIX is not supported by AMX 3.1.2 and later versions.

Solution: 

1. Make sure that the OS of the SystemHost machine and the OS bit mode (32-bit or 64-bit) are supported by the AMX version.
2. Clean up the DB schema, file systems (<config_home>), and EMS server
3. Recreate the AMX Admin server using TCT

E) Root cause: The Oracle Admin DB schema (username) has character quotes (“ ”) and hyphen(-), e.g., "test-user" (see AMX-15614)

Solution:

This is a known issue for AMX 3.3.0 and 3.4.0 . The solution is not to avoid character quotes (“ ”) and hyphen(-) in the Oracle schema (username).
 

Issue/Introduction

ActiveMatrix Administrator Server creation failing with "Error determining if action is finished"