Resolution:Title: Incorrect loading of TIBCO Collaborative Information Manager (CIM) EJBs (Sesson or Entity) into WebSphere causes the CORBA errors CORBA.TRANSACTION_ROLLEDBACK or CORBA.OBJECT_NOT_EXIST to be recorded in the file elink.log . CIM behavior to becomes unpredictable when CIM is accessed.
Resolution: Description:
============
CORBA.TRANSACTION_ROLLEDBACK or CORBA.OBJECT_NOT_EXIST errors returned by the WebSphere Application Server is usually a sign that the ECM.ear needs re-installing onto the WebSphere Server. This should be done when access to the Server has been shut down. When loading the TIBCO Collaborative Information Manager (CIM) EJBs (Session or Entity) into WebSphere, this loading may not work properly the first time around, causing CORBA errors. Often, these errors will clear as the beans are called again later.
Environment:
============
WebSphere Application Server 7.x ECM.ear file deployment (for all versions of CIM / MDM).
Symptoms:
=========
CORBA errors CORBA.TRANSACTION_ROLLEDBACK or CORBA.OBJECT_NOT_EXIST to be recorded in the file elink.log. CIM behavior to becomes unpredictable when CIM is accessed.
Cause:
======
When loading CIM EJBs (Session or Entity) into WebSphere, this loading may not work properly the first time around, causing CORBA errors. Often, these errors will clear as the beans are called again later on. This is because the beans can start off "in the wrong state", but as they are accessed, the beans are activated by WAS even though an error might be raised back to the client. (e.g., CORBA.OBJECT_NOT_EXIST: Session bean not properly initialized or CORBA.TRANSACTION_ROLLEDBACK: CMP Entity beans not properly connecting to the database) before the bean is activated.
Think of beans as callback handlers that listen on their business interfaces once they have been activated which is pretty much how EJB containers behave, - send a CORBA request to the EJB container port and let the handler direct the request to the appropriate bean.
When requests come in for beans that have not yet registered or are temporarily unavailable, a CORBA.OBJECT_NOT_EXIST exception (for a Session Bean) or a CORBA.TRANSACTION_ROLLEDBACK exception (for an Entity Bean) will be returned to the client.
In both cases, the application server will issue a request to activate the bean and register it with the handler ("location service daemon") after the exception has been raised if it finds the bean is available. Sometimes, code is written that waits and retries a bean access after a set time if one of the errors above occurs in order to give the server time to "set itself right".
A detailed description may be found at the following: http://www.ibm.com/developerworks/websphere/techjournal/0807_pape/0807_pape.html
IBM does provide the ability to force start EJBs as soon as they are loaded or as soon as the server comes up. Unfortunately, the CIM code is not written to handle this, so the EJBs must just be loaded without being started.
Resolution:
===========
Every redeployment of the ECM.ear file runs the slight risk of seeing this error again. To minimize the risk,it is recommended that all interactions with the server to which the new ECM.ear is being deployed are first shut off (for example, by bringing down the CIM Apache or IHS Web Servers or taking the whole CIM server machine off-line by temporarily closing all network ports to it - simple, at least, for Linux and AIX) before deploying the ear file. When (re)starting the WebSphere platform, the 'SystemOut' and 'SystemErr' log files for the server(s) to which ECM.ear has been deployed are examined to ensure that the servers came up without reporting any problems.
It is important to select the correct class loader policy (PARENT Last) when deploying an ear file to a server. This determines in which order classes are "looked for" on the server, as well as in what order they are registered with the ORB handler, and to synchronize all changes (including uploads) with all other servers in the cell / cluster via the cell master (maintained by the Deployment Manager server). It is also good practice to reboot all servers ("offline" - i.e., with network connections down) after installing any ear file, but especially those that do not dynamically start their EJBs, to ensure that all EJBs are loaded correctly.
Following this procedure ensures that all possible interactions with the server (and 'nodeagent', if in a WAS cell) to which the ECM.ear file is being deployed, have stopped. This allows 100% of the server's resources to be dedicated to installing the ECM.ear file and registering the CIM EJBs without the risk of any CORBA agent being kept busy and timing out on bean registration as well as the chance for beans to load into the appropriate caches on start up of the server without interruption.
Once the EJBs have been registered and loaded properly, the CORBA errors will cease, barring CORBA faults due to other reasons.