Probable causes for Remote TID Manager getting timed out

Probable causes for Remote TID Manager getting timed out

book

Article ID: KB0083129

calendar_today

Updated On:

Products Versions
TIBCO ActiveMatrix BusinessWorks Plug-in for SAP Solutions All

Description

Below are some of the probable causes for the Remote TID Manager getting timed out.

---Error string----

2017 Apr 25 20:48:18:309 GMT +0530 R3AdapterInstance01 Warn [Adapter] AER3-970276 Server Exception: 
    Gateway Host: 10.108.80.143
    Gateway Service: sapgw00
    Program ID: tibcosupport02
    Error Message: check TID fault: Remote TIDManager timeout 

AND

Error [Adapter] AER3-000102 [JCoServerThread-1]  TID operation timed out <TID>

---Error string----

1). Misconfiguration done for the Adapter Instance and Remote TID manager: Refer to KB article# 000031048 for more details. The URL for the same is pasted below.

URL: https://support.tibco.com/s/article/ka11a000000Ht1OAAS/Remote-TIDManager-getting-Timed-out

2). TID file is large in size and has many entries in it. The performance of the TID manager depends on the size of the TID file and the number of entries in it. In case the size of the TID file is big and has many entries in it then the performance will be hampered as all the operations performed by the TID Manager is I/O operation based and it may happen that the TID manager is unable to consume the TID requests within the default expiration time(30s) after which the messages will be expired and an error "TID operation timed out" will occur.

3). Use of a single TID file by multiple Remote TID Manager. During TID operations the TID file gets locked by that particular TID manager. In case if multiple TID manager configured/running in your environment are using the same TID file than it may cause issue and will keep other Remote TID manager waiting until the TID file is unlocked. The better option is to keep a separate TID file for different Remote TID managers. This will help in speeding up the I/O operation and make the TID manager consume the request within the expiration time.

4). TID file is corrupted. If the TID file is corrupted or has unwanted entries in it then there is a possibility that the Remote TID manager will be unable to perform the IO operation resulting in a timeout.

5).
Time difference between adapter machine and EMS machine. When additional tracing is enabled on the EMS side then in the EMS logs you will be able to see the detailed information for a request. In the EMS logs for each request one can see two JMS header properties "Time=" and "Exp=" both in epoch format which respectively stands for the time stamp when a message is sent out from an adapter and the timestamp when the message is expired by EMS. Based on the values of those properties, with default configuration one can see that the expiration time is 30s which means that if a message is sent out by adapter but not got consumed by TID Manager in 30s, then the message would be expired. The time difference comes into play when the time on the EMS machine is 15s earlier than the adapter machine, then the message will be expired by EMS in 15s after it is received by EMS. In this scenario, if the related TID IO operations are not completed within 15s then the Remote TID manager will be unable to consume the request and will get timed out.

6). Other EMS issues at network level which may cause issues in the delivery of messages between the two endpoints configured.

Issue/Introduction

Probable causes for Remote TID Manager being timed out

Environment

All

Resolution

In order to troubleshoot the issue do the following.

1). If a configuration issue is found then follow the steps as mentioned in KB article# 000031048.

2). Make sure that the same TID file is not being used by multiple Remote TID Managers. Use a different tid file when working with multiple Remote TID Managers.

3). Stop the adapter instance and Remote TID manager and then recycle the existing TID file by creating a backup and creating a new file and then start the Remote TID manager followed by the adapter instance.

4). Enable additional tracing at EMS side using the steps below to find out if there is a time difference between adapter machine and EMS machine.​ If in case a time difference is found between the Adapter machine and EMS machine then
log a ticket with EMS support to sync up the time between client and server machines.

***Enabling additional tracing at EMS**    
-Enable EMS Client trace by executing "set server client_trace=enabled" on tibemsadmin.
​-Enable MSG trace by executing "set server log_trace=+MSG" on tibemsadmin.
-Execute "addprop queue <queue_name> trace=body" on tibemsadmin to enable the trace property to the TID queue.
***************************************************