Advanced MSMQ Troubleshooting for TIBCO Scribe® Insight Publishers
Article ID: KB0078925
Updated On:
| Products | Versions |
|---|---|
| TIBCO Scribe Insight | 7.9.3 |
Description
Overview of the Scribe Publisher MSMQ Process
TIBCO Scribe® Insight Console utilizes Microsoft Message Queues for the AX, CRM, GP & NAV Publishers and for Query Publishers. Whenever a Publisher is created and connected to the target system, it performs the following tasks:
For CRM, a set of plugins are created within CRM that populate the new_ ScribePublisherQueue entity.
When a Publisher is active, it will pull data from the ScribePublisherQueue and create messages in the appropriate PubIn message queue.
Query Publishers generate messages directly into the ScribeIn queue. However using a Console Publisher to migrate data from MS Dynamic via MSMQ is a two-step process.
The first step is to create a message in the AX-GP-NAV-CRM PubIn queue. The message contains the minimal information required to query the Dynamics database to retrieve the data that is used to build the detail message in the next step.
The Publisher then creates the detailed message for the ScribeIn queue containing all of the fields required to update the target. This process can take longer when there is a lot of data to retrieve to build the message file. If for some reason the data can’t be retrieved, the message is sent to the PubFailed queue. If successful, the message is either discarded or sent to the PubSucceeded queue (if enabled).
Messages in the ScribeIn queue are picked up by the TIBCO Scribe® Insight Integration Processes (IP). Each message has a message label that is used by an IP to find messages that belong to them. Message labels that don’t match any of the IP processes will be sent to the ScribeDeadMessage queue. If there is an error during processing, messages are sent to the ScribeRetry queue. The IP process will attempt to reprocess the message based on the retry count in the IP configuration. Once the retry account is reached the message is sent to the ScribeDeadMessageQueue.
Troubleshooting Steps
Installation and Configuration Settings - Verify the following:
MSMQ is installed and the queues exist 
Are the Scribe & MQ services running
Verify Scribe Services Logon user is a Domain User that belongs to the local Administrators Group
Is the Scribe service LogOn user a member of the Scribe Console User group (windows group) 
Verify permissions on TIBCO Scribe® Insight related Private Queues (Scribe, CRM, etc) 
Have the number of messages reached the MSMQ disk space limit (or no messages are generated) 
Purge MSMQ System Queues Dead-Letter messages if necessary 
Check maximum storage allotment for messages (1048576 = 1 GB) – increase if necessary 
TIBCO Scribe® Insight Console utilizes Microsoft Message Queues for the AX, CRM, GP & NAV Publishers and for Query Publishers. Whenever a Publisher is created and connected to the target system, it performs the following tasks:
- Creates the ScribePublisherQueue (SPQ) table in the target database (AX,GP,NAV)
- For CRM 2011 - Creates new_ ScribePublisherQueue entity
- Creates the PubIn & PubFailed message queues in MSMQ on the Scribe server
For CRM, a set of plugins are created within CRM that populate the new_ ScribePublisherQueue entity.
When a Publisher is active, it will pull data from the ScribePublisherQueue and create messages in the appropriate PubIn message queue.
Query Publishers generate messages directly into the ScribeIn queue. However using a Console Publisher to migrate data from MS Dynamic via MSMQ is a two-step process.
The first step is to create a message in the AX-GP-NAV-CRM PubIn queue. The message contains the minimal information required to query the Dynamics database to retrieve the data that is used to build the detail message in the next step.
The Publisher then creates the detailed message for the ScribeIn queue containing all of the fields required to update the target. This process can take longer when there is a lot of data to retrieve to build the message file. If for some reason the data can’t be retrieved, the message is sent to the PubFailed queue. If successful, the message is either discarded or sent to the PubSucceeded queue (if enabled).
Messages in the ScribeIn queue are picked up by the TIBCO Scribe® Insight Integration Processes (IP). Each message has a message label that is used by an IP to find messages that belong to them. Message labels that don’t match any of the IP processes will be sent to the ScribeDeadMessage queue. If there is an error during processing, messages are sent to the ScribeRetry queue. The IP process will attempt to reprocess the message based on the retry count in the IP configuration. Once the retry account is reached the message is sent to the ScribeDeadMessageQueue.
Troubleshooting Steps
Installation and Configuration Settings - Verify the following:
MSMQ is installed and the queues exist
- MSMQ installed on the Scribe and Dynamics servers
- ScribeIn, ScribeRetry & ScribeDeadMessage queues exist on the TIBCO Scribe® Insight server
- AX-GP-NAV-CRM PubIn & PubFailed queues exist on the TIBCO Scribe® Insight server
Scribe AdminServer
Scribe BridgeServer (MSMQ must be running)
Scribe EventManager
Scribe MessageServer (MSMQ must be running)
Scribe MonitorServer
Message Queuing (MSMQ)
Message Queuing Triggers
Application specific services:
CRM Asynchronous Processing Service (MSCRMAsyncService)
NAS Service
Scribe BridgeServer (MSMQ must be running)
Scribe EventManager
Scribe MessageServer (MSMQ must be running)
Scribe MonitorServer
Message Queuing (MSMQ)
Message Queuing Triggers
Application specific services:
CRM Asynchronous Processing Service (MSCRMAsyncService)
NAS Service
Verify Scribe Services Logon user is a Domain User that belongs to the local Administrators Group
Is the Scribe service LogOn user a member of the Scribe Console User group (windows group)
Verify permissions on TIBCO Scribe® Insight related Private Queues (Scribe, CRM, etc)
- Windows 7: Computer Management > Services and Applications > Message Queuing
- Windows 2008: Open Server Manager, Expand Features node
- Right-click a queue > Security tab > Make sure that "Everyone" has "Full Control" Allowed (or the logon user for the Scribe services should have Full Control to queues)
Sharing Queues (Queue Browser)
- Share the AX-GP-NAV-CRM and Scribe queues. This makes it easier to manage the messages and troubleshoot.
- Scribe queues are there by default. The other queues must be manually added.
- Copy and paste the CRM or GP message queue path/name from the publisher into the Shared Message Queues
Verify there is only one Publisher per target (CRM Organization, GP Database – Each GP Company must be in its own DB)
Have the number of messages reached the MSMQ disk space limit (or no messages are generated)
- If you get a “Probable cause is a message larger than 4MB” error message then MSMQ may be full. Purge MSMQ System Queues Dead-Letter messages and/or increase the maximum storage allotment for messages (see below)
- Check the number of messages in the Scribe, MS Dynamics and MSMQ system queues.
- Do not purge the ScribeRetry or ScribeDeadMessage MSMQ unless you are certain the messages do not need to be processed.
For the MSMQ System Queues – Open Server Manager and expand the Features node
Purge MSMQ System Queues Dead-Letter messages if necessary
Check maximum storage allotment for messages (1048576 = 1 GB) – increase if necessary
Check the size of .mq files in C:\Windows\System32\msmq\storage
Query Publishers
Ensure the query is using the NOLOCK option on the Select
Select * From Account with (NOLOCK))
FROM dbo.MyTableX AS x WITH (NOLOCK)
INNER JOIN dbo.MyTableY As y WITH (NOLOCK)
ON …..
Keep the number of fields in the select to the minimum (e.g. don’t query unused fields)
Troubleshooting Steps
One of the more common issues encountered is that messages are not getting sent to the target. After verifying the installation and configuration steps above, the next step is to troubleshoot the message process.
Pause all MSMQ related Integration Processes and Publishers
Ensure the query is using the NOLOCK option on the Select
Select * From Account with (NOLOCK))
FROM dbo.MyTableX AS x WITH (NOLOCK)
INNER JOIN dbo.MyTableY As y WITH (NOLOCK)
ON …..
Keep the number of fields in the select to the minimum (e.g. don’t query unused fields)
Troubleshooting Steps
One of the more common issues encountered is that messages are not getting sent to the target. After verifying the installation and configuration steps above, the next step is to troubleshoot the message process.
Pause all MSMQ related Integration Processes and Publishers
- Restart the Bridge Sever service to ensure message processing has completely stopped.
Enable the PubSucceeded Message Queue
- If messages are not getting to the ScribeIn queue then enable the appropriate AX-GP-NAV-CRM PubSucceeded queue.
- Publishers can have filters and it is possible a message is discarded because of the filter. Any messages that are discarded for this reason would be sent to this queue if it is enabled.
Are records getting created in the ScribePublisherQueue table (SPQ)
Test the Message Queue 
Messages exist in the ScribeIn Queue but are not processed 
If the label and everything else seems correct and the messages are still not processed from ScribeIn then try the following:
Messages ultimately end up in the ScribeDeadMessage Queue 
- Make a change in MS Dynamics that should generate a message.
- Use SQL Server Management Studio to query the table and verify if:
- Records are getting created
- Large numbers of records exist that have not been processed
-
- If records are not getting created:
- Verify the appropriate Insert, Update or Delete options are checked in the Publisher
- May be a database trigger Or CRM Plugin issue
-
- For AX-GP-NAV - Remove and read the entity from the Publisher in the Console. This should attempt to remove and then re-add the database triggers.
- For CRM you can use the CRM Plugin Registration Tool to remove the Scribe plugins from CRM. Reconnecting to CRM from the Console Publisher and clicking Apply we recreate the plugins in CRM.
- If the GP ScribePublisherQueue contains an abnormally high number of records, contact support in order to review the GP triggers.
- Start the Publisher and verify if the message was sent to either the ScribeIn and/or AX-GP-NAV-CRM PubSucceeded queues.
- A message that is in the PubSucceeded queue but not in the ScribeIn queue may indicate it was discarded based on a Publisher entity filter.
- If the message is in neither queue check the AX-GP-NAV-CRM PubFailed queue.
- Use the use the key information in the PubFailed message to determine if the record exists in the AX-GP-NAV-CRM database using SQL Server queries. Messages will always fail if the data can’t be retrieved with the error “Matching Entity not Found”
- Messages can be manually reprocessed by using the Console Query Browser and dragging a message from a failed queue into the ScribeIn or AX-GP-NAV-CRM PubIn queue.
- Verify the “Ignore User” is not the cause of the problem. If you specified an "ignore user" in the Publisher settings and login/connect as this user, any changes are ignored by the Publishers.
Test the Message Queue
- If messages are not getting created, try using the TestMessageQueueUtility found in …\Scribe\TestMessageQueueUtility.exe
- Copy the executable to the server where you want to test sending a message to the message queue.
- This could be either the TIBCO Scribe® Insight server or MS Dynamics server.
Messages exist in the ScribeIn Queue but are not processed
- The message may be sent to the ScribeDeadMessage queue with the error “No matching Integration Process found”
- Verify the Integration Process is running and the Message Label matches the value specified in the Integration Process
- Move a message from ScribeIn to ScribeDeadMessage queue and then back to ScribeIn. This creates a new message with the same content.
- Recreate the connections in the DTS. In one instance, recreating the connection resolved the issue.
Messages ultimately end up in the ScribeDeadMessage Queue
- If the error is not apparent from looking at the ExecutionLog, try debugging the issue by manually running the message from the DTS in the Workbench.
- Any DTS that uses message queues utilizes an XML file. Select a failed message and use the Save Body option to save the file into the same folder as the DTS
- Do labels match (see above)
Issue/Introduction
This article provides an overview of the TIBCO Scribe® Insight Publishers and advanced troubleshooting steps.
Feedback
Yes
No
Powered by
