Bulk Operations In TIBCO Scribe® Insight
Article ID: KB0078906
Updated On:
| Products | Versions |
|---|---|
| TIBCO Scribe Insight | 7.9.1 |
Description
If you use Bulk Operations in TIBCO Scribe® Insight DTSs, you should be aware of the following:
For additional information, see the following in the TIBCO Scribe® Insight Help: Bulk Operations
- Bulk operations use an array based on batch size to store data. An array is created for both the source and target. It is best to limit the number of fields by using a query with specific fields. Be aware of fields that contain binary data which can impact array/batch size and performance.
- There is an overall 2 GB limit to the data that can be handled in a single bulk operation.
- Many features are not supported when using bulk operations, such as comparing fields or multi-record.
- Cross Reference tables are updated after the array is processed.
- Any data in the array is processed even if the job is cancelled or the process reaches an End Job in the DTS flow control.
- Bulk loading is 3-4 times faster than traditional processing (40-45 records/sec).
- Both bulk and non-bulk operations are supported in a single DTS. Bulk operation process design should be kept simple. Bulk operations in one DTS can chain to another DTS for more complex operations.
- A classic scenario for using mixed mode is having a non-bulk seek step for an Order Header and a bulk load step for processing the order detail data, which can contain multiple records.
- For MSMQ scenarios, the queue priority is set when the message is created for the ScribeIn MSMQ.
- Automatic Foreign Keys (AFK) can automatically be assigned when going from a Non-Bulk step to a Bulk step but not the other way around.
For additional information, see the following in the TIBCO Scribe® Insight Help: Bulk Operations
Issue/Introduction
General notes on Bulk Operations in TIBCO Scribe® Insight
Feedback
Yes
No
Powered by
