Introduction
To test a Map, it can be useful to stop after a certain number of rows, instead of processing all of them. Sometimes this is faster than debugging a Map step by step. Or, you may want to run though a subset of the records to help find subtle data or logic errors. You can also use this method to get performance metrics. Run the Map on a moderate # or records (i.e. 50k) and record the the rows/second in the Execution History. Use this rate to estimate how long your full migration will take.
Steps
Use the RECNUM() function, an If/Else Block and a Map Exit control Block.
- Edit your Map. It should already contain at least a top Query Block and a For Each Result control Block.
- Place a Group Comment Block at the top of the For Each Result control Block. Name it "Stop after # of records."
- Place an If / Else control Block in the Comment Block.
- Add a condition to the If Block
- Left Value: RECNUM()
- Operand: is greater than
- Right Value: [numeric value]
- Place a "Map Exit" control Block under the "If"
- Turn off Batch Processing in all Blocks where it's activated. This is necessary because the Map Exit Block flushes the cache in the current version of TIBCO Scribe® Online.
Here's a screen shot:
Notes
- Placing the If / Else Block at the top of the Map ensures it is always executed. If you put it at the end of the Map, it won't get executed on failed records.
- As a result, the Execution History reports 1 more record than is actually processed.
- Before running the Map on all records, remember to turn Batch Processing ON and remove the Group Comment, If / Else, and Map Exit Blocks.
Pro Tip: By putting the If/Else in a Comment Block, you can drag the comment Block to the scratch area. This de-activates the Block. Here's a screenshot of the Map with the Comment Block closed and moved to the scratch area.