Resolution:

Data cleansing fixes a set of deficiencies affecting access to external databases. Without this fix, certain external data could, for example, lead to unrecoverable conversion failures.

In some database systems such as IBM’s IMS, segments and fields are often initialized to low-values (x’00’s). In addition, many fields commonly remain undefined to the database, thus no enforced adherence to the associated syntax and data values is possible. Accessing such data through Object Service Broker could then lead to conversion failures.

Data Cleansing allows removal of customer identified characters in string type fields, such as linefeeds, carriage returns and other unwanted characters or their translation to acceptable characters such as spaces. As well, numeric fields such as packed decimal, containing low-values or non-numeric values, can be treated as if they were NULL values rather than suffering an unrecoverable conversion failure.

Environment:
============
* z/OS
* external data with invalid values

Resolution:
===========
The data cleansing fixes were introduced in TIBCO Object Service Broker for z/OS 5.2.0 by Hotfixes 02, 03, 04 and 05.

Table types supported for data cleansing are: IMP, VSM, MAP, IMS and IDM.
Syntaxes supported for string data cleansing are: A,C,Q,V,W,X, and Z.

Details on how to configure the data cleansing feature for specific tables can be found in the ++HOLD(AP13529) information shipped with the hotfix package.

The 3270 Table Definer can be used to specify cleansing options at a table level under the specification "Data Cleansing:". Possible specifications are:                                
LOWVALUES    - treat invalid numeric data as null if low values (all bytes X'00')                      
INVALID      - treat invalid numeric data as null    
TRLOAD=name  - load translate table(s) from load module                                                  
TRTABLE=name - load translate table(s) from @TRTABLES instance  
  
where "name" is a maximum of 8 characters, and specifies the name of a load module or the value of the parameter TRNAME for table @TRTABLES, specifying the table instance to be used for translation.                                

Specifics regarding further customization of this feature can be found in the hotfix ++HOLD(AP13529) text file or in the attachment associated with this solution. To review the attachment, logon to TIBCO Support at
   https://support.tibco.com/esupport/
and search the Knowledge Base (KB) for SOL1-BTPVAM, then open the attachment.                              
    
Known Issues:
=============
1) As shipped in Hotfix02/03/04, the feature for MAP table cleansing was not activated. The activation maintenance for MAP tables was shipped in TIBCO Object Service Broker for z/OS 5.2.0 Hotfix05. If you need further assistance, please open a service request with TIBCO Support.

References:
===========
* ++HOLD(AP13529) shipped with TIBCO Object Service Broker for z/OS 5.2.0 Hotfix03 and all subsequent hotfixes for this release since hotfixes are cumulative.
* TIBCO Object Service Broker Managing Data
* TIBCO Service Gateway™ for IDMS/DB Installing and Operating
* TIBCO Service Gateway™ for IMS/DB Installing and Operating
* TIBCO® Object Service Broker for z/OS Installing and Operating
* TIBCO® Object Service Broker Managing External Data
* LBN1-BVDIXV TIBCO(R) Object Service Broker for z/OS 5.2.0 Hotfix 05 is now available.