Understanding File Size Discrepancies in SBDF Export: Spotfire Custom Application vs. Spotfire Analyst Client

Understanding File Size Discrepancies in SBDF Export: Spotfire Custom Application vs. Spotfire Analyst Client

book

Article ID: KB0070426

calendar_today

Updated On:

Products Versions
Spotfire Analyst All

Description

Issue Description:
When exporting data to SBDF (Spotfire Binary Data Format) files from a custom Spotfire application versus the Spotfire Analyst client, users may notice a significant difference in file size. This KB article explains why this occurs and how to manage it effectively.

Root Cause:
The primary reason for the variance in file size lies in the utilization of Run-Length Encoding (RLE) blocks. Specifically, SBDF files generated by the custom application do not employ RLE blocks, resulting in larger file sizes. In contrast, when exporting from Spotfire Analyst, RLE is utilized for certain blocks, reducing the file size.

Technical Background:
The Spotfire.Dxp.Data.Formats.Sbdf.dll library provides support for RLE through the API. Users can leverage the SbdfValueArray.CreateRunLengthArrayEncoding method to enable RLE encoding. This array is then utilized within the SbdfColumnSliceBuilder constructor. In Spotfire 11.7, export and TERR data functions were enhanced to incorporate these methods, thereby enabling more efficient data compression through RLE (see article about What's New in TIBCO Spotfire 11.7)

Resolution

When implementing SBDF export functionality, developers should consider utilizing the more advanced API features provided by Spotfire.Dxp.Data.Formats.Sbdf.dll. This includes employing the SbdfTableSliceBuilder in conjunction with SbdfColumnSlice and SbdfValueArray to enable RLE encoding where applicable. By leveraging these components effectively, developers can optimize file size without compromising data integrity or performance.

Issue/Introduction

This KB article aims to explain why the SBDF (Spotfire Binary Data Format) file size exported from a custom Spotfire application is larger than the Spotfire Analyst client.

Additional Information

Spotfire Community Article: Spotfire SBDF Library

Doc: Spotfire.Dxp.Data.Formats.Sbdf Namespace