Statistica Python node function "ActiveDataSet" gives _ctypes.COMError when reading large input data to Pandas data frame
Article ID: KB0077055
Updated On:
| Products | Versions |
|---|---|
| Spotfire Statistica | 13.5 |
Description
In Statistica Python scripting node, "ActiveDataSet" function can be used to read an upstream input spreadsheet to a Pandas data frame, for example "trainDS = ActiveDataSet['trainData']" or "trainDS = ActiveDataSet[0]".
When reading a large input spreadsheet (e.g. 30000v * 2000c) with ActiveDataSet function to a Pandas dataframe, the python node execution gives below error message:
_ctypes.COMError: (-2147024882,'Not enough storage is available to complete this operation.')
This error happens to Python version 3.5.2, 3.6.3, 3.7.3 and 3.7.4.
Environment
Resolution
For large input dataset, the user could use below workaround to avoid above issue:
1. First export the input spreadsheet data into a CSV file with comma as separator through Statistica "Export Data" node
2. Then read the CSV file from local directory into a Pandas data frame through "pandas.read_csv" function in Python Node. For example,
import pandas as pd
#data = pd.read_csv("local CSV file path"), for example
data = pd.read_csv("C:/Users/lizliu.APACANALYTICS/Desktop/AllColumn4Test.csv")
Issue/Introduction
Feedback
