How to avoid the Out of Memory Exception in TDV while accessing TDV Odata services which has millions of Records?

How to avoid the Out of Memory Exception in TDV while accessing TDV Odata services which has millions of Records?

book

Article ID: KB0079985

calendar_today

Updated On:

Products Versions
TIBCO Data Virtualization 7.0.7,All, , ,

Description

I am  trying to access the TDV Odata service which has more than 10 million records (http://172.21.246.204:9800/odata/odata/complex10mview)
from the browser  and these below issues are occurring  
 

1) Memory spikes at a fast pace  and When this happens system is in accessible. 

2) During a data pull from a browser the user could click the browsers "stop" button or kill the browser itself. DV continues to process the query. The queries have consume resources in the system. 

3) Found below error message in the logs 

ContainerResponse - The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.OutOfMemoryError: Requested array size exceeds VM limit

Please let me know what is needed

Issue/Introduction

How to avoid the Out of Memory Exception in TDV while accessing TDV Odata services which has millions of Records?

Resolution

Please use paging functionality to avoid OOM when TDV hangs if client tries to get a large number of data.

1)PSB the sample  modified URL you need to try from browser.
http://172.21.246.204:9800/odata/odata/complex10mview?$top=1&$skip=2

The $skip System Query Option specifies that the result MUST NOT include the first n entities, where n is a non-negative integer value specified by the $skip query option.
The $top System Query Option specifies that only the first n records SHOULD be returned, where n is a non-negative integer value specified by the $top query option.