How to collect statistics related to shared memory usage in a Streaming application
Article ID: KB0072851
Updated On:
| Products | Versions |
|---|---|
| TIBCO Streaming | 10.6 and later |
Description
After running for some time, my application is exhausting shared memory. I see the following messages in the Streaming console log:
How can I monitor the application to see why shared memory is being exhausted? I'm unable to take a full node snapshot, as this is a production application, and we want to minimize any impacts on performance.
Low memory warning: 90 % of shared memory in use for segment 0 2021-07-27 23:38:37.955000-0700 [427552:442480] ERROR com.tibco.ep.dtm.sharedmemory: (smalloc.cpp:634) Shared memory exhausted in shared memory file C:\nodes\A.sbuser\ossm when trying to allocate 33589080 bytes. You must shutdown the node, remove shared memory, and increase the shared memory size.
How can I monitor the application to see why shared memory is being exhausted? I'm unable to take a full node snapshot, as this is a production application, and we want to minimize any impacts on performance.
Issue/Introduction
Outlines the steps needed to collect statistics that are relevant to issues relating to shared memory.
Resolution
Use the 'epadmin snapshot statistics' command to collect a sub-set of statistics that are related to shared memory usage. These are discussed in the Help under Home > EP Commands > epadmin and epdev References > epadmin-statistics.
To enable the collection of statistics relevant to shared memory, run:
Once enabled, you may collect regular snapshots of these statistics using the 'epadmin snapshot statistics' command:
This will collect relevant statistics for a period of 10 seconds, and write them to the file 'stats1.txt'. It is advised to run this collection at regular intervals, so the snapshots can be compared to each other. This should help reveal why shared memory is being exhausted over time.
To enable the collection of statistics relevant to shared memory, run:
epadmin --servicename=<YourNodeName> enable statistics --statistics="eventbus,eventlog,hash,mutex,native,object,objectmemory"
Once enabled, you may collect regular snapshots of these statistics using the 'epadmin snapshot statistics' command:
epadmin --servicename=<YourNodeName> snapshot statistics --statistics="eventbus,eventlog,hash,mutex,native,object,objectmemory" --seconds=10 > stats1.txt
This will collect relevant statistics for a period of 10 seconds, and write them to the file 'stats1.txt'. It is advised to run this collection at regular intervals, so the snapshots can be compared to each other. This should help reveal why shared memory is being exhausted over time.
Feedback
Yes
No
Powered by
