After running for a little while our application fails with this error:

 [A.X] default-engine-for-com.businessapp:2021-02-09 10:51:33.000000-0500 [8572] ERROR com.tibco.ep.dtm.sharedmemory: (smalloc.cpp:634) Shared memory exhausted in shared memory file   E:\PRODSYSTEM\A.X\ossm when trying to allocate 57344 bytes. You must shutdown the node, remove shared memory, and increase the shared memory size [A.X] default-engine-for-com.businessapp:2021-02-09 10:51:33.000000-0500 [8572] WARN  com.tibco.ep.dtm.runtime: (native_tran.cpp:267) CATCH(DSEException) [A.X] default-engine-for-com.businessapp:2021-02-09 10:51:33.000000-0500 [8572] WARN  com.tibco.ep.dtm.runtime: (dseexcpt.cpp:405) STACK TRACE: [A.X]   +++ START +++ [A.X]     smalloc.cpp (636): THROW SWSMAlloc::SMAllocResourceError: reason: 'Shared memory exhausted in shared memory file E:\PRODSYSTEM\A.X\ossm when trying to allocate 57344 bytes.   You must shutdown the node, remove shared memory, and increase the shared memory size' [A.X]     smalloc.cpp (707): RETHROW SWSMAlloc::SMAllocResourceError [A.X]     smalloc.cpp (860): RETHROW SWSMAlloc::SMAllocResourceError [A.X]     ostenv.cpp (1474): RETHROW SWSMAlloc::SMAllocResourceError [A.X]     ostenv.cpp (1495): RETHROW SWSMAlloc::SMAllocResourceError [A.X]     thread_resources.cpp (147): RETHROW SWSMAlloc::SMAllocResourceError [A.X]     ktvmmgr.cpp (4473): RETHROW SWSMAlloc::SMAllocResourceError [A.X]     native_tran.cpp (267): DUMP SWSMAlloc::SMAllocResourceError [A.X]   +++ END +++

How can we correct this problem?

Configuration guidance

This error is related to to the Transactional Memory configuration for the node which is configured too small for the amount of data being maintained in the running application.

If you have set your Query Table elements to be "in transactional memory", as so:

then instead of using JVM Heap Memory the Query Table is using shared Transactional Memory space, which is by default only 512MB.

See the " sharedMemory" setting in the ApplicationDefinition HOCON configuration file, documented here:
  TIBCO Streaming > Configuration Guide > Runtime Configuration Types > Runtime Application Configuration

Correct this by increasing the Transactional Memory that a node is allowed to use by using two configuration files, the ApplicationDefinition and NodeDeploy HOCON configuration files.

In the ApplicationDefinition, increase the " memorySizeBytes" value to big enough for your Query Table data. In the NodeDeploy configuration use the ApplicationDefinition through the nodeType setting.

ApplicationDefinition .conf example:

 name = "my_application" version = "1.0.0" type = "com.tibco.ep.dtm.configuration.application" configuration = {   ApplicationDefinition = {     execution = {       nodeTypes = {         "medium-capacity" = {                          sharedMemory = {             memoryAllocators = 16             memoryType = FILE             memorySizeBytes = 10g           }         }                                   }     }   } }

NodeDeploy configuration example:

 name = "NodeDeployment" version = "1.0" type = "com.tibco.ep.dtm.configuration.node" configuration = {   NodeDeploy = {     nodes = {       "${EP_NODE_NAME}" = {         nodeType = "medium-capacity"       }     }   } }

This example has given the nodeType configuration the arbitrary name "medium-capacity". You may name this anything with typical alphanumeric characters.

Use "${EP_NODE_NAME}" for the node name in the NodeDeploy configuration instead of a specific "{ nodename}.{ groupname}.{ clustername}" value (such as "A.USWEST.BIZAPP") to match any node and cluster you provide on the ' epadmin install node' command.

The ApplicationDefinition .conf file goes into the project " src/main/configurations" folder. The NodeDeploy .conf may also be in the project at the same place, or it may be maintained outside the project and referenced on the command-line as so:
  epadmin install node --nodename=A.X --application=app.zip --nodedeploy=myNodeDeploy.conf 

For more about Transactional Memory, please see the TIBCO Streaming product documentation:
  TIBCO Streaming > Concepts Guide > Transactional Memory
  TIBCO Streaming > Authoring Guide > Using Query Tables > Persistence of Query Tables, "Transactional Memory Option"
  TIBCO Streaming > Architecture Guide > Transactions

For performance, only put into Transactional Memory the information which must remain immediately available if a node in the cluster fails. A good example of this is short-term calculation results which are not available anywhere else. If the information is not immediately needed for speed, instead store it in a JDBC database. If the information is easily restored from external systems, use Query Table on Heap and fetch it externally as needed. If Transactional Memory is written frequently, then this produces many transactions which may slow the performance of the cluster (a cost of high reliability).