How can we find out for a running node how much memory is currently being used?

In a TIBCO® Streaming node there are several memory types to be aware of.

TYPES OF MEMORY

1. Total Memory used by a process as reported by the Operating System

At the OS level, the 'pmap {pid}' command shows on the last line the total memory used by a process. The process to check is the one with the PID shown on the "Listening" line of the console log:

2020-03-12 08:28:36.677000-0400 [55916:Thread- ThreadPool - 1] INFO  com.streambase.sb.sbd.net.StreamBaseHTTPServer: 
sbd at mysystem:10000; pid=55916; version=10.5.0_ccb32e4ca924f05a0d59933f58d9d573d97e7fa1; Listening

The 'jps' command also shows the PIDs:
> jps
39012 org.eclipse.equinox.launcher_1.5.400.v20190515-0925.jar
59524 Jps
27752
55916 com_example_testapp_TestApp1

2. Java Virtual Machine (JVM) Heap and Native Memory

Within the JVM, Heap memory allocation and usage is best monitored in real-time using the 'jconsole' application. The 'jconsole' application shows how much Heap is used, and in addition how much off-Heap native memory is used as well.

3. Transactional Memory in use within the node

Transactional Memory is the memory synchronized between nodes.

Within the node to see shared memory (ossm) usage, use commands:

> epadmin servicename=A.X display statistics statistics="memoryusage"
[A.X]
[A.X] ======== Memory Usage report for A.X ========
[A.X]
[A.X] Data gathered between 2020-03-12 08:28:13 Eastern Daylight Time and 2020-03-12 08:56:45 Eastern Daylight Time.
[A.X]
[A.X] Shared Memory Size (bytes)    % Utilized    Throttle State    Time
[A.X]                  536870912            35             Clear    2020-03-12 08:56:45.600000

and 

> epadmin servicename=A.X display statistics statistics="objectmemory"
[A.X]
[A.X] ======== Object Memory Usage report for A.X ========
[A.X]
[A.X] Data gathered at 2020-03-12 08:54:57 Eastern Daylight Time.
[A.X]
[A.X]   Instances  Megabytes Used Class Name
[A.X]           2           0.002 com.kabira.platform.Metadii
[A.X]           3           0.003 com.kabira.platform.component.NodeNotifier

This is described in the runtime documentation for the 'epadmin' command found here:
http://devzone.tibco.com/sites/streambase/latest/sb/sb-product/documentation/adminguide/ch09s02.html#sect_statistics_target

Transactional Memory is configured in the HOCON configuration:

type = "com.tibco.ep.dtm.configuration.application"
configuration = {
  ApplicationDefinition = {
    execution = {
      nodeTypes = {
        sample-node-type-0 = {
          sharedMemory = {
            memoryType = "SYSTEM_V_SHARED_MEMORY"
            memorySizeBytes = "512M"

When configured as SYSTEM_V_SHARED_MEMORY this is kept as an object in live RAM as part of the native (not on the Heap) memory allocated to the process by the OS. When configured as FILE this is kept on-disk in the physical 'ossm' file within the node directory.

MONITORING

For real-time memory monitoring use 'jconsole', linux 'top', and Windows Task Manager.

To log memory use for a TIBCO Streaming node, add to the node configuration the "additional memory use and garbage collection logs" shown here: 

StreamBaseEngine = {
    jvmArgs = [
      # heap memory size limits
      "-Xmx4096m"
      "-Xms512m"
      # recommended default settings
      "-XX:+UseG1GC"
      "-XX:MaxGCPauseMillis=500"
      "-XX:ConcGCThreads=1"
      # additional memory use and garbage collection logs
      "-verbose:gc"
      "-Xloggc:gc.log"
      "-XX:+PrintGCDetails"
      "-XX:+PrintGCDateStamps"
      "-XX:+PrintVMOptions"
    ]
  }

The gc.log file will show garbage collection activity, and incidentally how much Heap memory is allocated and used. For example, this line in the output shows Heap size changes:
[Eden: 1937.0M(1937.0M)->0.0B(1961.0M) Survivors: 67.0M->40.0M Heap: 3195.5M(3582.0M)->1216.6M(3582.0M)]

Eden: 1937.0M(1937.0M)->0.0B(1961.0M)
Eden's capacity was 1937MB and it was full. After this GC event the young generation occupied size came down to zero. The target capacity was changed to 1961MB.

Survivors: 67.0M->40.0M
The Suvivor space was 67MB before and 40MB after, indicating some prior survivors were collected.

Heap: 3195.5M(3582.0M)->1216.6M(3582.0M)
Heap capacity was 3582MB of which 3195MB was utilized. After this GC event, 1216MB was utilized and the capacity remained the same.

PROFILING

Heap Memory usage is also collected in the output of the 'sbprofile' command in the "System Statistics" record (lines beginning with 'S'). The 'sbprofile' command is described in the product documentation topics:
* TIBCO Streaming > StreamBase Admin Guide > Monitoring and Profiling > Profiling, "Real-Time Profiling with sbprofile"
* TIBCO Streaming > StreamBase References > Command Reference > sbprofile
 

Administration guidance