Example: <timestamp> <inferenceAgent> Error [ManagementCentral:ScheduledExecutorService:1] - [runtime.management] Metric publication failed java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at com.tibco.cep.runtime.management.impl.cluster.ASMetricTable.addMetricData(SourceFile:141)
or
<timestamp> <<inferenceAgent> Error [$default.be.mt$.Thread.7] - [runtime.scheduler] java.lang.OutOfMemoryError: Java heap space java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at com.tibco.cep.kernel.core.rete.BeTransaction.execute(SourceFile:122) at com.tibco.cep.kernel.core.rete.ReteWM.executeRules(SourceFile:1392)
Cause: ====== The Workers ThreadCount has been set to a large value (e.g., 100) in the Cluster Deployment Descriptor (CDD) file. Each Worker Thread reads events from a destination (e.g.,EMS queue) and the same ProcessUnit schedule event expires after a specific time. There is only one intrinsic CachePoller scheduler thread. It cannot keep up the rate of expirations compared to the incoming rate of events. This is why the memory grows until max, HeapSize is reached and the Exception will be thrown.
Resolution: =========== There are two options to avoid the issue.
1). Disable the scheduler for that InferenceAgent and CacheServers.Add the property be.engine.cluster.scheduler=false to the CDD file for all Inference Agents and CacheServers which are not required to behave as a scheduler (enable the schedule for at least one InferenceAgent).
2). Reduce the number of Worker Threads for the ProcessUnit (e.g., to default 10)