There is metrics data maintained in the agent. Along with the engine running, more and more metrics data is generated and stored with the agent which leads to the memory increase. With the default setting, metrics data for a week is maintained.
There are two options to prevent the agent from running into the java OutOfMemory issue:
1). Increase the Max heap setting for the agent to allow the persistence for one week metrics data. To determine what is a sufficient heap requires a bit of sizing exercises. You need to compute how much steady-state RAM it takes to hold a day's worthy data in memory and use 10x that number for a max heap size for a week's retention.
2). Reduce the persistence of metrics data by purging data more frequently. The retention policies settings are in BE_MM_Schema.xml located at <BE_HOME>\teagent\config\schema.
<schema name="BE_MM">
<retention-policies>
<retention-policy type="fact" period="1" unit="DAY" purge-time-of-day="-1" purge-frequency-period="86400000"/>
<retention-policy type="BEMMCube/instancehealthhierarchy" period="4" unit="HOUR" purge-time-of-day="-1" purge-frequency-period="3600000"/>
<retention-policy type="BEMMCube/eventhealthhierarchy" period="4" unit="HOUR" purge-time-of-day="-1" purge-frequency-period="3600000"/>
<retention-policy type="BEMMCube/clusterhealthhierarchy" period="4" unit="HOUR" purge-time-of-day="-1" purge-frequency-period="3600000"/>
<retention-policy type="BEMMCube/rulehealthhierarchy" period="4" unit="HOUR" purge-time-of-day="-1" purge-frequency-period="3600000"/>
<retention-policy type="BEMMCube/inferencehealthhierarchy" period="4" unit="HOUR" purge-time-of-day="-1" purge-frequency-period="3600000"/>
</retention-policies>
The above policy will purge all records older than one day and the purge will happen every day. You can change the period to other values to purge data after X days/hours. Possible values for unit - WEEK, DAY, HOUR.
