How would Concept TTL impact historical concepts in the space

How would Concept TTL impact historical concepts in the space

book

Article ID: KB0083807

calendar_today

Updated On:

Products Versions
TIBCO BusinessEvents Enterprise Edition 5.4.0

Description

We started supporting setting TTL for Concept starting with TIBCO BusinessEvents 5.4.0. If you decide to set TTL for a specific concept then the the historical entries created before setting the Concept TTL whether they are created in BE 5.2.2 or BE 5.4.0 will not expire. The reason for this is that there is a Space level TTL and Entry level TTL and for historical entries, the Entry level TTL is still -1 (never expires). If you run as-dump on space with old entries and entries created after setting TTL, you will see something that resembles the following:

Tuple Created when Entry TTL -1:

<EntryId> 7f000001-c351-593ed4b5-1 </EntryId>
<EntryTtl> -1 </EntryTtl>
<EntryTime> 0 </EntryTime>
<RemainingTtl> -1 </RemainingTtl>

Tuple Created when Entry TTL 300 seconds:

<EntryId> 7f000001-c351-593edf34-327-5aa </EntryId>
<EntryTtl> 300000 </EntryTtl>
<EntryTime> 3 </EntryTime>
<RemainingTtl> 290000 </RemainingTtl>

One caveat for this behavior is described below:

- Concept instance X was created with default (-1) TTL.
- You change the concept TTL to 300 seconds (5 minutes) and restart the cluster.
- After changing the TTL you load concept X and made a change to the concept that forced an RTC. 
- The RTC is processed and Concept X with changed fields is written to cache and backing store.
- If the TTL of the concept would change to new setting as you touched the concept.

Issue/Introduction

How would Concept TTL impact historical concepts in the space?

Environment

All Supported Platforms

Resolution

Solution is to clear unnecessary historical entries before setting Concept TTL is to add a cleanup engine to your cluster that can query entries, load them to memory and delete them. For instance, you can run your cleanup engine every week during weekends so that you are not impacting your cluster during your normal hours. Once the cleanup is done stop the engine.

Additional Information

case 01478532