How to configure an Aggregate Dimension for a rolling window
Article ID: KB0080618
Updated On:
| Products | Versions |
|---|---|
| TIBCO Streaming | - |
Description
The Aggregate operator has a number of options in the Dimension configuration. Which are important to set if I wanted the average value for the previous 30 seconds emitted every 5 seconds?
Issue/Introduction
StreamBase operator configuration guidance
Resolution
The settings to use to create a 30-second rolling window with an update every 5 seconds is to set the following options. Similar settings are used for a Tuple-based dimension where the tuple contains a timestamp field to be used to determine the current time.
Type = Time
The time-based dimension is controlled by the running system's internal clock. With every new data-point submitted to the Aggregate Operator, the system clock is checked to see if any of the following conditions have been met. In the absence of new data-points, the Aggregate is idle and will not emit even if the required time has passed. If you want the Aggregate to emit even if there is no regular data, use an upstream Heartbeat Operator to drive empty tuples into the Aggregate to trigger conditional evaluation.
Opening Policy, Open per: Advance = 5, Offset = 0
This causes a new window (a set of related tuples) to be created whenever a 5-second interval has passed. There's no Offset (0) because we want the window to open on the 5-second mark (example: 01:12:15.000 and not 01:12:17.000) and not some number of seconds after.
Window size, Close and emit after N seconds = 30
This closes any window that has been open at least 30 seconds, and the aggregate expressions are then evaluated on those window's contents.
Emission policy = No intermediate emissions based on this dimension
If set true, this would permit calculation on incomplete windows, every time a new window opened. That's not useful for a rolling average where inputs are expected to arrive regularly and therefore the quantity of underlying data-points for each calculated result remains close to the same.
Optional windows = Open only a single window for the first event or following a gap
This means that after an absence of updates which allow prior windows to expire, the next data-point causes one new window to open. The other setting causes the new data-point to open as many overlapping windows as it could be contained in.
Type = Time
The time-based dimension is controlled by the running system's internal clock. With every new data-point submitted to the Aggregate Operator, the system clock is checked to see if any of the following conditions have been met. In the absence of new data-points, the Aggregate is idle and will not emit even if the required time has passed. If you want the Aggregate to emit even if there is no regular data, use an upstream Heartbeat Operator to drive empty tuples into the Aggregate to trigger conditional evaluation.
Opening Policy, Open per: Advance = 5, Offset = 0
This causes a new window (a set of related tuples) to be created whenever a 5-second interval has passed. There's no Offset (0) because we want the window to open on the 5-second mark (example: 01:12:15.000 and not 01:12:17.000) and not some number of seconds after.
Window size, Close and emit after N seconds = 30
This closes any window that has been open at least 30 seconds, and the aggregate expressions are then evaluated on those window's contents.
Emission policy = No intermediate emissions based on this dimension
If set true, this would permit calculation on incomplete windows, every time a new window opened. That's not useful for a rolling average where inputs are expected to arrive regularly and therefore the quantity of underlying data-points for each calculated result remains close to the same.
Optional windows = Open only a single window for the first event or following a gap
This means that after an absence of updates which allow prior windows to expire, the next data-point causes one new window to open. The other setting causes the new data-point to open as many overlapping windows as it could be contained in.
Feedback
Yes
No
Powered by
