Using the Kafka Consumer adapter to read from last read message after restart
Article ID: KB0080264
Updated On:
| Products | Versions |
|---|---|
| TIBCO Streaming | - |
Description
While receiving Kafka data, when the StreamBase application with the Kafka adapter is restarted, what configuration is used to start reading data from the last read message?
Issue/Introduction
Configuration guidance
Resolution
After restarting the application, when sending a "subscribe" command to the Kafka Consumer "Control port", also set the optional field "clientId" to a non-null value which is the same value used from the previous session. When setting "clientId" Kafka's default behavior is to send new messages from the previous commit point. Do not also set either "time" or "offset" fields in the subscribe command. If "clientId" is not set then Kafka cannot determine what it has sent to the adapter.
Commit points happen at intervals controlled by Kafka, or can be set by using the Kafka Consumer Commit adapter.
Commit points happen at intervals controlled by Kafka, or can be set by using the Kafka Consumer Commit adapter.
Feedback
Yes
No
Powered by
