There are several integration points between TIBCO Streaming and Prometheus.
The Prometheus framework prefers Exporter implementations which pre-package the data as Counter, Gauge, Summary, and Histogram types. This architecture requires every new data source to have an Exporter created for it.
StreamBase (SB) output streams are custom to the schema of the tuple being emitted and have no required structure or naming conventions. StreamBase servers do have consistent APIs for monitoring overall server health independent of the specific application being run, as described in the product documentation here:
TIBCO Streaming Documentation > Administration Guide > Using StreamBase Containers > System Container and Control Streamsand
TIBCO Streaming Documentation > Administration Guide > Using StreamBase Containers > StreamBase Container Overview, "The System Container"Some integration approaches to consider are:
1) Create custom Java operators which use the Prometheus Java Client APISee the sample project loadable from Studio menu:
File > Import Samples and Community content > TIBCO Support Samples > Web > "
Web: Prometheus Metrics Integration"
NOTE: If you are not using the latest version of TIBCO Streaming, the default version of the project from GitHub may be incompatible. See "
Note 1" below for alternative download instructions to obtain specific versions of the samples.
The Prometheus API reference is here:
https://prometheus.io/docs/instrumenting/clientlibs/2) Use JMXSee:
https://github.com/prometheus/jmx_exporterYou will need to tell your SB server what JMX port to expose using jvm-args (example):
-Dcom.sun.management.jmxremote.port=3333
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=falseSee SB documentation page:
TIBCO StreamBase Documentation > Administration Guide > Monitoring and Profiling Applications > JMX Monitoring3) Create an HTTP output adapterPrometheus provides a client library which appears suitable for inclusion in a custom Java output adapter. The adapter would expose an HTTP end-point for the Prometheus server to connect to at will. The SB processTuple() command would submit tuples as they arrive to the Counter, Guage, and other classes as described here:
https://github.com/prometheus/client_java4) Write SB tuples to a CSV fileA 3rd-party developer has provided a Prometheus CSV file "tail" Exporter available here:
https://github.com/stohrendorf/csv-prometheus-exporterStreaming's
CSV File Writer output adapter may then be used to emit to this file.
Additionally, Prometheus metrics may be sent
into Streaming using approaches from the Streaming LiveView project found here:
https://github.com/TIBCOSoftware/tibco-streaming-community/tree/master/components/lv-prometheus-viewerNote 1: The GitHub recommended version of the sample "Web: Prometheus Metrics Integration" may not match your current Studio version, and at the time of this writing the StreamBase Sample Importer will only download the latest or recommended version.
To obtain the samples version matching your Studio version:1. Open in your web browser the "tibco-streaming-samples" webpage:
https://github.com/TIBCOSoftware/tibco-streaming-samples/tree/master2. Change the branch selection from "
master" to your preferred version, for example "10.6.1".
3. Click on the green "
Code v" button and select "
Download ZIP" and save to your computer.
4. Extract the "tibco-streaming-samples-10.6.1.zip" file.
5. In Studio, use menu:
File > Import... > Maven > Existing Maven Projects.
6. In the dialog, use "
Browse" to where you expanded the samples and to the "web\prometheus-metrics" folder and click "
Select Folder".
7. Click "
Finish" in the dialog.
This download includes a copy of all the GitHub samples for the selected Studio version, so for other samples you may use your single download and not have to repeat the download for other samples. There may be fixes in later versions of the samples, but correcting specific version incompatibilities is beyond the scope of this note.
Note 2: Non-TIBCO URLs identified in this article are subject to change without notice. The non-TIBCO references are provided as examples without specific recommendation.