When running a TIBCO Streaming Docker container in Kubernetes or OpenShift, the default probe is the docker ENTRYPOINT which is the launch script "start-node".

How can we configure the probes to monitor the actual server process?

Configuration guidance

This information is written for version 10.5 and is subject to change in future releases. Please consult the current product documentation.

When the Application fragment is configured to build a docker image, these are added to the project:

src/main/docker/application/
  Dockerfile - "ENTRYPOINT ${PRODUCT_HOME}/start-node"
src/main/docker/base/
  start-node

This is described in the product documentation here:
  TIBCO Streaming > StreamBase Admin Guide > Using Docker > Configuring Docker Images and Networks

Do not change the ENTRYPOINT. The start-node script needs to remain the ENTRYPOINT to prepare the runtime environment for the administrative 'epadmin' command which installs and starts the application.

The docker image itself cannot tell kubernetes what to probe for liveness or readiness. This must be configured outside the image using kubernetes .yaml configuration (ex: pods/probe/exec-liveness.yaml). The 'epadmin' commands run by 'start-node' only run briefly to start the Java VM for the application engine and are not suitable for a probe to monitor.

The persistent node processes are:

• swcoord - node coordinator process. Starts and monitors all other node processes and hosts the administration port.
• System::swcoordadmin - node coordinator administration IPC service. Provides an IPC mechanism between the node coordinator process and the rest of the processes on a node for administration commands.
• System::administration - node administration. Provides node administration commands.
• System::kssl - provides credential and access control enforcement.
• application::<engine-name> - the Java VM running the SB, LV, or Java application.

When 'epadmin install node' executes, all of these processes are started except the application engine JVM. When 'epadmin start node' executes, the "application::<engine-name>" process then starts. It is only after the application process starts that kubernetes liveness and readiness probes should begin. 

The kubernetes probe may monitor for any or all of these processes to confirm liveness.

For details about kubernetes probe configuration, see:
  Configure Liveness, Readiness and Startup Probes
  https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

A suitable probe command to confirm that the engine has not stopped is:
  test `epadmin servicename=A.X display engine name={engine-name} | grep -c "] State = Started"` -eq 1
where we expect to see this line once in the "display engine" output:
  [A.X] State = Started

When a node is freshly installed and started, the engine may take tens of seconds to several minutes depending on the application size to compile EventFlow code into Java byte-code, instantiate the StreamBase containers and LiveView Tables, and connect to external JDBC databases and other external services. The above test will report "Started" immediately, before data is ready to flow. 

A StreamBase application is ready for data when it reports in the console log:

2020-02-11 14:34:42.989000-0500 [92272:Thread- ThreadPool - 1] INFO  com.streambase.sb.sbd.net.StreamBaseHTTPServer: sbd at system:10000; pid=92272; version=10.5.0_ccb32e4ca924f05a0d59933f58d9d573d97e7fa1; Listening

A LiveView application is ready when it reports to the console:

2020-02-11 15:20:09.340000-0500 [15176:AsyncLogger] INFO  { Server Started } :

*** All tables have been loaded. LiveView is ready to accept client connections.  Total start time(ms): 52178 ***

You may find these phrases in the console log found within the node directory's "logs" sub-directory.

You may look for these phrases, or do your own application-specific readiness checks.