Pause a script until a Streaming application has fully started

Pause a script until a Streaming application has fully started

book

Article ID: KB0070222

calendar_today

Updated On:

Products Versions
TIBCO Streaming 10, 11, and later

Description

It is often needed when automating Streaming application startup using a script file to wait for the application engine to be ready before initiating client connections for initial data loading or other administrative actions.

What are the options for pausing a script after it executes ' epadmin start node' and wait for the application to be ready?
 

Issue/Introduction

Windows batch and Linux bash options for pausing a shell script

Resolution

When the EventFlow application engine is ready for client connections or administration commands, the engine log in the node directory will contain the "Listening" line like this:
 2023-10-02 12:35:51.632000-0400 [7948:Thread- ThreadPool - 1] INFO  com.streambase.sb.sbd.net.StreamBaseHTTPServer:    sbd at mysystem:7419; pid=7948; version=11.0.1; Listening
Programmatically examining the engine log is unreliable because the log may roll which changes the filename and there may be multiple starts and stops in the log.

Similarly, querying the node to get the state of the engine reports "Started" too soon during startup, so cannot be used to know for certain the application is ready. This command is therefore unreliable for this purpose:
  epadmin servicename=A.X display engine
or the same using the Admin REST API:
  curl -X 'POST' \
    'http://hostname:8008/admin/v1/targets/engine?servicename=A.X&command=display' \
    -H 'accept: application/json' \
    -H 'Content-Type: multipart/form-data' \
    -F 'parameters={}' \
    -F 'files='


The delay from startup to application readiness may take an arbitrarily long time based on the complexity of the application and the connections to external systems it must complete.

The Reliable Method

It is best to have the EventFlow application report that it is running by itself. One way is by writing to a known file which is easy to detect whether it exists or not with shell script commands. To do that, create a module (.sbapp) which contains an Input Stream connected to a File Writer output adapter. Configured as follows:
SB Properties for File Writer to automatically create a file at startup
Settings:
  • This Default File Name sets the path to not write the file into the default node and engine data area, but instead into the folder containing the node.
  • The InputStream stream has an empty schema with no fields. It simply completes the EventFlow syntax. It is otherwise unused.
Include this module in the main application for the StreamBase container you wish to confirm has started.

The file is created only once the StreamBase container in which the File Writer is located has been allowed to start, which is after the engine has initialized all adapters. Once the file is created by the File Writer adapter, then all public Streams are also available for connections and ports opened by server adapters will be live.

To wait until a file exists on Windows, use batch script commands:
 @echo off :waitforfile IF EXIST "ReportStart.txt" GOTO waitend timeout /t 1 /nobreak >NUL goto waitforfile :waitend echo "File found"
To wait until a file exists on Linux, use shell script commands:
 until [ -f ./ReportStart.txt ] do   sleep 1 done echo "File found"

These script portions may be placed after the command ' epadmin start node' in order to pause the script in the loop until the application creates the file.

Further Controlling the Delay

In order to pause until after specific EventFlow actions have been completed after application startup, still use the File Writer but enable the Control Port option and do not set the Default File Name in the adapter StreamBase properties. In your EventFlow implementation after startup events are complete, send into the File Writer control-port a command tuple with fields:
  • Command: "open"
  • FileName: "../../../../../ReportStart.txt"
The named file will not be created until the command-tuple is sent into the File Writer, permitting an additional application-controlled delay before the script (using the above commands) continues.