The reason for building entirely using command-line commands is to automate the build process as part of a Continuous Integration (CI) and Continuous Delivery (CD) environment.

By default, StreamBase Studio requires Docker CE installed on Microsoft Windows in order to build a Docker image (an Open Container Initiative (OCI, https://opencontainers.org/) compatible image suitable for running in any container manager (Docker, Kubernetes, OpenShift, etc.).

What follows are the steps to adapt your Streaming project to build a Docker image entirely from a Linux command-line.

In order to accomplish this, the fragment projects do not need to change, but the application project needs additions to the pom.xml and the start-node script and Dockerfile need to be provided. The application project specifically DOES NOT include Docker support (the "Enable Docker support" option was NOT selected when the application project was created). Obtain the 'start-node' script and Dockerfile as described below.

Note 1: The following steps also work on Linux systems with podman instead of docker when package podman-docker is installed.

Note 2: In the following examples my demo application is called "chirp" and consists of a Metronome and Log adapter. That's why you see "chirp" in the following examples.
 

One-Time Steps:

These steps prepare your build environment for building an image and running your application in a container. They only need to be performed once in each separate build environment.

A. Use Linux

Since the target container OS is Linux, the build process can only be executed in a Linux OS environment due to a current limitation in the Maven Assembly Plugin (MASSEMBLY-862 Assembly creation fails on Windows when extracting a tar.gz containing a symbolic link -- https://issues.apache.org/jira/browse/MASSEMBLY-862).

B. Install TIBCO Streaming for Linux

1. Install TIBCO Streaming according to documentation page:

TIBCO Streaming > Installation Guide > Installing on Linux

2. Configure your shell environment according to documentation page:

TIBCO Streaming > Installation Guide > StreamBase-Configured Shells

C. Prepare the local Maven repository

Run command:

   epdev install maven

This will take a little time to complete. It is copying the runtime artifacts which will be used to build the image to make them available for the Maven-build step.

D. Obtain the default ‘start-node’ script:

Create and unrelated StreamBase Application project with the setting "Enable Docker support" enabled. Obtain the 'start-node' script from this project from location:

    application-project/src/main/docker/base/start-node

and set this 'start-node' script aside for later use.
 

Repeated Steps:

These are the steps you will need to perform for any new project to prepare it for a command-line build process.

A. COPY PROJECTS FROM DEVELOPMENT ENV TO BUILD ENV

Typically development is done on a supported OS, such as Microsoft Windows or Apple MacOS. The build and runtime environments are typically a supported Linux version. These instructions cause all build activity that produces a runtime artifact to happen on Linux.

Make a faithful copy of your Streaming fragment and application projects into a blank working directory that will be a mirror of your StreamBase Studio workspace. This will not be used as a Studio workspace, but maintains the project file structure which is important.

(pic1: SB Studio view of project)

(pic2: linux directory of copied project)

Note: If you have already made a local build and the project target/ directory is populated, then before copying the projects run a "Maven clean" (mvn clean) in order to remove these old build artifacts. They tend to be large and they do not need to be copied into your Linux build environment.

B. PREPARE AND BUILD THE STREAMING APPLICATION PROJECT FROM THE COMMAND-LINE

The 'mvn install' and 'mvn package' commands are equivalent to performing a build from SB Studio. These are the alternative steps for preparing the project for a docker command-line build and also performing all steps from the command-line and not using SB Studio for any build step.

1. Build each EventFlow fragment project used by your application:

    cd {project folder where pom.xml is found}    mvn clean install -DskipTests=true

If this is the first time 'mvn' has been run on this system, then the Maven executables will be downloaded from "https://repo.maven.apache.org". Make sure you have internet access to this site. Use the same command for EventFlow, LiveView, and Java fragment projects that may be used by your application.

2. Update the Application project's pom.xml to include the additional packaging directive for "platform_linuxx86_64":

pom.xml (example):

 <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">     <!-- vim: set tabstop=4 softtabstop=0 expandtab shiftwidth=4 smarttab : -->     <modelVersion>4.0.0</modelVersion>     <groupId>com.example</groupId>     <artifactId>chirp_app</artifactId>     <packaging>ep-application</packaging>     <version>0.0.1-SNAPSHOT</version>     <name>chirp_app</name>     <description>My Application</description>     <!-- common definitions for this version of Streaming -->     <parent>         <groupId>com.tibco.ep.sb.parent</groupId>         <artifactId>ep-application</artifactId>         <version>10.6.1</version>     </parent>     <build>         <plugins>             <plugin>                 <groupId>com.tibco.ep</groupId>                 <artifactId>ep-maven-plugin</artifactId>                 <extensions>true</extensions>                 <configuration>                     <skipTests>${skipLocalTests}</skipTests>                     <nodes>                         <node>A</node>                     </nodes>                 </configuration>             </plugin>             <!-- add additional plugin -->             <plugin>                 <artifactId>maven-dependency-plugin</artifactId>                 <executions>                     <!-- copy dependencies to container build -->                     <execution>                         <id>install-dependencies</id>                         <phase>package</phase>                         <goals>                             <goal>unpack-dependencies</goal>                         </goals>                         <configuration>                             <outputDirectory>${project.build.directory}/container-source/maven</outputDirectory>                             <overWriteReleases>false</overWriteReleases>                             <overWriteSnapshots>false</overWriteSnapshots>                             <overWriteIfNewer>true</overWriteIfNewer>                             <includeScope>compile</includeScope>                             <silent>true</silent>                         </configuration>                     </execution>                 </executions>             </plugin>                           </plugins>     </build>     <dependencies>         <dependency>             <groupId>com.example</groupId>             <artifactId>chirp</artifactId>             <version>0.0.1-SNAPSHOT</version>             <type>ep-eventflow-fragment</type>         </dependency>         <!-- add dependency -->         <dependency>             <groupId>com.tibco.ep.sb.rt</groupId>             <artifactId>platform_linuxx86_64</artifactId>             <version>${sbrt.version}</version>             <type>zip</type>         </dependency>            </dependencies> </project>

3. Build application. In the application project folder run:

    mvn clean package -DskipTests=true

The 'mvn package' command using the above pom.xml produces files (example):

• target/container-source/maven -- full platform_linuxx86_64

(for "COPY maven ${STREAMING_PRODUCT_HOME}" command in Dockerfile)

• target/chirp_app-0.0.1-SNAPSHOT-ep-application.zip

This maven command would have attempted to build using the Docker CE environment if the application project was created with Docker support, which is what we do not want.

C. BUILD DOCKER IMAGE FROM THE COMMAND-LINE

4. Copy the application to the deployment directory:

    cd {app-project}/target     cp *.zip container-source/

Repeat this step after every repeated use of the 'mvn package' command to copy the most recent application to the deployment directory, overwriting older builds.

5. Copy the 'start-node' script from:

    {application-project}/src/main/docker/base/start-node

into:

    {application-project}/target/container-source/

Note: Be careful to change DOS line endings (^M or \r characters) to Linux line endings (\n only). You can see startup errors that cause the container to fail to start by attaching to the shell at startup, for example:

 $ docker start -i 7c427cdb4912 /bin/sh: /opt/tibco/streambase/start-node: /bin/sh^M: bad interpreter: No such file or directory

In this case the start-node script had DOS endings and could not start the 'sh' command. Correct this in the "target/container-source/" folder using a command like 'dos2unix filename'.

6. Create the following Dockerfile in {app-project}/target/container-source

 FROM almalinux:8.5 LABEL description="TIBCO Streaming application" LABEL PRODUCT_VERSION=${PROJECT_VERSION} # # Set environment # ENV STREAMING_PRODUCT_HOME       /opt/tibco/streambase ENV STREAMING_RUNTIME_HOME       /var/opt/tibco/streambase ENV JAVA_HOME            /etc/alternatives/jre ENV PATH                   /bin:/usr/sbin:${STREAMING_PRODUCT_HOME}/distrib/tibco/bin ENV USER_NAME            tibco # # Add required additional packages # RUN yum --assumeyes --exclude=gcc --exclude=binutils install \       sysstat \       gdb \       zip \       unzip \       libnsl \       less \       && yum --assumeyes install java-11-openjdk-headless \       && yum clean all # # Create a user to run nodes # RUN /usr/sbin/useradd \       --comment "TIBCO Streaming User" \       --create-home \       --user-group \       --password ${USER_NAME} \       --shell /bin/bash \       ${USER_NAME} # # Install runtime # RUN mkdir -p ${STREAMING_PRODUCT_HOME} COPY maven ${STREAMING_PRODUCT_HOME} COPY start-node ${STREAMING_PRODUCT_HOME}/start-node RUN chmod a+x ${STREAMING_PRODUCT_HOME}/start-node # # Allow user access to nodes and applications # RUN mkdir -p ${STREAMING_RUNTIME_HOME} RUN chown ${USER_NAME}:${USER_NAME} ${STREAMING_RUNTIME_HOME} # # Install application # RUN mkdir -p ${STREAMING_RUNTIME_HOME}/application/ COPY *.zip ${STREAMING_RUNTIME_HOME}/application/ # # Change to guest user # USER ${USER_NAME} # # entry point to start node # ENTRYPOINT ${STREAMING_PRODUCT_HOME}/start-node

7. Run in '{app-project}/target/container-source':

    docker build . --build-arg PROJECT_VERSION=0.0.1 --tag myapp:0.0.1

You can see the new image by running:

    docker images

Example:

 user@system:~/build_chirp/chirp_app/target/container-source$ docker images REPOSITORY	TAG	IMAGE ID	CREATED	SIZE myapp	0.0.1	41e3914cfc25	39 seconds ago	1.08GB hello-world	latest	feb5d9fea6a5	6 weeks ago	13.3kB docker.io/library/almalinux  8.5         4ca63ce1d8a9  3 months ago    205 MB 

8. Execute as:

    docker run -d --name myapp -e STREAMING_NODENAME=A.app myapp:0.0.1

9. Verify application is running (example):

    docker exec -it myapp /bin/bash 

Example:

 $ ps -ef UID	PID	PPID	C	STIME	TTY	TIME	CMD tibco	1	0	0	15:11	?	00:00:00	/bin/sh /opt/tibco/streambase/start-node tibco 	22	1	1	15:11	?	00:00:00	epadmin --username=tibco --password=tibco install node --nodename=A.app tibco	43	22	11	15:11	?	00:00:01	///opt/tibco/streambase/distrib/kabira/bin/swcoord -n A.app tibco	44	43	14	15:11	?	00:00:01	System::swcoordadmin -p /var/opt/tibco/streambase/node/A.app -n A.app -e tibco	71	43	99	15:11	?	00:00:27	System::administration -p /var/opt/tibco/streambase/node/A.app -n A.app tibco	169	0	2	15:11	pts/0	00:00:00	/bin/sh tibco	205	71	0	15:11	?	00:00:00	/opt/tibco/streambase/distrib/tibco/bin/epadmin schema=true delimiter=&& tibco	223	169	0	15:11	pts/0	00:00:00	ps -ef $ epadmin display services Service Name = A.app Service Type = node Network Address = dtm-adm://da9256a5f447:45563 ... $ epadmin servicename=A.app tail logging  ...

You can also execute these commands externally as (example):

    docker exec myapp epadmin display services

    docker exec myapp epadmin servicename=A.app tail logging

10. Stop the application as:

    docker stop --name myapp

Optionally remove the container using commands:

    docker container ls -a

    docker container rm myapp

Note that when using the default ‘start-node’ script stopping the container kills the node processes and does not allow clean shutdown and re-use of the contatiner at startup. This is why the ‘start-node’ script always installs a new node when the container starts. For smaller applications this may be fine, but or larger applications that are intended to preserve node-state between restarts, the stop process and ‘start-node’ script will need customization.