There may be a need to take a thread dump of the BPM process. See the resolution section for the steps to take a thread dump of the BPM process running in a Kubernetes node.

This article explains how to take Java thread dumps in BPM Enterprise 5.x

All supported Kubernetes environments

1. Logon to the control plane node machine.

2. Get the BPM pod name:

>kubectl get pods -n=<BPM namespace>

Example:

NAME                              READY   STATUS    RESTARTS        AGE
bpm-deployment-7f4b9c8d95-7k76z   1/1     Running   3 (4d16h ago)   15d

3. Get the shell prompt using the command:

>kubectl exec --stdin --tty <BPM pod name> -- /bin/bash

Example:

bpmuser@bpm-deployment-7f4b9c8d95-7k76z:/$

4. From the shell prompt of the pod, list the processes. The BPM container doesn't have the 'ps' so use /proc to get the list of processes. The following command can be used to get the list of processes:

for prc in /proc/*/cmdline; { (printf "$prc "; cat -A "$prc") | sed 's/\^@/ /g;s|/proc/||;s|/cmdline||'; echo; }

Example:

bpmuser@bpm-deployment-7f4b9c8d95-7k76z:/$ for prc in /proc/*/cmdline; { (printf "$prc "; cat -A "$prc") | sed 's/\^@/ /g;s|/proc/||;s|/cmdline||'; echo; }

1 /bin/bash /bin/bpmms
116 /opt/java/jdk-minimal/bin/java -Xmx4G -server -XX:+UseG1GC -XX:SurvivorRatio=128 -XX:MaxTenuringThreshold=0 -XX:+UseTLAB -XX:+CMSClassUnloadingEnabled -Djava.security.egd=file:///dev/urandom -Dspring.jdbc.getParameterType.ignore=true -Dcom.sun.jndi.ldap.connect.pool.maxsize=50 -Dcom.sun.jndi.ldap.connect.pool.prefsize=25 -Dcom.sun.jndi.ldap.connect.pool.initsize=10 --add-reads=java.xml=java.logging --add-exports=java.base/org.apache.karaf.specs.locator=java.xml,ALL-UNNAMED --patch-module java.base=lib/endorsed/org.apache.karaf.specs.locator-4.2.11.jar --patch-module java.xml=lib/endorsed/org.apache.karaf.specs.java.xml-4.2.11.jar --add-opens java.base/java.security=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.naming/javax.naming.spi=ALL-UNNAMED --add-opens java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.file=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.ftp=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.https=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.jar=ALL-UNNAMED --add-exports=java.base/sun.net.www.content.text=ALL-UNNAMED --add-exports=jdk.xml.dom/org.w3c.dom.html=ALL-UNNAMED --add-exports=jdk.naming.rmi/com.sun.jndi.url.rmi=ALL-UNNAMED --add-exports java.security.sasl/com.sun.security.sasl=ALL-UNNAMED -Dkaraf.instances=/opt/tibco/tibco-karaf-1.0.0/instances -Dkaraf.home=/opt/tibco/tibco-karaf-1.0.0 -Dkaraf.base=/opt/tibco/tibco-karaf-1.0.0 -Dkaraf.data=/opt/tibco/tibco-karaf-1.0.0/data -Dkaraf.etc=/opt/tibco/tibco-karaf-1.0.0/etc -Dkaraf.log=/opt/tibco/tibco-karaf-1.0.0/data/log -Dkaraf.restart.jvm.supported=true -Djava.io.tmpdir=/opt/tibco/tibco-karaf-1.0.0/data/tmp -Djava.util.logging.config.file=/opt/tibco/tibco-karaf-1.0.0/etc/java.util.logging.properties -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true -Dkaraf.log.console=ALL -classpath /opt/tibco/tibco-karaf-1.0.0/lib/boot/opendmk_jmxremote_optional_jar-1.0-b01-ea.jar:/opt/tibco/tibco-karaf-1.0.0/lib/boot/org.apache.karaf.diagnostic.boot-4.2.11.jar:/opt/tibco/tibco-karaf-1.0.0/lib/boot/org.apache.karaf.jaas.boot-4.2.11.jar:/opt/tibco/tibco-karaf-1.0.0/lib/boot/org.apache.karaf.main-4.2.11.jar:/opt/tibco/tibco-karaf-1.0.0/lib/boot/org.apache.karaf.specs.activator-4.2.11.jar:/opt/tibco/tibco-karaf-1.0.0/lib/boot/osgi.core-6.0.0.jar:/opt/tibco/tibco-karaf-1.0.0/lib/jdk9plus/istack-commons-runtime-3.0.10.jar:/opt/tibco/tibco-karaf-1.0.0/lib/jdk9plus/jakarta.xml.bind-api-2.3.3.jar:/opt/tibco/tibco-karaf-1.0.0/lib/jdk9plus/javax.activation-1.2.0.jar:/opt/tibco/tibco-karaf-1.0.0/lib/jdk9plus/javax.annotation-api-1.3.1.jar:/opt/tibco/tibco-karaf-1.0.0/lib/jdk9plus/jaxb-runtime-2.3.3.jar:/opt/tibco/tibco-karaf-1.0.0/lib/jdk9plus/txw2-2.3.3.jar org.apache.karaf.main.Main
71 /bin/sh /opt/tibco/tibco-karaf-1.0.0/bin/karaf clean run
9498 /bin/bash
self cat -A /proc/self/cmdline

5. From the list of running processes, identify the process id for BPM. In the above example, it is '116'.

6. Take the thread dump using kill -3. The thread dump will be logged into the pod log. 

Example: 

bpmuser@bpm-deployment-7f4b9c8d95-7k76z:/$ kill -3 116

To take more thread dumps repeat the above command. 

7. Exit from pod shell:

Example:

bpmuser@bpm-deployment-7f4b9c8d95-7k76z:/$ exit

8. Download the pod log using a command like: 

>kubectl logs <BPM pod name> > <file path/name of your choice>

Example:

>kubectl logs bpm-deployment-7f4b9c8d95-7k76z > /tmp/thread-dump.log

9. The pod log will have the thread dump starting from:

Full thread dump OpenJDK 64-Bit Server VM (11.0.16+8 mixed mode):

https://kubernetes.io/docs/tasks/debug/debug-application/get-shell-running-container/