If authentication for JMX is a requirement then the JMX documentation needs to be referred to for the comprehensive details on this.
Here are the 2 key requirements.

How to enable JMX Authentication for BW 5.x

All

1. The properties below are needed in the respective .tra file are 
Jmx.Enabled=true
java.property.com.sun.management.jmxremote=true
java.property.com.sun.management.jmxremote.port=11903   or any other available port 
java.property.com.sun.management.jmxremote.authenticate=true
java.property.com.sun.management.jmxremote.password.file=<tibco_home>/tibcojre64/1.8.0/lib/management/jmxremote.password
java.property.com.sun.management.jmxremote.access.file=<tibco_home>/tibcojre64/1.8.0/lib/management/jmxremote.access
java.property.com.sun.management.jmxremote.ssl=false

2. The JMX documentation states the following for the jmxremote.password file:
"You must ensure that only the owner has read and write permissions on this file, since it contains the passwords in clear text. For security reasons, the system checks that the file is only readable by the owner and exits with an error if it is not. Thus in a multiple-user environment, you should store the password file in private location such as your home directory."
This implies that the owner of the jmxremote.password file has to be the same user who starts the BW application in command line. 
If the BW application is being started in Administrator, the user starting Administrator has to be the same user who is the owner of the jmxremote.password file. 
The permission requirements may need to be considered in more detail depending on different scenarios.

For comprehensive details to configure JMX please refer to https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html