When TEA server is configured with one-way SSL using below properties, user cannot register bwagent with TEA server from bwadmin. registerteaagent command. 
tea.http.keystore
tea.http.truststore
tea.http.keystore-password
tea.http.truststore-password
tea.http.cert-alias

The error is displayed in bwagent.log.
javax.net.ssl.SSLException: Received fatal alert: unexpected_message
...
com.tibco.tea.agent - Auto registration failed with error: STATUS_EXCEPTED. There may be a problem with SSL configuration. For example, if HTTP Server in TEA Server was configured for mutual authentication; then the HTTP Client on TEA Agent side must be configured with a proper keystore/truststore; with correct client certificate file imported. For more details, please refer the user guide.

This is a known issue in BW 6.4.2 and TEA 2.3.0. The solution is to install TEA 2.3.0 HF-02. and add java.property.jdk.tls.client.protocols=SSLv2Hello,TLSv1,TLSv1.1,TLSv1.2 in TEA.tra.
 
Please refer to the defect TEA-3164 in the hotfix.

 

Cannot register bwagent with TEA from bwadmin command. Get error: Auto registration failed with error: STATUS_EXCEPTED

BW 6.4.2, TEA 2.3.0

install TEA 2.3.0 HF-02. As a workaround, user can register bwagent in TEA GUI