We are now required to make connections to Kafka from TIBCO Streaming over SSL/TLS. Until now, we've only been making insecure tcp connections. How can we configure our Streaming application and Kafka server to connect with each other over a secure port?

Summarizes how to enable 1-way and 2-way SSL/TLS connections between TIBCO Streaming and Kafka.

A successful SSL/TLS connection requires the Kafka server to supply its public certificate to the client.  It also requires the client to trust the Kafka server's identity, which is accomplished by adding the Kafka server's issuer (CA) certificate to the client's trust store.  In this case, the client is your TIBCO Streaming application that uses the Kafka adapters. To ensure the Kafka server supplies its public certificate, set the following in your Kafka server configuration file:

listeners=PLAINTEXT://:9092,SSL://:9093

advertised.listeners=PLAINTEXT://kafka.server.host:9092,SSL://kafka.server.host:9093

zookeeper.connect=kafka.server.host:2181

# SSL config
ssl.keystore.location=/path/to/kafka.server.keystore.jks
ssl.keystore.password=serversecret
ssl.key.password=serversecret
ssl.truststore.location=/path/to/kafka.server.truststore.jks
ssl.truststore.password=serversecret

..where:

• kafka.server.host = the publicly-accessible hostname of the Kafka server
• kafka.server.keystore.jks = the keystore containing the Kafka server's private key
• kafka.server.truststore.jks = the truststore containing the Kafka server's issuer (CA) certificate

In your Kafka adapter Properties, set the following:

Brokers = kafka.server.host:9093

Under the 'Advanced Options' tab, scroll down to the 'Advanced Config' section, and add the following:

security.protocol = SSL
ssl.truststore.location = /path/to/truststore.jks (e.g. C:/trusted/kafka.trusted.jks)
ssl.truststore.password = <TrustStorePassword>

In the above example, the kafka.trusted.jks must contain the Issuer (CA) certificate of the Kafka server. You may obtain any necessary certificates from your Kafka server administrator.

You are now configured for 1-way SSL/TLS.

For 2-way SSL/TLS the TIBCO Streaming client must provide a verifiable identity to the Kafka server. This identity has a private key (which requires a password to access), and may be stored in a similar location as the client's trust store shown above (e.g. C:/trusted/kafka.client.identity.jks). This identity then must be added to your Kafka adapter's 'Advanced Config' options:

ssl.keystore.location = C:/trusted/kafka.client.identity.jks
ssl.keystore.password = <KeystorePassword>
ssl.key.password = <PrivateKeyPassword>

The Kafka server must now be configured to trust the TIBCO Streaming client's identity.  This means the Streaming client's issuer (CA) certificate needs to be added to the Kafka server's truststore. Following the above example, the Streaming client's issuer certificate would need to be added to /path/to/kafka.server.truststore.jks on the Kafka server (as set in the Kafka server configuration file).

Finally, configure the Kafka server to authenticate clients by setting the following in your Kafka server configuration file:

ssl.client.auth=required

You are now configured for 2-way SSL/TLS.