This article outlines the steps and components needed to enable Kerberos/SASL_SSL authentication for the TIBCO Streaming Kafka adapters.

Outlines the steps and components needed to enable Kerberos/SASL_SSL authentication for the TIBCO Streaming Kafka adapters.

For secure Kerberos/SASL_SSL connections, you need to configure separate listeners for each on the Kafka server. Typically, the SSL listener is on port 9093, and the SASL_SSL listener is on port 9094. For example, in the Kafka server.properties:
 

 listeners=PLAINTEXT://:9092,SSL://:9093,SASL_SSL://:9094 advertised.listeners=PLAINTEXT://ec2-174-129-102-157.compute-1.amazonaws.com:9092,SSL://ec2-174-129-102-157.compute-1.amazonaws.com:9093,SASL_SSL://ec2-174-129-102-157.compute-1.amazonaws.com:9094

The SSL configuration on the Kafka server needs to specify the Kafka server's keystore, and a trust store to hold trusted client issuer (CA) certificates. In the Kafka server.properties, set:
 

 # SSL config ssl.keystore.location=/home/ec2-user/certs/kafka.server.keystore.jks ssl.keystore.password=password ssl.key.password=password ssl.truststore.location=/home/ec2-user/certs/kafka.truststore.jks ssl.truststore.password=password ssl.client.auth=required

For more detailed instructions on configuring SSL connections for Kafka, refer to the Knowledge article entitled "How to enable SSL/TLS connections to Kafka from TIBCO Streaming".

In addition, the SASL configuration on the Kafka server needs to specify the sasl mechanism and the Kerberos service name for Kafka. In the Kafka server.properties, set:
 

 # SASL config sasl.enabled.mechanisms=GSSAPI sasl.kerberos.service.name=kafka/ec2-174-129-102-157.compute-1.amazonaws.com@KAFKA.SECURE

..where ' KAFKA.SECURE' is the Kerberos realm name, ' kafka' is the service name, and the Kafka hostname is ' ec2-174-129-102-157.compute-1.amazonaws.com'. This comprises the full Kerberos service principal name for the Kafka service.

To connect to the Kafka service as a client, the client has to know about the Kerberos KDC server to which it will connect. This is done in a Kerberos krb5.conf configuration file located on the client machine. The krb5.conf should have details about the KAFKA.SECURE realm that is defined on the Kerberos KDC server. Following the above example, you should add the following to your client machine's krb5.conf:
 

 [realms] KAFKA.SECURE = {                 kdc = ec2-174-129-102-157.compute-1.amazonaws.com                 admin_server = ec2-174-129-102-157.compute-1.amazonaws.com  } [domain_realm]      .amazonaws.com = KAFKA.SECURE   amazonaws.com = KAFKA.SECURE

In the above example, we can see that the Kerberos KDC server runs on the same machine as the Kafka server (but in many cases, these are different machines).

In addition, the client also has to specify a JAAS configuration. This is typically done using another configuration file on the client machine. Here is an example KafkaClient JAAS configuration:
 

 KafkaClient {   com.sun.security.auth.module.Krb5LoginModule required   serviceName="kafka"   useKeyTab=true   debug=true   keyTab="C:/keytabs/user1.user.keytab"   principal="user1/@KAFKA.SECURE"; };

In this case, authentication will be accomplished using a keytab file. This means the client machine needs to have a local copy of that keytab file, and point to it using the ' keyTab' property as shown above.

Now, the TIBCO Streaming application must be configured so that it finds the krb5 and jaas configuration files on the client machine. This may be done in a StreamBase Engine configuration under your project's src/main/resources/ directory:
 

 StreamBaseEngine = {         jvmArgs = [             "-Djava.security.auth.login.config=C:/configs/kafka_client_jaas.conf"             "-Djava.security.krb5.conf=C:/configs/krb5.conf"         ]     }

Finally, in the Kafka adapter Properties view in Studio, set ' Brokers' to your Kafka server (e.g. ec2-107-23-170-187.compute-1.amazonaws.com:9094). Also under Advanced Options > Advanced Config, set the following:
 

 security.protocol = SASL_SSL sasl.mechanism = GSSAPI sasl.kerberos.service.name = kafka ssl.keystore.location = C:/stores/kafka.client.keystore.jks ssl.keystore.password = <YourKeystorePassword> ssl.key.password = <YourPrivateKeyPassword> ssl.truststore.location = C:/trusted/kafka.trusted.jks ssl.truststore.password = <YourTrustStorePassword>

To assist in troubleshooting, set the sun.security.krb5.debug system property to 'true'. Again, you may do this in your SB Engine configuration:
 

 StreamBaseEngine = {         jvmArgs = [             "-Djava.security.auth.login.config=C:/configs/kafka_client_jaas.conf"             "-Djava.security.krb5.conf=C:/configs/krb5.conf"             "-Dsun.security.krb5.debug=true"         ]     }

This will generate verbose messages relating to Kerberos auth in your Streaming application log.