How to connect to Kafka over TLS to a vanilla (Strimzi) Kafka cluster #1260
-
|
Hi I'm trying to set up Kafbat with TLS configuration. I run an internal Kafka cluster using Strimzi, and am trying to use the Helm Chart to set up kafbat. When running over a plain listener, everything works well, and I have managed to get scram-sha-512 working over a plain connection. However, when I try to connect over TLS, then this fails. My idea would be to run TLS connections with TLS authentication, but failing that TLS connection with scram-sha-512 over TLS would be fine also. The error I'm currently getting is Kafbat is given the following chart: On start-up, I get the following details output before the cert-path error, which seem to show that some of the configuration above is not being picked up? e.g. the truststore type for example? Any hints as to where I'm going wrong? I checked open_ssl s_client against my kafka, and it does seem like kafka-mycluster-kafka-bootstrap is listed, amongst others such as kafka-mycluster-kafka-bootstrap.kafka.svc, etc. For mutual TLS, i would also appreciate a hint also - is it just setting the kafka.clusters.properties.keystore values in the same way? Cheers |
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 2 replies
-
yamlApplicationConfig:
kafka:
clusters:
- name: mycluster
bootstrapServers: kafka-mycluster-kafka-bootstrap:9093
ssl:
truststoreLocation: "/etc/kafka/ca/ca.p12"
truststorePassword: "${SECRET_TRUSTSTORE_PASSWORD}"
verifySsl: false
properties:
security.protocol: SSL
ssl.keystore.location: "/etc/kafka/user/user.p12"
ssl.keystore.password: "${SECRET_TRUSTSTORE_PASSWORD}"also you need take another CA from another strimzi-secret |
Beta Was this translation helpful? Give feedback.
-
|
Hi Thanks for the prompt answer - i now still get the same issue, despite passing the user's p12 cert in the key store. The config i tried: The error i still get Here is the output from kafbat depth=1 O=io.strimzi, CN=cluster-ca v0 |
Beta Was this translation helpful? Give feedback.
-
|
not sure, but it looks like you're using the wrong CA. еhe error message indicates this - Java cannot build a trusted certificate chain. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Thanks for that - I actually thought of that too just as i posted and that worked - thanks for the hint. For posterity and for others, here is the final configuration I settled on: Thanks again for the help. |
Beta Was this translation helpful? Give feedback.
-
|
@fallen-up thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Hi, thanks for the good answer @landbaychrisburrell ; Just if someone wants to connect via SSL but authenticate via SCRAM here is the correct configuration: Strimzi kafka listener: spec:
kafka:
...
listeners:
- name: scramsha
port: 9093
type: internal
tls: true
authentication:
type: scram-sha-512
authorization:
type: simpleStrimzi kafka user: apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
name: kafbat-ui
labels:
strimzi.io/cluster: dev1-kafka
spec:
authentication:
type: scram-sha-512
password:
valueFrom:
secretKeyRef:
name: kafbat-ui-kafka-user-password
key: password
authorization:
type: simple
acls:
- resource:
type: topic
name: "*"
patternType: literal
operations:
- All
- resource:
type: group
name: "*"
patternType: literal
operations:
- AllKafbat-ui helm values: ...
yamlApplicationConfig:
kafka:
clusters:
- name: dev1-kafka
bootstrapServers: dev1-kafka-kafka-brokers:9093
ssl:
truststoreLocation: /etc/kafka/cluster-ca/ca.p12
truststorePassword: "${SECRET_TRUSTSTORE_PASSWORD}"
properties:
security.protocol: SASL_SSL
sasl.mechanism: SCRAM-SHA-512
sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="kafbat-ui" password="XXX";
volumes:
- name: dev1-kafka-cluster-ca-cert
secret:
secretName: dev1-kafka-cluster-ca-cert
volumeMounts:
- name: dev1-kafka-cluster-ca-cert
mountPath: /etc/kafka/cluster-ca
readOnly: true
env:
- name: SECRET_TRUSTSTORE_PASSWORD
valueFrom:
secretKeyRef:
name: dev1-kafka-cluster-ca-cert
key: ca.password
|
Beta Was this translation helpful? Give feedback.
Hi
Thanks for that - I actually thought of that too just as i posted and that worked - thanks for the hint. For posterity and for others, here is the final configuration I settled on: