Oracle connectors connect to wrong KafkaConnect

[vsfomin]

Even though the labels clearly indicate the name of the cluster the connector belongs to, in the logs of the neighboring Kafka Connect, I see connector names that shouldn’t be there.
In the logs of debezium-connect-cluster-wms-01-connect-0 (which I will call KafkaConnect01 below), I see connectors that actually belong to KafkaConnect02.
I have two KafkaConnect clusters running in the same Kubernetes namespace, each with about a dozen connectors.
In the logs of debezium-connect-cluster-wms-01-connect-0 (which I will call KafkaConnect01 below), I see connectors that actually belong to KafkaConnect02.

Additionally, I see this log message in KafkaConnect01: (Pay attention to the connection URL. debezium-connect-cluster-wms-02 should not appear on KafkaConnect debezium-connect-cluster-wms-01)

2025-10-14 19:23:57 INFO [DistributedHerder-nexus-debezium-1] DistributedHerder:2631 - [Worker clientId=nexus-debezium, groupId=nexus-debezium] Joined group at generation 11 with protocol version 2 and got assignment: Assignment{error=0, leader='nexus-debezium-9826221d-4265-43d5-95ab-ca75aaa59d16', leaderUrl='http://debezium-connect-cluster-wms-02-connect-0.debezium-connect-cluster-wms-02-connect.ldh-kafka-01.svc:8083/', offset=45, connectorIds=[], taskIds=[], revokedConnectorIds=[], revokedTaskIds=[], delay=287993} with rebalance delay: 287993

I also observe many error messages like this:

ERROR [qtp1999415998-124] RestClient:207 - IO error forwarding REST request to http://debezium-connect-cluster-wms-02-connect-0.debezium-connect-cluster-wms-02-connect.ldh-kafka-01.svc:8083/connectors/oracle-connector-wms-02-0472?forward=true :
java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: Connect Timeout
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]

What does this mean?

I need to run two separate Kafka Connect clusters in the same Kubernetes namespace. Pay atention: label strimzi.io/cluster:debezium-connect-cluster-wms-02 is present in connector config and points to the correct cluster with the corresponding name. Why do I see this connection in the logs on another KafkaConnect?

My Kubernetes manifests examples:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  annotations:
    meta.helm.sh/release-name: ldh-debezium
    meta.helm.sh/release-namespace: ldh-kafka-01
    strimzi.io/use-connector-resources: 'true'
  generation: 1
  labels:
    app.kubernetes.io/managed-by: Helm
  name: debezium-connect-cluster-wms-01

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  annotations:
    meta.helm.sh/release-name: ldh-debezium
    meta.helm.sh/release-namespace: ldh-kafka-01
  generation: 1
  labels:
    app.kubernetes.io/managed-by: Helm
    strimzi.io/cluster: debezium-connect-cluster-wms-02
  name: oracle-connector-wms-02-0149
  namespace: ldh-kafka-01

[scholzj]

Did you follow the docs - https://strimzi.io/docs/operators/latest/full/deploying.html#con-config-kafka-connect-multiple-instances-str - to make sure the Connect clusters are independent on each other? If not, the Connect clusters will form one big Connect cluster (that does not work progperly).

[vsfomin]

Yes, it was the root cause. Thanks!