Create valkey cluster with TLS

[tomiszili]

Hello,

I want to create a valkey cluster with TLS enabled within the valkey docker image. What valkey-cli arguments needed and prerequisites i have to do to make it work?

[tomiszili]

I have start to experiment with the configuration, but with no luck so far. Here is the error i got:

valkey-cluster  | + /usr/local/bin/valkey-cli --cluster create valkey-cluster:6380 valkey-cluster:6381 valkey-cluster:6382 valkey-cluster:6383 valkey-cluster:6384 valkey-cluster:6385 --cluster-replicas 1
valkey-cluster  | [ERR] Node valkey-cluster:6380 is not configured as a cluster node.
valkey-cluster  | + tail -f /valkey.log
valkey-cluster  | 2726:C 07 Nov 2024 12:12:33.899 * oO0OoO0OoO0Oo Valkey is starting oO0OoO0OoO0Oo
valkey-cluster  | 2726:C 07 Nov 2024 12:12:33.899 * Valkey version=7.2.6, bits=64, commit=00000000, modified=0, pid=2726, just started
valkey-cluster  | 2726:C 07 Nov 2024 12:12:33.899 * Configuration loaded
valkey-cluster  | 2726:M 07 Nov 2024 12:12:33.899 * monotonic clock: POSIX clock_gettime
valkey-cluster  | 2726:M 07 Nov 2024 12:12:33.900 * Running mode=cluster, port=6385.
valkey-cluster  | 2726:M 07 Nov 2024 12:12:33.900 * No cluster configuration found, I'm 20605c962f943742b260b76ba6a207954374f55f
valkey-cluster  | 2726:M 07 Nov 2024 12:12:33.906 * Server initialized
valkey-cluster  | 2726:M 07 Nov 2024 12:12:33.906 * Ready to accept connections tls
valkey-cluster  | 2666:M 07 Nov 2024 12:12:34.913 - Accepted 172.23.0.2:60752
valkey-cluster  | 2666:M 07 Nov 2024 12:12:34.913 # Error accepting a client connection: error:0A00010B:SSL routines::wrong version number (addr=172.23.0.2:60752 laddr=172.23.0.2:6380)

Here is my cluster_create.sh script mounted to the container.

#! /bin/bash

apt-get update && apt-get install -y wget
wget https://raw.githubusercontent.com/valkey-io/valkey/refs/heads/7.2/utils/gen-test-certs.sh -O /home/gen-test-certs.sh
chmod +x /home/gen-test-certs.sh
/home/gen-test-certs.sh
ls -la /data/tests/tls/

mkdir -p /nodes
touch /nodes/nodemap
if [ -z ${START_PORT} ]; then
    START_PORT=16379
fi
if [ -z ${END_PORT} ]; then
    END_PORT=16384
fi
if [ ! -z "$3" ]; then
    START_PORT=$2
    START_PORT=$3
fi
echo "STARTING: ${START_PORT}"
echo "ENDING: ${END_PORT}"

for PORT in `seq ${START_PORT} ${END_PORT}`; do
  mkdir -p /nodes/$PORT
  if [[ -e /valkey.conf ]]; then
    cp /valkey.conf /nodes/$PORT/valkey.conf
  else
    touch /nodes/$PORT/valkey.conf
  fi
  cat << EOF >> /nodes/$PORT/valkey.conf
port 0
cluster-enabled yes
cluster-announce-hostname valkey-cluster
cluster-preferred-endpoint-type hostname
daemonize yes
logfile /valkey.log
dir /nodes/$PORT
loglevel verbose
enable-debug-command yes
protected-mode yes
cluster-announce-port 0
cluster-announce-tls-port $PORT
cluster-port 0
cluster-announce-bus-port 2$PORT
tls-auth-clients optional
# tls-ca-cert-file /data/tests/tls/ca.crt
tls-ca-cert-dir /data/tests/tls
tls-cert-file /data/tests/tls/server.crt
tls-key-file /data/tests/tls/server.key
tls-client-cert-file /data/tests/tls/client.crt
tls-client-key-file /data/tests/tls/client.key
tls-cluster yes
tls-port $PORT
tls-replication no
tls-prefer-server-ciphers yes
tls-protocols "TLSv1 TLSv1.1 TLSv1.2 TLSv1.3"
EOF
  cat /nodes/$PORT/valkey.conf

  set -x
  /usr/local/bin/valkey-server /nodes/$PORT/valkey.conf
  sleep 1
  if [ $? -ne 0 ]; then
    echo "Valkey failed to start, exiting."
    continue
  fi
  echo valkey-cluster:$PORT >> /nodes/nodemap
done
if [ -z "${VALKEY_PASSWORD}" ]; then
    echo yes | /usr/local/bin/valkey-cli --cluster create `seq -f valkey-cluster:%g ${START_PORT} ${END_PORT}` --cluster-replicas 1
else
    echo yes | /usr/local/bin/valkey-cli -a ${VALKEY_PASSWORD} --cluster create `seq -f valkey-cluster:%g ${START_PORT} ${END_PORT}` --cluster-replicas 1
fi
tail -f /valkey.log

And here is my docker compose snippet.

services:
  valkey-cluster:
      container_name: valkey-cluster
      image: valkey/valkey:7.2.6
      entrypoint: /bin/sh -c 'export START_PORT="6380"; export END_PORT="6385"; /create_cluster.sh'
      ports:
        - 6380:6380
        - 6381:6381
        - 6382:6382
        - 6383:6383
        - 6384:6384
        - 6385:6385
      volumes:
        - "./create_cluster.sh:/create_cluster.sh"
        - "./cluster.valkey.conf:/valkey.conf:ro"