Why Rabbitmq is restarted in one time on after 24h hour (Bitnami Rabbitmq 3.13.3) in kubernetes.

[bhagyesh1]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I run RabbitMQ 4.x, the only series currently covered by community support
• I promise to provide all relevant information (versions, logs from all nodes, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

other (please specify)

Erlang version used

27.3.x

Operating system (distribution) used

RHEL9

How is RabbitMQ deployed?

Bitnami Helm chart

rabbitmq-diagnostics status output

See https://www.rabbitmq.com/docs/cli to learn how to use rabbitmq-diagnostics

# PASTE OUTPUT HERE, BETWEEN BACKTICKS

Logs from node 1 (with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 2 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 3 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

rabbitmq.conf

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find rabbitmq.conf file location

# PASTE rabbitmq.conf HERE, BETWEEN BACKTICKS

Steps to deploy RabbitMQ cluster

Run the argocd file on kubernetes-cluster

Steps to reproduce the behavior in question

Run the argocd file on kubernetes-cluster

advanced.config

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find advanced.config file location

# PASTE advanced.config HERE, BETWEEN BACKTICKS

Application code

# PASTE CODE HERE, BETWEEN BACKTICKS

Kubernetes deployment file

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: xxxxxxxxx-xxxxxxxx
namespace: argocd
spec:
project: xxxxx-xxxxxxxxxxxxx
source:
repoURL: 'https://charts.bitnami.com/bitnami'
targetRevision: 14.4.5
chart: rabbitmq
helm:
values: |
persistence:
enabled: true
size: xxxxxxx
storageClass: xxxxxx
service:
type: NodePort
nodePorts:
amqp: xxxxxxx
manager: xxxxxxx
metrics:
enabled: true
serviceMonitor:
enabled: true
resources:
requests:
memory: xxxx
cpu: xxxxxxxxxx
limits:
memory: xxxxxxx
cpu: xxxxxxx
livenessProbe:
enabled: true
initialDelaySeconds: 180
timeoutSeconds: 10
periodSeconds: 30
failureThreshold: 6
successThreshold: 1
readinessProbe:
enabled: true
initialDelaySeconds: 60
timeoutSeconds: 5
periodSeconds: 30
failureThreshold: 5
successThreshold: 2
auth:
username: xxxxxxxxsxxxx
existingPasswordSecret: xxxxxxxxxxxxx
existingPasswordSecretKey: xxxxxxxxxxxx
extraPlugins: |
rabbitmq_delayed_message_exchange
communityPlugins: "https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/download/v3.13.0/rabbitmq_delayed_message_exchange-3.13.0.ez"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "kubernetes.io/hostname"
operator: "In"
values:
- "xxxxxxxxxxxxxx"
- "xxxxxxxxxxxxxx"
tolerations:
- key: "w2"
operator: "Equal"
value: "app"
effect: "NoSchedule"
- key: "w3"
operator: "Equal"
value: "mongodb"
effect: "NoSchedule"
destination:
server: 'https://kubernetes.default.svc'
namespace: benchmark
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- ApplyOutOfSyncOnly=true
- Replace=true
- CreateNamespace=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m0s

What problem are you trying to solve?

Why Rabbitmq is restarted in one time on after 24h hour (Bitnami Rabbitmq 3.13.3) in kubernetes.

[michaelklishin]

We cannot know. RabbitMQ does not restart itself periodically, something in your environment does that.

It could also be an autoheal partition handling strategy in action but it's extremely unlikely that network partitions happen on a schedule in your cluster.

We won't troubleshoot your K8S for you. Our team does not maintain the Bitnami chart. Finally, 3.13.x is out of community support.

When in doubt, start with looking at the logs from all nodes. It's infinitely more efficient than guessing.