Skip to content

Conversation

@trevorwhitney
Copy link
Collaborator

What this PR does / why we need it:

Bumps a few helm dependencies to the latest versions for security fixes. Also published loki-helm-test image with release. We're not currently publishing this image and it's about ~1yr behind.

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@trevorwhitney trevorwhitney requested a review from a team as a code owner November 20, 2025 19:28
@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 20, 2025

💻 Deploy preview available (fix: bump helm deps, publish loki-helm-test w/ release):

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

Helm Diff Output - Summary

Default Values Scenario-diff-output
default, loki-backend, StatefulSet (apps) has changed:
  # Source: loki/templates/backend/statefulset-backend.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-backend
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: backend
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    updateStrategy:
      rollingUpdate:
        partition: 0
    serviceName: loki-backend-headless
    revisionHistoryLimit: 10
    
    persistentVolumeClaimRetentionPolicy:
      whenDeleted: Delete
      whenScaled: Delete
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: backend
    template:
      metadata:
        annotations:
          checksum/config: 6457764bbaed49d2a88ffc0b26082e5b83bf59c884c9bd0b327e36a23e468f89
          kubectl.kubernetes.io/default-container: "loki"
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: backend
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        automountServiceAccountToken: true
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: loki
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=backend
              - -legacy-read-mode=false
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: tmp
                mountPath: /tmp
              - name: data
                mountPath: /var/loki
              - name: sc-rules-volume
                mountPath: "/rules"
            resources:
              {}
          - name: loki-sc-rules
-           image: docker.io/kiwigrid/k8s-sidecar:1.30.10
+           image: docker.io/kiwigrid/k8s-sidecar:1.30.11
            imagePullPolicy: IfNotPresent
            env:
              - name: METHOD
                value: WATCH
              - name: LABEL
                value: "loki_rule"
              - name: FOLDER
                value: "/rules"
              - name: RESOURCE
                value: "both"
              - name: WATCH_SERVER_TIMEOUT
                value: "60"
              - name: WATCH_CLIENT_TIMEOUT
                value: "60"
              - name: LOG_LEVEL
                value: "INFO"
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            volumeMounts:
              - name: sc-rules-volume
                mountPath: "/rules"
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: backend
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: tmp
            emptyDir: {}
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: sc-rules-volume
            emptyDir: {}
    volumeClaimTemplates:
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: "10Gi"
default, loki-canary, DaemonSet (apps) has changed:
  # Source: loki/templates/loki-canary/daemonset.yaml
  apiVersion: apps/v1
  kind: DaemonSet
  metadata:
    name: loki-canary
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: canary
  spec:
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: canary
    updateStrategy:
      rollingUpdate:
        maxUnavailable: 1
      type: RollingUpdate
    template:
      metadata:
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: canary
      spec:
        serviceAccountName: loki-canary
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        containers:
          - name: loki-canary
            image: docker.io/grafana/loki-canary:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -addr=loki-release-gateway.default.svc.cluster.local.:80
              - -labelname=pod
              - -labelvalue=$(POD_NAME)
              - -user=self-monitoring
              - -tenant-id=self-monitoring
              - -pass=
              - -push=true
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            volumeMounts:
            ports:
              - name: http-metrics
                containerPort: 3500
                protocol: TCP
            env:
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              
            readinessProbe:
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 15
              timeoutSeconds: 1
        volumes:
default, loki-read, Deployment (apps) has changed:
  # Source: loki/templates/read/deployment-read.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: loki-read
    namespace: default
    labels:
      app.kubernetes.io/part-of: memberlist
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: read
  spec:
    replicas: 1
    strategy:
      rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: read
    template:
      metadata:
        annotations:
          checksum/config: 6457764bbaed49d2a88ffc0b26082e5b83bf59c884c9bd0b327e36a23e468f89
        labels:
          app.kubernetes.io/part-of: memberlist
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: read
      spec:
        serviceAccountName: loki-release
        automountServiceAccountToken: true
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 30
        containers:
          - name: loki
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=read
              - -legacy-read-mode=false
              - -common.compactor-grpc-address=loki-backend.default.svc.cluster.local:9095
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: tmp
                mountPath: /tmp
              - name: data
                mountPath: /var/loki
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: read
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: tmp
            emptyDir: {}
          - name: data
            emptyDir: {}
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
default, loki-release-chunks-cache, StatefulSet (apps) has changed:
  # Source: loki/templates/chunks-cache/statefulset-chunks-cache.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-chunks-cache
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: "memcached-chunks-cache"
      name: "memcached-chunks-cache"
    annotations:
      {}
    namespace: "default"
  spec:
    podManagementPolicy: Parallel
    replicas: 1
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: "memcached-chunks-cache"
        name: "memcached-chunks-cache"
    updateStrategy:
      type: RollingUpdate
    serviceName: loki-release-chunks-cache
    template:
      metadata:
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: "memcached-chunks-cache"
          name: "memcached-chunks-cache"
        annotations:
      spec:
        serviceAccountName: loki-release
        securityContext:
          fsGroup: 11211
          runAsGroup: 11211
          runAsNonRoot: true
          runAsUser: 11211
        initContainers:
          []
        nodeSelector:
          {}
        affinity:
          {}
        topologySpreadConstraints:
          []
        tolerations:
          []
        terminationGracePeriodSeconds: 60
        containers:
          - name: memcached
            image: memcached:1.6.39-alpine
            imagePullPolicy: IfNotPresent
            resources:
              limits:
                memory: 9830Mi
              requests:
                cpu: 500m
                memory: 9830Mi
            ports:
              - containerPort: 11211
                name: client
            args:
              - -m 8192
              - --extended=modern,track_sizes
              - -I 5m
              - -c 16384
              - -v
              - -u 11211
            env:
            envFrom:
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 6
              initialDelaySeconds: 5
              periodSeconds: 5
              tcpSocket:
                port: client
              timeoutSeconds: 3
            livenessProbe:
              failureThreshold: 3
              initialDelaySeconds: 30
              periodSeconds: 10
              tcpSocket:
                port: client
              timeoutSeconds: 5
          - name: exporter
-           image: prom/memcached-exporter:v0.15.3
+           image: prom/memcached-exporter:v0.15.4
            imagePullPolicy: IfNotPresent
            ports:
              - containerPort: 9150
                name: http-metrics
            args:
              - "--memcached.address=localhost:11211"
              - "--web.listen-address=0.0.0.0:9150"
            resources:
              limits: {}
              requests: {}
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 5
              periodSeconds: 5
              timeoutSeconds: 3
            livenessProbe:
              failureThreshold: 3
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 30
              periodSeconds: 10
              timeoutSeconds: 5
default, loki-release-results-cache, StatefulSet (apps) has changed:
  # Source: loki/templates/results-cache/statefulset-results-cache.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-results-cache
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: "memcached-results-cache"
      name: "memcached-results-cache"
    annotations:
      {}
    namespace: "default"
  spec:
    podManagementPolicy: Parallel
    replicas: 1
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: "memcached-results-cache"
        name: "memcached-results-cache"
    updateStrategy:
      type: RollingUpdate
    serviceName: loki-release-results-cache
    template:
      metadata:
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: "memcached-results-cache"
          name: "memcached-results-cache"
        annotations:
      spec:
        serviceAccountName: loki-release
        securityContext:
          fsGroup: 11211
          runAsGroup: 11211
          runAsNonRoot: true
          runAsUser: 11211
        initContainers:
          []
        nodeSelector:
          {}
        affinity:
          {}
        topologySpreadConstraints:
          []
        tolerations:
          []
        terminationGracePeriodSeconds: 60
        containers:
          - name: memcached
            image: memcached:1.6.39-alpine
            imagePullPolicy: IfNotPresent
            resources:
              limits:
                memory: 1229Mi
              requests:
                cpu: 500m
                memory: 1229Mi
            ports:
              - containerPort: 11211
                name: client
            args:
              - -m 1024
              - --extended=modern,track_sizes
              - -I 5m
              - -c 16384
              - -v
              - -u 11211
            env:
            envFrom:
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 6
              initialDelaySeconds: 5
              periodSeconds: 5
              tcpSocket:
                port: client
              timeoutSeconds: 3
            livenessProbe:
              failureThreshold: 3
              initialDelaySeconds: 30
              periodSeconds: 10
              tcpSocket:
                port: client
              timeoutSeconds: 5
          - name: exporter
-           image: prom/memcached-exporter:v0.15.3
+           image: prom/memcached-exporter:v0.15.4
            imagePullPolicy: IfNotPresent
            ports:
              - containerPort: 9150
                name: http-metrics
            args:
              - "--memcached.address=localhost:11211"
              - "--web.listen-address=0.0.0.0:9150"
            resources:
              limits: {}
              requests: {}
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 5
              periodSeconds: 5
              timeoutSeconds: 3
            livenessProbe:
              failureThreshold: 3
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 30
              periodSeconds: 10
              timeoutSeconds: 5
default, loki-write, StatefulSet (apps) has changed:
  # Source: loki/templates/write/statefulset-write.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-write
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: write
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    updateStrategy:
      rollingUpdate:
        partition: 0
    serviceName: loki-write-headless
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: write
    template:
      metadata:
        annotations:
          checksum/config: 6457764bbaed49d2a88ffc0b26082e5b83bf59c884c9bd0b327e36a23e468f89
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: write
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        automountServiceAccountToken: true
        enableServiceLinks: true
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: loki
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=write
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: write
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
    volumeClaimTemplates:
      - apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
          name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: "10Gi"
Distributed Scenario-diff-output
default, loki-canary, DaemonSet (apps) has changed:
  # Source: loki/templates/loki-canary/daemonset.yaml
  apiVersion: apps/v1
  kind: DaemonSet
  metadata:
    name: loki-canary
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: canary
  spec:
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: canary
    updateStrategy:
      rollingUpdate:
        maxUnavailable: 1
      type: RollingUpdate
    template:
      metadata:
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: canary
      spec:
        serviceAccountName: loki-canary
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        containers:
          - name: loki-canary
            image: docker.io/grafana/loki-canary:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -addr=loki-release-gateway.default.svc.cluster.local.:80
              - -labelname=pod
              - -labelvalue=$(POD_NAME)
              - -user=self-monitoring
              - -tenant-id=self-monitoring
              - -pass=
              - -push=true
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            volumeMounts:
            ports:
              - name: http-metrics
                containerPort: 3500
                protocol: TCP
            env:
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              
            readinessProbe:
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 15
              timeoutSeconds: 1
        volumes:
default, loki-release-chunks-cache, StatefulSet (apps) has changed:
  # Source: loki/templates/chunks-cache/statefulset-chunks-cache.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-chunks-cache
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: "memcached-chunks-cache"
      name: "memcached-chunks-cache"
    annotations:
      {}
    namespace: "default"
  spec:
    podManagementPolicy: Parallel
    replicas: 1
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: "memcached-chunks-cache"
        name: "memcached-chunks-cache"
    updateStrategy:
      type: RollingUpdate
    serviceName: loki-release-chunks-cache
    template:
      metadata:
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: "memcached-chunks-cache"
          name: "memcached-chunks-cache"
        annotations:
      spec:
        serviceAccountName: loki-release
        securityContext:
          fsGroup: 11211
          runAsGroup: 11211
          runAsNonRoot: true
          runAsUser: 11211
        initContainers:
          []
        nodeSelector:
          {}
        affinity:
          {}
        topologySpreadConstraints:
          []
        tolerations:
          []
        terminationGracePeriodSeconds: 60
        containers:
          - name: memcached
            image: memcached:1.6.39-alpine
            imagePullPolicy: IfNotPresent
            resources:
              limits:
                memory: 9830Mi
              requests:
                cpu: 500m
                memory: 9830Mi
            ports:
              - containerPort: 11211
                name: client
            args:
              - -m 8192
              - --extended=modern,track_sizes
              - -I 5m
              - -c 16384
              - -v
              - -u 11211
            env:
            envFrom:
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 6
              initialDelaySeconds: 5
              periodSeconds: 5
              tcpSocket:
                port: client
              timeoutSeconds: 3
            livenessProbe:
              failureThreshold: 3
              initialDelaySeconds: 30
              periodSeconds: 10
              tcpSocket:
                port: client
              timeoutSeconds: 5
          - name: exporter
-           image: prom/memcached-exporter:v0.15.3
+           image: prom/memcached-exporter:v0.15.4
            imagePullPolicy: IfNotPresent
            ports:
              - containerPort: 9150
                name: http-metrics
            args:
              - "--memcached.address=localhost:11211"
              - "--web.listen-address=0.0.0.0:9150"
            resources:
              limits: {}
              requests: {}
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 5
              periodSeconds: 5
              timeoutSeconds: 3
            livenessProbe:
              failureThreshold: 3
              httpGet:
                path: /metrics
                port: http-metrics
              initialDelaySeconds: 30
              periodSeconds: 10
              timeoutSeconds: 5
default, loki-release-compactor, StatefulSet (apps) has changed:
  # Source: loki/templates/compactor/statefulset-compactor.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-compactor
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: compactor
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    updateStrategy:
      rollingUpdate:
        partition: 0
    serviceName: loki-release-compactor-headless
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: compactor
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: compactor
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 30
        containers:
          - name: compactor
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=compactor
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            
            volumeMounts:
              - name: temp
                mountPath: /tmp
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: compactor
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: temp
            emptyDir: {}
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: data
            emptyDir: {}
default, loki-release-distributor, Deployment (apps) has changed:
  # Source: loki/templates/distributor/deployment-distributor.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: loki-release-distributor
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: distributor
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 3
    strategy:
      rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: distributor
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: distributor
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 30
        containers:
          - name: distributor
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=distributor
              - -distributor.zone-awareness-enabled=true
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: distributor
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
default, loki-release-index-gateway, StatefulSet (apps) has changed:
  # Source: loki/templates/index-gateway/statefulset-index-gateway.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-index-gateway
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: index-gateway
  spec:
    replicas: 2
    updateStrategy:
      type: RollingUpdate
    serviceName: loki-release-index-gateway-headless
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: index-gateway
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: index-gateway
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: index-gateway
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=index-gateway
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: index-gateway
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: data
            emptyDir: {}
default, loki-release-ingester-zone-a, StatefulSet (apps) has changed:
  # Source: loki/templates/ingester/statefulset-ingester-zone-a.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-ingester-zone-a
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: ingester
      app.kubernetes.io/part-of: memberlist
      name: ingester-zone-a
      rollout-group: ingester
    annotations:
      rollout-max-unavailable: "1"
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    serviceName: loki-release-ingester-zone-a-headless
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: ingester
        name: ingester-zone-a
        rollout-group: ingester
    updateStrategy:
      type: RollingUpdate
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: ingester
          app.kubernetes.io/part-of: memberlist
          name: ingester-zone-a
          rollout-group: ingester
      spec:
        topologySpreadConstraints:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/component: ingester
                app.kubernetes.io/instance: 'loki-release'
                app.kubernetes.io/name: 'loki'
            maxSkew: 1
            topologyKey: kubernetes.io/hostname
            whenUnsatisfiable: ScheduleAnyway
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: ingester
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -ingester.availability-zone=zone-a
              - -ingester.unregister-on-shutdown=false
              - -ingester.tokens-file-path=/var/loki/ring-tokens
              - -target=ingester
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                    - key: rollout-group
                      operator: In
                      values:
                        - ingester
                    - key: name
                      operator: NotIn
                      values:
                        - ingester-zone-a
                topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: data
            emptyDir: {}
default, loki-release-ingester-zone-b, StatefulSet (apps) has changed:
  # Source: loki/templates/ingester/statefulset-ingester-zone-b.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-ingester-zone-b
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: ingester
      app.kubernetes.io/part-of: memberlist
      name: ingester-zone-b
      rollout-group: ingester
    annotations:
      rollout-max-unavailable: "1"
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    serviceName: loki-release-ingester-zone-b-headless
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: ingester
        name: ingester-zone-b
        rollout-group: ingester
    updateStrategy:
      type: RollingUpdate
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: ingester
          app.kubernetes.io/part-of: memberlist
          name: ingester-zone-b
          rollout-group: ingester
      spec:
        topologySpreadConstraints:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/component: ingester
                app.kubernetes.io/instance: 'loki-release'
                app.kubernetes.io/name: 'loki'
            maxSkew: 1
            topologyKey: kubernetes.io/hostname
            whenUnsatisfiable: ScheduleAnyway
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: ingester
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -ingester.availability-zone=zone-b
              - -ingester.unregister-on-shutdown=false
              - -ingester.tokens-file-path=/var/loki/ring-tokens
              - -target=ingester
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                    - key: rollout-group
                      operator: In
                      values:
                        - ingester
                    - key: name
                      operator: NotIn
                      values:
                        - ingester-zone-b
                topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: data
            emptyDir: {}
default, loki-release-ingester-zone-c, StatefulSet (apps) has changed:
  # Source: loki/templates/ingester/statefulset-ingester-zone-c.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-ingester-zone-c
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: ingester
      app.kubernetes.io/part-of: memberlist
      name: ingester-zone-c
      rollout-group: ingester
    annotations:
      rollout-max-unavailable: "1"
  spec:
    replicas: 1
    podManagementPolicy: Parallel
    serviceName: loki-release-ingester-zone-c-headless
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: ingester
        name: ingester-zone-c
        rollout-group: ingester
    updateStrategy:
      type: RollingUpdate
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: ingester
          app.kubernetes.io/part-of: memberlist
          name: ingester-zone-c
          rollout-group: ingester
      spec:
        topologySpreadConstraints:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/component: ingester
                app.kubernetes.io/instance: 'loki-release'
                app.kubernetes.io/name: 'loki'
            maxSkew: 1
            topologyKey: kubernetes.io/hostname
            whenUnsatisfiable: ScheduleAnyway
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 300
        containers:
          - name: ingester
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -ingester.availability-zone=zone-c
              - -ingester.unregister-on-shutdown=false
              - -ingester.tokens-file-path=/var/loki/ring-tokens
              - -target=ingester
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                    - key: rollout-group
                      operator: In
                      values:
                        - ingester
                    - key: name
                      operator: NotIn
                      values:
                        - ingester-zone-c
                topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: data
            emptyDir: {}
default, loki-release-querier, Deployment (apps) has changed:
  # Source: loki/templates/querier/deployment-querier.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: loki-release-querier
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: querier
      app.kubernetes.io/part-of: memberlist
  spec:
    replicas: 3
    strategy:
      rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: querier
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: querier
          app.kubernetes.io/part-of: memberlist
      spec:
        topologySpreadConstraints:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/component: querier
                app.kubernetes.io/instance: 'loki-release'
                app.kubernetes.io/name: 'loki'
            maxSkew: 1
            topologyKey: kubernetes.io/hostname
            whenUnsatisfiable: ScheduleAnyway
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 30
        containers:
          - name: querier
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=querier
              - -distributor.zone-awareness-enabled=true
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
              - name: data
                mountPath: /var/loki
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: querier
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
          - name: data
            emptyDir: {}
default, loki-release-query-frontend, Deployment (apps) has changed:
  # Source: loki/templates/query-frontend/deployment-query-frontend.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: loki-release-query-frontend
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: query-frontend
  spec:
    replicas: 2
    strategy:
      rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: query-frontend
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: query-frontend
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 30
        containers:
          - name: query-frontend
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=query-frontend
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
            resources:
              {}
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: query-frontend
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
default, loki-release-query-scheduler, Deployment (apps) has changed:
  # Source: loki/templates/query-scheduler/deployment-query-scheduler.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: loki-release-query-scheduler
    namespace: default
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: query-scheduler
  spec:
    replicas: 2
    strategy:
      rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: query-scheduler
    template:
      metadata:
        annotations:
          checksum/config: 86e9512dd6ee479f2e728da8e5a8552cc803d3c249103c2b9b3fce3204b22084
        labels:
          helm.sh/chart: loki-6.46.0
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/version: "3.5.7"
          app.kubernetes.io/component: query-scheduler
          app.kubernetes.io/part-of: memberlist
      spec:
        serviceAccountName: loki-release
        
        securityContext:
          fsGroup: 10001
+         fsGroupChangePolicy: OnRootMismatch
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        terminationGracePeriodSeconds: 30
        containers:
          - name: query-scheduler
            image: docker.io/grafana/loki:3.5.7
            imagePullPolicy: IfNotPresent
            args:
              - -config.file=/etc/loki/config/config.yaml
              - -target=query-scheduler
            ports:
              - name: http-metrics
                containerPort: 3100
                protocol: TCP
              - name: grpc
                containerPort: 9095
                protocol: TCP
              - name: http-memberlist
                containerPort: 7946
                protocol: TCP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                drop:
                - ALL
              readOnlyRootFilesystem: true
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /ready
                port: http-metrics
              initialDelaySeconds: 15
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 1
            volumeMounts:
              - name: config
                mountPath: /etc/loki/config
              - name: runtime-config
                mountPath: /etc/loki/runtime-config
        affinity:
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app.kubernetes.io/component: query-scheduler
                  app.kubernetes.io/instance: 'loki-release'
                  app.kubernetes.io/name: 'loki'
              topologyKey: kubernetes.io/hostname
        volumes:
          - name: config
            configMap:
              name: loki
              items:
                - key: "config.yaml"
                  path: "config.yaml"
          - name: runtime-config
            configMap:
              name: loki-runtime
default, loki-release-results-cache, StatefulSet (apps) has changed:
  # Source: loki/templates/results-cache/statefulset-results-cache.yaml
  apiVersion: apps/v1
  kind: StatefulSet
  metadata:
    name: loki-release-results-cache
    labels:
      helm.sh/chart: loki-6.46.0
      app.kubernetes.io/name: loki
      app.kubernetes.io/instance: loki-release
      app.kubernetes.io/version: "3.5.7"
      app.kubernetes.io/component: "memcached-results-cache"
      name: "memcached-results-cache"
    annotations:
      {}
    namespace: "default"
  spec:
    podManagementPolicy: Parallel
    replicas: 1
    selector:
      matchLabels:
        app.kubernetes.io/name: loki
        app.kubernetes.io/instance: loki-release
        app.kubernetes.io/component: "memcached-results-cache"
        name: "memcached-results-cache"
    updateStrategy:
      type: RollingUpdate
    serviceName: loki-release-results-cache
    template:
      metadata:
        labels:
          app.kubernetes.io/name: loki
          app.kubernetes.io/instance: loki-release
          app.kubernetes.io/component: "memcached-results-cache"
          name: "memcached-results-cache"
        annotations:
      spec:
        serviceAccountName: loki-release
        secu
<!-- Sticky Pull Request CommentHelm Diff Output - Summary -->

@trevorwhitney trevorwhitney force-pushed the twhitney/helm-security-updates branch from cdc2274 to 60dfd56 Compare November 21, 2025 18:56
@trevorwhitney
Copy link
Collaborator Author

trevorwhitney commented Nov 21, 2025

@jkroepke any ideas why a few patch version bumps would be breaking the helm integration tests?

@jkroepke
Copy link
Contributor

The helm tests named Integration Test doesn't expose any debug informations, it's not possible to get a real reason why the run failed. I guess it's a flasky test, because helm-ci / Test Helm Chart is green.

Try a re-run.

@trevorwhitney
Copy link
Collaborator Author

@petewall it looks like you added these helm toolbox integration tests in #18565. do you know how to get more information out of them, such as why the queries are failing (the responses would be good to have), or how to get pod logs before destroying the cluster? I'm having a lot of trouble understanding why this small of a change is failing the tests and not sure how to get more info.

@jkroepke
Copy link
Contributor

@trevorwhitney I remember at least that we had already an internal discussion about this and a potential solution is tracked here: grafana/helm-chart-toolbox#42

@jkroepke
Copy link
Contributor

@trevorwhitney while looking deeper at the Test Helm Chart job, I can observe that the loki sidecar for rules is crashing:

------------------------------------------------------------------------------------------------------------------------
==> Logs of container loki-backend-0
------------------------------------------------------------------------------------------------------------------------
{"time": "2025-11-21T23:50:19.567006+00:00", "level": "INFO", "msg": "Starting collector"}
{"time": "2025-11-21T23:50:19.567186+00:00", "level": "INFO", "msg": "No folder annotation was provided, defaulting to k8s-sidecar-target-directory"}
{"time": "2025-11-21T23:50:19.567394+00:00", "level": "INFO", "msg": "Loading incluster config..."}
{"time": "2025-11-21T23:50:19.568087+00:00", "level": "INFO", "msg": "Config for cluster api at '[https://10.96.0.1:443](https://10.96.0.1/)' loaded."}
{"time": "2025-11-21T23:50:19.568192+00:00", "level": "INFO", "msg": "Unique filenames will not be enforced."}
{"time": "2025-11-21T23:50:19.568298+00:00", "level": "INFO", "msg": "5xx response content will not be enabled."}
{"time": "2025-11-21T23:50:19.581292+00:00", "level": "ERROR", "msg": "FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/app']", "exc_info": "Traceback (most recent call last):\n  File \"/app/sidecar.py\", line 183, in <module>\n    main()\n    ~~~~^^\n  File \"/app/sidecar.py\", line 137, in main\n    watch_for_changes(method, label, label_value, target_folder, request_url, request_method, request_payload,\n    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                      namespace, folder_annotation, resources, unique_filenames, script, enable_5xx,\n                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                      ignore_already_processed, resource_name)\n                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/app/resources.py\", line 429, in watch_for_changes\n    processes = _start_watcher_processes(current_namespace, folder_annotation, label,\n                                         label_value, request_method, mode, request_payload, resources,\n                                         target_folder, unique_filenames, script, request_url, enable_5xx,\n                                         ignore_already_processed, resource_name)\n  File \"/app/resources.py\", line 462, in _start_watcher_processes\n    proc.start()\n    ~~~~~~~~~~^^\n  File \"/usr/local/lib/python3.14/multiprocessing/process.py\", line 121, in start\n    self._popen = self._Popen(self)\n                  ~~~~~~~~~~~^^^^^^\n  File \"/usr/local/lib/python3.14/multiprocessing/context.py\", line 224, in _Popen\n    return _default_context.get_context().Process._Popen(process_obj)\n           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.14/multiprocessing/context.py\", line 300, in _Popen\n    return Popen(process_obj)\n  File \"/usr/local/lib/python3.14/multiprocessing/popen_forkserver.py\", line 35, in __init__\n    super().__init__(process_obj)\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.14/multiprocessing/popen_fork.py\", line 20, in __init__\n    self._launch(process_obj)\n    ~~~~~~~~~~~~^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.14/multiprocessing/popen_forkserver.py\", line 51, in _launch\n    self.sentinel, w = forkserver.connect_to_new_process(self._fds)\n                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^\n  File \"/usr/local/lib/python3.14/multiprocessing/forkserver.py\", line 89, in connect_to_new_process\n    self.ensure_running()\n    ~~~~~~~~~~~~~~~~~~~^^\n  File \"/usr/local/lib/python3.14/multiprocessing/forkserver.py\", line 156, in ensure_running\n    address = connection.arbitrary_address('AF_UNIX')\n  File \"/usr/local/lib/python3.14/multiprocessing/connection.py\", line 79, in arbitrary_address\n    return tempfile.mktemp(prefix='sock-', dir=util.get_temp_dir())\n                                               ~~~~~~~~~~~~~~~~~^^\n  File \"/usr/local/lib/python3.14/multiprocessing/util.py\", line 212, in get_temp_dir\n    base_tempdir = _get_base_temp_dir(tempfile)\n  File \"/usr/local/lib/python3.14/multiprocessing/util.py\", line 169, in _get_base_temp_dir\n    base_tempdir = tempfile.gettempdir()\n  File \"/usr/local/lib/python3.14/tempfile.py\", line 316, in gettempdir\n    return _os.fsdecode(_gettempdir())\n                        ~~~~~~~~~~~^^\n  File \"/usr/local/lib/python3.14/tempfile.py\", line 309, in _gettempdir\n    tempdir = _get_default_tempdir()\n  File \"/usr/local/lib/python3.14/tempfile.py\", line 224, in _get_default_tempdir\n    raise FileNotFoundError(_errno.ENOENT,\n                            \"No usable temporary directory found in %s\" %\n                            dirlist)\nFileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/app']"}
------------------------------------------------------------------------------------------------------------------------
<== Logs of container loki-backend-0
------------------------------------------------------------------------------------------------------------------------

Maybe this could be the reason, #19973 for a potential fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants