Skip to content

Commit 639919f

Browse files
committed
blog: Introduce ignore rules for drift detection
Signed-off-by: Dipti Pai <diptipai89@outlook.com>
1 parent aa2df3f commit 639919f

2 files changed

Lines changed: 234 additions & 0 deletions

File tree

218 KB
Loading
Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
---
2+
author: Dipti Pai
3+
date: 2026-07-20 12:00:00+00:00
4+
title: Selective drift correction with ignore rules
5+
description: "Let external controllers like HPA, VPA and external-dns manage specific fields without fighting Flux drift correction."
6+
url: /blog/2026/07/ignore-rules-drift-detection/
7+
tags: [ecosystem]
8+
resources:
9+
- src: "**.{png,jpg}"
10+
title: "Image #:counter"
11+
---
12+
13+
We are excited to introduce **drift ignore rules** for Flux Kustomizations, a long-requested
14+
capability that lets you tell Flux to leave specific fields alone during drift detection and
15+
correction, while continuing to reconcile everything else.
16+
17+
![](featured-image.png)
18+
19+
One of the core promises of GitOps with Flux is continuous reconciliation: the
20+
[kustomize-controller](https://fluxcd.io/flux/components/kustomize/) runs a periodic server-side
21+
apply dry-run to detect drift between the desired state in Git and the live state in the cluster,
22+
and corrects any divergence. This is exactly what you want for most fields; it guarantees that the
23+
cluster always matches the source of truth and that manual `kubectl` edits are reverted.
24+
25+
But not every field on a Kubernetes object should be owned by Flux. In real clusters, other
26+
controllers legitimately mutate parts of the resources that Flux manages, and Flux's drift
27+
correction ends up fighting them on every reconcile.
28+
29+
## The problem: when drift correction fights other controllers
30+
31+
Consider a Deployment that Flux reconciles from Git with `replicas: 2`. You also run a
32+
[HorizontalPodAutoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)
33+
that scales the workload to `5` replicas under load. On the next reconciliation, Flux sees the
34+
live `replicas: 5` as drift from the desired `replicas: 2` and scales the Deployment back down,
35+
undoing the autoscaler. The HPA scales it back up, Flux scales it back down, and the two
36+
controllers ping-pong indefinitely.
37+
38+
I ran into a memorable variant of this myself. On a small cluster, an HPA had been configured with a
39+
`minReplicas` floor well below the replica count baked into the GitOps manifest. Every time Flux
40+
reconciled, it bumped the Deployment back up to the manifest's much higher replica count. That surge
41+
was enough to exceed the cluster's capacity, so the cluster autoscaler dutifully spun up a brand new
42+
node to schedule the extra pods. Moments later the HPA scaled the Deployment back down to its
43+
`minReplicas` floor, the new node went idle, and the autoscaler tore it down again. Reconcile,
44+
scale up, add node, scale down, remove node, repeat, on every interval. What should have been a
45+
quiet steady state turned into constant node churn and a SEV2 incident.
46+
47+
The same pattern shows up with many other tools:
48+
49+
- **VerticalPodAutoscaler** adjusting `spec.template.spec.containers[*].resources`.
50+
- **external-dns** writing the `external-dns.alpha.kubernetes.io/hostname` annotation on a Service.
51+
- An operator that owns a single key inside a ConfigMap while Flux owns the rest.
52+
53+
Until now, the options were coarse: disable drift correction for the whole Kustomization, or
54+
exclude the entire resource from reconciliation. Both throw away the safety that GitOps gives you
55+
for all the *other* fields on those objects.
56+
57+
Drift ignore rules solve this at the field level.
58+
59+
## Introducing `.spec.ignore`
60+
61+
The Kustomization API now has an optional `.spec.ignore` list. Each rule combines a set of
62+
[JSON Pointer (RFC 6901)](https://datatracker.ietf.org/doc/html/rfc6901) `paths` with an optional
63+
`target` selector that scopes the rule to specific resources:
64+
65+
```yaml
66+
---
67+
apiVersion: kustomize.toolkit.fluxcd.io/v1
68+
kind: Kustomization
69+
metadata:
70+
name: app
71+
namespace: flux-system
72+
spec:
73+
interval: 10m
74+
prune: true
75+
sourceRef:
76+
kind: GitRepository
77+
name: app
78+
ignore:
79+
- paths:
80+
- "/spec/replicas"
81+
target:
82+
kind: Deployment
83+
```
84+
85+
With this rule in place, Flux ignores `/spec/replicas` on all Deployments during drift detection
86+
and correction, allowing an HPA to own the replica count, while still correcting drift on every
87+
other field (image tags, environment variables, labels, and so on).
88+
89+
The `target` selector supports `group`, `version`, `kind`, `name` and `namespace` (all matched as
90+
regular expressions), as well as `labelSelector` and `annotationSelector` using standard Kubernetes
91+
selector syntax. If you omit `target`, the rule applies to **all** resources managed by the
92+
Kustomization, so it is good practice to always scope rules to the resources you intend.
93+
94+
> **Tip:** For JSON Pointer paths that contain a `/` in the key name, the `/` must be escaped as
95+
> `~1`. For example, the annotation `external-dns.alpha.kubernetes.io/hostname` is referenced as
96+
> `/metadata/annotations/external-dns.alpha.kubernetes.io~1hostname`.
97+
98+
## How it works
99+
100+
Drift ignore rules build on the Kubernetes
101+
[server-side apply](https://kubernetes.io/docs/reference/using-api/server-side-apply/) field
102+
ownership model. When a reconciliation is triggered, for each ignored path the controller resolves
103+
one of two strategies based on who owns the field:
104+
105+
- **Strip** — if the ignored field is owned by another Apply-type field manager (for example,
106+
another controller using server-side apply such as an HPA or VPA), Flux removes the field from
107+
its apply payload. This relinquishes Flux's ownership and lets the other controller keep full
108+
control of the field.
109+
- **Adopt** — if Flux is the sole Apply-type field manager for the field, the in-cluster value is
110+
copied into the apply payload. This preserves the current value, even after a `kubectl patch` or
111+
`kubectl edit`, without Flux reverting it and while keeping Flux's field ownership intact.
112+
113+
A key property is that **changes to ignored fields alone do not trigger a reconciliation**. The
114+
controller excludes ignored paths when comparing the desired state against the live object, so a
115+
modification made by an external controller to an ignored field will not cause an unnecessary apply
116+
or a resource version bump. The earlier ping-pong simply does not happen.
117+
118+
## Real-world examples
119+
120+
### Let an autoscaler own replicas
121+
122+
The canonical use case: hand `replicas` over to a HorizontalPodAutoscaler.
123+
124+
```yaml
125+
spec:
126+
ignore:
127+
- paths:
128+
- "/spec/replicas"
129+
target:
130+
kind: Deployment
131+
name: podinfo
132+
```
133+
134+
If the HPA scales `podinfo` from 2 to 5 replicas, Flux leaves the count at 5. If you also bump the
135+
container image in Git, Flux still corrects the image; the ignore rule only protects `replicas`.
136+
137+
### Let a VerticalPodAutoscaler own container resources
138+
139+
VPA continuously recommends CPU and memory requests/limits. Ignore the whole `resources` subtree of
140+
the container so VPA's values stick:
141+
142+
```yaml
143+
spec:
144+
ignore:
145+
- paths:
146+
- "/spec/template/spec/containers/0/resources"
147+
target:
148+
kind: Deployment
149+
name: podinfo
150+
```
151+
152+
### Let external-dns own a Service annotation
153+
154+
external-dns manages a hostname annotation on a Service. Ignore just that annotation while keeping
155+
the rest of the Service under Flux control:
156+
157+
```yaml
158+
spec:
159+
ignore:
160+
- paths:
161+
- "/metadata/annotations/external-dns.alpha.kubernetes.io~1hostname"
162+
target:
163+
kind: Service
164+
name: frontend
165+
```
166+
167+
### Combine multiple rules
168+
169+
Rules compose, so you can cover several resources in a single Kustomization:
170+
171+
```yaml
172+
spec:
173+
ignore:
174+
- paths:
175+
- "/spec/replicas"
176+
- "/spec/template/spec/containers/0/resources"
177+
target:
178+
kind: Deployment
179+
name: podinfo
180+
- paths:
181+
- "/metadata/annotations/external-dns.alpha.kubernetes.io~1hostname"
182+
target:
183+
kind: Service
184+
name: frontend
185+
- paths:
186+
- "/data/managed-by-operator"
187+
target:
188+
kind: ConfigMap
189+
name: app-config
190+
```
191+
192+
## Preview changes with `flux diff`
193+
194+
The Flux CLI honors the ignore rules too. Running
195+
[`flux diff kustomization`](https://fluxcd.io/flux/cmd/flux_diff_kustomization/) gives you an
196+
accurate preview that matches what the controller will actually do, so ignored fields are not
197+
reported as spurious diffs:
198+
199+
```sh
200+
flux diff kustomization my-app --path ./path/to/local/manifests --kustomization-file ./path/to/local/my-app.yaml
201+
```
202+
203+
## Notes and limitations
204+
205+
- Drift ignore rules are designed to work with the server-side apply field ownership model. They are
206+
the right tool for delegating fields to other controllers that use server-side apply (HPA, VPA,
207+
external-dns, cert-manager, and similar).
208+
- Always scope rules with `target` unless you genuinely want to ignore the given paths across every
209+
resource in the Kustomization.
210+
- Because ignored fields are excluded from drift comparison, Flux will not update an ignored field
211+
even when its value changes in Git while another controller owns it. The field is intentionally
212+
delegated.
213+
214+
## Wrapping up
215+
216+
Drift ignore rules close a long-standing gap: you no longer have to choose between full GitOps drift
217+
correction and coexisting with autoscalers and other field-level controllers. You get both: Flux
218+
keeps the cluster aligned with Git for everything you declare, and steps back from the specific
219+
fields you delegate.
220+
221+
This capability landed across several Flux components:
222+
223+
- [fluxcd/pkg#1157](https://github.com/fluxcd/pkg/pull/1157) — field-level ignore support in the
224+
server-side apply engine.
225+
- [fluxcd/kustomize-controller#1627](https://github.com/fluxcd/kustomize-controller/pull/1627) — the
226+
`.spec.ignore` API on the Kustomization.
227+
- [fluxcd/flux2#5923](https://github.com/fluxcd/flux2/pull/5923) — `flux diff kustomization` support
228+
for ignore rules.
229+
230+
For the full reference, see the
231+
[Ignore Rules](https://fluxcd.io/flux/components/kustomize/kustomizations/#ignore-rules) section of
232+
the Kustomization documentation. As always, we would love to hear your feedback, join us on the
233+
[CNCF Slack](https://cloud-native.slack.com/messages/flux) or open an issue on
234+
[GitHub](https://github.com/fluxcd/flux2).

0 commit comments

Comments
 (0)