Skip to content

upgrading.md

Latest commit

 

History

History
232 lines (178 loc) · 8.41 KB

upgrading.md

File metadata and controls

232 lines (178 loc) · 8.41 KB
Raw

Upgrading

This page covers breaking changes and notable behavior shifts between versions. If you are upgrading from an earlier pre-release, apply every section below your current version (newest sections first).

Maintainers: before publishing a release after multi-version product changes, run the full E2E Nightly matrix on tip of main (see Releasing: full E2E matrix).

v0.1.22 to v0.1.23

v0.1.23 is a reliability patch. Existing policies keep working without YAML edits.

Requeue jitter is skipped during data collection

While Ready is InsufficientData or PrometheusUnavailable, the operator no longer adds --requeue-jitter (default 2m) on top of the cooldown. First recommendations and ConfigMap export can land sooner. Jitter still applies to steady-state cooldown requeues.

Fleet report savings stay finite

estimatedMonthlySavingsUSD adds only finite numbers. Unparseable values (NaN, Inf, or garbage) increment unparseableSavings and count as 0 in the USD total. Consumers of report.json should treat that field as additive.

v0.1.21 to v0.1.22

v0.1.22 ships scale defaults and capacity safety that were previously only on main. Review before upgrading from v0.1.21.

Default PromQL pod aggregation is Max

When metricsSource.podAggregation is unset, Attune now defaults to Max (max by (container) over the selected series) instead of leaving series unaggregated. Recommendations then follow the hottest pod for each container name, and Prometheus query cost stays proportional to containers rather than replicas.

If you relied on multi-pod sample pools (legacy unaggregated behavior), set explicitly:

spec:
  metricsSource:
    podAggregation: None   # or Avg

See Scaling: PromQL aggregation and Scaling: large fleets for operator flags related to large fleets (maxPodsInMetricsQuery, maxProfileSamples, informer field strip).

Default max concurrent reconciles is 2

The operator default for --max-concurrent-reconciles is 2 (was higher in some earlier builds). Helm clusterSize presets still override this when set. Large clusters that previously relied on more concurrent reconcilers should set the flag or a preset explicitly.

Request increases fail closed when node status is unavailable

If the operator cannot read node status for a pod, it skips request increases (decreases still allowed) and increments attune_capacity_skip_total{reason="unavailable"}. This is protective if node API access or informer lag fails.

Batch throttle chunking (no config change)

Safety observation batches CPU throttle PromQL queries and splits large pod/container sets into chunks of 64. No CRD field changes; Prometheus load for high-replica policies should drop further under rate limiting.

v0.1.20 to v0.1.21

v0.1.21 is a feature release. Existing policies keep working without YAML edits. Most new capabilities are opt-in. Read this section if you run Kubernetes 1.35+, GitOps export, multi-cluster rollups, or memory limit control.

Safe by default (no action required)

Area Behavior
GitOps pull request automation Off unless export.pullRequest.enabled: true
Fleet report ConfigMap export Off unless fleetReport (or Helm equivalent) is enabled
Runtime profiles Only apply when runtimeProfile is set
Export schema versioning Additive fields on recommendation ConfigMaps; consumers can ignore new keys

Behavior that can change without new fields

Memory limit decreases on Kubernetes 1.35+. On 1.35+, Attune no longer clamps memory limits the way it did on 1.33/1.34 when the platform allows live decreases and the policy uses controlledValues: RequestsAndLimits with decrease allowed. A usage floor still keeps the target limit above recent usage (default memory.decreaseUsageMarginPercent: 10). On 1.33–1.34, limit decreases remain clamped as before.

If you rely on “limits never go down in place,” pin an older cluster version, set memory.allowDecrease: false, use a restrictive runtime profile, or keep controlledValues: RequestsOnly (the default).

Capacity and node pressure. Resizes may be skipped more often when nodes are under pressure; metrics and status explain the skip. This is protective, not a CRD break.

Opt-in features worth enabling deliberately

Feature Where to start
GitOps PR automation GitOps integration
Multi-cluster fleet report Multi-cluster
Language runtime profiles Runtime profiles
Deferred / Infeasible UX Status conditions + troubleshooting
SLO PromQL guardrails (unchanged API; guide improved) SLO guardrails

Operator / install notes

  • Refresh CRDs with the release install path (helm upgrade or dist/crds.yaml / dist/install.yaml from the tag).
  • Grafana dashboard and PrometheusRule assets gain panels/alerts for GitOps PR outcomes, memory limit decrease safety, capacity skips, and related signals. Re-apply chart or dashboard ConfigMaps if you manage them out of band.
  • kubectl attune explain surfaces GitOps PR and runtime profile effective values; upgrade the plugin with the release for matching CLI help.

After upgrade, confirm policies with:

kubectl attune status -A
kubectl attune explain -n <namespace> <policy>

v1alpha1 Field Renames (v0.1.0)

Five CRD fields were renamed to align with ecosystem conventions. Existing AttunePolicy, AttuneDefaults, and AttuneNamespaceDefaults resources must be updated before applying the new CRDs.

Field mapping

Old field New field Conversion
safetyMargin: "1.2" overhead: "20" (old - 1) * 100
updateStrategy.mode updateStrategy.type rename only
bounds.min / bounds.max minAllowed / maxAllowed rename only
InPlaceOrEvict InPlaceOrRecreate rename only
excludeContainers excludedContainers rename only
updateStrategy.maxCpuChangePercent cpu.maxChangePercent move to cpu section
updateStrategy.maxMemoryChangePercent memory.maxChangePercent move to memory section

Overhead conversion examples

Old safetyMargin New overhead Meaning
"1.1" "10" 10% headroom
"1.15" "15" 15% headroom
"1.2" "20" 20% headroom (CPU default)
"1.3" "30" 30% headroom (memory default)
"1.5" "50" 50% headroom

Automated migration

Using sed (covers all five renames):

# All five renames in one pass
sed -i \
  -e 's/safetyMargin:/overhead:/g' \
  -e 's/overhead: "1.1"/overhead: "10"/g' \
  -e 's/overhead: "1.15"/overhead: "15"/g' \
  -e 's/overhead: "1.2"/overhead: "20"/g' \
  -e 's/overhead: "1.25"/overhead: "25"/g' \
  -e 's/overhead: "1.3"/overhead: "30"/g' \
  -e 's/overhead: "1.5"/overhead: "50"/g' \
  -e 's/InPlaceOrEvict/InPlaceOrRecreate/g' \
  -e 's/excludeContainers:/excludedContainers:/g' \
  manifests/*.yaml

# mode -> type (only in updateStrategy context to avoid false positives)
sed -i '/updateStrategy/,/^[^ ]/{s/mode:/type:/g}' manifests/*.yaml

# bounds.min/max -> minAllowed/maxAllowed (remove nesting manually if used)

Using yq (handles overhead conversion and bounds restructuring):

# Export current policies
kubectl get attunepolicies -n production -o yaml > policies.yaml

# Rename safetyMargin to overhead and convert values
yq -i '
  .items[].spec.cpu |= (
    .overhead = ((.safetyMargin | tonumber - 1) * 100 | tostring) |
    del(.safetyMargin)
  ) |
  .items[].spec.memory |= (
    .overhead = ((.safetyMargin | tonumber - 1) * 100 | tostring) |
    del(.safetyMargin)
  )
' policies.yaml

# Apply the new CRDs first, then re-apply policies
kubectl apply -f config/crd/bases/
kubectl apply -f policies.yaml

Helm values migration

If you use the Helm chart with custom defaults.cpu.overhead or defaults.memory.overhead in your values.yaml, update the values:

# Before
defaults:
  cpu:
    safetyMargin: "1.2"
  memory:
    safetyMargin: "1.3"

# After
defaults:
  cpu:
    overhead: "20"
  memory:
    overhead: "30"