[FEATURE] Allow the Cluster Autoscaler startup-taint prefix in NodeReadinessRule taint keys

[rajathagasthya]

Is your feature request related to a problem or existing issue? Please describe.

Cluster Autoscaler treats a taint as a startup taint in two ways: the --startup-taint flag, or the reserved key prefix startup-taint.cluster-autoscaler.kubernetes.io/, which is auto-detected without any flag.

On managed platforms the flag is not user-editable, so the reserved prefix is the only way to get startup-taint semantics:

• GKE documents the prefix as the only supported mechanism (docs).
• AKS supports boot-time taints via --node-init-taints, but its autoscaler does not expose --startup-taints (Azure/AKS#3276)

A single taint key cannot carry both prefixes. So on GKE and AKS, NRC cannot be the component that removes the startup taint.

Concrete use case

Gating scheduling on autoscaled GPU nodes: the node pool template applies a startup taint, Node Problem Detector publishes a GPU readiness node condition, and NRC removes the taint when the condition is True. This works with self-managed Cluster Autoscaler (where --startup-taint can be set) but not on GKE or AKS.

Describe the solution you'd like

Extend the validation to also accept the Cluster Autoscaler startup-taint prefix:

// +kubebuilder:validation:XValidation:rule="self.key.startsWith('readiness.k8s.io/') || self.key.startsWith('startup-taint.cluster autoscaler.kubernetes.io/')"

Optionally also allow the legacy ignore-taint.cluster-autoscaler.kubernetes.io/ prefix, which older autoscaler versions auto-detect.

Describe alternatives you've considered

• Admin-configured prefix allowlist (controller flag). More general, but CRD-level CEL cannot read controller flags, so apply-time validation would move to the reconciler (or webhook).
• Cluster Autoscaler auto-detecting readiness.k8s.io/ as a startup-taint namespace. A good long-term change on the autoscaler side, but this requires coordination among multiple providers.

[ajaysundark]

Hi @rajathagasthya - Thanks for raising this.

Cluster Autoscaler auto-detecting readiness.k8s.io/ as a startup-taint namespace. A good long-term change on the autoscaler side, but this requires coordination among multiple providers.

We already handled this for cluster-autoscaler in kubernetes/autoscaler#9243 and looking to add this in GKE as an inherent solution.

[rajathagasthya]

Thanks @ajaysundark!

kubernetes/autoscaler#9243 makes sense and is an improvement, but it's still a flag on the CA binary. Self-managed deployments can already use --startup-taint, but if you are on managed autoscalers (GKE, AKS) these flags cannot be set by the user.

Do you mean GKE will expose --startup-taint-prefix to admins, or enable readiness.k8s.io/ as a startup-taint prefix by default? Either way, my concern is that every provider would need to adopt it separately.

Allowing the reserved startup-taint.cluster-autoscaler.kubernetes.io/ prefix in NodeReadinessRule would be complementary. It works on those platforms today since their autoscalers already detect that prefix without any flag.

[ajaysundark]

The plan in GKE is not to expose the flag, but CA will have the --start-taint-prefix=readiness.k8s.io/ already baked in. Will that work for you?

I understand the concern that other managed autoscalers will not have it available yet. My concern is losing an unified taint-prefix for NRC will have longer implications, and I think more discussion / data-points are needed to decide on the direction. For start, we could start discussion in the SIG-autoscaling community and maybe capture 'recommended best-practices' in the project pages to encourage adopters to leverage this solution.

[rajathagasthya]

Yes, that works for GKE. And I agree keeping a single readiness.k8s.io/ prefix for NRC is the right call.

As concrete data point for the discussion: we're documenting a GPU Operator + Cluster Autoscaler integration that uses NRC to gate scheduling on GPU readiness. It works on self-managed CA today, and will work on GKE once startup taint baked in, but on AKS the pattern is blocked there for now.

So +1 to taking this to SIG-autoscaling. It would be helpful for adoption if all providers can support this taint prefix out of the box.

Please feel free to close this issue since we won't be doing what I proposed, or if you want to track it separately.