Customer onboarding: assess fallout from the invalid RBAC action in the cross-subscription ARM template

[cristim]

Found while root-causing #1794. The code fix ships with that issue's PR; this issue is about the operational fallout the code fix does not address.

What happened

arm/CUDly-CrossSubscription/template.json:51 contained Microsoft.Capacity/reservationOrders/purchase/action, which does not exist in Azure's operation catalog. Azure rejects an entire Microsoft.Authorization/roleDefinitions resource that references an unknown action with InvalidActionOrNotAction (400).

Verified against the live catalog and reproduced end to end on the Terraform twin of this role: the deployment fails, and removing the single action makes it succeed. See #1794 for the full evidence.

Why this needs its own issue

That ARM template is customer-facing. Customers deploy it to grant CUDly cross-subscription access during onboarding. Every customer who ran it should have hit the same 400.

The fix corrects the template going forward. It does not tell us:

1. How many customers attempted onboarding with the broken template, and since when. The invalid action needs a git log -S to date its introduction, which bounds the affected window.
2. Whether affected customers are in a partial state. The template also declares role assignments that dependsOn the role definition. Those should not have been created, but ARM partial-deployment behaviour should be confirmed rather than assumed, especially given the template declares an assignment at /providers/Microsoft.Capacity scope.
3. Whether onboarding failures were visible to us at all, or whether customers simply saw a failed ARM deployment and did not report it. If the latter, that is its own gap: a silent onboarding failure mode with no telemetry.
4. Whether affected customers need to re-run the corrected template, and whether re-running is idempotent for those left partially deployed.

Also worth checking while in here

The ARM template's assignableScopes includes /providers/Microsoft.Capacity, but the Terraform module deliberately removed that scope, documenting that a subscription-scoped principal cannot register an assignable scope above its own subscription and that including it makes the role-definition write 403 at the higher scope.

So ARM and Terraform have diverged on assignable scopes, and the ARM side carries the scope the TF side documented as unusable. The role-parity checker did not flag this. That may be a second latent onboarding failure, independent of the invalid action, and it should be tested rather than reasoned about.

Verification

Deploy the corrected template into a scratch subscription and confirm the role definition and both role assignments are created. Do this specifically for the /providers/Microsoft.Capacity-scoped assignment, which is the one most likely to still fail after the action fix.

[cristim]

Correction: the assignableScopes section of this issue is wrong. Please disregard it.

I claimed the ARM template's assignableScopes still includes /providers/Microsoft.Capacity while the Terraform module had removed it, and suggested that as a possible second latent onboarding failure.

That is not true on main:

"assignableScopes": [
  "[concat('/subscriptions/', subscription().subscriptionId)]"
]

#1658 ("sec(iac): scope Azure purchase role to onboarded subscription") already fixed exactly this for #1545, and the role-parity checker's scope axis plus the tenant-scope-arm.json fixture exist specifically to keep it fixed. There is nothing to investigate and nothing to file.

I read the file from a checkout that was behind origin/main, so I was looking at the pre-#1658 revision. Same mistake that made me undercount the affected fixtures at 8 when the real number was ~40.

The rest of this issue stands and is unaffected, because it rests on the invalid action, which is independently confirmed against Azure's live operation catalog:

• Microsoft.Capacity/reservationOrders/purchase/action does not exist, and Azure rejects any role definition referencing it with InvalidActionOrNotAction (400).
• That action is present in arm/CUDly-CrossSubscription/template.json on main, so customer onboarding through this template should be failing.
• The open questions about affected-customer count, partial deployment state, missing onboarding telemetry, and whether re-running the corrected template is idempotent are all still open.

One thing this correction strengthens: the assignableScopes regression guard demonstrably worked. The scope axis was caught, fixed, and pinned. What no test covered was whether the actions are things Azure will accept, which is the gap that let this ship.

[cristim]

Answering question 1: the affected window is 76 days

git log -S 'reservationOrders/purchase/action' -- arm/CUDly-CrossSubscription/template.json returns exactly one commit that introduces the string:

f34b036e2  2026-05-27  fix(iac/azure): custom role grants Microsoft.Capacity/...

Removed 2026-08-11 by PR #1800.

So any customer who deployed arm/CUDly-CrossSubscription/template.json between 2026-05-27 and 2026-08-11 should have hit InvalidActionOrNotAction (400) and failed to onboard. That is a 76-day window.

The template itself predates this (first appears 2026-04-08, 485344dbd), so onboarding was plausibly working before 2026-05-27. Worth confirming whether f34b036e2 introduced the custom roleDefinitions resource wholesale or only added the bad action to an existing one, since that decides whether the pre-May-27 path was healthy or merely differently broken. I have not established that.

What this does not answer

The remaining questions in this issue need data outside the repository and someone with access should take them:

1. How many customers attempted onboarding in that window. The repo cannot answer this.
2. Whether affected customers are in a partial state. The template declares role assignments that dependsOn the role definition, so they should not have been created, but ARM partial-deployment behaviour should be confirmed rather than assumed.
3. Whether we had any signal at all. If a customer's ARM deployment failed and nobody was notified, that is a separate and arguably worse defect than the invalid action: a silent onboarding failure with no telemetry. 76 days is a long time for a broken onboarding path to go unreported, which is itself weak evidence that either very few customers used it or that failures were invisible to us. Both readings are worth knowing about.
4. Whether re-running the corrected template is idempotent for anyone left partially deployed.

Correction carried forward

The assignableScopes concern originally raised in this issue was wrong and has been retracted in a separate comment. #1658 had already fixed it. The invalid-action finding is unaffected and independently confirmed against Azure's live operation catalog.