Make 401 challenge handling pluggable (IAuthChallengeHandler)

[akashsinghal]

Problem

Some registries mishandle a stale/expired cached bearer token so the client cannot recover:

• One returns 401 with no WWW-Authenticate challenge when a stale token is presented. A cold, credential-free request to the same URL does return a usable challenge.
• Another returns a 401 whose challenge realm points at a different host, which IRealmValidator (correctly) rejects. A cold request returns a same-host realm.

Today Client.SendCoreAsync hard-codes the challenge to token flow: an unknown/absent challenge falls through to return response1, and a disallowed/missing realm throws — neither is recoverable by a consumer, because both happen before any token-acquisition hook runs.

Proposal

Introduce IAuthChallengeHandler, letting the caller own the "401 → authorization" decision, with a default (DefaultAuthChallengeHandler) that preserves today's behavior exactly. The client keeps only the HTTP mechanics (cached-token attach, retry send, caching). Recovery for non-conformant registries lives in a consumer-supplied handler plus a shipped example — the core stays free of upstream-specific logic (unlike the previously-closed #400, which baked cold-retry into the core loop).

Contracts:

• AuthChallengeContext — the failed exchange (original request, 401 response, parsed scheme/parameters, host, scopes, whether a cached token was attached, whether the request is replayable) plus capabilities that delegate to the client: a credential-free probe, realm validation, scope merging, cached-token lookup, and Basic/Bearer token fetch.
• AuthChallengeResolution — scheme + token (+ cache scope key / cache flag). null means "give up; return the 401 unchanged."

Behavior-preserving: no existing test changes.

Is this direction acceptable? Implementation is ready and I am happy to open the PR.

[akashsinghal]

Composability review — and a proposed pivot

Reviewing the seam specifically for composability, I think it's drawn at the wrong boundary. It works and is behavior-preserving, but it's an awkward middle between two clean designs — a fat, Client-bound context feeding a narrow, lossy resolution — which splits "response handling" across the Client and the handler.

Where the coupling is

1. The context is a Client façade, not a context. AuthChallengeContext holds private readonly Client _client and re-exposes six client operations (SendWithoutAuthorizationAsync, IsRealmAllowedAsync, FetchBearerTokenAsync, FetchBasicTokenAsync, TryGetCachedToken, MergeChallengeScopes). A handler therefore cannot exist or be unit-tested without a live Client. Extension points that require the extended object to be present are callbacks, not seams.
2. Response handling is split, not owned. The handler can read the 401 and send a probe, but the retry send and caching stay in the Client, and the scope-shortcut sends its own request inside the Client before the handler even runs. Four actors touch the flow: Client (attempt-1) → Client (shortcut) → Handler (probe) → Client (retry).
3. The resolution is lossy. AuthChallengeResolution { Scheme, Token, CacheScopeKey, Cache } forces every outcome into "a token to cache and re-send." A handler that already has the final response (a cold probe that returned 200, or a bespoke flow) must instead hand back a token for the client to send again. Cache/CacheScopeKey leak the client's caching model into the output contract.

Root cause

Authentication-with-retry is fundamentally "given a request, produce an authenticated response" — a transport/middleware concern. Carving out "given a 401, return a token" forces the surrounding response mechanics to leak back in through the context.

Cleaner approach — IRegistryAuthenticator

Make the pluggable unit produce an authenticated response, with the raw transport injected:

public delegate Task<HttpResponseMessage> RegistryTransport(
    HttpRequestMessage request, bool allowAutoRedirect, CancellationToken ct);

public interface IRegistryAuthenticator
{
    Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, AuthContext context, RegistryTransport transport, CancellationToken ct);
}

• No Client in the contract. The cold probe is just transport(coldRequest, ...) — no façade method.
• No split. The authenticator owns attach → send → 401 → resolve → retry → cache in one place.
• Composable + testable. Fake the transport delegate; no Client/HttpClient needed. Recovery (dhi/nvcr) becomes a subclass/decorator of the default authenticator, not a new context capability.
• Client shrinks to: build the request, pick partitionId/allowAutoRedirect, hand its transport to the authenticator.

(An idiomatic DelegatingHandler variant is possible too, but it collides with oras-dotnet's two-HttpClient redirect design; passing allowAutoRedirect through the transport delegate sidesteps that.)

I'm prototyping IRegistryAuthenticator to compare shapes concretely.

[akashsinghal]

Design decision: scope the seam to challenge recovery, not auth ownership

Recording where we landed after working through the options here, and the reasoning.

The need (precisely)

Some registries mishandle a stale/expired cached bearer token:

• one returns 401 with no WWW-Authenticate when a token is presented (a credential-free request to the same URL does return a usable challenge);
• another returns a 401 whose challenge realm points at a different host (rejected by realm validation; a cold request returns a same-host realm).

For context on "conformance": the OCI image/distribution specs don't define an authentication scheme at all — the token flow is the docker/distribution spec's, which is the de-facto standard MCR / Docker Hub / most registries implement. The affected registries simply don't honor the spirit of that widely-adopted flow. So what's needed is a way to tolerate those deviations — recovery — not a way to reimplement auth.

Options considered

Design	Why not
#400 — cold-retry baked into SendCoreAsync	put registry-specific recovery unconditionally into the core loop
#406 — IAuthChallengeHandler + AuthChallengeContext	the context was bound to Client (held a ref and re-exposed ~6 of its ops) — a façade; a lossy {scheme, token} result split response handling across client and handler; not testable without a Client
prototype — IRegistryAuthenticator.SendAsync(request, ctx, transport, ct)	a near-duplicate of the existing IClient.SendAsync(request, partitionId, allowAutoRedirect, ct) — same "authenticated send" granularity; its real extension point turned out to be subclassing, and it doesn't chain like middleware
resolver — Client-held IAuthChallengeResolver (challenge → authorization)	the default resolver needs the same collaborators as Client (realm validator, scope manager, credential/access-token providers, token exchange, cache) → either the #406 façade returns (pass them in) or a silent config-sync footgun (duplicate them); plus a lossy authorization result and a split of scope/cache logic. Carrying it per-request via HttpRequestMessage.Options also reintroduced string-keyed late binding (a key/type drift silently falls back to default — no compile error).

The through-line: every design that exposed token acquisition dragged in the entire auth dependency graph (cache, scope, realm, providers, token exchange) — yielding a façade, duplication, or an IClient twin. But the consumer doesn't want to own auth. It wants to recover a broken challenge.

Where we're landing: a narrow, opt-in ChallengeRecovery hook

Keep all standard auth (Basic/Bearer/OAuth2, realm validation, scope handling, token exchange, caching, retry) internal to one owner — the engine. Expose one compile-safe seam, consulted only when standard resolution dead-ends on a structurally-unusable challenge:

public delegate Task<HttpResponseMessage?> ChallengeRecoveryHandler(
    FailedChallenge context, CancellationToken ct);

public class Client : IClient
{
    // default null == today's exact behavior; opt-in
    public ChallengeRecoveryHandler? ChallengeRecovery { get; init; }
}

public sealed class FailedChallenge
{
    public HttpResponseMessage UnauthorizedResponse { get; }   // the 401
    public string Host { get; }
    public bool AttachedCachedToken { get; }                   // stale-token hint
    public bool CanReplay { get; }                             // idempotent GET/HEAD
    public Func<CancellationToken, Task<HttpResponseMessage>> ProbeWithoutAuthorizationAsync { get; }
}

Call site and conditions — one place, after standard resolution fails and before give-up:

1. attempt-1 returned 401;
2. standard resolution produced no usable challenge (Unknown/absent scheme, or missing/invalid/disallowed realm). Credential and token-endpoint failures throw and never reach here, so recovery can never mask a real auth error;
3. a recovery is configured.

The handler then gates on AttachedCachedToken && CanReplay and cold-probes. The recovered response is run through standard resolution once — so a cold 200 for an anonymous resource is returned as-is, and a recovered challenge goes through the normal validate → fetch → cache → retry. Invoked at most once; the cold probe carries no token, so it can't re-arm.

Why a delegate, not an interface. There is realistically one recovery strategy — a credential-free cold re-derive — and it's completely generic / host-agnostic (no registry-specific code; a single handler covers both registries above). Realm allow-listing is already IRealmValidator; credential rotation is the providers'. So we're not designing for imagined implementations. Promoting a Func to an interface later is a non-breaking change if a genuinely different strategy ever appears.

Why opt-in (default off). This is not a performance decision: the gate makes it free for conformant registries — standard resolution succeeds, recovery never fires, no extra request and no allocation on that path. Recovery only fires on the already-broken stale-token + unusable-challenge case, adding one cold request to turn a guaranteed failure into a success. It's off by default because cold-probing around a challenge the server withheld is a deliberate tolerance of a registry that deviates from the de-facto distribution auth flow — the right place to make that call is the consumer, not the library default (and it eases adoption vs. changing behavior for everyone). oras can ship a ready-made cold-probe recovery that consumers opt into.

Net public surface: one nullable delegate property + one data type. IClient untouched; no façade, no subclassing, no IClient twin, no string bag, no duplicated auth graph. I'll revise the PR toward this shape.

[akashsinghal]

📊 Interactive design walkthrough

I put together a visual, progressively-disclosed explainer of the direction we landed on here — the opt-in ChallengeRecovery seam — and why, including the alternatives we worked through (#400, #406, the IRegistryAuthenticator prototype, and the resolver variant) and the through-line that drove the decision. It opens with the top-line direction and discloses more detail as you scroll (problem → key insight → rejected options → the chosen design → the finer points).

🔗 Rendered walkthrough: https://htmlpreview.github.io/?https://gist.githubusercontent.com/akashsinghal/44e213166b973018312bc7c03b4654ca/raw/challenge-recovery-explainer.html

@sajayantony @cshung I'd appreciate if you could take a look.