Skip to content

Rotating an API key should not rewind the conversation #5701

Description

@mmabrouk

The problem

Change a model or MCP API key today and the conversation restarts from the beginning. The warm session is thrown away and the next turn replays the whole transcript.

That is the wrong trade. Rotating a key should update the credential and leave the conversation alone. It is especially wrong for an MCP key, where the user is fixing an integration mid-conversation and gets punished with a full replay for it.

Why it happens today

computeCredentialEpoch in services/runner/src/engines/sandbox_agent/session-identity.ts captures the credential material a parked session was built with. When the incoming request's material differs, server.ts records a credentials-rotated mismatch, evicts the warm session, and cold-starts.

The reasoning was sound when it was written: the credential is baked into the sandbox environment at create time, and you cannot change a running process's environment. Once a harness is running with OPENAI_API_KEY=sk-old, the only safe way to stop it using a revoked key was to throw the session away.

Note that credential VALUES are already excluded from configFingerprint. This is entirely the epoch's doing.

What changed

The Daytona Secrets work (#5670) removes the premise on remote sandboxes. With AGENTA_RUNNER_DAYTONA_OPAQUE_SECRETS=process_local, the sandbox holds a dtn_secret_<id> placeholder, not the key. A value rotation can be handled by updating the Daytona Secret record behind the placeholder, and the running sandbox never needs to know.

Only a change to the plan's SHAPE (a different binding name, a different allowed host, a credential added or removed) genuinely requires a new sandbox, because that is what is actually baked in.

The wider problem this exposes

The runner has one invalidation signal, "the fingerprint changed", and one consequence, "cold replay". Several kinds of change get funnelled into it that do not deserve the same treatment:

  • A credential value changed. Should re-bind the credential. On Daytona that is a Secret update with no teardown; on local it needs a new process but never a transcript replay.
  • The model changed, or a skill was added. Should restart the harness and keep the transcript. Modern harnesses handle a mid-conversation model change themselves, and several have prompt caching that a needless replay throws away.
  • The conversation history diverged. This is the only case that genuinely needs a cold replay, because the parked session's state no longer corresponds to what the client believes happened.

The single signal collapses all three into the most expensive answer.

Proposed direction

Replace the one boolean mismatch with named change classes, each declaring what it actually invalidates. Roughly:

Change Consequence
Credential value Re-bind credentials. Daytona: update the Secret record in place. Local: new process, same transcript.
Model, skills, tools, prompt Restart the harness, keep the transcript.
History divergence, expired mount lease Cold replay. Unchanged.

The dispatch in server.ts then picks the cheapest consequence that covers every change it sees, instead of defaulting to the most expensive one.

Scope

Two pieces, and the first is useful on its own:

  1. Daytona in-place Secret update. Split daytonaCreateFingerprint (provider.ts) into a shape part and a value part, and add an update path to DaytonaSecretApi so a value-only change updates the record rather than tearing the sandbox down. Only affects the flag-on path.
  2. The change classes. Touches the warm pool for every kind of change and needs its own careful review.

Blocked on

Piece 1 cannot be verified against a real account until the runner's Daytona API key has permission to manage Secrets. See the ops note in #5670.

Origin

Raised by @MahmoudMabrouk in review of #5670: #5670 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions