[#5540] fix(api): price cached input tokens, and stop losing costs to a missing model

[mmabrouk]

Closes #5540.

Stacked on #5709. Set the base to that branch, so this diff shows only its own change.

The symptom

Three separate defects in how Agenta estimates the cost of a model call.

Cached input is charged at the full uncached rate. One measured span with 1 uncached prompt token, 25,182 cache-read tokens and 20 completion tokens was priced at $0.000303 against a harness-reported $0.0082. That is a 27x undercount, and it grows with every warm turn.

Estimation is dead for most agent spans. The model lookup checks the response model and a legacy parameters path. The agent runner sets the response model only for codex. Measured: 0 of 8,365 chat spans without a response model were priced, and 858 of 858 with one were. A perfect discriminator.

A reported cost of zero is treated as missing. The guard that stops the roll-up overwriting a producer-reported cost tested for a non-zero value, and the reader it used defaults every field to 0.0. So a genuine measured zero, such as a fully cached turn or a free model, was indistinguishable from an absent attribute and got replaced by an estimate.

The fix, and why the token math looks the way it does

Litellm derives ordinary input by subtracting the cache details from whatever prompt count you hand it. So it wants the inclusive figure, not the uncached remainder. Passing the uncached count plus the cache arguments silently drops the ordinary tokens.

Reproduced against the pinned litellm 1.92.0, with 93 uncached and 13,463 cache-read tokens:

Shape passed to litellm	gpt-5.3-codex	claude-sonnet-4-6
93, no cache arguments	$0.00016275	$0.000279
93, plus cache arguments	$0.002356025	$0.0040389
13,556 inclusive, plus cache arguments	$0.002518775	$0.0043179

The middle row loses the 93 ordinary tokens. Only the last row prices both kinds of input, and it holds on both the OpenAI and the Anthropic pricing routes.

One assumption, written into the code. Today's prompt bucket holds raw exclusive input, which is why summing the cache buckets into it is correct. A planned canonical change would make input inclusive at the boundary, at which point the same sum would double count. The comment at the summation names that condition, and the arithmetic sits in one place so the future migration has one thing to change. No heuristic tries to detect which convention is in force.

Model lookup now tries the response model, then ag.meta.request.model, then the legacy parameters path.

The guard now tests whether the cumulative total attribute is present, not whether it is non-zero.

Verification

• 1,631 passing in OSS unit tests, 1,890 in EE.
• 23 new tests: the pricing oracle parametrized over four cache shapes and two providers, the [tracing] Codex/Claude run cost ignores cached input tokens (understated ~9x on warm turns) #5540 regression itself, the case that guards the middle row of the table above, model lookup precedence, containment of an unpriceable alias and a missing model, and the reported-zero guard driven through the real ingest pipeline.
• Mutation checked. The new tests were run against the previous version of the module, loaded from HEAD under the production module name: 9 fail. They pin the fix rather than the framework. The one cache-creation case that passes on the old code does so legitimately, because OpenAI has no separate cache-write price and both shapes agree there.

Notes for the reviewer

• New noise, by design. Falling back to the request model starts feeding unpriceable aliases into litellm, such as the bare string "sonnet" that the model picker sends. Verified that the failure stays contained: litellm.BadRequestError is caught by the existing handler, one warning is logged, no cost is written, no exception escapes into ingest. Litellm also prints its own provider banner to stderr on each such failure, which is new noise at comparable volume.
• The estimate is still an estimate. Anthropic prices five-minute and one-hour cache writes differently. Litellm maps cache_creation_input_tokens to a single write rate and the runner emits a single cache-write number, so long-lived cache writes are still undercounted.
• Two pre-existing tests needed fixture updates, not behavior changes. One helper was unpriceable only because of the model-lookup defect, so its premise disappeared. Another monkeypatched cost_per_token with a fixed three-argument signature that rejected the new cache keywords.

Follow-ups this does not do

• The roll-up has the same missing-versus-zero conflation on its own write path: a parent whose children genuinely cost zero still gets no cumulative attribute rather than an explicit zero. The same shape exists for tokens and errors.
• calculate_costs assumes several attribute containers are dictionaries. A scalar there raises outside the try and would fail the whole ingest batch. Pre-existing.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agenta-documentation	Ready	Preview	Aug 3, 2026 10:20pm

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 962a7d2c-5018-4b63-a996-b3a072bced9b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Railway Preview Environment

Preview URL	https://gateway-pr-5714.up.railway.app/w
Project	agenta-oss-clone-spike
Image tag	pr-5714-c1bf6d4
Status	Deployed
Railway logs	Open logs
Workflow logs	View workflow run
Updated at 2026-08-03T22:21:58.704Z