Symptom
Run cost for ACP-path harnesses (Codex, Claude Code) ignores prompt-cache token detail. On a real warm Codex turn (93 non-cached + 13,463 cached input tokens), the platform computes $0.000855 while the true provider cost is ~$0.0076: the cached tokens are dropped entirely, so cost is understated roughly 9x whenever the cache is warm. (Naively summing them without cache-aware pricing would instead overstate to ~$0.068.)
Root cause
- The ACP bridge already delivers full detail:
PromptResponse.usage carries inputTokens, outputTokens, cachedReadTokens, cachedWriteTokens, thoughtTokens (disjoint buckets: input excludes cached).
- The runner's
mergePromptAndStreamUsage (services/runner/src/engines/sandbox_agent/usage.ts) keeps only input/output, so codex and claude spans never record cache tokens. The Pi in-process path already emits gen_ai.usage.cache_read.input_tokens / gen_ai.usage.cache_creation.input_tokens (services/runner/src/tracing/otel.ts), and the API's ingestion already maps those to ag.metrics.
calculate_costs (api/oss/src/core/tracing/utils/trees.py) calls litellm.cost_per_token without the cache kwargs. litellm 1.92.0 (the shipped version) supports cache_read_input_tokens / cache_creation_input_tokens and its codex model entries carry cache_read_input_token_cost (90% discount) — we just never pass them. Note litellm expects prompt_tokens to INCLUDE cached tokens (it subtracts internally), so the disjoint ACP buckets must be summed first.
Fix (small, no new dependency)
- Runner: extend
AgentUsage and stamp the cache attributes on ACP-path spans with the same gen_ai.usage.* names the Pi path uses (~15 lines; fixes Claude too).
- API: pass the cache kwargs to
litellm.cost_per_token in calculate_costs, summing buckets to litellm's inclusive convention (~10 lines).
Verify while implementing: whether ACP outputTokens includes thoughtTokens (all captured transcripts show 0 thought tokens), and claude-code-acp's bucket convention (inclusive vs disjoint) before reusing the summing logic.
Context
Surveyed alternatives (codex [otel] telemetry, OpenLLMetry, OpenInference, Langfuse): none instrument the codex CLI binary, and the ones that price do it server-side from token buckets exactly like us, so fixing our own pipeline is the smallest correct path. Full research doc from the Codex-harness build (PR #5509 review follow-up).
https://claude.ai/code/session_01TNqjpdGV3SBZUazJj7AgA9
Symptom
Run cost for ACP-path harnesses (Codex, Claude Code) ignores prompt-cache token detail. On a real warm Codex turn (93 non-cached + 13,463 cached input tokens), the platform computes $0.000855 while the true provider cost is ~$0.0076: the cached tokens are dropped entirely, so cost is understated roughly 9x whenever the cache is warm. (Naively summing them without cache-aware pricing would instead overstate to ~$0.068.)
Root cause
PromptResponse.usagecarriesinputTokens,outputTokens,cachedReadTokens,cachedWriteTokens,thoughtTokens(disjoint buckets: input excludes cached).mergePromptAndStreamUsage(services/runner/src/engines/sandbox_agent/usage.ts) keeps only input/output, so codex and claude spans never record cache tokens. The Pi in-process path already emitsgen_ai.usage.cache_read.input_tokens/gen_ai.usage.cache_creation.input_tokens(services/runner/src/tracing/otel.ts), and the API's ingestion already maps those toag.metrics.calculate_costs(api/oss/src/core/tracing/utils/trees.py) callslitellm.cost_per_tokenwithout the cache kwargs. litellm 1.92.0 (the shipped version) supportscache_read_input_tokens/cache_creation_input_tokensand its codex model entries carrycache_read_input_token_cost(90% discount) — we just never pass them. Note litellm expectsprompt_tokensto INCLUDE cached tokens (it subtracts internally), so the disjoint ACP buckets must be summed first.Fix (small, no new dependency)
AgentUsageand stamp the cache attributes on ACP-path spans with the samegen_ai.usage.*names the Pi path uses (~15 lines; fixes Claude too).litellm.cost_per_tokenincalculate_costs, summing buckets to litellm's inclusive convention (~10 lines).Verify while implementing: whether ACP
outputTokensincludesthoughtTokens(all captured transcripts show 0 thought tokens), and claude-code-acp's bucket convention (inclusive vs disjoint) before reusing the summing logic.Context
Surveyed alternatives (codex
[otel]telemetry, OpenLLMetry, OpenInference, Langfuse): none instrument the codex CLI binary, and the ones that price do it server-side from token buckets exactly like us, so fixing our own pipeline is the smallest correct path. Full research doc from the Codex-harness build (PR #5509 review follow-up).https://claude.ai/code/session_01TNqjpdGV3SBZUazJj7AgA9