feat(storage): report download throughput as per-step data via ctx.measure

[devin-ai-integration]

Summary

Makes per-step data load-bearing: every benchmark that computes a genuine non-latency metric (a rate/count the framework can't infer from step timing) now attaches it to the step that produced it, so it lands in benchmark_task_steps.step_data_json for the platform to read (see benchmarks-platform#49). All other benchmarks are latency-only and unchanged — their per-step signal is already step_latency_ms.

storage — throughputMbps is a download rate, not a duration. Compute it inside the download step and ctx.measure it there:

await step('download', async () => {
  await withTimeout(storage.download(key, { as: 'bytes' }), timeout, 'Download timed out');
  throughputMbps = (fileSizeBytes * 8) / (downloadMs / 1000) / 1_000_000;
  measure({ throughputMbps });          // → download step's step_data_json
});
return { data: { uploadMs, downloadMs, throughputMbps, fileSizeBytes } };  // task-level return unchanged (legacy results)

browser-throughput — the session lifecycle is now recorded via real ctx.step (not fabricated step records). runThroughputIteration wraps each real await phase in a step and reports throughput with ctx.measure inside the actions step, so the runner records true timing + real success/error per phase:

await step('create',  () => provider.session.create(...));
const page = await step('connect', async () => { /* connectOverCDP + get page */ });
await step('actions', async () => {
  await runActionLoop(page, actions, navigateUrl);
  measure({ actionsPerSecond, actionsCompleted });   // → actions step's step_data_json
});
await step('release', () => provider.session.destroy(...), { reportConcurrency: false });
// task returns { data } only — no hand-built steps[]

(Round mode's runTaskRecord fully supports ctx.step/ctx.measure, merging framework steps with any result.steps.)

ai-gateway — token throughput/count are non-latency, but dns/tcp/tls/ttfb/ttft are sub-timings of a single streamed HTTP request captured via socket hooks — there's no separate awaitable to wrap, so the pre-measured TaskResult.steps pattern is the correct escape hatch here. Attach via step.data (the analog of ctx.measure):

steps.push({ name: 'ttft', status, latencyMs: result.ttftMs,
             data: { outputTokens, outputTokensPerSec } });

Rule of thumb: if a phase is its own await, use ctx.step; only hand-build steps[] when the sub-timings live inside one indivisible operation (ai-gateway network phases, dax build phases parsed from one script).

Why not the others

browser (create/connect/navigate/release), dax (build phases), sandbox TTI (ttiMs is a create→exec duration, correctly task-level), snapshot-fork (all *Ms phase latencies) produce only durations — already captured by step_latency_ms. Forcing measure there would just duplicate latency.

Testing

• pnpm typecheck — clean
• no-cred bench run of storage exits 0 (providers skipped without creds)

Link to Devin session: https://app.devin.ai/sessions/3dbf4532c9e94ab6811ff85b49f1a203
Requested by: @HeyGarrison

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[open-cla]

Contributor License Agreement

All contributors are covered by a CLA.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

[github-actions]

Storage Benchmark Results

1MB Files

#	Provider	Score	Download	Throughput	Upload	Status
1	Azure Blob Storage	96.7	0.02s	365.9 Mbps	0.06s	10/10
2	AWS S3	94.9	0.10s	84.7 Mbps	0.21s	10/10
3	Vercel Blob	94.9	0.09s	97.5 Mbps	0.22s	10/10
4	Tensorlake	94.6	0.05s	168.8 Mbps	0.63s	10/10
5	Tigris	94.6	0.06s	149.3 Mbps	0.69s	10/10
6	Cloudflare R2	93.4	0.14s	60.8 Mbps	0.37s	10/10
7	Google Cloud Storage	0.0	0.00s	0.0 Mbps	0.00s	0/10

---

View full run · SVGs available as build artifacts

[github-actions]

Snapshot/Fork Benchmark Results

small dataset

#	Provider	Score	Snapshot create	Fork (snapshot)	Fork (live)	First read	Status
1	Tensorlake	99.7	0.04s	0.25s	0.30s	0.18s	1/1
2	Tigris	99.2	0.33s	0.50s	0.87s	0.29s	1/1
3	Azure-blob	98.3	1.12s	0.78s	1.77s	0.04s	1/1
4	Cloudflare R2	91.8	4.94s	4.50s	8.81s	0.46s	1/1
5	AWS S3	0.0	0.00s	0.00s	0.00s	0.00s	0/1

---

View full run

[github-actions]

Browser Benchmark Results

#	Provider	Score	Create	Connect	Navigate	Release	Total	Status
1	Tilion	98.0	0.02s	0.04s	0.12s	0.01s	0.20s	10/10
2	Kernel	97.3	0.01s	0.08s	0.11s	0.04s	0.27s	10/10
3	Browserbase	94.4	0.23s	0.11s	0.10s	0.14s	0.59s	10/10
4	Browseruse	92.5	0.25s	0.12s	0.07s	0.04s	0.50s	10/10
5	Hyperbrowser	86.4	0.33s	0.56s	0.74s	0.10s	1.77s	10/10
6	Steel	81.5	0.52s	0.55s	0.12s	0.39s	1.80s	10/10
7	Notte	60.3	0.60s	2.49s	0.59s	0.49s	4.67s	10/10

---

View full run · SVG available as build artifact

[github-actions]

Browser Throughput Benchmark Results

#	Provider	Score	APS (med)	Task (med)	Task (p95)	Screenshot	Status
1	Kernel	79.3	5.53/s	1.81s	1.85s	254ms	3/3
2	Tilion	73.3	4.26/s	2.35s	2.46s	270ms	3/3
3	Browseruse	69.3	3.45/s	2.90s	2.95s	263ms	3/3
4	Browserbase	66.8	2.99/s	3.34s	3.34s	328ms	3/3
5	Steel	53.5	1.33/s	7.50s	7.81s	690ms	3/3
6	Hyperbrowser	50.7	1.32/s	7.58s	11.61s	989ms	3/3
7	Notte	12.7	0.22/s	44.61s	50.36s	6361ms	3/3

---

View full run · SVG available as build artifact

[github-actions]

AI Gateway Benchmark Results

#	Gateway	Score	Cold E2E	Warm TTFT	Tok/sec	Status
1	LLM Gateway	90.1	747ms	622ms	90.3	20/20
2	Cloudflare AI Gateway	89.9	850ms	612ms	95.3	20/20
3	OpenRouter	89.8	1002ms	703ms	95.8	20/20
4	Pydantic AI Gateway	89.8	729ms	848ms	86.1	20/20
5	Vercel AI Gateway	89.4	877ms	690ms	86.3	20/20
6	Anthropic (direct)	87.7	787ms	644ms	83.5	20/20
7	Concentrate AI	84.5	1280ms	861ms	0.0	20/20

---

View full run · SVG available as a build artifact