Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions benchmarks/ai-gateway/ai-gateway.bench.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
/**
* AI Gateway benchmark, built on @benchsdk/runner's runBenchmark with
* `groupBy: 'round'`. Fairness is the whole point (see AI_GATEWAYS.md): every
* gateway's Nth iteration must run at roughly the same point in time as every
* other gateway's Nth iteration, so no gateway is favored by running during a
* different network condition. `groupBy: 'round'` provides exactly that — one
* task per gateway per round, taking turns — replacing the bespoke round-robin
* loop this file used to hand-roll.
* AI Gateway benchmark. Declarative — exports `config` + `task`; `bench run`
* owns the entrypoint. Configured with `groupBy: 'round'`. Fairness is the
* whole point (see AI_GATEWAYS.md): every gateway's Nth iteration must run at
* roughly the same point in time as every other gateway's Nth iteration, so no
* gateway is favored by running during a different network condition.
* `groupBy: 'round'` provides exactly that — one task per gateway per round,
* taking turns — replacing the bespoke round-robin loop this file used to
* hand-roll.
*
* Declared as two phases (cold, warm); the framework owns the phase boundary
* and exposes it via `ctx.phase`, so the task branches on phase identity, not
Expand Down
3 changes: 1 addition & 2 deletions benchmarks/browser/browser-task.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
* Shared browser-lifecycle workload for the browser benchmark. One iteration =
* create a session, connect over CDP, navigate to example.com, then release
* (always, even on failure). Orchestration (how many, how parallel) is owned by
* @benchsdk/runner's runBenchmark — this file only describes what one iteration
* does.
* @benchsdk/runner — this file only describes what one iteration does.
*/
import { chromium } from 'playwright-core';
import type { BenchmarkTask, TaskContext, TaskResult } from '@benchsdk/runner';
Expand Down
11 changes: 6 additions & 5 deletions benchmarks/sandbox/dax.bench.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
/**
* Dax benchmark: runs the OpenCode build (scripts/dax-benchmark.sh) inside a
* freshly created sandbox once per iteration and measures its build phases
* (prepare / bun-download / bun-unpack / clone / install / typecheck). Config
* lives here; platform orchestration is owned by @benchsdk/runner's runBenchmark.
* The phase-parsing + resource-sizing logic is reused from ./dax.ts so the
* legacy `results/sandbox-dax/` JSON shape is preserved verbatim via the
* legacy-results adapter (TEMPORARY local-JSON bridge — see legacy-results).
* (prepare / bun-download / bun-unpack / clone / install / typecheck).
* Declarative — exports `config` + `task`; `bench run` owns the entrypoint and
* platform orchestration. The phase-parsing + resource-sizing logic is reused
* from ./dax.ts so the legacy `results/sandbox-dax/` JSON shape is preserved
* verbatim via the legacy-results adapter (TEMPORARY local-JSON bridge — see
* legacy-results).
*
* `groupBy: 'round'` is used so the task owns its own measured `latencyMs`
* (the build's totalMs) and can attach pre-measured phase steps; the local
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/storage/snapshot-fork-task.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
* seed dataset -> snapshot -> fork(from snapshot) -> fork(from live) ->
* read-back-from-fork (verify) -> teardown.
* Every created resource is torn down in a `finally` so a mid-iteration failure
* does not leak real storage. Orchestration is owned by @benchsdk/runner's
* runBenchmark — this file only describes what one iteration does.
* does not leak real storage. Orchestration is owned by @benchsdk/runner
* this file only describes what one iteration does.
*/
import crypto from 'node:crypto';
import type { Storage } from '@storagesdk/core';
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/storage/snapshot-fork.bench.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/**
* Storage snapshot/fork benchmark: per-iteration seed -> snapshot -> fork ->
* verify, per provider (concurrency 1 = sequential; each iteration creates real
* snapshots/forks). Config lives here; orchestration is owned by @benchsdk/runner's
* runBenchmark.
* snapshots/forks). Declarative — exports `config` + `task`; `bench run` owns
* the entrypoint and orchestration.
*
* bench run benchmarks/storage/snapshot-fork.bench.ts
* bench run benchmarks/storage/snapshot-fork.bench.ts --dataset wide --iterations 5 --provider tigris
Expand Down
3 changes: 1 addition & 2 deletions benchmarks/storage/storage-task.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
* Shared storage upload/download workload for the storage benchmark. One
* iteration = upload a buffer, download it, compute throughput, then delete
* (best-effort cleanup). Orchestration (how many, how parallel) is owned by
* @benchsdk/runner's runBenchmark — this file only describes what one iteration
* does.
* @benchsdk/runner — this file only describes what one iteration does.
*/
import type { BenchmarkTask, TaskContext, TaskResult } from '@benchsdk/runner';
import { TaskError } from '@benchsdk/runner';
Expand Down
Loading