Skip to content

ci(workflows): add dry_run input to benchmark workflows - #237

Merged
kisernl merged 1 commit into
masterfrom
add-dry-runs-to-workflows
Jul 28, 2026
Merged

ci(workflows): add dry_run input to benchmark workflows#237
kisernl merged 1 commit into
masterfrom
add-dry-runs-to-workflows

Conversation

@kisernl

@kisernl kisernl commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Mirror the existing dry_run boolean from sandbox-tti-benchmarks and sandbox-dax-benchmarks to the remaining benchmark workflows (ai-gateway, browser, browser-throughput, snapshot-fork, storage). When set on a workflow_dispatch run, the ingest-to-platform and commit-and-push steps are skipped so the workflow can be invoked manually without ingesting data or writing back to master. The gate is benign for push, schedule, and pull_request events (inputs.dry_run is null).

Also drop migrate-secrets-to-org.yml, the one-time org-secret migration helper that has already completed.


Open in Devin Review

Mirror the existing dry_run boolean from sandbox-tti-benchmarks and
sandbox-dax-benchmarks to the remaining benchmark workflows (ai-gateway,
browser, browser-throughput, snapshot-fork, storage). When set on a
workflow_dispatch run, the ingest-to-platform and commit-and-push
steps are skipped so the workflow can be invoked manually without
ingesting data or writing back to master. The gate is benign for
push, schedule, and pull_request events (inputs.dry_run is null).

Also drop migrate-secrets-to-org.yml, the one-time org-secret migration
helper that has already completed.
@open-cla

open-cla Bot commented Jul 28, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@github-actions

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 Anthropic (direct) 91.7 619ms 626ms 94.1 4/4
2 Cloudflare AI Gateway 90.8 789ms 662ms 88.6 4/4
3 LLM Gateway 89.9 1181ms 692ms 91.1 4/4
4 OpenRouter 89.6 1149ms 1036ms 99.3 4/4
5 Pydantic AI Gateway 88.2 1557ms 640ms 81.0 4/4
6 Vercel AI Gateway 87.6 1465ms 1097ms 80.6 4/4

View full run · SVG available as a build artifact

@kisernl
kisernl merged commit 8384eae into master Jul 28, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant