Summary
When a PR modifies physics or synthesis logic, reviewers currently have no automatic signal about whether the benchmark numbers changed. Add a CI step that runs the three benchmark scenarios, diffs the key metrics against the base branch, and posts the delta as a PR comment.
Goal
Make numeric regressions visible in the PR review interface without requiring reviewers to check out and run examples locally.
Acceptance criteria
## Benchmark Delta (PR vs base)
| Scenario | Metric | Base | PR | Δ |
|-------------------|---------------------|--------|--------|--------|
| H100 parallel | Tj_rec (°C) | 67.4 | 67.4 | 0.0 |
| H100 series | Tj_hottest (°C) | 73.6 | 73.6 | 0.0 |
| B200-proxy | min_lpm | 6.2 | 6.2 | 0.0 |
Notes
- Only fires on PRs, not on pushes to main
- Should use a bot token or GITHUB_TOKEN for posting comments
- Related to snapshot tests issue — both are regression-detection mechanisms, but this one surfaces the delta in the PR view without requiring a snapshot approval step
Context
Relevant files: .github/workflows/, examples/decision_memo_examples.py
Summary
When a PR modifies physics or synthesis logic, reviewers currently have no automatic signal about whether the benchmark numbers changed. Add a CI step that runs the three benchmark scenarios, diffs the key metrics against the base branch, and posts the delta as a PR comment.
Goal
Make numeric regressions visible in the PR review interface without requiring reviewers to check out and run examples locally.
Acceptance criteria
python examples/decision_memo_examples.pyand extracts key metrics (Tj at recommended flow, risk level, min/recommended LPM, margin remaining)Notes
Context
Relevant files:
.github/workflows/,examples/decision_memo_examples.py