Skip to content

ci(integration-test): promote qp-topic into the required suite - #1235

Merged
modelmirror merged 1 commit into
stagingfrom
ci/qp-topic-required-suite
Aug 15, 2026
Merged

ci(integration-test): promote qp-topic into the required suite#1235
modelmirror merged 1 commit into
stagingfrom
ci/qp-topic-required-suite

Conversation

@modelmirror

Copy link
Copy Markdown
Collaborator

Maintainer-gated surfaces (promotion gate class) — ready for maintainer review and merge; not self-merged by design.
This PR touches scripts/promotion-gate.sh and .github/workflows/, both of which AGENTS.md holds for the maintainer even into staging.

Artifact served

The promotion gate's coverage of the qp-topic labeling path. qp-topic exercises the shared qp-topic-measure composite — the post-label half of the paid labeling run: the no-output guard, the fedcourts qp-topics publication gate, and the publish-and-validate path. Until now that path could regress and still promote. The scenario spends no tokens and binds no credentials, so requiring it costs a promotion nothing.

The scenario has a string of green live runs behind it — a standalone qp-topic run and several whole-suite scenario=all runs — which is the evidence its introduction deliberately deferred the gate promotion for.

What changed

Four synchronized sites, each pinned by an existing workflow-shape test (tests/test_workflow_promote.py fails on any partial update):

  1. scripts/promotion-gate.shqp-topic added to REQUIRED_SCENARIOS (8 → 9 entries).
  2. .github/workflows/integration-test.yml{"scenario": "qp-topic", "engine": "claude-code"} added to the all matrix (7 → 8 legs; collect still rides the run as its own environment-free job, not a leg). The scenario dispatch input already listed qp-topic.
  3. .github/workflows/promote.ymlqp-topic added to the per-scenario dispatch commands the freshness failure prints.
  4. docs/testing.md, docs/pipeline.md — the "outside the required suite" qualifiers dropped and the counts corrected.

New true counts: 9 required runs = 8 all matrix legs + the collect job, with engine-smoke counted once per each of three engines.

Freshness matches the new leg without further change: for an engine-less entry the gate builds ^integration-test: qp-topic / and requires the @ staging$ suffix, which the leg's title integration-test: qp-topic / claude-code @ staging satisfies.

Pre-existing drift retired in the same paragraphs

  • docs/testing.md still said "The collect scenario is outside the gate" and gave the required-run count as "seven … five of the six real scenarios". Both were already stale against the pre-diff 8-entry REQUIRED_SCENARIOScollect joined the suite earlier. The sentence is dropped (its still-true half, that collect binds no environment and so no deployment-branch policy can refuse it, already lives in the scenario inventory above it) and the count rewritten.
  • The qp-topic leg's own rationale comment in integration-test.yml argued from its non-required status — a premise this commit deletes. The justification inverts rather than merely losing a word: the environment binding the leg never reads is exactly what makes its run title's @ staging suffix a deployment-gated provenance claim, which is how freshness reads every required matrix leg. A collect-style environment-free job would have weakened the evidence.

Checks run

Python surface untouched (docs, workflows, one shell line), so the gate stages were scoped accordingly:

  • uv run pytest tests/test_workflow_promote.py tests/test_workflow_collect.py tests/test_workflow_cell_invariants.py — 50 passed
  • uvx zizmor@1.26.1 --persona=regular .github/workflows .github/actions — no findings
  • pinned actionlint 1.7.12 with shellcheck -e SC2016 — clean
  • shellcheck -e SC2016 scripts/promotion-gate.sh — clean

Review

workflow-reviewer and docs-reviewer both ran on the diff. No blockers from either. Both raised the same recommended finding — the now-contradictory qp-topic rationale comment — which is fixed above using workflow-reviewer's stronger framing. docs-reviewer's nits (the "every scenario" phrasing, which would have read as including the all option, and ragged reflow at the edit seams) and workflow-reviewer's nit on the environment parenthetical in docs/pipeline.md are all applied. workflow-reviewer independently confirmed against gh run list that a qp-topic / claude-code @ staging run has concluded success from the staging branch, and that the last green all predating this change cannot leak into the gate — freshness pins on head_sha, so a run at the promoted head executes the 8-leg matrix at that head.

Closes #1210 — note this does not fire from a staging merge; the orchestrator closes the issue by hand once this lands.

The qp-topic scenario drives the shared qp-topic-measure composite — the
post-label half of the paid labeling run — over canned inputs, token-free
and credential-free, so requiring it costs a promotion nothing and closes
the gate's blind spot on the labeling path.

Four synchronized sites move together (a workflow-shape test pins each
coupling): REQUIRED_SCENARIOS in the gate script, the `all` matrix in
integration-test.yml, the dispatch-commands help list promote.yml prints on
a freshness failure, and the docs. The required set goes 8 -> 9; the `all`
matrix goes 7 -> 8 legs, with collect still riding the run as its own
environment-free job.

Also retires drift the change makes visible: docs/testing.md still called
collect outside the gate and gave a required-run count of seven, both stale
since collect joined the set, and the qp-topic leg's own rationale comment
argued from its non-required status. The comment's justification inverts —
the environment binding the leg never reads is exactly what makes its run
title's `@ staging` suffix a deployment-gated provenance claim, which is how
freshness reads every required matrix leg.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@modelmirror
modelmirror merged commit ba5e47b into staging Aug 15, 2026
7 checks passed
@modelmirror
modelmirror deleted the ci/qp-topic-required-suite branch August 15, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant