Skip to content

Add a robust 'wait for CI fully green' tool for agents #31

Description

@jorisjonkers-dev-agents

Problem

Agents repeatedly hand-roll gh run list/view bash poll loops to wait for a PR/branch's CI to finish. These are fragile and waste turns/tokens:

  • []|all on an empty job list returns true → the loop exits before jobs even register (false "green").
  • Multiple concurrent workflows per push (Validate, CodeQL, Dependency review) — a single-workflow poll misses required checks.
  • State churn queued → in_progress → completed and re-queues mid-run confuse naive status==completed checks.
  • Long matrix jobs (e.g. 6 system-test shards) blow past a foreground command timeout, so the poll dies with no verdict.
  • Flaky jobs (Playwright 30s timeouts) need a bounded auto-rerun-and-recheck, currently done by hand.
  • Distinguishing "fully green" vs "some required check failed" vs "still running" is reimplemented every time, inconsistently.

Ask

A first-class tool (council CLI subcommand and/or an MCP tool) — e.g. council ci-wait / wait_for_ci — that, given a --branch/--pr/--sha (+ --repo):

  1. Discovers all check runs + workflow runs for the head commit (not one workflow), waits until every required check reaches a terminal state.
  2. Returns a single structured verdict: green | failed | timed_out, with the list of failing jobs and a short failing-log excerpt for each.
  3. Handles the empty/registering window correctly (don't conclude before checks appear; wait for the checks to be created).
  4. Bounded, cache-aware polling cadence (respect the 5-min prompt-cache window; long backoff for slow matrices) with an overall deadline.
  5. Optional --rerun-flaky N: re-run failed jobs up to N times and re-evaluate, reporting which passed only on retry (so genuine failures aren't masked but flakes don't block).
  6. Optional --required <checks> allowlist so non-blocking checks (e.g. CodeQL) don't gate the verdict.

Why

This is a universal need across every PR an agent opens; a correct shared implementation removes a recurring class of "false green / silent hang / timed-out poll" bugs and saves significant agent time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions