Skip to content

docs(pr): do not enqueue into a busy runner pool - #128

Merged
CybotTM merged 3 commits into
mainfrom
docs/queue-only-when-pool-quiet
Aug 3, 2026
Merged

docs(pr): do not enqueue into a busy runner pool#128
CybotTM merged 3 commits into
mainfrom
docs/queue-only-when-pool-quiet

Conversation

@CybotTM

@CybotTM CybotTM commented Aug 3, 2026

Copy link
Copy Markdown
Member

The arming gate lists three conditions, all about review state. A fourth decides whether the entry survives at all.

check_response_timeout_minutes (default 5) starts at enqueue and covers the wait for a runner, not just the run. With the pool saturated the required jobs sit in queued with no runner assigned, and the queue discards the entry having executed nothing. The PR looks fine throughout — CLEAN, approved, no failing check — which is what makes it hard to read.

Retrying amplifies the cause. Each attempt spawns a full set of runs on a gh-readonly-queue/* branch, and a dropped entry does not cancel them, so every retry leaves more runs holding slots and slows the next attempt.

Measured in netresearch/ofelia:

Four-attempt retry loop on a loaded pool dropped every time, 18 unfinished runs piled up
Same required checks in PR context concluded in 2.0 minutes — well inside the 5-minute window
Three later PRs, enqueued only at ≤2 unfinished runs merged on the first attempt each

So the timeout was not too short for the work; the entry never got to do the work. This adds the gate, the one call that reads the timeout when an entry drops on an already-quiet pool (then it genuinely is the setting), and the cleanup for one's own orphaned queue runs — scoped to your own PR, since other PRs thrash the same way and their runs are not yours to cancel.

The arming gate listed three conditions, all about review state. A fourth
decides whether the entry survives at all: check_response_timeout_minutes
starts at enqueue and covers the wait for a runner, not just the run. With
the pool saturated the required jobs sit in `queued`, never start, and the
queue discards the entry having executed nothing.

Retrying amplifies it. Each attempt spawns a full set of runs on a
gh-readonly-queue branch and a dropped entry does not cancel them, so every
retry leaves more runs holding slots. In netresearch/ofelia that piled up 18
unfinished runs and made a four-attempt loop fail by construction, while the
same required checks concluded in 2.0 minutes in PR context.

Adds the gate (at most ~2 unfinished runs), the one call that reads the
timeout when an entry drops on a quiet pool, and the cleanup for one's own
orphaned queue runs. Gating this way merged three consecutive PRs on the
first attempt.

Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>
Copilot AI review requested due to automatic review settings August 3, 2026 11:08
@github-actions github-actions Bot added documentation Improvements or additions to documentation skill labels Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Git workflow “Auto-Merge / Merge-Queue Arming Gate” documentation to address a failure mode where merge-queue entries are dropped due to runner-pool saturation (timeout starts at enqueue and includes waiting for a runner), and to provide operational guidance to avoid amplifying the problem with retries.

Changes:

  • Adds a new arming-gate condition to only enqueue when the runner pool is quiet.
  • Documents how check_response_timeout_minutes affects merge-queue drops and how to inspect the ruleset parameters.
  • Adds CLI snippets for estimating runner-pool load and canceling orphaned merge-queue runs for the current PR.
Suppressed comments (1)

skills/git-workflow/references/pull-request-workflow.md:1263

  • gh run list only returns the most recent runs up to --limit. With a busy repo, older queued/in_progress runs can be pushed outside the limit by newer completed runs, causing this count to under-report unfinished work and potentially enqueue into a still-busy pool.
gh run list --repo "$R" --limit 25 --json status \

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread skills/git-workflow/references/pull-request-workflow.md
The arming gate gained a fourth condition without its lead-in being
updated, so the section told the reader to check three and then listed
four.

Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (2)

skills/git-workflow/references/pull-request-workflow.md:1264

  • The "quiet runner pool" gate counts non-completed runs from only the most recent 25 workflow runs. Because gh run list is recency-ordered, older still-queued/in-progress runs can fall outside that window, producing an undercount and potentially a false "quiet" signal. Use a larger --limit (or otherwise ensure the query covers all active runs) so the gate is reliable on busy repos.
gh run list --repo "$R" --limit 25 --json status \
  --jq '[.[]|select(.status!="completed")]|length'

skills/git-workflow/references/pull-request-workflow.md:1283

  • This snippet uses xargs -r, which is a GNU extension and will fail on macOS/BSD xargs. Prefer a portable loop so the "cancel orphaned runs" step works across common dev environments.
gh run list --repo "$R" --limit 40 --json databaseId,status,headBranch \
  --jq '.[]|select(.status!="completed")
        |select(.headBranch|startswith("gh-readonly-queue/main/pr-'"$PR"'-"))|.databaseId' \
  | xargs -r -n1 gh run cancel --repo "$R"

Quote $R in the ruleset call, matching its neighbours, and move the
cancel filter to jq --arg. The previous form closed the single-quoted jq
expression mid-string to splice $PR in, which works but is easy to
mistranscribe -- and these snippets exist to be copied.

Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>
@CybotTM
CybotTM requested a review from Copilot August 3, 2026 11:46
@sonarqubecloud

sonarqubecloud Bot commented Aug 3, 2026

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (2)

skills/git-workflow/references/pull-request-workflow.md:1264

  • This gating command only inspects the most recent 25 runs, which can undercount unfinished runs in a backlogged repo and lead to a false “quiet pool” signal. It also relies on $R being set elsewhere, so it isn’t copy/paste-ready as written.
gh run list --repo "$R" --limit 25 --json status \
  --jq '[.[]|select(.status!="completed")]|length'

skills/git-workflow/references/pull-request-workflow.md:1284

  • xargs -r is a GNU extension and fails on macOS/BSD. Using a while read loop keeps the snippet portable while still skipping the no-IDs case.
  | xargs -r -n1 gh run cancel --repo "$R"

@CybotTM
CybotTM merged commit 307cb5a into main Aug 3, 2026
22 checks passed
@CybotTM
CybotTM deleted the docs/queue-only-when-pool-quiet branch August 3, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants