Skip to content

fix(pr-merge): verify the merge outcome before reporting success - #160

Merged
CybotTM merged 1 commit into
mainfrom
fix/pr-merge-verify-enqueue
Aug 10, 2026
Merged

fix(pr-merge): verify the merge outcome before reporting success#160
CybotTM merged 1 commit into
mainfrom
fix/pr-merge-verify-enqueue

Conversation

@CybotTM

@CybotTM CybotTM commented Aug 10, 2026

Copy link
Copy Markdown
Member

The defect

gh pr merge <n> --repo <r> --merge on a repository with a merge queue prints its usual success line and exits 0 while adding nothing to the queue, when an auto-merge request is already attached to the PR — the pending request swallows the enqueue. pr-merge.sh relayed that exit status as queued (--merge, strategy set by the queue).

Observed twice on netresearch/t3x-nr-llm. Measured on PR #559: immediately after the call isInMergeQueue was false and the merge queue was empty; after gh pr merge 559 --disable-auto the identical enqueue put it at position 1. Nothing else exposes the failure — gh pr checks is green, mergeStateStatus stays CLEAN, no added_to_merge_queue timeline event is written, and the shell sees a zero exit. The tool asserted a state that did not exist, and that cost several hours of misdiagnosis.

The change

After the merge call succeeds, the script reads the PR back with one GraphQL query for state, isInMergeQueue and autoMergeRequest { enabledBy { login } }, and reports only what it observed: merged when the state says so, queued when the PR really holds a queue entry, and a failure with exit 2 otherwise. The queued branch keys on isInMergeQueue rather than on the repo-level queue_active flag from pr-status.sh — an entry that exists is an entry, whatever the configuration said a moment earlier.

A queue entry is registered asynchronously, so the first read can legitimately answer false for an entry that lands a second later. The read is therefore retried a bounded five times at 2s (~8s worst case) rather than trusted on its first answer or waited out with a fixed sleep; it breaks as soon as the outcome is decided, so a successful merge pays one query.

A probe that failed is kept apart from a probe that succeeded and saw nothing. An unreachable API says something about the request, not about the PR, so that case reports the outcome as unknown and does not name a cause — naming --disable-auto there would send the operator to mutate a PR whose state was never read. Both cases exit 2, because the point of the block is to stop reporting an outcome that was never confirmed.

Where autoMergeRequest is non-null, the failure names the enabler and offers gh pr merge <n> --repo <r> --disable-auto plus a retry. The script does not clear the request itself: it reports, and does not mutate state the caller did not ask for.

Exit-code semantics are unchanged in shape — 0 merged or queued, 1 the gate is shut and nothing was attempted, 2 needs a human — with 2 extended to cover "the call exited 0 while nothing merged and nothing entered the queue". The header comment says so.

Validation

tests/test_pr_merge_verify_outcome.sh — 10 cases against a stubbed gh and a stubbed pr-status.sh, no network and no repo, ~0.6s. It covers the real enqueue, the swallowed enqueue with and without an attached auto-merge request, the non-queue merged and still-open cases, an entry that only appears on the third probe (asserting three GraphQL calls, so the retry is actually exercised), the bounded poll (five attempts, no hang), the unreachable API, and two regression guards that the gate-shut path still exits 1 without calling gh and that --dry-run still calls nothing.

Because green on a first run is not evidence, the suite was also run against the pre-fix pr-merge.sh from main: every new assertion fails there (case 2 reports queued and exits 0), and the four guards for existing behaviour pass on both. shellcheck clean on the script and the test; pre-commit run --files on all three paths passes; the full tests/ suite (2 shell, 1 python) passes the way tests.yml runs it.

Docs

references/pull-request-workflow.md gains the failure mode as a named subsection directly beside its converse, the existing "autoMergeRequest: null does NOT mean not armed" note, with the GraphQL probe and the three readings of it. The ## Then Merge: scripts/pr-merge.sh section gains two sentences saying the outcome is now read back and pointing there.

`gh pr merge <n> --repo <r> --merge` on a repository with a merge queue
prints its usual success line and exits 0 while adding NOTHING to the
queue when an auto-merge request is already attached to the PR — the
pending request swallows the enqueue. pr-merge.sh relayed that exit
status as "queued (--merge, strategy set by the queue)".

Measured on one PR: immediately after the call isInMergeQueue was false
and the queue was empty; after `gh pr merge <n> --disable-auto` the
identical call put it at position 1. Nothing else shows the failure —
checks green, mergeStateStatus CLEAN, no timeline event — so the tool
asserted a state that did not exist, twice, costing hours.

After the merge call succeeds the script now reads the PR back via one
GraphQL query (state, isInMergeQueue, autoMergeRequest.enabledBy) and
reports only what it observed: merged, queued, or a failure with exit 2.
A queue entry registers asynchronously, so the read is retried a bounded
five times at 2s rather than trusted on its first answer or waited out
with a fixed sleep.

A probe that failed is kept apart from a probe that saw nothing: an
unreachable API says something about the request, not about the PR, so
that case reports the outcome as unknown instead of naming a cause.
Where autoMergeRequest is non-null the failure names it and offers
`--disable-auto` plus a retry; the script never clears it itself.

Adds tests/test_pr_merge_verify_outcome.sh (10 cases, stubbed gh and
pr-status.sh, no network) and documents the failure mode in
references/pull-request-workflow.md next to its converse, the
`autoMergeRequest: null` note.

Signed-off-by: Sebastian Mendel <github@sebastianmendel.de>
Copilot AI lite review requested due to automatic review settings August 10, 2026 07:54
@sonarqubecloud

Copy link
Copy Markdown

@github-actions github-actions Bot added documentation Improvements or additions to documentation skill labels Aug 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@CybotTM

CybotTM commented Aug 10, 2026

Copy link
Copy Markdown
Member Author

The Copilot review failed twice on 168c31dd — both rows are error bodies ("unable to review"), not reviews — so no bot review exists on this head and re-requesting a third time would only repeat it. Noting it here rather than merging: this PR still needs a human review before the gate opens. All 22 checks pass and mergeStateStatus is CLEAN; the branch is ready apart from the review.

@CybotTM
CybotTM merged commit 5fc47d9 into main Aug 10, 2026
23 of 25 checks passed
@CybotTM
CybotTM deleted the fix/pr-merge-verify-enqueue branch August 10, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants