ci: enforce contract compatibility on pull requests - #161
Merged
Conversation
5 tasks
fullstackjam
added a commit
that referenced
this pull request
Aug 2, 2026
…#163) * docs: realign branch-protection sources of truth with live protection PR #161 made `contract schema (L2)`, `curl|bash smoke`, and `old-cli compat` run on pull_request and added them to branch protection on main, but left the in-repo files describing the previous three-check world. Live protection requires six contexts; the repo claimed three. The maintainer confirmed the six-check state is intended, so the in-repo files catch up to protection rather than the reverse. Live protection is unchanged by this commit. - .github/required-checks.txt: add the three missing contexts, so the file matches `.required_status_checks.contexts` exactly. - docs/MERGE_POLICY.md: drop the pre-merge/post-merge split, which no longer describes anything real — every job in test.yml fires on push, pull_request, and both dispatch events with no job-level `if:`, and vm-e2e on push and pull_request. "Why these three" becomes "Why these six". Adds the external-state trade-off these required checks carry, and notes the drift sensor's blind spot: it compares the file against workflow job names only, never against live protection, which is how this drift went unnoticed. - docs/HARNESS.md: the curl|bash smoke row said "push to main / dispatch" and the L2 contract row said "CI"; both now say "every PR". * docs: correct CI claims found by review fact-check A fact-check of the previous commit against the actual workflows found three wrong claims in the new text. All verified by reading the files. - `vm-e2e` does not run on push to `main`. vm-e2e-spike.yml scopes its push trigger to the `test/vm-e2e-speed` spike branch, so the claim "all six run on every PR and again on push to main" was false. It is PR-only, now stated as such. - `curl|bash smoke` never executes `scripts/install.sh`. The job curls `localhost:18888/testuser/test-config/install`, which mock-server.py serves as a synthetic stub that execs the freshly built binary in dry-run mode; no workflow references scripts/install.sh at all. The real installer is covered in L1 by install_script_test.go, which pipes it through `/bin/bash -s`. Both the table row (pre-existing error) and the new "no Go test exercises it end to end" bullet were wrong. - The network-dependency section said "three" but named two, and claimed a GitHub API blip or yanked asset blocks all PRs. `old-cli compat` ends its lookup with `|| true` and gates every later step on a non-empty version, so it passes green having tested nothing. Rewritten to name the real asymmetry: L2 blocks, cli-compat fails open. Also corrects "fails on PRs" for the drift sensor, which is continue-on-error, and notes its second blind spot: it only checks that listed checks have jobs, never that required contexts are listed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Runs the published contract fixtures through the real CLI decoders on every PR and restores all live branch-protection compatibility jobs on pull requests.
Why?
The L2 job only validated fixtures against schemas from the same repository and skipped pull requests. That allowed
normalizeRemoteConfigand other tolerant decoders to hide server drift while CI stayed green. GitHub branch protection already requirescontract schema (L2),curl|bash smoke, andold-cli compat, so skipping those jobs also left required checks with no result.Testing
go vet ./...passesOPENBOOT_CONTRACT_DIR=/private/tmp/openboot-contract-fixture go test -v -tags=contract ./test/contractmake test-unitpassesCross-repo checklist
openboot.dev; this changes CLI CI only.openboot-contractfixtures.Notes for reviewer
The new remote-config test decodes the canonical wire shape separately, then compares it with
UnmarshalRemoteConfigFlexiblefield-for-field. A canonical fixture that causes the CLI to repair, move, or drop fields now fails instead of being silently normalized.On
repository_dispatch, the contract checkout uses the dispatched commit SHA. Other events test the current contractmainbranch.