Commit d4d1eb9
Pipeline Phase 6 — release tail + learning loop (#722)
* feat(pipeline): Phase 6a — telemetry schema v3 + failure→asset-delta enforcement
Foundation for Phase 6 (release tail + learning loop, ADR-0068). The
failure→asset-delta protocol was prose-only in the pipeline skill; this
mechanizes it and lays the schema the miner/weekly-review (PR-C) import.
- telemetry schema v2 → **v3**: new run fields `asset_delta` (the class-fix a
non-ok run ships) and `assets_consulted` (usage accounting). Readers tolerate
older versions; the emitter writes v3.
- `telemetry.py finish --outcome {failed,parked}` now **requires**
`--asset-delta <type>:<dest>` from a closed delta-type taxonomy v1
(`alias|extension-point|phrasebook|hot-card|hlint-rule|hook|cli-utility|
skill-edit|telemetry-label|PRUNE|none`); `none:<reason>` is the honest escape.
- `telemetry.py consult --asset <kind>:<name>` — usage accounting so the miner
can PRUNE never-consulted assets.
- `telemetry.py --self-test` (new; wired into `./dev doctor`) covers the v3
fields, the enforcement, and a **cross-file label-taxonomy parity** check
(telemetry.py FAILURE_LABELS ≡ pipeline-state LABELS) — closes the duplicate-
enum drift risk.
- ADR-0068 formalizes the protocol, the recommendation contract, the
notify-only kill switch, and the deferred archive-sunset checklist.
- SCHEMA.md documents the taxonomy, the `recommendations.jsonl` record, and the
decided golden-retention policy (local/gitignored, prune after 4 weeks).
- pipeline SKILL.md + AGENTS.md: the failure policy now names the enforced
`--asset-delta` close and the `consult` usage logging.
Part of #715 (Phase 6). PR-A of the A/B/C sequence; PR-B (release tail) and
PR-C (learning loop) build on this schema.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(pipeline): Phase 6 release tail + learning loop (DoD, kill switch, changelog, miner)
Completes Phase 6 on top of the schema-v3 spine (ADR-0068). Everything is
self-tested on fixtures via `./dev doctor`; the data-dependent activations
(first weekly review, first miner report, archive sunset) are deferred until
real runs accumulate.
Release tail:
- **DoD gate**: `spec-check --criteria <spec>` emits the criteria table as JSON;
a tier lint rejects an `acceptance` criterion that names no `.hurl` test (and
a `unit` one that does). `dod.yml` (workflow_run on `Test`) flags a suite
failure on `main` as a **revert-candidate** — notify-only, never auto-reverts.
- **Kill switch**: `scripts/revert` + `./dev revert <commit>` opens a revert PR;
`revert.yml` runs it on a maintainer `/revert` comment (OWNER/MEMBER-gated,
distinct from claude.yml's `@claude` so they never double-fire).
- **Changelog**: `scripts/changelog` derives `CHANGELOG.md` from contract-delta
specs; breaking = a removed signature line ⇒ mandatory migration note;
`changelog --check` gates it at PR-ready (checks.yml).
- **`./dev testbed`**: the acceptance suite gets a verb (parity — was CI-only).
Learning loop:
- **`scripts/retrospect`**: the deterministic weekly digest (stage times,
failure labels, invented-API trend vs the first-5 baseline, waiting-on-human,
asset usage) + the recommendation-contract validator (≥2 runs, in-taxonomy,
measured cost) writing `telemetry/recommendations.jsonl`.
- **`neohaskell-retrospective-miner` skill**: the frontier-model procedure that
turns recurring friction into ≤5 contract-validated recommendations.
Governance: 4 new `./dev` verbs (testbed/changelog/revert/retrospect) + 3 new
self-tests wired into doctor (now 23 verbs, 7 skills); AGENTS.md gains a Phase 6
section. All 6 workflows parse.
Part of #715 (Phase 6). Consolidated into this PR per request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pipeline): Phase 6 review hardening — anti-rot parity gates, DoD honesty, ledger safety
Closes the 45 findings from the Phase 6 PR review. Every doc claim corrected
here is now backed by a gate (doctor/self-test), not just edited, so it can't
re-rot on the next change.
- Parity gates (telemetry.py self-test): schema-version (docs == CURRENT_SCHEMA)
and delta-type prose (SCHEMA.md + miner skill == DELTA_TYPES); fix stale v2 strings.
- ./dev telemetry verb (drop doctor exemption); all docs invoke it, not bare telemetry.py.
- Parallel-worktree ledger safety: runs/recommendations.jsonl merge=union + run_id
collision guard in `telemetry start`.
- changelog migration gate rejects placeholder impact (TBD/N/A/-/TODO/pending).
- DoD gate watches "Test" AND "Test macOS"; resolves PR via commits->pulls; honest comment.
- Kill switch: revert.yml and claude.yml provably disjoint (exact /revert match);
conflicting `git revert` aborts cleanly before any push.
- retrospect: golden pruner + inventory, `--set` lifecycle (gated status/validated),
week/<=5-per-week validation, schema-v1 dummy + malformed-line skip, PRUNE zero-saving.
- telemetry asset-delta: `none:<reason>` keeps colons (not split into ref).
- DoD honesty: new `spec-check --criteria-tests` existence gate wired into CI
(doctor asserts it has a caller); AGENTS.md de-claims the post-merge per-level checker.
- consult logging wired into pipeline skill localize/implement steps.
- Cleanups: tier-lint "hurl" substring, ADR 0067/0068 Accepted, SCHEMA header reword,
CHANGELOG promotion note, pipeline-bootstrap dogfood-exemption note, revert OWNER/MEMBER rationale.
Verified: ./dev doctor green; negative probes confirm the schema-version and
delta-type gates fail on drift; all workflow YAML valid.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipeline): address CodeRabbit on Phase 6 — revert hardening, ledger + gate safety
Five findings from CodeRabbit's review of the Phase 6 PR, all valid:
- revert.yml: add `concurrency` keyed on the PR number (cancel-in-progress) so
two quick `/revert` comments can't race on the revert branch; set
`persist-credentials: false` and authenticate the push via an explicit
tokenized remote instead of leaving the token in .git/config for later steps.
- scripts/revert: `gh pr create` is now checked — a push that succeeds but a PR
that fails no longer reports success (exits with recovery guidance); `--no-pr`
no longer falsely prints "opened revert PR".
- scripts/retrospect: `_load_recs()` tolerates a malformed recommendations.jsonl
line (warn + skip, mirroring load_runs) so one bad hand-edit can't crash the
weekly review; self-test covers it.
- scripts/spec-check: `_module_exists` uses an exact stem match (like
`_hurl_exists`) — a substring match let a criterion naming `FooSpec` be
"satisfied" by an unrelated `FooBarSpec`, weakening the DoD gate.
./dev doctor green (24 verbs, 7 skills, all self-tests); revert.yml parses.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci(test): job-level path gating so ci-gate is a safe required check
Enforce "the project compiles AND tests pass" without deadlocking tooling-only
PRs. A required status check that never runs stays pending forever and blocks
the merge; Test.yml's workflow-level `paths:` filter did exactly that for any PR
not touching compiled paths.
- Remove the `pull_request` `paths:` filter — Test now always triggers on PRs
(push keeps its paths filter; post-merge/dod.yml behavior unchanged).
- New cheap `changes` job detects whether a PR touched compiled surface
(core/testbed/integrations/nix/workflows/cabal.project/flake); heavy jobs skip
when it didn't (build+doctest gated; test-*/codemap-sync cascade via needs).
- `ci-gate` (always runs, real conclusion) now enforces build+doctest+all test
suites, accepting skips ONLY when there is nothing to test (draft or no
compiled change) and failing closed if path-detection itself failed.
This makes `ci-gate` sound to require in branch protection: green on tooling
PRs (nothing to build), red whenever a compiled PR fails to build or test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci(test): codemap-sync runs last and is enforced via ci-gate
codemap-sync now depends on build + every test suite, so it's the final job —
the last confirmation right before merge — and the codemap regen isn't spent on
a PR that's already failing. Folded into ci-gate's checked results (last, so a
real test failure surfaces before codemap-sync's transitive skip), so the
already-required ci-gate now enforces "committed codemap matches the code" too.
No branch-protection change needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci(test): keep codemap-sync parallel (enforced, no wall-clock cost)
Reverts the serial-last ordering — codemap-sync runs alongside the test suites
again (needs only build's artifact) on its own runner, so it adds no PR latency,
while staying in ci-gate's checked results so it remains a merge gate. The
"don't spend the regen if tests fail" saving was a runner-minute, not worth
lengthening every compiled PR.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: NickSeagullBot <bot@nickseagull.dev>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>1 parent b547cb5 commit d4d1eb9
22 files changed
Lines changed: 1830 additions & 72 deletions
File tree
- .claude/skills
- neohaskell-pipeline
- neohaskell-retrospective-miner
- .github/workflows
- docs/decisions
- scripts
- telemetry
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
29 | 32 | | |
30 | 33 | | |
31 | 34 | | |
| |||
56 | 59 | | |
57 | 60 | | |
58 | 61 | | |
59 | | - | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
60 | 65 | | |
61 | 66 | | |
62 | 67 | | |
| |||
69 | 74 | | |
70 | 75 | | |
71 | 76 | | |
72 | | - | |
| 77 | + | |
73 | 78 | | |
74 | 79 | | |
75 | 80 | | |
| |||
91 | 96 | | |
92 | 97 | | |
93 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
94 | 106 | | |
95 | 107 | | |
96 | 108 | | |
97 | 109 | | |
98 | 110 | | |
99 | 111 | | |
100 | | - | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
85 | 101 | | |
86 | 102 | | |
87 | 103 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | | - | |
| 25 | + | |
24 | 26 | | |
25 | 27 | | |
26 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
0 commit comments