Skip to content

Commit b301295

Browse files
NickSeagullNickSeagullBotclaude
authored
Phase 1: fast inner loop + telemetry foundations (#717)
* feat(pipeline): Phase 1 — fast inner loop + telemetry foundations Inner loop (all measured on this machine, GHC 9.8.4, -O0 dev flavor): - cabal.project.dev: committed dev build flavor (optimization: 0) - scripts/dev-loop: persistent ghcid typecheck loop writing .ghcid-errors.txt — measured 0.55s error feedback, 1.9s recovery (target <5s). Resolves ghcid via PATH or nix shell nixpkgs#ghcid. - scripts/test-match: link-free hspec --match inside cabal repl — measured 9.2s (target <30s); parses the hspec summary for a real exit code since ghci always exits 0. - scripts/refresh-dev-cache: warm-cache refresh + modules-rebuilt cache-health metric for pool worktrees. - session-start hook: best-effort background docker compose up -d (warm Postgres for the gated suites), local + remote. Telemetry (schema v1 frozen in telemetry/SCHEMA.md): - scripts/telemetry.py: one validated JSON line per pipeline run to telemetry/runs.jsonl; closed failure-label taxonomy ('other' requires a note); golden-task archiver (telemetry/golden/, gitignored for now). Dogfood run 2026-07-07-001 recorded. Profiling findings recorded in SCHEMA.md: full nhcore -O0 = 249 modules/54s; recompilation is content-hash based; leaf edit = 4.9s; interface-preserving edits don't cascade. Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipeline): Phase 1 follow-up — ghcid in dev shell, human-tool parity doctrine - nix/hix.nix: add ghcid to shell.tools (verified locally: resolves from cache.iog.io, no source build — the CI shell rebuild is a cache hit). - README: new "Fast inner loop" section — the pipeline scripts are the human dev commands, deliberately the same scripts; plus an honest warning on the Linting section (hlint config doesn't encode the dialect and doesn't run in CI until Phase 2). - Plan: second governing rule — "Every pipeline tool is a human tool": same script/environment/flavor for agents and humans, non-interactive by default, telemetry from pipeline runs only, human-runnable as an acceptance criterion for every pipeline asset. - AGENTS.md: parity + telemetry-discipline notes. Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipeline): address CodeRabbit review — panel-verified, all 5 findings valid - scripts/test-match: reject vacuous success — "0 examples, 0 failures" (typo'd pattern / wrong suite) now exits 1 with a distinct "no specs matched" message instead of reporting a pass for tests that never ran. A verification oracle must never return vacuous success. (Major) - scripts/telemetry.py: enforce the closed failure-label taxonomy on EVERY finish call, not only failed/parked; stricter than suggested — 'ok' runs must not carry a failure label at all. (Major) - scripts/telemetry.py: validate run_id against ^[A-Za-z0-9][A-Za-z0-9._-]*$ at start and before golden archiving (guards the telemetry/golden/<run_id>/ path); format documented in SCHEMA.md as the de-facto run-id contract. (flagged Critical; panel notes severity inflated for a single-operator tool — fixed anyway) - scripts/telemetry.py: reject negative --seconds/--repair-rounds/--count. (Minor) - AGENTS.md: un-glue "[suite]#" in the shell example. (Minor) All rejection paths exercised: 6 telemetry guards + vacuous/real test-match runs; runs.jsonl untouched by the validation tests. Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipeline): scripts self-provision the toolchain via with-toolchain Loop scripts no longer assume they run inside the flake dev shell — an agent (or human) can invoke them from any bare shell: - scripts/with-toolchain: runs a command with the pinned toolchain. Fast path only when IN_NIX_SHELL is set AND the command actually exists (an inherited shell can be STALE — e.g. predate ghcid's addition to the flake; discovered live when exactly that happened); otherwise wraps in `nix develop --command` (~0.4s warm); loud host-tools fallback when nix is absent. - dev-loop / test-match / refresh-dev-cache route every tool call through it; dev-loop's ad-hoc `nix shell nixpkgs#ghcid` fallback removed (ghcid now comes from the flake shell). - SCRIPT_DIR captured before cd (relative invocation from subdirs was broken). - Docs: SCHEMA.md wrapper overhead + config-hash caveat (first ghcid load re-interprets all modules if dist-newstyle was built under a different env); AGENTS.md + README notes. Verified: wrapped dev-loop full load (249 modules, "All good"), test-match pass through wrapper (3.9s), refresh-dev-cache (0 modules rebuilt), wrapper overhead 0.4s warm. Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipeline): ./dev — single entrypoint for the shared human/agent dev tools Panel-reviewed dispatcher (zero-dependency bash, deliberately not a task runner): ./dev with no args lists the menu; implementations stay in scripts/. - Verbs describe intent, not mechanism: watch / check / test / refresh / exec. - dev-loop's conflation split: `watch` = resident ghcid daemon (plumbing, once per session); `check` = the instantaneous typecheck status agents and humans actually call (reads the watcher file; one-shot -O0 typecheck fallback when no watcher runs). - telemetry.py deliberately NOT on the menu — emission is pipeline-only by doctrine; a discoverable verb would invite ad-hoc pollution. - Standing rule added to the entrypoint header: every future pipeline asset registers a verb here or it doesn't ship (operationalizes the human-runnable acceptance criterion). - README / AGENTS.md / SCHEMA.md updated to the verb interface. Verified: usage listing, check fallback (exit 0), test (exit 0), exec (ghc resolves), unknown verb (exit 2). Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipeline): ./dev doctor — mechanical enforcement of verb registration A rule without a check is a wish (this repo's founding lesson). The "every pipeline asset registers a ./dev verb" rule now has its gate: - scripts/doctor: enumerable domain (asset := file in scripts/) + bidirectional set check — every registered verb resolves to an existing executable script (no dangling menu entries, no missing +x), every script has a verb OR a reasoned exemption, and no stale exemptions (exempt+registered = contradiction). Exemptions live in one reviewable array with reasons (telemetry.py: pipeline-only doctrine; cloud-setup.sh: platform-invoked; install.sh: end-user). - .github/workflows/checks.yml: fast governance job (pure bash, no nix, seconds) running ./dev doctor on every PR — Phase 2's hlint gate joins this workflow later. - Legacy scripts triaged into verbs: ./dev doctest, ./dev test-all. - Plan: third governing rule — "every governing rule names its gate" — with the current rule→gate pairings recorded. Verified: pass case (8 verbs, 11 scripts, 3 exemptions), adversarial unregistered-script case fails with actionable message. Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipeline): address bot review round 2 — workflow hardening + input edges CodeQL/GitHub Advanced Security + CodeRabbit findings, all valid: - checks.yml: permissions: contents: read (closes the CodeQL alert), concurrency group with cancel-in-progress, checkout pinned to the v4.2.2 SHA with persist-credentials: false. - scripts/watch: TARGET is spliced into ghcid's shell-evaluated --command string (unlike sibling scripts where targets travel as argv) — now restricted to cabal target charset; metacharacters exit 2 before any shell sees them. - scripts/test-match: escape backslashes/quotes in PATTERN before it lands inside ghci's :main string literal — embedded quotes silently garbled the match, the exact vacuous-result bug this script guards against. - scripts/check: accepts [target] like its siblings; the one-shot fallback no longer hardcodes lib:nhcore. Verified live: injection attempt rejected (exit 2), quoted pattern survives escaping and reports vacuous match honestly (exit 1), ./dev doctor green. Part of #715 (Phase 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: NickSeagullBot <bot@nickseagull.dev> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 050c84e commit b301295

18 files changed

Lines changed: 688 additions & 2 deletions

.claude/hooks/session-start.sh

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,15 @@
2121
#
2222
set -euo pipefail
2323

24-
# Only run inside the remote (Claude Code on the web) environment.
24+
# --- 0. Warm Postgres for the test suites (local + remote, best-effort) -----
25+
# Service/integration suites gate on POSTGRES_AVAILABLE; keeping the container
26+
# warm removes a ~10s cold-start from the verification loop (pipeline Phase 1).
27+
# Backgrounded and failure-tolerant: no docker => no-op.
28+
if command -v docker >/dev/null 2>&1; then
29+
(cd "${CLAUDE_PROJECT_DIR:-$(pwd)}" && docker compose up -d >/dev/null 2>&1 || true) &
30+
fi
31+
32+
# Only run the rest inside the remote (Claude Code on the web) environment.
2533
if [ "${CLAUDE_CODE_REMOTE:-}" != "true" ]; then
2634
exit 0
2735
fi

.github/workflows/checks.yml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Fast governance checks (pipeline Phase 1+). Pure bash, no toolchain —
2+
# feedback in seconds, long before the build workflows finish.
3+
#
4+
# Currently: ./dev doctor — enforces "every pipeline asset registers a ./dev
5+
# verb or carries a reasoned exemption" (see scripts/doctor for the contract).
6+
# Phase 2 adds the dialect hlint gate as a separate job (needs the toolchain).
7+
name: checks
8+
9+
on:
10+
pull_request:
11+
push:
12+
branches: [main]
13+
14+
permissions:
15+
contents: read
16+
17+
concurrency:
18+
group: checks-${{ github.workflow }}-${{ github.ref }}
19+
cancel-in-progress: true
20+
21+
jobs:
22+
doctor:
23+
name: dev doctor (verb registration)
24+
runs-on: ubuntu-latest
25+
timeout-minutes: 5
26+
steps:
27+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
28+
with:
29+
persist-credentials: false
30+
- run: ./dev doctor

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,3 +82,8 @@ installer/target/
8282
.ruff_cache
8383
# Phase 0: untracked local pipeline-state archive (Phase 1 will manage telemetry/ properly)
8484
/telemetry/archive/
85+
86+
# Phase 1: inner-loop + telemetry local state
87+
.ghcid-errors.txt
88+
/telemetry/.current-run.json
89+
/telemetry/golden/

AGENTS.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,24 @@ cabal test # all suites (Postgres needed: docker-compose up
3434
./testbed/scripts/run-tests.sh # acceptance tests (auto-starts the app)
3535
```
3636

37+
## Fast inner loop (measured 2026-07-07 — use this in repair loops, NOT cabal build)
38+
39+
Single entrypoint: **`./dev`** (no-args lists all verbs; same tools for humans and agents, deliberately):
40+
41+
```bash
42+
./dev watch # start resident typecheck watcher → .ghcid-errors.txt (once per session)
43+
./dev check # quick typecheck status (instant from watcher; one-shot fallback)
44+
./dev test "pattern" [suite] # link-free hspec --match (~4-9s; default suite nhcore-test-core)
45+
./dev refresh # re-warm -O0 build after pull/switch; prints modules-rebuilt
46+
./dev exec <cmd> # any command with the pinned toolchain
47+
```
48+
49+
- Repair-loop protocol: edit → wait ~2s → **`./dev check`** (measured: error feedback 0.6s, recovery 1.9s). Never spawn `cabal build` inside the loop.
50+
- You do NOT need to be inside `nix develop`: every verb self-provisions the pinned toolchain (~0.4s warm overhead).
51+
- Everything uses the dev flavor (`cabal.project.dev`, `-O0`); full nhcore -O0 build = 249 modules / ~54s on this machine.
52+
- Pipeline telemetry: `scripts/telemetry.py` (schema: `telemetry/SCHEMA.md`, frozen v1). Every pipeline run emits one line to `telemetry/runs.jsonl`. Telemetry is pipeline-only: never emit lines for ad-hoc runs.
53+
- These are the same commands humans use (README "Fast inner loop") — parity is deliberate; don't create agent-only variants.
54+
3755
- Test discovery: **only `nhcore-test` uses hspec-discover**; `nhcore-test-core`, `-auth`, `-service`, `-integration` register specs manually in their `Main.hs` — new spec modules must be added there AND to the cabal `other-modules`.
3856
- Postgres-dependent specs self-gate on `POSTGRES_AVAILABLE=true`.
3957
-**hlint does NOT run in CI**, and the current `.hlint.yaml` does not encode NeoHaskell style — do not treat hlint output as style guidance until the Phase 2 rebuild lands.

README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,30 @@ To run manually:
8080
hlint .
8181
```
8282

83+
> ⚠ The current hlint configuration does not yet encode NeoHaskell's dialect
84+
> rules and does not run in CI — don't treat its output as style guidance.
85+
> The dialect-first rebuild is tracked in
86+
> [#715](https://github.com/neohaskell/NeoHaskell/issues/715) (Phase 2).
87+
88+
## Fast inner loop
89+
90+
The same tools the CI/agent pipeline uses work on demand for humans (they are
91+
the *same scripts* on purpose — if you can't reproduce what the pipeline saw,
92+
you can't debug it). One entrypoint, run `./dev` for the full menu:
93+
94+
```sh
95+
./dev watch # resident typecheck watcher (ghcid, -O0)
96+
./dev check # instant typecheck status
97+
./dev test "EventStore" # run only matching specs, no linking (~4-9s)
98+
./dev test "insert" nhcore-test-service # pick a suite
99+
./dev refresh # re-warm the -O0 build after pull/branch switch
100+
./dev exec ghc --version # anything else, with the pinned toolchain
101+
```
102+
103+
Typecheck feedback lands in under a second once the watcher is running. You
104+
don't need to be inside `nix develop`: every verb enters it on demand (pinned
105+
toolchain from any bare shell). Measured baselines: `telemetry/SCHEMA.md`.
106+
83107
## Running Tests
84108

85109
The core library tests are split into domain-specific suites that run in parallel on CI:

cabal.project.dev

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
-- Fast dev flavor for the agent inner loop (pipeline Phase 1).
2+
-- Usage: cabal build --project-file=cabal.project.dev <target>
3+
-- (scripts/dev-loop and scripts/test-match use this automatically)
4+
--
5+
-- Rationale: the repair loop must never pay -O1 compile times. The bake job
6+
-- and all pool worktrees build with THIS flavor so dist-newstyle stays valid
7+
-- for the loop (optimization level is part of the build-config hash).
8+
import: cabal.project
9+
10+
optimization: 0

dev

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Developer entrypoint — dispatches the shared human/agent dev tools.
4+
# (Pipeline doctrine: every pipeline tool is a human tool; every new pipeline
5+
# asset registers a verb here or it doesn't ship.)
6+
#
7+
# Implementations live in scripts/; all of them self-provision the pinned
8+
# toolchain via scripts/with-toolchain — no need to be inside `nix develop`.
9+
#
10+
set -euo pipefail
11+
cd "$(dirname "$0")"
12+
13+
usage() {
14+
cat <<'EOF'
15+
usage: ./dev <verb> [args]
16+
17+
watch [target] start the resident typecheck watcher (ghcid, -O0);
18+
writes errors to .ghcid-errors.txt (default: lib:nhcore)
19+
check [target] quick typecheck status — reads the watcher's file
20+
(instant), or one-shot typecheck of target if no watcher
21+
test "pattern" [suite] run only matching specs, link-free (~4-9s;
22+
default suite: nhcore-test-core)
23+
refresh [target] re-warm the -O0 build after pull/branch switch;
24+
prints modules-rebuilt (cache health)
25+
doctest run the doctest suite
26+
test-all run every test suite (colored human runner)
27+
doctor harness self-check: every script has a verb or a
28+
reasoned exemption; no dangling verbs (CI-enforced)
29+
exec <cmd> [args] run any command with the pinned toolchain
30+
(e.g. ./dev exec ghc --version)
31+
32+
Measured baselines: telemetry/SCHEMA.md
33+
EOF
34+
}
35+
36+
VERB="${1:-}"
37+
[ $# -gt 0 ] && shift
38+
39+
case "${VERB}" in
40+
watch) exec scripts/watch "$@" ;;
41+
check) exec scripts/check "$@" ;;
42+
test) exec scripts/test-match "$@" ;;
43+
refresh) exec scripts/refresh-dev-cache "$@" ;;
44+
doctest) exec scripts/run-doctest "$@" ;;
45+
test-all) exec scripts/run-all-tests.sh "$@" ;;
46+
doctor) exec scripts/doctor "$@" ;;
47+
exec) exec scripts/with-toolchain "$@" ;;
48+
""|help|-h|--help) usage ;;
49+
*)
50+
echo "dev: unknown verb '${VERB}'" >&2
51+
usage >&2
52+
exit 2
53+
;;
54+
esac

docs/plans/2026-07-07-continuous-generation-pipeline-plan.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,15 @@
2323
| Invented-API events (GHC "not in scope") per PR | unmeasured | measured, trending ↓ |
2424
| Style violations reaching CI | unmeasured | 0 (caught at edit-hook/hlint layer) |
2525

26-
## Governing rule (the anti-rot constitution)
26+
## Governing rules (the anti-rot constitution)
2727

2828
> **No agent-visible document without a CI check or a generation source.**
2929
> Every asset in this plan is either compiler/script-generated (with a regenerate-and-diff sync check) or human-curated (with a structural validity check). Anything else gets archived.
3030
31+
> **Every pipeline tool is a human tool.** (Added 2026-07-07, panel-reviewed.) Same script, same environment, same build flavor for agents and humans — parity is what makes agent failures reproducible by a human in minutes. Non-interactive by default; TTY niceties never break the agent path. Telemetry records *pipeline runs only* — ad-hoc human use never emits. **Human-runnable is an acceptance criterion for every pipeline asset**, operationalized as: every asset registers a verb in `./dev`.
32+
33+
> **Every governing rule names its gate.** (Added 2026-07-07.) A rule without a named enforcement mechanism is a wish — rejected at plan review. Current pairings: doc-truth → `codemap check` + generated-file sync checks (Phase 3); verb registration → `./dev doctor` (CI: `checks.yml`, seconds, no toolchain); telemetry discipline → emitter validation in `scripts/telemetry.py`; dialect → hlint CI gate + edit hook (Phase 2); spec fidelity → signatures drift check (Phase 5); doc coverage → the ratchet (Phase 3).
34+
3135
---
3236

3337
## Phase 0 — Archive & truth reset

nix/hix.nix

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
hspec-discover = "latest";
2424
haskell-language-server = "latest";
2525
cabal-gild = "latest";
26+
ghcid = "latest"; # powers scripts/dev-loop (agent + human inner loop)
2627
};
2728
shell.buildInputs = with pkgs; [ git nixfmt-classic postgresql hurl poppler_utils ];
2829
}

scripts/check

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Quick typecheck status (pipeline Phase 1). THE verb agents and humans call
4+
# after an edit — instantaneous when the watcher is running.
5+
#
6+
# ./dev check
7+
#
8+
# Reads the watcher's output file (.ghcid-errors.txt, rewritten by
9+
# `./dev watch` on every save). If no watcher is running, falls back to a
10+
# one-shot incremental typecheck (a few seconds at -O0).
11+
#
12+
# Exit code: 0 = typechecks, 1 = errors.
13+
#
14+
set -euo pipefail
15+
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
16+
cd "${SCRIPT_DIR}/.."
17+
18+
TARGET="${1:-lib:nhcore}"
19+
OUT=".ghcid-errors.txt"
20+
21+
if [ -f "${OUT}" ]; then
22+
# mtime shown so a stale file (killed watcher) is detectable at a glance.
23+
echo "check: from watcher (file updated $(date -r "${OUT}" '+%H:%M:%S')):"
24+
cat "${OUT}"
25+
if [ ! -s "${OUT}" ] || grep -q "All good" "${OUT}"; then
26+
exit 0
27+
else
28+
exit 1
29+
fi
30+
else
31+
echo "check: no watcher running (start one with ./dev watch) — one-shot typecheck of ${TARGET}:" >&2
32+
"${SCRIPT_DIR}/with-toolchain" cabal build "${TARGET}" --project-file=cabal.project.dev 2>&1 | tail -5
33+
fi

0 commit comments

Comments
 (0)