Skip to content

Track disposition of the #185 explain-lab probe framework (.scratch/) #186

Description

@mobileskyfi

Parent: #185 (phase-0 lab, now complete) · #90 (explain umbrella).

Phase 0 (#185) is answered and the ratification PR is up. The lab's probe framework currently lives only in .scratch/ (gitignored, non-authoritative, uncommitted). Per the constitution the probes are throwaway and the findings are the deliverable — but a good part of this framework is reusable when explain is actually implemented, and right now its only durable record is the sha256 hashes embedded in the #185 exit-artifact comments. This issue tracks the disposition so the work is neither silently lost nor smuggled into mainline.

What exists

.scratch/explain-lab-*76 files, ~16 MB (56 .ts probes/SUTs + JSON fixtures/captures). The two 7.5 MB files are the Q13 per-character highlight streams (explain-lab-q13-streams.v7.23.2.json / v7.24rc2.json), captured from CHR; everything else is small.

Rough shape:

  • The offline SUT (a schema-free parser, written before scoring): explain-lab-segmenter.ts, -pathresolve.ts, -symbols.ts, -verbsplit.ts. This is the closest thing to a reference for the eventual grown canonicalizeExecuteCommandreference only, it is throwaway and has the known Q17 DoS + Q14 fabrication defects on purpose.
  • The oracle side: -q0-projector.ts (IL→{path,verb,args,blocks}), -cmdtree.ts, -cliref.ts, plus the frozen gold set -q0-gold-rev3.json.
  • The frozen partition: -partition.ts / -partition.json (dev/holdout split, the method precondition for Q1–Q4/Q13).
  • Per-question scorers + corner suites: -q{1,2,3,4,6,8,10,11,13,14,15,16,17}-*.ts.

Reusable as product test fixtures (the point of tracking this)

Several exit artifacts already flagged their probe assets as reusable when explain is built — these should become real test/ fixtures/anchors at that time, not be re-derived:

  • Q14 mutation suite (-q14-recovery.ts) → recovery/diagnostics anchor tests (the fail-closed bare-word rule + defect-region contract).
  • Q15 coordinate fixtures + reference mapper (-q15-coords.ts) → the coordinate-invariant anchor test when the coordinate mapper ships.
  • Q17 adversarial generators (18 + 5 nesting shapes, -q17-stress.ts) → determinism / span-well-formedness / depth-bound property tests.
  • Q0 gold set (-q0-gold-rev3.json) → if any IL projection is ever productized.
  • Q13 highlight streams → the per-occurrence oracle for a span-vocabulary (Q12) build; large, CHR-derived, reproducible from -q13-recapture.ts against a CHR.

Decision needed

Pick a home so this survives a .scratch/ wipe without polluting mainline. Options (not mutually exclusive):

  1. Keep in .scratch/, recover-by-hash. Cheapest. The exit artifacts already pin every file's sha256; re-runnable against the two CHRs. Risk: .scratch/ is explicitly ephemeral and uncommitted — one cleanup loses it.
  2. Archive branch (e.g. lab/explain-185-archive) holding the framework verbatim, never merged. Durable, out of mainline history, git-recoverable. The 15 MB of streams would need .gitattributes/LFS or exclusion.
  3. Promote-on-implement. Leave in .scratch/ now; when explain implementation starts, lift the four reusable suites above into test/ as fixtures/anchors and drop the rest. Cleanest long-term; depends on implementation actually starting.

Suggested: (1) now + (3) at implementation, with this issue as the durable index so nothing is orphaned. Streams (option-2 concern) stay CHR-reproducible and need not be committed.

Not blocking #185 closure or the ratification PR — this is bookkeeping for the reusable half.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions