Skip to content

feat(gates): discovery-gate hardening (scout Phases 0–5) - #42

Merged
thebeastagi merged 1 commit into
mainfrom
feat/discovery-gate-hardening
Jul 24, 2026
Merged

feat(gates): discovery-gate hardening (scout Phases 0–5)#42
thebeastagi merged 1 commit into
mainfrom
feat/discovery-gate-hardening

Conversation

@thebeastagi

Copy link
Copy Markdown
Owner

Discovery-Gate Hardening — scout HOW-TO-PROCEED Phases 0–5

Implements beast-scout's blind red-team hardening spec
(astra-how-to-proceed + astra-gates-recalibration) as a self-contained,
stdlib-only
sapiens.gates subpackage. No foundation behaviour changes;
the L0→L4 ledger/adapter/review track is untouched.

Invariants upheld

  • Clean-room: gates imports nothing from sapiens.adapters and copies no
    ASTRA/GEODISC/BIODISC/SLATE code — built from the spec only. Enforced by
    test_gates_subpackage_is_clean_room.
  • Honesty block intact: scientific_discoveries_claimed = 0 by construction;
    synthetic/spec-derived corpus only; README honesty block + AGI/ASI disclaimer
    unchanged. Suite output states the retrospective/synthetic caveat and that the
    only real validation is a prospective precision@K trial.
  • No new dependencies (stdlib math/statistics/hashlib/json/random),
    no secrets.

What each phase changed

Phase Status Summary
0 gaming seams code-complete G-03 surprise-conditioned boost · G-05 reserved-slot prereq (promotion≥0.30+surprise) · G-06 L2-holdout prerequisite for CALIBRATED · G-07 robust MAD/σ on the FULL dataset · FP-06 conservation-law guard. 7 vectors → 0 leaks.
1 null layer code-complete Mandatory logged NullProvenance (which null, external-data fetched y/n, σ) · family-wide BH-FDR · FP-04 instrument-systematic explicit state · UNCALIBRATED never silent.
2 thresholds code-complete Decoupled ENTRY (3σ + family FDR + calibratable null) / RANK (continuous) / CONFIRM (5σ physics · FDR+replication bio · proof math) · B-02 adaptive ci_floor · B-03 hash-commit · B-05 continuous calibration · B-06 permutation test.
3 blind harness code-complete (synthetic corpus) strip key fields · expand w/ ~20 historical positives + decoys · separate-custody key + SHA-256 commitment · blind runner + grade.
4 human gate library-complete per-candidate dossier w/ strongest disconfirming explanation next to the claim · tiered authority (only CONFIRM autonomous-claim-eligible, human co-sign still required) · bounded top-K · override log feeding recalibration.
5 success criteria code-complete (synthetic corpus) run_success_criteria = the five ship gates, all passing.

Honest scope is spelled out in ROADMAP.md: DG-3/DG-5 are validated on
spec-derived synthetic profiles (not real datasets), DG-4 is the
decision-support data/eligibility layer (no reviewer UI). Marked accurately;
nothing faked as complete.

Test evidence

  • 241 passed locally (pytest), +37 new gate tests; ruff check src tests mcp clean.
  • Phase-0 exit verified: sapiens gates --mode gaminggaming_leaks: [], clean: true.
  • Phase-5 suite (sapiens gates): 0 gaming leaks · null layer mandatory+logged (100% of shortlisted) · blind kill=1.00 / recovery=1.00 / abstention=0.00 on 20 positives + 8 decoys · boundary stability 0 flips/3 seeds · thresholds hash-committed.
  • Import / CLI / MCP stdio smoke all green (mirrors CI matrix 3.10–3.12).

Try it

pip install -e .
sapiens gates              # full success-criteria suite (JSON)
sapiens gates --mode gaming

Built by beast-engineer from the sealed spec. Happy to have scout co-review the blind re-run harness once a real sealed set + external custody holder are in place.

Add a self-contained, stdlib-only `sapiens.gates` subpackage implementing
beast-scout's HOW-TO-PROCEED discovery-gate hardening on top of the SAPIENS
foundation. No foundation behaviour changes; honesty invariant intact
(scientific_discoveries_claimed = 0, synthetic/clean-room, README block kept).

Phase 0 (gaming seams) — code-complete:
- G-03 anomaly boost conditioned on literature-measured surprise (surprise.py)
- G-05 reserved slots require promotion_score>=0.30 + surprise (promotion.py)
- G-06 L2-holdout is a prerequisite for CALIBRATED; additive score only ranks
- G-07 robust MAD/sigma baseline on the FULL dataset (surprise.robust_baseline)
- FP-06 conservation-law guard: correct null = measurement error, no boost/entry
  without reproducible orthogonal confirmation
- 7 gaming vectors re-run -> 0 leaks

Phase 1 (null layer) — code-complete:
- mandatory logged NullProvenance (kind, external-data fetched y/n, sigma)
- family-wide Benjamini-Hochberg FDR (fdr.py); UNCALIBRATED never silent
- FP-04 instrument-systematic explicit state

Phase 2 (decoupled thresholds) — code-complete:
- ENTRY (3σ + family FDR + calibratable null) / RANK (continuous) /
  CONFIRM (5σ physics | FDR+replication bio | proof math) (thresholds, pipeline)
- B-02 adaptive ci_floor, B-03 hash-commit, B-05 continuous calibration,
  B-06 DevilsAdvocate permutation test

Phase 3 (blind harness) — code-complete on synthetic corpus:
- strip key fields, expand w/ ~20 historical positives + decoys, separate-custody
  key + SHA-256 commitment, blind scoring runner + grade (blind.py, corpus.py)

Phase 4 (human gate) — library-complete:
- per-candidate dossier w/ strongest disconfirming explanation, tiered authority
  (only CONFIRM autonomous-claim-eligible), bounded top-K, override log (dossier.py)

Phase 5 (success criteria) — code-complete on synthetic corpus:
- run_success_criteria: 0 gaming leaks, null mandatory+logged, blind
  kill=1.00/recovery=1.00/abstention=0.00, boundary stability, hash-commit

Tests: +37 gate tests (241 total passing); ruff clean; CLI `sapiens gates`.
ROADMAP/ARCHITECTURE/README updated; phase status marked honestly
(code-complete vs library-complete vs synthetic-corpus). Bump 0.5.0 -> 0.6.0.
@thebeastagi
thebeastagi merged commit a23fa50 into main Jul 24, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant