feat: real validation framework (Phase 2 v1)

[Tilanthi]

What

The first piece of Phase 2: a real statistical validation framework that replaces the adapters' rigged scores with genuine, reproducible evidence — so pass/fail reflects held-out reality, not a planted answer. Pure standard library (no numpy/scipy), staying inside the Phase-0 runtime constraint.

Why

Phase 0 adapters deliberately return planted scores to exercise the pipeline. Before any real adapter (Phase 4) can be trusted, the system needs honest statistics: confidence intervals, p-values, and — critically for a discovery loop that tests many candidates — multiple-comparison correction. This PR builds that machinery and demonstrates it flowing through the existing kernel/ledger/ladder.

Changes

• src/sapiens/validation.py — the framework:
  • regularized incomplete beta function (Lentz continued fraction) as the backbone;
  • exact Clopper-Pearson binomial confidence interval;
  • Pearson correlation with a two-sided p-value;
  • Benjamini-Hochberg FDR correction (without it, testing many candidates yields floods of false positives);
  • leakage-safe holdout_split;
  • PassPolicy + evaluate() — a gate that decides pass/fail from real statistics (point estimate, CI floor, p-value).
• src/sapiens/adapters/regression.py — SyntheticRegressionAdapter that evaluates each candidate via a genuine held-out Pearson test through the gate, not a planted score.
• Tests — numerical correctness vs analytic Beta CDFs and known CI / p-value / FDR values, plus regression-adapter promotion and wrong-direction-rejection tests.

Notable behaviour

A candidate whose correlation is significant but in the wrong direction fails on effect size (point -0.988 below min_effect 0.5) even though its p-value is ~1e-49. The gate deliberately rejects p-value-hunting — a "significant" result that contradicts the hypothesis is not evidence for it.

Verified: proportion_ci(8, 10) → [0.4439, 0.9748]; betai matches Beta(1,1)/Beta(2,2)/Beta(0.5,0.5) CDFs exactly; BH on [0.01,0.04,0.03,0.20] → [0.04, 0.0533, 0.0533, 0.20].

Phase 0 invariants

Synthetic data, real methodology, scientific_discoveries_claimed stays 0. validation.py is a core module (statistics only; claims nothing). ruff clean, pytest → 28 passed.

Scope note

This is Phase 2 v1 (the statistical foundation). Not included here: calibration reports, leakage-control fixtures beyond holdout_split, and the structured L3 review panels (Phase 3) that consume this real evidence. Real adapters (Phase 4) remain gated behind the licensing/clean-room wall.

🤖 Generated with Claude Code