[Stack 16/27] Fix D5: match Clojure prop_test formula (Wilson-score-like with +1 pseudocount)

[jucor]

Summary

Stacked on #2446 (Fix D9: z-score thresholds from two-tailed to one-tailed). Please review and merge #2446 first.
Next in stack: #2449 (Fix D6: match Clojure two-proportion test formula (+1 pseudocount))

Replace Python's standard one-proportion z-test prop_test(p, n, p0) with
Clojure's Wilson-score-like formula prop_test(succ, n) from stats.clj:10-15:

2 * sqrt(n+1) * ((succ+1)/(n+1) - 0.5)

The Clojure formula has a built-in +1 pseudocount (Laplace smoothing / Beta(1,1)
prior) that regularizes extreme values for small Polis groups. This is separate
from the PSEUDO_COUNT=2.0 used for pa/pd estimation (Beta(2,2) prior):

• pa = (na + 1) / (ns + 2) — Beta(2,2) prior for probability estimation
• pat = 2 * sqrt(ns+1) * ((na+1)/(ns+1) - 0.5) — Beta(1,1) prior for significance testing

What changed in the output: pat, pdt values (proportion test z-scores),
and downstream agree_metric / disagree_metric values. The z-scores are
now slightly different due to sqrt(n+1) vs sqrt(n) and (succ+1)/(n+1) vs
(na+1)/(n+2) denominators.

Changes

• repness.py: prop_test(p, n, p0) → prop_test(succ, n) with Clojure formula
• repness.py: prop_test_vectorized(p, n, p0) → prop_test_vectorized(succ, n)
• repness.py: Callers updated to pass raw counts (na, ns) instead of (pa, ns, 0.5)
• test_discrepancy_fixes.py: Removed xfail from D5 formula test, added 8 test cases + edge case
• test_repness_unit.py, test_old_format_repness.py: Updated for new signature
• Golden snapshots re-recorded for all datasets

Test plan

• D5 formula tests pass (8 input pairs + edge cases)
• D5 Clojure blob consistency check passes (all datasets)
• Full test suite passes (public + private, 19/19 regression tests)
• Only pre-existing failure: pakistan-incremental D2 (unrelated)

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Updates Delphi’s representativeness (“repness”) proportion-test implementation to match the Clojure reference, aligning downstream pat/pdt (and derived agree/disagree metrics) with the parity plan.

Changes:

• Replaced the Python one-proportion z-test with Clojure’s Wilson-score-like prop_test(succ, n) formula (with +1 pseudocount) and updated vectorized equivalent.
• Updated repness callers to pass raw counts (na/nd, ns) rather than smoothed proportions.
• Refreshed and expanded tests (including removing the D5 xfail) and updated docs/journal plan status.

Reviewed changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
delphi/polismath/pca_kmeans_rep/repness.py	Implements new prop_test(succ, n) + vectorized formula and updates callers to use raw counts.
delphi/tests/test_discrepancy_fixes.py	Enables D5 parity assertions (removes xfail) and expands coverage across multiple cases/edges.
delphi/tests/test_repness_unit.py	Updates unit + vectorized tests for the new prop-test signature/formula.
delphi/tests/test_old_format_repness.py	Updates backwards-compatible interface tests for the new prop-test signature.
delphi/docs/PLAN_DISCREPANCY_FIXES.md	Marks D5 / PR 4 as DONE in the discrepancy plan table.
delphi/docs/CLJ-PARITY-FIXES-JOURNAL.md	Adds PR4 journal entry describing the D5 change, rationale, and validation steps.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Delphi Coverage Report

File	Stmts	Miss	Cover
init.py	2	0	100%
benchmarks/bench_pca.py	76	76	0%
benchmarks/bench_repness.py	81	81	0%
benchmarks/bench_update_votes.py	38	38	0%
benchmarks/benchmark_utils.py	34	34	0%
components/init.py	1	0	100%
components/config.py	165	133	19%
conversation/init.py	2	0	100%
conversation/conversation.py	1107	320	71%
conversation/manager.py	131	42	68%
database/init.py	1	0	100%
database/dynamodb.py	387	234	40%
database/postgres.py	305	205	33%
pca_kmeans_rep/init.py	5	0	100%
pca_kmeans_rep/clusters.py	257	22	91%
pca_kmeans_rep/corr.py	98	17	83%
pca_kmeans_rep/pca.py	52	16	69%
pca_kmeans_rep/repness.py	297	38	87%
regression/init.py	4	0	100%
regression/clojure_comparer.py	188	17	91%
regression/comparer.py	887	720	19%
regression/datasets.py	135	27	80%
regression/recorder.py	36	27	25%
regression/utils.py	138	87	37%
run_math_pipeline.py	260	114	56%
umap_narrative/500_generate_embedding_umap_cluster.py	210	109	48%
umap_narrative/501_calculate_comment_extremity.py	112	53	53%
umap_narrative/502_calculate_priorities.py	135	135	0%
umap_narrative/700_datamapplot_for_layer.py	502	502	0%
umap_narrative/701_static_datamapplot_for_layer.py	310	310	0%
umap_narrative/702_consensus_divisive_datamapplot.py	432	432	0%
umap_narrative/801_narrative_report_batch.py	785	785	0%
umap_narrative/802_process_batch_results.py	265	265	0%
umap_narrative/803_check_batch_status.py	175	175	0%
umap_narrative/llm_factory_constructor/init.py	2	2	0%
umap_narrative/llm_factory_constructor/model_provider.py	157	157	0%
umap_narrative/polismath_commentgraph/init.py	1	0	100%
umap_narrative/polismath_commentgraph/cli.py	270	270	0%
umap_narrative/polismath_commentgraph/core/init.py	3	3	0%
umap_narrative/polismath_commentgraph/core/clustering.py	108	108	0%
umap_narrative/polismath_commentgraph/core/embedding.py	104	104	0%
umap_narrative/polismath_commentgraph/lambda_handler.py	219	219	0%
umap_narrative/polismath_commentgraph/schemas/init.py	2	0	100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py	160	9	94%
umap_narrative/polismath_commentgraph/tests/conftest.py	17	17	0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py	74	74	0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py	55	55	0%
umap_narrative/polismath_commentgraph/tests/test_storage.py	87	87	0%
umap_narrative/polismath_commentgraph/utils/init.py	3	0	100%
umap_narrative/polismath_commentgraph/utils/converter.py	283	237	16%
umap_narrative/polismath_commentgraph/utils/group_data.py	354	336	5%
umap_narrative/polismath_commentgraph/utils/storage.py	584	518	11%
umap_narrative/reset_conversation.py	159	50	69%
umap_narrative/run_pipeline.py	453	312	31%
utils/general.py	62	41	34%
Total	10770	7613	29%

[Copilot]

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The return-value description is slightly inaccurate with the +1 pseudocount: the sign is determined by (succ+1)/(n+1) relative to 0.5 (so e.g. succ==n/2 yields a positive value for n>0). Consider rewording to avoid implying it’s based on the raw succ/n proportion.

Suggested change

	Z-score (positive means succ/n > 0.5)
	Z-score (sign determined by (succ + 1) / (n + 1) relative to 0.5; positive when (succ + 1) / (n + 1) > 0.5)

[jucor]

Superseded by spr-managed PR stack. See the new stack starting at #2508.