Skip to content

corpus: v_script_summary multiplies parse capture counts through one-to-many joins #39

Description

@mobileskyfi

Problem

v_script_summary joins artifact_files, highlight_snapshots, and parseil_results before summing parse capture outcomes. These one-to-many joins multiply the parse rows. The distinct counters survive, but SUM(CASE WHEN p.ok ...) does not.

Reproduced on the pinned schema-v4 corpus:

  • Repository revision: 7d623557d257e0a4d50f47c87d6a842ff2699401
  • Database SHA-256: 4f1560f757265570b5b04c9f93d6d57b432064cf655ce2d1852da95536e313a9
Source Actual successful capture rows View's parseil_ok_count
edge-cases/empty.rsc 6 72
edge-cases/unicode-heavy.rsc 6 72
corpus row 228 6 72

edge-cases/oversize-32k.rsc has six failed capture rows, but the view reports parseil_error_count=36.

Reproduction

Run against that pinned database:

SELECT s.id, s.path, v.parseil_version_count,
       v.parseil_ok_count, v.parseil_error_count,
       (SELECT COUNT(*) FROM parseil_results p
        WHERE p.script_id=s.id AND p.ok=1) AS actual_capture_ok,
       (SELECT COUNT(*) FROM parseil_results p
        WHERE p.script_id=s.id AND p.ok=0) AS actual_capture_error
FROM v_script_summary v
JOIN source_scripts s ON s.id=v.id
WHERE s.id IN (3,4,6,228);

Source: scripts/build-corpus-db.ts, CREATE VIEW v_script_summary.

Scope and acceptance

  • Aggregate capture outcomes per script before joining unrelated one-to-many tables, or use equivalent independent aggregates.
  • Assert that adding an artifact or highlight snapshot cannot change parse outcome counts.
  • Verify every source's view counts against direct queries of parseil_results, including failures and missing captures.
  • Regenerate the database reproducibly and update downstream pins deliberately.
  • Keep capture success distinct from script validity (corpus: separate parse outcomes from rejection causes and bind captures to exact inputs and device environments #37): ok=1 means capture succeeded, even when its response is a diagnostic.

This finding affects this summary view. It does not explain the Glossema comparison's direct-join counts; those have separate classification problems in #37.

Independent offline review by Codex, 2026-09-21; filed with maintainer authorization.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions