conformance: re-pin corpus to c7906bc (PRs #249 + #250, with #246/#251/#252) + authored help and mobile analyzer bundles by jaredmixpanel · Pull Request #253 · mixpanel/mixpanel-headless

jaredmixpanel · 2026-09-23T23:58:19Z

Summary

This is step 2 of the two-step protocol (conformance/record/README.md, "Which SHA to stamp"), following the #245 precedent. It re-pins the corpus and contract to c7906bcd6d7d6222fb225f5fd7e48e8ed39dd259, the tip of main. That commit is exactly the src/ + tests/ + conformance/ these files were built from. git log 6b23b75..c7906bc -- src tests conformance is #245, #246, #251, #249, #252 and #250, and none of those PRs touched conformance/vectors/ or conformance/contract/.

The PR has four commits:

Re-extraction. Only stamps move: 171 bundle headers and the manifest. One count changes: exclusions.no_seam_hit goes from 4,022 to 4,151. Of those, +122 come from Plugin overhaul: skills built on the built-in reference (mp help) and a plugin-owned Python environment #251's offline tests and +7 from feat(conformance): help registry contract and authored help vectors #249's; none of them reaches a recording seam. api-index.json is byte-identical, and the extracted total stays at 3,237 vectors in 171 bundles.
Authored bundles:
- authored/help/reference.jsonl: 157 vectors for the 13 help.* apis, from feat(conformance): help registry contract and authored help vectors #249.
- authored/replays/rrweb-analyze-mobile.jsonl: 17 rrweb_analyzer.analyze vectors over mobile streams, from feat(conformance): authored analyzer vectors over mobile replay streams #250.
Contract:
- help-registry.json is written for the first time (schema_version 1).
- error-codes.json, literal-aliases.json, model-coverage.json and tag-universe.json change generated_from only.
- help-registry.json leaves check_stamps.AWAITING_FIRST_REPIN, which is now empty. The guard's tests patch an awaiting name in instead of relying on a non-empty set.
Ledger. An EXTRACTION-LEDGER.md entry records the invocation, the count deltas and the verification.

The contract was generated after both bundles were written, so tag-universe.json and model-coverage.json scan the final corpus.

Invocation

SHA=c7906bcd6d7d6222fb225f5fd7e48e8ed39dd259
just conformance-record --mp-record-date=2026-09-23 --mp-record-commit=$SHA
uv run python -m conformance.record.gen_help_vectors --commit $SHA
uv run python -m conformance.record.gen_replay_analyze_vectors --commit $SHA
uv run python -m conformance.contract.generate_contract --generated-from $SHA

Test plan

uv run pytest conformance/tests -o addopts="" -q: 617 passed. The generator-equals-committed-bundle tests for both new bundles now run instead of skipping.
uv run pytest conformance/runner -o addopts="" -q: 3,654 passed (3,480 + 157 + 17).
just conformance-stamps: CLEAN.
CI drift command (re-extract with the committed stamps, then conformance.record.diff): record run 9,982 passed / 1 skipped / 563 deselected; drift check: CLEAN (byte-identical within D8 scope).
just conformance-smoke: smoke result: PASS (control clean, 14/14 sabotage patches caught).
just check: exit 0; 9,955 passed; coverage 93.71%.

After merge: TS port

Set conformance-runner/corpus.config.json sourceCommit to c7906bcd6d7d6222fb225f5fd7e48e8ed39dd259 and run npm run sync:corpus. The sync refuses unless manifest.source_commit equals the pin.
New in the snapshot:
- corpus/contract/help-registry.json. The TS generator reads it in place of the pending copy.
- authored/help/reference.jsonl: bind the 13 help.* apis.
- authored/replays/rrweb-analyze-mobile.jsonl: covered by the existing rrweb_analyzer.analyze binding.
Expected: 3,654 vectors with 0 FAIL / 0 UNPORTED once the help bindings land. The TS error-codes and literal-alias censuses are unchanged (36 / 39).
Note: 2 of the 17 analyzer vectors (the 1.05 / 0.95 tolerance-boundary taps) differ from the earlier local /tmp/rrweb-analyze copy, because 54ef784 moved those taps before feat(conformance): authored analyzer vectors over mobile replay streams #250 merged. The committed bundle is authoritative.

🤖 Generated with Claude Code

Record run with --mp-record-date=2026-09-23 --mp-record-commit=c7906bcd6d7d6222fb225f5fd7e48e8ed39dd259. Only the 171 bundle headers and the manifest stamps move, plus exclusions.no_seam_hit 4022 -> 4151 (#251 +122, #249 +7). api-index.json is byte-identical. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

gen_help_vectors --commit c7906bc writes authored/help/reference.jsonl (157 vectors, #249). gen_replay_analyze_vectors --commit c7906bc writes authored/replays/rrweb-analyze-mobile.jsonl (17 vectors, #250). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

generate_contract --generated-from c7906bc writes help-registry.json for the first time (schema_version 1); the other four artifacts change generated_from only. help-registry.json leaves check_stamps.AWAITING_FIRST_REPIN, now empty; the guard tests patch an awaiting name in instead of relying on a non-empty set. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

greptile-apps · 2026-09-24T00:01:16Z

Confidence Score: 5/5

The PR appears safe to merge; no actionable regression or repository-rule violation remains in the reviewable changes.

_{Reviews (2) · Last reviewed commit: "docs(conformance): exact #251 test count..."}

Copilot

Copilot review overview

🟢 Approval recommended

No blocking issues remain; the only finding is a non-blocking documentation nit.

Review effort: Lite
Findings: None

What changed in this PR

Re-pins the conformance corpus and contract to c7906bcd, adds authored help/mobile replay vectors, and finalizes stamp validation.

Changes:

Refreshes vector bundle and contract provenance stamps.
Adds 157 help and 17 mobile replay vectors.
Updates stamp-guard tests and extraction provenance.

File	Summary
`conformance/vectors/validation/test_validation.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_validation_funnel.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_validation_flow.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_validation_cohort.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_user_validators.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_query_validation.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_query_validation_pbt.jsonl`	Re-pinned vector bundle.
`conformance/vectors/validation/test_query_user_edge_cases.jsonl`	Re-pinned vector bundle.
`conformance/vectors/streaming/test_query_workspace_scoping.jsonl`	Re-pinned vector bundle.
`conformance/vectors/streaming/test_query_user_structural.jsonl`	Re-pinned vector bundle.
`conformance/vectors/streaming/test_api_client_data_governance.jsonl`	Re-pinned vector bundle.
`conformance/vectors/segmentation/test_expressions.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_workspace_report_links.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_validation_bypass_r2.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_query_limit.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_custom_property_types.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_custom_property_query.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_coverage_cases.jsonl`	Re-pinned vector bundle.
`conformance/vectors/retention/test_api_client_bookmarks.jsonl`	Re-pinned vector bundle.
`conformance/vectors/replays/test_types_signed_replay.jsonl`	Re-pinned vector bundle.
`conformance/vectors/replays/test_types_replay.jsonl`	Re-pinned vector bundle.
`conformance/vectors/replays/test_types_replay_summary.jsonl`	Re-pinned vector bundle.
`conformance/vectors/replays/test_types_replay_event.jsonl`	Re-pinned vector bundle.
`conformance/vectors/replays/test_rrweb_analyzer.jsonl`	Re-pinned vector bundle.
`conformance/vectors/replays/test_replay_bundle.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_workspace_funnel.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_validation_funnel.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_types_funnel.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_query_workspace_scoping.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_query_limit.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_discovery.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_custom_property_types.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_custom_property_query.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_coverage_cases.jsonl`	Re-pinned vector bundle.
`conformance/vectors/funnels/test_api_client.jsonl`	Re-pinned vector bundle.
`conformance/vectors/flows/test_workspace_report_links.jsonl`	Re-pinned vector bundle.
`conformance/vectors/flows/test_workspace_flow.jsonl`	Re-pinned vector bundle.
`conformance/vectors/flows/test_validation_bypass_r2.jsonl`	Re-pinned vector bundle.
`conformance/vectors/flows/test_types_flow.jsonl`	Re-pinned vector bundle.
`conformance/vectors/flows/test_run_flow_user_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_segfilter.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_query_types.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_query_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_custom_property_builders.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_build_cohort_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_bookmark_builders.jsonl`	Re-pinned vector bundle.
`conformance/vectors/filters/test_bookmark_builders_pbt.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_workspace_resolution.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_workspace_lazy_resolve.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_workspace_business_context.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_settings_headers.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_schema_graph.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_query_workspace_scoping.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_lexicon_schemas.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_app_api_client.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_webhooks.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_session.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_schemas.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_experiments.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_data_governance.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_crud.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_crud_edge.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_annotations.jsonl`	Re-pinned vector bundle.
`conformance/vectors/entities/test_api_client_alerts.jsonl`	Re-pinned vector bundle.
`conformance/vectors/engage/test_workspace_query_user_integration.jsonl`	Re-pinned vector bundle.
`conformance/vectors/engage/test_workspace_build_user_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/engage/test_run_flow_user_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/engage/test_query_user_structural.jsonl`	Re-pinned vector bundle.
`conformance/vectors/engage/test_query_user_edge_cases.jsonl`	Re-pinned vector bundle.
`conformance/vectors/engage/test_api_base_url_override.jsonl`	Re-pinned vector bundle.
`conformance/vectors/discovery/test_query_workspace_scoping.jsonl`	Re-pinned vector bundle.
`conformance/vectors/discovery/test_discovery.jsonl`	Re-pinned vector bundle.
`conformance/vectors/discovery/test_api_client.jsonl`	Re-pinned vector bundle.
`conformance/vectors/data-governance/test_api_client_data_governance.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_workspace_crud.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_workspace_cohort.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_workspace_build_user_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_validation_bypass.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_types_cohort_behaviors.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_coverage_cases.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_cohort_definition.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_build_cohort_params.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_api_client_crud.jsonl`	Re-pinned vector bundle.
`conformance/vectors/cohorts/test_api_client_crud_edge.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_workspace_report_links.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_workspace_crud.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_workspace_cohort.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_workspace_alerts.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_validation.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_validation_retention.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_validation_funnel.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_validation_bypass.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_validation_bypass_r2.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_types_pbt.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_query_validation.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_query_types.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_query_limit.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_query_integration.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_custom_property_types.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_custom_property_builders.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_bookmark_builders.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_bookmark_builders_pbt.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_api_client_crud.jsonl`	Re-pinned vector bundle.
`conformance/vectors/bookmarks/test_api_client_alerts.jsonl`	Re-pinned vector bundle.
`conformance/vectors/auth/test_workspace_resolution.jsonl`	Re-pinned vector bundle.
`conformance/vectors/auth/test_app_api_client.jsonl`	Re-pinned vector bundle.
`conformance/tests/test_check_stamps.py`	Updated stamp-guard tests; documentation nit noted.
`conformance/record/check_stamps.py`	Finalized first-repin stamp handling.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

jaredmixpanel and others added 4 commits September 23, 2026 16:57

docs(conformance): ledger entry for the c7906bc re-pin

7c1cb05

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Copilot AI lite review requested due to automatic review settings September 23, 2026 23:58

Copilot started reviewing on behalf of jaredmixpanel September 23, 2026 23:58 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

docs(conformance): exact #251 test counts in the c7906bc ledger entry

7fe1aac

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

jaredmixpanel merged commit 86ba6a9 into main Sep 24, 2026
11 checks passed

jaredmixpanel deleted the conformance/repin-c7906bc branch September 24, 2026 00:27

Navigation Menu

conformance: re-pin corpus to c7906bc (PRs #249 + #250, with #246/#251/#252) + authored help and mobile analyzer bundles - #253

conformance: re-pin corpus to c7906bc (PRs #249 + #250, with #246/#251/#252) + authored help and mobile analyzer bundles#253
jaredmixpanel merged 5 commits into
mainfrom
conformance/repin-c7906bc

jaredmixpanel commented Sep 23, 2026

Uh oh!

greptile-apps Bot commented Sep 24, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jaredmixpanel commented Sep 23, 2026

Summary

Invocation

Test plan

After merge: TS port

Uh oh!

greptile-apps Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

greptile-apps Bot commented Sep 24, 2026 •

edited

Loading