Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: conformance

# The parity number in the README is only worth what a third party can re-run.
# This pins the suite commit and the spec digest, demands the full 149/149, and
# This pins the suite commit and the spec digest, demands the full 153/153, and
# compares a fresh run against the checked-in evidence, so the repository cannot
# drift away from its own claim.
on:
Expand All @@ -14,8 +14,8 @@ permissions:
contents: read

env:
SUITE_COMMIT: ea25a1e218e94843e018dffc0eae4f3fcab1749e
SPEC_DIGEST: 39233b27b7f27b94ed727a3852030c69c7a64e5706b73519848a2b02f244e661
SUITE_COMMIT: 7098f4e6b7d04c8394969ed81b4025d4d9038324
SPEC_DIGEST: 606215de629d5f5eda9e62826cf511733b1ec0b9ca8ed07662a5c8bfe181d0b9

jobs:
parity:
Expand Down Expand Up @@ -67,11 +67,11 @@ jobs:
# something is already wrong, so it gets held shut here.
run: python3 tests/test_compare_report.py

- name: Full corpus, 149/149 required
- name: Full corpus, 153/153 required
run: |
set -euo pipefail
cargo run --locked --release -- aee-conformance/vectors --json fresh.json | tee run.txt
grep -q 'parity: accepts 35/35, rejects 114/114' run.txt
grep -q 'parity: accepts 36/36, rejects 117/117' run.txt

- name: Fresh run must match the checked-in evidence
run: python3 scripts/compare-report.py fresh.json reports/suite-revision-5.json
run: python3 scripts/compare-report.py fresh.json reports/suite-revision-6.json
10 changes: 6 additions & 4 deletions NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
| 3 | `cf0d5402327ae5a451efebc914852d1c687753ca` | `d3872a02875b2da8de0263e93fb92ca6f5ab0fd75f07ed3762a1b18b0c1712a3` (unchanged) | 140 | 140/140 |
| 4 | `b886c0a` | — | — | not run here |
| 5 | `ea25a1e218e94843e018dffc0eae4f3fcab1749e` | `39233b27b7f27b94ed727a3852030c69c7a64e5706b73519848a2b02f244e661` | 149 | 149/149 (148/149 unchanged) |
| 6 | `7098f4e6b7d04c8394969ed81b4025d4d9038324` | `606215de629d5f5eda9e62826cf511733b1ec0b9ca8ed07662a5c8bfe181d0b9` | 153 | 153/153 (151/153 unchanged) |

Revision 4 vendored the new encoding and nesting rules into the spec and this
checker was never run against it, so there is no record for it and the table
Expand Down Expand Up @@ -45,10 +46,11 @@ reads as 129 to a per-value counter and 128 to a per-container one; at 129 both
reject, and at 128 with an empty-container leaf both accept. The boundary
therefore lives in `src/json.rs`'s own tests.

All five are inspectable and reproducible by hand from those pins; the
revision-5 run at 149/149 is the one re-verified continuously by CI, which
follows the current suite pin. Revision 3 was continuously verified until
revision 5 replaced it and remains reproducible from its own pins.
All six are inspectable and reproducible by hand from those pins; the
revision-6 run at 153/153 is the one re-verified continuously by CI, which
follows the current suite pin. Each earlier record was continuously verified
until the next replaced it, and each remains reproducible from its own pins:
the fixed build still reproduces the revision-5 record exactly.

## Vendored spec vs branch head

Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ An independent validity-gate checker for the **Adversarial Execution Evidence (A

**suiteRevision 5: 149/149** (35/35, 114/114) after a parser fix. The unchanged revision-3 build scored **148/149** against it, and the single miss is worth stating plainly because the causation runs the other way this time: the revision pins a nesting bound the earlier text did not state, this checker had picked 256, and the new `bad-741` vector found it. Not blind, and not a case of the two meeting. The bound was only the visible half. The revision also states the counting rule, and this parser had been incrementing per parsed value rather than per open container, which read exactly one level deeper than the spec rule on **every document in the corpus** — all 149 statements and every record payload inside them, measured on the raw bytes so the deliberately ill-formed vectors are covered too — because every deepest path in the corpus ends in a scalar. Changing only the constant scores 149/149 as well, and still rejects a statement at depth 128 that the spec calls valid. What keeps the corpus from telling the two fixes apart is not its maximum depth, since `bad-741`'s payload sits at 130, but that nothing in it sits at 128, the one depth where the two readings disagree: a scalar leaf inside 128 open containers reads as 129 to a per-value counter and 128 to a per-container one, and at 129 both reject. That boundary is pinned in this parser's own tests instead. The revision's other half, the encoding rules, needed no change here: this checker rejected ill-formed UTF-8, CESU-8, overlong forms and unpaired surrogate escapes from the first build.

**suiteRevision 6: 153/153** (36/36, 117/117) after implementing the noncharacter exclusion. The unchanged revision-5 build scored **151/153**. Two of the four new vectors are the depth-boundary pair, `ok-036` and `bad-742`, and the container-branch counter already handled both; the other two, `bad-743` and `bad-744`, carry Unicode noncharacters in a vocabulary label and a payload value, which this checker admitted. It admitted them because the earlier text scoped its MUST to well-formed sequences of Unicode scalar values, and a noncharacter is one; the revision widens the rule to the RFC 7493 section 2.1 exclusion the strict-I-JSON label had always implied. **This one is directed, and more so than revision 2 was:** the rule was written and the vectors named before this checker ran, so what it demonstrates is that the corrected rule is implementable from the text, not that an independent reader found it.

[PARITY-REPORT.md](PARITY-REPORT.md) carries the scores, the interpretation decisions the spec text forced, the four formerly-open corners and how each was closed, and the from-spec discipline attestation listing exactly what was and was not read for each revision. [NOTES.md](NOTES.md) compares the vendored spec against the branch-head spec.

No dependency on the reference implementation: this crate carries its own strict I-JSON parser, RFC 8785 canonicalization with ECMAScript number formatting, RFC 6962 domain-separated Merkle root over DSSE PAE bytes, run-binding derivation, and Ed25519 tier verification against the suite's seed-derived test key.
Expand All @@ -18,13 +20,13 @@ No dependency on the reference implementation: this crate carries its own strict

```
git clone https://github.com/astrogilda/aee-conformance
git -C aee-conformance checkout ea25a1e218e94843e018dffc0eae4f3fcab1749e
git -C aee-conformance checkout 7098f4e6b7d04c8394969ed81b4025d4d9038324
cargo run --locked --release -- aee-conformance/vectors --json fresh.json
python3 scripts/compare-report.py fresh.json reports/suite-revision-5.json
python3 scripts/compare-report.py fresh.json reports/suite-revision-6.json
```

The checkout is pinned deliberately. `main` moves, and a later revision would run
a different corpus against the 149/149 claim on this page, which is the one thing
a different corpus against the 153/153 claim on this page, which is the one thing
a reproduction recipe must not do quietly. Earlier revisions are reproducible the
same way by taking their suite pin and checker commit from
[`reports/INDEX.json`](reports/INDEX.json).
Expand Down
16 changes: 15 additions & 1 deletion reports/INDEX.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,22 @@
"vectors": 149,
"acceptParity": "35/35",
"rejectParity": "114/114",
"continuouslyVerified": false,
"note": "Revision 5 pins the two things the v0.6 text left open, encoding and nesting depth. The unchanged revision-3 checker scored 148/149 against it, missing only bad-741: this implementation had picked 256 where the text now says 128, and counted depth per parsed value where the text now says per open container. Both halves are fixed here. The constant alone would also have scored 149/149, which is why the boundary the corpus does not reach is pinned in the parser's own tests instead.",
"checkerCommit": "88c37d19c63854341778fd40fb31108d74975ce1"
},
{
"file": "suite-revision-6.json",
"reportSha256": "sha256:0362e1768849e56f2e2df3b5ce188ffcead28cebfc41511ec03530ee7552fd87",
"checkerSourceDigest": "sha256:1c3e2e7843fc021e20c33d3bdc726fbb704ad0438654a0444fa7a06d6613aaba",
"suiteRevision": 6,
"suiteCommit": "7098f4e6b7d04c8394969ed81b4025d4d9038324",
"specDigest": "sha256:606215de629d5f5eda9e62826cf511733b1ec0b9ca8ed07662a5c8bfe181d0b9",
"vectors": 153,
"acceptParity": "36/36",
"rejectParity": "117/117",
"continuouslyVerified": true,
"note": "Revision 5 pins the two things the v0.6 text left open, encoding and nesting depth. The unchanged revision-3 checker scored 148/149 against it, missing only bad-741: this implementation had picked 256 where the text now says 128, and counted depth per parsed value where the text now says per open container. Both halves are fixed here. The constant alone would also have scored 149/149, which is why the boundary the corpus does not reach is pinned in the parser's own tests instead."
"note": "Revision 6 adds the depth boundary pair and the noncharacter pair. The unchanged revision-5 checker scored 151/153 against it: ok-036 and bad-742 passed on the container-branch counter already in place, and bad-743 and bad-744 did not, because this checker did not implement the RFC 7493 section 2.1 noncharacter exclusion the revision makes normative. A directed fix, not a blind run: the rule was stated before this checker was run."
}
]
}
Loading