Skip to content

Commit f8bd3a7

Browse files
Rul1anclaude
andauthored
Reject the Unicode noncharacters, and record suiteRevision 6 at 153/153 (#4)
* fix(json): reject the Unicode noncharacters in every string literal suiteRevision 6 makes the RFC 7493 section 2.1 exclusion normative: the sixty-six noncharacters, U+FDD0 through U+FDEF and U+nFFFE and U+nFFFF in each of the seventeen planes, are malformed wherever a string literal appears, at any depth and in member-name as well as value position. This checker admitted them, and the reason is worth recording because it was not an oversight. The earlier text scoped its MUST to string literals being well-formed sequences of Unicode scalar values, and a noncharacter is a scalar value, so the narrower rule was implemented faithfully. What the revision changes is the rule, not the reading: the strict-I-JSON label above that MUST had always implied the wider RFC 7493 exclusion, and the revision closes the gap between the label and the rule underneath it. Both routes into a string body are covered, because the exclusion is over code points and a producer reaches them either way: the raw UTF-8 byte, and the escape including a surrogate pair resolving into a plane-end noncharacter. The plane-end pairs differ only in their lowest bit, so one mask covers all thirty-four, and a test walks the whole code space to confirm the predicate selects exactly sixty-six. Refs in-toto/attestation#570. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: record suiteRevision 6 at 153/153 and repin CI The unchanged revision-5 checker scored 151/153. The depth-boundary pair ok-036 and bad-742 passed on the container-branch counter already in place; bad-743 and bad-744 did not, and those two are what the previous commit fixes. The record says plainly that this run is directed, and more so than revision 2 was: the rule was written and the vectors named before this checker ran, so what it shows is that the corrected rule is implementable from the text, not that an outside reader found something. Revision 5 is retired from continuous verification and keeps its checkerCommit, 88c37d1, so its provenance stays checkable after the pin moves. The workflow follows: suite pin, spec pin, the compared report, and the parity string the corpus step greps for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 88c37d1 commit f8bd3a7

6 files changed

Lines changed: 261 additions & 21 deletions

File tree

.github/workflows/conformance.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name: conformance
22

33
# The parity number in the README is only worth what a third party can re-run.
4-
# This pins the suite commit and the spec digest, demands the full 149/149, and
4+
# This pins the suite commit and the spec digest, demands the full 153/153, and
55
# compares a fresh run against the checked-in evidence, so the repository cannot
66
# drift away from its own claim.
77
on:
@@ -14,8 +14,8 @@ permissions:
1414
contents: read
1515

1616
env:
17-
SUITE_COMMIT: ea25a1e218e94843e018dffc0eae4f3fcab1749e
18-
SPEC_DIGEST: 39233b27b7f27b94ed727a3852030c69c7a64e5706b73519848a2b02f244e661
17+
SUITE_COMMIT: 7098f4e6b7d04c8394969ed81b4025d4d9038324
18+
SPEC_DIGEST: 606215de629d5f5eda9e62826cf511733b1ec0b9ca8ed07662a5c8bfe181d0b9
1919

2020
jobs:
2121
parity:
@@ -67,11 +67,11 @@ jobs:
6767
# something is already wrong, so it gets held shut here.
6868
run: python3 tests/test_compare_report.py
6969

70-
- name: Full corpus, 149/149 required
70+
- name: Full corpus, 153/153 required
7171
run: |
7272
set -euo pipefail
7373
cargo run --locked --release -- aee-conformance/vectors --json fresh.json | tee run.txt
74-
grep -q 'parity: accepts 35/35, rejects 114/114' run.txt
74+
grep -q 'parity: accepts 36/36, rejects 117/117' run.txt
7575
7676
- name: Fresh run must match the checked-in evidence
77-
run: python3 scripts/compare-report.py fresh.json reports/suite-revision-5.json
77+
run: python3 scripts/compare-report.py fresh.json reports/suite-revision-6.json

NOTES.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
| 3 | `cf0d5402327ae5a451efebc914852d1c687753ca` | `d3872a02875b2da8de0263e93fb92ca6f5ab0fd75f07ed3762a1b18b0c1712a3` (unchanged) | 140 | 140/140 |
1010
| 4 | `b886c0a` ||| not run here |
1111
| 5 | `ea25a1e218e94843e018dffc0eae4f3fcab1749e` | `39233b27b7f27b94ed727a3852030c69c7a64e5706b73519848a2b02f244e661` | 149 | 149/149 (148/149 unchanged) |
12+
| 6 | `7098f4e6b7d04c8394969ed81b4025d4d9038324` | `606215de629d5f5eda9e62826cf511733b1ec0b9ca8ed07662a5c8bfe181d0b9` | 153 | 153/153 (151/153 unchanged) |
1213

1314
Revision 4 vendored the new encoding and nesting rules into the spec and this
1415
checker was never run against it, so there is no record for it and the table
@@ -45,10 +46,11 @@ reads as 129 to a per-value counter and 128 to a per-container one; at 129 both
4546
reject, and at 128 with an empty-container leaf both accept. The boundary
4647
therefore lives in `src/json.rs`'s own tests.
4748

48-
All five are inspectable and reproducible by hand from those pins; the
49-
revision-5 run at 149/149 is the one re-verified continuously by CI, which
50-
follows the current suite pin. Revision 3 was continuously verified until
51-
revision 5 replaced it and remains reproducible from its own pins.
49+
All six are inspectable and reproducible by hand from those pins; the
50+
revision-6 run at 153/153 is the one re-verified continuously by CI, which
51+
follows the current suite pin. Each earlier record was continuously verified
52+
until the next replaced it, and each remains reproducible from its own pins:
53+
the fixed build still reproduces the revision-5 record exactly.
5254

5355
## Vendored spec vs branch head
5456

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ An independent validity-gate checker for the **Adversarial Execution Evidence (A
1010

1111
**suiteRevision 5: 149/149** (35/35, 114/114) after a parser fix. The unchanged revision-3 build scored **148/149** against it, and the single miss is worth stating plainly because the causation runs the other way this time: the revision pins a nesting bound the earlier text did not state, this checker had picked 256, and the new `bad-741` vector found it. Not blind, and not a case of the two meeting. The bound was only the visible half. The revision also states the counting rule, and this parser had been incrementing per parsed value rather than per open container, which read exactly one level deeper than the spec rule on **every document in the corpus** — all 149 statements and every record payload inside them, measured on the raw bytes so the deliberately ill-formed vectors are covered too — because every deepest path in the corpus ends in a scalar. Changing only the constant scores 149/149 as well, and still rejects a statement at depth 128 that the spec calls valid. What keeps the corpus from telling the two fixes apart is not its maximum depth, since `bad-741`'s payload sits at 130, but that nothing in it sits at 128, the one depth where the two readings disagree: a scalar leaf inside 128 open containers reads as 129 to a per-value counter and 128 to a per-container one, and at 129 both reject. That boundary is pinned in this parser's own tests instead. The revision's other half, the encoding rules, needed no change here: this checker rejected ill-formed UTF-8, CESU-8, overlong forms and unpaired surrogate escapes from the first build.
1212

13+
**suiteRevision 6: 153/153** (36/36, 117/117) after implementing the noncharacter exclusion. The unchanged revision-5 build scored **151/153**. Two of the four new vectors are the depth-boundary pair, `ok-036` and `bad-742`, and the container-branch counter already handled both; the other two, `bad-743` and `bad-744`, carry Unicode noncharacters in a vocabulary label and a payload value, which this checker admitted. It admitted them because the earlier text scoped its MUST to well-formed sequences of Unicode scalar values, and a noncharacter is one; the revision widens the rule to the RFC 7493 section 2.1 exclusion the strict-I-JSON label had always implied. **This one is directed, and more so than revision 2 was:** the rule was written and the vectors named before this checker ran, so what it demonstrates is that the corrected rule is implementable from the text, not that an independent reader found it.
14+
1315
[PARITY-REPORT.md](PARITY-REPORT.md) carries the scores, the interpretation decisions the spec text forced, the four formerly-open corners and how each was closed, and the from-spec discipline attestation listing exactly what was and was not read for each revision. [NOTES.md](NOTES.md) compares the vendored spec against the branch-head spec.
1416

1517
No dependency on the reference implementation: this crate carries its own strict I-JSON parser, RFC 8785 canonicalization with ECMAScript number formatting, RFC 6962 domain-separated Merkle root over DSSE PAE bytes, run-binding derivation, and Ed25519 tier verification against the suite's seed-derived test key.
@@ -18,13 +20,13 @@ No dependency on the reference implementation: this crate carries its own strict
1820

1921
```
2022
git clone https://github.com/astrogilda/aee-conformance
21-
git -C aee-conformance checkout ea25a1e218e94843e018dffc0eae4f3fcab1749e
23+
git -C aee-conformance checkout 7098f4e6b7d04c8394969ed81b4025d4d9038324
2224
cargo run --locked --release -- aee-conformance/vectors --json fresh.json
23-
python3 scripts/compare-report.py fresh.json reports/suite-revision-5.json
25+
python3 scripts/compare-report.py fresh.json reports/suite-revision-6.json
2426
```
2527

2628
The checkout is pinned deliberately. `main` moves, and a later revision would run
27-
a different corpus against the 149/149 claim on this page, which is the one thing
29+
a different corpus against the 153/153 claim on this page, which is the one thing
2830
a reproduction recipe must not do quietly. Earlier revisions are reproducible the
2931
same way by taking their suite pin and checker commit from
3032
[`reports/INDEX.json`](reports/INDEX.json).

reports/INDEX.json

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,22 @@
6666
"vectors": 149,
6767
"acceptParity": "35/35",
6868
"rejectParity": "114/114",
69+
"continuouslyVerified": false,
70+
"note": "Revision 5 pins the two things the v0.6 text left open, encoding and nesting depth. The unchanged revision-3 checker scored 148/149 against it, missing only bad-741: this implementation had picked 256 where the text now says 128, and counted depth per parsed value where the text now says per open container. Both halves are fixed here. The constant alone would also have scored 149/149, which is why the boundary the corpus does not reach is pinned in the parser's own tests instead.",
71+
"checkerCommit": "88c37d19c63854341778fd40fb31108d74975ce1"
72+
},
73+
{
74+
"file": "suite-revision-6.json",
75+
"reportSha256": "sha256:0362e1768849e56f2e2df3b5ce188ffcead28cebfc41511ec03530ee7552fd87",
76+
"checkerSourceDigest": "sha256:1c3e2e7843fc021e20c33d3bdc726fbb704ad0438654a0444fa7a06d6613aaba",
77+
"suiteRevision": 6,
78+
"suiteCommit": "7098f4e6b7d04c8394969ed81b4025d4d9038324",
79+
"specDigest": "sha256:606215de629d5f5eda9e62826cf511733b1ec0b9ca8ed07662a5c8bfe181d0b9",
80+
"vectors": 153,
81+
"acceptParity": "36/36",
82+
"rejectParity": "117/117",
6983
"continuouslyVerified": true,
70-
"note": "Revision 5 pins the two things the v0.6 text left open, encoding and nesting depth. The unchanged revision-3 checker scored 148/149 against it, missing only bad-741: this implementation had picked 256 where the text now says 128, and counted depth per parsed value where the text now says per open container. Both halves are fixed here. The constant alone would also have scored 149/149, which is why the boundary the corpus does not reach is pinned in the parser's own tests instead."
84+
"note": "Revision 6 adds the depth boundary pair and the noncharacter pair. The unchanged revision-5 checker scored 151/153 against it: ok-036 and bad-742 passed on the container-branch counter already in place, and bad-743 and bad-744 did not, because this checker did not implement the RFC 7493 section 2.1 noncharacter exclusion the revision makes normative. A directed fix, not a blind run: the rule was stated before this checker was run."
7185
}
7286
]
7387
}

0 commit comments

Comments
 (0)