RUE-1045: ADR-0072 — runtime benchmarking anchored by a Rue static site generator

[DorianListens]

Adds ADR-0072 (status: proposal), the design document for the Runtime performance benchmarking project, with a first concrete workload: gazette, a static site generator written in Rue that builds the live rue-lang.dev corpus and is measured tool-vs-tool against pinned Zola and Hugo.

Key decisions:

• One workload, two roles. Gazette is the first runtime benchmark and also joins the maintained compiler scaling curve, so it feeds both the compile-time and runtime series.
• Tool-vs-tool first. The headline cross-language comparison uses production peers (Zola v0.21.0 already pinned in-repo; Hugo to be pinned the same way) on the same corpus, aiming for order-of-magnitude placement rather than microsecond precision. The translation tracks in RUE-1048 stay in future scope, untouched.
• Live corpus, recorded identity. The workload builds the live site content as website/build.sh assembles it — no frozen snapshot. Cross-tool comparison is internally valid because all tools build the identical corpus within a run; every observation records corpus identity (tree hash, file count, bytes), corpus changes are annotated chart events, and the pinned peers act as a drift control — the gazette-to-peer ratio is the drift-immune longitudinal signal. Freezing returns only as a prerequisite of a future regression ratchet.
• v1 equivalence subset. Front matter, a corpus-scoped Markdown subset, shortcodes, template inheritance, section indexes, RSS, output writing. Syntax highlighting, search indexing, and minification are disabled for all tools and listed as the first parity expansions.
• Work-equivalence validation. Per-run output determinism within each tool, identical output file sets across tools, spot goldens on stable pages; wrong output fails regardless of speed.
• Infrastructure reuse. A runtime mode in rue-bench with a performance/runtime.toml manifest on the ADR-0067 suite/epoch model; observations append to the performance-data-v1 orphan branch; the website derives comparison tables and time series at build, per the plan in RUE-1049.
• Std gaps on the canonical path. Directory enumeration (RUE-1481) and substring operations (RUE-1482) land in std as ordinary reviewed work, not benchmark shims.

The remaining phase issues are RUE-1483 (gazette libraries), RUE-1484 (gazette + live-corpus fixture prep), RUE-1485 (cross-tool comparison), and RUE-1046 (harness mode). The ADR index in docs/designs/README.md is regenerated via scripts/validate-adrs.py --write.

Fixes RUE-1045

[DorianListens]

Adversarial review — ADR-0072 + RUE-1045/1046/1047/1048/1049/1481–1485

This is a strong, unusually self-aware ADR — the corpus-drift argument (Decision 2) and the event-driven peer cadence (Decision 9) are both better than the obvious alternatives, and the Negative section is honest about most of what it costs. RUE-1481 and RUE-1482 are correctly scoped and their premises check out against the tree.

I read the ADR against the ten Linear issues, the four ADRs it builds on (0057/0067/0070/0071), the actual corpus and templates, performance/*.toml, and the collection workflows. Below, findings ordered by how much they'd change the document. Verification commands/paths inline so you can check rather than trust.

---

1. The ADR-0067 stall gate turns "intended dogfooding pressure" into a repo-wide hard stop

ADR-0072:210 and :389 say a page using constructs outside the equivalence subset "breaks the runtime leg of performance collection," framed as "intended dogfooding pressure." Decision 9 says regressions get "maintainer triage and no hard CI gate in v1."

ADR-0067 has a second gate that isn't about regressions at all:

A series that has stopped advancing — more than five merged trunk commits with no new plotted point on some platform — fails every pull request until it is resolved… Neither has a bypass.

If the runtime series sits inside that scope, then one blog post containing a Markdown table stops the runtime series, and five merged commits later every PR in the repository is blocked — with no bypass, and with a remedy that isn't the one ADR-0067 anticipates. ADR-0067 offers "declaring the next epoch, which needs no baseline and so is a small manifest change"; here the fix is extending a Markdown parser written in Rue.

"No hard CI gate in v1" and "inherits ADR-0067's machinery" can't both hold. Please state explicitly whether the runtime series is in or out of the stall gate, and if out, by what mechanism (separate suite? advisory record kind, like ADR-0068's incremental suite, which "intentionally does not append to ADR-0067's raw history"?).

2. "That compile doubles as a recorded fresh-process compile observation" is a free lunch the protocol forbids

Decision 7 (:280) puts gazette's compile-time measurement in performance/scaling.toml — the weekly performance-scaling.yml, with samples = 3, workers = ["one","two","four","eight","automatic"], compiler_build_profile = "release_thin_lto", boundary = "fresh_source_to_native_v1".

Decision 9 (:307–312) then has the per-push runtime leg's single opportunistic compile also become "a recorded fresh-process compile observation."

Those are incompatible. ADR-0067 §5 pins sampling policy in the epoch, and §"The product boundary" is categorical: "Validation refuses to append a run whose pinned components do not match its epoch." A one-sample, single-worker-setting compile from the per-push job satisfies no declared epoch in either manifest.toml or scaling.toml. Either name the series it enters and declare its epoch, or drop the claim — as written it reads as a cost argument for per-push collection that the measurement protocol won't honor.

Two related gaps in Decision 7: scaling.toml is at revision = 3, so adding gazette is a revision-4 event (worth saying, given ADR-0067 §4's rules about what a revision bump means); and performance-scaling.yml has timeout-minutes: 60 for the current 4 workloads × 3 samples × 5 worker settings — gazette is intended to be a large program, and that budget deserves a sentence.

3. No parallelism parity — the biggest single threat to the headline number

Zola renders pages in parallel via rayon; Hugo is goroutine-parallel by design. Rue has no concurrency: there is no thread/sync/atomic module in std/ and nothing there spawns. The standard ubuntu-24.04 hosted runner has 4 vCPUs.

So the default configuration measures single-threaded Rue against 4-way-parallel peers, and up to ~4× of whatever ratio gets published is thread count rather than codegen, language, or std quality. Decision 3, Decision 5, and RUE-1485 don't mention threads at all.

For an order-of-magnitude claim this may well be acceptable — but it has to be a decision, not an omission, because it's the first thing a skeptical reader will raise. Concretely: pin the peers single-threaded (RAYON_NUM_THREADS=1, GOMAXPROCS=1) and make that the primary ratio, or publish both single- and multi-threaded peer numbers, and either way record thread policy in the epoch next to the parity config.

4. The fairness section covers disabled features but not the specialization asymmetry

:383 frames the "unfair to Zola/Hugo" critique entirely as disabled features. The larger asymmetry is structural: gazette implements exactly the subset this corpus uses and by design "reject[s] constructs outside their documented subset" (:207), while Zola and Hugo run general engines and pay for generality even with features off.

Concretely, from the corpus: of 1,225 shortcode invocations, 1,224 are rule() and exactly one is preview_feature(). website/templates/shortcodes/rule.html is a single line. gazette can render rule() as a hardcoded interpolation while Zola pays generic Tera shortcode rendering 1,224 times — and the output is correct, so determinism checks, file-set equality, and spot goldens (Decision 4) cannot see it. That's exactly the wrong-but-fast class Decision 4 exists to close, in the one form it's blind to.

The defensible headline claim is "a corpus-specialized Rue program vs. general-purpose tools," not "Rue vs. Rust vs. Go." I'd add that as a stated Negative and as a required caption on the published comparison table, alongside the disabled-feature list.

5. RUE-1483's template-engine scope is smaller than the templates it must cover

RUE-1483 scopes the engine to "variable interpolation, conditionals, loops over pages/sections, template inheritance (base/child blocks), HTML escaping." What the 17 templates actually use (grep -rhoE '\| *[a-z_]+|get_[a-z_]+\(' website/templates/):

• globals: get_url( ×22, get_section( ×6, paginator ×6, load_data ×1
• filters: safe ×11, default ×11, date ×4, pluralize ×2, plus upper, lower, truncate, striptags, sort, slice, join

Filters and globals are absent from the scope, and the very shortcode template the ADR names uses two of them ({{ adr | lower }}, {{ adr | upper }} in preview_feature.html). Two consequences that need decisions:

• paginator is used by production templates, yet Decision 3 (:202) lists pagination as out of scope for v1. So the benchmark's Zola port must drop a feature the live site uses — fine, but say so.
• load_data is website/templates/performance.html, reading static/performance-data.json. The ADR never says whether the performance dashboard page is in the corpus. Excluded → "builds the live corpus" needs a carve-out. Included → gazette needs JSON loading (std/json.rue exists) plus that page's template.

6. "Live corpus" is content-only — templates and config are not live, and there are four sets, not three

:175 introduces a benchmark Zola template/config set separate from the production website/templates/ and website/config.toml (which today has minify_html = true, build_search_index = true, highlight_code = true, extra_syntaxes_and_themes = ["syntaxes"] — all disabled for parity). So the workload is: live content + benchmark templates + benchmark config. The Negative at :392 counts "three template sets"; it's really four, and the fourth (production) is the one whose drift away from the ports quietly erodes the "live" framing.

Worth answering directly in the ADR: why port Zola templates at all, rather than run the site's own 17 templates minus the dashboard page? A port that's simpler than production flatters Zola or gazette depending on which way it simplifies, and it's a maintenance surface the ADR is otherwise trying to avoid.

7. Flagging on the raw runtime series will almost never have a clean trailing window

ADR-0067 §5's flagging rule compares the current median against a trailing-window median, and corpus identity changes break that window's comparability. From GitHub history: 57 commits touched docs/spec/src in the last month, against a trunk rate of ≥100 commits/week — roughly one corpus change per 7–8 trunk commits. With per-push runtime collection, the raw gazette series takes a discontinuity about every 7 points, so nearly every trailing window spans one.

Decision 2 is right that the peer ratio is the drift-immune signal. But Decision 9 still promises "runtime regressions surface attached to the compiler change that caused them" against the raw series, and Open Question 4 leaves the threshold open without noting this. The ADR should say explicitly that flagging operates on corpus-identity-matched segments (or on the ratio), not on the raw median — otherwise the per-push regression story doesn't actually work. (The good news: since the peer leg fires on corpus change, each corpus-identity segment has a same-run peer denominator, so segment-scoped ratio flagging is well defined.)

8. Recording-instead-of-pinning the corpus is a genuine deviation from ADR-0067, not reuse

:242: "corpus identity is recorded per observation rather than pinned." ADR-0067 §4 is the opposite rule:

A workload edit changes its source identity, fails validation, and therefore requires a new suite revision — a workload can never silently reset its own baseline while the headline continues.

and lists "each workload's own sources" among what stays pinned, specifically because they are not the product. The std precedent the ADR leans on doesn't transfer: std is unpinned because std is the product; the website corpus is an input.

I think the decision is right — the RUE-1256 lesson (pinning an input that moves stops the series rather than describing it) applies cleanly here. But it introduces a category ADR-0067's validator has no concept of ("recorded input"), so the ADR's claim that it "reuses that machinery rather than inventing a parallel system" understates what's happening. Please say plainly that ADR-0067's identity rule is amended for the runtime suite, and what schema addition that implies.

9. Factual corrections

• ~780 KB is wrong. The assembled corpus is 525,479 bytes (513 KiB) of Markdown: website/content 88,118 B / 25 files + docs/spec/src 437,361 B / 71 files. 784 KiB is du -sk block-rounded disk usage; du -sk --apparent-size gives 515 KiB. The figure is ~49% high, and it appears at :74, :151, and in RUE-1484's description.
• Everything else in that paragraph checks out: 96 files, 25 + 71 split, 17 templates (recursive, including the two shortcode templates), ~1,200 shortcode invocations (1,225 exactly).
• "1,200 shortcode invocations" is accurate in aggregate but is 1,224 rule() + 1 preview_feature() — material to Finding 4, and to RUE-1483's "correctness bar is the corpus" claim.
• "Zola … runs in CI today" holds, but indirectly — nothing in .github/workflows/ mentions zola; deploy-website.yml:75 runs website/build.sh, which invokes ./zola build. Fine as written; noting the chain since the claim carries weight in Decision 1's justification.
• Hardware counters (Decision 5, "join where the pinned runners support them"): hosted GitHub runners don't expose PMU counters — perf_event access isn't available in that virtualized environment — so on the ADR-0071 regime this is effectively "never," not "portable extension." Given RUE-1046 argues counters are the low-noise way to catch codegen regressions, and Finding 7 shows wall-time flagging is already fighting corpus drift, I'd state plainly that counters are gated on moving to controlled hardware.

10. Two of RUE-1045's five open questions are dropped rather than answered

RUE-1045 asked the ADR to address five open questions. Regression detection, cadence, and peer pinning are all answered well. Two are not:

• "Relationship to the Agent Outcomes Gauntlet (RUE-487) — shared prompts, shared corpus, or fully independent?" RUE-487 is a relatedTo link on RUE-1045 and appears nowhere in ADR-0072 — not in a Decision, not in relates, not carried forward as an Open Question. It should be explicitly deferred rather than silently dropped.
• "Repository layout: in-tree with the compiler vs. separate repo." Answered implicitly (examples/gazette/, performance/runtime.toml) but never stated as a decision with reasoning.

RUE-1045 also specifies sequencing as "harness first, then incremental corpus expansion, then peer translations," which this ADR inverts (harness is Phase 4). That's defensible but deserves a sentence, because of Finding 11.

11. Sequencing: the ADR's own "ships as soon as" promise is gated behind three phases of language work

:299: "The longitudinal Rue-only view ships as soon as the harness produces data and does not wait for the cross-tool comparison." RUE-1049 makes the same promise. But under Implementation Phases the harness is Phase 4, behind read_dir + recursive walk, substring/split/replace/join, a TOML subset parser, a Markdown subset renderer, a template engine with inheritance, and the SSG itself. "As soon as" is a long way out — and the project goal it delays longest is precisely "a trusted system to iterate against while improving performance."

Concrete alternative worth an explicit rejection if you don't take it: stand up Phase 4 first against a workload that exists today. examples/wordfreq, examples/jsonfmt, examples/tinydb, examples/ruelex, and examples/sudoku are all already compiled in CI and already string/allocation/map-bound. rue-bench runtime mode + performance/runtime.toml against wordfreq needs zero std work and starts the longitudinal series immediately; gazette then lands into working infrastructure instead of being a prerequisite for it. That also de-risks Phases 1–3, since the std work would land with a live runtime series already able to show what it cost.

This is the alternative I'd most want to see in an "Alternatives Considered" section (8 of 73 ADRs have one, so no convention is being broken — but this document has three live alternatives worth recording: harness-first-on-existing-example, frozen snapshot corpus, and translation-first per RUE-1048; the latter two are argued away well in prose already).

12. Scale-variant validity (for RUE-1484)

:178, "deterministic 10x and 100x scale variants, generated by path-prefixed duplication." At 100× that's ~9,600 pages / ~52 MB, hashed in full per sample per tool. Two wrinkles worth pre-empting:

• The corpus's 15 @/… internal links resolve to the original copies, so cross-reference resolution stays constant while per-page work scales 100×. The curve is therefore not a site-size curve, and will be read as one unless the ADR says otherwise.
• Duplicated _index.md sections carry identical weight, title, and dates, so section-index and RSS ordering fall through to slug tie-breaks. Deterministic, but tool-specifically deterministic — worth asserting in the validation rules rather than assuming.

13. Parity/determinism traps to pre-empt in RUE-1485

• Hugo's default RSS template emits <lastBuildDate> from build time. With it, Hugo fails Decision 4's per-run byte-identical check on literally every run. The Hugo port must strip it (same class: .Site.LastChange, <generator>).
• "Identical output file sets across tools" needs a documented allowlist, not just an assertion: Zola emits sitemap.xml and copies static/; Hugo emits sitemap.xml and defaults RSS to index.xml (config'd to rss.xml here); gazette must match, or the criterion needs stated exclusions.

14. Verified-clean, for the record

• docs/designs/README.md index row is correct; scripts/validate-adrs.py passes on this branch (73 records) and --write is idempotent (clean tree after).
• ADR-0065 is accepted with empty implemented: ✓. Slices are preview-gated ✓ (PreviewFeature::Slices, crates/rue-error/src/lib.rs:760). -O3 is a real flag ✓ (crates/rue/src/main.rs:552). std/fs.rue:21-22 does defer getdents64 as "the largest remaining chunk" ✓. std/strings.rue is byte-only — find/rfind/count all take u8 ✓. StrBuf.contains is bool-returning with no index counterpart (std/strbuf.rue:349) ✓. RUE-1481 and RUE-1482's premises are accurate as written.
• Decision 7's "alongside ruelex, mosaic, harbor, and lattice" ✓ matches performance/scaling.toml. Minor: meridian (266 files / 1.5 MB) and caldera (803 files / 4.6 MB) are both larger than lattice (131 files / 557 KB) and are not on the curve — so gazette joins at the small-to-middle end. Fine, just don't let it read as extending the curve's reach.
• performance/runtime.toml as a fourth manifest is consistent with existing practice (manifest/scaling/incremental) ✓. Riding the existing single-writer performance-collection concurrency group preserves the one-writer property ✓.
• Implementation note for Phase 6, not an ADR change: website/build.sh currently passes only --manifest performance/manifest.toml to rue-bench derive, so the runtime record kind will need that call extended.

---

What I'd want changed before acceptance

Findings 1, 2, 3, 5, 9, and 10 — each is either an internal contradiction, a claim the surrounding machinery won't honor, a number that's wrong, or a charter question dropped without a decision. 4, 7, 8, and 11 are substantive enough that I'd want at least a paragraph each, since they change how the results must be presented and how long the project takes to produce its first useful output. 12 and 13 are fine as follow-ups into RUE-1484/1485.

None of that touches the core judgment, which I think is right: an SSG is a genuinely good first realistic workload, tool-vs-tool before translation is the right sequencing, and refusing to freeze a corpus that changes daily is the correct call for a comparative observatory.

---

Generated by Claude Code

[DorianListens]

All 14 findings addressed in b01f74b; the six blocking ones are all taken as written. I re-verified the load-bearing claims before editing (corpus byte count, shortcode split, the ADR-0067 stall-gate text, scaling.toml revision/policy, the template filter census) — every one checks out. Disposition per finding:

1 — Stall gate. Taken. Decision 9 now states the runtime series is its own record kind sitting outside ADR-0067's required-CI gates, including the no-bypass stall gate, with the reasoning spelled out (that gate presumes an epoch-declaration remedy; this series' stall remedy is parser work). A stalled runtime series is a dashboard staleness flag and triage item, never a repo-wide block — explicitly the ADR-0068 advisory posture.

2 — Free-lunch compile observation. Taken; the claim is dropped rather than papered over. Decision 9's parenthetical now says the compile is shared cost only, and Decision 7 states the scaling curve is the only recorded compile-time series for gazette, that a one-sample compile matches no declared epoch and validation would rightly refuse it, that joining scaling.toml is a suite-revision event (currently revision 3), and that the workflow's time budget gets re-checked.

3 — Thread parity. Taken, as a decision: the primary published ratio pins the peers single-threaded (RAYON_NUM_THREADS=1, GOMAXPROCS=1), with the same rationale as ADR-0071's one-worker primary — expose per-unit work, not core count. The peers' default-parallel configuration is measured and published as a labeled secondary row so the real-tool number is never hidden. Thread policy lives in the epoch.

4 — Specialization asymmetry. Taken, in both directions: a fairness rule on gazette (shortcodes/filters/templates must go through general engine paths; corpus-specific fast paths are out of bounds — the 1,224/1 rule skew is cited in the ADR as the motivating case) and a required caption on every published comparison ("a corpus-specialized Rue program against general-purpose tools"), plus a Negative acknowledging the asymmetry is structural and can't be validated away.

5 — Template-engine scope. Taken. The in-scope subset now enumerates the filter/helper set the ports actually use (safe, default, date, upper, lower, truncate, striptags, pluralize, sort, slice, join, URL/section lookups), and both live-site conflicts are explicit carve-outs: the ports omit paginated views, and the dashboard page is excluded because load_data of derived benchmark data would make the benchmark an input to its own workload. RUE-1483's scope is updated to match.

6 — Four sets, why port at all. Taken. The carve-outs answer the "why port" question and the ADR now says so: each port is the production template set minus the excluded pages and features, kept as close to production as the subset allows. The Negative now counts four template/config sets and asks website-template changes to include a port review.

7 — Flagging across discontinuities. Taken. Decision 9 now states raw medians are compared only within corpus-identity-matched segments, never across a change, and that the cross-segment signal is the per-segment peer ratio — well-defined because the peer leg fires on exactly the corpus-change runs. Open Question 4 is reworded to be about segment/ratio thresholds.

8 — Amendment, not reuse. Taken. Decision 2 now names the deviation: a third input category ("recorded input") that ADR-0067's compile-time suites don't have, named explicitly in the runtime record schema, with ADR-0067's rules unchanged for compile-time suites.

9 — Facts. All corrected: 513 KiB / 525,479 bytes in both ADR locations and RUE-1484; shortcode split stated (1,224 rule / 1 preview_feature); hardware counters now "gated on a future controlled-hardware epoch" since hosted runners expose no PMU — wall time and RSS are the portable core.

10 — Dropped charter questions. Taken. RUE-487 is in relates and explicitly deferred as Open Question 5; repository layout is a stated rejection in the new Alternatives Considered section; the sequencing inversion is resolved by taking finding 11.

11 — Harness-first. Adopted, not just recorded: phases are reordered so Phase 1 stands rue-bench runtime mode up on existing CI-compiled examples (wordfreq at minimum, optionally jsonfmt/ruelex), which needs zero std work and starts the longitudinal series and publication path immediately; gazette lands into working infrastructure at Phase 4. An Alternatives Considered section now records the three live alternatives (frozen corpus, gazette-first, translation-first) plus the separate-repo rejection. Linear phase issues are renumbered to match.

12 — Scale-variant validity. Taken into both the ADR (Decision 2 now notes the curve is a page-count curve, not a site-shape curve, since @/… links resolve to originals, and requires validation to assert per-tool ordering tie-break determinism) and RUE-1484.

13 — Parity/determinism traps. Taken into Decision 4 (documented file-set allowlist for sitemap/feed-name/static differences; ports strip build-time-varying feed output, Hugo lastBuildDate/generator named) and RUE-1485 in full (including .Site.LastChange).

14 — Verified-clean notes. The build.sh derive-call extension is recorded on RUE-1046 so it isn't lost, and RUE-1484 now notes gazette joins the scaling curve at the small-to-middle end (meridian/caldera are larger and not on it), so the rung doesn't read as extending the curve's reach.

One judgment call to flag rather than bury: finding 11's adoption means RUE-1046 is now the first implementation issue and its scope statement ("first implementation step") is true again. If gazette-first was actually preferred despite the review's argument, that's a one-commit revert — but the harness-first case looks correct to me on the project's own goal of "a trusted system to iterate against while improving performance."

---

Generated by Claude Code

[steveklabnik]

I reviewed the revised ADR at b01f74b, including both discussion comments and the disposition of the earlier 14 findings. That rewrite materially improves the proposal, but I think five design gaps remain before acceptance:

1. Output validation can still publish wrong-but-fast results. Decision 4 validates determinism, filenames, non-empty pages, and a few spot goldens. A renderer that consistently drops body content from every un-goldened page passes all of those checks. The design needs an exhaustive semantic oracle for every page—e.g. normalized metadata, headings, text, links, shortcode results, section membership, and feed ordering—checked outside the timed window. Per-tool spot goldens are still useful for rendered-form coverage, but they are not a complete work-equivalence oracle.

2. The recorded corpus identity does not clearly cover the complete workload input. Decision 2 describes the identity as the Markdown content tree, while Decision 4 includes static passthrough in the file-set contract. website/static currently contains 35 files and roughly 1.2 MiB, more bytes than the Markdown corpus. A static change would therefore alter copied work/output without necessarily changing the stated identity or triggering peer remeasurement. Either the recorded identity and event detector must cover every live input—including static assets—or static copying should be excluded equally from every tool's measured boundary.

3. The peer ratio is corpus-normalized, not drift-immune. Pinned peers still move with hosted-runner variation, not only corpus changes. With the event-only cadence, one noisy peer observation becomes the denominator for an entire corpus segment, so cross-segment ratio flagging can inherit that noise. I would either run at least one cheap, single-threaded 1× peer canary alongside every Gazette observation, or explicitly keep cross-segment ratios advisory and stop describing them as drift-immune. The latter may be enough for the stated order-of-magnitude objective.

4. The runtime platform and calibration contract is incomplete. Decision 5 says runtime measurement inherits the compiler suites' pinned GitHub Actions regime and calibration, while Decision 6 makes directory enumeration Linux-first. The current per-push compiler collector covers x86-64 Linux, AArch64 Linux, and AArch64 macOS; runtime sample counts also cannot inherit calibration from unrelated compiler workloads. Please name the exact v1 runtime platform matrix and require runtime-specific, per-platform calibration before using regression flags.

5. Harness-first now conflicts with “Gazette is the first runtime benchmark.” Decision 1 says Gazette is first, while Phase 1 begins a durable runtime series with “wordfreq at minimum, optionally jsonfmt or ruelex.” That phase should declare an exact initial workload, a meaningful fixed fixture, arguments, correctness oracle, and whether the resulting series is permanent. Reusing the current tiny CLI fixture would mostly measure process startup and is not enough to establish a useful runtime trend.

The overall direction still looks good: an SSG is a strong realistic workload, harness-first is the right implementation order, single-threaded peers as the primary comparison is defensible, and the revised ADR is now honest about specialization and corpus discontinuities. CI is green; these findings are about completing the measurement contract rather than implementation failures.

[DorianListens]

All five gaps addressed in 1d935e7. Per-finding:

1 — Semantic oracle. Taken. Decision 4 gains a fourth validation layer: outside the timed window, a normalized extraction (front-matter metadata, heading tree, visible text, link targets, shortcode expansion results, section membership, feed entry ordering) is computed for every emitted page and compared across all three tools. The consistent-body-drop case you describe now fails by construction, and spot goldens are re-scoped to what the oracle deliberately ignores — rendered markup-level form.

2 — Complete input identity. Taken, on the include side rather than the exclude side: static passthrough is real SSG work all three tools perform, so it stays in the measured boundary, and the recorded identity is now a tree hash over everything the tools consume — content tree, static assets (verified at 33 files / ~1.2 MiB on this branch, more bytes than the Markdown, as you said), and the versioned template-port/parity-config revision. The peer-leg event detector triggers on that full fixture identity, so no input class can change the job invisibly.

3 — Ratio honesty. Both of your options, actually: the term "drift-immune" is gone everywhere in favor of corpus-normalized, with the denominator's runner noise stated explicitly; and a per-run canary — one single-threaded Zola build of the 1× corpus alongside every gazette observation — gives every observation a same-run denominator so no segment leans on a stale or singleton peer sample. The full peer matrix (Hugo, scale variants, default-parallel row) stays event-driven, and the canary sits under the same cost safety valve as the scale variants. Even with the canary, cross-segment ratio flags are explicitly advisory until finding 4's calibration exists.

4 — Platform matrix and calibration. Taken. Decision 5 now names the exact v1 matrix: x86_64-linux on the pinned ubuntu-24.04 regime, aarch64-linux joining once the Phase 2 std work is CI-verified there, macOS deferred behind the @syscall error-detection gap. And it states the principle you implied: dispersion is a property of the workload, so each runtime workload is calibrated per platform from its own repeated samples — nothing inherited from compiler-workload calibration — and regression flags on a platform are advisory until that calibration exists.

5 — Phase 1 workload declaration. Taken. Decision 1 is reworded — gazette is the anchor workload and first cross-tool benchmark, not the first measured program — and the phase plan now declares Phase 1's initial workload exactly: wordfreq over a large deterministic text fixture generated by a checked-in seeded generator (seed and generator revision pinned in performance/runtime.toml, generated identity recorded per observation), sized on the order of tens of MiB so it measures word counting and map pressure rather than process startup, with fixed arguments and a byte-exact golden output oracle. The series is declared permanent — it joins RUE-1047's corpus as a string/map-bound workload rather than being scaffolding — and jsonfmt/ruelex may join later only with their own declared fixtures and oracles.

Linear is synced (RUE-1046 carries the full Phase 1 declaration; RUE-1484/1485 carry the oracle, identity, and canary changes). Validator passes; CI running on the push.

---

Generated by Claude Code