Skip to content

PERF: Cut the HTML and TextFormatter tree-walk overhead#61

Merged
gschlager merged 3 commits into
mainfrom
perf-html-textformatter
Jul 9, 2026
Merged

PERF: Cut the HTML and TextFormatter tree-walk overhead#61
gschlager merged 3 commits into
mainfrom
perf-html-textformatter

Conversation

@gschlager

@gschlager gschlager commented Jul 9, 2026

Copy link
Copy Markdown
Member

Follow-up to #58/#59/#60, covering the two remaining parsers. New bench variants separate nokogiri's C parse from the Ruby tree walk by feeding the parsers pre-parsed nodes — and for HTML, the walk turned out to cost more than the parse (36.7 vs 21.7 µs/post).

HTML, profiler-ranked:

  • Whitespace preservation is tracked with a counter during descent instead of walking node.ancestors for every text node. The counter is seeded once per parse from the parse root's real ancestry, so pre-parsed subtrees inside a <pre> keep their exact semantics.
  • The default HandlerRegistry is built once per process (shared, deep-frozen, same pattern as the other parsers).
  • process_text_node only pays the whitespace-collapsing gsub when the text contains something collapsible, and trim_trailing_whitespace exits early when there is nothing to strip.

TextFormatter was already lean; it gets the shared registry, frozen constants for the %w[s e] / %w[t r] arrays it allocated per element, and the direct-fetch lookup.

corpus bench before after
HTML walk (Ruby side only) 36.7 µs/post 16.9 µs/post
HTML conversion, ASCII / multibyte 87.9 / 117.3 µs/post 60.8 / 81.3 µs/post
HTML walk allocations 285 objects/post 143 objects/post
TF conversion, ASCII / multibyte 35.4 / 41.4 µs/post 29.8 / 36.4 µs/post

Output is unchanged: zero diff of rendered Markdown (including unknown_tags) over all 400 HTML and 400 TextFormatter corpus posts plus Unicode edge cases, old vs new.

Mutation coverage stays at 100% on both namespaces. The gsub/rstrip gates are output-equivalent fast paths and join the existing ignore entries; the preserve-depth counter is pinned by new specs, including nesting through a custom preserving tag (the built-in ones use RawHandler and never walk children).

After this, HTML conversion is roughly half nokogiri — there is not much left on the Ruby side of these two parsers.

@gschlager gschlager force-pushed the perf-other-parsers branch from ba1cd32 to b140416 Compare July 9, 2026 07:51
@gschlager gschlager force-pushed the perf-html-textformatter branch from 129d584 to 41004c3 Compare July 9, 2026 07:51
@gschlager gschlager force-pushed the perf-other-parsers branch from b140416 to c0afe79 Compare July 9, 2026 19:33
@gschlager gschlager force-pushed the perf-html-textformatter branch from 41004c3 to 7e6e142 Compare July 9, 2026 19:33
@gschlager gschlager changed the title DEV: Cut the HTML and TextFormatter tree-walk overhead PERF: Cut the HTML and TextFormatter tree-walk overhead Jul 9, 2026
@gschlager gschlager force-pushed the perf-other-parsers branch from c0afe79 to bc3fc4e Compare July 9, 2026 19:39
@gschlager gschlager force-pushed the perf-html-textformatter branch from 7e6e142 to cfb1d9f Compare July 9, 2026 19:39
@gschlager gschlager force-pushed the perf-other-parsers branch from bc3fc4e to c069882 Compare July 9, 2026 19:40
@gschlager gschlager force-pushed the perf-html-textformatter branch from cfb1d9f to a312398 Compare July 9, 2026 19:40
@gschlager gschlager force-pushed the perf-other-parsers branch from c069882 to ff0eb41 Compare July 9, 2026 19:40
@gschlager gschlager force-pushed the perf-html-textformatter branch from a312398 to c28c90d Compare July 9, 2026 19:40
@gschlager gschlager force-pushed the perf-other-parsers branch from ff0eb41 to 53166fb Compare July 9, 2026 19:41
@gschlager gschlager force-pushed the perf-html-textformatter branch from c28c90d to 40378af Compare July 9, 2026 19:41
@gschlager gschlager marked this pull request as ready for review July 9, 2026 19:48
gschlager added 3 commits July 9, 2026 21:49
@gschlager
DEV: Add TextFormatter corpus and nokogiri-share bench variants
5eaafe2 
The s9e/TextFormatter corpus mirrors the other formats (ASCII +
multibyte, mixed constructs, <s>/<e> markup preservation). New
html_walk/html_nokogiri and tf_walk/tf_nokogiri variants separate
nokogiri's C parse from the Ruby tree walk by pre-parsing the corpus
and feeding the parsers Nokogiri nodes.

Baseline these produced: the HTML walk costs 36.7 µs/post against
21.7 µs for the nokogiri parse itself — the Ruby side is the bigger
half. The allocation profile blames node.text.gsub copies per text
node, a node.ancestors walk per text node, and rstrip+pop+re-add
churn per element. TextFormatter is much leaner (7.6 µs walk); its
only notable waste is per-call registry construction and two array
literals allocated per element.
@gschlager
PERF: Cut the HTML tree walk's per-node overhead
e7df37d 
The walk cost more than nokogiri's actual parse (36.7 vs 21.7 µs/post
on the corpus). Four changes, profiler-ranked:

- Whitespace preservation is now tracked with a counter during descent
  instead of walking node.ancestors for every text node. The counter
  is seeded once per parse from the parse root's real ancestry, so
  pre-parsed subtrees inside a <pre> keep their exact semantics (new
  specs pin that, plus custom-preserving-tag nesting — the built-in
  preserving tags use RawHandler and never walk children).
- The default HandlerRegistry is built once per process (shared,
  deep-frozen — same pattern as the other parsers). All three internal
  collections are publicly reachable, so every freeze line is pinned
  by a raising spec.
- process_text_node only pays the whitespace-collapsing gsub when the
  text contains something collapsible; single-space prose passes
  through without the copy.
- trim_trailing_whitespace exits early when the last text node has no
  strippable tail, instead of rstrip+pop+re-add on every element.

The two gsub/rstrip gates are output-equivalent fast paths and join
the mutant ignore list with the registry fetch lookups; everything
else is at 100% with killing specs.

Corpus bench: html_walk 36.7 → 16.9 µs/post (allocations 285 → 143),
full HTML conversion 87.9 → 60.8 ASCII and 117.3 → 81.3 multibyte.
Output verified unchanged against the old implementation over all 400
corpus posts plus Unicode edge cases (zero diff, unknown_tags
included).
@gschlager
PERF: Trim TextFormatter per-parse and per-element overhead
7eac804 
Small wins — the TextFormatter walk was already lean:

- The default HandlerRegistry is built once per process (shared,
  deep-frozen), instead of a full construction per parse call.
- The %w[s e] / %w[t r] literals allocated two Arrays per element
  node; they're frozen constants now.
- Registry lookups fetch directly (s9e stores tag names uppercase, so
  the fetch hits without the upcase copy); the fallback keeps
  case-insensitive lookups working.

Corpus bench: tf_fresh 35.4 → 29.8 µs/post ASCII, tf_parse allocations
drop by the per-parse registry share. Output verified unchanged (zero
diff over the corpus + Unicode edge cases).
@gschlager gschlager force-pushed the perf-other-parsers branch from 53166fb to d2cda94 Compare July 9, 2026 19:51
@gschlager gschlager force-pushed the perf-html-textformatter branch from 40378af to 7eac804 Compare July 9, 2026 19:51
Base automatically changed from perf-other-parsers to main July 9, 2026 19:51
@gschlager gschlager merged commit 7eac804 into main Jul 9, 2026
7 checks passed
@gschlager gschlager deleted the perf-html-textformatter branch July 9, 2026 19:51
@gschlager gschlager mentioned this pull request Jul 9, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gschlager