perf(bench): collect mimalloc memory in benchmark setup

[stormslowly]

Why

Between two measured iterations a benchmark leaves the allocator dirty: thread-local free lists and retired pages are laid out according to the previous iteration's allocation and free order. The next iteration then walks a different allocator fast path, which was the suspected source of run-to-run variance.

This calls mi_collect(true) in every iter_batched / iter_batched_ref setup closure, so each measured iteration starts from a comparable heap state.

Setup is outside the measured region — codspeed-criterion-compat only wraps the routine with start_benchmark() / end_benchmark() (src/compat/bencher.rs:127-139), and criterion's walltime bencher excludes setup from timing — so the collect cost is not counted.

What

• rspack_benchmark::collect_memory() — thin wrapper over libmimalloc_sys::mi_collect(true), no-op on wasm.
• libmimalloc-sys added with the extended feature (that feature is what exposes the mi_collect binding); feature set kept aligned with the existing mimalloc dependency.
• Call inserted as the first statement of all 21 setup closures across compilation_stages, build_chunk_graph, bundle, persistent_cache, scan_dependencies and walltime.

Measured effect: no stability gain

Ran the simulation benchmark 5× on this branch and 5× on its base (faba242cd6) as independent CI runs, then read the per-benchmark instruction counts straight out of the uploaded callgrind dumps (50 benchmarks, matching the 50 CodSpeed reports).

	bit-identical across the 5 runs	CV median	CV mean	CV max
with mi_collect	16/50	0.0079%	0.133%	1.21%
base	16/50	0.0065%	0.134%	1.93%

Paired per benchmark: with-collect is tighter on 18, base is tighter on 16, 16 ties. Restricted to the 25 benchmarks that are not already deterministic (max CV > 0.01%): 12 vs 13. That is a coin flip, so allocator carry-over is not what drives the residual noise.

The noise that remains is concentrated in the small rust@<stage> benchmarks (a few million Ir); every sources@*, bundle@* and build_* benchmark is already deterministic. rust@create_chunk_assets is the worst case in both arms (1.21% / 1.93%), which is the benchmark previously traced to glibc memcpy ifunc dispatch — something mi_collect cannot address.

One real effect did show up: collecting shifts the measured level slightly down (22 of the 28 benchmarks that moved by more than 0.01%, up to −0.78%, mean −0.086%). That is a one-off bias absorbed at the next baseline, not a variance reduction.

Wall-clock cost is nil: the Run benchmark (simulation) step took 727s here versus a 759s median over the last 22 main runs on the same runner type.

[github-actions]

📦 Binary Size-limit

Comparing 0573afd to chore(deps): migrate to @rstackjs/doc-ui v1.14.9 (#15349) by Jiahan Chen

⏳ The base commit triggered a linux binding build, but its binary size data has not been generated yet, so the size comparison is skipped.

Please re-run this workflow once the ecosystem-benchmark data for that commit is published.

Warning

Reference only — not the real baseline. The base commit's data isn't ready yet, so this compares against the nearest earlier commit that has data (9e19541) for a rough estimate:

🙈 Size remains the same at 68.18MB

[github-actions]

Rsdoctor Bundle Diff Analysis

⚠️ Note: The latest commit (faba242cd6) does not have baseline artifacts. Using commit 9e195410dc for baseline comparison instead. If this seems incorrect, please wait a few minutes and try rerunning the workflow.

Found 5 projects in monorepo, 0 projects with changes.

📊 Quick Summary

Project	Total Size	Gzip Size	Change	Gzip Change
popular-libs	1.7 MB	556.5 KB	0	0
react-10k	5.6 MB	1.3 MB	0	0
react-1k	823.1 KB	218.4 KB	0	0
ui-components	4.9 MB	1.4 MB	0	0
react-5k	2.7 MB	669.3 KB	0	0

Generated by Rsdoctor GitHub Action

[codspeed-hq]

Merging this PR will not alter performance

✅ 50 untouched benchmarks
⏩ 47 skipped benchmarks¹

---

_{Comparing perf/bench-mi-collect-in-setup (0573afd) with main (22dda99)²}

Footnotes

1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (faba242) during the generation of this report, so 22dda99 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[stormslowly]

Closing: measured, and it does not improve stability.

I ran the simulation benchmark 5× on this branch and 5× on its base (faba242cd6) as independent CI runs, and read the per-benchmark instruction counts straight out of the uploaded callgrind dumps (50 benchmarks, matching the 50 CodSpeed reports).

	bit-identical across the 5 runs	CV median	CV mean	CV max
with mi_collect	16/50	0.0079%	0.133%	1.21%
base	16/50	0.0065%	0.134%	1.93%

Paired per benchmark: with-collect is tighter on 18, base is tighter on 16, 16 ties. Restricted to the 25 benchmarks that are not already deterministic (max CV > 0.01%), it is 12 vs 13 — a coin flip. Allocator carry-over between iterations is not what drives the residual noise.

Where the noise actually lives: entirely in the small rust@<stage> benchmarks of a few million Ir. Every sources@*, bundle@* and build_* benchmark is already bit-identical across runs. The worst case in both arms is rust@create_chunk_assets (1.21% / 1.93%), the benchmark previously traced to glibc memcpy ifunc dispatch — which mi_collect cannot address, and which CI already pins as far as it can via GLIBC_TUNABLES.

The one real effect is a small systematic shift, not a variance reduction: of the 28 benchmarks that moved by more than 0.01%, 22 moved down (max −0.78%, mean −0.086%). That is a one-off bias absorbed at the next baseline.

Cost was not the problem — the Run benchmark (simulation) step took 727s here versus a 759s median over the last 22 main runs on the same runner type. But since there is no measurable benefit, this is not worth a new libmimalloc-sys dependency and 21 call sites.