[DharmaBytesX]

The backend-benchmark job fails with a BufferError on runners that have valgrind-codspeed build 10 or later.
This adds a step that raises the soft stack limit to 16 MiB before the CodSpeed steps.
Decide: this is a CI workflow change ("Ask First" in AGENTS.md), and a runner-wide setting is the alternative.

Why

Problem: since 2026-10-26 the job fails at test setup on 4 of the 24 huge-runners runners, always with BufferError: Existing exports of data: object cannot be re-sized in asyncio/streams.py, and the job then ends with exit code 139 (failing job, CodSpeed 5.5.0, Python 3.14.7). Each of the 4 runners received valgrind-codspeed 3.26.0-0codspeed10 during a job of the CodSpeed bump pull requests (#11223, #11342), whose CodSpeed runner 5.6.x requires that build (binary_pins.rs at v5.6.0). The CodSpeed runner 5.5.0 pinned here accepts a newer build and does not downgrade (setup.rs), so these runners keep failing.

Cause: since build 10 (PR #53, startup stack layout), valgrind reports a 4 KiB stack to the program when the soft stack limit is 8 MiB. CPython 3.14 reads its stack bounds at startup (ceval.c) and, with no stack margin, queues garbage-collected objects instead of freeing them (_Py_Dealloc). The memoryview in bytes(memoryview(buf)[:n]) is then not freed, and del buf[:n] fails. CodSpeed said in the shared channel that it looks like a regression in that pull request, recommended a stack limit of 16384 KiB or unlimited as a workaround, and will follow up when a release is ready (no public link).

Goal: the job passes on any runner, whatever valgrind build is installed.
Non-goals: changing the runners, pinning or downgrading valgrind, fixing valgrind.

What changed

• A new step, "Raise the stack limit for valgrind", runs before both CodSpeed steps. It raises the soft stack limit of its parent process (the runner worker) to 16 MiB with prlimit; the CodSpeed steps inherit it. A higher existing limit is kept. The step prints the resulting limit.
• Why not ulimit in the benchmark command: valgrind reads the limit when it starts, and CodSpeed runs the command inside valgrind (measure.rs). In the local test, ulimit there gave the error in 1000 of 1000 runs.
• Unchanged: the runners, the CodSpeed action pin (5.5.0), the benchmark commands and every other job.

How to review

• The step assumes its shell is a direct child of the runner worker. On live runners the process tree is worker, step shell, command (read with ps, 8 runners). I did not run a job with the step on a runner (see below).
• prlimit (util-linux 2.39.3) exists in the 3 runner containers I checked.
• The step fails (exit code 1) if the hard stack limit is below 16 MiB. The runners have an unlimited hard limit (4 runner containers, and the live job workers of 8 runners). Failing is intended: skipping would keep the BufferError.
• Alternative: set the stack limit for all runner containers in the runner configuration. This pull request does not do that.
• Remove the step when the CodSpeed runner in use installs a valgrind build that fixes this. The fixed build is not known yet.

How to test

Local reproduction (arm64 Docker, Ubuntu 24.04, the CI's Python builds from actions/python-versions, bytes(memoryview(buf)[:n]) then del buf[:n], 1000 runs):

valgrind-codspeed	soft stack limit	BufferError
9	8 MiB	0
10, 11, 12	8 MiB	1000
10, 11, 12	9 MiB or more, 16 MiB, unlimited	0

• The workflow step text, run as a child of a stand-in for the worker followed by the CodSpeed launch sequence (valgrind 10, Python 3.14.10): without the step 1000 errors, with the step 0; a worker already at a higher limit is not lowered; a hard limit of 10 MiB makes the step fail with a clear message.
• On a failing x86-64 runner (valgrind 10, Python 3.14.10, no file written): at 8 MiB the reported stack is 4 KiB and the error occurs in 1000 of 1000 runs; at 16 MiB the reported stack is 8196 KiB and there is no error.
• Deep recursion under valgrind 10 crashes with exit code 139 at 8 MiB and ends with a normal RecursionError at 16 MiB. The exit-time crash seen in the CI jobs was not reproduced exactly.
• yamllint -s and zizmor report nothing new.
• Not verified: a real CI job with the step. It must land on one of the 4 affected runners (about 1 in 6 per run); I will re-run until it does and add the link here. Expected: the job passes and the step log shows max stack size 16777216.

Impact & rollout

• Backward compatibility: none. CI only.
• Config/env changes: none.
• Deployment notes: safe to merge; the change applies to the job's own worker process.

Checklist

• Tests added/updated: not applicable to a workflow change; tested as described above
• Changelog entry: not needed, CI-config changes get none (AGENTS.md)
• I have reviewed AI generated content

Code review

• The code-review skill ran at level high in a separate agent, not the author, on the uncommitted diff against origin/stable (1 file, 9 added lines, before the comment change listed below); the agent checked the checkout (branch sas-ci-benchmark-stack-limit, only .github/workflows/ci.yml modified).
• Findings: 0 CRITICAL, 2 IMPORTANT, 3 MINOR.
• IMPORTANT: nothing in the diff proves the fix, and the branch pins CodSpeed 5.5.0, which does not install the build. Kept: on stable the job fails today on the 4 runners that have build 10 with CodSpeed 5.5.0, and the local and runner tests above show the effect; the proof in a real job is still open (see How to test).
• IMPORTANT: the step fails the job if the hard stack limit is below 16 MiB. Kept, with the reason given in How to review; tested.
• MINOR left as they are: the printed limit does not prove that the CodSpeed steps inherit it (the CI run will); no removal condition in the comment (it is in How to review).
• MINOR changed: the comment now says the CodSpeed steps below inherit the limit, so the step must stay before them. The bash -e {0} shell is documented by GitHub; I did not run it.

Security review

• The built-in security-review skill ran in a separate agent, not the author, on git diff origin/HEAD... (commit 10af29865). origin/HEAD is origin/stable, the base of this pull request, and the diff is the one file shown here.
• Findings: 0 High, 0 Medium, 0 Low. Nothing to fix; no Low findings to list.
• The reviewer did not run the step on a runner. The runner facts it could not check (process tree, prlimit present, hard limit) are covered in How to review.

[DharmaBytesX]

Closing: the 4 affected runners were repaired by reinstalling valgrind-codspeed 3.26.0-0codspeed9, so all 24 runners now run the same build, and CodSpeed has confirmed the regression and will follow up with a fix. This step is not needed while the CodSpeed action stays on 5.5.0. If a CodSpeed bump that requires build 10 or later is merged before a fix ships, the same workaround applies: a soft stack limit of 16384 KiB or more before the CodSpeed steps (see the description).

[codspeed]

Merging this PR will not alter performance

✅ 13 untouched benchmarks

---

_{Comparing sas-ci-benchmark-stack-limit (10af298) with stable (67b523d)}