Skip to content

solx-tester SIGSEGV on macOS x86-64: rustc 1.95/1.96 enum-niche miscompilation (rust-lang/rust#159035) — fix by bumping to 1.97.1 #547

Description

@nebasuke

I read about rust-lang/rust#159035 a few days ago on Hackernews, and now I regret thinking it would be unlikely enough to apply to us. So here you go, trying to get the integration tests to run on both Mac runners actually triggered it.

Claude Summary

solx-tester is killed by SIGSEGV within seconds on macOS x86-64 whenever it processes the Ethereum semantic-test corpus. The root cause is not in solx or solx-tester: it is a rustc/LLVM enum-niche miscompilation present in our pinned toolchain (1.96.1) and in 1.95.0, tracked upstream as rust-lang/rust#159035 (P-critical, I-miscompile, LLVM fix merged upstream, 1.97.1 point release with the LLVM bump confirmed by the release team).

Proposed fix: bump rust-toolchain.toml to 1.97.1 as soon as it ships. No code changes needed. The debug branch below verifies the bump in a single push.

Impact

  • macOS x86-64 is the only platform where we have observed the crash; it fires deterministically on the first Ethereum-corpus Call input, single-threaded or multi-threaded, so solx-tester effectively cannot run on Intel macs today. (This has no CI coverage, which is why it went unnoticed — it surfaced while validating perf(core): reuse persistent worker subprocesses across translation units #524 on macOS runners, PR Integration macos large hack #545.)
  • Per the upstream analysis, the miscompile family either crashes loudly or copies data into memory that Rust considers uninitialized (never validly read), so Linux/arm builds passing our suites are unlikely to be silently affected — but staying on an affected toolchain is not comfortable.

Crash signature

exception: EXC_BAD_ACCESS (EXC_I386_GPFLT), "UNKNOWN_0xD at 0x0000000000000000"
frame #0: solx-tester`try_from_ethereum [inlined] new at runtime.rs:54:9 [opt]
->  0x1000ee62c <+3420>: movaps %xmm0, -0xf8(%rbp)

An aligned 16-byte SSE store (zero-initializing the Option<u128> field of Runtime during Input::try_from_ethereum) is emitted against a stack slot at rbp-0xf8. With a standard frame (rbp ≡ 0 mod 16) that slot is ≡ 8 mod 16 and can never be 16-byte aligned — the store faults every time its basic block executes. The 0x12… patterns in the registers at crash time are the solc test-framework caller address (0x1212…0012) being copied into the struct; early readings of them as a wild pointer were a red herring.

How we ruled out solx-tester bugs

All on branch mac-tester-debug (PR #546, plain main + a debug workflow), runner macos-15-large:

Experiment Run Result
Baseline, full suite, 12 threads 29364158123 SIGSEGV ~30 s in; arm64 legs green (5m31s xlarge / 15m33s macos-15)
Single-threaded, corpus only (--path semanticTests) 29366844647 SIGSEGV in 2 s → deterministic, not a data race
MallocScribble/PreScribble/GuardEdges same run identical registers (no 0x55/0xAA patterns) → not use-after-free, not malloc-uninitialized
ASAN-instrumented tester same run 33 min clean → no heap violation to find
lldb --batch autopsy 29371120991 inline-resolved crash site + disassembly above
Toolchain probes 29372752832 1.95.0: SIGSEGV in ~7 s. 1.96.1: SIGSEGV in ~30 s. nightly 1.99.0 (2026-07-13, fixed LLVM): 23.5 min clean until the step's time budget expired

The 1.95-also-crashes result matches the upstream finding that the latent LLVM bug is observable from ~1.87 onward (it merely became easy to hit in 1.97), so rolling back is not a fix — only the LLVM-side fix in 1.97.1 is.

Reproducing

On any macOS x86-64 machine (CI runners are 2012 Mac minis; SIP state irrelevant):

cargo build --release --bin solx --bin solx-tester   # pinned 1.96.1
./target/release/solx-tester \
  --solidity-compiler ./target/release/solx \
  --path semanticTests --threads 1
# → "Segmentation fault: 11" within seconds

A single simple test passes (--path tests/solidity/simple/default.sol); the crash needs the Ethereum corpus path. The mac-tester-debug branch automates the whole thing (LLVM/solc from main's CI artifact caches, symbols kept, crash reports uploaded) — one push reruns everything, which is also how we'll verify the 1.97.1 bump.

Next step

  • When 1.97.1 is released: bump rust-toolchain.toml, verify green on mac-tester-debug (macos-15-large leg), then land the bump on main.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions