Skip to content

feat: add --deterministic-time to make observed time reproducible - #61

Draft
not-matthias wants to merge 5 commits into
masterfrom
deterministic-time
Draft

not-matthias wants to merge 5 commits into
masterfrom
deterministic-time

Conversation

@not-matthias

@not-matthias not-matthias commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

TLDR: Runtimes make decisions from elapsed time: benchmark harnesses pick iteration and warmup counts, GCs and schedulers use time-based heuristics, and time-bounded loops run until a deadline. Under Callgrind the host clock runs 20 to 100 times faster relative to the program than natively, and varies between runs, so all of these become instruction-count noise. --deterministic-time=yes derives every clock the client can read from the number of guest instructions executed, so the same instruction stream always observes the same time (Linux only; --deterministic-rng already lists this as a limitation).

Model

  • 1 ns per executed guest instruction: a 1 GHz CPU retiring one instruction per cycle. Valgrind's own work (translation, syscall wrappers, tool instrumentation) does not advance the clock.
  • CLOCK_REALTIME is a fixed epoch (2024-01-01T00:00:00Z) plus the virtual time; the monotonic, boottime and CPU-time clocks are the virtual time.

Core

  • m_vclock.c holds the counter and a VEX instrument2 pass that counts IMarks and adds them to VG_(vclock_ns) before every conditional exit and at the end of each superblock, so every path adds exactly the instructions it executed. It runs after the tool's pass, so tool costs are unchanged and tools never see the counter accesses. Cost is three IR statements per SB exit.
  • The pass is composed with the existing SP-update pass in vg_core_instrument_pass.

Syswrap

  • clock_gettime, gettimeofday and time are completed in PRE from the virtual clock. Dynamic and unknown clock ids still reach the kernel.
  • nanosleep and clock_nanosleep move the clock forward by the requested duration (or up to the absolute deadline) and return at once; EINVAL/EFAULT checks match the kernel. A completed sleep sets SfYieldAfter so other threads still get to run.

VEX

  • New LibVEX_SetVirtualTSC(ticks, hz) hook, mirroring LibVEX_SetRandom32. RDTSC/RDTSCP (amd64) and CNTVCT_EL0 (arm64) return the virtual time in ns; CNTFRQ_EL0 reports 1 GHz; RDTSCP reports TSC_AUX as 0.

Review notes

  • Not covered on purpose (documented in CODSPEED-CHANGELOG.md): timeouts on blocking calls (futex, poll, epoll_wait), interval timers, getrusage/times, and CPUID's TSC frequency. Doing timeouts right needs a discrete-event scheduler; I left it out until a workload needs it.
  • The clock is global. Thread interleaving is still host-driven, so multithreaded programs can still observe different times.
  • Regtest callgrind/tests/deterministic_time checks properties that hold on every host (epoch, clocks agree, sleeps skip ahead, TSC tracks the clock, identical work takes identical time, kernel errors preserved). The full callgrind suite passes with VALGRIND_OPTS=--deterministic-time=yes with the same pre-existing failures as without.
  • Variance tests (zero Ir variance across runs under the flag, non-zero without) are in valgrind-helpers: AvalancheHQ/valgrind-helpers#6 (stacked on AvalancheHQ/valgrind-helpers#3).

Smoke (5 ms monotonic budget loop, then rdtsc delta, three runs each):

host time:          iterations=7880   ticks=29771478   secs=1793787211
                    iterations=9958   ticks=28456395   secs=1793787211
                    iterations=13037  ticks=27578133   secs=1793787211
deterministic time: iterations=192307 ticks=7002395    secs=1704067200  (x3, identical)

LibVEX_SetVirtualTSC installs a counter that the RDTSC, RDTSCP (amd64,
x86) and CNTVCT_EL0 (arm64) dirty helpers return instead of reading the
host counter, and a frequency that CNTFRQ_EL0 reports. RDTSCP reports
TSC_AUX as 0 so the CPU number cannot leak either.

This mirrors LibVEX_SetRandom32: the core decides whether to install it,
and NULL keeps the host behaviour.
@codspeed

codspeed Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 11.01%

⚡ 1 improved benchmark
✅ 83 untouched benchmarks
⏩ 60 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_valgrind[valgrind.codspeed, stress-ng --cpu 1 --cpu-ops 10, full-with-inline] 7.5 s 6.8 s +11.01%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing deterministic-time (8e8a952) with master (d034e80)

Open in CodSpeed

Footnotes

  1. 60 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

With --deterministic-time=yes the core keeps a virtual clock that
advances by 1 ns per executed guest instruction, so the same instruction
stream always observes the same time, however long Valgrind takes to
run it. Valgrind's own work never advances it.

A core instrument2 pass counts the IMarks of each superblock and adds
the count to VG_(vclock_ns) before every conditional exit and at the end
of the block, so every path adds exactly the instructions it executed.
It runs after the tool's instrumentation, so tool costs are unchanged
and tools never see the counter accesses. The counter is also installed
as VEX's virtual TSC, so RDTSC and CNTVCT_EL0 tick at 1 GHz.

m_vclock.c also maps Linux clock ids onto the clock: the realtime clocks
are a fixed epoch (2024-01-01T00:00:00Z) plus the virtual time, the
monotonic, boottime and CPU-time clocks are the virtual time, and
sleeps move the clock forward. The syscall wrappers use these in the
next commit.
With --deterministic-time=yes, clock_gettime, gettimeofday and time
return the virtual clock, and nanosleep and clock_nanosleep move it
forward by the requested duration (or up to the absolute deadline) and
return at once instead of blocking. Clock ids the virtual clock does not
cover (dynamic and unknown ones) still reach the kernel, as do the
kernel's EINVAL and EFAULT checks on the request.

A completed sleep yields to other threads, as a real sleep would let
them run. The vDSO is already disabled under Valgrind, so libc reaches
these syscalls.
Checks the properties of the virtual clock that hold on every host and
compiler: the realtime clocks start at the fixed epoch, the monotonic
clocks agree with each other, sleeps advance the clock without waiting,
the TSC ticks once per virtual nanosecond, identical work takes
identical time, and argument errors still match the kernel. Absolute
values are not pinned because the instruction count before main()
depends on libc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant