Skip to content

v3: stop re-reading released worker args in the monomorphize merges - #28700

Open
Jengro777 wants to merge 12 commits into
vlang:masterfrom
Jengro777:fix/28489-monomorphize-released-args
Open

Jengro777 wants to merge 12 commits into
vlang:masterfrom
Jengro777:fix/28489-monomorphize-released-args

Conversation

@Jengro777

Copy link
Copy Markdown
Contributor

Why

The monomorphize worker merge copied its []string tables shallowly: the array
was copied, but every element still pointed into the worker scratch arena that
transform_worker_scope_leave() releases right after the merge. A later pass
(the driver compiles twice on this path) re-seeds its specializations from
generic_specialization_args / monomorph_cache_specs, so it read freed type
arguments. The strings turn into NUL bytes, and V3 then reports bogus
diagnostics for perfectly valid calls:

error: unknown function `ctrl.use`
error: unknown function `ctrl.route_use`

...and silently falls back to the compatibility compiler - exactly the
diagnostic family of #28489.

What changed

  • run_parallel_monomorphize_specs(): deep-copy the merged
    generic_specialization_args entry into the parent arena instead of
    spec_args.clone() (which copies the slice header only).
  • run_scoped_monomorphize_specs(): re-record every emitted specialization
    through record_monomorph_cache_spec() from the master, so the
    parent-owned copy wins over one a worker recorded for the same key.
  • record_monomorph_cache_spec(): clone the key, declaration key, module and
    every argument string, and re-record existing keys instead of skipping them
    (the worker's copy dies with its arena).

Test plan

vlib/v/compiler_tests/scoped_monomorphize_closure_test.v builds a V3 compiler
with -prealloc and compiles the small veb program from #28489 (embedded
Middleware[Ctx]/Controller structs, a generic controller helper that calls
ctrl.use() / ctrl.route_use(), and lifted closures). The fixture fails on a
shallow merge (freed arguments -> bogus diagnostics/SIGSEGV); the follow-up PR
for #28640 keeps it as the shared regression fixture for the bounded path.

v -g -keepc -o vnew cmd/v
./vnew -silent test vlib/v/compiler_tests/scoped_monomorphize_closure_test.v

Notes

Fixes #28489.

Copilot AI lite review requested due to automatic review settings September 18, 2026 03:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The regression test does not deterministically exercise V3’s scoped path and may pass through fallback compilation.

Pull request overview

This pull request fixes V3 monomorphization cache lifetime issues by preserving specialization data across worker merges.

Changes:

  • Deep-copy specialization arguments during merges.
  • Re-record cache entries with parent-owned strings.
  • Add regression coverage for generic closure compilation.
File summaries
File Summary
vlib/v/transform/transform_parallel_notd_v3_no_parallel.v Safely merges specialization data from workers.
vlib/v/transform/monomorphize.v Owns cache strings and refreshes duplicate entries.
vlib/v/compiler_tests/scoped_monomorphize_closure_test.v Adds scoped monomorphization regression coverage.
Review details

Suppressed comments (3)

vlib/v/compiler_tests/scoped_monomorphize_closure_test.v:89

  • This invocation does not force V3. On the regression described here, V3 can emit the bogus diagnostics and then transparently fall back to V1, which still returns exit code 0 and produces the binary, so all three assertions pass without exercising the fix. Run the generated compiler with -new-compiler so the test fails on the stale-argument path.
	compile := os.execute('${v3_bin} -nocache -o ${out} ${dir}')

vlib/v/compiler_tests/scoped_monomorphize_closure_test.v:22

  • Calling scoped_monomorph_v3_bin() here builds the compiler when it is absent and then immediately deletes it, so every test run performs the expensive self-build twice. Remove the path directly in the suite hook instead of invoking the build helper.
	os.rm(scoped_monomorph_v3_bin()) or {}

vlib/v/compiler_tests/scoped_monomorphize_closure_test.v:16

  • -prealloc only enables scope_parallel_workers; run_scoped_monomorphize_specs() is selected separately when the AST reaches scoped_monomorph_node_threshold (currently 1,000,000). The small program compiled at line 89 does not force that gate, so this test can pass while the changed scoped merge/re-recording path is never executed. Please make the regression deterministically select the bounded path, for example by providing a test-only threshold override or a fixture large enough to cross it.
	// `-prealloc` is what enables `scope_parallel_workers` and the scoped
	// monomorphize path, matching how the distributed compiler is built.
	build := os.execute('${scoped_monomorph_vexe} -gc none -prealloc -path "${scoped_monomorph_vlib_dir}|@vlib|@vmodules" -o ${bin} ${scoped_monomorph_v3_src}')
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@Jengro777
Jengro777 force-pushed the fix/28489-monomorphize-released-args branch from a9a45f2 to 5c4151a Compare September 18, 2026 06:24
@Jengro777

Copy link
Copy Markdown
Contributor Author

request for review

The monomorphize worker merge copied its `[]string` tables shallowly: the array
was copied, but every element still pointed into the worker scratch arena that
is released right after the merge. A later pass (the driver compiles twice on
this path) re-seeds its specializations from `generic_specialization_args` /
`monomorph_cache_specs`, so it read freed type arguments: they turn into NUL
bytes, and V3 then reports bogus `unknown function` diagnostics for perfectly
valid calls (`ctrl.use` / `ctrl.route_use` in veb apps) and silently falls back
to the compatibility compiler.

Deep-copy the merged arguments into the parent arena, re-record every emitted
specification from the master (instead of skipping keys a worker already
recorded) so the last, parent-owned copy wins, and copy the emitted
specialization keys/arguments into the parent arena while the worker scope is
still alive but suspended.

Needed to compile RuoQi-v (https://github.com/RuoQi-DoDo/RuoQi-v) with
`-new-compiler`.
Fixes vlang#28489.
@Jengro777
Jengro777 force-pushed the fix/28489-monomorphize-released-args branch from 5c4151a to e7ac3a1 Compare September 18, 2026 09:17

@medvednikov medvednikov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current code changes only; CI status intentionally not considered. The deep-copy changes themselves look coherent, but the new regression does not actually force the scoped merge path this PR also changes.

Comment thread vlib/v/compiler_tests/scoped_monomorphize_closure_test.v
…ize-released-args

# Conflicts:
#	vlib/v/transform/monomorphize.v
#	vlib/v/transform/monomorphize_test.v
The fixture compiles its sample with the compiler built from cmd/v, but
without -new-compiler a V3 failure silently retries with the V 0.5.2
compatibility compiler, which can still produce a binary and exit 0, so
the assertions would pass without covering the scoped merge path.
Pass -new-compiler so a regression fails the test instead.
@Jengro777

Copy link
Copy Markdown
Contributor Author

fixed

Monomorph workers resolve and lift specialization declarations inside their own
scratch arena, which is released right after the batch merges. The merges
published the worker's own `GenericFnDecl` (module, key, file and the
`flat.Node` text), so the master's signature tables and the unresolved-type
cache ended up holding strings from released arenas: after a later batch
released its block, `same_transform_text()` compared `cache.module` and read
unmapped memory, which made the veb program from vlang#28489 abort with a
segfault instead of compiling.

Publish a copy the master arena owns when a worker's queued or emitted
specialization is merged, in both the scoped and the chunk-worker paths, like
the argument lists and queue keys already are.

The merged upstream state could not be built by V3 at all, which hid this:
`transform_late_used_fn_bodies` still dereferenced `(*names)` after the
parameter became a value, and `transform_selector_expr` shadowed `iface_name`.
Fix both so `./v -new-compiler` can compile the transform module again.

Fixes vlang#28489.
…ize-released-args

# Conflicts:
#	vlib/v/transform/transform_parallel_notd_v3_no_parallel.v

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

V3 reports bogus unknown function diagnostics for generic functions that call methods on a generic parameter, then silently falls back to V1

3 participants