Skip to content

Add precompilation instructions for lazy artifacts.#88

Merged
maleadt merged 2 commits into
masterfrom
tb/precompile_lazy
May 28, 2026
Merged

Add precompilation instructions for lazy artifacts.#88
maleadt merged 2 commits into
masterfrom
tb/precompile_lazy

Conversation

@maleadt

@maleadt maleadt commented Apr 22, 2026

Copy link
Copy Markdown
Member

Avoids the load-time compilation seen with lazy artifacts, such as in LLVMExtra_jll.

Before:

julia> @time_imports using LLVMExtra_jll
[ Info: Precompiling LLVMExtra_jll [dad2f222-ce93-54a1-a47d-0025e8a3acab](cache misses: wrong source (3), incompatible header (3), mismatched flags (2))
Precompiling LLVMExtra_jll finished.
  2 dependencies successfully precompiled in 2 seconds. 28 already precompiled.
      0.9 ms  Printf
      8.5 ms  Dates
      0.8 ms  TOML
      0.6 ms  NetworkOptions
               ┌ 1.9 ms OpenSSL_jll.__init__()
      5.6 ms  OpenSSL_jll 52.83% compilation time
               ┌ 0.7 ms LibSSH2_jll.__init__()
      1.6 ms  LibSSH2_jll
               ┌ 0.9 ms LibGit2_jll.__init__()
      1.5 ms  LibGit2_jll
      4.7 ms  LibGit2
      3.9 ms  ArgTools
               ┌ 0.7 ms nghttp2_jll.__init__()
      1.4 ms  nghttp2_jll
               ┌ 1.2 ms Zlib_jll.__init__()
      1.9 ms  Zlib_jll
               ┌ 1.0 ms LibCURL_jll.__init__()
      1.7 ms  LibCURL_jll
               ┌ 0.0 ms MozillaCACerts_jll.__init__()
      0.7 ms  MozillaCACerts_jll
               ┌ 0.0 ms LibCURL.__init__()
      1.3 ms  LibCURL
               ┌ 3.2 ms Downloads.Curl.__init__()
      8.4 ms  Downloads
      1.4 ms  Tar
               ┌ 0.0 ms p7zip_jll.__init__()
      0.9 ms  p7zip_jll
      0.7 ms  UUIDs
      0.8 ms  Logging
               ┌ 0.0 ms Pkg.__init__()
     80.3 ms  Pkg
      1.0 ms  LazyArtifacts
      1.0 ms  Preferences
      1.0 ms  JLLWrappers
               ┌ 42.3 ms LLVMExtra_jll.__init__() 99.44% compilation time
    178.2 ms  LLVMExtra_jll 23.59% compilation time

After:

julia> @time_imports using LLVMExtra_jll
[ Info: Precompiling LLVMExtra_jll [dad2f222-ce93-54a1-a47d-0025e8a3acab](cache misses: wrong source (3), incompatible header (3), mismatched flags (2))
Precompiling LLVMExtra_jll finished.
  2 dependencies successfully precompiled in 2 seconds. 28 already precompiled.
      0.9 ms  Printf
      8.9 ms  Dates
      0.8 ms  TOML
      0.7 ms  NetworkOptions
               ┌ 1.9 ms OpenSSL_jll.__init__()
      5.3 ms  OpenSSL_jll 51.79% compilation time
               ┌ 0.6 ms LibSSH2_jll.__init__()
      1.3 ms  LibSSH2_jll
               ┌ 1.0 ms LibGit2_jll.__init__()
      1.6 ms  LibGit2_jll
      4.6 ms  LibGit2
      3.9 ms  ArgTools
               ┌ 0.6 ms nghttp2_jll.__init__()
      1.3 ms  nghttp2_jll
               ┌ 1.2 ms Zlib_jll.__init__()
      1.9 ms  Zlib_jll
               ┌ 1.0 ms LibCURL_jll.__init__()
      1.7 ms  LibCURL_jll
               ┌ 0.0 ms MozillaCACerts_jll.__init__()
      0.7 ms  MozillaCACerts_jll
               ┌ 0.0 ms LibCURL.__init__()
      1.2 ms  LibCURL
               ┌ 3.0 ms Downloads.Curl.__init__()
      7.9 ms  Downloads
      1.3 ms  Tar
               ┌ 0.0 ms p7zip_jll.__init__()
      0.8 ms  p7zip_jll
      0.7 ms  UUIDs
      0.7 ms  Logging
               ┌ 0.0 ms Pkg.__init__()
     81.3 ms  Pkg
      0.9 ms  LazyArtifacts
      1.0 ms  Preferences
      0.9 ms  JLLWrappers
               ┌ 0.2 ms LLVMExtra_jll.__init__()
    135.4 ms  LLVMExtra_jll

I don't particularly like the explicit nasty precompile statements, but AFAIU a precompilation workload was tried in the past and removed because it triggered actual compilation.

@topolarity

Copy link
Copy Markdown
Member

Is this better done in a precompile workload in LazyArtifacts / Pkg itself?

None of the types look specific to JLLWrappers

@maleadt

maleadt commented Apr 22, 2026

Copy link
Copy Markdown
Member Author

Yes, but then we don't get this into user's hands anytime soon.

@topolarity

topolarity commented Apr 22, 2026

Copy link
Copy Markdown
Member

That's good motivation, just wanted to verify

We should still add it in both places so that newer Julia versions don't end up with the machine code duplicated in all of their JLL cache files needlessly. Julia should be able to recognize the CodeInstance is in a pre-existing pkgimage and avoid re-serializing it, if our precompilation policy is implemented correctly now

@maleadt maleadt force-pushed the tb/precompile_lazy branch from 0acc212 to df7a02f Compare May 14, 2026 12:06
`Val{LA}` where `LA` is a `Module` is only legal on Julia 1.11+; older
Julia rejects it with `TypeError: in Type, in parameter, expected Type,
got a value of type Module`. The stdlib `Artifacts._artifact_str` /
`__artifact_str` internals also only exist starting Julia 1.11. Move the
precompile body behind a `@static if VERSION >= v"1.11" && isdefined(...)`
gate so 1.6-1.10 still get the `get_julia_libpaths` precompile but skip
the unsupported path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@maleadt

maleadt commented May 14, 2026

Copy link
Copy Markdown
Member Author

CI fixed, and precompilation upstreamed to LazyArtifacts/Pkg.

@maleadt maleadt merged commit 351e5f6 into master May 28, 2026
65 of 72 checks passed
@maleadt maleadt deleted the tb/precompile_lazy branch May 28, 2026 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants