Skip to content

Commit 2ca88ad

Browse files
authored
Document type-piracy / type-leakage restrictions for require_stdlib (#56005)
I was a recent offender in JuliaLang/Pkg.jl#4017 (comment) This PR tries to lay down some guidelines for the behavior that stdlibs and the callers of `require_stdlib` must adhere to to avoid "duplicate stdlib" bugs These bugs are particularly nasty because they are experienced semi-rarely and under pretty specific circumstances (they only occur when `require_stdlib` loads another copy of a stdlib, often in a particular order and/or with a particular state of your pre-compile / loading cache) so they may make it a long way through a pre-release cycle without an actionable bug report.
1 parent 43f4afe commit 2ca88ad

File tree

1 file changed

+40
-0
lines changed

1 file changed

+40
-0
lines changed

base/loading.jl

+40
Original file line numberDiff line numberDiff line change
@@ -2610,6 +2610,46 @@ function _require_from_serialized(uuidkey::PkgId, path::String, ocachepath::Unio
26102610
end
26112611

26122612
# load a serialized file directly from append_bundled_depot_path for uuidkey without stalechecks
2613+
"""
2614+
require_stdlib(package_uuidkey::PkgId, ext::Union{Nothing, String}=nothing)
2615+
2616+
!!! warning "May load duplicate copies of stdlib packages."
2617+
2618+
This requires that all stdlib packages loaded are compatible with having concurrent
2619+
copies of themselves loaded into memory. It also places additional restrictions on
2620+
the kinds of type-piracy that are allowed in stdlibs, since type-piracy can cause the
2621+
dispatch table to become visibly "torn" across multiple different packages.
2622+
2623+
The specific requirements are:
2624+
2625+
The import side (caller of `require_stdlib`) must not leak any stdlib types, esp.
2626+
to any context that may have a conflicting copy of the stdlib(s) (or vice-versa).
2627+
- e.g., if an output is forwarded to user code, it must contain only Base types.
2628+
- e.g., if an output contains types from the stdlib, it must be consumed "internally"
2629+
before reaching user code.
2630+
2631+
The imported code (loaded stdlibs) must be very careful about type piracy:
2632+
- It must not access any global state that may differ between stdlib copies in
2633+
type-pirated methods.
2634+
- It must not return any stdlib types from any type-pirated public methods (since
2635+
a loaded duplicate would overwrite the Base method again, returning different
2636+
types that don't correspond to the user-accessible copy of the stdlib).
2637+
- It must not pass / discriminate stdlib types in type-pirated methods, except
2638+
indirectly via methods defined in Base and implemented (w/o type-piracy) in
2639+
all copies of the stdlib over their respective types.
2640+
2641+
The idea behind the above restrictions is that any type-pirated methods in the stdlib
2642+
must return a result that is simultaneously correct for all of the stdlib's loaded
2643+
copies, including accounting for global state differences and split type identities.
2644+
2645+
Furthermore, any imported code must not leak any stdlib types to globals and containers
2646+
(e.g. Vectors and mutable structs) in upstream Modules, since this will also lead to
2647+
type-confusion when the type is later pulled out in user / stdlib code.
2648+
2649+
For examples of issues like the above, see:
2650+
[1] https://github.com/JuliaLang/Pkg.jl/issues/4017#issuecomment-2377589989
2651+
[2] https://github.com/JuliaLang/StyledStrings.jl/issues/91#issuecomment-2379602914
2652+
"""
26132653
function require_stdlib(package_uuidkey::PkgId, ext::Union{Nothing, String}=nothing)
26142654
@lock require_lock begin
26152655
# the PkgId of the ext, or package if not an ext

0 commit comments

Comments
 (0)