Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from JuliaLang:master #1814

Open
wants to merge 681 commits into
base: master
Choose a base branch
from

Conversation

pull[bot]
Copy link

@pull pull bot commented Jul 31, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added the ⤵️ pull label Jul 31, 2024
topolarity and others added 29 commits October 15, 2024 09:07
…56159)

These are invalidated by our own stdlibs (Dates and REPL) unfortunately
so we need to put this barrier in.

This fix is _very_ un-satisfying, because it doesn't do anything to
solve this problem for downstream libraries that use e.g. `displaysize`.
To fix that, I think we need a way to make sure callers get these
invalidation barriers by default...
Documenter.jl requires all trailing list content to follow the same
indentation as the header. So, in the current view
(https://docs.julialang.org/en/v1/manual/installation/#Command-line-arguments)
the list appears broken.
)

Follow up to #55994 and #55997. This should basically be a
non-functional change and I see no performance difference, but the
comments and the definition of a helper function should make the code
easier to follow (I initially struggled in #55997) and extend to other
types.
PerProcess: once per process
PerThread: once per thread id
PerTask: once per task object
Somewhat generalizes our support for changing Ptr to C_NULL. Not
particularly fast, since it is just using the builtins implementation of
setfield, and delaying the actual stores, but it should suffice.
Address reviewer feedback, add more fixes and more tests,
rename to add Once prefix.
Closes #56109 

Simulating a bad manifest by having `LibGit2_jll` missing as a dep of
`LibGit2` in my default env, say because the manifest was generated by a
different julia version or different master julia commit.

## This PR, it just works
```
julia> using Revise

julia>
```
i.e.
```
% JULIA_DEBUG=loading ./julia --startup-file=no
julia> using Revise
...
┌ Debug: Stdlib LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] is trying to load `LibGit2_jll`
│ which is not listed as a dep in the load path manifests, so resorting to search
│ in the stdlib Project.tomls for true deps
└ @ Base loading.jl:387
┌ Debug: LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] indeed depends on LibGit2_jll in project /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/LibGit2/Project.toml
└ @ Base loading.jl:395
...

julia>
```

## Master
```
julia> using Revise
Info Given Revise was explicitly requested, output will be shown live
ERROR: LoadError: ArgumentError: Package LibGit2 does not have LibGit2_jll in its dependencies:
- Note that the following manifests in the load path were resolved with a potentially
  different DEV version of the current version, which may be the cause of the error.
  Try to re-resolve them in the current version, or consider deleting them if that fails:
    /Users/ian/.julia/environments/v1.12/Manifest.toml
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have LibGit2 checked out for development and have
  added LibGit2_jll as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with LibGit2
...
```
…op (#55802)

Closes #55785

I'm not sure if we want to backport this like this. Because that removes
some functionality (the pass itself). So LLVM.jl and friends might need
annoying version code. We can maybe keep the code there and just not run
the pass in a backport.
This removes several `convert(String, ...)` from this code, which really
shouldn't be something we invalidate on in the first place (see
#56173) but this is still an
improvement in code quality so let's take it.
Due to popular demand, the type of annotations is to be changed from a
`Tuple{UnitRange{Int}, Pair{Symbol, Any}}` to a `NamedTuple{(:region,
:label, :value), Tuple{UnitRange{Int}, Symbol,
Any}}`.

This requires the expected code churn to `strings/annotated.jl`, and
some changes to the StyledStrings and JuliaSyntaxHighlighting libraries.

Closes #55249 and closes #55245.
Follow up to #56034, ref:
#56034 (comment).

---------

Co-authored-by: Cody Tapscott <[email protected]>
This also has some changes that move platform selection to compile time
together with
JuliaPackaging/JLLWrappers.jl@45cc049,
move the platform selection to compile time.

(this helps juliac a ton)
Rebase and extension of @alexfanqi's initial work on porting Julia to
RISC-V. Requires LLVM 19.

Tested on a VisionFive2, built with:

```make
MARCH := rv64gc_zba_zbb
MCPU := sifive-u74

USE_BINARYBUILDER:=0

DEPS_GIT = llvm
override LLVM_VER=19.1.1
override LLVM_BRANCH=julia-release/19.x
override LLVM_SHA1=julia-release/19.x
```

```julia-repl
❯ ./julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.0-DEV.1374 (2024-10-14)
 _/ |\__'_|_|_|\__'_|  |  riscv/25092a3982* (fork: 1 commits, 0 days)
|__/                   |

julia> versioninfo(; verbose=true)
Julia Version 1.12.0-DEV.1374
Commit 25092a3* (2024-10-14 09:57 UTC)
Platform Info:
  OS: Linux (riscv64-unknown-linux-gnu)
  uname: Linux 6.11.3-1-riscv64 #1 SMP Debian 6.11.3-1 (2024-10-10) riscv64 unknown
  CPU: unknown:
              speed         user         nice          sys         idle          irq
       #1  1500 MHz        922 s          0 s        265 s     160953 s          0 s
       #2  1500 MHz        457 s          0 s        280 s     161521 s          0 s
       #3  1500 MHz        452 s          0 s        270 s     160911 s          0 s
       #4  1500 MHz        638 s         15 s        301 s     161340 s          0 s
  Memory: 7.760246276855469 GB (7474.08203125 MB free)
  Uptime: 16260.13 sec
  Load Avg:  0.25  0.23  0.1
  WORD_SIZE: 64
  LLVM: libLLVM-19.1.1 (ORCJIT, sifive-u74)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Environment:
  HOME = /home/tim
  PATH = /home/tim/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
  TERM = xterm-256color


julia> ccall(:jl_dump_host_cpu, Nothing, ())
CPU: sifive-u74
Features: +zbb,+d,+i,+f,+c,+a,+zba,+m,-zvbc,-zksed,-zvfhmin,-zbkc,-zkne,-zksh,-zfh,-zfhmin,-zknh,-v,-zihintpause,-zicboz,-zbs,-zvknha,-zvksed,-zfa,-ztso,-zbc,-zvknhb,-zihintntl,-zknd,-zvbb,-zbkx,-zkt,-zvkt,-zicond,-zvksh,-zvfh,-zvkg,-zvkb,-zbkb,-zvkned


julia> @code_native debuginfo=:none 1+2.
	.text
	.attribute	4, 16
	.attribute	5, "rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0_zifencei2p0_zmmul1p0_zba1p0_zbb1p0"
	.file	"+"
	.globl	"julia_+_3003"
	.p2align	1
	.type	"julia_+_3003",@function
"julia_+_3003":
	addi	sp, sp, -16
	sd	ra, 8(sp)
	sd	s0, 0(sp)
	addi	s0, sp, 16
	fcvt.d.l	fa5, a0
	ld	ra, 8(sp)
	ld	s0, 0(sp)
	fadd.d	fa0, fa5, fa0
	addi	sp, sp, 16
	ret
.Lfunc_end0:
	.size	"julia_+_3003", .Lfunc_end0-"julia_+_3003"

	.type	".L+Core.Float64#3005",@object
	.section	.data.rel.ro,"aw",@progbits
	.p2align	3, 0x0
".L+Core.Float64#3005":
	.quad	".L+Core.Float64#3005.jit"
	.size	".L+Core.Float64#3005", 8

.set ".L+Core.Float64#3005.jit", 272467692544
	.size	".L+Core.Float64#3005.jit", 8
	.section	".note.GNU-stack","",@progbits
```

Lots of bugs guaranteed, but with this we at least have a functional
build and REPL for further development by whoever is interested.

Also requires Linux 6.4+, since the fallback processor detection
used here relies on LLVM's `sys::getHostCPUFeatures`, which for
RISC-V is implemented using hwprobe introduced in 6.4. We could
probably add a fallback that parses `/proc/cpuinfo`, either by building
a CPU database much like how we've done for AArch64, or by parsing the
actual ISA string contained there. That would probably also be a good
place to add support for profiles, which are supposedly the way forward
to package RISC-V binaries. That can happen in follow-up PRs though.
For now, on older kernels, use the `-C` arg to Julia to specify an ISA.

Co-authored-by: Alex Fan <[email protected]>
There are tests elsewhere that i) make sure there is no zero-arg methods
of these functions and ii) tests that e.g. `+()` throws a `MethodError`.
Without this patch there are test errors whenever the same test process
runs both of these tests.
E.g. this allows `finalizer` inlining in the following case:
```julia
mutable struct ForeignBuffer{T}
    const ptr::Ptr{T}
end
const foreign_buffer_finalized = Ref(false)
function foreign_alloc(::Type{T}, length) where T
    ptr = Libc.malloc(sizeof(T) * length)
    ptr = Base.unsafe_convert(Ptr{T}, ptr)
    obj = ForeignBuffer{T}(ptr)
    return finalizer(obj) do obj
        Base.@assume_effects :notaskstate :nothrow
        foreign_buffer_finalized[] = true
        Libc.free(obj.ptr)
    end
end
function f_EA_finalizer(N::Int)
    workspace = foreign_alloc(Float64, N)
    GC.@preserve workspace begin
        (;ptr) = workspace
        Base.@assume_effects :nothrow @noinline println(devnull, "ptr = ", ptr)
    end
end
```
```julia
julia> @code_typed f_EA_finalizer(42)
CodeInfo(
1 ── %1  = Base.mul_int(8, N)::Int64
│    %2  = Core.lshr_int(%1, 63)::Int64
│    %3  = Core.trunc_int(Core.UInt8, %2)::UInt8
│    %4  = Core.eq_int(%3, 0x01)::Bool
└───       goto #3 if not %4
2 ──       invoke Core.throw_inexacterror(:convert::Symbol, UInt64::Type, %1::Int64)::Union{}
└───       unreachable
3 ──       goto #4
4 ── %9  = Core.bitcast(Core.UInt64, %1)::UInt64
└───       goto #5
5 ──       goto #6
6 ──       goto #7
7 ──       goto #8
8 ── %14 = $(Expr(:foreigncall, :(:malloc), Ptr{Nothing}, svec(UInt64), 0, :(:ccall), :(%9), :(%9)))::Ptr{Nothing}
└───       goto #9
9 ── %16 = Base.bitcast(Ptr{Float64}, %14)::Ptr{Float64}
│    %17 = %new(ForeignBuffer{Float64}, %16)::ForeignBuffer{Float64}
└───       goto #10
10 ─ %19 = $(Expr(:gc_preserve_begin, :(%17)))
│    %20 = Base.getfield(%17, :ptr)::Ptr{Float64}
│          invoke Main.println(Main.devnull::Base.DevNull, "ptr = "::String, %20::Ptr{Float64})::Nothing
│          $(Expr(:gc_preserve_end, :(%19)))
│    %23 = Main.foreign_buffer_finalized::Base.RefValue{Bool}
│          Base.setfield!(%23, :x, true)::Bool
│    %25 = Base.getfield(%17, :ptr)::Ptr{Float64}
│    %26 = Base.bitcast(Ptr{Nothing}, %25)::Ptr{Nothing}
│          $(Expr(:foreigncall, :(:free), Nothing, svec(Ptr{Nothing}), 0, :(:ccall), :(%26), :(%25)))::Nothing
└───       return nothing
) => Nothing
```

However, this is still a WIP. Before merging, I want to improve EA's
precision a bit and at least fix the test case that is currently marked
as `broken`. I also need to check its impact on compiler performance.

Additionally, I believe this feature is not yet practical. In
particular, there is still significant room for improvement in the
following areas:
- EA's interprocedural capabilities: currently EA is performed ad-hoc
for limited frames because of latency reasons, which significantly
reduces its precision in the presence of interprocedural calls.
- Relaxing the `:nothrow` check for finalizer inlining: the current
algorithm requires `:nothrow`-ness on all paths from the allocation of
the mutable struct to its last use, which is not practical for
real-world cases. Even when `:nothrow` cannot be guaranteed, auxiliary
optimizations such as inserting a `finalize` call after the last use
might still be possible (#55990).
1. Adjust the docstring for `Future`, which had its design changed late
in that PR and is now confusing.
2. Add additional assertions validating the API assumptions of the
`Future` API. I found it too easy to accidentally misuse this and cause
hard-to-debug failures. The biggest change is that `isready` accounts
for delayed assignments again, which allows an additional invariant that
incomplete tasks must always have other pending tasks, allowing for
infinite loop detection in the scheduler.
3. A small fix to use the AbstractInterpreter that created the
InferenceState for the callback. We haven't fully defined the semantics
of mixed-interpreter inference stacks, but downstream packages were
using is and this at least makes it mostly work again.
This code was originally added in
df81bf9 where Core.Compiler would keep
an array of all the things it inferred, which could then be provieded to
the runtime to be included in the package image. In
113efb6 keeping the array itself became
a runtime service for locking considerations. As a result, the role of
Core.Compiler here is a bit weird. It has the enable switch and the GC
root, but all the actual state is being managed by the runtime.

It would be desirable to remove the Core.Compiler reference, so that
loading.jl can function even if `Core.Compiler` does not exist (which is
in theory supposed to be possible, even though we currently never run in
such a configuration; that said, post trimming one might imagine useful
instances of such a setup).

To do this, put the runtime fully in charge of managing this array.
Core.Compiler will call the callback unconditionally for all newly
inferred cis and the runtime can decide whether to save it or not.

Extracted from #56128
Also use a round robin to only return stacks one thread at a time to
avoid contention on munmap syscalls.
Using

https://github.com/gbaraldi/cilkbench_julia/blob/main/cilk5julia/nqueens.jl
as a benchmark it's about 12% faster wall time. This benchmark has other
weird behaviours specially single threaded. Where if calls `wait`
thousandas of times per second, and if single threaded every single one
does a `jl_process_events` call which is a syscall + preemption. So it
looks like a hang. With threads the issue isn't there

The idea behind the round robin is twofold. One we are just freeing too
much and talking with vtjnash we maybe want some less agressive
behaviour, the second is that munmap takes a lock in most OSs. So doing
it in parallel has severe negative scaling.
Fixes #51277, though we give no guarantee that it keeps working this
way, or that calling `setrounding_raw` won't lead to other undefined
behavior.

To give some examples:

```julia
julia> t = Base.Rounding.setrounding_raw(Float64, Base.Rounding.to_fenv(RoundDown)) do
           Task(() -> println(rounding(Float64)))
       end
Task (runnable) @0x000000010dff04c0

julia> rounding(Float64)
RoundingMode{:Nearest}()

julia> wait(schedule(t))
RoundingMode{:Down}() # currently gives RoundingMode{:Nearest}()

julia> rounding(Float64)
RoundingMode{:Nearest}()

julia> Base.Rounding.setrounding_raw(Float64, Base.Rounding.to_fenv(RoundDown)) do
           Threads.@threads :static for i = 1:Threads.nthreads()
               println(Threads.threadid() =>  rounding(Float64))
           end
       end
1 => RoundingMode{:Down}()
2 => RoundingMode{:Down}() # currently gives RoundingMode{:Nearest}()
4 => RoundingMode{:Down}() # currently gives RoundingMode{:Nearest}()
3 => RoundingMode{:Down}() # currently gives RoundingMode{:Nearest}()
```
_cf_: #56147

.html and .pdf rendering may be resolved for this content

the ticket's mentioned build.md file was left unattended to
This still points to 1.10 and should be updated.
…ces (#56185)

The `reflection.jl` file provides a large amount of functionality
covering everything from helpers for access to core runtime data
structures to setting up particular inference problems. It is included
by both Base and Core.Compiler, but the functions that use the compiler,
don't really make sense in the latter. In preparation for #56128, and
stop including the compiler-dependent pieces in Core.Compiler.

While we're here, also move a few generically useful reflection
functions out of Core.Compiler, so users that access
them don't have to load the compiler.

Split out from #56128, but doesn't make any semantic changes by itself,
so should be quick/easy to merge.
It makes sense that we originally added this to the compiler, but these
annotations are really a runtime feature that the compiler simply reads
to allow it to make additional assumptions. The runtime should not
semantically depend on the compiler for this, so move these definitions
to expr.jl. The practical effect of this right now is that Base gains a
second copy of this code. Post #56128, the compiler will use the Base
copy of this.

Split out from #56128.
Also, add an aggressive constprop annotation to `generic_matvecmul!`.

Together, these improve performance:
```julia
julia> A = rand(Int,100,100);

julia> @Btime $A' * $A';
  290.203 μs (405 allocations: 175.98 KiB) # v"1.12.0-DEV.1364"
  270.008 μs (5 allocations: 79.11 KiB) # This PR
```
With this, the error message in
https://buildkite.com/julialang/julia-master/builds/41054#019294ca-e2c5-41f2-a897-e2959715f154
would become
```julia
Error in testset errorshow:
Test Failed at /home/jishnu/juliaPR/usr/share/julia/test/errorshow.jl:226
  Expression: typeof(err) === $(Expr(:escape, :MethodError))
   Evaluated: StackOverflowError === MethodError
     Context: expr = :(+())
```
Having the failing expression displayed makes it easier to locate the
source of the error.
Keno and others added 30 commits November 11, 2024 16:04
This was part of #56509, but is an independent bugfix. The basic issue
is that these macro were using `do` block internally. This is
undesirable for test macros, because we would like them not to affect
the behavior of what they're testing. E.g. right now:
```
julia> using Test

julia> const x = 1
1

julia> @test_nowarn const x = 1
ERROR: syntax: `global const` declaration not allowed inside function around /home/keno/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:927
Stacktrace:
 [1] top-level scope
   @ REPL[3]:1
```

This PR just writes out the try/finally manually, so the above works
fine after this PR.
This appears to improve performance.
```julia
julia> A = zeros(100_000);

julia> x = (i for i in axes(A,1));

julia> @Btime copyto!($A, 1, $x, 1, length($A));
  64.162 μs (0 allocations: 0 bytes) # nightly v"1.12.0-DEV.1593"
  52.532 μs (0 allocations: 0 bytes) # this PR
```
A function to obtain a view of a diagonal of a matrix is useful, and
this is clearly being used widely within `LinearAlgebra`.

The implementation here iterates according to the `IndexStyle` of the
array:
```julia
julia> using LinearAlgebra

julia> A = reshape(1:9, 3, 3)
3×3 reshape(::UnitRange{Int64}, 3, 3) with eltype Int64:
 1  4  7
 2  5  8
 3  6  9

julia> diagview(A,1)
2-element view(::UnitRange{Int64}, 4:4:8) with eltype Int64:
 4
 8

julia> T = Tridiagonal(1:3, 3:6, 4:6)
4×4 Tridiagonal{Int64, UnitRange{Int64}}:
 3  4  ⋅  ⋅
 1  4  5  ⋅
 ⋅  2  5  6
 ⋅  ⋅  3  6

julia> diagview(T,1)
3-element view(::Tridiagonal{Int64, UnitRange{Int64}}, StepRangeLen(CartesianIndex(1, 2), CartesianIndex(1, 1), 3)) with eltype Int64:
 4
 5
 6
```

Closes #30250
This improves performance, as the loops in `istriu` and `istril` may be
fused to improve cache-locality.
This also changes the quick-return behavior, and only returns after the
check over all the upper or lower bands for a column is complete.

```julia
julia> using LinearAlgebra

julia> A = zeros(2, 10_000);

julia> @Btime isdiag($A);
  32.682 μs (0 allocations: 0 bytes) # nightly v"1.12.0-DEV.1593"
  9.481 μs (0 allocations: 0 bytes) # this PR

julia> A = zeros(10_000, 2);

julia> @Btime isdiag($A);
  10.288 μs (0 allocations: 0 bytes)  # nightly 
  2.579 μs (0 allocations: 0 bytes) # this PR

julia> A = zeros(100, 100);

julia> @Btime isdiag($A);
  6.616 μs (0 allocations: 0 bytes) # nightly
  3.075 μs (0 allocations: 0 bytes) # this PR

julia> A = diagm(0=>1:100);  A[3,4] = 1;

julia> @Btime isdiag($A);
  2.759 μs (0 allocations: 0 bytes) # nightly
  85.371 ns (0 allocations: 0 bytes) # this PR
```

A similar change is added to `istriu`/`istril` as well, so that
```julia
julia> A = zeros(2, 10_000);

julia> @Btime istriu($A); # trivial
  7.358 ns (0 allocations: 0 bytes) # nightly
  13.779 ns (0 allocations: 0 bytes) # this PR

julia> @Btime istril($A);
  33.464 μs (0 allocations: 0 bytes) # nightly
  9.476 μs (0 allocations: 0 bytes) # this PR

julia> A = zeros(10_000, 2);

julia> @Btime istriu($A);
  10.020 μs (0 allocations: 0 bytes) # nightly
  2.620 μs (0 allocations: 0 bytes) # this PR

julia> @Btime istril($A); # trivial
  6.793 ns (0 allocations: 0 bytes) # nightly
  14.473 ns (0 allocations: 0 bytes) # this PR

julia> A = zeros(100, 100);

julia> @Btime istriu($A);
  3.435 μs (0 allocations: 0 bytes) # nightly
  1.637 μs (0 allocations: 0 bytes) # this PR

julia> @Btime istril($A);
  3.353 μs (0 allocations: 0 bytes) # nightly
  1.661 μs (0 allocations: 0 bytes) # this PR
```

---------

Co-authored-by: Daniel Karrasch <[email protected]>
This PR replaces `MulAddMul` arguments by `alpha, beta` pairs in the
multiplication methods involving `Diagonal` matrices, and constructs the
objects exactly where they are required. Such an approach improves
latency.
```julia
julia> D = Diagonal(1:2000); A = rand(size(D)...); C = similar(A);

julia> @time mul!(C, A, D, 1, 2); # first-run latency is reduced
  0.129741 seconds (180.18 k allocations: 9.607 MiB, 88.87% compilation time) # nightly v"1.12.0-DEV.1505"
  0.083005 seconds (146.68 k allocations: 7.442 MiB, 82.94% compilation time) # this PR

julia> @Btime mul!($C, $A, $D, 1, 2); # runtime performance is unaffected 
  4.983 ms (0 allocations: 0 bytes) # nightly
  4.938 ms (0 allocations: 0 bytes) # this PR
```

This PR sets the stage for a similar change for
`Bidiagonal`/`Tridiaognal` matrices, which would lead to a bigger
reduction in latencies.
Stdlib: SparseArrays
URL: https://github.com/JuliaSparse/SparseArrays.jl.git
Stdlib branch: main
Julia branch: master
Old commit: 0dd8d45
New commit: 14333ea
Julia version: 1.12.0-DEV
SparseArrays version: 1.12.0
Bump invoked by: @ViralBShah
Powered by:
[BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl)

Diff:
JuliaSparse/SparseArrays.jl@0dd8d45...14333ea

```
$ git log --oneline 0dd8d45..14333ea
14333ea Break recursion (#579)
07cf4a6 Update ci.yml (#578)
33491e0 added diagonal-sparse multiplication (#564)
8f02b7f doc: move solvers doc to `src\solvers.md` (#576)
485fd4b Inline sparse-times-dense in-place multiplication (#567)
f10d4da added specialized method for 3-argument dot with diagonal matrix (#565)
70c06b1 Diagonal-sandwiched triple product for SparseMatrixCSC (#562)
313a04f Change default QR tolerance to match SPQR (#557)
81d49e9 Update ci.yml (#558)
```

Co-authored-by: Dilum Aluthge <[email protected]>
Since the expressions generated by those macros are almost identical,
the implementation could be changed to use a common helper function to
create expressions for each case.
This seems to have negligible impact on size or performance, but makes
code quality much better.
This does not yet make the compiler tests independently runnable using
`] test Compiler`; it only moves the files and wires them up to continue
running as part of the Base test runner.

---------

Co-authored-by: Shuhei Kadowaki <[email protected]>
Split out from #56509 to facilitate adjusting downstream packages.
This could previously cause any use of llvmcall to crash during ji
generate or generate bad code. Now it uses the llvm attribute to specify
this correctly.
These didn't get merged when the Compiler moved out, because the Base
function doesn't get defined until very late in the build process.
However, they are semantically the same, so merge their method tables.
Includes a bunch small change, but importantly the fix for the search
not opening.

I think we should also backport this to 1.10 and 1.11, but those should
probably be done manually, to avoid manifest conflicts. Should I just PR
the relevant `release-*` branches?
Co-authored-by: Lilith Orion Hafner <[email protected]>
Co-authored-by: Neven Sajko <[email protected]>
The new `Compiler` package has a special stub that bypasses compilig a
separate copy if you have `dev`'ed the version that's already compiled
into the sysimg. It wasn't quite working properly in the final version
of that PR if a compiler so loaded was a dependency of another
precompiled package. Fix that and add a test to make sure it doesn't
regress.
I'm cleaning up some downstream packages following the compiler split
and these were commonly found in compat code. One is the `Base.copy`
method for Phi(C)Node, which we had an implementation for, just not
wired up to `copy`. The other is `block_for_inst` with an SSAValue,
which exists for IncremetalCompact, but not IRCode, so add the latter
for consistency.
Previously, definitions of overloaded `Base.show` and `IRShow.show` were
mixed, causing `show` to not function properly for `Compiler` as a
standard library. This commit fixes that issue and also includes some
minor cleanups.
`Compiler.:(==)` is now identical to `Base.:(==)`, so the following `==`
methods defined in typelattice.jl are considered type piracy:
https://github.com/JuliaLang/julia/blob/1edc6f1b7752ed67059020ba7ce174dffa225954/Compiler/src/typelattice.jl#L194-L197

In fact, loading `Compiler` as a standard library with this code can
sometimes result in errors like the following:
```julia
julia> using Compiler

julia> Int == Core.Const(1)
ERROR: MethodError: ==(::Type{Int64}, ::Core.Const) is ambiguous.
...
```

Since these `==` definitions no longer seem necessary, this commit
simply removes them to resolve the issue.

@nanosoldier `runbenchmarks("inference", vs=":master")`
…6496)

This fixes some bugs that prevent compile-all from working correctly at
all, and uses more of it for normal compile. Increases sysimg size from
about 140 to 170 MB of data and 11 to 15 MB of code
#56553)

Since #56409, Compiler.jl as a standard library has
become available. However, for Julia versions prior to this change, even
though the stdlib can be installed via Pkg.jl, the precompilation fails
due to code compatibility issues. Consequently, when an external package
that uses the Compiler stdlib adds Compiler.jl to its Project.toml, the
package would stop working on older Julia versions.

To address this, this commit adopts the same approach as JET.jl.
Specifically, on older Julia versions, a dummy `Compiler` module is
defined, allowing dependent packages to switch between using the
Compiler.jl stdlib or the previous `Core.Compiler`. While this is a
somewhat hacky solution, it should resolve the issue for now.

Also includes a change to include `ssair/show.jl` in the context of
`Compiler` to ensure that stale precompilation caches are not used.

And as a result this commit bumps the version of the Compiler.jl
standard library.
We care more and more now that our inference graph exactly matches the
callgraph, even in cases like this where we can easily prove that the
inference graph is simpler than the full callgraph. However, given when
the optimizer runs, it expects this information to be available and
valid as soon as the cycles are completed.
We don't have frontend syntax for it, but there is a use case for having
`:new_opaque_closure` take an OC constructed from an optimized
OpaqueClosure (and just replacing the capture environment). In this
case, there is nothing inference can do to introspect into the opaque
closure, so it just needs to bail out early.
…56563)

Caught this by running threads test repeatedly locally: the sweep needs
to acquire engine_lock, so we need to make sure to release that first
(the other jl_unique_gcsafe_lock users shouldn't care about this
ordering since they don't acquire their locks during sweeping)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.