typeof(...) in Core failed to infer error in `@code_warntype`

[bvdmitri]

The issue after the discussion

The @code_warntype typeof(1) gives an uninformative error message.

julia> @code_warntype typeof(1)
typeof(...) @ Core none:0
  failed to infer

The original issue

Consider we write a structure:

struct TypeConverter{T} end

TypeConverter(::Type{T}) where {T} = TypeConverter{T}()

(converter::TypeConverter{T})(something) where {T} = convert(T, something)
(converter::TypeConverter{T})(something::AbstractArray) where {T} = map(converter, something)

Works perfectly fine at first glance:

julia> converter = TypeConverter(Float32)

julia> converter(1.0)
1.0f0

julia> converter([ 1.0, 1.0 ])
2-element Vector{Float32}:
 1.0
 1.0

EDIT: missing $. The problem is that this code does not really work as I would expect it to and it does allocates (type-instability?):

julia> @btime converter(1.0)
  10.803 ns (1 allocation: 16 bytes)
1.0f0

But if I call @code_warntype I get this cryptic error message:

julia> @code_warntype typeof(converter(1.0))
typeof(...) @ Core none:0
  failed to infer

Interestingly enough, @code_warntype does not show any error if I call it directly on converter(1.0) and it does infer Float32. Why does not typeof work in a combination with @code_warntype? And why does the converter call allocates something?

julia> @code_warntype converter(1.0)
MethodInstance for (::TypeConverter{Float32})(::Float64)
  from (converter::TypeConverter{T})(something) where T @ Main REPL[3]:1
Static Parameters
  T = Float32
Arguments
  converter::Core.Const(TypeConverter{Float32}())
  something::Float64
Body::Float32
1 ─ %1 = Main.convert($(Expr(:static_parameter, 1)), something)::Float32
└──      return %1

If I look at the @code_native I also do not see any place where it could allocate:

@code_native converter(1.0)
.section	__TEXT,__text,regular,pure_instructions
	.build_version macos, 13, 0
	.globl	_julia_TypeConverter_12357      ; -- Begin function julia_TypeConverter_12357
	.p2align	2
_julia_TypeConverter_12357:             ; @julia_TypeConverter_12357
; ┌ @ /Users/bvdmitri/.julia/dev/ReactiveMP.jl/src/helpers/helpers.jl:223 within `TypeConverter`
	.cfi_startproc
; %bb.0:                                ; %top
; │┌ @ number.jl:7 within `convert`
; ││┌ @ float.jl:233 within `Float32`
	fcvt	s0, d0
; │└└
	ret
	.cfi_endproc
; └
                                        ; -- End function
.subsections_via_symbols

but it does and the code is slow. How could it be that single fcvt s0, d0 instruction allocate 16 bytes?

@btime converter(1.0)
17.327 ns (1 allocation: 16 bytes)

Motivation

There are a couple of famous issues like #15276 and #47760.

Basically the issue itself slowdowns a code of the form map(e -> convert(T, e), collection).

julia> foo(T, x) = map((e) -> convert(T, e), x)
foo (generic function with 2 methods)

julia> foo(x) = map((e) -> convert(Float64, e), x)
foo (generic function with 2 methods)

julia> x = ones(100);

julia> @btime foo($x);
  33.359 ns (1 allocation: 896 bytes)

julia> @btime foo(Float64, $x);
  9.333 μs (119 allocations: 3.22 KiB)

The structure above is an attempt to circumvent this related issue.

Version

I checked on Julia 1.9-rc1. The issue is present there as well.

Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.5.0)
  CPU: 10 × Apple M2 Pro
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1)
  Threads: 1 on 6 virtual cores

[bvdmitri]

Ok, the problem with allocations are due to the missing interpolation in the @btime macro.
@btime $converter(1.0) works fine, but the

typeof(...) @ Core none:0
  failed to infer

is still strange to me.

julia> foo(x) = x
foo (generic function with 1 method)

julia> @code_warntype typeof(foo(1))
typeof(...) in Core
  failed to infer

julia> bar(x) = typeof(foo(x))
bar (generic function with 1 method)

julia> @code_warntype bar(1)
MethodInstance for bar(::Int64)
  from bar(x) in Main at REPL[8]:1
Arguments
  #self#::Core.Const(bar)
  x::Int64
Body::Type{Int64}
1 ─ %1 = Main.foo(x)::Int64
│   %2 = Main.typeof(%1)::Core.Const(Int64)
└──      return %2

[vtjnash]

typeof is a builtin function, so it can't be "inferred". The error message text for that is awkward though, since it is the same for anything that fails here for any reason

[bvdmitri]

Yeah, the error message misleaded me, and it combined with the fact that I forgot $ sign under the @btime macro. It would be nice to show a better error message. I can give more context and initially I got this in my test, like

converter = TypeConverter(Float32)

@test @inferred(typeof(converter(T)(1.0))) === T

shows me

julia> @inferred(typeof(converter(1.0))) === Float32
ERROR: return type Type{Float32} does not match inferred return type Any

So I called the @code_warntype and got the misleading error message. The code works as I would expect it to, but both @inferred and @code_warntype could handle the typeof better.

[uniment]

I don't see how this is related to #15276 or #47760.

However, what motivated your issue is that foo was not specializing on the type T. We can fix that (see Performance Tips):

julia> foo(::Type{T}, x) where T = map(e -> convert(T, e), x)
       foo(x) = map(e -> convert(Float64, e), x)
foo (generic function with 2 methods)

julia> x = ones(100)
       @btime foo($x)
       @btime foo(Float64, $x);
  49.011 ns (1 allocation: 896 bytes)
  49.848 ns (1 allocation: 896 bytes)

Of course, it makes more sense to define foo(x) = foo(Float64, x).

[bvdmitri]

I agree with you. That indeed fixes the problem, but the motivation for my issue was from a larger context and I only showed a minimal working example. In the real code it is more closer (though still not exactly) to:

function foo(x, y)
     T = eltype(y)
     return map(e -> convert(T, e), x)
end
 
foo(x) = map(e -> convert(Float64, e), x) 

julia> @btime foo($x);
  34.072 ns (1 allocation: 896 bytes)

julia> @btime foo($x, $y);
  7.583 μs (104 allocations: 2.52 KiB)

It's just too much of a footgun for many Julia developers and I still think it is related. In simple cases yes, I can fix it easily with explicit type specialization. Even in the code above I can replace T = eltype(y) with map(e -> convert(eltype(y), e), x), which also would fix the performance issue. It's just a real footgun, both codes are semantically equivalent, but have drastically different performance characteristics (I understand the reasoning behind and why is that happening though).
Its quite hard to achieve the best performance for all cases in general code without some "hacks". TypeConverter is one of the hacks I came up with (maybe not ideal).

The real issue here was that I couldn't really test my implementation because in the process @code_warntype failed on typeof and it misguided me.

[uniment]

Okay, I see what's going on.

Yes, it's a type system performance footgun relating to closures, but it's not related to unnecessary boxing (the anonymous function's T field is not a Core.Box). The issue is that calling typeof on Float64 returns DataType instead of Type{Float64}.

When constructing the anonymous function, the struct's type parameter is typeof(Float64), which is DataType. For example:

julia> f = let T=Float64
           e -> convert(T, e)
       end
       typeof(f)
var"#31#32"{DataType}

As you can see, you cannot tell f's return type just from calling typeof(f), so dynamic dispatch must be used when processing its return value.

One workaround is to construct a Returns, which uses a custom Base._stable_typeof that returns Type{Float64} (source code here):

julia> foo(x, y) = let T=Returns(eltype(y))
           map(e -> convert(T(), e), x)
       end
       foo(x) = foo(x, Vector{Float64})
foo (generic function with 2 methods)

julia> x = ones(100)
       y = ones(100)
       @btime foo($x)
       @btime foo($x, $y);
  47.720 ns (1 allocation: 896 bytes)
  47.533 ns (1 allocation: 896 bytes)

[uniment]

Maybe the question to ask here is:

When a closure captures a local variable, should its constructor call the default vanilla typeof when type-parameterizing the functor, or should it call Base._stable_typeof? Where is the optimal compile time/run time balance on this issue?

[bvdmitri]

I think that this question is out of the scope of the issue here. It has been and is being discussed in many places.
This issue is about the fact that @code_warntype typeof(1) does not work and throws an uninformative error message.
The behaviour of capturing in closures was only a motivation for me. I modified my original message to reflect that better.