Distressing performance unless promotion is manual

[timholy]

Given the impact of the splatting penalty in other areas, I've long wondered why these definitions aren't problematic. It turns out that, in some circumstances, they are:

julia> using FixedPointNumbers

julia> x = 0.2f0
0.2f0

julia> y = N0f8(0.2)
0.2N0f8

julia> @code_llvm x*y

define float @"julia_*_68175"(float, %Normed*) #0 !dbg !5 {
top:
  %ptls_i8 = call i8* asm "movq %fs:0, $0;\0Aaddq $$-10896, $0", "=r,~{dirflag},~{fpsr},~{flags}"() #3
  %ptls = bitcast i8* %ptls_i8 to i8****
  %2 = alloca [4 x i8**], align 8
  %.sub = getelementptr inbounds [4 x i8**], [4 x i8**]* %2, i64 0, i64 0
  %3 = getelementptr [4 x i8**], [4 x i8**]* %2, i64 0, i64 2
  %4 = bitcast i8*** %3 to i8*
  call void @llvm.memset.p0i8.i32(i8* %4, i8 0, i32 16, i32 8, i1 false)
  %5 = bitcast [4 x i8**]* %2 to i64*
  store i64 4, i64* %5, align 8
  %6 = getelementptr [4 x i8**], [4 x i8**]* %2, i64 0, i64 1
  %7 = bitcast i8* %ptls_i8 to i64*
  %8 = load i64, i64* %7, align 8
  %9 = bitcast i8*** %6 to i64*
  store i64 %8, i64* %9, align 8
  store i8*** %.sub, i8**** %ptls, align 8
  %10 = getelementptr [4 x i8**], [4 x i8**]* %2, i64 0, i64 3
  %11 = call [2 x float] @julia_promote_68176(float %0, %Normed* %1)
  %.elt = extractvalue [2 x float] %11, 0
  %.elt2 = extractvalue [2 x float] %11, 1
  store i8** inttoptr (i64 139997130231784 to i8**), i8*** %3, align 8
  %12 = call i8** @jl_gc_pool_alloc(i8* %ptls_i8, i32 1440, i32 16)
  %13 = getelementptr i8*, i8** %12, i64 -1
  %14 = bitcast i8** %13 to i8***
  store i8** inttoptr (i64 139997180171536 to i8**), i8*** %14, align 8
  %15 = bitcast i8** %12 to float*
  store float %.elt, float* %15, align 8
  %.sroa_raw_cast5 = bitcast i8** %12 to i8*
  %.sroa_raw_idx6 = getelementptr inbounds i8, i8* %.sroa_raw_cast5, i64 4
  %16 = bitcast i8* %.sroa_raw_idx6 to float*
  store float %.elt2, float* %16, align 4
  store i8** %12, i8*** %10, align 8
  %17 = call i8** @jl_f__apply(i8** null, i8*** %3, i32 2)
  %18 = bitcast i8** %17 to float*
  %19 = load float, float* %18, align 16
  %20 = load i64, i64* %9, align 8
  store i64 %20, i64* %7, align 8
  ret float %19
}

julia> function myprod1(x, y)
           xp, yp = promote(x, y)
           xp*yp
       end
myprod1 (generic function with 1 method)

julia> function myprod2(x, y)
           T = promote_type(typeof(x), typeof(y))
           convert(T, x) * convert(T, y)
       end
myprod2 (generic function with 1 method)

julia> @code_llvm myprod1(x, y)

define float @julia_myprod1_68215(float, %Normed*) #0 !dbg !5 {
top:
  %2 = call [2 x float] @julia_promote_68176(float %0, %Normed* %1)
  %.elt = extractvalue [2 x float] %2, 0
  %.elt2 = extractvalue [2 x float] %2, 1
  %3 = fmul float %.elt, %.elt2
  ret float %3
}

julia> @code_llvm myprod2(x, y)

define float @julia_myprod2_68216(float, %Normed*) #0 !dbg !5 {
top:
  %2 = getelementptr inbounds %Normed, %Normed* %1, i64 0, i32 0
  %3 = load i8, i8* %2, align 1
  %4 = uitofp i8 %3 to float
  %5 = fmul fast float %4, 0x3F70101020000000
  %6 = fmul float %5, %0
  ret float %6
}

julia> using BenchmarkTools

julia> @benchmark $x*$y seconds=1
BenchmarkTools.Trial: 
  memory estimate:  64 bytes
  allocs estimate:  4
  --------------
  minimum time:     73.504 ns (0.00% GC)
  median time:      74.888 ns (0.00% GC)
  mean time:        79.331 ns (3.29% GC)
  maximum time:     1.387 μs (87.63% GC)
  --------------
  samples:          10000
  evals/sample:     972
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark myprod1($x, $y) seconds=1
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     5.037 ns (0.00% GC)
  median time:      5.041 ns (0.00% GC)
  mean time:        5.101 ns (0.00% GC)
  maximum time:     18.131 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark myprod2($x, $y) seconds=1
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     3.703 ns (0.00% GC)
  median time:      3.706 ns (0.00% GC)
  mean time:        3.738 ns (0.00% GC)
  maximum time:     14.419 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
  time tolerance:   5.00%
  memory tolerance: 1.00%

Note that this is in stark contrast to the case with "conventional" types:

julia> z = 3
3

julia> @which x*z
*(x::Number, y::Number) in Base at promotion.jl:247

julia> @which x*y
*(x::Number, y::Number) in Base at promotion.jl:247

julia> @benchmark $x*$z seconds=1
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     2.025 ns (0.00% GC)
  median time:      2.030 ns (0.00% GC)
  mean time:        2.060 ns (0.00% GC)
  maximum time:     12.631 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @code_llvm x*z

define float @"julia_*_68579"(float, i64) #0 !dbg !5 {
top:
  %2 = sitofp i64 %1 to float
  %3 = fmul float %2, %0
  ret float %3
}

Before I make the obvious fix, is there a deeper issue to understand here?

Thanks to @Evizero for the observation that led to this investigation. CC @vchuravy, since it seriously affects FPN.

[martinholters]

julia> @code_typed x*z
CodeInfo(:(begin 
        return (Base.mul_float)(x, (Base.sitofp)(Float32, y)::Float32)::Float32
    end))=>Float32

julia> @code_typed x*y
CodeInfo(:(begin 
        return (Core._apply)(Base.*, $(Expr(:invoke, MethodInstance for promote(::Float32, ::FixedPointNumbers.Normed{UInt8,8}), :(Base.promote), :(x), :(y))))::Float32
    end))=>Float32

although promote(x,y) is perfectly inferable. Why isn't the _apply elided here?

Even more mysterious to me:

julia> @code_typed x+y
CodeInfo(:(begin 
        $(Expr(:inbounds, false))
        # meta: location REPL[24] promote 2
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl convert 66
        SSAValue(3) = (Base.checked_trunc_uint)(UInt8, 1)::UInt8
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl rawone 32
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl one 31
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl one 28
        SSAValue(1) = (Base.checked_trunc_uint)(UInt8, 255)::UInt8
        SSAValue(0) = (Core.sizeof)($(QuoteNode(UInt8)))::Int64
        SSAValue(2) = (Base.sub_int)((Base.mul_int)(8, SSAValue(0))::Int64, $(QuoteNode(8)))::Int64
        # meta: pop location
        # meta: pop location
        # meta: pop location
        y@_4 = (Base.FastMath.mul_float_fast)((Base.uitofp)(Float32, (Core.getfield)(y@_3, :i)::UInt8)::Float32, (Base.FastMath.div_float_fast)((Base.uitofp)(Float32, SSAValue(3))::Float32, (Base.uitofp)(Float32, (Base.select_value)((Base.sle_int)(0, SSAValue(2))::Bool, (Base.lshr_int)(SSAValue(1), (Base.bitcast)(UInt64, SSAValue(2)))::UInt8, (Base.shl_int)(SSAValue(1), (Base.bitcast)(UInt64, (Base.neg_int)(SSAValue(2))::Int64))::UInt8)::UInt8)::Float32)::Float32)::Float32
        # meta: pop location
        # meta: pop location
        $(Expr(:inbounds, :pop))
        return (Base.add_float)(x, y@_4)::Float32
    end))=>Float32

julia> mytimes(x::Number, y::Number) = *(promote(x,y)...)
mytimes (generic function with 1 method)

julia> @code_typed mytimes(x,y)
CodeInfo(:(begin 
        $(Expr(:inbounds, false))
        # meta: location REPL[24] promote 2
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl convert 66
        SSAValue(3) = (Base.checked_trunc_uint)(UInt8, 1)::UInt8
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl rawone 32
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl one 31
        # meta: location /home/hol/.julia/v0.6/FixedPointNumbers/src/normed.jl one 28
        SSAValue(1) = (Base.checked_trunc_uint)(UInt8, 255)::UInt8
        SSAValue(0) = (Core.sizeof)($(QuoteNode(UInt8)))::Int64
        SSAValue(2) = (Base.sub_int)((Base.mul_int)(8, SSAValue(0))::Int64, $(QuoteNode(8)))::Int64
        # meta: pop location
        # meta: pop location
        # meta: pop location
        y@_4 = (Base.FastMath.mul_float_fast)((Base.uitofp)(Float32, (Core.getfield)(y@_3, :i)::UInt8)::Float32, (Base.FastMath.div_float_fast)((Base.uitofp)(Float32, SSAValue(3))::Float32, (Base.uitofp)(Float32, (Base.select_value)((Base.sle_int)(0, SSAValue(2))::Bool, (Base.lshr_int)(SSAValue(1), (Base.bitcast)(UInt64, SSAValue(2)))::UInt8, (Base.shl_int)(SSAValue(1), (Base.bitcast)(UInt64, (Base.neg_int)(SSAValue(2))::Int64))::UInt8)::UInt8)::Float32)::Float32)::Float32
        # meta: pop location
        # meta: pop location
        $(Expr(:inbounds, :pop))
        return (Base.mul_float)(x, y@_4)::Float32
    end))=>Float32

Why does it work for + and even mytimes, but not for *?

[KristofferC]

FWIW this happened for us too: Ferrite-FEM/Tensors.jl@691481c. It is an easy performance trap to fall in.

[martinholters]

Also interesting:

julia> @benchmark $x*$y
BenchmarkTools.Trial: 
  memory estimate:  64 bytes
  allocs estimate:  4
  --------------
  minimum time:     71.349 ns (0.00% GC)
  median time:      76.081 ns (0.00% GC)
  mean time:        79.529 ns (3.42% GC)
  maximum time:     1.282 μs (90.51% GC)
  --------------
  samples:          10000
  evals/sample:     974
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> import Base.*

julia> *(x::Number, y::Number) = (z=promote(x,y); *(z...))
WARNING: Method definition *(Number, Number) in module Base at promotion.jl:247 overwritten in module Main at REPL[8]:1.
* (generic function with 182 methods)

julia> @benchmark $x*$y
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     3.073 ns (0.00% GC)
  median time:      3.632 ns (0.00% GC)
  mean time:        3.642 ns (0.00% GC)
  maximum time:     11.594 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @code_typed x*y
CodeInfo(:(begin 
        z = $(Expr(:invoke, MethodInstance for promote(::Float32, ::FixedPointNumbers.Normed{UInt8,8}), :(Main.promote), :(x), :(y)))
        return (Base.mul_float)((Core.getfield)(z, 1)::Float32, (Core.getfield)(z, 2)::Float32)::Float32
    end))=>Float32

So introducing a helper variable circumvents the problem.

If I read the code in inlining_pass correctly (and that's a big if!), _apply is elided if the expression to be splatted is effect free? Could we be more clever by introducing a temporary for the expression to be splatted (if its type is a tuple) and then using that? Or am I completely off track here?

[timholy]

Cross-ref #21067; if I've done it correctly, this issue won't be closed if it merges, since I suspect this is something we want a deeper fix for. But if that merges, anyone who benchmarks should beware!

[timholy]

The hack sounds good, but why on earth does introducing a temporary help?

[martinholters]

I guess the splatting would essentially transform

foo(expr...) # with expr an expression inferred as yielding (a subtype of) Tuple{Any,Any}

into

foo(expr[1], expr[2])

which is ok if expr is just a variable but would be incorrect if expr contained a function call that may not be effect-free. The difference between x*y and x*z might be caused by different inlining of the promote?

[timholy]

Good thought. promote unfortunately doesn't inline automatically with FixedPoint numbers.

[JeffBezanson]

@martinholters has is right here. We should introduce a temporary location so we can elide apply.