State dependent type construction?

[ivirshup]

Basically, instantiating one of these types before the other causes instantiation of the latter to throw an UndefRefError.

julia> Dict{Tuple{Vararg{Any,2}},Float64}()
Dict{Tuple{Vararg{Any,2}},Float64} with 0 entries

julia> Dict{Tuple{Any, Any},Float64}()
ERROR: UndefRefError: access to undefined reference
 in copy!(::Base.LinearFast, ::Array{Tuple{Any,Any},1}, ::Base.LinearFast, ::Array{Tuple{Vararg{Any,2}},1}) at ./abstractarray.jl:254
 in Dict{Tuple{Any,Any},Float64}() at ./dict.jl:429
 in eval(::Module, ::Any) at ./boot.jl:225
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

In a new session

julia> Dict{Tuple{Any, Any},Float64}()
Dict{Tuple{Any,Any},Float64} with 0 entries

julia> Dict{Tuple{Vararg{Any,2}},Float64}()
ERROR: UndefRefError: access to undefined reference
 in copy!(::Base.LinearFast, ::Array{Tuple{Vararg{Any,2}},1}, ::Base.LinearFast, ::Array{Tuple{Any,Any},1}) at ./abstractarray.jl:254
 in Dict{Tuple{Vararg{Any,2}},Float64}() at ./dict.jl:429
 in eval(::Module, ::Any) at ./boot.jl:225
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

julia> versioninfo()
Julia Version 0.5.0-dev+4361
Commit bc56e32 (2016-05-27 12:01 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin15.5.0)
  CPU: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, has well)

I found this in JuMP occurring with calls to JuMP._similar from getvalue. Similar errors didn't occur for Arrays, or a user defined type, so I think it's Dict specific. The value type doesn't seem to matter to this error.

[ivirshup]

@dpsanders and I were taking a look at this and found there are issues with the dispatch of convert when creating arrays inside the dictionary.

julia> convert(Array{Tuple{Any, Any}}, Array{Tuple{Any, Any}}())
0-dimensional Array{Tuple{Any,Any},0}:
#undef

julia> convert(Array{Tuple{Vararg{Any, 2}}}, Array{Tuple{Vararg{Any, 2}}}())
ERROR: UndefRefError: access to undefined reference
 in unsafe_copy!(::Array{Tuple{Any,Any},0}, ::Int64, ::Array{Tuple{Any,Any},0}, ::Int64, ::Int64) at ./array.jl:51
 in copy!(::Array{Tuple{Any,Any},0}, ::Int64, ::Array{Tuple{Any,Any},0}, ::Int64, ::Int64) at ./array.jl:62
 in convert(::Type{Array{Tuple{Vararg{Any,2}},N}}, ::Array{Tuple{Any,Any},0}) at ./array.jl:200
 in eval(::Module, ::Any) at ./boot.jl:225
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

julia> @which convert(Array{Tuple{Any, Any}}, Array{Tuple{Any, Any}}())
convert{T,n}(::Type{Array{T,N<:Any}}, x::Array{T,n}) at array.jl:197

julia> @which convert(Array{Tuple{Vararg{Any, 2}}}, Array{Tuple{Vararg{Any, 2}}}())
convert{T,n,S}(::Type{Array{T,N<:Any}}, x::AbstractArray{S,n}) at array.jl:200

The reciprocal has similar results.

I believe the crux of the bug is based around:

julia> Array{Tuple{Any, Any}}()
0-dimensional Array{Tuple{Any,Any},0}:
#undef

julia> Array{Tuple{Vararg{Any,2}}}()
0-dimensional Array{Tuple{Any,Any},0}:
#undef

Where, in a new session, the reciprocal holds as well. It seems to me that there is an issue with dispatch on object instantiation, since Core.apply_type has the expected behaviour, so no conversion happens while everything is in the type domain.

julia> Array{Tuple{Any,Any}}
Array{Tuple{Any,Any},N}

julia> Array{Tuple{Vararg{Any,2}}}
Array{Tuple{Vararg{Any,2}},N}

I think the following probably has something to do with all of this:

julia> Base.typeseq(Tuple{Vararg{Any, 2}}, Tuple{Any,Any})
true

This also occurs with user defined types

julia> type A{T}
       x::T
       A() = new()
       end

julia> A{Tuple{Any,Any}}()
A{Tuple{Any,Any}}(#undef)

julia> A{Tuple{Vararg{Any, 2}}}()
A{Tuple{Any,Any}}(#undef)

[vtjnash]

NTuple{2, Any} and Tuple{Any, Any} are simply alternative spellings of the same type. the type system is supposed to know this (and usually does). sometimes it normalizes them to be easier to deal with / compare.

[ivirshup]

Huh. Why is it that NTuple{2, Any} is constructed as Tuple{Any, Any} while Tuple{Vararg{Any, 2}} isn't?

Is what you're saying that the bug here is the dispatch at convert, and the state-dependent object instantiation should be inconsequential?

[vtjnash]

NTuple is a convenience typealias for Tuple{Vararg{...}}

[ivirshup]

Yeah, I checked that when you mentioned it. That's why I'm confused by the types NTuple constructs:

julia> NTuple
Tuple{Vararg{T,N}}

julia> NTuple{2, Any} === Tuple{Vararg{Any,2}}
false

julia> NTuple{2, Any} === Tuple{Any, Any}
true

[JeffBezanson]

Fixed by #17361

[tkelman]

may as well add the additional test case from here, just in case