You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a tracking issue for distinguishing between encodings and scalar functions.
Motivation
We know that scalar functions arrays are a specific type of Array / physical encoding. We have roughly defined them as such:
Scalar functions operate over a single row of values and produce a single output.
This means that its child arrays must be the same length, and the execution of the ScalarFnArray produces an array of the same length as all of its children.
But that is it. We also don't really have a good sense of what is not a scalar function other than the fact that children with different lengths are not scalar functions.
Grey Area
There are a few instances where we are unsure if an algorithm should be represented as a scalar function or as a normal physical encoding.
L2Denorm (which we are renaming to NormalizedAdd a Normalized encoding for tensor columns #9138): This is a very weird scalar function that we wanted to change to an encoding because there were internal invariants and it was just very awkward to use (and there were some hacks in different places just for it).
FoR: This is literally just an addition!
ZigZag: There is a well-understood single function, almost just like a multiplication.
ALP: This is basically just a scalar function with a type coercion.
Dict: This is scalar only in the codes, and the main reason this is not a proper scalar function is because the values child might be large, so it exists as a full array child.
Unresolved Questions
It would be good for us to come up with a formal definition, and if that definition means we need to migrate some of the existing encodings, we should do that.
This is a tracking issue for distinguishing between encodings and scalar functions.
Motivation
We know that scalar functions arrays are a specific type of
Array/ physical encoding. We have roughly defined them as such:ScalarFnArrayproduces an array of the same length as all of its children.But that is it. We also don't really have a good sense of what is not a scalar function other than the fact that children with different lengths are not scalar functions.
Grey Area
There are a few instances where we are unsure if an algorithm should be represented as a scalar function or as a normal physical encoding.
L2Denorm(which we are renaming toNormalizedAdd a Normalized encoding for tensor columns #9138): This is a very weird scalar function that we wanted to change to an encoding because there were internal invariants and it was just very awkward to use (and there were some hacks in different places just for it).FoR: This is literally just an addition!ZigZag: There is a well-understood single function, almost just like a multiplication.ALP: This is basically just a scalar function with a type coercion.Dict: This is scalar only in the codes, and the main reason this is not a proper scalar function is because thevalueschild might be large, so it exists as a full array child.Unresolved Questions
It would be good for us to come up with a formal definition, and if that definition means we need to migrate some of the existing encodings, we should do that.
Related