`Update` performance

[Jolanrensen]

Looking into the implementation of Update, it seems to me that the filtering of rows happens again and again for each column while this could be a one-time operation.

As a broad overview, the current situation looks like:

fun updateImpl() {
    df.replace(columns).with { it: DataColumn -> /* filter, given entire df row and replace column */ }
}

while it could be

fun updateImpl() {
    df.filter { /* Filter rows */ }.replace(columns).with { it: DataColumn -> /* replace column */ }
}

There is of course also a trade-off, because this doesn't work with Update.perCol {}, since it would supply a filtered column instead of the entire column like now. So we need to think this through thoroughly.

[nikitinas]

df.filter { /* Filter rows */ }.replace(columns)...

would be incorrect, as it will return DataFrame with reduced number of rows, but update should keep all original rows and just update some values based on given predicate.

We can not avoid filtering inside every column with current API, because filter uses it argument that represents current value in the column: RowValueFilter = DataRow<T>.(it: C) -> Boolean

[nikitinas]

To implement this performance optimization we need another where clause without access to currently updated value. It may look like this:

df.update { columns }.whereRow { /* expression: RowFilter */ }...

[Jolanrensen]

I see, perRow might be better since it's similar to perCol/perRowCol. That said, having multiple functions do a similar thing with different optimizations for each will definitely confuse users, so let's keep it as is for now.

[nikitinas]

perRow can be used for new value computation, similar to perCol / perRowCol, but not for filtering.

[Jolanrensen]

So it's similar to with. Still confusing for users to choose between with and perRow in that case, right?