Skip to content

DropSimilar should not compute Pearson correlation #2187

Description

@jeromedockes

DropSimilar requires pyarrow to be installed when the input is a polars dataframe, because compute_associations uses pandas to compute associations. However, DropSimilar does not use the Pearson correlations (which is what actually requires pyarrow). compute_associations should have a parameter to compute either Cramer V, Pearson or both (both by default), and DropSimilar should skip Pearson, since it's not used.

We may also want to use that option in the tablereport and compute only cramerv with a warning when we have a polars dataframe and no pyarrow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions