You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 1, 2025. It is now read-only.
map_partitions is an opaque layer for utilizing dask-expr's optimization features. We frequently use map_partitions so it may be of limited use, but it would be useful to get in front of map_partitions with column pre-filtering whenever we can. reduce is a straightforward first case.
The advantage being that we can potentially avoid loading columns we don't use, though it will absolutely be dependent on the user workflow and where it invokes other map_partitions calls