Skip to content

Redesign group-by aggregation API#79

Draft
gene-bordegaray wants to merge 3 commits into
gabotechs:gene.bordegaray/2026/06/join-api-redesignfrom
gene-bordegaray:gene.bordegaray/2026/07/groupby-api-redesign
Draft

Redesign group-by aggregation API#79
gene-bordegaray wants to merge 3 commits into
gabotechs:gene.bordegaray/2026/06/join-api-redesignfrom
gene-bordegaray:gene.bordegaray/2026/07/groupby-api-redesign

Conversation

@gene-bordegaray

Copy link
Copy Markdown
Collaborator

Summary

  • Redesign the root CuDFGroupBy aggregation API around GroupByRequest, Aggregation, and CuDFGroupByResult.
  • Update DataFusion aggregate planning to build the new request type and consume flattened grouped results.
  • Fix group-by/view lifetime ordering so non-owning cuDF handles drop before their backing storage, and retain group-by destruction readiness after aggregation.

Stack

This PR is stacked on #78 (gene.bordegaray/2026/06/join-api-redesign).

Root Cause

The previous group-by wrapper exposed a less composable request API and returned raw nested tuples. While adding NUNIQUE and MEDIAN coverage, cleanup crashes exposed a wrapper lifetime issue: cuDF groupby/view handles could be dropped after the Rust-owned key/view storage or producer stream readiness they depended on.

Validation

  • cargo test --workspace

@gene-bordegaray gene-bordegaray changed the title [codex] Redesign group-by aggregation API Redesign group-by aggregation API Jul 4, 2026
@gene-bordegaray gene-bordegaray force-pushed the gene.bordegaray/2026/07/groupby-api-redesign branch from 0a549e5 to df15a00 Compare July 4, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant