Skip to content

Redesign join API and internals#78

Draft
gene-bordegaray wants to merge 2 commits into
gabotechs:gene.bordegaray/2026/06/root-execution-readiness-corefrom
gene-bordegaray:gene.bordegaray/2026/06/join-api-redesign
Draft

Redesign join API and internals#78
gene-bordegaray wants to merge 2 commits into
gabotechs:gene.bordegaray/2026/06/root-execution-readiness-corefrom
gene-bordegaray:gene.bordegaray/2026/06/join-api-redesign

Conversation

@gene-bordegaray

Copy link
Copy Markdown
Collaborator

Summary

This PR is stacked on top of the root execution/readiness API redesign in #77. It reshapes the libcudf-rs join API and internals around smaller, explicit join paths.

It replaces the monolithic join implementation with focused modules for common selection logic, equi joins, hash joins, join indices, masks, and output gathering.

Stream API details

The reusable hash join can be converted into a streaming join with into_streaming_join(). Streaming probes record matched build indices as each probe batch runs, and unmatched_build_rows(...) finalizes left/full-style output by gathering build rows that were never matched.

Filtered streaming probes use the same builder-style API as regular probes: filter(predicate), optional condition_tables(...), optional payload overrides, and selected build/probe outputs. The filtered path computes equality candidates first, applies the AST predicate, records only surviving build matches, and appends unmatched probe rows for left probes.

The stream readiness model keeps hash join state, filtered build state, and join index vectors alive until dependent GPU work is ready.

Internal structure

  • Split src/join.rs into src/join/{common,equi,hash,indices,masks,output}.rs.
  • Localized join state next to the code that owns it.
  • Kept output gathering and filtered-index argument structs only where they protect same-typed FFI arguments.
  • Removed thin compatibility wrappers and stale join abstractions.
  • Converted production join panics into CuDFError paths.

Validation

  • git diff --cached --check
  • cargo fmt --check
  • cargo check --workspace
  • cargo clippy -p libcudf-rs -- -D warnings
  • cargo test -p libcudf-rs join:: -- --nocapture
  • cargo test -p libcudf-datafusion physical::hash_join -- --test-threads=1 --nocapture
  • cargo test -p libcudf-datafusion expr::ast -- --test-threads=1 --nocapture
  • cargo doc -p libcudf-rs --no-deps

@gene-bordegaray gene-bordegaray changed the title Redesign join API internals Redesign join API and internals Jul 4, 2026
@gene-bordegaray gene-bordegaray force-pushed the gene.bordegaray/2026/06/join-api-redesign branch from 2295418 to 4bffe3a Compare July 4, 2026 12:35
@gene-bordegaray gene-bordegaray force-pushed the gene.bordegaray/2026/06/join-api-redesign branch from 4bffe3a to 8478b4a Compare July 4, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant