Skip to content

Permanently back cuDF column by a pylibcudf.Column #20306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open

Permanently back cuDF column by a pylibcudf.Column #20306

wants to merge 31 commits into from

Conversation

mroeschke
Copy link
Contributor

Description

On top of #20233
Supersedes #20087

closes #17816
closes #16469
closes #18726

This PR changes Column construction to pass a backing pylibcudf.Column instead of the cuDF Buffers. There are still some followup simplifications to to_pylibcudf/from_pylibcudf to minimize shallow copies.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

mroeschke and others added 27 commits September 29, 2025 18:07
@mroeschke
Make ColumnBase.deserialize construct via pylibcudf
e789cb6
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
c64df29 
…lumn_serialize
@mroeschke
remove initial warning filter
fe4caa1
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
d6d3aec 
…lumn_serialize
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
d96929e 
…lumn_serialize
@mroeschke
Fold in Tom's review from set_mask PR
1711292
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
8f66659 
…lumn_serialize
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
3420a3d 
…lumn_serialize
@madsbk
deserialize(): wrap SpillableBuffer in a spillable_gpumemoryview
606a820
@madsbk
Merge branch 'branch-25.12' into ref/cudf/column_serialize
ec5dc5e
@mroeschke
Merge branch 'ref/cudf/column_serialize' of https://github.com/mroesc…
abf5b53 
…hke/cudf into ref/cudf/column_serialize
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
7777e0b 
…lumn_serialize
@mroeschke
Undo warning changes in test_spilling.py
522941a
@mroeschke
Remove default arguments from Column constructors
8755ec4
@mroeschke
CategoricalColumn size was never None
2c0f8f2
@mroeschke
TemporalColumns never have None size
e61defc
@mroeschke
NumericalColumn never gets None size
366311e
@mroeschke
StringColumn never has None size
508ccba
@mroeschke
Validate that null_count >=0
aa9a118
@mroeschke
Try annotating dtype
22ff2fa
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
f586eba 
…nstructors
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
068ae77 
…nstructors
@mroeschke
Merge remote-tracking branch 'upstream/branch-25.12' into ref/cudf/co…
c29c9ee 
…nstructors
@mroeschke
Adjust constructors to just take pylibcudf columns
1b13b5a
@mroeschke
Create functions to convert data, mask, children to cudf compatable o…
5522d58 
…bjects
@mroeschke
Fix naming of method
2137b9d
@mroeschke
Pass dtype to children construction
f6621d0
@mroeschke mroeschke self-assigned this Oct 18, 2025
@mroeschke mroeschke requested a review from a team as a code owner October 18, 2025 00:36
@mroeschke mroeschke requested a review from bdice October 18, 2025 00:36
@mroeschke mroeschke requested a review from rjzamora October 18, 2025 00:36
@mroeschke mroeschke added Python Affects Python cuDF API. improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 18, 2025
@mroeschke mroeschke changed the title Permanently back cuDF column bya pylibcudf.Column Permanently back cuDF column by a pylibcudf.Column Oct 18, 2025
@mroeschke mroeschke mentioned this pull request Oct 18, 2025
Closed
3 tasks
@GPUtester GPUtester moved this to In Progress in cuDF Python Oct 18, 2025
@mroeschke mroeschke changed the base branch from main to branch-25.12 October 20, 2025 23:07
@mroeschke mroeschke changed the base branch from branch-25.12 to main October 20, 2025 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bdice bdice Awaiting requested review from bdice bdice is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@rjzamora rjzamora Awaiting requested review from rjzamora rjzamora is a code owner automatically assigned from rapidsai/cudf-python-codeowners

At least 1 approving review is required to merge this pull request.

Assignees

@mroeschke mroeschke

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

cuDF Python
Status: In Progress

Milestone

No milestone

2 participants

@mroeschke @madsbk