Skip to content

Conversation

@mbutrovich
Copy link
Collaborator

DataFusion is preparing for their 51.0.0 release, so this is a test PR to see how CI goes with main on a recent commit that likely reflects what DF 51 will be. Marking as draft since we wouldn't want this merged until crates are available and the community discusses the changes.

Which issue does this PR close?

  • N/A.

What changes are included in this PR?

  • Bump DF to 51
  • Bump Arrow/Parquet to 57
  • Page index tests are skipped right now because of API changes in Arrow-rs. I will have to revisit what we can do to fix these since they were relying on a pretty low-level API
  • Toolchain bump to support DF's 1.88. I need to understand if this would be within iceberg-rust's MSRV policy

Are these changes tested?

Creating this to run CI.

@alamb
Copy link

alamb commented Nov 8, 2025

i can probably help with the page index changes

@mbutrovich
Copy link
Collaborator Author

It was green at one point, so I think from DataFusion's standpoint DF 51 is okay. This branch quickly fell behind with merge conflicts. I'll update it once DF 51 crates are available and we want to consider upgrading for real.

@mbutrovich mbutrovich self-assigned this Nov 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants