-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Open
Description
Is your feature request related to a problem or challenge?
- Follow on to Release DataFusion 50.2.0 (September 2025) Release DataFusion
50.2.0
(minor) #17849
There are some things that we may way to backport into the 50.0.0 line, and this ticket tracks doing so.
Steps
- Update version and changelog
- Test with DataFusion Python
- Test with DataFusion Comet Release DataFusion
50.3.0
(minor) #18072 - Test with delta.rs:
- Test Vortex
- Test with iceberg-rust:
- Test with LakeSail:
- Test with parquet viewer
- Test with datafusion-materialized-views
- Voting Thread
- Publish to crates.io
- Create ticket for next release:
Potential Content:
Minor Features
- feat: support Spark
concat
string function #18063 + chore: useNullBuffer::union
for Sparkconcat
#18087 @comphead
Bug Fixes
- AmbiguousReference on project following join when expression name matches left or right column name #17294 / Fix ambiguous column names in substrait conversion as a result of literals having the same name during conversion. #17299 @xbailey
- LimitPushPastWindows returns incorrect results for queries with
lead()
#18028 / Fix bug in LimitPushPastWindows #18029 @avantgardnerio - Semantic equality check for DataType::Timestamp doesn't exist #17776 / Extend datatype semantic equality check to include timestamps #17777 @shivbhatia10
-
change_redundant_column
lossy approach breaks logical optimizer and physical planner #17405 / fix(SubqueryAlias): use maybe_project_redundant_column #17478 @notfilippo -
DateAdd
/DateSub
do not perform overflow checks in release builds datafusion-comet#2539 / fix: Add overflow checks to SparkDateAdd/Sub to avoid panics #18013 @andygrove - Datafusion 50 Performance Regression (array_has style filter/join for Parquet data set) #18070
Backport PR list:
- [branch-50]: chore: cherry pick
concat
to 50.3.0 #18128 - [branch-50] Fix ambiguous column names in substrait conversion #17299 #18077
- [branch-50] Backport Fix bug in LimitPushPastWindows (#18029) #18107
- [branch-50] Extend datatype semantic equality check to include timestamps (#17777) #18129
- [branch-50] fix(SubqueryAlias): use maybe_project_redundant_column (#17478) #18130
- [branch-50]: fix: Add overflow checks to SparkDateAdd/Sub to avoid panics (#18013) #18131
Metadata
Metadata
Assignees
Labels
No labels