Skip to content

Provide min/max/sum info from zone maps if possible - #9040

Open
myrrc wants to merge 2 commits into
developfrom
zonemap-stats-layout
Open

Provide min/max/sum info from zone maps if possible#9040
myrrc wants to merge 2 commits into
developfrom
zonemap-stats-layout

Conversation

@myrrc

@myrrc myrrc commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

For Zoned layout reader, provide info for min/max/sum functions directly from zone maps if zone maps are included fully.
Decode data for chunks if some zone maps arent't included fully.

This only works if zone maps are precise so we check this both for normal and legacy stats.

@myrrc myrrc added the changelog/feature A new feature label Jul 28, 2026
@myrrc
myrrc requested a review from joseph-isaacs July 28, 2026 17:38
Base automatically changed from myrrc/zonemap-aggregate-stats to develop July 29, 2026 08:43
@myrrc
myrrc force-pushed the zonemap-stats-layout branch from f4686ae to 4c70126 Compare July 29, 2026 08:43
@joseph-isaacs

Copy link
Copy Markdown
Contributor

What is the intended semantics of this change/expression. Would you expect this to work over both a layout-reader plan and an array?

@myrrc
myrrc force-pushed the zonemap-stats-layout branch 2 times, most recently from 55619ee to 9e871d6 Compare July 29, 2026 11:22
@codspeed-hq

codspeed-hq Bot commented Jul 29, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 1842 untouched benchmarks
⏩ 55 skipped benchmarks1


Comparing zonemap-stats-layout (276e7c7) with develop (a1c2853)

Open in CodSpeed

Footnotes

  1. 55 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@myrrc
myrrc force-pushed the zonemap-stats-layout branch 5 times, most recently from 06dc827 to fc3bebf Compare July 29, 2026 15:20
@myrrc
myrrc requested a review from robert3005 July 29, 2026 15:50
@myrrc

myrrc commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Things need to be checked via talking with @robert3005 and @joseph-isaacs

  1. Are zone maps storing exact values? For aggregate vtable, there is "satisfy" which can be used to check this.
  2. String stats are truncated. How does this work vs. non-zone map aggregation?
  3. How will stat function semantics change from "upper bound" to "exact" reflect its users?

@myrrc
myrrc force-pushed the zonemap-stats-layout branch 3 times, most recently from 49e131b to 615d8ce Compare July 29, 2026 17:19
@myrrc
myrrc changed the base branch from develop to myrrc/duckdb-aggregate-float-pushdown July 29, 2026 17:19
Base automatically changed from myrrc/duckdb-aggregate-float-pushdown to develop July 29, 2026 17:27
@myrrc
myrrc force-pushed the zonemap-stats-layout branch 2 times, most recently from c6e5678 to 615d8ce Compare July 29, 2026 17:30
@myrrc
myrrc requested a review from gatesn July 29, 2026 17:33
@myrrc
myrrc force-pushed the zonemap-stats-layout branch from 615d8ce to 276e7c7 Compare July 31, 2026 16:36
@myrrc

myrrc commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Additional issue to check: Struct array with nullability may produce invalid zone maps (containing invalid but defined values)

@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.92308% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.37%. Comparing base (a1c2853) to head (276e7c7).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
vortex-layout/src/layouts/zoned/reader.rs 81.09% 31 Missing ⚠️
vortex-layout/src/layouts/zoned/zone_map.rs 96.77% 3 Missing ⚠️

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants