Skip to content

Prune plan scans with zoned statistics - #9244

Draft
joseph-isaacs wants to merge 1 commit into
vortex-plan-scan-v2from
vortex-plan-zoned-pruning
Draft

Prune plan scans with zoned statistics#9244
joseph-isaacs wants to merge 1 commit into
vortex-plan-scan-v2from
vortex-plan-zoned-pruning

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • derive a statistics falsifier from each scan filter and optimize it as a physical plan
  • bind the filter once and share that BoundExpression between exact filtering and pruning-plan
    construction
  • add the static (ExpressionPlan, ZonedPlan) rule, representing pruning as bound state on
    ZonedPlan rather than introducing another plan-node type
  • drop the zoned data child from pruning plans, cache the zone map, preserve unknown proof values,
    and expand zone proofs into the data row domain
  • keep the temporary unbind only at the existing raw stats-rewrite adapter boundary
  • execute pruning before constructing exact-filter or projection reads, so fully falsified splits
    do not request data
  • preserve concurrent filter/projection registration so both consumers share an in-flight segment
    request
  • cover before/after optimization, mixed row-index plus zoned struct-field partitioning, short
    final zones, skipped data-segment reads, and shared filter/projection reads

Stack

This is part 6 of the plan-native scan stack and is based on #9240. Its history is one coherent
pruning commit. The focused planning optimization is split into follow-up #9254.

Performance follow-up

The current implementation prioritizes correctness and keeps pruning inside the general array-plan
execution contract. On the 100-shard ClickBench scan, pruning eliminated 51.8% of rows and fully
skipped 264 of 1,100 splits, but prepared v2 execution was 44.41 ms with pruning versus 30.69 ms
without it. Profiling attributes roughly 5.3 ms to materializing a row-aligned boolean/validity
proof and another 8.4 ms to filtering the encoded input before the exact comparison.

A follow-up should make pruning mask-native and preserve encoded filter evaluation before combining
the pruning mask. That execution work is deliberately separate from this planning and correctness
PR.

Validation

  • RUSTC_WRAPPER= cargo nextest run -p vortex-layout -p vortex-scan-v2
  • RUSTC_WRAPPER= cargo clippy -p vortex-layout -p vortex-scan-v2 --all-targets --all-features -- -D warnings
  • RUSTC_WRAPPER= cargo test --doc -p vortex-layout -p vortex-scan-v2
  • cargo +nightly fmt --all -- --check
  • git diff --check

@codspeed-hq

codspeed-hq Bot commented Aug 6, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 45.36%

❌ 1 regressed benchmark
✅ 1924 untouched benchmarks
⏩ 51 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation decompress[u64, (1000, 16)] 73 µs 133.6 µs -45.36%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing vortex-plan-zoned-pruning (1bdf135) with vortex-plan-scan-v2 (a8c35cb)

Open in CodSpeed

Footnotes

  1. 51 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@joseph-isaacs
joseph-isaacs force-pushed the vortex-plan-zoned-pruning branch from 7d7bdf1 to 57d4c88 Compare August 6, 2026 20:40
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant