Problem
HLL and TopN are currently easy to maintain for append-only writes, because newly generated stats can be merged into existing table-level stats.
DELETE is different. Both HLL and TopN do not support subtraction:
- HLL cannot remove values contributed by deleted rows from the sketch.
- TopN cannot safely decrement counts, and it cannot recover values that were not kept in the previous TopN set.
Therefore, after DELETE, existing table-level HLL/TopN may no longer match the current live data. If they are still treated as fresh, optimizer estimates may become inaccurate.
Expected behavior
HLL and TopN should use consistent freshness handling after DELETE changes table data:
- append: keep the current incremental merge behavior;
- compact/recluster operations that do not change logical data: preserve existing stats;
- delete/update/merge-like mutations: do not keep old HLL/TopN as fresh;
- analyze: rebuild fresh HLL/TopN stats.
Main question
For DELETE
- Invalidate table-level HLL/TopN and wait for ANALYZE/auto-analyze to rebuild them.
- Rebuild HLL/TopN during mutation commit from the remaining live block stats.
Option 1 is cheaper and more conservative. Option 2 keeps stats fresher, but makes mutation commit heavier and depends on complete block-level stats.
Problem
HLL and TopN are currently easy to maintain for append-only writes, because newly generated stats can be merged into existing table-level stats.
DELETE is different. Both HLL and TopN do not support subtraction:
Therefore, after DELETE, existing table-level HLL/TopN may no longer match the current live data. If they are still treated as fresh, optimizer estimates may become inaccurate.
Expected behavior
HLL and TopN should use consistent freshness handling after DELETE changes table data:
Main question
For DELETE
Option 1 is cheaper and more conservative. Option 2 keeps stats fresher, but makes mutation commit heavier and depends on complete block-level stats.