Background
TiFlash rough set filter can use storage-level min/max metadata to skip packs before reading full column data. Decimal columns are common for monetary and metric fields, but decimal predicates are currently not handled end-to-end by the rough set filter path, so queries with selective Decimal filters cannot benefit from pack pruning.
For example:
SELECT *
FROM t
WHERE decimal_col >= 100.00 AND decimal_col < 200.00;
Without Decimal rough set support, TiFlash may need to scan more packs even when the Decimal predicate is highly selective.
Proposal
Support Decimal columns in the rough set min/max filter path.
Main scope:
- Build and persist min/max index metadata for Decimal columns.
- Read Decimal min/max metadata from DMFile and expose it to rough set checks.
- Allow Decimal column predicates to be parsed into rough set operators only when the non-null literal is encoded as
tipb::ExprType::MysqlDecimal.
- Keep mixed Decimal/non-Decimal predicates conservative: parser should mark them as unsupported when statically detectable, and runtime
CanNotCompare -> Some should remain as a safety net.
- Use scale-aware Decimal comparison for Decimal32/Decimal64/Decimal128/Decimal256, including different precision/scale combinations between the predicate constant and the column/index value.
- Support comparison predicates, null-safe equality,
IN, and nullable Decimal columns where applicable.
Acceptance Criteria
- Decimal predicates can create effective rough set operators and prune packs with Decimal min/max indexes.
- Mixed Decimal/non-Decimal predicates do not create misleading rough set operators and remain conservative.
- Unit tests cover parser behavior, min/max rough check behavior, nullable Decimal behavior, and DMFile write/read of Decimal min/max indexes.
Background
TiFlash rough set filter can use storage-level min/max metadata to skip packs before reading full column data. Decimal columns are common for monetary and metric fields, but decimal predicates are currently not handled end-to-end by the rough set filter path, so queries with selective Decimal filters cannot benefit from pack pruning.
For example:
Without Decimal rough set support, TiFlash may need to scan more packs even when the Decimal predicate is highly selective.
Proposal
Support Decimal columns in the rough set min/max filter path.
Main scope:
tipb::ExprType::MysqlDecimal.CanNotCompare -> Someshould remain as a safety net.IN, and nullable Decimal columns where applicable.Acceptance Criteria