Skip to content

Support rough set filter for Decimal columns #11031

Description

@windtalker

Background

TiFlash rough set filter can use storage-level min/max metadata to skip packs before reading full column data. Decimal columns are common for monetary and metric fields, but decimal predicates are currently not handled end-to-end by the rough set filter path, so queries with selective Decimal filters cannot benefit from pack pruning.

For example:

SELECT *
FROM t
WHERE decimal_col >= 100.00 AND decimal_col < 200.00;

Without Decimal rough set support, TiFlash may need to scan more packs even when the Decimal predicate is highly selective.

Proposal

Support Decimal columns in the rough set min/max filter path.

Main scope:

  • Build and persist min/max index metadata for Decimal columns.
  • Read Decimal min/max metadata from DMFile and expose it to rough set checks.
  • Allow Decimal column predicates to be parsed into rough set operators only when the non-null literal is encoded as tipb::ExprType::MysqlDecimal.
  • Keep mixed Decimal/non-Decimal predicates conservative: parser should mark them as unsupported when statically detectable, and runtime CanNotCompare -> Some should remain as a safety net.
  • Use scale-aware Decimal comparison for Decimal32/Decimal64/Decimal128/Decimal256, including different precision/scale combinations between the predicate constant and the column/index value.
  • Support comparison predicates, null-safe equality, IN, and nullable Decimal columns where applicable.

Acceptance Criteria

  • Decimal predicates can create effective rough set operators and prune packs with Decimal min/max indexes.
  • Mixed Decimal/non-Decimal predicates do not create misleading rough set operators and remain conservative.
  • Unit tests cover parser behavior, min/max rough check behavior, nullable Decimal behavior, and DMFile write/read of Decimal min/max indexes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects-8.5This bug affects the 8.5.x(LTS) versions.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions