[CH] fallback for unsupported regex in re2

[exmy]

Backend

CH (ClickHouse)

Bug description

OptimizedRegularExpression: cannot compile re2: d(?!d), error: invalid perl operator: (?!. Look at https://github.com/google/re2/wiki/Syntax for reference.

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

No response

[FelixYBW]

Does CH also use re2?

@PHILO-HE we may reuse the preprocessing script as well as the skip list.

[PHILO-HE]

@PHILO-HE we may reuse the preprocessing script as well as the skip list.

@FelixYBW, currently, we have no script, but have a pre-validation function on Gluten native side. It validate all listed regrex functions by letting RE2 try to compile the pattern. If the compilation fails, we definitely need to make it fall back. This pre-validation can be reused by both backend.

[FelixYBW]

@FelixYBW, currently, we have no script, but have a pre-validation function on Gluten native side. It validate all listed regrex functions by letting RE2 try to compile the pattern. If the compilation fails, we definitely need to make it fall back. This pre-validation can be reused by both backend.

I remember we have some preprocessing of the pattern before we send to re2, which can make more pattern workable on re2. Is the code still there?

[PHILO-HE]

I remember we have some preprocessing of the pattern before we send to re2, which can make more pattern workable on re2. Is the code still there?

@FelixYBW, I just found Velox has such preprocessing for presto: see code. And Meituan also proposed a pr to re-use this code for Spark and also make some improvement. See facebookincubator/velox#10981.