Skip to content

Commit 698699d

Browse files
authored
PG17 compatibility (#7653): Fix test diffs in columnar schedule (#7768)
This PR fixes diffs in `columnnar_chunk_filtering` and `columnar_paths` tests. In `columnnar_chunk_filtering` an expression `(NOT (SubPlan 1))` changed to `(NOT (ANY (a = (SubPlan 1).col1)))`. This is due to [aPG17 commit](postgres/postgres@fd0398fc) that improved how scalar subqueries (InitPlans) and ANY subqueries (SubPlans) are EXPLAINed in expressions. The fix uses a helper function which converts the PG17 format to the pre-PG17 format. It is done this way because pre-PG17 EXPLAIN does not provide enough context to convert to the PG17 format. The helper function can (and should) be retired when 17 becomes the minimum supported PG. In `columnar_paths`, a merge join changed to a hash join. This is due to [this PG17 commit](postgres/postgres@f7816ae), which improved the PG optimizer's ability to estimate the size of a CTE scan. The impacted query involves a CTE scan with a point predicate `(a=123)` and before the change the CTE size was estimated to be 5000, but with the change it is correctly (given the data in the table) estimated to be 1, making hash join a more attractive join method. The fix is to have an alternative goldfile for pre-PG17. I tried, but was unable, to force a specific kind of join method using the GUCs (`enable_nestloop`, `enable_hashjoin`, `enable_mergejoin`), but it was not possible to obtain a consistent plan across all supported PG versions (in some cases the join inputs switched sides).
1 parent 5f479d5 commit 698699d

8 files changed

+697
-15
lines changed

src/test/regress/expected/columnar_chunk_filtering.out

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -977,6 +977,7 @@ DETAIL: unparameterized; 1 clauses pushed down
977977
(1 row)
978978

979979
SET hash_mem_multiplier = 1.0;
980+
SELECT public.explain_with_pg16_subplan_format($Q$
980981
EXPLAIN (analyze on, costs off, timing off, summary off)
981982
SELECT sum(a) FROM pushdown_test where
982983
(
@@ -989,13 +990,18 @@ SELECT sum(a) FROM pushdown_test where
989990
)
990991
or
991992
(a > 200000-2010);
993+
$Q$) as "QUERY PLAN";
992994
NOTICE: columnar planner: adding CustomScan path for pushdown_test
993995
DETAIL: unparameterized; 0 clauses pushed down
996+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
994997
NOTICE: columnar planner: cannot push down clause: must match 'Var <op> Expr' or 'Expr <op> Var'
995998
HINT: Var must only reference this rel, and Expr must not reference this rel
999+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
9961000
NOTICE: columnar planner: cannot push down clause: must not contain a subplan
1001+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
9971002
NOTICE: columnar planner: adding CustomScan path for pushdown_test
9981003
DETAIL: unparameterized; 1 clauses pushed down
1004+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
9991005
QUERY PLAN
10001006
---------------------------------------------------------------------
10011007
Aggregate (actual rows=1 loops=1)
@@ -1092,14 +1098,14 @@ BEGIN;
10921098
END;
10931099
EXPLAIN (analyze on, costs off, timing off, summary off)
10941100
SELECT id FROM pushdown_test WHERE country IN ('USA', 'BR', 'ZW');
1095-
QUERY PLAN
1101+
QUERY PLAN
10961102
---------------------------------------------------------------------
10971103
Custom Scan (ColumnarScan) on pushdown_test (actual rows=3 loops=1)
1098-
Filter: (country = ANY ('{USA,BR,ZW}'::text[]))
1099-
Rows Removed by Filter: 1
1100-
Columnar Projected Columns: id, country
1101-
Columnar Chunk Group Filters: (country = ANY ('{USA,BR,ZW}'::text[]))
1102-
Columnar Chunk Groups Removed by Filter: 2
1104+
Filter: (country = ANY ('{USA,BR,ZW}'::text[]))
1105+
Rows Removed by Filter: 1
1106+
Columnar Projected Columns: id, country
1107+
Columnar Chunk Group Filters: (country = ANY ('{USA,BR,ZW}'::text[]))
1108+
Columnar Chunk Groups Removed by Filter: 2
11031109
(6 rows)
11041110

11051111
SELECT id FROM pushdown_test WHERE country IN ('USA', 'BR', 'ZW');

src/test/regress/expected/columnar_chunk_filtering_0.out

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -977,6 +977,7 @@ DETAIL: unparameterized; 1 clauses pushed down
977977
(1 row)
978978

979979
SET hash_mem_multiplier = 1.0;
980+
SELECT public.explain_with_pg16_subplan_format($Q$
980981
EXPLAIN (analyze on, costs off, timing off, summary off)
981982
SELECT sum(a) FROM pushdown_test where
982983
(
@@ -989,13 +990,18 @@ SELECT sum(a) FROM pushdown_test where
989990
)
990991
or
991992
(a > 200000-2010);
993+
$Q$) as "QUERY PLAN";
992994
NOTICE: columnar planner: adding CustomScan path for pushdown_test
993995
DETAIL: unparameterized; 0 clauses pushed down
996+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
994997
NOTICE: columnar planner: cannot push down clause: must match 'Var <op> Expr' or 'Expr <op> Var'
995998
HINT: Var must only reference this rel, and Expr must not reference this rel
999+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
9961000
NOTICE: columnar planner: cannot push down clause: must not contain a subplan
1001+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
9971002
NOTICE: columnar planner: adding CustomScan path for pushdown_test
9981003
DETAIL: unparameterized; 1 clauses pushed down
1004+
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
9991005
QUERY PLAN
10001006
---------------------------------------------------------------------
10011007
Aggregate (actual rows=1 loops=1)

src/test/regress/expected/columnar_paths.out

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
CREATE SCHEMA columnar_paths;
22
SET search_path TO columnar_paths;
3+
-- columnar_paths has an alternative test output file because PG17 improved
4+
-- the optimizer's ability to use statistics to estimate the size of a CTE
5+
-- scan.
6+
-- The relevant PG commit is:
7+
-- https://github.com/postgres/postgres/commit/f7816aec23eed1dc1da5f9a53cb6507d30b7f0a2
38
CREATE TABLE full_correlated (a int, b text, c int, d int) USING columnar;
49
INSERT INTO full_correlated SELECT i, i::text FROM generate_series(1, 1000000) i;
510
CREATE INDEX full_correlated_btree ON full_correlated (a);
@@ -296,20 +301,16 @@ SELECT * FROM w AS w1 JOIN w AS w2 ON w1.a = w2.d
296301
WHERE w2.a = 123;
297302
QUERY PLAN
298303
---------------------------------------------------------------------
299-
Merge Join
300-
Merge Cond: (w2.d = w1.a)
304+
Hash Join
305+
Hash Cond: (w1.a = w2.d)
301306
CTE w
302307
-> Custom Scan (ColumnarScan) on full_correlated
303308
Columnar Projected Columns: a, b, c, d
304-
-> Sort
305-
Sort Key: w2.d
309+
-> CTE Scan on w w1
310+
-> Hash
306311
-> CTE Scan on w w2
307312
Filter: (a = 123)
308-
-> Materialize
309-
-> Sort
310-
Sort Key: w1.a
311-
-> CTE Scan on w w1
312-
(13 rows)
313+
(9 rows)
313314

314315
-- use index
315316
EXPLAIN (COSTS OFF) WITH w AS NOT MATERIALIZED (SELECT * FROM full_correlated)

0 commit comments

Comments
 (0)