Skip to content

Commit

Permalink
PG17 compatibility (#7653): Fix test diffs in columnar schedule (#7768)
Browse files Browse the repository at this point in the history
This PR fixes diffs in `columnnar_chunk_filtering` and `columnar_paths`
tests.

In `columnnar_chunk_filtering` an expression `(NOT (SubPlan 1))` changed
to `(NOT (ANY (a = (SubPlan 1).col1)))`. This is due to [aPG17
commit](postgres/postgres@fd0398fc) that
improved how scalar subqueries (InitPlans) and ANY subqueries (SubPlans)
are EXPLAINed in expressions. The fix uses a helper function which
converts the PG17 format to the pre-PG17 format. It is done this way
because pre-PG17 EXPLAIN does not provide enough context to convert to
the PG17 format. The helper function can (and should) be retired when 17
becomes the minimum supported PG.

In `columnar_paths`, a merge join changed to a hash join. This is due to
[this PG17
commit](postgres/postgres@f7816ae),
which improved the PG optimizer's ability to estimate the size of a CTE
scan. The impacted query involves a CTE scan with a point predicate
`(a=123)` and before the change the CTE size was estimated to be 5000,
but with the change it is correctly (given the data in the table)
estimated to be 1, making hash join a more attractive join method. The
fix is to have an alternative goldfile for pre-PG17. I tried, but was
unable, to force a specific kind of join method using the GUCs
(`enable_nestloop`, `enable_hashjoin`, `enable_mergejoin`), but it was
not possible to obtain a consistent plan across all supported PG
versions (in some cases the join inputs switched sides).
  • Loading branch information
colm-mchugh authored Dec 3, 2024
1 parent 5f479d5 commit 698699d
Show file tree
Hide file tree
Showing 8 changed files with 697 additions and 15 deletions.
18 changes: 12 additions & 6 deletions src/test/regress/expected/columnar_chunk_filtering.out
Original file line number Diff line number Diff line change
Expand Up @@ -977,6 +977,7 @@ DETAIL: unparameterized; 1 clauses pushed down
(1 row)

SET hash_mem_multiplier = 1.0;
SELECT public.explain_with_pg16_subplan_format($Q$
EXPLAIN (analyze on, costs off, timing off, summary off)
SELECT sum(a) FROM pushdown_test where
(
Expand All @@ -989,13 +990,18 @@ SELECT sum(a) FROM pushdown_test where
)
or
(a > 200000-2010);
$Q$) as "QUERY PLAN";
NOTICE: columnar planner: adding CustomScan path for pushdown_test
DETAIL: unparameterized; 0 clauses pushed down
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
NOTICE: columnar planner: cannot push down clause: must match 'Var <op> Expr' or 'Expr <op> Var'
HINT: Var must only reference this rel, and Expr must not reference this rel
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
NOTICE: columnar planner: cannot push down clause: must not contain a subplan
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
NOTICE: columnar planner: adding CustomScan path for pushdown_test
DETAIL: unparameterized; 1 clauses pushed down
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
QUERY PLAN
---------------------------------------------------------------------
Aggregate (actual rows=1 loops=1)
Expand Down Expand Up @@ -1092,14 +1098,14 @@ BEGIN;
END;
EXPLAIN (analyze on, costs off, timing off, summary off)
SELECT id FROM pushdown_test WHERE country IN ('USA', 'BR', 'ZW');
QUERY PLAN
QUERY PLAN
---------------------------------------------------------------------
Custom Scan (ColumnarScan) on pushdown_test (actual rows=3 loops=1)
Filter: (country = ANY ('{USA,BR,ZW}'::text[]))
Rows Removed by Filter: 1
Columnar Projected Columns: id, country
Columnar Chunk Group Filters: (country = ANY ('{USA,BR,ZW}'::text[]))
Columnar Chunk Groups Removed by Filter: 2
Filter: (country = ANY ('{USA,BR,ZW}'::text[]))
Rows Removed by Filter: 1
Columnar Projected Columns: id, country
Columnar Chunk Group Filters: (country = ANY ('{USA,BR,ZW}'::text[]))
Columnar Chunk Groups Removed by Filter: 2
(6 rows)

SELECT id FROM pushdown_test WHERE country IN ('USA', 'BR', 'ZW');
Expand Down
6 changes: 6 additions & 0 deletions src/test/regress/expected/columnar_chunk_filtering_0.out
Original file line number Diff line number Diff line change
Expand Up @@ -977,6 +977,7 @@ DETAIL: unparameterized; 1 clauses pushed down
(1 row)

SET hash_mem_multiplier = 1.0;
SELECT public.explain_with_pg16_subplan_format($Q$
EXPLAIN (analyze on, costs off, timing off, summary off)
SELECT sum(a) FROM pushdown_test where
(
Expand All @@ -989,13 +990,18 @@ SELECT sum(a) FROM pushdown_test where
)
or
(a > 200000-2010);
$Q$) as "QUERY PLAN";
NOTICE: columnar planner: adding CustomScan path for pushdown_test
DETAIL: unparameterized; 0 clauses pushed down
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
NOTICE: columnar planner: cannot push down clause: must match 'Var <op> Expr' or 'Expr <op> Var'
HINT: Var must only reference this rel, and Expr must not reference this rel
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
NOTICE: columnar planner: cannot push down clause: must not contain a subplan
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
NOTICE: columnar planner: adding CustomScan path for pushdown_test
DETAIL: unparameterized; 1 clauses pushed down
CONTEXT: PL/pgSQL function explain_with_pg16_subplan_format(text) line XX at FOR over EXECUTE statement
QUERY PLAN
---------------------------------------------------------------------
Aggregate (actual rows=1 loops=1)
Expand Down
19 changes: 10 additions & 9 deletions src/test/regress/expected/columnar_paths.out
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
CREATE SCHEMA columnar_paths;
SET search_path TO columnar_paths;
-- columnar_paths has an alternative test output file because PG17 improved
-- the optimizer's ability to use statistics to estimate the size of a CTE
-- scan.
-- The relevant PG commit is:
-- https://github.com/postgres/postgres/commit/f7816aec23eed1dc1da5f9a53cb6507d30b7f0a2
CREATE TABLE full_correlated (a int, b text, c int, d int) USING columnar;
INSERT INTO full_correlated SELECT i, i::text FROM generate_series(1, 1000000) i;
CREATE INDEX full_correlated_btree ON full_correlated (a);
Expand Down Expand Up @@ -296,20 +301,16 @@ SELECT * FROM w AS w1 JOIN w AS w2 ON w1.a = w2.d
WHERE w2.a = 123;
QUERY PLAN
---------------------------------------------------------------------
Merge Join
Merge Cond: (w2.d = w1.a)
Hash Join
Hash Cond: (w1.a = w2.d)
CTE w
-> Custom Scan (ColumnarScan) on full_correlated
Columnar Projected Columns: a, b, c, d
-> Sort
Sort Key: w2.d
-> CTE Scan on w w1
-> Hash
-> CTE Scan on w w2
Filter: (a = 123)
-> Materialize
-> Sort
Sort Key: w1.a
-> CTE Scan on w w1
(13 rows)
(9 rows)

-- use index
EXPLAIN (COSTS OFF) WITH w AS NOT MATERIALIZED (SELECT * FROM full_correlated)
Expand Down
Loading

0 comments on commit 698699d

Please sign in to comment.