Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PG17 compatibility: Fix Test Failure in local_table_join #7732

Draft
wants to merge 4 commits into
base: naisila/pg17_support
Choose a base branch
from

Conversation

m3hm3t
Copy link
Contributor

@m3hm3t m3hm3t commented Nov 11, 2024

PostgreSQL 17 seems to have introduced improvements in how correlated subqueries are handled during plan generation. Instead of generating a trivial subplan with WHERE true, it now applies more specific filtering (WHERE (key = 5)), which makes the execution plan more efficient.

postgres/postgres@b262ad44

diff -dU10 -w /__w/citus/citus/src/test/regress/expected/local_table_join.out /__w/citus/citus/src/test/regress/results/local_table_join.out
--- /__w/citus/citus/src/test/regress/expected/local_table_join.out.modified	2024-11-05 09:53:50.423970699 +0000
+++ /__w/citus/citus/src/test/regress/results/local_table_join.out.modified	2024-11-05 09:53:50.463971296 +0000
@@ -1420,32 +1420,32 @@
   ) as subq_1
 ) as subq_2;
 DEBUG:  Wrapping relation "custom_pg_type" to a subquery
 DEBUG:  generating subplan 204_1 for subquery SELECT typdefault FROM local_table_join.custom_pg_type WHERE true
 ERROR:  direct joins between distributed and local tables are not supported
 HINT:  Use CTE's or subqueries to select from local tables and use them in joins
 -- correlated sublinks are not yet supported because of #4470, unless we convert not-correlated table
 SELECT COUNT(*) FROM distributed_table d1 JOIN postgres_table using(key)
 WHERE d1.key IN (SELECT key FROM distributed_table WHERE d1.key = key and key = 5);
 DEBUG:  Wrapping relation "postgres_table" to a subquery
-DEBUG:  generating subplan XXX_1 for subquery SELECT key FROM local_table_join.postgres_table WHERE true
+DEBUG:  generating subplan 206_1 for subquery SELECT key FROM local_table_join.postgres_table WHERE (key OPERATOR(pg_catalog.=) 5)

@m3hm3t m3hm3t self-assigned this Nov 11, 2024
Copy link

codecov bot commented Nov 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.63%. Comparing base (b29c332) to head (f69de66).

Additional details and impacted files
@@                   Coverage Diff                    @@
##           naisila/pg17_support    #7732      +/-   ##
========================================================
+ Coverage                 89.61%   89.63%   +0.02%     
========================================================
  Files                       274      274              
  Lines                     59689    59672      -17     
  Branches                   7446     7444       -2     
========================================================
- Hits                      53490    53488       -2     
+ Misses                     4069     4054      -15     
  Partials                   2130     2130              

@m3hm3t m3hm3t marked this pull request as ready for review November 11, 2024 20:47
@@ -1438,7 +1442,7 @@ set citus.local_table_join_policy to 'prefer-distributed';
SELECT COUNT(*) FROM distributed_table d1 JOIN postgres_table using(key)
WHERE d1.key IN (SELECT key FROM distributed_table WHERE d1.key = key and key = 5);
DEBUG: Wrapping relation "distributed_table" "d1" to a subquery
DEBUG: generating subplan XXX_1 for subquery SELECT key FROM local_table_join.distributed_table d1 WHERE true
DEBUG: generating subplan XXX_1 for subquery SELECT key FROM local_table_join.distributed_table d1 WHERE (key OPERATOR(pg_catalog.=) 5)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have just two lines with the same change (key OPERATOR(pg_catalog.=) 5)
How about a normalized line, we can change the table names such that the normalization rule detects the line easily.
e.g.
postgres_table_key5_filtering
distributed_table_key5_filtering

Just an idea, could be better. But I am reluctant of merging a new file with 1659 new lines just for 2 lines change in the test output. What do you think?

@naisila naisila changed the title Fix Test Failure in local_table_join in PG17 PG17 compatibility: Fix Test Failure in local_table_join Nov 12, 2024
@m3hm3t m3hm3t marked this pull request as draft November 14, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants