PG17 compatibility: Resolve compilation issues #7651

naisila · 2024-07-10T16:03:58Z

PG17 compatibility - Part 1

This PR provides successful compilation against PG17Beta2.

Notes to reviewer:

Review commit by commit - it's much cleaner that way.
When merged, Enable configure and add pg17 build test commits will be reverted. They are part of the PR to make sure that the PG17 build is actually successful - see Build for PG17 in the PR checks. These changes will be merged to the main branch later, when the PG17 support is fully added.

(This PR is converted to draft, and work is continued in another PR against release-13.0 branch #7699)

codecov · 2024-07-10T16:09:57Z

Codecov Report

Attention: Patch coverage is 97.00240% with 25 lines in your changes missing coverage. Please review.

Project coverage is 89.69%. Comparing base (2a263fe) to head (76f60a7).
Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7651      +/-   ##
==========================================
- Coverage   89.70%   89.69%   -0.01%     
==========================================
  Files         283      283              
  Lines       60507    60512       +5     
  Branches     7539     7539              
==========================================
  Hits        54275    54275              
- Misses       4076     4080       +4     
- Partials     2156     2157       +1

PG17 added foreach_ptr, foreach_int and foreach_oid macros Relevant PG commit 14dd0f27d7cd56ffae9ecdbe324965073d01a9ff postgres/postgres@14dd0f2 We already have these macros, but they are different with the PG17 ones because our macros take a DECLARED variable, whereas the PG16 macros declare a locally-scoped loop variable themselves. Hence I am renaming our macros to foreach_declared_

Relevant PG commit d060e921ea5aa47b6265174c32e1128cebdbc3df postgres/postgres@d060e92

Relevant PG commit 041b96802efa33d2bc9456f2ad946976b92b5ae1 postgres/postgres@041b968

Relevant PG commit: 89e5ef7e21812916c9cf9fcf56e45f0f74034656 postgres/postgres@89e5ef7

Relevant PG commit: 08e6344fd6423210b339e92c069bb979ba4e7cd6 postgres/postgres@08e6344

Relevant PG commit: f696c0cd5f299f1b51e214efc55a22a782cc175d postgres/postgres@f696c0c

Relevant PG commit: de3600452b61d1bc3967e9e37e86db8956c8f577 postgres/postgres@de36004

Relevant PG commit: ecb0fd33720fab91df1207e85704f382f55e1eb7 postgres/postgres@ecb0fd3

Relevant PG commit: 4f622503d6de975ac87448aea5cea7de4bc140d5 postgres/postgres@4f62250

Relevant PG commit: 012460ee93c304fbc7220e5b55d9d0577fc766ab postgres/postgres@012460e

Relevant PG commit: 50c67c2019ab9ade8aa8768bfe604cd802fe8591 postgres/postgres@50c67c2

Relevant PG commit: 509199587df73f06eda898ae13284292f4ae573a postgres/postgres@5091995

Relevant PG commit: 75680c3d805e2323cd437ac567f0677fdfc7b680 postgres/postgres@75680c3

Relevant PG commit: 0294df2f1f842dfb0eed79007b21016f486a3c6c postgres/postgres@0294df2

Relevant PG commit: 5de890e3610d5a12cdaea36413d967cf5c544e20 postgres/postgres@5de890e

…ON_COUNT Relevant PG commit: a6be0600ac3b71dda8277ab0fcbe59ee101ac1ce postgres/postgres@a6be060

Relevant PG commit: 9e9931d2bf40e2fea447d779c2e133c2c1256ef3 postgres/postgres@9e9931d

Relevant PG commits: 28f3915b73f75bd1b50ba070f56b34241fe53fd1 postgres/postgres@28f3915 ab355e3a88de745607f6dd4c21f0119b5c68f2ad postgres/postgres@ab355e3 024c521117579a6d356050ad3d78fdc95e44eefa postgres/postgres@024c521

eaydingol · 2024-07-26T11:36:14Z

src/backend/columnar/columnar_customscan.c

Did you consider using the macros from pg17 (e.g. removing the local variables)? Would it be feasible?

We still have to support PG15 and PG16. One way to do it would be to change Citus's current definition of these macros such that they are identical to PG17's definitions, and then iterate over all uses and remove the local variables.
Do you think that is better?

That would require to add the macro definition for PG < 17. Both (the current / alternative) seem feasible options, and both require quite some editing. You already implemented one of them. It seems OK to me.

@naisila, personally I think that using PG17's definitions when PG_VERSION_NUM >= PG_VERSION_17 and having identical definitions for PG15 and PG16 is better for you will naturally transition to PostgreSQL's macros when you no longer have to support PG15 and PG16. If you keep your own macros, you'll still have to support them even when PG15 and PG16 are no longer supported, or replace them with PostgreSQL's macros in the future.
Anyway, that's just my opinion, I'm not the one who'll have to support Citus, so it's not my call.

You are theoretically right, but in practice, we would have to update hundreds on instances in the codebase, all of which include if PG_VERSION_NUM >= PG_VERSION_17.

naisila marked this pull request as draft July 11, 2024 09:31

naisila force-pushed the pg17_kickoff branch from 0a57bae to ce1a029 Compare July 11, 2024 11:28

naisila added the pg17_support label Jul 23, 2024

Enable configure

ae3ed7d

naisila force-pushed the pg17_kickoff branch 2 times, most recently from cd85aa0 to e7e64c9 Compare July 25, 2024 10:39

naisila added 19 commits July 25, 2024 13:17

Remove ExecFreeExprContext call

a816d1b

Relevant PG commit d060e921ea5aa47b6265174c32e1128cebdbc3df postgres/postgres@d060e92

PG17 uses streaming IO in analyze, fix scan_analyze_next_block function

646ff0a

Relevant PG commit 041b96802efa33d2bc9456f2ad946976b92b5ae1 postgres/postgres@041b968

Define ObjectClass for PG17+ only since it's removed

85a3ed0

Relevant PG commit: 89e5ef7e21812916c9cf9fcf56e45f0f74034656 postgres/postgres@89e5ef7

No new node-wide object added so far in PG17, change assert

2434eff

Remove ReorderBufferTupleBuf structure.

2076339

Relevant PG commit: 08e6344fd6423210b339e92c069bb979ba4e7cd6 postgres/postgres@08e6344

Define colliculocale and daticulocale since they have been renamed

e9fe257

Relevant PG commit: f696c0cd5f299f1b51e214efc55a22a782cc175d postgres/postgres@f696c0c

makeStringConst defined in PG17

e33736e

Relevant PG commit: de3600452b61d1bc3967e9e37e86db8956c8f577 postgres/postgres@de36004

RangeVarCallbackOwnsTable was replaced by RangeVarCallbackMaintainsTable

4eafd3a

Relevant PG commit: ecb0fd33720fab91df1207e85704f382f55e1eb7 postgres/postgres@ecb0fd3

attstattarget is nullable, define pg compatible functions for it

be2bec0

Relevant PG commit: 4f622503d6de975ac87448aea5cea7de4bc140d5 postgres/postgres@4f62250

stxstattarget is nullable in PG17, write compat functions for it

0176974

Relevant PG commit: 012460ee93c304fbc7220e5b55d9d0577fc766ab postgres/postgres@012460e

Use ResourceOwner to track WaitEventSet in PG17

dd19847

Relevant PG commit: 50c67c2019ab9ade8aa8768bfe604cd802fe8591 postgres/postgres@50c67c2

getIdentitySequence now uses Relation instead of relation_id

022ca99

Relevant PG commit: 509199587df73f06eda898ae13284292f4ae573a postgres/postgres@5091995

Remove no-op tuplestore_donestoring function

e0a0874

Relevant PG commit: 75680c3d805e2323cd437ac567f0677fdfc7b680 postgres/postgres@75680c3

MergeAction can have 3 merge kinds (now enum) in PG17, write compat

49f7bff

Relevant PG commit: 0294df2f1f842dfb0eed79007b21016f486a3c6c postgres/postgres@0294df2

EXPLAIN (MEMORY) is added, make changes to ExplainOnePlan

a23bf88

Relevant PG commit: 5de890e3610d5a12cdaea36413d967cf5c544e20 postgres/postgres@5de890e

LIMIT_OPTION_DEFAULT has been removed as it's useless, use LIMIT_OPTI…

651bcc7

…ON_COUNT Relevant PG commit: a6be0600ac3b71dda8277ab0fcbe59ee101ac1ce postgres/postgres@a6be060

write compat for create_foreignscan_path bcs of more arguments in PG17

ec9beb5

Relevant PG commit: 9e9931d2bf40e2fea447d779c2e133c2c1256ef3 postgres/postgres@9e9931d

naisila force-pushed the pg17_kickoff branch from e7e64c9 to 3572f98 Compare July 25, 2024 11:19

add pg17 build test

76f60a7

naisila force-pushed the pg17_kickoff branch from 3572f98 to 76f60a7 Compare July 25, 2024 11:29

naisila mentioned this pull request Jul 25, 2024

Adds PG17.1 support - Regression tests sanity #7661

Merged

naisila marked this pull request as ready for review July 25, 2024 11:53

eaydingol reviewed Jul 26, 2024

View reviewed changes

naisila marked this pull request as draft October 6, 2024 19:15

This was referenced Oct 16, 2024

Rename foreach_ macros to foreach_declared_ macros #7700

Merged

PG17 compatibility: Resolve compilation issues #7699

Merged

naisila closed this Oct 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PG17 compatibility: Resolve compilation issues #7651

PG17 compatibility: Resolve compilation issues #7651

naisila commented Jul 10, 2024 •

edited

Loading

codecov bot commented Jul 10, 2024 •

edited

Loading

eaydingol Jul 26, 2024

naisila Jul 29, 2024

eaydingol Aug 6, 2024

Green-Chan Oct 17, 2024

naisila Nov 12, 2024

PG17 compatibility: Resolve compilation issues #7651

PG17 compatibility: Resolve compilation issues #7651

Conversation

naisila commented Jul 10, 2024 • edited Loading

codecov bot commented Jul 10, 2024 • edited Loading

Codecov Report

eaydingol Jul 26, 2024

Choose a reason for hiding this comment

naisila Jul 29, 2024

Choose a reason for hiding this comment

eaydingol Aug 6, 2024

Choose a reason for hiding this comment

Green-Chan Oct 17, 2024

Choose a reason for hiding this comment

naisila Nov 12, 2024

Choose a reason for hiding this comment

naisila commented Jul 10, 2024 •

edited

Loading

codecov bot commented Jul 10, 2024 •

edited

Loading