-
Notifications
You must be signed in to change notification settings - Fork 9
[PG-1637] Remove any existing keys for tables on create #388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[PG-1637] Remove any existing keys for tables on create #388
Conversation
Codecov ReportAttention: Patch coverage is
❌ Your project status has failed because the head coverage (85.55%) is below the target coverage (90.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## TDE_REL_17_STABLE #388 +/- ##
=====================================================
- Coverage 85.72% 85.55% -0.18%
=====================================================
Files 22 22
Lines 2529 2547 +18
Branches 387 388 +1
=====================================================
+ Hits 2168 2179 +11
- Misses 287 292 +5
- Partials 74 76 +2
🚀 New features to boost your workflow:
|
@@ -93,6 +93,7 @@ _PG_init(void) | |||
|
|||
check_percona_api_version(); | |||
|
|||
pg_tde_init_data_dir(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what was the reason to move it here? I'm not objecting it, just curious
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I mentioned in the commit message, this directory is used by the SMgr, and the SMgr is loaded and used regardless of whether any database have run CREATE EXTENSION. So the directory should be created regardless.
Practically it was because the call to pg_free_key_map_entry() crashed when the directory didn't exist :).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, sorry, I missed it. Thanks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My first thought is: why can't we always delete keys, even for encrypted relations? And I do not find an answer in the PR.
|
||
if (key) | ||
if (unlikely(key)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When can this even happen? Is it a thing which only happens if there are leftover keys or we are doing a recovery/replay on the replica? This branch should be explained.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New version up that is much more strict about what can happen in each case.
I have to think about this. I was worried about the recovery case, but there will be a WAL record for creating the key after this function in called, so it shouldn't be an issue. That would also remove the need for the branch you asked about I'm pretty sure. I need to verify though. I also need to understand exactly how and when the forks are created et.c. because my understanding of this is a little hazy. |
Only create keys when MAIN fork is created, and trust tde_smgr_should_encrypt() to know when to encrypt. Also trust that the key has already been created if we're in recovery or replication.
Instead of it being done down in tdemap code controlled with a boolean, we just do it where the decision to do it is made instead.
There are crash scenarios where keys are left behind in the key file even though the OID for the table goes unused. This meant that we could have keys laying around for newly created plaintext relations after OID wraparound. Simply removing any existing keys when relations are created seems appriopriate. Also move creation of pg_tde data dir to library init. This directory is used by the SMgr which is loaded regardless of whether any database are yet to create extension or not.
de81c34
to
45303eb
Compare
Some crash scenarios can leave keys behind in the relation key file for OIDs that are unused. If these OIDs are later used when creating a non-encrypted relation the key has to be removed or we will do the wrong thing since the existence of a key makes us assume the relation is encrypted.