[PG-1637] Remove any existing keys for tables on create #388
Conversation
Codecov ReportAttention: Patch coverage is
❌ Your project status has failed because the head coverage (85.55%) is below the target coverage (90.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## TDE_REL_17_STABLE #388 +/- ##
=====================================================
- Coverage 85.72% 85.55% -0.18%
=====================================================
Files 22 22
Lines 2529 2547 +18
Branches 387 388 +1
=====================================================
+ Hits 2168 2179 +11
- Misses 287 292 +5
- Partials 74 76 +2
🚀 New features to boost your workflow:
|
| @@ -93,6 +93,7 @@ _PG_init(void) | |||
|
|
|||
| check_percona_api_version(); | |||
|
|
|||
| pg_tde_init_data_dir(); | |||
There was a problem hiding this comment.
what was the reason to move it here? I'm not objecting it, just curious
There was a problem hiding this comment.
As I mentioned in the commit message, this directory is used by the SMgr, and the SMgr is loaded and used regardless of whether any database have run CREATE EXTENSION. So the directory should be created regardless.
Practically it was because the call to pg_free_key_map_entry() crashed when the directory didn't exist :).
There was a problem hiding this comment.
Yes, sorry, I missed it. Thanks
|
|
||
| if (key) | ||
| if (unlikely(key)) |
There was a problem hiding this comment.
When can this even happen? Is it a thing which only happens if there are leftover keys or we are doing a recovery/replay on the replica? This branch should be explained.
There was a problem hiding this comment.
New version up that is much more strict about what can happen in each case.
I have to think about this. I was worried about the recovery case, but there will be a WAL record for creating the key after this function in called, so it shouldn't be an issue. That would also remove the need for the branch you asked about I'm pretty sure. I need to verify though. I also need to understand exactly how and when the forks are created et.c. because my understanding of this is a little hazy. |
Only create keys when MAIN fork is created, and trust tde_smgr_should_encrypt() to know when to encrypt. Also trust that the key has already been created if we're in recovery or replication.
Instead of it being done down in tdemap code controlled with a boolean, we just do it where the decision to do it is made instead.
There are crash scenarios where keys are left behind in the key file even though the OID for the table goes unused. This meant that we could have keys laying around for newly created plaintext relations after OID wraparound. Simply removing any existing keys when relations are created seems appriopriate. Also move creation of pg_tde data dir to library init. This directory is used by the SMgr which is loaded regardless of whether any database are yet to create extension or not.
AndersAstrand
force-pushed
the
tde/remove-unused-relation-keys
branch
from
de81c34 to
45303eb
Compare
AndersAstrand
changed the title
Some crash scenarios can leave keys behind in the relation key file for OIDs that are unused. If these OIDs are later used when creating a non-encrypted relation the key has to be removed or we will do the wrong thing since the existence of a key makes us assume the relation is encrypted.