New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: backup vault k8s secrets to s3 encrypted. #142

Open

artsyjian wants to merge 13 commits into main from artsyjian/vault

+618 −228

Collaborator

artsyjian commented Feb 23, 2025 •

edited

Loading

This repo is public.

The type of this PR is: Feat

This PR solves PHIRE-1618

Description

Add script to backup Vault K8S secrets to S3. The secrets are AES256 encrypted.

foreman run --env .env.shared,.env python src/vault_backup/main.py staging --s3

Add a cronjob for the script for each env, to be run daily.
Rename bucket name for Vault snapshot script from "BACKUP" to "SNAPSHOT", in order to not collide with the new script. (Other changes are just formatting)

Shared env has been updated, with instruction on how to decrypt.

Before-merging tasks:

Generate encryption key, one per env.
Update Staging env
Update Production env

artsyjian requested a review from cavvia

February 23, 2025 21:55

artsyjian self-assigned this

artsy-peril bot added the Jira Synced label

cavvia reviewed

View reviewed changes

src/vault_backup/backup.py Outdated Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/backup.py Outdated Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/main.py Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/context.py Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/backup.py Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/backup.py Outdated Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/main.py Outdated Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/main.py Outdated Show resolved Hide resolved

cavvia requested changes

View reviewed changes

cavvia left a comment •

edited

Loading

Thanks for this! The non-standard formatting is a blocker. I have some questions about the layout of the code and your cli opts as well.


          backup vault k8s secrets in ascii

dcf298c

artsyjian force-pushed the artsyjian/vault branch from 3886d27 to dcf298c Compare

February 25, 2025 15:32

artsyjian added 2 commits

February 25, 2025 08:16


          use defaultdict

946224e


          kebab log level arg

8fae273

mc-jones requested changes

View reviewed changes

Contributor

mc-jones left a comment •

edited

Loading

Collaborator Author

artsyjian commented Feb 25, 2025

let's hold this PR pending this ticket.

artsyjian added 5 commits

March 5, 2025 09:00


          encrypt backup

51340f0


          update prefix

8bd092a


          use var for sub prefix

ef32b04


          change var names for snapshot script

1e8de84


          cronjobs for vault backup

3ee9b98

artsyjian requested review from cavvia, mc-jones, brainbicycle and joeyAghion

March 5, 2025 17:57

artsyjian changed the title ~~feat: backup vault k8s secrets in ascii~~ feat: backup vault k8s secrets to s3 encrypted.


          template to say repo public

40e1ee8

cavvia reviewed

View reviewed changes

src/vault_backup/backup.py Outdated Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_backup/backup.py Outdated Show resolved Hide resolved

cavvia reviewed

View reviewed changes

src/vault_snapshot/vault_snapshot/snap.py Outdated Show resolved Hide resolved


          better docstring

54dcce2

artsyjian removed request for joeyAghion, mc-jones and brainbicycle

March 11, 2025 14:12

artsyjian added 3 commits

March 11, 2025 07:48


          flatten vault snapshot dir

ff93ca2


          update cronjob

e4577c4


          break encrypt function out to lib

8aa8709

artsyjian requested a review from cavvia

March 11, 2025 15:39

cavvia reviewed

View reviewed changes

src/lib/encrypt.py

Comment on lines +4 to +8

    
              def symmetric_encrypt(key, iv, message):

                  """AES256 encrypt message and return ciphertext

                  params are all str

                  ciphertext returned is in bytes

                  """

cavvia Mar 12, 2025

Suggested change

      
            def symmetric_encrypt(key, iv, message):
          
                """AES256 encrypt message and return ciphertext
          
                params are all str
          
                ciphertext returned is in bytes
          
                """
          
            def symmetric_encrypt(key: str, iv: str, message: str) -> bytes:
          
                """AES256 encrypt a message.
          
                Returns
          
                -------
          
                ciphertext: bytes
          
                """

cavvia Mar 12, 2025 •

edited

Loading

Switches to using a typed function signature and use a standard docstring format (numpydoc in this case, which is commonly used in python and IDEs support).

cavvia reviewed

View reviewed changes

src/vault_snapshot/main.py

    
              def validate(artsy_env, vault_host, vault_port, s3, s3_bucket):

                  """validate config obtained from env and command line"""

                  if not (vault_host and vault_port):

                      raise Exception(

cavvia Mar 12, 2025 •

edited

Loading

This and the following exceptions could use the built-in ValueError instead. Use of generic Exception is generally to be avoided in favour of more specific errors.

cavvia approved these changes

View reviewed changes

cavvia left a comment

A couple of non-blocking suggestions, otherwise looks good from a python coding conventions perspective.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels