Deploy Lakekeeper with fine-grained authorization enabled

[martyngigg]

The local development setup has the OpenFGA backend enabled for fine-grained authorization within Lakekeeper.

We need to replicate this in the cloud-deployed configuration. It will require a new deployment of [OpenFGA] along with a database supporting it. To begin with a single Trino client will be registered and authorized and permissions will entirely be controlled within Superset.

Further development with explore Open Policy Agent to enable unify permissions handling.

[martyngigg]

Requests for new databases have been made:

• keycloak: Moving from locally managed
• openfga: New to support OpenFGA

[martyngigg]

QA deployment

Prerequisites

• Contact users to check when when we can take Superset down for a few hours

On the day

• Stop current services
• Backup:
  • Lakekeeper DB
  • Keycloak DB
  • Superset DB
• Wipe Lakekeeper & Keycloak DB
• Run Terraform scripts to provision QA Openstack resources
• Run Ansible scripts to deploy services
  • Check all services running
• Run ELT jobs to reingest all data

[martyngigg]

Deployment has been completed.