Spike/argocd demo

[maltesander]

ArgoCD demo depolying stackable components and running a spark job via airflow.

What it does:

• Stack ArgoCD

  • sealed secrets controller (https://github.com/bitnami-labs/sealed-secrets)
  • deploys all operators (except secret, listener and commons which are currently installed via stackablectl, since there must be operators installed in a stack) via applicationset
  • generated tls certs for sealed secrets

• Demo based on the airflow demo, deployed via ArgoCD

  • deploy minio for airflow kubernetes executor logs
  • gitsync to fetch dags
  • automatically deploy minio connection to airflow
  • all secrets sealed
  • deploy spark job via UI

Clone the demos repo and checkout the branch spike/argocd-demo and run

stackablectl demo install -n argo-cd -s stacks/stacks-v2.yaml -d demos/demos-v2.yaml argocd

I currently port forward the argocd-server and airflow webserver. In the airflow webUI you can start the pyspark job manually.

Feedback would be very welcome :)