feat(taskbroker): add deployment

[TartanLeGrand]

This pull request introduces a new "taskbroker" component to the Sentry Helm chart, including its deployment, configuration, and associated Kubernetes resources. The changes enable task scheduling and execution capabilities by adding a task broker, task scheduler, and task worker. Additionally, it includes updates to support persistence, autoscaling, and environment-specific configurations.

Taskbroker Component Additions:

• Taskbroker Deployment: Added a new Deployment for the taskbroker, including support for custom annotations, labels, resource requests/limits, persistence, and environment variables (charts/sentry/templates/sentry/taskbroker/deployment-taskbroker.yaml).
• Taskbroker Service: Introduced a headless Service for the taskbroker to facilitate communication on port 50051 (charts/sentry/templates/sentry/taskbroker/service-taskbroker.yaml).
• Taskbroker PersistentVolumeClaim: Added support for persistent storage for the taskbroker when persistence is enabled (charts/sentry/templates/sentry/taskbroker/pvc-taskbroker.yaml).
• Taskbroker HorizontalPodAutoscaler: Added autoscaling capabilities for the taskbroker based on CPU utilization (charts/sentry/templates/sentry/taskbroker/hpa-taskbroker.yaml).
• Taskbroker ServiceAccount: Created a dedicated ServiceAccount for the taskbroker to manage permissions (charts/sentry/templates/sentry/taskbroker/serviceaccount-sentry-taskbroker.yaml).

Task Scheduling Enhancements:

• Task Scheduler Deployment: Added a Deployment for the task scheduler, which runs the taskworker-scheduler command to manage task scheduling (charts/sentry/templates/sentry/taskbroker/deployment-taskscheduler.yaml).
• Task Worker Deployment: Added a Deployment for task workers to execute tasks, including support for concurrency and RPC communication with the taskbroker (charts/sentry/templates/sentry/taskbroker/deployment-taskworker.yaml).

Configuration Updates:

• Helper Template Updates: Added helper functions for generating taskbroker-specific image references, environment variables, and hostnames (charts/sentry/templates/_helper.tpl). [1] [2]
• Redis Configuration: Updated common Sentry environment variables to include Redis host and port (charts/sentry/templates/_helper.tpl).

Values Configuration:

• Taskbroker Values: Introduced a new taskbroker section in values.yaml for configuring replicas, autoscaling, persistence, resource requests/limits, and environment variables (charts/sentry/values.yaml). [1] [2]

These changes collectively enhance the Sentry Helm chart by introducing a robust task management system and providing flexibility for deployment configurations.

[Mokto]

Could you explain why this is needed please?

[cjcormack]

Taskbroker is a new service that Sentry added in 25.6.0 (as a replacement to Celery), see getsentry/self-hosted#3738.

[TartanLeGrand]

Wait for getsentry/sentry#96966 because getsentry/self-hosted#3845

[Mokto]

👋 Hi, @TartanLeGrand,
I detected conflicts against the base branch 🙊
You'll want to sync 🔄 your branch with upstream!

[manishrawat1992]

@TartanLeGrand Thanks for taking this initiative,
I will also try to contribute to this PR.

But the way this should be implemented is as follows.
We need statfulset for taskbroker, as it will require disk

the way this file should be is

for each taskbroker
{{- range $name, $config := .Values.taskbroker }}
taskworker deployment with n replicas yaml pointing to this taskbroker(n can be 1-32)
{{-end}}

[manishrawat1992]

This should ideally be a statefulset because, embedded sqlite might require pvc also.

Take example of symbolicator statefulset