Skip to content

Repository files navigation

CI/CD & Monitoring

License: MIT

Production-ready CI/CD pipelines and monitoring configurations for AWS, GCP, and On-Premise Kubernetes environments.


Tech Stack

KubernetesDockerHelmPrometheusGrafanaLokiThanosArgoCDJenkinsGitHub ActionsGitLab CIElasticsearchKibanaFluent BitNginx


Platform Coverage

Monitoring

Tool AWS GCP On-Premise
Kube-Prometheus-Stack O O O
Grafana O O O
Thanos - - O
Node Exporter - - O
Prometheus MySQL Exporter - - O
Prometheus Redis Exporter - - O
Prometheus Elasticsearch Exporter - - O

Logging

Tool AWS GCP On-Premise
Loki O O O
Promtail - - O
Elasticsearch - - O
Kibana - - O
ECK Operator - - O
Fluent Bit O - O
Fluentd - - O

CI/CD

Tool AWS GCP On-Premise
ArgoCD O O O
Jenkins - - O

Registry

Tool AWS GCP On-Premise
Harbor - - O

Security

Tool AWS GCP On-Premise
Vaultwarden - - O

Directory Structure

aws/ - AWS EKS Monitoring & GitOps
aws/
├── argocd/                  # ArgoCD with ALB Ingress
├── grafana/                 # Grafana (grafana-community chart v11.3.5)
├── kube-prometheus-stack/   # Prometheus + Alertmanager + Operator
├── loki/                    # Loki log aggregation
├── fluent-bit-aws/          # Fluent Bit log collector
└── gitlab-runner-aws/       # GitLab Runner for AWS
Component Chart Version
ArgoCD argo/argo-cd 9.4.16
Grafana grafana-community/grafana 11.3.5
Kube-Prometheus-Stack prometheus-community/kube-prometheus-stack 82.14.1
Loki grafana/loki 6.55.0
Fluent Bit fluent/fluent-bit 0.49.3
GitLab Runner gitlab/gitlab-runner 0.87.0
gcp/ - GCP GKE Monitoring & GitOps
gcp/
├── argocd/                  # ArgoCD with GKE Ingress
├── grafana/                 # Grafana (grafana-community chart v11.3.5)
├── kube-prometheus-stack/   # Prometheus + Alertmanager + Operator
└── loki/                    # Loki log aggregation
Component Chart Version
ArgoCD argo/argo-cd 9.4.16
Grafana grafana-community/grafana 11.3.5
Kube-Prometheus-Stack prometheus-community/kube-prometheus-stack 82.14.1
Loki grafana/loki 6.55.0
onpremise/ - On-Premise Full Stack
onpremise/
├── argocd/                              # ArgoCD with Dex SSO & Slack notifications
├── elk-stack/                           # Elasticsearch, Kibana, Filebeat, Logstash, APM
├── grafana/                             # Grafana dashboards
├── harbor-helm/                         # Harbor container registry
├── ingress-nginx-sidecar-fluentbit/     # Nginx ingress with Fluent Bit sidecar
├── jenkins/                             # Jenkins CI server
├── kube-prometheus-stack/               # Full Kubernetes monitoring stack
├── loki/                                # Log aggregation
├── node-exporter/                       # Node Exporter (Ansible-deployed on hosts)
├── prometheus-elasticsearch-exporter/   # Elasticsearch metrics exporter
├── prometheus-mysql-exporter/           # MySQL metrics exporter
├── prometheus-redis-exporter/           # Redis metrics exporter
├── promtail/                            # Log collector for Loki
├── thanos/                              # Long-term metrics storage
└── vaultwarden/                         # Self-hosted Bitwarden-compatible password manager
Component Link
ArgoCD GitLab SSO, Redis HA, Slack notifications
ELK Stack APM, Elasticsearch, Filebeat, Kibana, Logstash, Metricbeat
Grafana Dashboards and data source configuration
Harbor Container registry with Helm chart
Ingress Nginx + Fluent Bit Access log collection sidecar
Jenkins CI/CD server
Kube-Prometheus-Stack Prometheus, Alertmanager, Operator
Loki Log aggregation system
Node Exporter Host-level metrics via Ansible (playbook, rollback, uninstall)
Prometheus Elasticsearch Exporter Chart prometheus-community/prometheus-elasticsearch-exporter 7.2.1
Prometheus MySQL Exporter Chart prometheus-community/prometheus-mysql-exporter 2.13.1
Prometheus Redis Exporter Chart prometheus-community/prometheus-redis-exporter 6.22.0
Promtail Log collector agent
Thanos Long-term Prometheus storage
Vaultwarden Self-hosted password manager (chart 0.35.1, appVersion 1.35.4)
github-cicd/ - GitHub Actions Workflows
github-cicd/
├── aws/                     # AWS build & deploy workflows
│   ├── build-deploy-repo/   # ECR build, Helm deploy, S3 upload
│   └── data-repo/           # Data validation & upload pipeline
├── gcp/                     # GCP build & deploy workflows
│   ├── build-deploy-repo/   # GAR build, Helm deploy, GCS upload
│   ├── data-repo/           # Data validation & upload pipeline
│   ├── data-build-deploy-repo/          # Combined data + build + deploy
│   ├── matrix-strategy/                 # Matrix strategy example
│   ├── upgrade-data-build-deploy-repo/  # Enhanced pipeline v1
│   └── upgrade-data-build-deploy-repo-v2/  # Enhanced pipeline v2
└── aws-gcp/                 # Hybrid multi-cloud workflow

See github-cicd/README.md for details.

gitlab-cicd/ - GitLab CI/CD Templates & Pipelines
gitlab-cicd/
├── templates/               # Reusable CI/CD components
│   ├── variables/           # Common variables and service definitions
│   ├── auth/                # AWS ECR authentication
│   ├── build/               # Kaniko build (Harbor & ECR)
│   ├── deploy/              # ArgoCD GitOps deployment
│   ├── backup/              # Google Drive backup
│   └── examples/            # Template usage examples
├── pipelines/               # Complete pipeline patterns (48 files)
│   ├── server/              # Server service (harbor/aws/module versions)
│   ├── data/                # Data pipeline (harbor/aws/module versions)
│   ├── client/              # Client CI pipelines
│   ├── admin/               # Admin service pipelines
│   ├── battle/              # Battle service pipelines
│   ├── build-deploy/        # Basic build & deploy patterns
│   ├── gcp-artifact-registry/  # GCP Artifact Registry patterns
│   ├── data-upload/         # Data upload utilities
│   └── backup/              # Google Drive backup
└── scripts/                 # Utility scripts (Python, TypeScript)

See gitlab-cicd/README.md for details.

github-runner/ - Self-hosted GitHub Actions Runner
github-runner/
└── actions-runner-controller/   # ARC with Helmfile
Component Link
Actions Runner Controller Self-hosted runner on Kubernetes
gitlab-runner/ - Self-hosted GitLab Runner
gitlab-runner/
├── helmfile.yaml            # Helmfile release definition
├── values/                  # Environment-specific values
└── upgrade.sh               # Version upgrade script

See gitlab-runner/README.md for details.

jenkins/ - Jenkins Pipeline Examples
jenkins/
├── init-jenkinsfile         # Initialization pipeline
├── ios-jenkinsfile          # iOS build pipeline
├── lua-jenkinsfile          # Lua build pipeline
└── res-jenkinsfile          # Resource build pipeline

See jenkins/README.md for details.

scripts/ - Shared Tooling
scripts/
└── upgrade-sync/            # Canonical upgrade.sh templates + drift sync tool
    ├── sync.sh              # --check / --apply to keep all component upgrade.sh in sync
    └── templates/           # Source-of-truth upgrade.sh bodies

See scripts/upgrade-sync/README.md for the sync workflow.


Documentation

Document Description
Architecture Overview Platform architecture, component dependency graph, network flow, storage layout
Disaster Recovery Recovery procedures for each major component

Getting Started

  1. Choose your target platform (aws/, gcp/, or onpremise/)
  2. Navigate to the specific tool directory
  3. Follow the README instructions in each directory
  4. Customize values files for your environment
# Example: Deploy kube-prometheus-stack on AWS
cd aws/kube-prometheus-stack
helmfile apply

# Example: Deploy ArgoCD on GCP
cd gcp/argocd
helmfile apply

Helmfile Pattern

All monitoring components follow a consistent Helmfile remote chart pattern:

component/
├── Chart.yaml          # Version tracking (upstream format)
├── helmfile.yaml       # Helmfile release definition (remote chart)
├── values.yaml         # Upstream default values (auto-managed by upgrade.sh)
├── values/
│   └── mgmt.yaml       # Environment-specific overrides
├── examples/           # Storage, ingress, and other examples
├── upgrade.sh          # Automated version upgrade script
├── backup/             # Auto backup on upgrade
└── README.md
# Common operations
helmfile lint     # Validate
helmfile diff     # Preview changes
helmfile apply    # Deploy
helmfile destroy  # Delete

# Upgrade
./upgrade.sh              # Upgrade to latest
./upgrade.sh --dry-run    # Preview only
./upgrade.sh --rollback   # Restore from backup

Related Repositories

For platform-specific application deployment templates (EKS Fargate, GKE with Firestore/PD-CSI/NFS, on-premise Kubernetes), see helm-chart-template.


License

This project is licensed under the MIT License - see the LICENSE file for details.

Releases

Packages

Used by

Contributors

Languages