diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 6eec6395..95f18285 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -169,22 +169,25 @@ graph TD ### Production-Grade Features - **Zero-configuration GPG**: Uses GPG_PRIVATE_KEY secret automatically - **AES256 encryption**: Local state files encrypted with GPG -- **Custom naming**: Pattern: `{provider}-{component}-{branch}-terraform.tfstate` +- **Custom naming**: + - for base-infra and observ-infra: `{provider}-{component}-{branch}-terraform.tfstate` + - for infra: `{provider}-{component}-{profile}-{branch}-terraform.tfstate` - **Git safety**: Encrypted state files tracked in repository - **Branch isolation**: Complete separation of environment states ### State File Organization ```bash # Repository Structure (Encrypted) -.terraform-state/ -├── aws-base-infra-testgrid-terraform.tfstate.gpg -├── aws-infra-testgrid-terraform.tfstate.gpg -└── aws-observ-infra-testgrid-terraform.tfstate.gpg +terraform/implementations/aws/base-infra/aws-base-infra--terraform.tfstate.gpg +terraform/implementations/aws/infra/profiles//aws-infra---terraform.tfstate.gpg +terraform/implementations/aws/observ-infra/aws-observ-infra--terraform.tfstate.gpg # Decrypted for Terraform Use (Temporary) -terraform/base-infra/aws-base-infra-testgrid-terraform.tfstate -terraform/infra/aws-infra-testgrid-terraform.tfstate -terraform/observ-infra/aws-observ-infra-testgrid-terraform.tfstate +terraform/implementations/aws/base-infra/aws-base-infra--terraform.tfstate +terraform/implementations/aws/infra/profiles//aws-infra---terraform.tfstate +terraform/implementations/aws/observ-infra/aws-observ-infra--terraform.tfstate + +where = mosip/esignet ``` ### GPG Key Management @@ -214,7 +217,7 @@ GPG_PRIVATE_KEY: | ```yaml BACKEND_TYPE: local # State files encrypted with GPG and stored in repository -# Custom naming: aws-infra-testgrid-terraform.tfstate +# Custom naming: aws-infra--terraform.tfstate ``` #### Remote Backend (Legacy Support) @@ -303,13 +306,15 @@ TERRAFORM_APPLY: true CLOUD_PROVIDER: aws TERRAFORM_COMPONENT: infra BACKEND_TYPE: local -# State: aws-infra-main-terraform.tfstate.gpg +INFRA_PROFILE: mosip/esignet +# State: profiles//aws-infra--main-terraform.tfstate.gpg # Staging deployment (staging branch) CLOUD_PROVIDER: aws TERRAFORM_COMPONENT: infra BACKEND_TYPE: local -# State: aws-infra-staging-terraform.tfstate.gpg +INFRA_PROFILE: mosip/esignet +# State: profiles//aws-infra--staging-terraform.tfstate.gpg ``` ### Sequential Workflow Deployment diff --git a/README.md b/README.md index 0726ffe8..cd103c8c 100644 --- a/README.md +++ b/README.md @@ -205,6 +205,18 @@ AWS_SECRET_ACCESS_KEY: "..." # What it's for: Secret key that pairs with access key ID (like a password) # IMPORTANT: Keep this SECRET! Never commit to Git or share publicly +# GitHub Personal Access Token +GH_INFRA_PAT: "github_pat_..." +# What it's for: Required for repository operations during deployment +# How to get: GitHub Settings → Developer Settings → Personal access tokens (Fine-grained) +# Permissions Required: +# - Contents: Read and write (critical, Read only causes 403 on push) +# - Metadata: Read +# - Actions: Read and write +# - Environments: Read and write +# - Variables: Read and write +# NOTE: No Secrets permission needed (intentionally excluded) + # SSH Private Key (must match ssh_key_name in tfvars) YOUR_SSH_KEY_NAME: | # Replace YOUR_SSH_KEY_NAME with actual ssh_key_name value from your tfvars @@ -222,6 +234,7 @@ YOUR_SSH_KEY_NAME: | - [ ] GPG Passphrase created (16+ characters) - [ ] AWS Access Key ID obtained from IAM - [ ] AWS Secret Access Key saved securely +- [ ] GitHub PAT (GH_INFRA_PAT) generated with correct permissions - [ ] SSH key pair generated (public + private) - [ ] SSH public key uploaded to AWS EC2 Key Pairs - [ ] SSH private key added to GitHub secrets @@ -327,6 +340,7 @@ Add the required secrets as follows: - `GPG_PASSPHRASE` - `AWS_ACCESS_KEY_ID` - `AWS_SECRET_ACCESS_KEY` +- `GH_INFRA_PAT` - `YOUR_SSH_KEY_NAME` (replace with actual ssh_key_name value from tfvars, e.g., `mosip-aws`) - **Environment Secrets** (Settings → Secrets and variables → Actions → Environment secrets): - All other secrets mentioned in the Prerequisites section above (KUBECONFIG, WireGuard configs, etc.) @@ -403,7 +417,7 @@ For detailed information about GitHub Actions workflow parameters, terraform mod - **Backend**: Choose backend configuration: - **(5)** `local` - GPG-encrypted local state (recommended for development) - Stores state in your GitHub repository (encrypted) - - **(6)** `s3` - Remote S3 backend (recommended for production) + - **(6)** `s3` - Remote S3 backend (If you want to store the state file in a S3 bucket, provide the bucket name. Otherwise, leave it empty to use the local backend) - Stores state in AWS S3 bucket (centralized) - **(7)** **SSH_PRIVATE_KEY**: GitHub secret name containing SSH private key for instance access - Must match the `ssh_key_name` in your terraform.tfvars @@ -630,7 +644,7 @@ For complete workflow usage instructions, inputs, secrets configuration, and tro #### Step 3d: MOSIP Infrastructure -This step creates MOSIP Kubernetes cluster, PostgreSQL (if enabled), networking, and application infrastructure +This step creates MOSIP Kubernetes cluster, PostgreSQL (if enabled), ActiveMQ (if enabled), networking, and application infrastructure 1. **Update infra variables in `terraform/implementations/aws/infra/aws.tfvars`:** @@ -841,16 +855,17 @@ After updating `aws.tfvars`, deploy or update your main infra cluster: - **(3)** **Branch**: Select your deployment branch (e.g., `release-0.1.0`) - **(4)** **Cloud Provider**: Select `aws` (Azure/GCP are placeholder implementations) - **(5)** **Component**: Select `infra` (MOSIP application infrastructure) -- **(6)** **Backend**: Choose backend configuration: - - `local` - GPG-encrypted local state (recommended for development) - - `s3` - Remote S3 backend (recommended for production) -- **(7)** **SSH_PRIVATE_KEY**: GitHub secret name containing SSH private key for instance access +- **(6)** **Profile**: Select `mosip`/`esignet` (Select profile which you want to use for deployment) +- **Backend**: Choose backend configuration: + - **(7)** `local` - GPG-encrypted local state (recommended for development) + - **(8)** `s3` - Remote S3 backend (If you want to store the state file in a S3 bucket, provide the bucket name. Otherwise, leave it empty to use the local backend) +- **(9)** **SSH_PRIVATE_KEY**: GitHub secret name containing SSH private key for instance access - Must match the `ssh_key_name` in your terraform.tfvars -- **Terraform apply**: - - **(8)** ☐ **Unchecked** — Plan mode: runs terraform plan (shows changes without applying). - - **(8)** ✅ **Checked** — Apply mode: runs terraform apply (creates/updates infrastructure). +- **(10)** **☐ Terraform apply**: + - ☐ **Unchecked** — Plan mode: runs terraform plan (shows changes without applying). + - ✅ **Checked** — Apply mode: runs terraform apply (creates/updates infrastructure). - Tip: For your first deployment, run in plan mode first to review changes. If the plan looks correct, re-run the workflow with Apply checked. -- **(9)** **Run Workflow** +- **(11)** **Run Workflow** **If Workflow Fails - How to View Error Logs:** @@ -1314,17 +1329,23 @@ The Helmsman deployment process follows a specific sequence with automated trigg ![Deploy External Services - Helmsman](docs/_images/helmsman-external-services.png) -- **(1)** Actions → **"Deploy External services of mosip using Helmsman"** (or "Helmsman External Dependencies") +- **(1)** Actions → **"Deploy External services of mosip using Helmsman"** - **Can't find it?** Search for "External" in the workflows list -- **(2)** **Select Run workflow** -- **(3)** **Select Branch** +- **(2)** Click **Run workflow** button in the top right corner +- **(3)** **Branch**: Select your deployment branch (e.g., `develop`) +- **(4)** **Deployment profile to use**: `mosip-platform-java11` (or other appropriate profile) +- **(5)** **Choose Helmsman mode**: `apply` (dry-run will fail due to namespace dependencies) +- **(6)** **Domain name for this environment**: Enter the domain name (e.g., `example.xyz.net`) +- **(7)** **Environment name**: Enter the environment name (e.g., `sandbox`, `dev`, `staging`) +- **(8)** **Slack channel name for alerting** (optional): e.g., `#mosip-alerts` +- **(9)** **Slack webhook URL for alerting** (optional) +- **(10)** **Rancher cluster ID for rancher-monitoring**: e.g., `c-xxxxx` +- **(11)** Click **Run workflow** green button - This workflow handles both deployments in parallel: - **Prerequisites**: `prereq-dsf.yaml` (monitoring, Istio, logging) - **External Dependencies**: `external-dsf.yaml` (databases, message queues, storage) -- **(4)** **Mode**: `apply` (required - dry-run will fail!) - - **Important:** DO NOT select dry-run mode for Helmsman - - **Time required:** 20-40 minutes - - **Automatic Trigger**: Upon successful completion, this workflow automatically triggers the MOSIP services deployment +- **Time required:** 20-40 minutes +- **Automatic Trigger**: Upon successful completion, this workflow automatically triggers the MOSIP services deployment **What You Should See:** @@ -1341,7 +1362,7 @@ The Helmsman deployment process follows a specific sequence with automated trigg - **Automatically triggered** after successful completion of step 1 - Workflow: **Deploy MOSIP services using Helmsman** (`helmsman_mosip.yml`) - DSF file: `mosip-dsf.yaml` -- Mode: `apply` (required - dry-run will fail due to namespace dependencies) +- Mode: `apply` (dry-run will fail due to namespace dependencies) **Error Handling:** @@ -1379,19 +1400,23 @@ The Helmsman deployment process follows a specific sequence with automated trigg - **Prerequisites**: All MOSIP core services must be running, partner onboarding completed successfully and secrets required for esignet should be updated. - **(1)** Actions → **Deploy eSignet using Helmsman** (`helmsman_esignet.yml`) -- **(2)** **Select Branch** -- **(3)** **Mode**: `apply` (required - dry-run will fail due to namespace dependencies) -- **(4)** **Additional Options** (optional): +- **(2)** Click **Run workflow** button in the top right corner +- **(3)** **Select Branch** +- **(4)** **Select Profile**: `mosip-platform-java11` or `mosip-platform-java21` or `esignet` or any other profile you want to deploy for +- **(5)** **Mode**: `apply` (dry-run will fail due to namespace dependencies) +- **(6)** **Additional Options** (optional): - **skip_mosip_dsf_check**: ☐ Unchecked by default - **When to enable (✅)**: Standalone eSignet deployment without full MOSIP stack - **What it does**: Bypasses validation check for MOSIP core services completion - **Use case**: Testing eSignet independently or deploying eSignet to a separate cluster - - **(5)** **delete_existing_jobs**: ☐ Unchecked by default + - **(7)** **delete_existing_jobs**: ☐ Unchecked by default - **When to enable (✅)**: Re-running eSignet deployment after a previous failed attempt - **What it does**: Removes existing partner onboarder jobs before creating new ones - **Use case**: Cleanup before retry deployment to avoid "job already exists" errors - **Important**: Only enable this on re-runs, not on first deployment -- **(6)** **Run Workflow**: +- **(8)** **Domain name**: Enter the domain name for this environment (e.g., `example.xyz.net`) +- **(9)** **Environment name**: Enter the environment name (e.g., `sandbox`, `dev`, `staging`) +- **(10)** **Run Workflow** - **Time required:** 15-25 minutes **What You Should See:** @@ -1420,9 +1445,15 @@ The Helmsman deployment process follows a specific sequence with automated trigg - **Prerequisites**: All pods from steps 1-2 must be in `Running` state and onboarding completed successfully - **(1)** Actions → **Deploy Testrigs of mosip using Helmsman** (`helmsman_testrigs.yml`) -- **(2)** workflow - **select Run workflow in right side** -- **(3)** Branch - **Select Branch** -- **(4)** Mode: `apply` (required - dry-run will fail due to namespace dependencies) +- **(2)** Click **Run workflow** button in the top right corner +- **(3)** **Branch**: Select your deployment branch (e.g., `develop`) +- **(4)** **Choose MOSIP platform profile**: `mosip-platform-java11` (or other appropriate profile) +- **(5)** **Choose Helmsman mode**: `apply` (dry-run will fail due to namespace dependencies) +- **(6)** **Domain name for this environment**: Enter the domain name (e.g., `example.xyz.net`) +- **(7)** **Environment name**: Enter the environment name (e.g., `sandbox`, `dev`, `staging`) +- **(8)** **Slack channel name for alerting** (optional): e.g., `#mosip-alerts` +- **(9)** **Slack webhook URL for alerting** (optional) +- **(10)** Click **Run workflow** green button **Post-Deployment Steps:** diff --git a/docs/ENVIRONMENT_DESTRUCTION_GUIDE.md b/docs/ENVIRONMENT_DESTRUCTION_GUIDE.md index 59419a0d..0b426820 100644 --- a/docs/ENVIRONMENT_DESTRUCTION_GUIDE.md +++ b/docs/ENVIRONMENT_DESTRUCTION_GUIDE.md @@ -45,6 +45,7 @@ Parameters: ├─ Branch: release-0.1.0 (or your deployment branch) ├─ Cloud Provider: aws ├─ Component: infra +├─ Profile: mosip/esignet └─ Backend: local (or s3, match your deployment configuration) ``` @@ -171,7 +172,8 @@ kubectl exec -n keycloak keycloak-0 -- /opt/jboss/keycloak/bin/standalone.sh \ ```bash # Export Terraform state (if using local backend) cd terraform/implementations/aws/infra/ -tar -czf terraform-state-backup-$(date +%Y%m%d).tar.gz terraform.tfstate* +# For profile-isolated deployments, check the profiles/ directory: +tar -czf terraform-state-backup-$(date +%Y%m%d).tar.gz profiles//terraform.tfstate* # Export Kubernetes configurations kubectl get all --all-namespaces -o yaml > k8s-resources-backup.yaml @@ -406,6 +408,7 @@ kubectl get namespaces | grep -E "cattle|istio" - Branch: release-0.1.0 (your deployment branch) - Cloud Provider: aws - Component: infra + - Profile: mosip/esignet - Backend: local (or s3, match your deployment) ``` @@ -664,6 +667,7 @@ aws ec2 describe-volumes \ ```bash # Remove local Terraform state (if using local backend and no longer needed) rm -rf terraform/implementations/aws/*/terraform.tfstate* +rm -rf terraform/implementations/aws/*/profiles/*/terraform.tfstate* rm -rf terraform/implementations/aws/*/.terraform/ # Remove backed-up kubeconfig diff --git a/docs/SECRET_GENERATION_GUIDE.md b/docs/SECRET_GENERATION_GUIDE.md index f7c5d7ee..d0a6a5c1 100644 --- a/docs/SECRET_GENERATION_GUIDE.md +++ b/docs/SECRET_GENERATION_GUIDE.md @@ -7,10 +7,11 @@ This guide provides step-by-step instructions for generating all required secret 1. [SSH Keys](#1-ssh-keys) 2. [GPG Passphrase](#2-gpg-passphrase) 3. [AWS Credentials](#3-aws-credentials) -4. [WireGuard VPN Configuration](#4-wireguard-vpn-configuration) -5. [Kubernetes Config (KUBECONFIG)](#5-kubernetes-config-kubeconfig) -6. [reCAPTCHA Keys](#6-recaptcha-keys) -7. [How to Add Secrets to GitHub](#how-to-add-secrets-to-github) +4. [GitHub Personal Access Token (GH_INFRA_PAT)](#4-github-personal-access-token-gh_infra_pat) +5. [WireGuard VPN Configuration](#5-wireguard-vpn-configuration) +6. [Kubernetes Config (KUBECONFIG)](#6-kubernetes-config-kubeconfig) +7. [reCAPTCHA Keys](#7-recaptcha-keys) +8. [How to Add Secrets to GitHub](#8-how-to-add-secrets-to-github) --- @@ -277,7 +278,60 @@ Add as **Repository Secrets** in GitHub: --- -## 4. WireGuard VPN Configuration +## 4. GitHub Personal Access Token (GH_INFRA_PAT) + +### What is it? +A Fine-grained Personal Access Token that allows automated processes to interact with your GitHub repository. + +### Why do you need it? +- Required for repository operations during deployment +- Allows workflow to configure environments and variables +- If not provided or configured incorrectly, deployment will fail with a 403 error on push. + +### How to Generate GH_INFRA_PAT + +1. **Go to GitHub Settings** + - Click your profile picture (top right) → Settings + - Scroll down to bottom left and click **Developer settings** + - Click **Personal access tokens** → **Fine-grained tokens** + - Click **Generate new token** + +2. **Configure Token Settings** + - **Token name**: `GH_INFRA_PAT` (or similar) + - **Expiration**: Set as needed (e.g., 90 days) + - **Resource owner**: Select your organization or admin user(e.g., `mosip`) + - **Repository access**: Select **Only select repositories** and choose your infra repository (e.g., `mosip/infra`) + +3. **Set Permissions** + Set exactly these permissions: + - Click on Add permissions button under Permissions section + - **Contents**: Read and write *(critical, Read-only causes 403 on push)* + - **Metadata**: Read-Only + - **Actions**: Read and write + - **Environments**: Read and write + - **Variables**: Read and write + + *NOTE: No Secrets permission needed (intentionally excluded)* + +4. **Generate and Save** + - Click **Generate token** + - Copy the token immediately + - Save it securely in your password manager + +### Where to Use It +Add as **Repository Secret** in GitHub: +- **Name**: `GH_INFRA_PAT` +- **Value**: `github_pat_...` (your generated token) + +### Common Pitfalls +- ❌ Choosing "Read-only" for Contents (causes 403 error on push) +- ❌ Adding Secrets permission (not needed, violates least-privilege) +- ❌ Selecting "All repositories" instead of "Only select repositories" +- ❌ Not saving the token after generating + +--- + +## 5. WireGuard VPN Configuration ### What is it? WireGuard is a modern VPN that creates secure connections to your private infrastructure. @@ -367,7 +421,7 @@ Add as **Environment Secrets** in GitHub (not repository secrets): --- -## 5. Kubernetes Config (KUBECONFIG) +## 6. Kubernetes Config (KUBECONFIG) ### What is it? KUBECONFIG is a configuration file that contains credentials and connection details for your Kubernetes cluster. @@ -471,7 +525,7 @@ users: --- -## 6. reCAPTCHA Keys +## 7. reCAPTCHA Keys ### What is it? reCAPTCHA is Google's service that protects websites from bots and spam by verifying users are human. @@ -557,7 +611,7 @@ hooks: --- -## 7. How to Add Secrets to GitHub +## 8. How to Add Secrets to GitHub ### Understanding Secret Types @@ -591,6 +645,7 @@ hooks: - `GPG_PASSPHRASE` - `AWS_ACCESS_KEY_ID` - `AWS_SECRET_ACCESS_KEY` + - `GH_INFRA_PAT` - `mosip-aws` (or your SSH key name) ### Step-by-Step: Adding Environment Secrets @@ -626,6 +681,7 @@ Repository Structure: │ ├── GPG_PASSPHRASE │ ├── AWS_ACCESS_KEY_ID │ ├── AWS_SECRET_ACCESS_KEY +│ ├── GH_INFRA_PAT │ └── mosip-aws (SSH private key) │ └── Environments @@ -662,6 +718,7 @@ Use this checklist to ensure you've generated and configured all required secret - [ ] GPG Passphrase generated and added - [ ] AWS Access Key ID obtained and added - [ ] AWS Secret Access Key obtained and added +- [ ] GitHub PAT (GH_INFRA_PAT) generated with correct permissions and added ### Infrastructure Deployment - [ ] Terraform base-infra deployed successfully diff --git a/docs/TERRAFORM_WORKFLOW_GUIDE.md b/docs/TERRAFORM_WORKFLOW_GUIDE.md index 5cef42dc..3ea580c9 100644 --- a/docs/TERRAFORM_WORKFLOW_GUIDE.md +++ b/docs/TERRAFORM_WORKFLOW_GUIDE.md @@ -13,6 +13,9 @@ - **base-infra**: VPC, networking, jump server (deploy FIRST) - **observ-infra**: Rancher management cluster (optional) - **infra**: MOSIP Kubernetes cluster (main deployment) +- **`INFRA_PROFILE`**: Profile name for isolated deployments (e.g., `mosip`, `esignet`, `mosip-dev`) + - Use alphanumeric characters and hyphens only; no slashes or special characters + - Ensures state file separation when deploying different services - **`SSH_PRIVATE_KEY`**: GitHub secret name containing SSH private key for instance access - Must match the `ssh_key_name` in your terraform.tfvars - [How to create SSH keys](SECRET_GENERATION_GUIDE.md#1-ssh-keys) diff --git a/docs/_images/ARCHITECTURE_DIAGRAMS.md b/docs/_images/ARCHITECTURE_DIAGRAMS.md index 76f8f81d..df08d510 100644 --- a/docs/_images/ARCHITECTURE_DIAGRAMS.md +++ b/docs/_images/ARCHITECTURE_DIAGRAMS.md @@ -168,13 +168,13 @@ Production (main branch): ├── mosip-terraform-bucket-main/ │ ├── aws-base-infra-main-terraform.tfstate │ ├── aws-observ-infra-main-terraform.tfstate -│ ├── aws-infra-main-terraform.tfstate +│ ├── aws-infra--main-terraform.tfstate │ ├── azure-base-infra-main-terraform.tfstate │ ├── azure-observ-infra-main-terraform.tfstate -│ ├── azure-infra-main-terraform.tfstate +│ ├── azure-infra--main-terraform.tfstate │ ├── gcp-base-infra-main-terraform.tfstate │ ├── gcp-observ-infra-main-terraform.tfstate -│ └── gcp-infra-main-terraform.tfstate +│ └── gcp-infra--main-terraform.tfstate Staging (staging branch): ├── mosip-terraform-bucket-staging/ @@ -199,7 +199,7 @@ Azure: Storage Account + Container Isolation GCP: Google Cloud Storage + Versioning ├── Bucket: mosip-terraform-bucket-{branch} -├── Objects: {cloud}-{component}-{branch}-terraform.tfstate +├── Objects: {cloud}-{component}-{profile}-{branch}-terraform.tfstate └── Versioning: Object versioning enabled ``` diff --git a/docs/_images/esignet.png b/docs/_images/esignet.png index 359747a3..eef7513e 100644 Binary files a/docs/_images/esignet.png and b/docs/_images/esignet.png differ diff --git a/docs/_images/helmsman-external-services.png b/docs/_images/helmsman-external-services.png index f546e228..1b8f013b 100644 Binary files a/docs/_images/helmsman-external-services.png and b/docs/_images/helmsman-external-services.png differ diff --git a/docs/_images/helmsman-testrigs.png b/docs/_images/helmsman-testrigs.png index 10bec4b4..a5c0e490 100644 Binary files a/docs/_images/helmsman-testrigs.png and b/docs/_images/helmsman-testrigs.png differ diff --git a/docs/_images/infra-terraform-apply.png b/docs/_images/infra-terraform-apply.png index 7950e437..27717489 100644 Binary files a/docs/_images/infra-terraform-apply.png and b/docs/_images/infra-terraform-apply.png differ diff --git a/docs/_images/infra-terraform-destroy.png b/docs/_images/infra-terraform-destroy.png index b5d2adfb..287a7818 100644 Binary files a/docs/_images/infra-terraform-destroy.png and b/docs/_images/infra-terraform-destroy.png differ diff --git a/terraform/README.md b/terraform/README.md index 0a5f1cf6..86a24fc3 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -79,7 +79,7 @@ graph TD - **PostgreSQL Integration**: External PostgreSQL 15 setup via Terraform + Ansible (configurable) - **Workloads**: Authentication, Registration, Partner Management, etc. - **Lifecycle**: Can be destroyed and recreated as needed -- **State File**: `{cloud}-infra-terraform.tfstate` +- **State File**: `profiles//{cloud}-infra--terraform.tfstate` ### Observation Infrastructure (observ-infra) @@ -239,6 +239,7 @@ terraform/ │ │ ├── aws-resource-creation/ # VPC, subnets, security groups, EC2 instances │ │ ├── nginx-setup/ # Load balancer and reverse proxy configuration │ │ ├── postgresql-setup/ # PostgreSQL database setup and configuration +│ │ ├── activemq-setup/ # Activemq setup │ │ ├── rke2-cluster/ # RKE2 Kubernetes cluster provisioning │ │ ├── rancher-keycloak-setup/ # Identity management and SSO setup │ │ └── nfs-setup/ # Network File System configuration @@ -288,18 +289,22 @@ Encrypted State Management =========================== Local State Files (Encrypted with GPG): -├── .terraform-state/ -│ ├── aws-base-infra-testgrid-terraform.tfstate.gpg ← Encrypted -│ ├── aws-infra-testgrid-terraform.tfstate.gpg ← Encrypted -│ └── aws-observ-infra-testgrid-terraform.tfstate.gpg ← Encrypted +├── terraform/implementations/aws/base-infra/ +│ └── aws-base-infra--terraform.tfstate.gpg ← Encrypted +├── terraform/implementations/aws/infra/profiles// +│ └── aws-infra---terraform.tfstate.gpg ← Encrypted (Profile-isolated) +└── terraform/implementations/aws/observ-infra/ + └── aws-observ-infra--terraform.tfstate.gpg ← Encrypted Temporary Decrypted Files (During Operations): -├── terraform/base-infra/ -│ └── aws-base-infra-testgrid-terraform.tfstate ← Temporary -├── terraform/infra/ -│ └── aws-infra-testgrid-terraform.tfstate ← Temporary -└── terraform/observ-infra/ - └── aws-observ-infra-testgrid-terraform.tfstate ← Temporary +├── terraform/implementations/aws/base-infra/ +│ └── aws-base-infra--terraform.tfstate ← Temporary +├── terraform/implementations/aws/infra/profiles// +│ └── aws-infra---terraform.tfstate ← Temporary +└── terraform/implementations/aws/observ-infra/ + └── aws-observ-infra--terraform.tfstate ← Temporary + +where = mosip/esignet ``` ### GPG Encryption Benefits @@ -313,14 +318,16 @@ Temporary Decrypted Files (During Operations): ### Custom State File Naming -State files use descriptive naming pattern: `{provider}-{component}-{branch}-terraform.tfstate` +State files use descriptive naming pattern: `{provider}-{component}-[{profile}]-{branch}-terraform.tfstate` ``` Examples: ├── aws-base-infra-main-terraform.tfstate.gpg # Production base infrastructure -├── aws-infra-staging-terraform.tfstate.gpg # Staging MOSIP cluster +├── profiles//aws-infra--staging-terraform.tfstate.gpg # Staging MOSIP cluster ├── aws-observ-infra-development-terraform.tfstate.gpg # Dev monitoring cluster -└── azure-infra-testgrid-terraform.tfstate.gpg # Test environment +└── profiles//azure-infra---terraform.tfstate.gpg # Test environment + +where = mosip/esignet ``` ### Isolated State Files @@ -332,19 +339,21 @@ Encrypted State File Isolation Structure ========================================= AWS Encrypted States (.gpg files): -├── aws-base-infra-{branch}-terraform.tfstate.gpg (VPC, Jumpserver, WireGuard) -├── aws-infra-{branch}-terraform.tfstate.gpg (K8s for MOSIP Core/External) -└── aws-observ-infra-{branch}-terraform.tfstate.gpg (K8s for Rancher UI, Keycloak) +├── base-infra/aws-base-infra-{branch}-terraform.tfstate.gpg (VPC, Jumpserver, WireGuard) +├── infra/profiles//aws-infra--{branch}-terraform.tfstate.gpg (K8s for Profile) +└── observ-infra/aws-observ-infra-{branch}-terraform.tfstate.gpg (K8s for Rancher UI, Keycloak) Azure Encrypted States (.gpg files): -├── azure-base-infra-{branch}-terraform.tfstate.gpg (VNet, Jumpserver, WireGuard) -├── azure-infra-{branch}-terraform.tfstate.gpg (RKE2 for MOSIP Core/External) -└── azure-observ-infra-{branch}-terraform.tfstate.gpg (RKE2 for Rancher UI, Keycloak) +├── base-infra/azure-base-infra-{branch}-terraform.tfstate.gpg (VNet, Jumpserver, WireGuard) +├── infra/profiles//azure-infra--{branch}-terraform.tfstate.gpg (RKE2 for Profile) +└── observ-infra/azure-observ-infra-{branch}-terraform.tfstate.gpg (RKE2 for Rancher UI, Keycloak) GCP Encrypted States (.gpg files): -├── gcp-base-infra-{branch}-terraform.tfstate.gpg (VPC, Jumpserver, WireGuard) -├── gcp-infra-{branch}-terraform.tfstate.gpg (RKE2 for MOSIP Core/External) -└── gcp-observ-infra-{branch}-terraform.tfstate.gpg (RKE2 for Rancher UI, Keycloak) +├── base-infra/gcp-base-infra-{branch}-terraform.tfstate.gpg (VPC, Jumpserver, WireGuard) +├── infra/profiles//gcp-infra--{branch}-terraform.tfstate.gpg (RKE2 for Profile) +└── observ-infra/gcp-observ-infra-{branch}-terraform.tfstate.gpg (RKE2 for Rancher UI, Keycloak) + +where = mosip/esignet ``` ### Benefits of GPG Encrypted Isolated States