Skip to content

Commit 8470f91

Browse files
florianowflorianow
committed
feat: adding private Clusters
1 parent 7bea02b commit 8470f91

File tree

8 files changed

+242
-11
lines changed

8 files changed

+242
-11
lines changed

‎modules/azure/aks/backplane/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@ No modules.
2424
| Name | Type |
2525
|------|------|
2626
| [azurerm_role_assignment.buildingblock_deploy](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment) | resource |
27+
| [azurerm_role_assignment.buildingblock_deploy_hub](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment) | resource |
2728
| [azurerm_role_definition.buildingblock_deploy](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_definition) | resource |
29+
| [azurerm_role_definition.buildingblock_deploy_hub](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_definition) | resource |
2830

2931
## Inputs
3032

@@ -39,6 +41,10 @@ No modules.
3941
| Name | Description |
4042
|------|-------------|
4143
| <a name="output_documentation_md"></a> [documentation\_md](#output\_documentation\_md) | Markdown documentation with information about the AKS Building Block building block backplane |
44+
| <a name="output_hub_role_assignment_ids"></a> [hub\_role\_assignment\_ids](#output\_hub\_role\_assignment\_ids) | The IDs of the hub role assignments for the service principals. |
45+
| <a name="output_hub_role_assignment_principal_ids"></a> [hub\_role\_assignment\_principal\_ids](#output\_hub\_role\_assignment\_principal\_ids) | The principal IDs of the service principals that have been assigned the hub role. |
46+
| <a name="output_hub_role_definition_id"></a> [hub\_role\_definition\_id](#output\_hub\_role\_definition\_id) | The ID of the role definition that enables deployment of the building block to the hub. |
47+
| <a name="output_hub_role_definition_name"></a> [hub\_role\_definition\_name](#output\_hub\_role\_definition\_name) | The name of the role definition that enables deployment of the building block to the hub. |
4248
| <a name="output_role_assignment_ids"></a> [role\_assignment\_ids](#output\_role\_assignment\_ids) | The IDs of the role assignments for the service principals. |
4349
| <a name="output_role_assignment_principal_ids"></a> [role\_assignment\_principal\_ids](#output\_role\_assignment\_principal\_ids) | The principal IDs of the service principals that have been assigned the role. |
4450
| <a name="output_role_definition_id"></a> [role\_definition\_id](#output\_role\_definition\_id) | The ID of the role definition that enables deployment of the building block to subscriptions. |

‎modules/azure/aks/backplane/main.tf‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ resource "azurerm_role_definition" "buildingblock_deploy" {
2323
"Microsoft.Network/virtualNetworks/subnets/write",
2424
"Microsoft.Network/virtualNetworks/subnets/delete",
2525
"Microsoft.Network/virtualNetworks/subnets/join/action",
26+
"Microsoft.Network/virtualNetworks/virtualNetworkPeerings/read",
27+
"Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write",
28+
"Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete",
29+
"Microsoft.Network/virtualNetworks/peer/action",
2630
"Microsoft.Network/networkInterfaces/read",
2731
"Microsoft.Network/networkSecurityGroups/read",
2832
"Microsoft.Network/networkSecurityGroups/write",
@@ -33,6 +37,12 @@ resource "azurerm_role_definition" "buildingblock_deploy" {
3337
"Microsoft.Network/loadBalancers/read",
3438
"Microsoft.Network/loadBalancers/write",
3539
"Microsoft.Network/loadBalancers/delete",
40+
"Microsoft.Network/privateDnsZones/read",
41+
"Microsoft.Network/privateDnsZones/write",
42+
"Microsoft.Network/privateDnsZones/delete",
43+
"Microsoft.Network/privateDnsZones/virtualNetworkLinks/read",
44+
"Microsoft.Network/privateDnsZones/virtualNetworkLinks/write",
45+
"Microsoft.Network/privateDnsZones/virtualNetworkLinks/delete",
3646
"Microsoft.Resources/deployments/read",
3747
"Microsoft.Resources/deployments/write",
3848
"Microsoft.Resources/deployments/delete",
@@ -57,3 +67,29 @@ resource "azurerm_role_assignment" "buildingblock_deploy" {
5767
principal_id = each.value
5868
scope = var.scope
5969
}
70+
71+
resource "azurerm_role_definition" "buildingblock_deploy_hub" {
72+
name = "${var.name}-deploy-hub"
73+
description = "Enables deployment of the ${var.name} building block to the hub (for private cluster peering)"
74+
scope = var.scope
75+
76+
permissions {
77+
actions = [
78+
"Microsoft.Resources/subscriptions/resourceGroups/read",
79+
"Microsoft.Network/virtualNetworks/read",
80+
"Microsoft.Network/virtualNetworks/virtualNetworkPeerings/read",
81+
"Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write",
82+
"Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete",
83+
"Microsoft.Network/virtualNetworks/peer/action",
84+
]
85+
}
86+
}
87+
88+
resource "azurerm_role_assignment" "buildingblock_deploy_hub" {
89+
for_each = var.principal_ids
90+
91+
role_definition_id = azurerm_role_definition.buildingblock_deploy_hub.role_definition_resource_id
92+
description = azurerm_role_definition.buildingblock_deploy_hub.description
93+
principal_id = each.value
94+
scope = var.scope
95+
}

‎modules/azure/aks/backplane/outputs.tf‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,23 @@ output "scope" {
2222
value = var.scope
2323
description = "The scope where the role definition and role assignments are applied."
2424
}
25+
26+
output "hub_role_definition_id" {
27+
value = azurerm_role_definition.buildingblock_deploy_hub.id
28+
description = "The ID of the role definition that enables deployment of the building block to the hub."
29+
}
30+
31+
output "hub_role_definition_name" {
32+
value = azurerm_role_definition.buildingblock_deploy_hub.name
33+
description = "The name of the role definition that enables deployment of the building block to the hub."
34+
}
35+
36+
output "hub_role_assignment_ids" {
37+
value = { for id in var.principal_ids : id => azurerm_role_assignment.buildingblock_deploy_hub[id].id }
38+
description = "The IDs of the hub role assignments for the service principals."
39+
}
40+
41+
output "hub_role_assignment_principal_ids" {
42+
value = { for id in var.principal_ids : id => azurerm_role_assignment.buildingblock_deploy_hub[id].principal_id }
43+
description = "The principal IDs of the service principals that have been assigned the hub role."
44+
}

‎modules/azure/aks/buildingblock/APP_TEAM_README.md‎

Lines changed: 84 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Azure Kubernetes Service (AKS)
22

33
## Description
4-
This building block provides a production-grade Azure Kubernetes Service (AKS) cluster with integrated security, monitoring, and networking features. It delivers a fully managed Kubernetes environment with Azure AD authentication, workload identity support, and comprehensive observability through Log Analytics.
4+
This building block provides a production-grade Azure Kubernetes Service (AKS) cluster with integrated security, monitoring, and networking features. It delivers a fully managed Kubernetes environment with Azure AD authentication, workload identity support, and comprehensive observability through Log Analytics. The cluster supports both public and private deployment scenarios with optional hub-and-spoke network connectivity.
55

66
## Usage Motivation
77
This building block is for application teams that need to deploy containerized applications on a secure, scalable, and managed Kubernetes platform. The AKS cluster comes pre-configured with enterprise-grade security features, eliminating the operational complexity of managing Kubernetes infrastructure while maintaining the flexibility to run any containerized workload.
@@ -21,6 +21,7 @@ This building block is for application teams that need to deploy containerized a
2121
| Configuring Azure AD authentication and RBAC | ✅ | ❌ |
2222
| Setting up Log Analytics and monitoring infrastructure | ✅ | ❌ |
2323
| Managing virtual network and subnet configuration | ✅ | ❌ |
24+
| Managing hub network peering (for private clusters) | ✅ | ❌ |
2425
| Deploying and managing applications and workloads | ❌ | ✅ |
2526
| Configuring application-level resource limits and quotas | ❌ | ✅ |
2627
| Managing Kubernetes namespaces and RBAC within the cluster | ❌ | ✅ |
@@ -56,25 +57,102 @@ This building block is for application teams that need to deploy containerized a
5657
## Cluster Features
5758

5859
### Authentication & Authorization
59-
- **Azure AD Integration**: Cluster uses Azure AD for authentication, enabling centralized identity management
60+
- **Azure AD Integration**: Cluster uses Azure AD for authentication, enabling centralized identity management (optional)
6061
- **OIDC Issuer**: Workload Identity enabled for secure pod-to-Azure-resource authentication
6162

6263
### Monitoring & Logging
63-
- **Log Analytics Workspace**: Centralized logging for cluster and application logs
64+
- **Log Analytics Workspace**: Centralized logging for cluster and application logs (optional)
6465
- **Container Insights**: Integrated monitoring for container performance and health
6566
- **Diagnostic Settings**: Cluster metrics and logs forwarded to Log Analytics
6667

6768
### Networking
6869
- **Custom VNet**: Dedicated virtual network and subnet for cluster isolation
6970
- **Azure CNI**: Advanced networking capabilities with pod-level networking
71+
- **Private Cluster**: Optional private API server accessible only via private endpoint
72+
- **Hub Connectivity**: Optional VNet peering to central hub network for on-premises connectivity
7073

7174
### Auto-Scaling
72-
- **Cluster Autoscaler**: Automatically adjusts node count based on resource requirements
73-
- **System Node Pool**: Dedicated node pool for system workloads with auto-scaling enabled
75+
- **Cluster Autoscaler**: Automatically adjusts node count based on resource requirements (when enabled)
76+
- **System Node Pool**: Dedicated node pool for system workloads with optional auto-scaling
77+
78+
## Deployment Scenarios
79+
80+
### Public Cluster (Default)
81+
```hcl
82+
module "aks" {
83+
source = "./buildingblock"
84+
85+
aks_cluster_name = "my-public-aks"
86+
resource_group_name = "aks-rg"
87+
location = "West Europe"
88+
aks_admin_group_object_id = "12345678-1234-1234-1234-123456789012"
89+
log_analytics_workspace_name = "my-law"
90+
}
91+
```
92+
93+
### Private Cluster with Hub Connectivity
94+
```hcl
95+
provider "azurerm" {
96+
alias = "hub"
97+
subscription_id = "hub-subscription-id"
98+
# hub credentials
99+
}
100+
101+
module "aks" {
102+
source = "./buildingblock"
103+
104+
providers = {
105+
azurerm = azurerm
106+
azurerm.hub = azurerm.hub
107+
}
108+
109+
aks_cluster_name = "my-private-aks"
110+
resource_group_name = "aks-rg"
111+
location = "West Europe"
112+
113+
# Private cluster settings
114+
private_cluster_enabled = true
115+
private_dns_zone_id = "System"
116+
private_cluster_public_fqdn_enabled = false
117+
118+
# Hub connectivity
119+
hub_subscription_id = "hub-subscription-id"
120+
hub_resource_group_name = "hub-network-rg"
121+
hub_vnet_name = "hub-vnet"
122+
123+
# Azure AD and monitoring
124+
aks_admin_group_object_id = "12345678-1234-1234-1234-123456789012"
125+
log_analytics_workspace_name = "my-law"
126+
}
127+
```
128+
129+
### Private Cluster without Hub (Isolated)
130+
```hcl
131+
module "aks" {
132+
source = "./buildingblock"
133+
134+
aks_cluster_name = "my-isolated-aks"
135+
resource_group_name = "aks-rg"
136+
location = "West Europe"
137+
private_cluster_enabled = true
138+
private_dns_zone_id = "System"
139+
aks_admin_group_object_id = "12345678-1234-1234-1234-123456789012"
140+
log_analytics_workspace_name = "my-law"
141+
}
142+
```
74143

75144
## Getting Started
76145

77-
1. **Access the cluster**: Use `az aks get-credentials` to configure kubectl access
146+
### Public Cluster
147+
1. **Access the cluster**: Use `az aks get-credentials --resource-group <rg> --name <cluster-name>` to configure kubectl access
78148
2. **Verify connectivity**: Run `kubectl get nodes` to confirm cluster connectivity
79149
3. **Deploy your application**: Use `kubectl apply` or Helm to deploy applications
80150
4. **Monitor your workloads**: View logs and metrics in Azure Monitor or Log Analytics
151+
152+
### Private Cluster
153+
1. **Ensure network connectivity**: Access must be from a network peered with the AKS VNet or the hub network
154+
2. **Access the cluster**: Use `az aks get-credentials --resource-group <rg> --name <cluster-name>` from a machine with network access
155+
3. **Use Azure Bastion or VPN**: Connect via Azure Bastion, VPN Gateway, or ExpressRoute for management access
156+
4. **Verify connectivity**: Run `kubectl get nodes` to confirm cluster connectivity
157+
5. **Deploy your application**: Use `kubectl apply` or Helm to deploy applications
158+
6. **Monitor your workloads**: View logs and metrics in Azure Monitor or Log Analytics

‎modules/azure/aks/buildingblock/README.md‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,18 +60,25 @@ No modules.
6060
| [azurerm_resource_group.aks](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/resource_group) | resource |
6161
| [azurerm_subnet.aks_subnet](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/subnet) | resource |
6262
| [azurerm_virtual_network.vnet](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_network) | resource |
63+
| [azurerm_virtual_network_peering.aks_to_hub](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_network_peering) | resource |
64+
| [azurerm_virtual_network_peering.hub_to_aks](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_network_peering) | resource |
6365
| [time_sleep.wait_for_subnet](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
66+
| [azurerm_resource_group.hub_rg](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/resource_group) | data source |
67+
| [azurerm_virtual_network.hub_vnet](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/data-sources/virtual_network) | data source |
6468

6569
## Inputs
6670

6771
| Name | Description | Type | Default | Required |
6872
|------|-------------|------|---------|:--------:|
69-
| <a name="input_aks_admin_group_object_id"></a> [aks\_admin\_group\_object\_id](#input\_aks\_admin\_group\_object\_id) | Object ID of the Azure AD group used for AKS admin access | `string` | n/a | yes |
73+
| <a name="input_aks_admin_group_object_id"></a> [aks\_admin\_group\_object\_id](#input\_aks\_admin\_group\_object\_id) | Object ID of the Azure AD group used for AKS admin access. If null, Azure AD RBAC will not be configured. | `string` | `null` | no |
7074
| <a name="input_aks_cluster_name"></a> [aks\_cluster\_name](#input\_aks\_cluster\_name) | Name of the AKS cluster | `string` | `"prod-aks"` | no |
7175
| <a name="input_dns_prefix"></a> [dns\_prefix](#input\_dns\_prefix) | DNS prefix for the AKS cluster | `string` | `"prodaks"` | no |
7276
| <a name="input_dns_service_ip"></a> [dns\_service\_ip](#input\_dns\_service\_ip) | IP address for Kubernetes DNS service (must be within service\_cidr) | `string` | `"10.0.0.10"` | no |
7377
| <a name="input_enable_auto_scaling"></a> [enable\_auto\_scaling](#input\_enable\_auto\_scaling) | Enable auto-scaling for the default node pool | `bool` | `false` | no |
74-
| <a name="input_kubernetes_version"></a> [kubernetes\_version](#input\_kubernetes\_version) | Kubernetes version for the AKS cluster | `string` | `"1.29.2"` | no |
78+
| <a name="input_hub_resource_group_name"></a> [hub\_resource\_group\_name](#input\_hub\_resource\_group\_name) | Resource group name of the hub virtual network. Required when private\_cluster\_enabled is true and connecting to a hub. | `string` | `null` | no |
79+
| <a name="input_hub_subscription_id"></a> [hub\_subscription\_id](#input\_hub\_subscription\_id) | Subscription ID of the hub network. Required when private\_cluster\_enabled is true and connecting to a hub. | `string` | `null` | no |
80+
| <a name="input_hub_vnet_name"></a> [hub\_vnet\_name](#input\_hub\_vnet\_name) | Name of the hub virtual network to peer with. Required when private\_cluster\_enabled is true and connecting to a hub. | `string` | `null` | no |
81+
| <a name="input_kubernetes_version"></a> [kubernetes\_version](#input\_kubernetes\_version) | Kubernetes version for the AKS cluster | `string` | `"1.33.0"` | no |
7582
| <a name="input_location"></a> [location](#input\_location) | Azure region where resources will be deployed | `string` | `"Germany West Central"` | no |
7683
| <a name="input_log_analytics_workspace_name"></a> [log\_analytics\_workspace\_name](#input\_log\_analytics\_workspace\_name) | Name of the Log Analytics Workspace. If null, no LAW or monitoring will be created. | `string` | `null` | no |
7784
| <a name="input_log_retention_days"></a> [log\_retention\_days](#input\_log\_retention\_days) | Number of days to retain logs in Log Analytics Workspace | `number` | `30` | no |
@@ -81,6 +88,9 @@ No modules.
8188
| <a name="input_network_policy"></a> [network\_policy](#input\_network\_policy) | Network policy to use (azure, calico, or cilium) | `string` | `"azure"` | no |
8289
| <a name="input_node_count"></a> [node\_count](#input\_node\_count) | Initial number of nodes in the default node pool | `number` | `3` | no |
8390
| <a name="input_os_disk_size_gb"></a> [os\_disk\_size\_gb](#input\_os\_disk\_size\_gb) | OS disk size in GB for the node pool | `number` | `100` | no |
91+
| <a name="input_private_cluster_enabled"></a> [private\_cluster\_enabled](#input\_private\_cluster\_enabled) | Enable private cluster (API server only accessible via private endpoint) | `bool` | `false` | no |
92+
| <a name="input_private_cluster_public_fqdn_enabled"></a> [private\_cluster\_public\_fqdn\_enabled](#input\_private\_cluster\_public\_fqdn\_enabled) | Enable public FQDN for private cluster (allows public DNS resolution but API server remains private) | `bool` | `false` | no |
93+
| <a name="input_private_dns_zone_id"></a> [private\_dns\_zone\_id](#input\_private\_dns\_zone\_id) | Private DNS Zone ID for private cluster. Use 'System' for Azure-managed zone, or provide custom zone ID. Only used when private\_cluster\_enabled is true. | `string` | `"System"` | no |
8494
| <a name="input_resource_group_name"></a> [resource\_group\_name](#input\_resource\_group\_name) | Name of the resource group to create for the AKS cluster | `string` | `"aks-prod-rg"` | no |
8595
| <a name="input_service_cidr"></a> [service\_cidr](#input\_service\_cidr) | CIDR for Kubernetes services (must not overlap with VNet or subnet) | `string` | `"10.0.0.0/16"` | no |
8696
| <a name="input_subnet_address_prefix"></a> [subnet\_address\_prefix](#input\_subnet\_address\_prefix) | Address prefix for the AKS subnet | `string` | `"10.240.0.0/20"` | no |

0 commit comments

Comments
 (0)