diff --git a/content/vault/v1.21.x/content/docs/deploy/run-as-zcx-cluster.mdx b/content/vault/v1.21.x/content/docs/deploy/run-as-zcx-cluster.mdx new file mode 100644 index 0000000000..23be589a58 --- /dev/null +++ b/content/vault/v1.21.x/content/docs/deploy/run-as-zcx-cluster.mdx @@ -0,0 +1,362 @@ +--- +layout: docs +page_title: Run Vault on IBM z/OS Container Extensions +description: >- + Step-by-step guide to deploying a secure 3-node Vault cluster inside IBM zCX using HAProxy and TLS. +--- + +# Run Vault on IBM z/OS Container Extensions (zCX) + +Deploy a fully secured 3-node HashiCorp Vault Enterprise cluster on IBM z/OS +Container Extensions (zCX) with: + +- Three independent zCX instances running Vault on unique IP addressed. +- A Layer-4 HAProxy load balancer to distribute traffic. +- End-to-end TLS encryption to ensure secure communication throughout the cluster. + +![Vault zCX cluster deployment](/img/run-as-zcx-cluster.png) + +## Before you start + +- **Read about [sealing and unsealing](/vault/docs/concepts/seal) Vault**. You + cannot test your deployment without unsealing Vault. + +- **Install the [Docker CLI](https://www.docker.com/products/cli/)**. You must + be able to interact with Docker containers as part of the deploy process. + +- **Check your permissions**. You must have permission to: + - Deploy and update Docker containers. + - Run `vault operator` commands. + - Configure and deploy your load balancer. + +- **You must have your encryption certificates**. To secure communication in the + cluster between Vault clients, HAProxy, and other nodes you must have the + following certificates: + - **`vault.pem`** - the TLS certificate that secures communication between nodes + - **`vault.key`** - the private encryption key used by Vault + - **`ca.pem`** - the certificate authority (CA) used to validate mutual TLS + between nodes +## Step 1: Get the Vault Enterprise image from Docker + +1. Use the Docker CLI to pull official Vault Enterprise container images on all three zCX nodes: + + ```shell-session + $ docker pull hashicorp/vault-enterprise + ``` + +1. Verify image integrity. + + ```shell-session + $ docker images | grep vault-enterprise + ``` + +## Step 2: Create a persistent volume on each node + +Create Docker volumes for configuration and data persistence on each node. + +1. Create volume for Vault data storage including an internal storage backend and space for audit logs: + + ```shell-session + $ docker volume create vault-data + ``` + +1. Create a separate volume for Vault configuration files: + + ```shell-session + $ docker volume create vault-config + ``` + +1. Verify volume creation for both volumes: + + ```shell-session + $ docker volume ls | grep vault + ``` + +You now have two explicit volume mount points for Vault: + +Volume name | Mount path | Stores +-------------- | ----------------- | ------ +`vault-data` | `/vault/data` | Internal storage snapshots, encrypted storage backend +`vault-config` | `/vault/config.d` | Vault configuration files + +## Step 3: Create your Vault configuration files + +Create individual Vault configuration files for the leader node and follower +nodes with node-specific parameters. + + +1. Create a configuration file, `vault-leader.hcl` for Vault leader nodes. For + example, the following configuration file exposes Vault on port 8200 for API + and UI traffic and sets `tls_disable` to `0` to force TLS communication + within the cluster: + + ```hcl + ui = true + disable_mlock = true + + # --- Listener Configuration --- + listener "tcp" { + address = "0.0.0.0:8200" + tls_disable = 0 + tls_cert_file = "/vault/vault.pem" + tls_key_file = "/vault/vault.key" + tls_client_ca_file = "/vault/ca.pem" + } + + # --- Raft Storage (Leader Node) --- + storage "raft" { + path = "/vault/data" + node_id = "" # e.g., "node1" + + # No retry_join block needed for leader + } + + # --- Cluster Networking --- + api_addr = "https://:8200" + cluster_addr = "https://:8201" + ``` + + + +1. Create a configuration file, `vault-follower.hcl` for Vault follower nodes. + For example, the following configuration file exposes Vault on port 8200 for + API and UI traffic, sets `tls_disable` to `0` to force TLS communication + within the cluster, and sets the leader information so that followers know + how to join the cluster: + +```hcl +ui = true +disable_mlock = true + +# --- Listener Configuration --- +listener "tcp" { + address = "0.0.0.0:8200" + tls_disable = 0 + tls_cert_file = "/vault/vault.pem" + tls_key_file = "/vault/vault.key" + tls_client_ca_file = "/vault/ca.pem" +} + +# --- Raft Storage (Follower Node) --- +storage "raft" { + path = "/vault/data" + node_id = "" # e.g., "node2" or "node3" + + retry_join { + leader_api_addr = "https://:8200" + leader_ca_cert_file = "/vault/ca.pem" + } +} + +# --- Cluster Networking --- +api_addr = "https://:8200" +cluster_addr = "https://:8201" +``` + +## Step 4: Push the configuration files and certificates + +1. Use the Docker CLI to access the `vault-config` volume: + + ```shell-session + $ docker run --rm -it -v vault-config:/vault alpine sh + ``` + +1. Add the leader configuration files to the volume: + + ```shell-session + $ TBD + ``` + +1. Add the follower configuration files to the volume: + + ```shell-session + $ TBD + ``` + +1. Use the Docker CLI to access the `vault-data` volume: + + ```shell-session + $ docker run --rm -it -v vault-data:/vault alpine sh + ``` + +1. 2. Add the certificate files to the volume: + ```shell-session + $ TBD + ``` +## Step 5: Deploy the Vault container + +Deploy the Vault container on each zCX node. + +1. Set your Vault Enterprise license (export before running). + + ```shell-session + $ export VAULT_LICENSE="02MV4UU43BK5HGYYTOJZWFQMTMNNEWU33JJVVGC..." + ``` + +1. Deploying the Vault Enterprise container on zCX. + + ```shell-session + $ docker run -d \ + --name vault-zcx \ + --cap-add IPC_LOCK \ + -p 8200:8200 \ + -p 8201:8201 \ + -v vault-config:/vault/config.d \ + -v vault-data:/vault/data \ + -e VAULT_LICENSE="$VAULT_LICENSE" \ + hashicorp/vault-enterprise:1.19.12-ent \ + server -config=/vault/config.d/ + ``` + +1. Check the deployment by verifying the container status: + + ```shell-session + $ docker ps | grep vault-zcx + ``` + + 1. Check the container log. + + ```shell-session + $ docker logs vault-zcx + ``` + +## Step 6: Initialize and unseal the leader node + + + +Perform the initialization only on the leader node. + + + +1. Use Docker to call the Vault CLI so you can initialize the leader node and + save the unseal keys to a JSON file, `vault-init.json`: + + ```shell-session + $ docker exec -it vault-zcx vault operator init \ + -format=json > /secure/location/vault-init.json + ``` + + By default, `vault operator init` generates five unseal keys and configures + Vault to require three keys as the unseal threshold. + +1. Extract unseal keys and root token. + + ```shell-session + $ cat /secure/location/vault-init.json + ``` + +1. Unseal Vault on all nodes. + + Vault requires a certain threshold of shares to reconstruct the unseal key. Unseal Vault with 3 different unseal keys (repeat on each node). + + ```shell-session + $ docker exec -it vault-zcx vault operator unseal + $ docker exec -it vault-zcx vault operator unseal + $ docker exec -it vault-zcx vault operator unseal + ``` + +1. Remember to unseal with same key that you got from leader. Check the seal status. + + ```shell-session + $ vault status + ``` + +## Step 7: Configure HAProxy load balancer + +In a Vault cluster, the load balancer plays a critical role in directing traffic to the active leader +while maintaining full security. This example uses an HAProxy Layer-4 (TCP) load balancer, +but you can use other load balancers as well depending on your environment and requirements. + +- End-to-end TLS encryption: L4 simply forwards TCP connections, so Vault’s TLS traffic remains fully encrypted without terminating SSL at the load balancer. + +- Simpler configuration: No need to manage certificates on HAProxy or re-encrypt traffic. + +- Leader-aware routing: Vault handles leader redirects natively, so the load balancer doesn’t need to interpret HTTP or Vault protocols. + +- Better performance: Forwarding raw TCP reduces overhead, giving lower latency and higher throughput. + +### Configuration + +```plaintext +global + maxconn 4096 + log stdout format raw local0 + +defaults + mode tcp # TCP mode for Layer-4 forwarding + timeout connect 5s + timeout client 1m + timeout server 1m + log global + +# --- Stats Page (HTTPS) --- +frontend stats + bind *:8404 ssl crt /usr/local/etc/haproxy/haproxy.pem + mode http + stats enable + stats uri /stats + stats refresh 10s + stats admin if TRUE + +# --- Vault API Frontend (TLS Passthrough) --- +frontend vault_api + bind *:8200 # Listen on Vault API port + mode tcp # Layer-4 forwarding to preserve TLS + default_backend vault_nodes + +# --- Vault Nodes Backend --- +backend vault_nodes + mode tcp + balance roundrobin # Simple load distribution among nodes + option tcp-check # Health check at TCP level + server node1 :8200 check + server node2 :8200 check + server node3 :8200 check +``` + +1. Copy config to volume. + + ```shell-session + $ docker cp haproxy.cfg haproxy-tmp:/usr/local/etc/haproxy/haproxy.cfg + ``` + +1. Deploy HAProxy load-balancer. + + ```shell-session + $ docker run -d \ + --name vault-lb \ + -p 8300:8200 \ + -p 8404:8404 \ + -v haproxy-config:/usr/local/etc/haproxy \ + ibmz-hc-registry.ngrok.dev/haproxy:3.2 + ``` + +## Verify cluster + +1. List all the leader and follower nodes. + + ```shell-session + $ vault operator raft list-peers + ``` + +1. Test container health on browser using HAProxy endpoint. + + ```plaintext + https://:/stats + ``` + +1. Test secure Vault connection and view Raft configuration. + + ```shell-session + $ curl \ + --cacert \ + --header "X-Vault-Token: " \ + https://:/v1/sys/storage/raft/configuration \ + | jq . + ``` + +## Additional resources + +- [Vault configuration parameters](/vault/docs/configuration) +- [CLI command - `operator init`](/vault/docs/commands/operator/init) +- [CLI command - `operator unseal`](/vault/docs/commands/operator/unseal) \ No newline at end of file diff --git a/content/vault/v1.21.x/data/docs-nav-data.json b/content/vault/v1.21.x/data/docs-nav-data.json index 407bb6f9c3..2edeb5b8a0 100644 --- a/content/vault/v1.21.x/data/docs-nav-data.json +++ b/content/vault/v1.21.x/data/docs-nav-data.json @@ -724,6 +724,15 @@ "title": "Run as a service", "path": "deploy/run-as-service" }, + { + "title": "Run on IBM zCX", + "badge": { + "text": "ENT", + "type": "filled", + "color": "neutral" + }, + "path": "deploy/run-as-zcx-cluster" + }, { "title": "Run on AWS", "routes": [ diff --git a/content/vault/v1.21.x/img/run-as-zcx-cluster.png b/content/vault/v1.21.x/img/run-as-zcx-cluster.png new file mode 100644 index 0000000000..ab118453d2 Binary files /dev/null and b/content/vault/v1.21.x/img/run-as-zcx-cluster.png differ