Using local-exec to apply Kubernetes manifests is fragile as it depends on kubectl being installed and configured on the machine running Terraform. The README instructs users to configure kubectl after terraform apply, which creates a chicken-and-egg problem. A more robust, Terraform-native approach is to use the http data source to fetch the manifest and the kubernetes_manifest resource to apply it. This ensures the process is self-contained within Terraform's execution.

data "http" "inference_gateway_crds" {
  url = "https://github.com/kubernetes-sigs/gateway-api-inference-extension/releases/download/v1.0.0/manifests.yaml"
}

resource "kubernetes_manifest" "apply_crds" {
  for_each   = { for i, v in split("\n---\n", data.http.inference_gateway_crds.body) : i => v if trimspace(v) != "" }
  manifest   = yamldecode(each.value)
  depends_on = [module.gke]
}

Hi @SinaChavoshi - I would recommend using the kubectl module which handles installation, auth, etc:

module "kubectl" {
  source = "terraform-google-modules/gcloud/google//modules/kubectl-wrapper"

  project_id              = var.project_id
  cluster_name            = var.cluster_name
  cluster_location        = var.cluster_location
  kubectl_create_command  = "kubectl create deploy nginx --image=nginx"
  kubectl_destroy_command = "kubectl delete deploy nginx"
}

Since null_resource.apply_crds is being replaced by kubernetes_manifest.apply_crds, this depends_on needs to be updated to reflect the new resource.

  depends_on = [kubernetes_deployment.vllm, kubernetes_manifest.apply_crds]

The helm provider is used in main.tf but it is not declared in the required_providers block. This will cause terraform init to fail. Please add the helm provider to this block.

  required_providers {
    google = {
      source = "hashicorp/google"
    }
    google-beta = {
      source = "hashicorp/google-beta"
    }
    kubernetes = {
      source = "hashicorp/kubernetes"
    }
    helm = {
      source = "hashicorp/helm"
    }
  }

The vllm deployment is configured with 3 replicas, but the gpupool node pool is configured with only one node (node_count = 1). The a3-highgpu-2g machine type has 2 GPUs, and each pod requests 1 GPU. This means only 2 of the 3 replicas can be scheduled, leaving one pod in a Pending state. To fix this, you should either increase the node_count to 2 in the node pool definition, or reduce the number of replicas to 2 to match the available GPUs on a single node.

    replicas = 2

The temperature parameter in the JSON payload is specified as a string ("0.7"). While some servers might be lenient, the OpenAI API specification (which vLLM aims to be compatible with) expects this to be a number. It's better to provide it as a numeric value for correctness and broader compatibility.

For accelerator-optimized machine types like a3-highgpu-2g, the accelerator type and count are inferred from the machine type. This guest_accelerator block is redundant and can be removed for clarity and to rely on the machine type's implicit configuration.

Using the :latest tag for container images is not recommended as it can lead to unpredictable behavior and makes deployments difficult to reproduce. It's a best practice to pin images to a specific, immutable version tag (e.g., v0.5.1).

Accessing metadata via metadata[0] is unnecessary and less readable. The Terraform Kubernetes provider schema defines metadata as a single block, not a list. You can access its attributes directly. Please use kubernetes_secret.hf_secret.metadata.name instead. This applies to other resource references in this file as well (e.g., lines 299, 341, 345, 355, 368, 380).

                name = kubernetes_secret.hf_secret.metadata.name

Using a floating tag like :main is not recommended for the same reasons as using :latest. It can cause unexpected changes and break reproducibility. Please use a specific release tag or commit SHA if available.

The description for this output mentions GKE Autopilot, but this example provisions a GKE Standard cluster. The description should be updated to be accurate.

  description = "The configuration for addons supported by GKE."