Support for configuring Hyperdisk provisioned Throughput and IOPS

[FearlessHyena]

TL;DR

Since Hyperdisks allow you to specify the Throughput and IOPS that should be provisioned when creating them, it would be great if the node_pools property added support for them

Terraform Resources

Detailed design

Two new variables under the `node_pools` property would be needed which would only be used in case of Hyperdisks and ignored for Persistent disks. The [google_compute_disk resource] (https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_disk) can be used for reference
- disk_provisioned_iops
- disk_provisioned_throughput

There is also the [enable_confidential_compute](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_disk#enable_confidential_compute-1) property but not sure if it would be in scope of this feature request

Additional information

No response

[DrFaust92]

https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/hyperdisk#create according to docs its not directly related to the GKE config and needs to be done by creating a relevant storage class and setting it on relevant PVC:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: balanced-storage
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
  type: hyperdisk-balanced
  provisioned-throughput-on-create: "250Mi"
  provisioned-iops-on-create: "7000"

and

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: podpvc
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: balanced-storage
  resources:
    requests:
      storage: 20Gi

and using that for your workloads
the above may differ based on your config.

as this is not a module issue I think it cant be closed

[FearlessHyena]

Certain newer GCP instance types like the C4/C4D/C4A only support Hyperdisk volumes so when you create a nodepool with any of those instance types, the disk_type needs to be set to a hyperdisk variant like hyperdisk-balanced

I believe you don't need a PVC if you plan to use the Hyperdisk as ephemeral storage, as GKE automatically reserves part of the boot disk to serve as ephemeral storage for Pods
Unfortunately unless the Throughput/IOPS are explicitly specified, the default values are used which isn't ideal for many production environments

[marinflorin]

Hey @FearlessHyena, it looks like the documentation needs to be updated, but there is no validation against the disk_type. You can go ahead and use hyperdisk-balanced. I gave it a try, and it worked. Also, below, you can find an example that is using hyperdisk-balanced.

https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/blob/main/examples/confidential_safer_cluster/main.tf#L102

[voron]

There is no such option in GKE node pool API support to configure disk IOPS/throughput for the node's main/boot disk. Thus I don't expect this feature to appear anytime soon.
Possible work-around is to use daemon set on the corresponding node pool to decrease provisioned IOPS/throughput to free values in runtime.