Support for MIG configuration

[fketelaars]

Is your feature request related to a problem? Please describe.
MIG configuration in OpenShift is quite involved, as documented here: https://docs.nvidia.com/datacenter/cloud-native/openshift/latest/mig-ocp.html. There are quite a few decisions that need to be made and not all are applicable to watsonx.

Create an opinionated configuration for MIG and automate the setup.

https://docs.nvidia.com/datacenter/cloud-native/openshift/23.9.2/mig-ocp.html#example-single-mig-strategy

[fketelaars]

Steps to be performed for "single" strategy (https://docs.nvidia.com/datacenter/cloud-native/openshift/23.9.2/mig-ocp.html#example-single-mig-strategy):

NODE_NAME=myworker.redhat.com
STRATEGY=single
MIG_CONFIGURATION=all-3g.40gb

oc patch clusterpolicy/gpu-cluster-policy --type='json' \
    -p='[{"op": "replace", "path": "/spec/mig/strategy", "value": '$STRATEGY'}]'

oc label node $NODE_NAME nvidia.com/mig.config=$MIG_CONFIGURATION --overwrite

Describe to check that GPU resources are available. Reboot of the node may be needed.

oc describe node | egrep "Name:|Roles:|Capacity|nvidia.com/gpu:|nvidia.com/mig-.* |Allocatable:|Requests +Limits"

Also, allow models to be installed with multiple replicas.

apiVersion: watsonxaiifm.cpd.ibm.com/v1beta1
kind: Watsonxaiifm
metadata:
  creationTimestamp: "2024-12-11T16:08:32Z"
  generation: 8
  name: watsonxaiifm-cr
  namespace: cpd
  resourceVersion: "831218"
  uid: a73e9bd8-241a-41ee-8f1a-bf043bdf7dbd
spec:
  blockStorageClass: ocs-storagecluster-ceph-rbd
  docker_registry_prefix: cp.icr.io/cp/cpd
  fileStorageClass: ocs-storagecluster-cephfs
  ibm_granite_13b_instruct_v2_replicas: 3
  ignoreForMaintenance: false
  install_model_list:
  - ibm-granite-13b-instruct-v2
  license:
    accept: true
  size: small
  storageClass: ocs-storagecluster-cephfs
  version: 9.3.0