Allow mutating some Bucket(Access)Class fields after create/grant #119
Description
Enhancement
Is your feature request related to a problem?/Why is this needed
Copied from kubernetes-retired/container-object-storage-interface-api#105
Initial discussion suggested that we can use Volume Attribute Class (or an analogue of it) for COSI
Use for changing vendor-specific quotas after 1st provisioning VAC KEP - kubernetes/enhancements#3751
The initial intent here is to allow updating vendor-specific parameters that might need to be updated after initial creation of a resource based on BClass/BAClass.
For example, the Ceph COSI driver wishes to allow setting bucket size quotas to prevent users from consuming too much of the limited, on-prem cluster storage. Apps might need to have their BC or BA upgraded to increase their quota space as their app grows.
Similarly, admins may wish to apply newly-available driver features to existing B/BC/BA resources.
This may also be important during the evaluation phase of an application. When an application with unique needs is being onboarded and evaluated, Class parameters may need to be modified somewhat frequently. Allowing these modifications without requiring application redeployment could save users time in this scenario.
I [Guy Margalit] see many cases where the class is effectively a "one-off" set of parameters to the backend with some custom parameters, and I wish it was simpler to use for that.
This is a good statement as to what the pain points seem to be with the current COSI API. This is something that weighs on my mind also since I see and get feedback from Rook users about this, and the current OBC system.
Given the importance that the K8s and sig-storage communities have put on portability, I am reluctant to start with the standpoint that portability can be so easily thrown out the window for ease-of-use, but I also don't want to throw this valid concern.
One of the things we have talked about in the COSI meeting recently is potentially having XxClass as well as XxAttributeClasses in a way that mirrors VACs: https://kubernetes.io/docs/concepts/storage/volume-attributes-classes/
I think this is a way to still have portability as well as users who can have more control over their own destiny in a way that is gated by admins.
Another alternative that is more directly to the title of this issue is allowing XxClass parameters to be mutable (potentially even for pre-existing bucket/access resources?). For a trusted user who is trying to get things running preproduction, that could also allow for an easier time getting going with a backend that has many tunables, as I can imagine many backends will eventually have.