CI using sample driver

[BlaineEXE]

Enhancement

Why is this needed?:

COSI must have CI.

COSI should have a sample driver that shows vendors how to implement drivers in some reasonably realistic way.

It will be efficient if COSI is able to use its sample driver for CI tests.

For comparison, CSI's hostpath sample driver is used for CI as well.

The current COSI sample driver does not have a storage backend. It merely implements gRPC server calls with no implementation. This is okay but not ideal for actually exemplifying how a driver can interact with a backend.

Describe the solution you'd like in detail:

Ideally, the sample driver will interact with a real storage backend. However, there are challenges. All 3 object protocols currently supported (S3, Azure blob, GCS) are proprietary protocols.

We must have CNCF approval to implement a driver using these protocols.

The ideal sample driver would use S3, since it is the most widely used protocol and since it has the most open source implementations.

In addition, the ideal sample driver would not rely on vendor-specific features and instead rely only on the most minimal S3 APIs shared by any S3 server implementation so that COSI is not acting as a vendor driver maintainer.

If we are using the sample driver to help us with CI, The sample driver would require a backend that is functional and provides basic S3 APIs and features. It must be able to manage buckets, users, and assign permissions to users for bucket access.

There are a number of open source software projects that provide S3 backends/servers, but attention to licensing is important. COSI maintainers have identified these "open source" S3 server providers:

• Ceph RGW - LGPL-2.1 or LGPL-3
• GarageFS - AGPL v3
• SeaweedFS - Apache 2.0 (approved license)
• Storj + S3 adapter - AGPL v3
• MinIO - AGPL v3 and commercial licenses

Of these, only SeaweedFS uses a CNCF-approved license.

In full, the proposal is for 3 layers:

• COSI core: Controller+Sidecar (Apache 2.0)
• COSI sample driver: user-consumable driver (Apache 2.0, using the proprietary S3 protocol)
• S3 provider: used in CI only

Based on the above discussion, the pre-approved S3 provider we could use is SeaweedFS (Apache 2.0). The S3 provider could be swapped out in the future or swapped out for downstream vendors. If SeaweedFS becomes deprecated, swapping might be necessary.

Describe alternatives you've considered:

The current sample driver connects to no backend. If licensing is an issue, especially regarding usage of S3, this is the non-preferred fallback option.

Additional context:

[BlaineEXE]

A colleague pointed out that the SMB CSI driver has some overlap with this proposal: https://github.com/kubernetes-csi/csi-driver-smb

I haven't looked deeply to see what they do for their CI, but my understanding is that they stand up an SMB server to run CI against that is some flavor of GPL. If true, this suggests that COSI can follow their lead in our CI and use a backend service for testing that isn't Apache-2.0-licensed, provided we are not incorporating the code into any delivered binaries.

I think we will need to continue discussing what license issues may be at play if our sample driver implements non-backend-specific S3 support though, since S3's API is a proprietary interface (though with a lot of prior art of inclusion in open source projects).