Feature Request: Templated creation of Sandboxes

[barney-s]

Currently we have Sandbox which allows user/agent to create a sandbox for agent use.
For each sandbox the pod template fields need to be provided to create a sandbox.

We want an ability to provide curated Sandbox templates. An use can then create sandboxes using that template.

At the same time we want the ability to separate usage concerns.
Typically an administrator creates the sandbox templates.
Using a claim model further separates concerns.

template -> claim -> sandbox

template - created by admin.
claim - created by user.
sandbox - created by sandbox controller from the template.

[justinsb]

Is the primary use case curation or is it "pooling" of pods to reduce launch time?

I think if it's curation we have a bunch of solutions to that, like helm / KRO / kubebuilder / kubebuilder-declarative-pattern etc. Historically we (in kubernetes) have embraced a wide variety of solutions here, rather than picking one.

If on the other hand the use case is to pre-launch some Sandboxes, to launch them faster, then I concur that a claim might be useful. We might also be able to support this with just Sandbox.

[tomergee]

Good question, the general idea is to promote 3 things - Fast factory sandbox creation, separation of concerns and security for different class of personas:

1. Administrators will create templates representing different sandbox families (computer use, browser use, "Java-dev"...) including all necessary config for the users to consume in bulk when running thousands of agents. Admins have control of the image, dependencies, config and policies in their perimeter (Also allows Admins to have more granular control managing sandbox families and life cycle)
2. Users consuming only Sandboxes from templates introduces the opportunity to limit cluster object access, also helps to simplify consuming Sandboxes by users that do not have K8s knowledge.
3. Pre-launching templates of sandbox families is definitely a way to tackle any latency

[justinsb]

OK so maybe we should split out the Template use case and the Claim use case? A Template could be solved with helm / KRO / kubebuilder / kubebuilder-declarative-pattern. I can imagine that different users might want to expose different things. I have no objection to our putting an opinionated SandboxTemplate into examples/ to experiment with, but I do think that if people want to use Sandbox but not SandboxTemplate (say because they are using helm), that is OK.

The pre-launching use case I think is very interesting, and maybe we should split that out into its own issue. I don't know whether the right way to express that is via a Claim, or whether we make the Sandbox itself the claim (and have the controller manage a pool of Pods that are not yet associated to Sandboxes?). WDYT?

[janetkuo]

@justinsb Let me provide more context. The original idea was to just have a SandboxTemplate (mostly for UX) and a Sandbox that references a sandbox template by name.

However, this will cause several problems:

1. SandboxTemplate is now tightly coupled with Sandbox (Sandbox has a field that references SandboxTemplate, e.g. spec.templateName or spec.fromTemplate)
2. Users can't see the actual fields of the Sandbox without looking at the SandboxTemplate, whose content is subject to change (do we handle the upgrade case?)
3. If the controller were to copy the fields from SandboxTemplate to Sandbox spec, it violates the contract (controller shouldn't modify user's intent)

I suggested using a SandboxClaim to address these problem. It allows SandboxTemplate and Sandbox to be loosely coupled, and keeps a record of the actual fields being used in the Sandbox. It also allows us to easily remove SandboxTemplate/SandboxClaim altogether if we find this unnecessary from our experiments.

[justinsb]

So a SandboxClaim + a SandboxTemplate creates a Sandbox? I'm fine with that, though I do think that should be separable from the core Sandbox flow, as some people will prefer helm / KRO / their own operators. So as long as we clearly delineate the Sandbox from SandboxClaim / SandboxTemplate, and emphasize that they will be promoted to beta separately and that users don't have to use SandboxClaim / SandboxTemplate , I think that should be fine ... we do want to experiment.

My 2c is that it will either be too specific to be useful to many people, or end up rebuilding KRO/helm etc, which is why I'm wary ... but as you point out, if we find it unnecessary we will remove it ... we are very much in alpha after all!

[janetkuo]

Right, SandboxClaim allows the Sandbox to be completely decoupled from SandboxTemplate so that users can create Sandbox directly. SandboxClaim will be managed by a separate controller from Sandbox.

[aditya-shantanu]

@justinsb @janetkuo @barney-s @tomergee
Hey folks. Checking in. We're good to go with this design ?
any open questions that remain ?

[barney-s]

@aditya-shantanu @justinsb @janetkuo

Updated #8 to reflect our intent of not having the Template and Claim as part of core CRDs of sandbox.
Created an extensions/ folder and added the CRDs there. The crd group had been changed to extensions.agents.x-k8s.io as well.

PTAL.

[barney-s]

CRDs are merged in #8

Need controller.