Apple container

[NN---]

Is it possible to integrate lima with Apple containers ?
https://github.com/apple/container

[afbjorklund]

It might be possible to do something similar to WSL2 with it, but it might be more "plain" and not support systemd and containers for the first iterations... (despite the name, it launches virtual machines and not containers)

Lima launches a regular cloud image for each instance, whereas WSL2 launches one optimized virtual machine for all "distributions" (system containers). Apple Containerization launches one MicroVM* for each "container" (virtual machine), and is not sharing the kernel.

* See also https://firecracker-microvm.github.io/ and https://katacontainers.io/

Note: Apple is using the kata-containers kernel by default, in their documentation.

[afbjorklund]

One could add Docker and Podman variants of the first image, but it would just be the same but with HTTP instead of SSH (or perhaps with HTTP running on top of SSH) - where with Podman you can choose between lima podman and podman-remote.

[jandubois]

I wonder if its more beneficial if we integrate some new VZ functionality into https://github.com/Code-Hex/vz to speed up or make existing Lima VZ VMs more stable, instead of trying to integrate with Apple's from-scratch containerization (Swift) framework?

Agreed. Not even sure it has to go into VZ; maybe trying to create a custom Linux image with the vminitd from Apple, and just enough OS to run containerd or mobyd, and see if that will significantly speed up boot time.

Personally I hope that macOS 26 will have significant improvements to vmnet that will make our lives easier.

[afbjorklund]

If boot time is important, there are also plenty of things that can be done with QEMU and a custom distro*. The closest thing we have now is the Alpine Virt-derivate (https://github.com/lima-vm/alpine-lima) but it is more focused on size than speed...

* Like "Optimizing QEMU boot time"

[afbjorklund]

My experiments with the minikube OS carried over into the Bean OS, which was configurable enough to be repurposed for such experiments. That is, using Buildroot to create a custom distribution... For Linux, one could have a Firecracker driver.

[afbjorklund]

https://github.com/afbjorklund/microvm-buildroot

It boots in under a second, even with the most naive approach (default kernel config, regular qemu setup)
Including support for containers, but excluding systemd (and sharing).

[nirs]

Lima can support new instance type "container" implemented using a virtualization container, so it can manage more workload types. But I'm not use how lima vm can map to much limited virtualization container instance. Also not clear what is the use case.

[jandubois]

Yeah, and the same as the wsl2 driver, it would not support regular images. wsl2 requires a tarball of the filesystem, and containers will need a container image of the OS.

You can run a full OS in a container (I've seen someone run an OpenSUSE container in an Apple container, and then install xeyes using zypper and have it show up on the macOS desktop). But I don't know what the advantage of that is vs. running directly in VZ:

[afbjorklund]

There is not a fundamental difference between tarballs and container images, in that you can do docker import to make one.

But they are both different from the cloud images that are normally used in Lima, that are typically a virtual HD or virtual CD.

[jandubois]

Yeah, my point was that you cannot take a template for qemu or vz and expect it to work with wsl2 by just changing the vmType. You also need to specify different image sources.

Right now we have only a single wsl2.yaml template that actually works on Windows. But if we had a containers driver, you would expect to be able to have a template that would work for all drivers supported on macOS.

We are also lacking a way to specify the image format in the images section right now and would have to rely on the file extension, but that is a different topic.

[afbjorklund]

We could add features for bloatable containers and for "adding a kernel and init to a container image and make it into a vm".

But it does sound like feature creep, and you could just run the container? Like with the nerdbox experiment that I did.

[afbjorklund]

Apparently one can use wsl --import for the same feature, to be able to start virtual machines from container images:

"Import any Linux distribution to use with WSL" https://learn.microsoft.com/en-us/windows/wsl/use-custom-distro

[afbjorklund]

Added as a feature:

• Lima driver for Apple Container virtual machines #3788