`minikube cache` config path queries host Docker daemon, adding 10-30s per image on non-Docker drivers

[nirs]

Summary

When images are registered via minikube cache add, they are persisted in
~/.minikube/config/config.json under the cache key. On every subsequent
minikube start, minikube reads this config and runs two phases per image:

1. Save phase — verify/download the image tarball to ~/.minikube/cache/images/
2. Load phase — transfer the tarball to the guest and load it

Both phases attempt to query the host's Docker daemon for image digest
optimization. On non-Docker drivers (vfkit, hyperkit, QEMU, KVM), no host Docker
daemon exists. Each query blocks for a full 10-second timeout before falling back.

This adds 10-30 seconds per cached image to every minikube start.

Evidence

With cache config entry (image file already exists on disk)

% minikube config view
- cache: map[nginx:latest:<nil>]

% timestamp minikube start -d vfkit --log_file start-after.log
[   14.560]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5
[   14.928] * Enabled addons: default-storageclass, storage-provisioner
[   27.009] * Done!    ← 27s total

Log shows the save phase is instant (file exists), but the load phase hits the
10-second timeout:

cache.go:96]  cache image "nginx:latest" -> ".../nginx_latest" took 80.458µs
cache.go:80]  save to tar file nginx:latest -> .../nginx_latest succeeded

cache_images.go:90]  LoadCachedImages start: [nginx:latest]
cache_images.go:117] "nginx:latest" needs transfer: needs transfer timed out in 10.000000 seconds
image.go:80]          couldn't find image digest nginx:latest from local daemon:
                      Get "http://root%40docker.local/v1.51/images/nginx:latest/json": context deadline exceeded
cache_images.go:94]  duration metric: took 12.036926292s to LoadCachedImages

With cache config entry (image file deleted from disk)

% rm ~/.minikube/cache/images/arm64/nginx_latest
% timestamp minikube start -d vfkit --log_file start-no-cached-image.log
[   13.273] * Enabled addons: default-storageclass, storage-provisioner
[   53.699] * Done!    ← 53s total

Now BOTH phases hit the Docker daemon timeout:

image.go:188] daemon lookup for nginx:latest: failed to connect to the docker API
              at ssh://root@docker.local: lookup root@docker.local: no such host
cache.go:96]  cache image "nginx:latest" -> ".../nginx_latest" took 28.570988208s  ← 25s daemon timeout + download

cache_images.go:90]  LoadCachedImages start: [nginx:latest]
cache_images.go:117] "nginx:latest" needs transfer: needs transfer timed out in 10.000000 seconds
image.go:80]          couldn't find image digest nginx:latest from local daemon:
                      Get "http://root%40docker.local/v1.51/images/nginx:latest/json": context deadline exceeded
cache_images.go:94]  duration metric: took 11.996759708s to LoadCachedImages

Without cache config entry (image file still on disk)

% minikube cache delete nginx:latest
% timestamp minikube start -d vfkit --log_file start-no-cached-image.log
[   14.609] * Enabled addons: default-storageclass, storage-provisioner
[   15.092] * Done!    ← 15s total

No LoadCachedImages in the log at all. The image file in
~/.minikube/cache/images/ is inert without the config entry.

Root Cause

minikube cache add persists images to config:

// cmd/minikube/cmd/cache.go
if err := cmdConfig.AddToConfigMap(cacheImageConfigKey, args); err != nil { ... }

On every minikube start, CacheAndLoadImagesInConfig reads this config and calls
machine.CacheAndLoadImages, which:

1. Calls image.SaveToDir → saveToTarFile → tries the Docker daemon to get the
   image (via go-containerregistry's daemon client which resolves DOCKER_HOST)
2. Calls LoadCachedImages → creates a Docker client.Client via
   client.FromEnv → calls DigestByDockerLib with a 10-second timeout

In pkg/minikube/machine/cache_images.go:

var imgClient *client.Client
if cr.Name() == "Docker" {
    imgClient, err = client.NewClientWithOpts(client.FromEnv)
}

The condition cr.Name() == "Docker" checks the guest container runtime, not
whether a Docker daemon is available on the host. For VM-based drivers (vfkit,
hyperkit, QEMU, KVM), the host Docker daemon is irrelevant — images must be
transferred via SCP regardless.

In pkg/minikube/image/image.go:

func DigestByDockerLib(imgClient *client.Client, imgName string) string {
    ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
    defer cancel()
    img, err := imgClient.ImageInspect(ctx, imgName)
    // ... blocks for 10s if daemon is unreachable
}

Impact

• +10s per cached image (load phase) on every minikube start for VM drivers
• +25-30s per cached image (save + load) if the tarball file was deleted
• Multiplies with number of cached images
• Silent — no user-visible error or warning, just slow startup
• Users who followed minikube cache add docs are penalized on every start

Proposed Fix

The Docker daemon optimization is only valid when minikube uses the Docker
driver (KIC mode), where the host daemon IS the container runtime.

if cc.Driver == driver.Docker {
    imgClient, err = client.NewClientWithOpts(client.FromEnv)
}

For the save phase, skip the daemon source when the driver is not Docker.

Minikube has no dependency on the host Docker daemon unless using the Docker
driver. Any code path that attempts to access it for other drivers is broken by
design. On macOS vmnet networks, connecting to a non-existing Docker VM blocks
indefinitely instead of failing with "connection refused" (as on Linux), making
timeout-based workarounds impractical.

Environment

• macOS 26.5.2 (arm64)
• minikube v1.38.1
• Driver: vfkit
• Container runtime in guest: docker
• No Docker Desktop on host

Logs

• start-with-cached-image.log
• start-no-cached-image.log

[afbjorklund]

I don't think that is a new bug, it has been there for a while (I think I tried to fix it before, but CI was broken):

• Calling Docker daemon even for other container runtimes #8577

There is some similar bug where it is required to install a docker binary for none driver, like it "used to be":

• none driver can't be used with non-Docker runtimes (looks for "docker" executable) #5549

[OchukoWH]

Hi @afbjorklund @nirs,

I'd be interested in working on a fix for this.

I noticed this issue (and similar ones) has been around for quite some time, so I wanted to check first: would a PR addressing this still be welcome, or is there another direction planned for handling it?

If a fix would be accepted, I'd be happy to work on it.

[afbjorklund]

I noticed this issue (and similar ones) has been around for quite some time, so I wanted to check first: would a PR addressing this still be welcome, or is there another direction planned for handling it?

I consider all of these issues to be part of the same legacy (where "Docker" was the only runtime), and I think they are supposed to be addressed as part of going from minikube 1.x to minikube 2.x - with switching to containerd as default and not "assuming" docker.

But you can probably fix the bug meanwhile, there is no use of waiting 10 seconds for a possibly non-existing Docker connection.

i.e. "Minikube has no dependency on the host Docker daemon unless using the Docker driver."

[OchukoWH]

/assign

[nirs]

@OchukoWH note that the cache command is deprecated for years - any issue with it will be resolved by removing this command.

What we want to fix is the dependency on docker daemon in the start command. Any code accessing docker daemon or docker command line must be guarded by check for docker driver.

[OchukoWH]

Thanks for the clarification! That makes sense. I'll focus on the start path instead of the deprecated cache command. I'll investigate the Docker daemon accesses and make sure they're guarded so they're only used when running with the Docker driver. I'll put together a PR.