Pull from Spegel/Embedded Registry Mirror

[macrokernel]

Is your feature request related to a problem? Please describe.
I would like to be able to pull images from Spegel like from an ordinary registry. I think this should help me with this issue: banzaicloud/banzai-charts#1353.

[brandond]

You can. That is how it works. When the embedded registry is enabled, containerd is configured to pull from spegel as a mirror for other upstream registries. You can also pull directly from it, if you want, and assuming the content is available in containerd.

See the docs: https://docs.k3s.io/installation/registry-mirror#enabling-the-distributed-oci-registry-mirror

When enabled at a cluster level, all nodes will host a local OCI registry on port 6443

[macrokernel]

Hi Brandon,
I have already read the doc you referring. There is an example of how to pull images into the registry, but I cannot find how to pull from it. Could you please point me at the exact place in the doc?

[brandond]

all nodes will host a local OCI registry on port 6443

Just do as the doc says - enable Spegel mirroring for an upstream registry, load content for that registry into the containerd image store, and then pull it using crictl or by referencing the image in a pod spec. Pulls from that upstream registry will be served by Spegel.

Spegel must be used as a mirror for an upstream registry, but that registry does not need to actually exist or be reachable. So you can just make up a registry hostname, tag and import images into the containerd image store as if they came from that registry, and Spegel will happily share that content.

[macrokernel]

I did, and the registry mirror is working fine for starting up pods, but it is not working for vault-secrets-webhook, so I opened an issue in that project as well. I tend to think that vault-secrets-webhook needs something special from the registry side, which K3s embedded registry is lacking. Perhaps the webhook is using a different (from the standard image pull) HTTP method or something like that. The webhook is querying the registry to determine the container's ENTRYPOINT and CMD from the image metadata.

[brandond]

Spegel must be used as a mirror (proxy), with the upstream registry host set in the ns query parameter. You cannot pull directly from it without specifying the upstream host that it is mirroring.

See: https://github.com/opencontainers/distribution-spec/blob/main/spec.md#registry-proxying

[macrokernel]

Thank you for the clarification. I've reported this to the vault-secrets-webhook issue tracker since I don't think I can resolve it myself. Hopefully, they'll eventually make the webhook compatible with K3s's embedded registry mirror.

Just for the record, I found a very similar issue: #9870.