From f29fbd7c9b858246813334b58af53480690b46a6 Mon Sep 17 00:00:00 2001 From: Ramkumar Chinchani Date: Wed, 29 May 2024 17:03:10 +0000 Subject: [PATCH] Add support for erofs and squashfs layer media types OCI artifacts support has landed in various OCI specs v1.1.0 which allows for arbitrary artifact types, small and large. Large artifacts (even existing container images) pose a particular challenge that: 1) it takes too long to download 2) it takes too long to unpack This PR **begins** to address 2) above. OCI image layers have traditionally been packaged as tar archives which must be unpacked after downloading. Read-only compressed filesystems such as squashfs and erofs present viable alternatives such that the unpack step can be skipped since they are directly mountable and overlayfs compatible. squashfs is older but has wider platform support (Linux, *BSD and Windows). erofs is newer and has strong community support but very Linux-specific. Build tools will now produce these new layers directly instead of tar archives. Additional reading: https://www.cyphar.com/blog/post/20190121-ociv2-images-i-tar https://groups.google.com/a/opencontainers.org/g/dev/c/Zk3yf45HIdA?pli=1 Signed-off-by: Ramkumar Chinchani --- config.md | 2 ++ layer.md | 4 ++++ specs-go/v1/mediatype.go | 8 ++++++++ 3 files changed, 14 insertions(+) diff --git a/config.md b/config.md index 5d6953083..a7c458838 100644 --- a/config.md +++ b/config.md @@ -30,6 +30,8 @@ Layers SHOULD be packed and unpacked reproducibly to avoid changing the layer Di NOTE: Do not confuse DiffIDs with [layer digests](manifest.md#image-manifest-property-descriptions), often referenced in the manifest, which are digests over compressed or uncompressed content. +For erofs and squashfs layers, a layer DiffID is the digest over the layer itself. + ### Layer ChainID For convenience, it is sometimes useful to refer to a stack of layers with a single identifier. diff --git a/layer.md b/layer.md index a5c692f41..035df8f1c 100644 --- a/layer.md +++ b/layer.md @@ -20,6 +20,8 @@ This section defines the `application/vnd.oci.image.layer.v1.tar`, `application/ - Layer Changesets for the [media type](media-types.md) `application/vnd.oci.image.layer.v1.tar` MUST be packaged in [tar archive][tar-archive]. - Layer Changesets for the [media type](media-types.md) `application/vnd.oci.image.layer.v1.tar` MUST NOT include duplicate entries for file paths in the resulting [tar archive][tar-archive]. +- Layer Changesets for the [media type](media-types.md) `application/vnd.oci.image.layer.v1.erofs` MUST be packaged in [erofs file][erofs]. +- Layer Changesets for the [media type](media-types.md) `application/vnd.oci.image.layer.v1.squashfs` MUST be packaged in [squashfs file][squashfs]. ## Change Types @@ -365,3 +367,5 @@ Implementations SHOULD NOT upload layers tagged with this media type; however, s [rfc1952_2]: https://tools.ietf.org/html/rfc1952 [tar-archive]: https://en.wikipedia.org/wiki/Tar_(computing) [rfc8478]: https://tools.ietf.org/html/rfc8478 +[squashfs]: https://github.com/plougher/squashfs-tools +[erofs]: https://erofs.docs.kernel.org/ diff --git a/specs-go/v1/mediatype.go b/specs-go/v1/mediatype.go index ce8313e79..e0df5b3a2 100644 --- a/specs-go/v1/mediatype.go +++ b/specs-go/v1/mediatype.go @@ -45,6 +45,14 @@ const ( // MediaTypeImageLayerZstd is the media type used for zstd compressed // layers referenced by the manifest. MediaTypeImageLayerZstd = "application/vnd.oci.image.layer.v1.tar+zstd" + + // MediaTypeErofs specifies the media type as erofs filesystem layer + // https://www.iana.org/assignments/media-types/application/vnd.erofs + MediaTypeErofs = "application/vnd.oci.image.layer.v1.erofs" + + // MediaTypeSquashfs specifies the media type as squashfs filesystem layer + // https://github.com/plougher/squashfs-tools + MediaTypeSquashfs = "application/vnd.oci.image.layer.v1.squashfs" ) // Non-distributable layer media-types.