Skip to content

[CI]: git clone revamp, part II #4031

Open
@apostasie

Description

@apostasie

What is the problem you're trying to solve

This is a second breakout from #4021.

  1. git clones are currently done for every target platform. There is no reason for that, and clone could (should) be done only once per run, at maximum.

  2. The same goes for go modules.
    They are currently retrieved at build time - which means they are retrieved as many times as we build (per architecture, per version).
    They could (should) instead be retrieved once, at clone time, using go mod vendor.

  3. Also, go mod could (should) use a caching mount for the download cache location, to avoid redownloading modules that have been fetched already (eg: by another project), and also avoid putting every independent stage go mod cache into buildkit cache

  4. Finally, it is convenient to be able to override the git repository location of a given project (be it because of maintenance or to test a fork). I recently faced exactly that problem (I am building libslirp, but they went offline for maintenance for a week).

The proposal here is to address these issues.

Describe the solution you'd like

The updated form of the Dockerfile would look something like (pseudo code, should be very close to working):

FROM        --platform=$BUILDPLATFORM builder AS download-bypass4netns
ARG         BYPASS4NETNS_VERSION
ARG         BYPASS4NETNS_REPO
RUN         --mount=target=/root/go/pkg/mod,type=cache \
            git clone --quiet --depth 1 --branch "${BYPASS4NETNS_VERSION%@*}" https://"$BYPASS4NETNS_REPO".git . && \
            git-checkout-tag-with-hash.sh "$VERSION" && \
            go mod vendor

FROM        --platform=$BUILDPLATFORM builder AS build-bypass4netnsd
ARG         TARGETARCH
RUN         --mount=from=download-bypass4netns,type=bind,target=/src,source=/src,rw \
            exec 42>.lock; flock -x 42; \
            GOPROXY=off GOFLAGS=--mod=vendor make blablbabla; \
            flock -u 42

Let me know your thoughts on this to infzasx (cat is typing :-)) to inform upcoming PR.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions