Greetings! I have a potentially curious issue with pre-downloading dependencies for network-isolated container builds.
What I am trying to do:
- via pre-download dependencies
go mod downloadin a local directory
- Mount the directory in container build, set GOMODCACHE to point to this directory
- Build container without internet access
Problem: Part of the build process is running
go generate, This command wants to download additional modules that were not downloaded by Go Mod Downloads. It fails because there is no internet access.
go: downloading golang.org/x/tools v0.1.13-0.20220804200503-81c7dc4e4efa go: downloading github.com/olekukonko/tablewriter v0.0.5 /firstname.lastname@example.org/cmd/internal/printer/printer.go:16:2: email@example.com: Get "https://proxy.golang.org/github.com/olekukonko/tablewriter/@v/v0.0.5.zip": dial tcp: lookup proxy.golang.org: i/o timeout /firstname.lastname@example.org/entc/load/load.go:27:2: email@example.com: Get "https://proxy.golang.org/golang.org/x/tools/@v/v0.1.13-0.20220804200503-81c7dc4e4efa.zip": dial tcp: lookup proxy.golang.org: i/o timeout /firstname.lastname@example.org/entc/gen/graph.go:25:2: email@example.com: Get "https://proxy.golang.org/golang.org/x/tools/@v/v0.1.13-0.20220804200503-81c7dc4e4efa.zip": dial tcp: lookup proxy.golang.org: i/o timeout storage/ent/generate.go:3: running "go": exit status 1
reproducer (with go1.19.2):
git clone https://github.com/dexidp/dex cd dex git checkout v2.35.1 env GOMODCACHE=$(realpath ../gocache) go mod download cd ..
FROM golang:1.19.1-alpine3.16 AS builder # alpine-sdk would be needed if we could get past the download step # RUN apk add alpine-sdk COPY ./dex /src/dex WORKDIR /src/dex ARG GOMODCACHE=/tmp/gocache RUN export GOMODCACHE="$GOMODCACHE" && \ export GOOS=linux && \ go generate -mod=mod /src/dex/storage/ent/
sudo podman network create --internal internal sudo podman build --network=internal -v $(realpath ./gocache):/tmp/gocache:Z --build-arg GOMODCACHE=/tmp/gocache .
The ultimate goal is to make the container as secure and reproducible as possible. In fact, it is part of a build system that does roughly as described: pre-downloads dependencies and then builds containers with network isolation.
With that in mind, I’ve found two sub-optimal solutions:
- Before build, run
go generatein addition to the following
go mod download
go mod download allinstead (downloads generated dips, not sure why)
Problem with both: the go.sum file for the example project does not include additional dependencies. This means that additional downloads are not verified.
Problem generating Go: The host system does not know what parameters to pass to generate Go. It also may not have the various non-golang dependencies needed to generate it successfully.
Problem downloading all: Downloads even more dependencies, most of which aren’t even needed for Go generates.
Is there any other way I missed downloading it?
go generate dependency? The ideal solution would be to include them in
go.sum of the project. However, it detracts from the normal golang dev workflow. even running
go mod tidy removes them again.