mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-25 20:46:53 +09:00
Aleksei Sviridkin
2ddd55ce5b
## What this PR does A local build that asks to load images into docker no longer pushes them. `hack/common-envs.mk` set `PUSH` and `LOAD` with plain `:=`, and a makefile assignment beats the environment, so `LOAD=1 PUSH=0 make image` quietly pushed a multi-arch build. Both now take the environment value, and an empty one falls back to the default. I also made `PUSH` default to 0 when `LOAD=1`, so `LOAD=1` alone only loads. Before this, `make image LOAD=1` pushed and loaded at once. An explicit `PUSH` from the environment or the command line still wins, and `OCI_EXPORT_DIR` still forces both off. Every buildx target gets the values through `BUILDX_ARGS`, so the fix lives in that one file. The Talos installer was the one image that ignored `PUSH`. It is published with skopeo, not buildx, so it never saw `--push=$(PUSH)`. Its registry copies now run only when `PUSH=1`. A build that pushes nothing no longer writes `PUSHED_TAGS_LOG`. That log tells the multi-arch stitch which tags to move, so it must hold only tags that were really pushed. The cache helper lost an argument nobody used. No caller passes the second argument of `cache-args`, and an explicit tag there would bypass `CACHE_TAG`, so the amd64 and arm64 builds of that image would share one cache ref. CI sets neither `PUSH` nor `LOAD`, so CI builds do the same thing as before. Each fix has a `make --dry-run` test in `hack/common-envs_test.bats`. Closes #4499 Closes #4504 Closes #4551 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The ccp skills pass `PUSH=1` explicitly, which still works. The website docs do not mention `PUSH`, `LOAD` or `cache-args`. ### Release note ```release-note fix(build): `LOAD` and `PUSH` set in the environment are now honoured, `LOAD=1` no longer pushes unless `PUSH=1` is also set, and the Talos installer is no longer published when `PUSH=0`. ```
161 lines
8.9 KiB
Makefile
161 lines
8.9 KiB
Makefile
# macOS-compatible sed in-place
|
|
ifeq ($(shell uname),Darwin)
|
|
SED_INPLACE := sed -i ''
|
|
else
|
|
SED_INPLACE := sed -i
|
|
endif
|
|
|
|
REGISTRY ?= ghcr.io/cozystack/cozystack
|
|
|
|
# CACHE_REGISTRY: registry holding the `:buildcache` mode=max cache images that
|
|
# `--cache-from`/`--cache-to` read and write (see the cache-args macro below).
|
|
# Defaults to $(REGISTRY) so the cache is co-located with the build registry —
|
|
# in CI that is the per-CI registry the runners are closest to, keeping cache
|
|
# transfers fast and egress-free.
|
|
CACHE_REGISTRY ?= $(REGISTRY)
|
|
|
|
# IMAGE_TAG: build-unique tag pushed for every image. Set by CI to a value
|
|
# that does not collide between concurrent builds:
|
|
# * pull-requests.yaml -> pr-<N>-<sha>
|
|
# * tags.yaml -> <ref_name> (e.g. v1.5.0)
|
|
# * local -> dev (pushes unless PUSH=0; LOAD=1 defaults PUSH to 0)
|
|
IMAGE_TAG ?= dev
|
|
|
|
# Opt-in extra tags. Workflows set these explicitly; defaults are off so a
|
|
# local `make build` never races with CI or accidentally moves :latest.
|
|
# PUBLISH_VERSIONED=1 -> also push :<component-version> (release semantics)
|
|
# PUBLISH_FLOATING=1 -> also push :latest (release/main only)
|
|
PUBLISH_VERSIONED ?= 0
|
|
PUBLISH_FLOATING ?= 0
|
|
|
|
# WRITE_CACHE=1 enables `--cache-to` (mode=max) to CACHE_REGISTRY/<img>:buildcache.
|
|
# Default off: only the serialized main/release cache-warming build writes the
|
|
# shared cache ref. PR builds keep it 0 and read-only, so concurrent PR builds
|
|
# never race on the cache manifest (the 409 collisions PR #2711 fixed for tags).
|
|
WRITE_CACHE ?= 0
|
|
|
|
# Environment and command line both override these, and an empty value from the
|
|
# environment counts as unset. LOAD=1 loads into the local docker store, so PUSH
|
|
# follows it to 0 unless PUSH is set too.
|
|
LOAD := $(or $(strip $(LOAD)),0)
|
|
PUSH := $(or $(strip $(PUSH)),$(if $(filter 1,$(LOAD)),0,1))
|
|
|
|
# OCI_EXPORT_DIR: when set, build every image to a per-image OCI archive
|
|
# (<dir>/<name>.oci.tar) instead of pushing or loading it. Fork PRs use this:
|
|
# the unprivileged `pull_request` build carries no registry credentials, so it
|
|
# exports the images as workflow artifacts and a privileged `workflow_run` later
|
|
# pushes them to the registry BY DIGEST (see .github/workflows/e2e-fork.yaml).
|
|
# The image digest is content-addressed and captured via --metadata-file
|
|
# regardless of output type, so the values.yaml/.tag refs baked during this
|
|
# build match exactly what the privileged run pushes. Setting it forces
|
|
# PUSH/LOAD off so buildx emits only the archive — a `--push` alongside the OCI
|
|
# --output would attempt an anonymous push and die with `denied` (#3257).
|
|
OCI_EXPORT_DIR ?=
|
|
ifneq ($(strip $(OCI_EXPORT_DIR)),)
|
|
PUSH := 0
|
|
LOAD := 0
|
|
endif
|
|
|
|
BUILDER ?=
|
|
# Published images are amd64+arm64 by default, so a build on either
|
|
# architecture cannot publish an image the other one cannot run. LOAD=1 builds
|
|
# the host architecture only: the classic docker image store cannot load a
|
|
# multi-platform index. Set PLATFORM explicitly to build anything else.
|
|
DEFAULT_PLATFORMS := linux/amd64,linux/arm64
|
|
PLATFORM ?= $(if $(filter 1,$(LOAD)),,$(DEFAULT_PLATFORMS))
|
|
BUILDX_EXTRA_ARGS ?=
|
|
COZYSTACK_VERSION = $(patsubst v%,%,$(shell git describe --tags --match 'v*'))
|
|
|
|
# SBOM=1 attaches a CycloneDX SBOM as a buildx build attestation (--sbom=true).
|
|
# Requires the docker-container buildx driver and registry referrers/attestation
|
|
# support. Kept OFF by default until an OCIR referrers/attestation probe confirms
|
|
# support — otherwise the attestation push fails the build. See #2937.
|
|
SBOM ?= 0
|
|
|
|
BUILDX_ARGS := --provenance=false --push=$(PUSH) --load=$(LOAD) \
|
|
--label org.opencontainers.image.source=https://github.com/cozystack/cozystack \
|
|
$(if $(filter 1,$(SBOM)),--sbom=true) \
|
|
$(if $(strip $(BUILDER)),--builder=$(BUILDER)) \
|
|
$(if $(strip $(PLATFORM)),--platform=$(PLATFORM)) \
|
|
$(BUILDX_EXTRA_ARGS)
|
|
|
|
# image-tags <repo> <versioned-tag>
|
|
# Expands to one or more `--tag` flags for `docker buildx build`:
|
|
# - always: :$(IMAGE_TAG) (build-unique handle)
|
|
# - if PUBLISH_VERSIONED=1: :<versioned-tag> (skipped when arg2 is empty
|
|
# or equals IMAGE_TAG)
|
|
# - if PUBLISH_FLOATING=1: :latest
|
|
# Consumers reference images by digest, so the build-unique tag is enough
|
|
# for cluster-side pulls; the versioned and floating tags exist for human
|
|
# discoverability and downstream tooling on releases.
|
|
# When OCI_EXPORT_DIR is set it also appends a per-image `--output type=oci`
|
|
# (via the oci-output macro) so the build writes <dir>/<name>.oci.tar (the tag
|
|
# becomes the archive's image name). Every package's buildx call routes its tags
|
|
# through this macro, so this is the single point that turns the whole build into
|
|
# an artifact-export build — except recipes that bypass image-tags, which must
|
|
# call oci-output directly (see #3257).
|
|
|
|
# comma emits a literal comma inside the lazily-expanded macros below
|
|
# (oci-output, cache-args) without make mis-reading it as an $(if …)/$(call …)
|
|
# argument separator. Defined here, above its first user, so it is unambiguously
|
|
# in scope wherever those macros are $(call)ed from a recipe.
|
|
comma := ,
|
|
|
|
# oci-output <image-name> — when OCI_EXPORT_DIR is set, expands to a per-image
|
|
# `--output type=oci,dest=<dir>/<name>.oci.tar`; empty otherwise. EVERY buildx
|
|
# call reachable from a fork build must include this — via image-tags, or
|
|
# directly for recipes that bypass image-tags (#3257). A missing archive is
|
|
# caught loudly by the fork build's `if-no-files-found: error` upload.
|
|
oci-output = $(if $(strip $(OCI_EXPORT_DIR)), --output type=oci$(comma)dest=$(OCI_EXPORT_DIR)/$(subst /,-,$(1)).oci.tar)
|
|
|
|
# image-tags <image-name> [<versioned-tag>] — the --tag/--output flags for one
|
|
# image. Under OCI_EXPORT_DIR the versioned/floating tags are suppressed: a
|
|
# multi-`--tag` `--output type=oci` build writes every ref into the archive's
|
|
# index.json, and `skopeo copy oci-archive:<f>` then refuses it ("more than one
|
|
# image in oci"). PUBLISH_* are 0 on fork PRs (the only export case today), so
|
|
# this is belt-and-suspenders for that trap.
|
|
#
|
|
# PUSHED_TAGS_LOG=<file> appends every pushed <repo>:<tag> to <file>, one per
|
|
# line. hack/stitch-multiarch.sh moves exactly those tags onto the multi-arch
|
|
# index; no ref in the tree names a versioned tag, so this is how it learns
|
|
# one was pushed. The append happens when the recipe is expanded, `make -n`
|
|
# included, and is skipped when the build pushes nothing (PUSH other than 1, or
|
|
# OCI_EXPORT_DIR).
|
|
image-tag-refs = $(REGISTRY)/$(1):$(IMAGE_TAG)$(if $(strip $(OCI_EXPORT_DIR)),,$(if $(filter 1,$(PUBLISH_VERSIONED)),$(if $(filter-out $(IMAGE_TAG),$(strip $(2))), $(REGISTRY)/$(1):$(strip $(2))))$(if $(filter 1,$(PUBLISH_FLOATING)), $(REGISTRY)/$(1):latest))
|
|
|
|
define image-tags
|
|
$(foreach r,$(call image-tag-refs,$(1),$(2)),--tag $(r))$(call oci-output,$(1))$(if $(and $(strip $(PUSHED_TAGS_LOG)),$(filter 1,$(PUSH)),$(if $(strip $(OCI_EXPORT_DIR)),,1)),$(shell printf '%s\n' $(call image-tag-refs,$(1),$(2)) >>'$(PUSHED_TAGS_LOG)'))
|
|
endef
|
|
|
|
# cache-args <image-name>
|
|
# Expands to buildx cache flags for one image:
|
|
# --cache-from is always emitted (a missing cache 404s harmlessly -> cold build)
|
|
# --cache-to is emitted only when WRITE_CACHE=1 (main/release), writing mode=max
|
|
# so ALL stages are cached -- including the multistage `builder` layers that
|
|
# `--cache-to type=inline` could never export. oci-mediatypes + image-manifest
|
|
# keep the cache manifest portable across registries (OCIR/ghcr/ECR).
|
|
# CACHE_TAG lets a build for another platform keep a cache of its own
|
|
# (buildcache-arm64) instead of overwriting the amd64 one with every write.
|
|
# $(comma) (defined above) escapes the literal commas in the --cache-to value so
|
|
# make does not mis-parse them as $(if ...) argument separators.
|
|
CACHE_TAG ?= buildcache
|
|
cache-args = --cache-from type=registry,ref=$(CACHE_REGISTRY)/$(1):$(CACHE_TAG)$(if $(filter 1,$(WRITE_CACHE)), --cache-to type=registry$(comma)ref=$(CACHE_REGISTRY)/$(1):$(CACHE_TAG)$(comma)mode=max$(comma)oci-mediatypes=true$(comma)image-manifest=true)
|
|
|
|
ifeq ($(COZYSTACK_VERSION),)
|
|
$(shell git remote add upstream https://github.com/cozystack/cozystack.git || true)
|
|
$(shell git fetch upstream --tags)
|
|
COZYSTACK_VERSION = $(patsubst v%,%,$(shell git describe --tags --match 'v*'))
|
|
endif
|
|
|
|
# Go module fetches in image builds. IMAGE_GOPROXY reaches every Go builder
|
|
# stage as the GOPROXY build arg, which a stage sees only once it declares
|
|
# `ARG GOPROXY`. The pipe falls back to direct on any proxy error; Go's default
|
|
# comma does so only on a 404 or 410, so an HTTP/2 stream reset from
|
|
# proxy.golang.org fails the build. A module hosted in git can only be fetched
|
|
# directly where the stage has git, which the alpine golang images lack; there
|
|
# the retry around `go mod download` is the only cover. The value is not read
|
|
# from the host's GOPROXY: a mirror on localhost or a file:// proxy is
|
|
# unreachable from inside buildkit.
|
|
IMAGE_GOPROXY ?= https://proxy.golang.org|direct
|
|
BUILDX_ARGS += --build-arg 'GOPROXY=$(IMAGE_GOPROXY)'
|