mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 12:13:55 +09:00
fix(ci): limit the drifted-file bare-ref overlay to first-party images
Main can bump a third-party image together with its config, so a PR behind it must keep the committed pair rather than take main's image alone. Only ghcr.io/cozystack/cozystack refs, matched against an artifact line for the same first-party image, are replaced now. State the trade-off for first-party refs that do exist, and name the packages that took main's images in their own annotation. Assisted-by: LLM Signed-off-by: Maxim Belyy <maximbel2003@gmail.com>
This commit is contained in:
1 parent
54b76fcdf1
commit
854a3f40e4
3 files changed
+83
-27
No files matched your search
@@ -42,7 +42,7 @@ If a package stamps a reference somewhere the two globs do not cover, add that f
|
||||
|
||||
## Invariants
|
||||
|
||||
- **Every first-party reference is tag+digest.** A bare tag with no digest is not reproducible and must not be committed. The one exception is the first commit of a new image: it has no digest until it is built, so it lands as a bare `<repo>:<tag>` that the producing `make image` replaces. PR CI and release preparation both stamp it before anything pulls it, but a tree applied without a build pulls a tag that does not exist. In PR CI the stamp comes from `hack/overlay-main-images.sh` when the PR does not rebuild the package, and it takes a bare ref even from a file whose configuration drifted from the artifact, because keeping a tag that was never pushed cannot install.
|
||||
- **Every first-party reference is tag+digest.** A bare tag with no digest is not reproducible and must not be committed. The one exception is the first commit of a new image: it has no digest until it is built, so it lands as a bare `<repo>:<tag>` that the producing `make image` replaces. PR CI and release preparation both stamp it before anything pulls it, but a tree applied without a build pulls a tag that does not exist. In PR CI the stamp comes from `hack/overlay-main-images.sh` when the PR does not rebuild the package, and it takes a bare first-party ref even from a file whose configuration drifted from the artifact, because a tag that was never pushed cannot install. The cost is that any other digest-less first-party ref in such a file also runs main's image with the PR's older config; third-party refs in it keep their committed pair.
|
||||
- **Exactly one producer stamps each reference.** A reference no `Makefile` writes will never be refreshed by anything, including release preparation.
|
||||
- **A package that stamps a reference is in the root `Makefile` `build:` list.** `build-matrix.sh`, the main build and release preparation all iterate that one list, so a package outside it is never rebuilt by any path and its pin freezes permanently. `hack/build-matrix_test.bats` computes the set of packages whose Makefile runs `docker buildx build` and fails on any the list misses.
|
||||
- **Every consumer of references reads all three storage shapes**, via `hack/lib/image-refs.sh`.
|
||||
|
||||
+44
-19
@@ -43,7 +43,7 @@
|
||||
# If any OTHER line differs — the PR branched from a
|
||||
# main whose config for that file differs from the artifact's base — the file is
|
||||
# left on its committed ref (safe degradation, logged, never fatal), except
|
||||
# that a bare digest-less ref in it, which no registry serves, still takes the
|
||||
# that a bare first-party ref in it, one with no digest, still takes the
|
||||
# artifact's ref (see replace_bare_refs). CI checks out the pull_request merge
|
||||
# commit (main + PR), so for an unbuilt file the config matches main as of
|
||||
# that merge commit. It is fixed when the run starts, so a rerun, or a PR not
|
||||
@@ -127,23 +127,33 @@ img_line='(@sha256:|^[[:space:]]*(- )?(image|repository|registry|tag|digest):|^[
|
||||
# match branch for this same shape and for this same reason.
|
||||
bare_ref_line='^[^[:space:]#][^[:space:]]*/[^[:space:]]+$'
|
||||
|
||||
# A drifted file keeps its committed refs. For a ref pinned by digest that is a
|
||||
# pullable older image, which is the safe answer. A bare `<repo>:<tag>` with no
|
||||
# digest is different: it is the first commit of a new image
|
||||
# (docs/agents/image-refs.md), its tag was never pushed, and keeping it makes the
|
||||
# component fail to pull instead of run an older image. The merge commit CI
|
||||
# checks out is fixed when the run starts, so a PR whose merge commit predates a
|
||||
# config change to such a file drifts on it, and every new image is exposed
|
||||
# until a release stamps its ref (#5328).
|
||||
# A drifted file keeps its committed refs. For a ref pinned by digest that is an
|
||||
# older image which pulls and matches the PR's config, so it is the safe answer.
|
||||
# A first-party ref with no digest is not: docs/agents/image-refs.md forbids it
|
||||
# except as the first commit of a new image, whose tag was never pushed, and
|
||||
# keeping that makes the component fail to pull. The merge commit CI checks out
|
||||
# is fixed when the run starts, so a PR whose merge commit predates a config
|
||||
# change to such a file drifts on it, and a new image fails every such PR until
|
||||
# a release stamps its ref (#5328).
|
||||
#
|
||||
# replace_bare_refs <cur> <new> rewrites, in <cur> only, each line whose value is
|
||||
# such a bare ref under an image key, taking the line from <new> that has the
|
||||
# same key and indentation, the same image name and an @sha256: digest. Exactly
|
||||
# one such line must exist; two are ambiguous and none means build-main did not
|
||||
# stamp it, so there is nothing more pullable to take. Every other line of <cur>
|
||||
# stays as the PR has it. The current image may expect configuration the PR's
|
||||
# older chart lacks, but the committed ref cannot run at all. Prints the number
|
||||
# of lines replaced.
|
||||
# a bare first-party ref (ghcr.io/cozystack/cozystack/<name>:<tag>, no digest)
|
||||
# under an image key, taking the line from <new> that has the same key and
|
||||
# indentation, an @sha256: digest and the same first-party image, i.e. a path
|
||||
# ending in /cozystack/<name>. Exactly one such line must exist; two are
|
||||
# ambiguous and none means build-main did not stamp it. Every other line of
|
||||
# <cur> stays as the PR has it.
|
||||
#
|
||||
# The trade-off is deliberate. A bare first-party ref that does exist (an older
|
||||
# release tag or a moving `latest`, #4264) also takes main's image, so it runs
|
||||
# with the PR's older config rather than as committed. It is the same component
|
||||
# built from the tree, and a digest-less ref is not reproducible anyway; the
|
||||
# alternative is a ref that may not pull at all. Third-party refs are never
|
||||
# touched: main can bump one alongside its config, and the committed pair is
|
||||
# the one known to work together. The split shape (`repository:` + `tag:`
|
||||
# without `digest:`) is not handled; no package uses it for a bare ref today.
|
||||
#
|
||||
# Prints the number of lines replaced.
|
||||
replace_bare_refs() {
|
||||
tmp="$1.overlay-bare.$$"
|
||||
n=$(awk -v newf="$2" -v out="$tmp" '
|
||||
@@ -167,7 +177,10 @@ replace_bare_refs() {
|
||||
while ((getline l < newf) > 0) {
|
||||
p = key_prefix(l)
|
||||
if (p == "" || index(l, "@sha256:") == 0) continue
|
||||
k = p SUBSEP image_name(ref_value(l, p))
|
||||
v = ref_value(l, p)
|
||||
name = image_name(v)
|
||||
if (index(v, "/cozystack/" name ":") == 0) continue
|
||||
k = p SUBSEP name
|
||||
cnt[k]++
|
||||
line[k] = l
|
||||
}
|
||||
@@ -177,7 +190,7 @@ replace_bare_refs() {
|
||||
p = key_prefix($0)
|
||||
if (p != "") {
|
||||
v = ref_value($0, p)
|
||||
if (v ~ /^[^@[:space:]]+\/[^@[:space:]\/]+:[^@[:space:]\/]+$/) {
|
||||
if (v ~ /^ghcr\.io\/cozystack\/cozystack\/[^@[:space:]\/]+:[^@[:space:]\/]+$/) {
|
||||
k = p SUBSEP image_name(v)
|
||||
if (cnt[k] == 1) { print line[k] > out; r++; next }
|
||||
}
|
||||
@@ -197,6 +210,7 @@ replace_bare_refs() {
|
||||
overlaid=0
|
||||
same=0
|
||||
bare=0
|
||||
bare_files=""
|
||||
skipped=0
|
||||
drift=0
|
||||
drift_files=""
|
||||
@@ -240,8 +254,10 @@ for new in $(find "$MAINPKGS" -type d -name charts -prune -o \
|
||||
*.yaml)
|
||||
r=$(replace_bare_refs "$cur" "$new") || r=0
|
||||
if [ "$r" -gt 0 ]; then
|
||||
echo "bare ref in $cur -> current-main ($r line(s))"
|
||||
echo "bare first-party ref in $cur -> current-main ($r line(s)); its other refs stay committed"
|
||||
bare=$((bare + r))
|
||||
d=$(dirname "${cur#packages/}")
|
||||
bare_files="$bare_files $d"
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
@@ -277,3 +293,12 @@ if [ "$drift" -gt 0 ]; then
|
||||
"$(printf '%s\n' "$drift_pkgs" | wc -l | tr -d ' ')" \
|
||||
"$(printf '%s' "$drift_pkgs" | tr '\n' ' ' | sed 's/ $//')"
|
||||
fi
|
||||
# Among those, a package whose bare first-party ref took main's image runs that
|
||||
# image with the PR's older config, not its committed ref; name it separately
|
||||
# so the warning above does not read as the whole story for it.
|
||||
if [ "$bare" -gt 0 ]; then
|
||||
bare_pkgs=$(echo "$bare_files" | tr ' ' '\n' | grep -v '^$' | sort -u)
|
||||
printf '::warning title=Image overlay bare refs::%s drifted package(s) took current-main images for bare first-party refs, with the config committed in this PR: %s\n' \
|
||||
"$(printf '%s\n' "$bare_pkgs" | wc -l | tr -d ' ')" \
|
||||
"$(printf '%s' "$bare_pkgs" | tr '\n' ' ' | sed 's/ $//')"
|
||||
fi
|
||||
@@ -525,7 +525,8 @@ load test_helper
|
||||
if grep -q 'prefix:' "$w/packages/system/proxy/values.yaml"; then
|
||||
echo "FAIL: a config line was taken from the artifact" >&2; exit 1
|
||||
fi
|
||||
grep -q 'bare ref in packages/system/proxy/values.yaml -> current-main' "$w/out.txt"
|
||||
grep -q 'bare first-party ref in packages/system/proxy/values.yaml -> current-main' "$w/out.txt"
|
||||
grep -q '::warning title=Image overlay bare refs::1 drifted package(s) .*: system/proxy$' "$w/out.txt"
|
||||
grep -q 'bare-refs=1' "$w/out.txt"
|
||||
rm -rf "$w"
|
||||
}
|
||||
@@ -560,15 +561,45 @@ load test_helper
|
||||
}
|
||||
|
||||
@test "a bare ref is kept when the artifact carries no digest for it either" {
|
||||
# A ref build-main does not stamp (third-party, or not built yet) has no
|
||||
# digest in the artifact, so there is nothing more pullable to take.
|
||||
# A ref build-main does not stamp has no digest in the artifact, so there is
|
||||
# nothing more pullable to take.
|
||||
root=$(pwd)
|
||||
w=$(mktemp -d)
|
||||
mkdir -p "$w/packages/system/tp" "$w/main/system/tp"
|
||||
printf 'tuning: old\nimage: docker.io/library/busybox:1.37.0\n' > "$w/packages/system/tp/values.yaml"
|
||||
printf 'tuning: new\nimage: docker.io/library/busybox:1.38.0\n' > "$w/main/system/tp/values.yaml"
|
||||
mkdir -p "$w/packages/system/nd" "$w/main/system/nd"
|
||||
printf 'tuning: old\nimage: ghcr.io/cozystack/cozystack/nd:v0.0.0\n' > "$w/packages/system/nd/values.yaml"
|
||||
printf 'tuning: new\nimage: iad.ocir.io/x/cozystack/nd:main\n' > "$w/main/system/nd/values.yaml"
|
||||
( cd "$w" && "$root/hack/overlay-main-images.sh" main '[]' ) > "$w/out.txt"
|
||||
grep -q '^image: docker.io/library/busybox:1.37.0$' "$w/packages/system/tp/values.yaml"
|
||||
grep -q '^image: ghcr.io/cozystack/cozystack/nd:v0.0.0$' "$w/packages/system/nd/values.yaml"
|
||||
grep -q 'bare-refs=0' "$w/out.txt"
|
||||
rm -rf "$w"
|
||||
}
|
||||
|
||||
@test "a bare third-party ref in a drifted file is kept even when main pins one" {
|
||||
# Main can bump a third-party image together with its config in one file. A
|
||||
# PR behind it has the older pair, which is the one known to work together;
|
||||
# taking main's image alone would run it with the config it replaced.
|
||||
root=$(pwd)
|
||||
w=$(mktemp -d)
|
||||
mkdir -p "$w/packages/system/kc" "$w/main/system/kc"
|
||||
printf 'features: old\nimage: quay.io/keycloak/keycloak:26.6.3\n' > "$w/packages/system/kc/values.yaml"
|
||||
printf 'features: new\nimage: quay.io/keycloak/keycloak:27.0.0@sha256:abcd\n' > "$w/main/system/kc/values.yaml"
|
||||
( cd "$w" && "$root/hack/overlay-main-images.sh" main '[]' ) > "$w/out.txt"
|
||||
grep -q '^image: quay.io/keycloak/keycloak:26.6.3$' "$w/packages/system/kc/values.yaml"
|
||||
grep -q 'bare-refs=0' "$w/out.txt"
|
||||
if grep -q 'Image overlay bare refs' "$w/out.txt"; then
|
||||
echo "FAIL: a bare-refs annotation with nothing replaced" >&2; exit 1
|
||||
fi
|
||||
rm -rf "$w"
|
||||
}
|
||||
|
||||
@test "a bare first-party ref is not replaced by a third-party image of the same name" {
|
||||
root=$(pwd)
|
||||
w=$(mktemp -d)
|
||||
mkdir -p "$w/packages/system/same" "$w/main/system/same"
|
||||
printf 'tuning: old\nimage: ghcr.io/cozystack/cozystack/redis:v0.0.0\n' > "$w/packages/system/same/values.yaml"
|
||||
printf 'tuning: new\nimage: docker.io/library/redis:8.0@sha256:abcd\n' > "$w/main/system/same/values.yaml"
|
||||
( cd "$w" && "$root/hack/overlay-main-images.sh" main '[]' ) > "$w/out.txt"
|
||||
grep -q '^image: ghcr.io/cozystack/cozystack/redis:v0.0.0$' "$w/packages/system/same/values.yaml"
|
||||
grep -q 'bare-refs=0' "$w/out.txt"
|
||||
rm -rf "$w"
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user