Files
cozystack/hack/e2e-chainsaw/harbor/chainsaw-test.yaml
T
Aleksei Sviridkin 49cefe0dff fix(tests): size HR-Ready asserts on nested and Package releases
A nested <name>-system release takes its install timeout from the
template that renders it, not from cozystack-api, and a Package release
takes it from cozystack-operator. The external-dns, openbao and qdrant
templates set 10m and harbor's sets 15m; the operator gives 10m. Their
asserts sat at 5m or 10m, so a failed install reported as a bare
timeout. Raise each above its own floor: 11m, and 16m for
harbor-test-system. harbor-test was on its 600s floor exactly, which
expires first because its clock starts before the install's; it goes
to 11m with the generated parents here.

kafka keeps its 6m: kafka-rd sets a 7h install timeout that no budget
inside the E2E job cap can clear, and the assert now says so.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-01 17:40:21 +03:00

267 lines
11 KiB
YAML

# Chainsaw port of hack/e2e-apps/harbor.bats ("Create Harbor").
#
# The BATS test hand-rolled a large diagnostic dump on the harbor-test-system
# HelmRelease wait; here that becomes the test-wide catch block, which fires on
# any step failure.
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: harbor
spec:
catch:
- events: {}
- describe:
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
name: harbor-test
- describe:
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
name: harbor-test-system
- describe:
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketClaim
name: harbor-test-registry
- describe:
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketAccess
name: harbor-test-registry
# The backend view: cluster-scoped Buckets plus the COSI controller pods,
# where a frozen-claim readiness propagation failure actually surfaces.
- get:
apiVersion: objectstorage.k8s.io/v1alpha1
kind: Bucket
format: yaml
- script:
content: kubectl -n cozy-objectstorage-controller get pods
# The app chart wraps the workload in a nested HelmRelease named
# `<release>-system`, and the chart behind it stamps
# app.kubernetes.io/instance from its own release name. So the Harbor pods
# carry `harbor-test-system`, and the unsuffixed value this collector used to
# hold reached none of them, on every failed run.
#
# Coverage boundary, so the absence of a component here is not read as its
# health: this selector reaches the workload of the nested release — core,
# registry, portal, jobservice, nginx, exporter, and trivy when it is
# enabled (the manifest for this suite disables it). The Redis pods carry
# `harbor-test-redis`, so this selector misses them, but the script below
# collects their logs by `rf[rs]-` name prefix. The CNPG database is covered
# by the podLogs op below, on a different label.
#
# Nothing here collects the parent chart's post-delete cleanup Job, which
# labels its pod template with the unsuffixed parent release name. That is
# not an oversight: this suite has no `delete` op, so the parent release is
# uninstalled only in Chainsaw's cleanup phase, and catch cannot observe it
# -- catch is a defer inside the step, while the cleaner runs from t.Cleanup,
# after the test function has returned. A collector on the unsuffixed value
# would be a permanently-empty capture, the defect these selectors exist to
# avoid. The qdrant suite deletes inside a `try:`, so there the Job is at
# least in scope, and that suite carries the collector with its own note on
# when it captures anything.
- podLogs:
selector: app.kubernetes.io/instance=harbor-test-system
tail: 50
# harbor-test readiness gates on this database, so its logs are the evidence
# for a whole class of failed runs. Selected on `cnpg.io/cluster`, CNPG's own
# primary label, rather than on app.kubernetes.io/instance, which CNPG only
# derives from the Cluster name: the primary label has nothing to drift from,
# and it is what the postgres suite and the WorkloadMonitors already use.
- podLogs:
selector: cnpg.io/cluster=harbor-test-db
tail: 50
# podLogs cannot report that it matched nothing, so an empty capture above and
# a selector that has gone stale look identical. Print the instance label of
# every pod in the namespace instead. Deliberately no selector is repeated
# here: a copy would only prove itself right, and would keep matching after
# the collector's own selector drifted. The census shows the values that
# actually exist, so a reader can see whether the selector above can match.
#
# What is and is not guarded, so the next reader does not assume more:
# the `-system` value above is also pinned on a resource in a `steps:` assert
# below, so the chart renaming that label turns a step red -- that is the
# drift this pin exists to catch. It does NOT catch a wrong value typed into the
# selector here: the assert pins the label on the resource, this pins it in
# the collector, and they are two literals. It also catches nothing when the
# run fails before reaching that assert. The census below covers neither, and
# is not meant to -- it answers what the labels actually are.
# The `cnpg.io/cluster` selector has no pin at all; it rides on CNPG's
# primary label instead, which is the reason for choosing it.
- script:
# The helper bounds each kubectl attempt with --request-timeout, which on
# a degraded cluster costs about five times the flag; one call is ~75s at
# worst, and this op timeout sits above that so an inner budget is what
# fires. A SIGKILL at the op boundary would take the census out mid-write
# and yield nothing, where a kubectl that gives up leaves an error to
# print. The helper carries the rest of the rationale and is unit-tested
# in hack/pod-label-census_test.bats.
timeout: 2m
content: |
set -eu
cd ../../..
. hack/e2e-chainsaw/_lib/pod-label-census.sh
pod_label_census "$NAMESPACE"
# harbor-test readiness is gated by the inline RedisFailover
# harbor-test-redis: its three sentinels stay NotReady until the spotahome
# redis-operator elects a master and rewrites their monitored-master config.
# None of the collectors above see that path: the instance selector above
# matches neither the operator (namespace cozy-redis-operator) nor the
# rfr-/rfs- pods, which carry `harbor-test-redis`, and the RedisFailover CRD
# ships no status subresource, so there is nothing to assert on declaratively.
- describe:
apiVersion: databases.spotahome.com/v1
kind: RedisFailover
name: harbor-test-redis
- podLogs:
namespace: cozy-redis-operator
selector: app.kubernetes.io/name=redis-operator
tail: 200
- script:
# A script rather than more podLogs entries: the -o wide pod tables and
# the rfr-/rfs- name-prefix match (independent of the operator's
# pod-label scheme) are not expressible declaratively. Every command is
# best-effort (|| true) so diagnostics never mask the test's own error;
# no inner timeout wrappers, so the 2m op timeout is the only budget.
timeout: 2m
content: |
set -u
echo "== redis-operator pods (cozy-redis-operator) =="
kubectl -n cozy-redis-operator get pods -o wide || true
echo "== rfr-/rfs- pod states ($NAMESPACE) =="
kubectl -n "$NAMESPACE" get pods -o wide | grep -E 'rf[rs]-harbor-test-redis' || true
for p in $(kubectl -n "$NAMESPACE" get pods -o name | grep -E 'rf[rs]-harbor-test-redis'); do
echo "== logs $p =="
kubectl -n "$NAMESPACE" logs "$p" --all-containers --tail=100 --prefix || true
done
steps:
- name: create-harbor
try:
- apply:
file: harbor.yaml
- name: wait-helmrelease-ready
try:
# harbor-test readiness is gated by the inline RedisFailover, whose three
# sentinels only go Ready after the freshworks-oss redis-operator elects a
# master and rewrites their config; normal convergence is 2-3m, but on a
# CPU-committed runner the tail runs long (steal-induced probe misses,
# documented in pull-requests.yaml). 11m sits above the 600s install
# timeout cozystack-api gives this release, so a failed install has
# recorded its reason first; see "Sizing an HR-Ready budget" in
# docs/agents/e2e-testing.md. On a timeout, the catch block's
# redis-operator log shows which reconcile gate stalled.
- assert:
timeout: 11m
resource:
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: harbor-test
status:
(conditions[?type == 'Ready']):
- status: "True"
- name: wait-cosi-bucket
try:
# 2m, not the default 5m: the objectstorage-controller requeues the
# BucketClaim until the backend Bucket is ready, so readiness converges
# within tens of seconds; a claim still not ready after 2m is frozen
# (readiness-propagation regression), and the tight bound surfaces that
# instead of masking it (mirrors the bats 120s budget from 63d14e748).
- assert:
timeout: 2m
resource:
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketClaim
metadata:
name: harbor-test-registry
status:
bucketReady: true
- assert:
timeout: 1m
resource:
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketAccess
metadata:
name: harbor-test-registry
status:
accessGranted: true
- name: wait-system-and-deployments
try:
# 16m: this release is rendered by
# packages/apps/harbor/templates/harbor.yaml, whose spec.timeout of 15m is
# its install timeout; see "Sizing an HR-Ready budget" in
# docs/agents/e2e-testing.md.
- assert:
timeout: 16m
resource:
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: harbor-test-system
status:
(conditions[?type == 'Ready']):
- status: "True"
- assert:
timeout: 2m
resource:
apiVersion: apps/v1
kind: Deployment
metadata:
name: harbor-test-core
# Pins the label the catch collector selects on. Harbor's matchLabels
# carry only release+app, so pin the pod template, which is what
# podLogs actually matches. Asserts are subset matches, so this costs
# nothing and turns a silent drift of the instance label into a red
# step instead of an empty capture.
spec:
template:
metadata:
labels:
app.kubernetes.io/instance: harbor-test-system
status:
(conditions[?type == 'Available']):
- status: "True"
- assert:
timeout: 2m
resource:
apiVersion: apps/v1
kind: Deployment
metadata:
name: harbor-test-registry
status:
(conditions[?type == 'Available']):
- status: "True"
- assert:
timeout: 2m
resource:
apiVersion: apps/v1
kind: Deployment
metadata:
name: harbor-test-portal
status:
(conditions[?type == 'Available']):
- status: "True"
- name: verify-credentials-and-service
try:
- assert:
resource:
apiVersion: v1
kind: Service
metadata:
name: harbor-test
spec:
(ports[0].port): 80
- script:
content: |
set -eu
pw=$(kubectl -n "$NAMESPACE" get secret harbor-test-credentials \
-o jsonpath='{.data.admin-password}' | base64 -d)
url=$(kubectl -n "$NAMESPACE" get secret harbor-test-credentials \
-o jsonpath='{.data.url}' | base64 -d)
test -n "$pw"
echo "$url" | grep -q 'https://'
check:
($error == null): true