mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
Mattia Eleuteri
f859b710a4
Addresses the four findings on #3524. **The `fail` in the bucket chart (MAJOR).** The <bucket>-<user>-credentials Secret was made to `fail` the render while its COSI lookup was empty, so the release would retry and converge instead of skipping the Secret permanently. Helm cannot render a partial set, so that fail aborts the WHOLE <bucket>-system release: the other users' Secrets, and the bucket UI Deployment/Service/Ingress/HTTPRoute with them. One declared user whose BucketAccess never provisions takes down every other user of that bucket, and on an already-installed release parks it in Failed, blocking every later upgrade. **No per-bucket escape (MAJOR).** Rather than add one, remove the need for it. The gate already forces the release that renders the lookup-gated Strategy CRs; the credentials Secret is the same trap one release earlier, and the gate reads that Secret to resolve the bucket name in the first place. So while it is absent, or carries no bucket name, force the bucket's <bucket>-system release instead of doing nothing. That cannot deadlock its own precondition: the BucketClaim and the BucketAccess whose COSI Secret the lookup reads are rendered unconditionally by the parent release, not by the one being forced. The two releases are throttled independently — they are forced in sequence on a bootstrap, so a shared timestamp would delay the second by a full MinForceInterval for no reason. Coordinates come from the chart under provisionBucket, and are omitted on external S3 where the Secret is admin-managed and no release renders it. No RBAC delta: patch on helmreleases is already cluster-scoped. The chart goes back to a partial render, with the comment now explaining why the skip is not self-healing and why failing is the wrong lever, and a test that renders the UI with every user unresolvable so a reintroduced `fail` fails the suite. Tenant buckets keep their pre-existing behaviour; generalising the repair to every bucket needs a controller that owns them and is tracked separately. **Stale gauge (MINOR).** cozystack_backup_default_objects_missing was only written on the happy path, so the state the gate exists to catch reported 0 and the documented alert never fired. It is now written on the unresolved path too — the credentials Secret counts as one of the objects the default backups depend on, which is what it always was. On an API error the gauge is deliberately left alone rather than flapping, so a new cozystack_backup_default_objects_check_errors_total marks it stale, and the doc pairs the two instead of overstating the gauge. **Suspended releases (MINOR).** forceHelmRelease now does a point Get and skips a release with spec.suspend: true. helm-controller ignores forceAt and requestedAt while suspended (`cozyhr suspend` sets exactly that), so the gate was re-stamping every MinForceInterval for the whole suspension and climbing the force counter — which the runbook attributes to a render that is not producing the objects, the wrong diagnosis. The skip is logged and not counted, so a climbing counter keeps its documented meaning. Reported-by: IvanHunters <xorokhotnikov@gmail.com> Signed-off-by: Mattia Eleuteri <mattia@hidora.io> Assisted-By: Claude <noreply@anthropic.com>
157 lines
6.1 KiB
YAML
157 lines
6.1 KiB
YAML
suite: DefaultObjectsGate wiring — the release the gate forces, and its RBAC
|
|
|
|
# The Strategy CRs and the Velero BSL are rendered behind a `lookup` of the
|
|
# BucketClaim this same chart creates, so a fresh install renders them empty
|
|
# and helm-controller never re-renders (unchanged chart + values, drift
|
|
# detection off) — the skip is permanent. The controller's DefaultObjectsGate
|
|
# repairs that by forcing a real Helm upgrade on THIS release, so the release
|
|
# coordinates it is given must be this chart's own release, and it must hold
|
|
# the verbs to do it.
|
|
|
|
templates:
|
|
- templates/deployment.yaml
|
|
- templates/rbac.yaml
|
|
|
|
release:
|
|
name: backupstrategy-controller
|
|
namespace: cozy-backup-controller
|
|
|
|
tests:
|
|
- it: "gate targets this chart's own HelmRelease and the cozy-default BackupClass"
|
|
template: templates/deployment.yaml
|
|
asserts:
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_HELMRELEASE_NAME
|
|
value: backupstrategy-controller
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_HELMRELEASE_NAMESPACE
|
|
value: cozy-backup-controller
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_BACKUPCLASS
|
|
value: cozy-default
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_VELERO_NAMESPACE
|
|
value: cozy-velero
|
|
|
|
# The credentials Secret is rendered one release earlier, by the platform
|
|
# bucket's <bucket>-system release, behind the same kind of install-time
|
|
# lookup. Its name is assembled from the bucket-rd release prefix and the
|
|
# "-system" suffix the parent bucket chart appends, and it must stay in
|
|
# lockstep with systemSecretName: the gate resolves the bucket name from
|
|
# that Secret and forces exactly this release when it is absent. Two
|
|
# independently written strings, so pin the relationship, not just the value.
|
|
- it: "gate targets the bucket release that renders the credentials Secret"
|
|
template: templates/deployment.yaml
|
|
asserts:
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_CREDENTIALS_HELMRELEASE_NAME
|
|
value: bucket-cozy-backups-system
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_CREDENTIALS_HELMRELEASE_NAMESPACE
|
|
value: tenant-root
|
|
# BACKUP_STORAGE_SECRET_NAME is exactly the release name above with
|
|
# "-credentials" appended — that is what makes forcing this release the
|
|
# thing that creates that Secret.
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_STORAGE_SECRET_NAME
|
|
value: bucket-cozy-backups-system-credentials
|
|
|
|
- it: "a renamed bucket moves both the forced release and the Secret together"
|
|
template: templates/deployment.yaml
|
|
set:
|
|
backupStorage.bucketName: platform-backups
|
|
backupStorage.systemSecretName: bucket-platform-backups-system-credentials
|
|
asserts:
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_CREDENTIALS_HELMRELEASE_NAME
|
|
value: bucket-platform-backups-system
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_STORAGE_SECRET_NAME
|
|
value: bucket-platform-backups-system-credentials
|
|
|
|
# External S3: the Secret is admin-managed and no release renders it, so
|
|
# there is nothing to force. Handing the gate stale coordinates would make
|
|
# it patch a HelmRelease that does not exist every MinForceInterval.
|
|
- it: "provisionBucket=false: no bucket release to force"
|
|
template: templates/deployment.yaml
|
|
set:
|
|
backupStorage.provisionBucket: false
|
|
asserts:
|
|
- notContains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_CREDENTIALS_HELMRELEASE_NAME
|
|
value: bucket-cozy-backups-system
|
|
- notContains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_CREDENTIALS_HELMRELEASE_NAMESPACE
|
|
value: tenant-root
|
|
|
|
- it: "velero.bslEnabled=false: the BSL is not rendered, so the gate must not look for it"
|
|
template: templates/deployment.yaml
|
|
set:
|
|
velero.bslEnabled: false
|
|
asserts:
|
|
- notContains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_VELERO_NAMESPACE
|
|
value: cozy-velero
|
|
# The rest of the gate stays wired: the Strategy CRs still need it.
|
|
- contains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_HELMRELEASE_NAME
|
|
value: backupstrategy-controller
|
|
|
|
- it: "reconcileDefaultObjects=false: the gate is fully unwired (main.go disables it on an empty release name)"
|
|
template: templates/deployment.yaml
|
|
set:
|
|
backupStorage.reconcileDefaultObjects: false
|
|
asserts:
|
|
- notContains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_HELMRELEASE_NAME
|
|
value: backupstrategy-controller
|
|
- notContains:
|
|
path: spec.template.spec.containers[0].env
|
|
content:
|
|
name: BACKUP_DEFAULT_OBJECTS_BACKUPCLASS
|
|
value: cozy-default
|
|
|
|
- it: "ClusterRole grants patch on helmreleases (forceAt stamp) and get on backupstoragelocations"
|
|
template: templates/rbac.yaml
|
|
asserts:
|
|
- contains:
|
|
path: rules
|
|
content:
|
|
apiGroups: ["helm.toolkit.fluxcd.io"]
|
|
resources: ["helmreleases"]
|
|
verbs: ["get", "create", "update", "patch", "delete"]
|
|
- contains:
|
|
path: rules
|
|
content:
|
|
apiGroups: ["velero.io"]
|
|
resources: ["backupstoragelocations"]
|
|
verbs: ["get"]
|