mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-04 02:57:27 +09:00
feat/ca-extraction-controller
5079
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
156ded19d0 |
refactor(controller): use new() for bool-literal owner-reference flags
Go 1.26's extended new() builtin takes an expression, so new(true) and new(false) replace ptr.To for the owner reference's Controller and BlockOwnerDeletion flags. ptr.Deref, which has no builtin equivalent, stays. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
8f2395ea69 |
fix(controller): flag a missing tenant-ca label at the canonical name
A key-free Secret already sitting at the canonical trust-anchor name is left untouched and reported Ready. But tenant visibility comes from the internal.cozystack.io/tenant-ca label, and an engine-owned Secret at that name carries none, so the lineage webhook marks it tenantresource=false and the tenant is locked out while the controller reports "published". Add the missing label to the canonical-name contract deviations so the warning names it, and collect all deviations rather than only the first so a single warning describes every way the object falls short. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d0af407744 |
perf(controller): gate the ApplicationDefinition watch on spec.secrets
The ApplicationDefinition mapping fans out to every TenantProjection in the cluster on every definition event, including Flux's periodic no-op re-apply of the many *-rd definitions. The selectors-digest drift check already suppresses the resulting writes, but not the load: each enqueued reconcile still issues several uncached reads. Gate the watch with a predicate that delivers an update only when spec.secrets — the field the selectors digest is computed from — changed, while letting creates and deletes through. Comparing the same field the digest digests keeps the gate and the digest from drifting apart. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b8b3b1945b |
docs(controller): describe both projection withdrawal mechanisms
The package doc claimed there is no withdrawal logic and that a retired declaration is only ever a deleted sentinel. withdrawProjection has since become the second mechanism: it deletes a projection when the sentinel outlives its CACert entry, which owner-reference garbage collection never reaps because the owner still exists. State both paths. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
a121bec10b |
fix(controller): resolve an empty sourceSecretName to a condition
An entry with an empty sourceSecretName sent the reconciler to Get a Secret with an empty name. The apiserver answers that with "resource name may not be empty" — not NotFound — so it escaped the not-found branch, was returned from Reconcile, and wedged the sentinel on exponential backoff. The error path returns before any status write, so no Ready condition was ever recorded: the silent retry the sentinel was introduced to make visible. Pin sourceSecretName to MinLength=1 so a live apiserver rejects it at admission, and guard the reconcile so an object admitted by an older CRD resolves to Ready=False/SourceInvalid instead of an error. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
dd8ee25db4 |
fix(controller): withdraw a projection this sentinel owns by name
withdrawProjection recognised its projection by an exact owner-reference match while adoption uses a name match, so the two disagreed about what counts as "ours" — and withdrawal was the stricter, which is backwards: "safe to overwrite" is a stronger claim than "safe to delete". Two projections a name match adopts and rewrites were read as foreign by the exact match and left in place: one carrying a previous incarnation's UID (delete-and-recreate under the same name), which owner-reference garbage collection never reaps once the CACert entry is gone, so the retired anchor stayed tenant-readable forever; and one whose BlockOwnerDeletion flag had drifted, the very drift the adoption path normalizes. Decide ownership with isOurProjection so both paths answer the question the same way. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1f9736226f |
fix(controller): store the full selectors digest and correct the withdraw note
The selectors digest kept only the first 8 bytes of the SHA-256. A collision skips the re-admission the doc calls load-bearing — the revocation direction, where a tenant would silently keep read access after a definition stopped selecting the anchor — and the annotation has no length limit, so store the full hash. Also correct the withdraw-branch comment: MinItems=1 prevents a single-entry sentinel from emptying its list, so the branch is reached when a future non-CACert entry replaces the CACert one, not by gating a lone entry off. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4f02c6ae1b |
fix(controller): requeue a contested sentinel so it recovers on its own
The two-sentinel refusal returned without a RequeueAfter. NoRelease and MultipleCACert can do that because their fix edits the same sentinel, which the For() watch delivers; a contest is cleared by deleting the SIBLING sentinel, and nothing enqueues this one when a sibling changes. So after an operator removed the duplicate, the surviving sentinel withheld its anchor until the informer's global resync — hours. Return RequeueAfter: resyncInterval on the contest branch so recovery rides the resync backstop, and clarify that an anchor an earlier uncontested reconcile already published is left in place rather than withdrawn. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
612a73971a |
test(e2e): compare CA projection by certificate fingerprint, not raw PEM
Both suites compared the projection's ca.crt against the source PEM byte for byte. The controller re-encodes ca.crt from the DER it parsed, so the two match today only because openssl and the re-encoder both emit header-free 64-column PEM; a header line or column-width change upstream would redden a correct projection. Compare the parsed certificates via openssl x509 -noout -fingerprint instead, which is identity-stable across any re-encoding. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d558920c84 |
docs(cozystack-basics): distinguish the two VAP capability-gate cases
The dependsOn comment warned that a capability gate renders the policy out at first install and never re-adds it, which read as contradicting the .Capabilities.APIVersions.Has guard the policy templates now carry. They do not conflict: the warning is about gating on the matched types' CRDs, which may not exist yet at first install, while the template guard names the ValidatingAdmission Policy API itself — GA on every supported cluster, so always present. Spell out that a maybe-absent matched-type CRD needs dependsOn ordering while an always-present core API can be gated in the template. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6bf2390c54 |
test(e2e): run the cacert suite when the writer policy changes
select-e2e.sh mapped reverse dependencies to Chainsaw suites only through *-application sources and a hardcoded external-dns, so a change confined to the tenant-projection writer policy (shipped by cozystack-basics) selected every suite that depends on it but not cacert — whose Chainsaw is the policy's only live deny/allow proof. Add the cozystack-basics reverse mapping, mirroring the external-dns precedent, and pin both directions: a cozystack-basics change selects cacert, and a cacert run installs cozystack-basics. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
17653906a6 |
refactor(controller): rename CA-cert annotations and match app group
The projection carries two annotations whose keys still named a publish-ca-cert discovery leg that no longer exists in the tree. Rename them to internal.cozystack.io/ca-cert-source and internal.cozystack.io/ca-cert-selectors while the API is unshipped; both stay functional (the selectors digest drives the re-admission drift check, the source ref is a drift input and a traceability record). Resolving the ApplicationDefinition matched on Kind alone from a List of unspecified order. Gate the match on the release's group label too, so a release stamped with an unexpected group cannot resolve a same-kind definition by accident. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
9ac06cd1fc |
fix(controller): withdraw the CA projection and refuse contested releases
Two gaps in the sentinel-to-canonical-name lifecycle: Withdrawal — the reconciler published each declared entry but never deleted one. A sentinel that keeps existing with its CACert entry removed (a chart gating it on tls.enabled, say) left the old <release>.tenant-ca served forever, because owner-reference GC only fires when the whole sentinel is pruned. A sentinel that reconciles with zero CACert entries now deletes the projection at the canonical name, but only when it still carries that sentinel's owner reference, so a foreign Secret or another sentinel's projection is never touched. Contention — two sentinels in one namespace with the same release label both resolve to the single canonical name. Whichever reconciled first would publish an arbitrary CA and the two could flap ownership of one projection. Refuse both, mirroring the more-than-one-CACert-entry refusal, so nothing is published until the declaration lives on exactly one sentinel. The ClusterRole comments claimed the reconciler never deletes a Secret; corrected to describe the owner-scoped withdrawal delete. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e6e68fadf6 |
fix(controller): wake the reconciler when the CA projection is tampered
The only watches on the projection Secret were the source-Secret index (keyed on
spec.projections.sourceSecretName) and the sentinel itself. The projection is
named <release>.tenant-ca, which is no sentinel's source, so an in-place rewrite
of its ca.crt was corrected only by the five-minute resync — a window in which a
forged trust anchor is served.
Map the projection back to its owning sentinel through the controller owner
reference it carries, on the same dedicated metadata cache the source watch uses.
Owns(&corev1.Secret{}) would register against the manager cache, whose Secret
informer is label-scoped to the WildcardSecret replicas, so a projection carrying
no wildcard label would never be delivered — it would compile and pass a fake
client yet never fire in a real cluster.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
9be5f51083 |
fix(controller): adopt the CA projection by owner reference, not a strippable label
isOurProjection gated adoption on the ManagedLabel marker plus the Opaque type. The marker is an ordinary label any Secret writer in the namespace can strip: stripping it off the genuine projection made the controller disown its own object, refuse it as a collision, and keep serving whatever bytes the same actor wrote under ca.crt — a trust anchor stuck forged and never self-healing. Key adoption on the owner reference back to this sentinel instead, matched by name so a delete-and-recreate under the same name is still re-homed. Forging the reference only causes an adopt-and-overwrite with the genuine key-free CA, the self-healing direction; a Secret with no such reference is a stranger and is refused as a collision, never overwritten. The Opaque type stays as a cheap sanity guard but is no longer load-bearing — the update path is a read-modify-write that never writes the immutable Type. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
9f924101b6 |
test(controller): cover the sentinel discovery, status, and GC paths
Rewrite the cacert unit suite for the TenantProjection model: the happy-path projection, the Ready=False/SourceNotFound status a missing source now surfaces, ownership solely by the sentinel, the write-path guards driven through a real source (private key and non-certificate refused), the canonical-name cases, and the forged-projection heal. The pure write-path guard tests — including the DER-reassembly guarantee — carry over unchanged. Replace the cacert chainsaw suite: it probes the writer policy against the live API server (RBAC denies a tenant, the VAP denies a non-flux writer, flux is allowed) and drives the controller end to end (a flux-created sentinel reports SourceNotFound, then lifts a real CA into a key-free, sentinel-owned projection). Drop the declared-leg wiring test. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
93b496be95 |
feat(cozystack-basics): pin the TenantProjection writer to flux
Replace the publish-ca-cert label-leg admission control with a policy on the sentinel. A ValidatingAdmissionPolicy restricts TenantProjection CREATE and UPDATE to the flux ServiceAccount helm-controller runs as, so a tenant (who has no RBAC on internal.cozystack.io) cannot forge a declaration that publishes a CA of their choosing. DELETE stays ungated for owner-ref GC. cozystack runs its flux controllers under one shared cozy-fluxcd:flux ServiceAccount, so the pin admits any flux controller — a deliberate platform-tier widening, since only helm-controller renders charts and no tenant reaches this group. The literal is guarded by a bats check on its shape, name and namespace. Remove the label-leg policy, its unit test, its pin check and its chainsaw suite — the sentinel model retires that leg. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
4503538f91 |
feat(postgres): render a TenantProjection for the CA trust anchor
Postgres is the first consumer of the sentinel model. The chart renders a TenantProjection named <release>-ca declaring CNPG's <release>-ca as the CACert source; the CA-extraction controller lifts ca.crt alone into the tenant-readable <release>.tenant-ca projection and owns it through the sentinel. Drop the spec.caCert declaration leg from the ApplicationDefinition API and from the postgres definition — the sentinel replaces it. The tenant-ca include selector that makes the projection tenant-visible is unchanged. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
b41b6cce89 |
feat(controller): discover the CA source through a TenantProjection sentinel
Drive the CA-extraction reconciler off the namespaced TenantProjection sentinel a chart renders, replacing the publish-ca-cert label leg and the cluster-scoped spec.caCert declaration leg. The sentinel names the source Secret and key; the projected trust anchor takes an owner reference to the sentinel, so the lineage webhook resolves the tenant through sentinel -> HelmRelease and garbage collection prunes the projection when the chart stops rendering the sentinel. The source Secret is watched as metadata only, so the informer holds no private key material, and the one source actually read goes through the uncached reader. The sentinel's Ready status makes a missing or unpopulated source visible with kubectl instead of a silent requeue. The write-path guard, canonical-name contract, adoption rules and selectors-digest re-admission are unchanged. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
564e1d95d9 |
feat(api): add TenantProjection CRD for declared tenant-visible projections
Introduce the namespaced internal.cozystack.io/v1alpha1 group and its TenantProjection kind. A chart renders this sentinel to declare which of an application's Secrets should be published to its tenant as a key-free copy; the controller reconciles the projected Secret from it. No tenant RBAC role grants any verb on internal.cozystack.io, so the sentinel cannot be forged from a tenant namespace. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d2944051c9 |
fix(cozystack-basics): gate ValidatingAdmissionPolicies on API availability
The four VAP policies in this chart (publish-ca-cert-writer plus the route, gateway, and ingress hostname policies) rendered unconditionally, so a cluster whose apiserver does not serve the ValidatingAdmissionPolicy API would receive an unrenderable resource and fail the cozystack-basics install or upgrade. Wrap all four in the same .Capabilities.APIVersions.Has guard the platform deletion-protection template already uses, and cover both the served and unavailable cases in helm-unittest. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ad1f40f8ff |
test(cacert): reclaim writer-policy probe resources on partial setup
The writer-policy suite runs against the shared, pre-existing tenant-test namespace, which Chainsaw never reclaims, so every object the script creates must be pruned by its own trap. Arm that trap before the RBAC apply, so a partial apply that creates the Role but not the RoleBinding cannot leak the privileged Role into the shared namespace, and delete vap-ca-source-forged too, since it is created precisely when the deny path regresses and the test fails. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f73ffc54a8 |
test(cacert): surface cleanup failures in the writer-policy suite
The script step grants the tenant default ServiceAccount Secret write via a RoleBinding so its probe requests reach admission, and Chainsaw does not reclaim objects created with kubectl inside a script step, so a trap prunes them by hand. Masking those deletes with `|| true` let a failed delete leave the elevated RoleBinding behind for the next suite. Run every delete, preserve the test's own exit status, and fail the step on a genuine cleanup error instead of swallowing it. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
fa7372a554 |
test(cacert): map the e2e suite to its install source
hack/select-install.sh --validate requires every chainsaw suite directory to resolve to a PackageSource, and the new cacert suite had no mapping, so the install-side test-impact validation — run by the unit-test gate — failed. Map it to cozystack.cozystack-basics, the package that ships the policy the suite exercises, mirroring the securitygroup suite's mapping to its own controller package. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
7e7cf7a377 |
docs(cozystack-basics): record the source/destination gating asymmetry
The writer policy gates the CA-source surface but not the destination surface — the <release>.tenant-ca projection a tenant reads. Document why that is accepted rather than closed: a platform-tier writer that could forge a projection already has an ungated path through the declared-leg source, so it is unequal depth rather than an escalation; a destination gate could not be a clean single-writer pin because the tenant-ca label has two legitimate producers (this controller and helm-controller); and the surgical fix is a UID check in lineage owner resolution, tracked as separate follow-up work. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
c106a28026 |
feat(api): constrain the application release prefix to dot-free characters
The CA-extraction controller projects a tenant's trust anchor to "<release>.tenant-ca" and relies on the dot being a separator no release name can contain: release names are "<prefix><app name>", app names are DNS-1035 labels with no dot, so the projection name is unambiguous only while the prefix is dot-free too. That property was asserted in the controller but never enforced on the field it depends on. Add a validation pattern restricting the prefix to lowercase DNS-1123 characters, which excludes the dot by construction. Every shipped application definition already conforms. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
82c959ac4d |
fix(cozy-lib): anchor the CA trust-anchor guard to the whole value
The caCertSecret helper emits ca.crt verbatim but only checked that a certificate block appeared SOMEWHERE in the value. Bytes before the first block or after the last one — a human-readable preamble, or a raw DER/JWK key that carries no PEM private-key header and so slips the key guard — therefore passed the check and were published to the tenant inside the trust anchor. Anchor the certificate regex to the entire value, so it matches only one or more complete blocks surrounded by whitespace and refuses anything else. This mirrors the CA-extraction controller, which reaches the same result by rebuilding the projection from the parsed blocks. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d298d6edd4 |
fix(cozystack-controller): normalize BlockOwnerDeletion drift on the projection owner
releaseOwnerRef pins BlockOwnerDeletion to false so a trust anchor never holds up the teardown of the application it belongs to, but the ownership drift check compared every owner-reference field except that one. A sole owner reference whose only drift was BlockOwnerDeletion flipped to true therefore passed as already-correct and was never re-homed, leaving a projection that blocks a foreground deletion of its HelmRelease — the opposite of what the flag is set for. Include BlockOwnerDeletion in the owner comparison so the write path replaces a drifted reference and restores the flag to false. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
2855ba4677 |
test(cozystack-basics): drive the CA writer policy deny path through admission
The CA-source writer policy is two CEL expressions, and until an engine converges onto the label leg nothing exercises them against the API server: helm-unittest pins the expressions verbatim and the bats check pins the writer identity, but a CEL that is wrong as written would satisfy both while admitting a forged trust-anchor source. Add a chainsaw suite that impersonates a non-cert-manager identity and asserts the API server denies both attack shapes — forging a new labelled source and stripping the label off an existing one to evade the gate — while cert-manager itself stays allowed, which also guards against a policy that denies everyone. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1a7339537f |
fix(cozystack-controller): rebuild CA projection from validated blocks
The write-path guard validated the CA value as a certificate chain but then published a clone of the original input. pem.Decode silently skips bytes before the first PEM block and between blocks, and the private-key header guard only matches a "-----BEGIN ... PRIVATE KEY-----" line, so a raw DER or JWK private key placed ahead of or between real certificates passes both checks and is copied verbatim into the tenant-readable projection. Assemble the projection from the re-encoded certificate DER instead of cloning the input, so preamble and interstitial key material are structurally unreachable. The empty-value, wrong-type, non-parseable and trailing-byte rejections are unchanged. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6eeac7f880 |
test(cozystack-basics): run the CA writer-pin check under POSIX sh
hack/cozytest.sh runs each .bats file under /bin/sh, which is dash on the CI runner. The publish-ca-cert writer-pin check used bash [[ ]] string-glob conditionals, so the file aborted with "[[: not found" before any assertion executed — reddening the unit-test job. Rewrite both assertions as POSIX case statements, which glob-match identically in dash and bash. What the check pins is unchanged. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ca8863470d |
chore: ignore Go binaries built into the repository root
`go build ./cmd/<name>` without -o writes the executable to the working directory under the package name, so each of the thirteen packages under cmd/ has a root-level binary waiting to happen. Nothing in .gitignore matched them: an ~80 MB Mach-O sat untracked in a working tree, one `git add .` from being committed against the repo's own rule about built artifacts. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e8631ae920 |
feat(cozystack-basics): restrict CA-source Secret writes to cert-manager
The CA-extraction controller's label-driven leg treats a core Secret labelled internal.cozystack.io/publish-ca-cert as a trust-anchor source and lifts its ca.crt into the tenant-readable projection. That label is the whole attribution on that leg and sits on an ordinary Secret, so without a platform-side control any principal that can write a Secret in the namespace could forge a source or rewrite an existing one's ca.crt and swap the trust anchor a tenant is handed as vouched. Add a ValidatingAdmissionPolicy that allows a Secret carrying that label to be created or updated only by cert-manager's controller ServiceAccount, the sole legitimate writer of these Secrets. It covers both the forge-new (CREATE) and tamper-existing (UPDATE, including a relabel-away evasion via the old object) shapes, and by keying on the publish-ca-cert label it never touches the declared-leg source, the projection, or any unlabelled Secret. The allowed identity is pinned as a literal to the shipped cert-manager package's controller ServiceAccount; a comment marks it as the fragile seam that must move if cert-manager is repackaged. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f2c2abfe88 |
feat(postgres): publish the CA trust anchor to tenants
Declare CNPG's operator-created CA Secret as the postgres trust-anchor
source (spec.caCert names {{ .release }}-ca), and add the generic
internal.cozystack.io/tenant-ca selector to spec.secrets so the key-free
<release>.tenant-ca projection the extraction controller publishes becomes
readable by the tenant.
postgres is the first converged engine, and the hardest: CNPG creates the
CA Secret itself, key-bearing, asynchronously, and offers no way to label
it, so it exercises the name-driven leg end to end. The test drives the
shipped ApplicationDefinition rather than a fixture, so the wiring and the
controller cannot drift apart unseen: a CNPG-shaped key-bearing
<release>-ca appears, and the projection must be produced key-free AND
selected for tenant visibility by the shipped selector.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
||
|
Aleksei Sviridkin
|
689e493ccf |
feat(cozy-lib): converge the trust-anchor helper on the canonical anchor
The CA-extraction controller publishes every engine's trust anchor as an Opaque "<release>.tenant-ca" Secret labelled internal.cozystack.io/tenant-ca. That label is the single, engine-agnostic selector an ApplicationDefinition puts in spec.secrets, and it is what ultimately reaches the tenant. This helper is the other producer of that same object, for charts that already hold the CA PEM at render time. It agreed with neither half of the contract: it documented "<release>-ca-cert" as the name, and it stamped no tenant-ca label at all. Both are now fixed, and both mattered. The name "<release>-ca-cert" is not merely a second choice, it is unusable: Percona Server for MongoDB creates a Secret of exactly that name itself and puts a private key in it, so a chart rendering a trust anchor there would be writing over live key material. Without the tenant-ca label, the two producers would converge on the name but not on the label. A helper-rendered anchor would be missed by the generic selector, the lineage webhook would mark it tenantresource=false, and the tenant would be locked out of the very Secret the helper exists to publish — while a controller-projected one, of the same name, worked. Also record what the tenantresource label does and does not do. The lineage webhook recomputes it from the definition's spec.secrets selectors on every admission and overwrites whatever the chart wrote, so rendering it is necessary but not sufficient for tenant read access. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d7232831f4 |
feat(cozystack-controller): run the CA-extraction reconciler
Wire the reconciler into the platform controller manager and grant its ServiceAccount the Secret verbs the projection needs: create/update/patch to publish the trust anchor, and delete to withdraw it when an application stops publishing one. Read was already covered by the catch-all rule, and no tenant RBAC is widened. Give the reconciler its own dedicated, scoped Secret cache rather than narrowing the manager's shared cache. Only labelled CA sources are held in that informer, so a cluster-wide Secret watch never caches every Secret, and so every private key, in the cluster; the operator-owned name-driven sources and the projections are read through the uncached API reader instead. A private cache also keeps this scoping independent of any other controller on the same manager that needs to scope Secrets differently. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
08ad87e24f |
feat(controller): publish a key-free CA trust anchor per application
Every engine in the catalog issues a per-release CA, but the object that holds ca.crt almost always holds a private key next to it, so any RBAC path that hands a tenant the trust anchor hands over key material too. Add a controller that publishes, for each application release, a canonical Opaque Secret <release>.tenant-ca holding only ca.crt. That name is claimed by none of the operators the platform ships, which matters: <release>-ca-cert, the obvious candidate, is already taken and means opposite things across engines — a key-free CA on one, a key-BEARING Secret on another — so projecting onto it could only be done by overwriting live key material. The certificate is found two ways. A Secret labelled internal.cozystack.io/publish-ca-cert is how a chart opts in, and it is watched, so a rotation propagates at once. For the engines whose operator owns the CA Secret and will not label it, the application's caCert declaration names that Secret, which is read by name and re-read on a resync. The two legs differ only in how the source is found — extraction, sanitization, ownership and collision handling are one shared write path, because forking it is where a key-leak bug would hide. Exactly one whitelisted key is ever copied, and every byte written is re-checked for a PEM private-key header first: on the name-driven leg the source is key-bearing by construction. The projection is owner-referenced to the release, so deleting the application collects it with no prune logic, and a Secret the controller did not create is never overwritten — a Secret carrying key material on the canonical name is refused with a Warning Event rather than adopted. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5da3fab457 |
feat(api): declare where an application's CA lives
The trust-anchor contract is a label on the CA Secret, but two engines cannot honour it. CloudNativePG builds its PKI Secrets without applying spec.inheritedMetadata, so the chart's labels never reach <release>-ca, and Percona Server for MongoDB exposes no secretTemplate at all and reconciles an out-of-band patch away. Add an optional ApplicationDefinition.spec.caCert declaring, per engine, the name of the Secret its operator creates and the key inside it that holds the certificate. Registering an engine is then data, not code. The declaration only names an object the platform already reads; nothing a tenant can read is widened by it, and a labelled Secret still takes precedence where an engine can label its own output. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
myasnikovdaniil
|
741a5acc4a |
fix(opensearch-operator): use RollingUpdate maxSurge=0 instead of Recreate (#3319)
## What this PR does
Fixes a Helm-upgrade failure in the `opensearch-operator` chart that
blocks the new release-upgrade E2E lane (added in #3276).
Upgrading from a chart that shipped no explicit strategy (v1.5.x) to a
v1.6.0-rc build fails validation immediately, helm-controller retries
it, and the HelmRelease never becomes Ready:
```
Helm upgrade failed for release cozy-opensearch-operator/opensearch-operator:
Deployment.apps "opensearch-operator-controller-manager" is invalid:
spec.strategy.rollingUpdate: Forbidden: may not be specified when strategy type is Recreate
```
Observed in the ["Upgrade E2E Test" job of run
29480724982](https://github.com/cozystack/cozystack/actions/runs/29480724982)
— `UpgradeFailed ... (x19 over 11m)`, with the HelmRelease never
reaching Ready.
### Root cause
#3040 (commit
|
||
|
myasnikovdaniil
|
960d853706 |
feat(api): typed waitStrategy/healthCheckExprs for kstatus readiness (#2642) (#3273)
## What Adds typed `waitStrategy` + `healthCheckExprs` fields so a package/app can declare the Flux helm-controller v1.5.0 kstatus health gate for the custom resource it wraps — making `HelmRelease.Ready` mean "the CR is actually healthy" instead of "helm applied the CR". This is the operator/apiserver foundation of the kstatus epic (#2642); the API surface was reviewed and approved by the API owner in #3263. Commits (per the review's "isolated step-0" request): 1. `build(deps)` — the dependency bump (see below). 2. `feat(api)` — the fields, plumbing, tests, and regenerated code. 3. `fix(api)` — match the openapi code-generator pin to the 0.35 runtime (fixes an aggregated-apiserver crash the bump surfaced). 4. `fix(api)` — declare `OpenAPIModelName` on the apps types so SSA resolves catalog kinds. 5. `docs(api)` — note that `disableWait` silently defeats `healthCheckExprs`. ## API surface Typed fields on **both** HelmRelease-building paths: - `ApplicationDefinition.spec.release` (apiserver / catalog apps → `convertApplicationToHelmRelease`) - `ComponentInstall` (operator / platform packages → `buildHelmReleaseSpec`) ```go // WaitStrategy maps to HelmReleaseSpec.WaitStrategy.Name — a deliberate scalar // simplification of the upstream {name} object. One of poller|legacy. WaitStrategy string // HealthCheckExprs maps to HelmReleaseSpec.HealthCheckExprs (the upstream // kustomize.CustomHealthCheck type verbatim). HealthCheckExprs []kustomize.CustomHealthCheck ``` `config.ResolveWaitStrategy` is shared by both builders and **couples the default**: `healthCheckExprs` are only evaluated under the `poller` strategy, so when expressions are set and no strategy is given, the generated HelmRelease defaults to `poller`. A package that sets only `healthCheckExprs` is therefore self-contained, independent of the controller's global default. ## The dependency bump (commit 1) The `WaitStrategy` / `kustomize.CustomHealthCheck` / `HelmReleaseSpec.HealthCheckExprs` types do not exist below `helm-controller/api` v1.5.0 — the runtime is already v1.5.0 (shipped by #2602), so this closes a Go-types-vs-runtime skew. It is **not** an isolated bump: v1.5.x requires **k8s 0.35 + controller-runtime 0.23**, cascading the k8s stack 0.34 → 0.35 (and the go directive to 1.26). cozystack replaces `k8s.io/apimachinery` with a fork carrying the still-open upstream `ConvertToVersion` patch (k/k#135537). That single commit rebases cleanly onto apimachinery v0.35.0; published at `cozystack/apimachinery@release-1.35-pr135537`, and `go.mod` pins its pseudo-version. **The repo builds with zero source changes under the new stack.** > Maintainer note: please confirm the `release-1.35-pr135537` fork branch naming, or fold it into the `cozystack` branch as you prefer — the pin can be updated to match. ## Not included (conscious YAGNI) `disableWait` is intentionally omitted. The only known need (the kubernetes tenant-addon deadlock) was resolved child-side, and `waitStrategy: legacy` is not equivalent (legacy still waits Helm-v3-style; `disableWait` skips waiting entirely). Add it later when a concrete leaf needs a true no-gate escape hatch. ## Tests - `config.TestResolveWaitStrategy` — the couple-the-default logic (unset+exprs→poller; explicit legacy honored; unset+no-exprs→nil). - Operator + apiserver cases mirror each other (`TestBuildHelmReleaseSpecHealthCheckExprs` / `TestConvertApplicationToHelmRelease_HealthCheckExprs`). - `go build ./...` clean, `go vet` clean, `gofmt` clean, codegen regenerated and committed. The 4 backupstrategy CRDs regenerate with benign k8s-0.35 PodSpec description drift (DynamicResourceAllocation alpha→stable, toleration Lt/Gt, resizePolicy note) — no functional change. ## Follow-up Phase 3 (per-leaf adoption — postgres/kafka/mongodb/etc. declaring their CEL via these fields) builds on this. The full CRD→CEL catalog is maintained in the epic's readiness audit. Refs #2642 ```release-note feat(api): add `waitStrategy` and `healthCheckExprs` to ApplicationDefinition and ComponentInstall, so a HelmRelease reports Ready only once the custom resource it wraps is actually healthy ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable CEL health-check expressions for application releases and package components. * Added `waitStrategy` support (`poller` and `legacy`) with automatic `poller` selection when health checks are configured and no strategy is set. * Propagated these settings into generated Helm releases, affecting readiness/wait behavior. * **Documentation** * Updated Application and PackageSource schemas to include `healthCheckExprs` and `waitStrategy`. * Extended related Kubernetes CRD documentation, including projected pod certificate annotations and workload reference fields. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
e73ad68a85 |
fix(backups): support cnpg-operator with barman plugin (#3313)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
Fixes #3300. Migrates CNPG Postgres backups off the deprecated native
`spec.backup.barmanObjectStore` (removed in CNPG 1.29; the
`barman-cloud-*` binaries it needs are absent from the `standard` image
variant keycloak-db pins, and the bare system-flavor tags apps/postgres
pins are deprecated upstream) onto the [CloudNativePG Barman Cloud
plugin](https://github.com/cloudnative-pg/plugin-barman-cloud), across
every Postgres backup surface: `apps/postgres`, `system/keycloak`, and
the `cozy-default-cnpg` BackupClass strategy driver. The chart renders
`spec.plugins` + a `barmancloud.cnpg.io/ObjectStore` CR; the
backup-controller driver SSA-applies an `ObjectStore` and patches
`spec.plugins`; `Backup`/`ScheduledBackup` use `method: plugin`. The
`postgres-operator` chart vendors and co-locates the
`plugin-barman-cloud` operator (with its cert-manager-issued mTLS) so
CNPG can discover it.
One platform-level change is required for the plugin to run and ships
with this PR (see the upgrade notes in
`docs/operations/backup-classes.md` and the release note below): the
CloudNativePG operator is bumped to 1.28.1 (also fixing an operator/CRD
version skew that broke all backups). The other prerequisite — the
LINSTOR scheduler admission webhook stripping
`initContainers[].restartPolicy` cluster-wide, which broke the plugin's
restartable sidecar — is already fixed on `main` (
|
||
|
myasnikovdaniil
|
cc66e22952 |
fix(ci): mirror and retag images whose digest is embedded in the values tag (#3375)
## What's broken
`collect_refs` in both `hack/nightly-mirror.sh` and
`hack/promote-retag.sh` recognises three values shapes. The one most
package Makefiles actually write is not among them:
```yaml
image:
repository: <registry>/linstor-csi
tag: main@sha256:<digest> # digest embedded in the tag
```
Package Makefiles set `.image.tag` to `"$(IMAGE_TAG)@$(digest)"` in a
single `yq` call rather than maintaining a separate `digest` key. That
matches neither the `{repository, tag, digest}` rule (no `digest` key)
nor, usefully, the single-string rule — that one matches the bare tag
value, which carries no repository, so `ref_repo()` reduces
`main@sha256:...` to the repository `main` and the ownership filter
drops it.
Being dropped is silent. Both scripts then report success having simply
never seen those refs.
The same emit-then-drop hits a second class, found in review: refs whose
registry host does not live inside `repository` at all.
`keycloak-operator` splits it into a sibling `registry` key, and
`kubeovn` keeps it in the document-level `global.registry.address` with
a bare `repository: kubeovn`. Both are built and pushed to `$REGISTRY`
by cozystack — `kubeovn` by the `cozystack/kubeovn-chart` wrapper repo,
whose own `make image` writes all three keys — and both were emitted
host-less and discarded as third-party.
## Impact
**Nightly (`nightly-mirror.sh`) — this is the one that broke.** The
image is never copied to GHCR, *and* the unconditional host rewrite at
the end of the script still repoints its repository at the destination
registry. The result is a published reference to an image that was never
pushed there.
Confirmed against the `:nightly` tag on GHCR — the two packages using a
recognised shape are present, and eight images across six packages using
this shape are absent:
| shape | packages | `:nightly` |
|---|---|---|
| `{repository,tag,digest}` | cilium | present |
| single-string full ref | cozystack-operator | present |
| `{repository, tag:<t>@sha256}` | kamaji, kilo, linstor-csi,
piraeus-server, linstor-gui, metallb-controller, metallb-speaker,
redis-operator | **absent** |
This surfaced as the first full nightly run failing at install: the
`e2e-image-prepuller` DaemonSet never became ready (0/3, 10-minute
timeout) because `linstor-csi` and `piraeus-server` 404 on GHCR. The
prepuller only templates kubeovn/linstor/cert-manager, so it exposed two
of the eight; the other six would have failed later at HelmRelease
install.
**Promotion (`promote-retag.sh`) — same blind spot, milder
consequence.** This retags within a single registry, so the digests
already exist and digest-pinned installs still resolve — there is no
dangling reference and no 404. What is missing is the `:<version>` tag
(and `:latest` under `MOVE_LATEST=1`) for those images, so a promoted
release does not carry the tags it claims to.
The registry shows how long that has been true for the two host-split
images. `keycloak-operator` carries exactly one tag, `latest`, though
the package has shipped since 2024-11. `kubeovn`'s newest
cozystack-versioned tag is `v1.14.11-v0.38.8`, from before the 1.0
versioning switch — while `cilium`, which uses an already-recognised
shape, carries `v1.5.0` through `v1.6.0-rc.2`. Neither has received a
tag from any 1.x release.
Measured against the committed tree, the dry-run selects **19 unique
repositories before this change and 29 after** — the eight digest-in-tag
images plus `keycloak-operator` and `kubeovn`, nothing lost.
### What the nightly does *not* recover
Four of the ten are not in the root `Makefile`'s `build` target: `kilo`,
`redis-operator`, `keycloak-operator` and `kubeovn`. Their baked refs
keep the committed GHCR host, so the `SRC_REGISTRY` filter still drops
them and their `:nightly` tags remain absent after this PR. Installs are
unaffected — those digests already resolve on GHCR. The nightly recovers
the six OCIR-built images; promotion recovers all ten.
## Why this was invisible
Latent since `nightly-mirror.sh` was added — the nightly had never
completed a full run, so its mirror step was never exercised end to end.
Both test suites were green throughout. `nightly-mirror_test.bats`'s
fixture covered the single-string, `{repository,tag,digest}` and
platformSource shapes — a layout only cilium actually uses — and omitted
the dominant one. `promote-retag_test.bats` asserted "at least one
cozystack-owned image is selected", which is satisfied by the shapes
that already worked and cannot detect an omission.
Two comments actively pointed the wrong way, and are corrected here. The
header presented its list as the shapes the build writes, which reads as
exhaustive; it now says the list is empirical, names what an audit found
outside it, and says to re-run that audit when a package is added. The
ownership-filter comment named `kilo`, `kube-ovn` and
`keycloak-operator` as third parties "not pushed to `$REGISTRY`" — all
three are pushed there, and that wrong comment is part of why the
missing tags went unnoticed.
Both fixtures are extended here. The promotion test names concrete
packages rather than asserting a count or a generic pattern, since
either would drift back to proving nothing.
## Type guards
Both digest-in-tag rules check `.tag` is a string before `test()`.
Without it a non-string tag aborts the whole expression (`cannot match
with !!int, can only match strings`), and since these invocations
swallow stderr and status, that abort would silently discard **every**
ref of this shape in the same file — reintroducing this very bug in any
file that happens to carry an unquoted `tag: 1.24` next to a cozystack
image. Rule 1 is immune because its `select(tag == "!!str")` already
precedes its `test()`.
`registry`, `repository` and `digest` are guarded too, against a
narrower failure. Those keys are concatenated rather than `test()`ed,
and yq coerces int, bool, null and seq on `+` — each merely yields a ref
the ownership filter discards. Only a `!!map` aborts, raising `!!str ()
cannot be added to a !!map`, which takes the file's whole expression
with it. So that guard is insurance against a nested-map typo in a
vendored chart, not against unquoted scalars. It is a `select()`, so an
offending map drops only itself and its neighbours still resolve.
No non-string value of either kind exists in the tree today, so both are
prevention rather than an active fix.
## Testing
- Both suites pass: `nightly-mirror_test.bats` 2/2,
`promote-retag_test.bats` 4/4.
- Every new assertion was mutation-tested — removing the rule under test
turns it red. Verified individually for the shape-3 registry join, the
shape-2 registry join, the `global.registry.address` rule, and the
`repository` type guard. Also verified that a rule matching the bare tag
instead of the repository (the subtle wrong fix) fails, which is why the
assertions check the full source ref rather than only the destination.
- `promote-retag.sh --dry-run` against the real committed tree: 19 → 29
unique repositories, no losses.
- An audit of every `@sha256:` digest under `packages/*/*/values.yaml`
against the selector output accounts for all 35: 30 are selected, and
the remaining five are third-party
(`ghcr.io/lexfrei/{kuberture,ouroboros}`, `docker.io/clastix/kubectl`,
`ghcr.io/kvaps/...`) or sit outside `$REGISTRY`
(`ghcr.io/cozystack/ingress-nginx-with-protobuf-exporter/*`).
- `shellcheck` clean on both scripts at default severity and `-S style`.
## Ordering note for reviewers
Worth landing before the next release candidate is cut, rather than
treating this as routine CI cleanup.
`promote-rc.yaml` builds the release branch from the rc's staging
branch, and `pull-requests-release.yaml` runs `./hack/promote-retag.sh`
from that merged tree — so the copy of the script that executes at
promotion is the one frozen into the rc tag's tree, not whatever is on
the default branch at promotion time. An rc cut before this merges will
carry the old script and silently skip those retags when promoted.
The nightly half is also what unblocks the full-suite e2e lane, which
cannot currently publish a working install closure to GHCR.
### Downstream repositories
Walked the trigger map in `docs/agents/contributing.md` against the
diff: it touches only `hack/nightly-mirror.sh`, `hack/promote-retag.sh`
and their two `.bats` suites. No `packages/`, no chart, no
`values.schema.json`, no `ApplicationDefinition`, no CRD or RBAC. The
`cozystack/ccp` trigger covers moving or renaming files under `hack/`
and changing what a make target does; neither applies — the anchor files
it gates on (`hack/package.mk`, `hack/common-envs.mk`) are untouched and
no make target changes behaviour.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
fix(ci): mirror and retag every cozystack-owned image, including those whose digest is embedded in the values `tag` and those whose registry host is stored outside `repository`. Ten images across eight packages were silently skipped: the nightly published references that 404 at install, and promoted releases did not carry the `:<version>` tags they claim.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved nightly mirroring and retag promotion to recognize
digest-pinned image references across more YAML value shapes, including
digests embedded in the `tag` field and optional registry wrappers.
* Expanded support for chart-global image definitions and safer parsing
to avoid silently skipped references.
* Strengthened selection logic to exclude third-party images from copy
plans.
* **Tests**
* Expanded dry-run coverage with additional fixture cases for split
maps, `tag`-embedded digests, numeric-tag variants, and more “owned”
image assertions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
v1.6.0-rc.4
|
||
|
myasnikovdaniil
|
1b777d8077 |
fix(kubernetes): re-pin deleted centos:stream9 digest for kubevirt-csi-driver (#3386)
## What this PR does `quay.io/centos/centos:stream9` is a **rolling tag** — upstream deletes the previously-pinned digest whenever it publishes a new `stream9` build. The digest re-pinned in |
||
|
Andrey KolkovandClaude Opus 4.8
|
9c3bae012a |
fix(kubernetes): re-pin deleted centos:stream9 digest for kubevirt-csi-driver
quay.io/centos/centos:stream9 is a rolling tag; upstream deletes the pinned
digest whenever it publishes a new stream9 build. The digest re-pinned in
|
||
|
Myasnikov Daniil
|
292f8ec65a |
fix(ci): type-guard the concatenated keys in the ref selectors
Address non-blocking review feedback from lexfrei on hack/nightly-mirror.sh:81 and hack/promote-retag.sh:84. `registry`, `repository` and `digest` are joined into a ref by string concatenation with no type guard, so a non-string value could abort the whole per-file expression. Because collect_refs swallows both stderr and status, that abort silently discards every co-located owned ref — the same per-file blast radius the `tag` guard already prevents. The mechanism differs from `tag`'s, though, and the narrower reading is worth recording. These keys are never fed to test(), so the "cannot match with !!int" abort does not apply to them: yq coerces int, bool, null and seq on `+`, and each of those merely yields a ref the ownership filter discards. Only a !!map actually aborts, raising "!!str () cannot be added to a !!map". The guard is therefore narrow insurance against a nested-map typo in a vendored chart, not protection against unquoted scalars. Implemented as select() rather than a test(), so an offending map drops only itself and its neighbours in the same file still resolve. The nightly fixture gains a file pairing a map-typed `repository` with an owned ref, asserting the latter is still mirrored; removing the guard turns it red. Selection over the committed tree is unchanged at 30 repositories. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
acd88bdd00 |
fix(ci): collect image refs whose host is outside repository
Address review feedback from lexfrei and IvanHunters on
hack/promote-retag.sh:84 and hack/nightly-mirror.sh:81.
The shape-3 rule assumed the registry host lives inside `repository`.
Two cozystack-built images do not satisfy that and were emitted
host-less, so the ownership filter dropped them — the same silent
emit-then-drop this PR fixes for the other shapes:
- keycloak-operator splits the host into a sibling `registry` key
({registry: ghcr.io, repository: cozystack/cozystack/keycloak-
operator, tag: latest@sha256:...}), and its own Makefile writes
exactly that layout.
- kubeovn keeps the host in the document-level
global.registry.address with a bare `repository: kubeovn`. It is
built and pushed to $REGISTRY by the cozystack/kubeovn-chart
wrapper, whose `make image` writes all three keys.
Neither has received a 1.x release tag: on GHCR keycloak-operator
carries only `latest`, and kubeovn's newest cozystack-versioned tag is
v1.14.11-v0.38.8, from before the 1.0 versioning switch — while cilium,
which uses an already-covered shape, has v1.5.0 through v1.6.0-rc.2.
Shapes 2 and 3 now rejoin an optional `registry` sibling, and a new
shape-4 rule handles the chart-global layout. An audit of every
@sha256: digest under packages/*/*/values.yaml against the selector
output confirms the remaining five misses are third-party or sit
outside $REGISTRY. Dry-run over the committed tree: 28 -> 30
repositories, exactly these two gained, nothing lost.
Also correct two comments that overstated coverage. The header no
longer presents its list as the shapes the build is limited to — the
list is empirical and charts are vendored — and the ownership-filter
comment no longer names kilo, kube-ovn and keycloak-operator as
third parties not pushed to $REGISTRY. All three are pushed there, and
that wrong comment is part of why the missing tags went unnoticed.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Andrey KolkovandClaude Fable 5
|
b897b53107 |
test(postgres): fix S3 CA secret name in backup roundtrip e2e
The backup roundtrip e2e copied the seaweedfs S3 CA from a Secret named seaweedfs-system-ca-cert in tenant-root, a name captured off a dev cluster. On current main the seaweedfs chart pins fullnameOverride: seaweedfs, so the cert-manager CA Certificate's secret is seaweedfs-ca-cert; the old name is absent and the first CI run of the e2e failed at step 05a with 'secrets "seaweedfs-system-ca-cert" not found'. Default S3_CA_SECRET to seaweedfs-ca-cert and, when that Secret is absent, auto-discover the seaweedfs CA Certificate (the seaweedfs-labelled one with spec.isCA=true) and read its secretName, so a future fullnameOverride change does not silently break the copy again. Fail with a clear message when neither resolves. README and chainsaw comment updated to match. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Myasnikov Daniil
|
770314b3cb |
fix(ci): retag images whose digest is embedded in the values tag
promote-retag.sh carries the same collect_refs blind spot the previous commit fixes in nightly-mirror.sh, and nightly-mirror.sh's own comment asserted the two were "identical to" each other -- which is how the gap stayed invisible once one of them was fixed. Measured against the committed tree, the dry-run selects 19 repositories before this change and 27 after. The eight it was missing are exactly the set that broke the nightly: kamaji, kilo, linstor-csi, piraeus-server, linstor-gui, metallb-controller, metallb-speaker, redis-operator. Nothing is lost from the previous selection. The consequence differs from the nightly case and is milder. This retag is within a single registry, so the digests already exist and every digest-pinned install still resolves -- there is no dangling reference and no 404. What is missing is the :<version> tag (and :latest under MOVE_LATEST=1) for those eight images, so a promoted release does not carry the tags it claims to, and anything resolving a cozystack component by version tag rather than by digest does not find it. Nothing in the promotion detects this: the script reports success having simply never seen those refs. The existing test could not catch it. It asserts "at least one cozystack-owned image is selected" and that nothing third-party leaks in -- both satisfied by the shapes that already worked, which is why eight omissions sat under a green suite. It now names concrete shape-3 packages, since a count or a generic pattern would drift back to proving nothing. Verified the assertion fails against the unfixed script. Same type guard on `.tag` as the previous commit, for the same reason: without it an unquoted numeric tag anywhere in a file silently discards every shape-3 ref in that file, and the swallowed stderr leaves no trace. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
06278ac18b |
fix(ci): mirror images whose digest is embedded in the values tag
The nightly's full-suite e2e failed at install: the e2e-image-prepuller
DaemonSet never became ready, because
ghcr.io/cozystack/cozystack/linstor-csi@sha256:2f314cb9... and
.../piraeus-server@sha256:39fdacec... both 404 on GHCR. They are not there
because nightly-mirror.sh never copied them.
collect_refs handled three values shapes, and the one most package Makefiles
actually write is not among them: a split map whose digest is embedded in the
tag, {repository, tag: <tag>@sha256:<digest>}, produced by a single yq call
setting .image.tag to "$(IMAGE_TAG)@$(digest)". It matches neither the
{repository, tag, digest} rule (no `digest` key) nor, usefully, the
single-string rule -- that one matches the bare tag value, which carries no
repository, so ref_repo() reduces "main@sha256:..." to the repository "main"
and the SRC_REGISTRY case drops it.
Being dropped there is silent, and the host rewrite at the end of the script
runs over every depth-2 values.yaml regardless of what was mirrored. So the
image was never copied AND its repository was still repointed at
DST_REGISTRY, leaving a reference to an image that was never pushed there --
exactly the dangling-ref failure the comment above that rewrite warns about,
reached by a path it did not anticipate. The 404 then surfaces at pull time,
one job later, as a DaemonSet rollout timeout that names no image.
This is not limited to linstor. Checking the :nightly tag on GHCR, the two
packages using a recognised shape (cilium, cozystack-operator) are present
and eight images across six packages using this shape are absent: kamaji,
kilo, linstor-csi, piraeus-server, linstor-gui, metallb-controller,
metallb-speaker, redis-operator. The prepuller only templates
kubeovn/linstor/cert-manager, so it surfaced two of the eight; the rest would
have failed later at HelmRelease install.
Latent since the script was added in
|
||
|
myasnikovdaniil
|
f5227ee60f |
fix(seaweedfs): close the 4.31 rename fallout on the 1.5.x→1.6 upgrade path (#3339)
## What this PR does Cozystack v1.5.0 bumped the vendored SeaweedFS chart 4.0.405 → 4.31.0, which renamed every workload after the Helm release (`<name>-system-*`). StatefulSet names are immutable, so upgrades through 1.5.x stood up a second, empty set beside the running one instead of renaming. #3282 pinned `fullnameOverride: seaweedfs` so 1.6 adopts running workloads in place — this PR closes the remaining holes on that adoption path, found by driving a disposable 3-node cluster through a real v1.4.5 → v1.5.3 → main upgrade with SeaweedFS tenants planted in every reachable state. - **The naming guard moves into `packages/system/seaweedfs`** — the render a platform upgrade actually re-renders: the `<name>-system` HelmRelease pulls this chart from a platform-managed ExternalArtifact, so the previous guard in `extra/seaweedfs` was never in the path. Upgrading a fresh-1.5.x tenant therefore created an empty chart-named set beside its live data and flipped the guard's own legacy-data signal. `extra/` keeps a sibling copy for operator visibility; a bats parity suite pins the two detection blocks byte-identical. - **The guard refuses instead of guessing when both naming generations exist.** Nothing durable distinguishes a duplicate that never served from one that served and crashed: claim timestamps invert during the recovery runbook's own re-bind, `readyReplicas: 0` is a snapshot, and Helm birth order answers "which generation is original", not "is the other one empty". Exactly one generation present is decidable, and the render proceeds (or refuses as class S) on its own. - **Cluster-scoped RBAC is named per namespace again.** 4.31 named four cluster-scoped objects after the release — identical for every tenant — so all tenants collided on one ClusterRole/ClusterRoleBinding and only the last-reconciled tenant's COSI provisioner kept its RBAC. Names return to `global.seaweedfs.serviceAccountName`; three of four are byte-identical to pre-4.31 and adopt in place. - **The `seaweedfs-db` hand-over runs for every instance name.** Migration 43 compared the owning release against the literal `seaweedfs-system`, so an instance named `foo` was skipped and its CNPG Cluster — the filer metadata for every object in that tenant's S3 — was pruned on the next reconcile, PVC included. The comparison now matches the `-system` suffix (shared `lib/seaweedfs-db-adopt.sh`), and new migration 53 re-runs the hand-over for clusters already past 43, before anything re-renders. - **`hack/seaweedfs-naming-audit.sh` + `docs/operations/seaweedfs-431-rename-recovery.md`** — what the guard's refusal points operators at: read-only classification (`L` / `S` / `MIXED`, naming the candidate duplicate from relative PV vintage, never a clock window) and the recovery procedures. **Scope.** The supported SeaweedFS deployment is the tenant module, which hardcodes the instance name (`packages/apps/tenant/templates/seaweedfs.yaml`); a tenant only enables or disables it. The runbook and its selectors are scoped to that name; instances created directly against the API under other names are classified by the audit but routed to escalation. Zone/pool keys of ~40 or more characters fall outside the guard's reconstruction — an accepted limit, recorded in `_naming.tpl` and the runbook. **Upgrade impact.** 1.4.x → 1.6: no manual steps — one generation, adopted in place. 1.5.x → 1.6 with SeaweedFS: migration 53 protects every `seaweedfs-db` first; then the upgrade **refuses** for any tenant holding both naming generations until the operator resolves the duplicate. This is deliberate — which generation holds the data is not decidable from inside a render, and guessing wrong destroys it. Release notes should present the refusal as expected behavior. **Testing.** 55 chart unit tests across both packages (including new MultiZone zone-component guard cases — the suite previously had no zone shapes), 11 audit bats, 8 guard-parity bats, migration bats. Validated end-to-end on a disposable 3-node cluster driven v1.4.5 → v1.5.3 → main with five tenants covering: never-saw-4.31 (renders untouched), fresh-1.5.x (refused as class S), wedged duplicate, split duplicate, and a non-default-named instance (its database survives only with this fix). The audit classifies all five correctly, and its two independent signals — revision-1 birth scheme and relative PV vintage — agree on every MIXED tenant. Related: #3282 (fullnameOverride pin), #3335 (etcd adoption backup gate — separate, also required for the 1.5.x→1.6 path). ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change Walked the trigger map against the diff: no package added/renamed under `packages/{apps,extra}/`, the `packages/core/platform/values.yaml` change is only the migrations `targetVersion` bump (no `spec.components.platform.values.*` key changes), no variant/bundle/component changes, no asset renames, no ApplicationDefinition semantic changes. ### Release note ```release-note fix(seaweedfs): the 1.6 upgrade no longer renames a SeaweedFS instance away from its data. The naming guard now runs in the chart a platform upgrade actually re-renders and refuses when both pre- and post-4.31 naming generations exist; hack/seaweedfs-naming-audit.sh and docs/operations/seaweedfs-431-rename-recovery.md guide recovery, and the refusal is expected for tenants that passed through 1.5.x. Cluster-scoped COSI RBAC is named per namespace again (the 4.31 release-based names collided across tenants), and the seaweedfs-db hand-over runs for every instance name — previously an instance not named `seaweedfs` had its filer metadata database pruned on upgrade; new migration 53 repairs clusters that already ran the old hand-over. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added fail-closed upgrade protections for SeaweedFS naming migrations (mixed/damaged classification), including safer behavior when cluster visibility is limited. * Added cluster-scoped RBAC uniqueness safeguards to prevent cross-tenant name collisions during rendering. * Improved SeaweedFS database adoption/repair migrations and strengthened post-delete cleanup ownership checks. * **Bug Fixes** * Hardened SeaweedFS 4.31 rename recovery and PV rebind flow, including reclaim policy preservation and long/non-default instance-name edge cases. * **Documentation** * Expanded the SeaweedFS 4.31 rename-recovery runbook with clarified auditing, verification, and escalation. * **Tests** * Added/expanded integration and Helm rendering tests for the above scenarios and refusal/fail-closed behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.6.0-rc.3 |