mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-22 04:33:36 +09:00
f004b35fb4267fdfc55b77083605634801ecee5a
404
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f004b35fb4 |
fix(mongodb): record the CR's flow on the artifact and bound the two open loops
Reading a backup's flow off its CR's finalizer had reached the state switch but not the artifact: the snapshot was still written from the app flag, re-read on the completing reconcile. A flag flipped to false while a system-bucket dump streamed produced a Backup whose snapshot called a cozy-backups archive legacy, so the point-in-time refusal and the S3 backfill both switched off for it. The artifact now takes the same flow the switch does, pinned by a test that flips the flag between mint and completion. Two waits had no upper bound. A live-state read that kept failing at the deadline deferred every poll behind a debug line with no condition, so an unreadable CR pinned the job Running with nothing to act on; it is now a named wait and fails after a grace, with the CR left in place rather than deleted blind. The storage-race retry minted a fresh operator CR per poll while the previous ones terminated, for the whole deadline; it now stops after a small budget and surfaces the operator's error. On useSystemBucket the chart already guarantees backup.enabled, so a strategy without s3 coordinates failed with a remedy the tenant had in place; the wait names the missing coordinates instead. The missing-credentials message no longer blames useSystemBucket on a release that never enabled it. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
5e3b8c217b |
fix(mongodb): refuse useSystemBucket without backup.enabled and pin the retry's flow
The pbm agents run only when backup.enabled is true, so useSystemBucket with backups disabled renders a cluster that can never service a BackupJob, and the tenant learned that thirty minutes into the first job when the driver's precondition gave up. The chart already fails closed on the bootstrap clash; do the same here so the combination is a render-time error with the remedy in it. The storage-race retry keyed on the app flag while everything else in the state switch had moved to the flow recorded on the CR's own finalizer, and that flag term was the only thing keeping a legacy tenant's operator CR from being deleted and retried on the same operator error — with no test holding it. Key it on the CR's flow like its neighbours and pin the legacy side: the error is terminal and the CR survives. The retry matches the operator's own error wording, which is correct at the pinned crVersion and nothing else ties together: a bump that rewords it turns the retry off with the suite still green. Name the coupling at the pin and at the matcher so a bump re-checks it. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
4233e90792 |
fix(mongodb): decide a backup's flow from its own CR, not the app flag
The opt-out refusal ran before the lookup of the job's existing operator CR, so it was a per-poll gate rather than a mint-time one: flipping backup.useSystemBucket to false while a dump streamed routed every later reconcile into the refusal, failed the job at the deadline and left the CR minted under the old flag streaming into the platform bucket with a finalizer no Backup would ever resolve — the stranded archive the running exemption exists to prevent. The refusal now applies only while the job has no CR, and everything after the mint reads the flow off the CR's delete-backup finalizer, which is fixed at mint time, instead of the mutable app flag. A late dump on the legacy flow is likewise handled by its CR's flow: the deadline fails the job and leaves the operator CR, as before this driver existed. The refusal also identified the injected entry by bucket name and told the operator to wait for the chart to re-render it, which the docs say never happens. The entry is now recognised by its credentialsSecret, the one field on it the driver chose (bucket names collide with an admin's external bucket and drift when the platform bucket is re-provisioned), and the message names the manual removal the docs prescribe. At the pinned operator `requested` is set right after the command is sent to pbm, so deleting a CR in that state cannot recall it and a late dump would land with nothing to own it. Only an unobserved or queued CR is cancelled; a requested one is left as the handle to whatever pbm writes, and the job fails. The credential pre-check failed at once where every sibling precondition waits for the restore deadline, and it re-ran against a restore already replaying, so a Secret pruned mid-replay marked an in-progress restore Failed. It now waits like its neighbours and is skipped once the operator restore exists. The status-write suppression on the waiting helper is pinned by counting status updates across repeated reconciles. The legacy deadline's justification claimed psmdb task retention covers the archive; the operator prunes only its own scheduled backups, so the object is the tenant's to reclaim, and the comments say that instead. The bootstrap refusal names the value that actually clears it (bootstrap.enabled is not reset after a bootstrap restore). Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
42a29e3597 |
fix(mongodb): keep the legacy deadline on a running dump and name the wait
Exempting state=running from the wall-clock deadline was justified by the shared bucket: on the useSystemBucket flow a failed BackupJob leaves pbm streaming an archive no Backup object represents. The exemption was applied to both flows, though, so a legacy dump whose operator or node died mid-run went from failing on the deadline to polling forever with no condition and no Event. Gate the exemption on the flow: a legacy archive lands in the tenant's own bucket under psmdb task retention and the pre-existing contract failed the job while leaving the operator CR alone, so that is restored. On the system-bucket flow the wait is now named on the BackupJob (Ready=False with the start time) instead of being silent; the waiting helper writes status only when the condition changes so a dump that runs for hours does not cost a status write per poll. The point-in-time refusal read the snapshot flag with the decode error discarded, so an undecodable snapshot fell open and a recoveryTime restore of a system-bucket archive reached the operator to die on a pbm-level message. Treat a decode failure as "flow unknown" and refuse. Adopting a target credential only for a storage on the same bucket left the source's own reference in place when nothing matched, and opting an app into the system bucket prunes exactly that Secret while older Backups still name it. Check the reference resolves before the operator sees it and fail with a named reason instead of a missing-Secret error deep in the restore. Opting back out stops the injection but leaves the driver's storage on the live cluster until the chart re-renders; a BackupJob in that window passed the presence-only precondition, minted a CR without the delete-backup finalizer and wrote an object nothing owns into the platform bucket. Refuse to mint onto a storage whose bucket is the platform's while the flag reads false, bounded by the deadline like the other waits. Also realign the snapshot literal for gofmt, and fix two doc slips: the pbm agents start on backup.enabled alone (a declared storage is needed only to service a backup), and the opt-in cross-reference targeted the parent heading instead of its subsection. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
69f80dbdc2 |
fix(mongodb): drop the false-positive stall detector and harden the cancel
The pinned psmdb operator persists a backup's status only when the state string or error changes, so status.lastTransition freezes at the instant a dump enters running and never advances while it streams. A stall detector keyed on that field fires on every dump that outruns its window and raises the same signal a genuine wedge would, so it cannot be acted on either way. Drop it: a running dump is polled quietly and never cancelled, since this CR carries no field a driver-side timer can use to tell a slow dump from a wedged one. The deadline cancel re-read the live state but could not tell "could not read" from "not started": the helper returned "" for a nil client, a throttle, a 403 and a real absence alike, and a live ready or error state also fell through, so an unanswerable read or a just-completed CR was deleted, taking a finished archive with it via the delete-backup finalizer. The helper now returns an error, and the cancel deletes only on a positive not-started answer, deferring on anything else. The restore's point-in-time refusal keyed on the resolved credentialsSecret name, which a BackupClass can set to any Secret; record the useSystemBucket flag on the backup snapshot and key the refusal on that instead. The storage-race retry deleted the errored CR, but the lookup kept returning it while it lingered Terminating, so a fresh CR was never minted; skip a CR carrying a deletion timestamp in the lookup. Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
2293c0c073 |
fix(mongodb): bound and surface the useSystemBucket backup race-recovery waits
The useSystemBucket driver injects storage onto the live cluster at BackupJob time and works around the psmdb operator resolving that storage from a cache that has not yet observed the apply. Several of those waits were unbounded or silent, and a PITR restore of a system-bucket backup reached the operator despite no oplog ever being captured on that flow: - the first-injection requeue now fails terminally once the backup deadline passes, instead of polling a cluster whose cache never reflects the storage - a state=error the operator raises because it has not yet seen the injected storage is retried (delete the CR, requeue) within the deadline rather than failing the BackupJob on the driver's own race - a backup wedged in state=running past the stall window surfaces Ready=False and a Warning once its status stops transitioning, so it no longer parks later BackupJobs in silence, and is never cancelled (that would strand the partial archive pbm keeps writing to the shared bucket) - the timeout-cancel path re-reads the CR live before deleting, so a waiting CR whose live state just turned running at the boundary is not deleted mid-dump - a failed delete-backup finalizer strip now emits a Warning naming the stuck object instead of the success-case event - a recoveryTime restore over a system-bucket backup fails with a named error rather than handing the operator a point-in-time target it cannot serve Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
d091c9951c |
fix(mongodb): requeue after a fresh storage injection so the operator's cache can observe it
The first BackupJob of a useSystemBucket app SSA-injected the storage and minted the operator PerconaServerMongoDBBackup in one reconcile. The psmdb operator resolves spec.backup.storages from a cached cluster read when it services that CR, and a cache that has not yet observed the apply takes the not-found branch and latches the CR at State=error — which the operator never re-drives (it returns for a terminal-state CR and watches only the Backup CR and Pods) and the driver reads as a terminal failure. Only the first job per app is exposed; later ones re-apply an already-declared storage. When the pre-apply cluster did not already declare the storage, requeue after the apply instead of minting in the same pass. The next reconcile finds the storage in this driver's own cache (a good proxy that the operator's has caught up too) and only then mints the CR — one 5s poll per app, well inside the deadline. A fake client has no informer lag, so this is pinned by asserting the first pass requeues and mints nothing rather than by reproducing the timing. Also correct the RBAC delete-grant comment: the 30m timeout branch cancels a not-yet-started operator CR on either flow, not just the useSystemBucket cleanup, so a Failed BackupJob never leaves a CR the operator would run later. And pin the wantBucket=="" restore-credential short-circuit with a no-bucket source case. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
e653a4fa5d |
fix(mongodb): correct the finalizer-prune claims, pin the pre-read release CR, and fix the retention field text
The delete-backup finalizer prunes storage only for a `ready` backup at v1.22.0; a timed-out backup never reaches `ready`, so the deadline-cancel comment no longer claims the finalizer "takes anything partially written" — nothing is written until the backup runs, which the cancel prevents. The psmdbBackupTimedOut godoc no longer says the operator "has not started" the backup, which is wrong for `requested` (the operator sets it as it dispatches); it now describes the not-yet-running states it actually gates. The pre-read CR handed to releaseMongoDBCleanup on the terminating path had no test that failed when the argument was dropped, because the existing cases either call the function directly or let the second Get succeed. Added a cleanupMongoDBBackup case whose operator-CR Get fails after the first: the release must strip from the passed object, not re-read, or the teardown wedges. The useSystemBucket field said "retention move[s] to the Plan", but a Plan has no retention field; corrected to "scheduling moves to a Plan; this flow has no retention yet" and regenerated the schema, README and mongodb-rd. Softened the "never pins a namespace in Terminating" wording in the cleanup godoc and RBAC comment: the driver only bounds its own finalizer, the operator's release-lock finalizer can still hold the CR while the operator is down. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
a89321fa8c |
fix(mongodb): keep retentionPolicy a genuine no-op, and cancel a timed-out backup so it cannot orphan an archive
Every prose surface (the useSystemBucket field, the retentionPolicy field, and backup-classes.md) tells a tenant that schedule and retentionPolicy are ignored on this flow, and cozyvalues-gen copies that into the schema, README and the dashboard. A render-time reject of a non-default retentionPolicy contradicted all of them: a tenant who followed the documented opt-in with a retentionPolicy still set got a failed render and a HelmRelease that stopped converging, pointed at the very doc that says the field does nothing. Drop the reject so retentionPolicy is inert exactly as documented, handled the same way as schedule (gated out, no error). The absence of retention on this flow is real but belongs to a follow-up, so the docs now point at the BackupRetentionPolicy design (#4192) and the teardown-GC issue (#3966), and recommend an S3 lifecycle policy until they land. Cancel a timed-out operator backup instead of only failing the BackupJob. The deadline fires for a backup the operator has not begun streaming; but `waiting` is the state pbm reports for a backup queued behind another, and the operator starts it once the slot frees — after the BackupJob is Failed and never reconciled again, writing an archive into the shared bucket that no Backup object represents. The deadline branch now deletes the operator CR before failing, which prevents that late run entirely: at v1.22.0 the operator returns for a CR with a deletion timestamp before it dispatches to pbm, so nothing is written. The delete is best-effort so it never blocks the failure. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
f8554d289d |
fix(mongodb): stop the system-bucket flow stranding archives, and reject an inert retentionPolicy
Two ways the shared cozy-backups bucket could accumulate objects nothing prunes. A dump that outran the 30-minute deadline was declared failed while pbm kept writing. The BackupJob's terminal-phase guard then never runs the branch that mints the Cozystack Backup, so the archive lands with no Backup object to represent it and no retention can reach it (the operator CR carries no ownerRef either). Leave a running dump to finish — a real dataset routinely outruns 30m, and a broken one still terminates through the operator's error/rejected state. The deadline keeps bounding the case it was written for: the operator never starts the backup (state stays "", requested or waiting), where nothing has been written and failing strands nothing. retentionPolicy is inert on this flow (the chart's tasks[].keep is gated out and no Plan-level retention exists yet), so a value a tenant set expecting it to work would grow the bucket silently. Reject a retentionPolicy changed from the chart default when useSystemBucket is on, rather than ignore it; the default still renders so an untouched install is unaffected. Also on the cleanup path: hand the terminating-namespace release the operator CR the caller already read and confirmed owned, so the finalizer strip is not gated on a second Get whose transient failure would silently drop it and leave the CR finalized in a namespace being torn down. Correct the ownership comments — the delete-backup finalizer is not one the operator never sets (it finalizes task-minted backups on the legacy flow); what keeps it unambiguous is that cleanup only ever resolves the one CR named in driverMetadata. Add the missing isolation test that a legacy app's own s3-storage is never overwritten by the injection call site, and move the strategy's storage comment into a Go-template comment so it no longer ships into every rendered Strategy CR. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
cc51e040ae |
fix(mongodb): keep the cleanup escape hatch off legacy operator CRs, and surface a stuck finalizer
The skip-artifact-cleanup annotation reaches releaseMongoDBCleanup before the ownership check, and the operator-CR delete sat outside the finalizer gate, so a legacy backup (no delete-backup finalizer) lost its PerconaServerMongoDBBackup record when a tenant set the annotation to release a stuck Backup or applied it across a namespace during teardown. That record is what a psmdb restore-by-name resolves against, and both the RBAC comment and the docs promise it is never touched. Gate every write in the release on the finalizer being present: an unowned (legacy) CR or one that cannot be read at all is left intact, and the Cozystack Backup still releases. Surface a failed finalizer strip. The release must proceed even against an unreachable CR, but the swallowed patch error left the operator CR holding percona.com/delete-backup — pinning its namespace in Terminating, the outcome the branch exists to prevent — while the Event reported the success case. Emit a FinalizerNotStripped Warning naming the object so an operator can find it. Mark the sibling backup fields (schedule, retentionPolicy, destinationPath, endpointURL, insecureSkipTLSVerify, s3AccessKey, s3SecretKey) as ignored under useSystemBucket, so a tenant editing one instead of the flag itself is not misled by a description that describes only the legacy flow; regenerate the schema, README and mongodb-rd. Pair the insecureSkipTLSVerify rendering assertion with a case that flips the platform knob on, so the two together discriminate the rendered value from a hardcoded default. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
d58322e253 |
fix(mongodb): let the system-bucket cleanup escape hatch survive an unreadable operator CR
The two give-up paths sat behind the operator-CR Get, so a Get that hard-fails the way a removed CRD (NoKindMatchError) or a revoked verb (Forbidden) does returned the error and never reached the release — leaving a Backup that keeps its cleanup finalizer and requeues forever, exactly in the situation someone sets the escape-hatch annotation to get out of. Read the skip annotation before any apiserver call (as the Redis cleanup does) so it always releases, and classify NoMatchError alongside NotFound (as the Job/Redis cleanups do) so a missing CRD reads as "gone". releaseMongoDBCleanup now takes the CR name and treats every apiserver step as best-effort, so the release is never blocked by the same unreachable CR. Also correct the docs, which promised an automatic release when "the operator is down": only a terminating namespace or the annotation release. A down operator in a live namespace keeps the Backup in Terminating as a visible signal, which is what the code does. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
542f42ef67 |
fix(mongodb): make the system-bucket cleanup teardown-safe and single-source its ownership
Four issues in the system-bucket cleanup path. Teardown wedge. The cleanup requeued forever waiting for the operator to prune, and the escape-hatch annotation released the Cozystack Backup without touching the delete-backup finalizer the driver stamps on the operator CR. So a namespace teardown, where the operator's delete-backup step cannot run (it execs against a pbm agent that is going away), left the operator CR — and the namespace — stuck in Terminating with no way out but hand-patching finalizers. Cleanup now detects a terminating namespace (as Redis/Rabbitmq do) and, on that or the annotation, strips its own finalizer and releases, leaving the object for a bucket lifecycle policy. Needs patch on perconaservermongodbbackups for the finalizer strip. Ownership was read from two places written at different times: the finalizer (stamped when the operator CR is created) and a driverMetadata flag (written when the artifact is created, a later reconcile). If the app's useSystemBucket flag flipped between them they disagreed. Read ownership from the finalizer on the one CR named in driverMetadata (the one the driver minted): resolving it by name, never by scan, keeps the operator's own cron-* task-minted backups (which do carry percona.com/delete-backup at v1.22.0) out of scope, so the finalizer on that named CR is an unambiguous single marker. Drops the driverMetadata flag. Injection error was terminal. A transient server-side apply — an apiserver hiccup, or a conflict while the operator writes the same cluster — failed the BackupJob outright. Requeue with backoff instead, bounded by the existing deadline, matching the cluster/app read errors alongside it. Injection ran before the enabled check, mutating a cluster that can never back up. Gate it on backup.enabled so a disabled cluster is left untouched. Also: reframe the restore S3 backfill as the defensive guard it is (the pinned operator always echoes status.s3, so the live path normally carries coordinates), and correct the psmdbtypes doc that still claimed the driver never patches the operator CRs. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
1d05dafb2e |
fix(mongodb): own and prune the system-bucket archive, and heal the restore read path
Two correctness gaps on the useSystemBucket flow. Retention. Gating spec.backup.tasks out of the chart also dropped the keep that psmdb task retention used, and an on-demand PerconaServerMongoDBBackup is not covered by task retention, so nothing pruned the shared cozy-backups bucket — a retention-swept Plan grew it unboundedly for every tenant at once. Own the archive the way the Redis/Rabbitmq drivers own theirs: stamp the operator's percona.com/delete-backup finalizer onto the backups minted on this flow, and on Cozystack Backup deletion delete that operator CR and wait for the finalizer to free the object before releasing. A legacy backup (tenant's own bucket) keeps the pre-existing hands-off no-op. Adds the delete verb on perconaservermongodbbackups and a skip-artifact-cleanup escape hatch for a wedged operator, matching the sibling drivers. Also drops the unused update verb from the psmdb cluster RBAC (server-side apply is a patch, not an update). Restore. resolveMongoDBBackupSource returned early off the live operator Backup CR whenever it reported a destination, copying status.s3 as-is. On this flow the operator can echo a destination with status.s3 empty (the storage was injected, not declared), and the target has no declared storage to resolve from either, so the restore stranded with no coordinates. Backfill a missing live S3 block from the snapshot the BackupJob recorded. Also correct the precondition godoc: the pbm agents start on spec.backup.enabled alone; the declared storage is required by the backup CR's resolution, not by the agents. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
|
47fe7c9630 |
test(mongodb): pin the restore credential swap and document the opt-out
The restore-time decision to re-point a backup's credentialsSecret at the
target cluster lived inline in reconcileMongoDBRestore with no test, so the
same-bucket guard that keeps a cross-flow restore from authenticating
against the wrong bucket could be dropped unnoticed. Extract it into
mongodbRestoreCredentialsSecret and cover all four
{legacy,system}x{legacy,system} combinations; neutralising the bucket
match now reddens the suite. The projected-secret skip is kept as well: a
target storage sharing the platform bucket but naming a different secret is
better restored with the platform-projected cozy-backups-creds than with
that secret.
Document the reverse toggle: flipping useSystemBucket true->false leaves
the driver-injected s3-storage on the live PerconaServerMongoDB under the
driver's field manager, which the chart does not own and never prunes. It
is inert once backups are disabled; the operator must delete it by hand to
return the release to the legacy flow.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
|
9545bee233 |
fix(mongodb): correct system-bucket prose, pin strategy rendering, make restore credential swap flow-aware
Several comments and the useSystemBucket field description had drifted from
what the code does.
The strategy header still said the $bucketName gate was "not because it
needs the value" and that the sibling strategies needed the bucket name
while this one only asked for ordering, but the strategy now renders
bucket: {{ $bucketName }} for the injected coordinates, so it needs the
value; the precondition comment claimed the s3-storage entry exists on
backup.enabled=true, which is only the legacy flow. The rendering-test
comment carried the same "needs no bucket coordinates" claim. The
useSystemBucket field description said injection happens "at first BackupJob
time" (it now runs on every BackupJob) and that backups scope to
<namespace>/<release> (the prefix templates the application name, not the
release), and it ran ~700 chars of implementation detail into the CRD that
backup-classes.md already carries; trimmed and corrected, then regenerated
schema/README/types/cozyrds. Same <release> slip fixed in the driver
comment and the docs.
Nothing asserted the new strategy rendering. Added the MongoDB row to the
endpoint-form suite (s3.endpointUrl, scheme preserved both ways) and field
assertions to the rendering suite — bucket, endpointUrl, region,
credentialsSecret, forcePathStyle, and prefix, which must reach the CR as a
literal go-template string so the driver can template it per-application;
a stray brace would empty it with every doc-count test still green.
The restore credential swap broke across flows. It replaced the source
credentialsSecret with whatever the target declared, comparing no buckets,
so a system-bucket backup restored into a legacy target kept the platform
bucket against the tenant's own -s3-creds, and a legacy backup restored
into a system-bucket target got cozy-backups-creds against the tenant
bucket - both AccessDenied. Skip the swap when the source already names the
platform-projected cozy-backups-creds (it outlives the source app, nothing
to repair), and only adopt a target credential whose storage is on the
same bucket as the archive. psmdbStorageS3 now reads .s3.bucket alongside
.s3.credentialsSecret; TestPsmdbTargetCredentialsSecret covers the match,
the cross-bucket refusal, and the empty-bucket no-op.
Also drop the post-apply re-Get: the typed SSA Patch decodes the server's
merged object back into the passed struct, so the precondition reads the
injected storage from it instead of a follow-up Get that hit the informer
cache and briefly reported Ready=False on the happy path.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
|
4ea4888685 |
fix(mongodb): key system-bucket injection off the app flag, not live storage
The MongoDB backup driver decided what to inject and what to record from the live cluster's storage set rather than from the app's opt-in flag. That had three consequences. The snapshot fallback fired for legacy apps too. Its gate was `snap.S3 == nil && rendered.S3 != nil`, but the coordinates moved onto the shared cozy-default strategy, so rendered.S3 is now non-nil for every MongoDB app. A legacy backup written to the tenant's own bucket got its snapshot rewritten to the platform bucket, sending restore to the wrong place. Gate the fallback on the app's backup.useSystemBucket flag instead, threaded through createMongoDBBackupArtifact, and pin the legacy case. The injected storage never caught up with the strategy. Injection was gated on the named storage being absent, so it ran once per cluster and froze the coordinates at the first backup; a later endpoint/region change or bucket re-provision was a silent no-op, and Helm could not repair it because the useSystemBucket chart renders no storages. Key injection off the app flag and re-apply on every BackupJob (the prefix is deterministic, so this never splits the archive the way CNPG's serverName would). Add UseSystemBucket to mongodbapp.MongoDBSpec so a legacy app is protected by the flag, not by whatever happens to be on the cluster. The docs said the opposite of what the driver does. backup-classes.md claimed "the MongoDB driver cannot inject its S3 target per-backup"; correct it and add an opt-in-to-the-system-bucket section that names which values stop being read and that scheduled backups + PITR must move to a Plan. Fix the endpoint table row, the tuning reference, the RBAC comment that said the driver never patches, the strategy header/inline comments, and the example comment. Also assert notExists on spec.backup.pitr in the helm test, and lift the duplicated cozy-backups-creds default into a const. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
|
c036efaa5a |
test(mongodb): cover system-bucket storage injection gate and builder
Extract the two untested contracts from applyMongoDBSystemStorage into
pure functions and unit-test both, mirroring the CNPG driver's
buildBarmanObjectStore split:
- shouldInjectMongoDBSystemStorage decides whether the driver injects the
system-bucket storage; the tests pin both branches, including the one
that leaves a legacy app's manual bucket untouched.
- buildMongoDBSystemStorageEntry builds the {type:s3,s3:{...}} entry,
defaulting credentialsSecret and omitting forcePathStyle when nil.
Also add a helm-unittest for the restore.yaml guard rejecting
bootstrap.enabled together with backup.useSystemBucket.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
|
730592f323 |
feat(mongodb): support the system-bucket backup flow (useSystemBucket)
MongoDB was the only cozy-default app with no useSystemBucket flow: its backup storage was always static on the app CR (bucket/endpoint/creds from backup.* values + a chart-emitted <release>-s3-creds Secret), so a tenant could not back up to the platform system bucket out of the box the way postgres/clickhouse/etc. do. PBM takes bucket/endpointUrl/region/prefix as static CR fields (not secret refs) and the platform bucket name/endpoint are only known at BackupJob time, so - unlike ClickHouse's runtime env secretKeyRef - the chart cannot render them. The fix mirrors the CNPG driver: the chart adds a useSystemBucket flag that keeps spec.backup.enabled (so the pbm agents run) but omits the static storages/tasks/pitr and the creds Secret; the cozy-default MongoDB strategy carries the coordinates (endpointUrl already scheme'd by the endpoint helper, credentialsSecret cozy-backups-creds); and the psmdb driver SSA-injects spec.backup.storages[s3-storage] onto the live PerconaServerMongoDB at BackupJob time under its own field manager with ForceOwnership, so a Flux re-render never reverts it. Injection is gated on the storage being absent, so a legacy app that ships its own static storage is left untouched. The backup snapshot falls back to the strategy coordinates when the operator status echo is empty, so the restore path rebuilds backupSource (credentialsSecret cozy-backups-creds is projected into the restore namespace before dispatch). RBAC grows patch/update on perconaservermongodbs for the injection. A bootstrap.enabled + useSystemBucket render guard mirrors postgres. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
|
aa20495df3 |
feat(marketplace): register a tapped repository's apps on connect
Connecting a marketplace repository materialized its PackageSource but created no Package, so its ApplicationDefinitions never rendered and its apps never appeared in the dashboard catalog until a manual `cozypkg add` the dashboard cannot drive (the package list is computed from the ApplicationDefinitions, which do not exist yet). The tap materializer now creates a tap-managed registration Package for each materialized PackageSource, the way the platform ships a Package per built-in app, so the ApplicationDefinition registrations deploy on connect and the apps become browsable; the app components carry no install block and are still instantiated per user resource. Privileged install components of an unconfirmed tap auto-registration (a Package that still carries the marketplace-tap label) are refused on several best-effort layers: the Package reconciler skips their HelmRelease and prunes a lingering one at the install site, and the PackageSource reconciler withholds their ExternalArtifact so the helm-controller has no privileged content to deploy, re-running on confirmation via a Package watch. `cozypkg add --allow-privileged` sheds the label as an explicit ownership handover, which is the confirmation; a platform Package never carries the label. A benign component flipped to privileged in a new revision can still be materialized by source-watcher before these layers observe the flip; closing that race fully requires decoupling the catalog registration from install and is tracked in issue #4359. `cozypkg untap` and the dashboard disconnect remove the registration Package (de-registering the apps) under a UID + ResourceVersion precondition, so a concurrent `cozypkg add` handover is never clobbered; running instances are left but unmanaged. `cozypkg add` re-opens a tap auto-registration for variant selection and hands it over. The cozystack-api ClusterRole gains delete on packages for the disconnect path. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
4dc6065ec9 |
fix(backups): give the sidecar sizing its reason per caller
The Keycloak comment and the helper doc claimed the tenant LimitRange as the reason for the resources, but cozy-keycloak has no LimitRange and neither does a tenant that leaves resourceQuotas empty: there the sidecar had no requests and no limit at all, and the change gives it a 1Gi ceiling where there was none. The comments and both doc blocks now say what applies to each caller. No rendered object changes. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
|
9214430bd8 |
fix(backups): give the barman-cloud sidecar its own resources
The barman-cloud plugin injects its sidecar without resource requests or limits. Tenant namespaces ship a LimitRange that defaults containers to 128Mi, so the sidecar inherits that default and is OOMKilled mid-backup: the ObjectStore stays healthy, the Cluster stays Ready, and only the backup fails. A cgroup high-water mark of 254 MiB on a 38 MB database shows the default cannot hold it. The shared cozy-lib helper and the Go path that builds the platform's own ObjectStore now request 100m/256Mi and cap memory at 1Gi. No CPU limit is set, matching the entityOperator precedent in the kafka chart: a throttled sidecar stalls WAL archiving instead of failing it. The helper is renamed from checksumSidecarConfiguration to sidecarConfiguration, since it no longer carries only the checksum pin. Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
|
f9ea0da3de |
feat(backups): Kafka topic-metadata backup via a dedicated strategy driver (#4000)
## What this PR does Adds a dedicated `strategy.backups.cozystack.io/Kafka` Go driver that backs up the **topic metadata** of a Cozystack-managed `Kafka` application and wires it into the platform `cozy-default` flow — the counterpart to the app-specific drivers (CNPG, MariaDB, Altinity, MongoDB). It is a typed variant of the generic Job strategy: the controller renders a one-shot batch Job that talks to the Kafka Admin API, and adds on top an applicationRef Kind gate, a `Ready` precondition on the Strimzi `Kafka` cluster, and a per-run object key via `.BackupName` so distinct runs never overwrite each other. What it captures: every non-internal topic's partition count, replication factor and non-default configs (via `kafka-topics --describe`), stored as one small object per run in the system bucket; restore recreates them with `kafka-topics --create` + `kafka-configs --alter`. Deliberately **out of scope**: message payloads (a volume-snapshot strategy's job), consumer-group offsets (a log position, only meaningful with a byte-identical data restore), ACLs, quotas, and `KafkaUser`s. No PITR. Restored topics are created directly through the Admin API, so on the target they are unmanaged (no `KafkaTopic` CR) — the accepted tradeoff of an Admin-API-only driver. Both restore flows are implemented: **in-place** recreates dropped topics into the source, and **to-copy** applies the source's topics onto a freshly-bootstrapped empty `Kafka` (the "bootstrap a cluster, then apply the metadata state" flow). Contents: - Driver: `api/backups/strategy/v1alpha1/kafka_types.go`, `internal/backupcontroller/kafkastrategy_controller.go` (+ unit tests for its pure helpers), `internal/backupcontroller/kafkatypes/` (Strimzi `Kafka` readiness gate), dispatch cases in both controllers + `supportedBackupStrategyKinds()`, scheme registration, generated CRD + deepcopy. - Chart: `strategy-kafka-default.yaml` (gated on the bucket name), a `cozy-default` binding, read RBAC on `kafka.strimzi.io/kafkas`, and the Strimzi Kafka client image value. - Docs: a Kafka section in `docs/operations/backup-classes.md`. - Example + e2e: `examples/backups/kafka-metadata/` and `hack/e2e-chainsaw/kafka-metadata/` (`kafka-3-metadata-roundtrip`), an honest round-trip that proves a per-run `retention.ms` sentinel survives both restores. The original `packages/apps/kafka` chart is left untouched (it has no legacy backup mechanism). Validated: `go build ./...`, `go vet`, `go test ./internal/backupcontroller/...` (pure helpers plus a reconcile Kind-gate test and the RestoreJob cleanup regression test), `helm unittest` on the package (which now renders `strategy-kafka-default.yaml` and asserts its kind/name/`artifactURITemplate`), the `select-install`/`select-e2e` bats suites, `helm template` of the strategy CR, `bash -n` of the rendered script, and `make generate` (clean). The end-to-end round-trip (`kafka-3-metadata-roundtrip`) is enabled and mapped like the sibling backup suites, so CI builds this branch's controller and runs it live against a fresh sandbox; it has not been merged-and-run before, so that first live run must go green before merge. ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. This PR adds a new `strategy.backups.cozystack.io/Kafka` CRD, a Go driver, chart wiring, docs and examples. It does not add or rename a `packages/apps`/`packages/extra` package, does not change any `ApplicationDefinition`, `values.schema.json`, version enum, `release.prefix`, output Secret/Service, the `Package`/`Plan`/`RestoreJob` CRDs the provider types by hand, any `hack/*.mk` anchor or make-target behaviour, node prerequisites, labels/annotations, or telemetry metrics. A brand-new `strategy.backups.cozystack.io` CRD travels to the Terraform provider as opaque JSON (it types only `Plan`/`RestoreJob`/`Package`), so it keeps working untouched. None of the downstream triggers match. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(backups): add a Kafka topic-metadata backup strategy bound by cozy-default — backs up topic definitions and configs (partitions, replication factor, non-default configs) via the Kafka Admin API, with in-place and to-copy restore; message data, consumer offsets, ACLs, quotas and KafkaUsers are out of scope ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added Kafka topic-metadata backup and restore support, excluding message data. - Supports ad-hoc and scheduled backups, in-place restoration, and restoration to a separate Kafka cluster. - Recreates missing topics and restores compatible configurations while validating partition and replication-factor constraints. - Waits for Kafka readiness and cleans up backup artifacts after completion. - Backup and restore jobs now enforce configurable deadlines to prevent stalled operations. - **Documentation** - Added guidance covering Kafka metadata backups, restoration limitations, and S3 configuration. - **Tests** - Added end-to-end coverage for backup, restoration, validation, and cleanup scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
ca2a6d6429 |
fix(backups): warn when a Kafka backupTimeout is rejected
kafkaRunDeadline silently substituted the 30m default for a backupTimeout that time.ParseDuration rejects or that falls below the one-minute floor, with no Event, condition or log line. An operator who set an unparseable or sub-minute value, then raised it, saw every run keep dying at DeadlineExceeded with nothing naming the discarded parameter. Return the applied deadline together with a message when the configured value is rejected, and record it as a BackupTimeoutInvalid Warning Event on the BackupJob and RestoreJob (the reconcilers already hold a Recorder). An honoured or unset value stays silent. TestKafkaRunDeadline now asserts the warning fires for the rejected cases and not otherwise. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
b3b280ea73 |
feat(gateway)!: add TLS-passthrough listeners to TenantGateway (#3342)
## What this PR does Phase 1 of external database exposure via Gateway API TLS-passthrough (#2815): adds a `tlsPassthroughListeners` field to `TenantGateway` and renders one layer-4 `mode: Passthrough` listener per entry, on a native database port with a per-engine subdomain hostname, alongside the Gateway's terminate listeners. No engine is wired: no TLSRoute, no certificate or CA changes. The existing layer-7 `tlsPassthroughServices` field gains a 62-entry count bound, an item length and a DNS-1123 item pattern, and a repeated entry now fails the reconcile with `Ready=False` instead of rendering the same listener twice. `spec.apex` gains a 253-character bound and nothing else: the CEL rules concatenate it, so the install-time cost estimator refuses the CRD without a declared bound. It gains no format pattern, and that is not a claim that a mixed-case host works. Such a host reaches the field verbatim from a namespace label, the renderer then composes listener hostnames Gateway API refuses, and that tenant has no Gateway either way; what a pattern would change is only that the write is refused, which fails this HelmRelease where the same fault costs the one Gateway. An apex no listener hostname can sit within is reported by the controller instead, and only when `tlsPassthroughListeners` is in use. The constraints and what they do to objects that already exist are described further down. The bound on an entry is what the schema can state on its own. An entry also becomes a listener name and a hostname built from the apex, so the controller checks both composed values against the 253 characters Gateway API allows and names whichever one overflows. Validation rejects duplicate listener names or ports, ports reserved by the platform Gateway (80 and 443), out-of-range ports, invalid DNS-1123 names, and hostnames outside the tenant apex (mirroring the Gateway hostname admission policy so an invalid entry cannot make the whole Gateway rejected). The SNI-overlap rule refuses two passthrough listeners that answer the same name on different ports. That is what [cilium#42898](https://github.com/cilium/cilium/issues/42898) reports: only one of the two routes ever works. It does not extend to a terminate listener and a passthrough listener sharing a name across ports. No failure is reported for that shape, and Cilium counts a terminate/passthrough pair as conflicting only when they share a port. The per-engine subdomain hostname scheme follows the phase-1 gate. The flat-hostname alternative (pending the Cilium listener-isolation fix targeting 1.20) is a later scheduling decision that does not change this API field. One shape differs from #2815 on purpose. The ticket has `hostname` optional with a `*.<name>.<apex>` default; here it is required. Deriving the SNI from the listener name hides the one value an operator has to get right, and the two drift apart the moment the naming scheme changes. Found while checking that pairing, unrelated to this field: a stock `dns01` gateway already renders `tls-api` (passthrough, 443) beside `https` (terminate, 443, `*.<apex>`), one port with overlapping SNI, which is the shape Cilium 1.19.6 marks `Conflicted`. That belongs to the Cilium bump and is recorded on #3343. Also named rather than fixed: the controller-side rules on this field stop the whole reconcile when they refuse a spec. `renderGateway` returns an error, so the Gateway is not updated at all, and the tenant sees `Ready=False` with `ReconcileError` while new app listeners and every certificate the controller would issue or amend wait behind it. Certificates already issued go on renewing, since cert-manager drives that from the objects already in the cluster and an aborted reconcile writes nothing to them. Dropping only the conflicting listener and reporting it on status would be the better shape, but that changes the error model of the whole renderer rather than one rule, so it is out of scope here. The current behaviour is at least observable: `Ready=False` is a signal, not silent drift. On HTTP-01, a hostname a passthrough listener answers earns no HTTPS-terminate listener once a TLSRoute claiming that hostname can be served on the listener, matched by SNI overlap so a wildcard entry reaches a name beneath it as readily as the name written in the spec, and the field itself is accepted under certMode http01 alone. That refusal is stated as the set of modes that can carry a passthrough listener rather than the set that cannot, because a mode added to the enum would otherwise be admitted by default: dns01 and existingSecret serve the whole apex from one wildcard terminate listener the pinned Cilium cannot keep apart from a passthrough listener beneath it, and edge ends TLS at the class provider and renders no TLS listener for one to sit beside. The route half of that condition is load-bearing rather than incidental. On the pinned v1.19.5, tlsPassthroughFilterChains walks a listener's routes and skips one with no backends, so a declared passthrough listener nothing attaches to contributes no filter chain and matches no ClientHello, while the terminate listener's chain is built from its own Secret and carries the name alone. Keying the withdrawal on the declaration would therefore take a name that is being served and hand it to a listener that forwards nowhere, and that is the ordinary shape rather than a corner: the chart puts api, vm-exportproxy and cdi-uploadproxy on every tenant Gateway, while there is one platform TLSRoute per name cluster-wide and each names one tenant's Gateway as its parent, so every other tenant renders all three entries with a route on none. The backend half is that same rule reached by a different road: the chain is built per route out of the backends that survived resolution, so a route that attached and forwards nowhere puts no chain on the name either, a backendRef naming a Service that does not exist yet because manifests applied out of order, one deleted since, a name with a typo, or one in another namespace with no ReferenceGrant, and without that half an ordinary backend mistake takes a live endpoint offline while Cilium builds nothing to replace it. Resolution asks what v1.19.5 asks and no more: the reference allowed, the Service present, and neither its ports nor its endpoints read, because the translator reads neither. Services and ReferenceGrants are watched alongside routes, so both halves are deferrals rather than exemptions and the withdrawal lands on the pass that sees the missing piece arrive. The withdrawal reaches HTTP-01 alone, because `collectHostnameClaims` returns nothing under the other two modes and they render no per-hostname listener to withdraw. A TLSRoute that declares no `spec.hostnames` claims the hostname of each rendered passthrough listener its parentRef selects, because that is the name Cilium serves it on: `ComputeHosts` at v1.19.5 substitutes the listener hostname for a route that names none, so such a route takes the name over exactly as one that spells it out does. The tenant-namespace hostname policy requires a route there to name its hostnames, so this shape reaches a Gateway from the system namespaces where the platform routes live, or from an administrator, not from a tenant chart. A claim broader than the listener's counts the same way from the other side: a TLSRoute claiming `*.<apex>` is served on `api.<apex>` rather than on the wildcard, because `ComputeHosts` returns the listener's own hostname for a route hostname covering it, so a claim covering a reserved name is judged as a claim on that name while the route keeps one entry under the key its owner wrote. Both restrictions are tied to the Cilium pin, currently v1.19.5. That version translates the Gateway into a single Envoy listener whose filter chains match on SNI without the port, so a terminate chain and a passthrough chain carrying one name collide regardless of which Gateway port declared them; upstream states the consequence above NeedsCrossProtocolSplit in operator/pkg/model/model.go as of v1.19.6, the release that starts splitting Envoy listeners per port when NeedsPerPortListeners holds. That is a disjunction rather than a blanket change, and the passthrough side of every one of its three clauses is counted from attached routes, so a listener with no TLSRoute behind it contributes to none of them: the split arrives with the route, not with the bump. The same single-listener translation has a consequence that is not a restriction: the declared port is where the listener is published, not a bound on reach, because the backend answers its hostname's SNI on every port the Gateway exposes. Firewalling a tenant address down to 80 and 443 does not put a database out of reach; the hostname is the access surface, and the field's own doc and the chart README now say so. Both restrictions are validation-only, and they lift together on the pin plus a configuration the split actually covers rather than on the bump alone. A route whose hostname is withdrawn this way is told so under this controller's own name: Accepted=False with `NoMatchingListenerHostname`, and a message naming the passthrough hostname that answers it, which for a wildcard entry the route cannot derive on its own. It matters most there. A wildcard entry can withdraw the listener from several published names beneath it at once, each one a TLSRoute claims and can be served on, and the certificates are garbage-collected with them, and a rollback is cheaper than that sounds: cert-manager here runs with `enableCertificateOwnerRef: false`, so deleting a Certificate leaves its Secret behind and a re-created Certificate adopts a still-valid one rather than placing an order. Before this the only signal was Cilium's own RouteParentStatus entry under a different controllerName. A TLSRoute whose hostname no passthrough listener answers is told so as well, on the same condition and for the same reason: before this change that hostname got a terminate listener Cilium refused with ResolvedRefs=False, so withdrawing the listener would otherwise have removed the only object saying the route went nowhere. Two things stay named rather than fixed. A `tlsPassthroughListeners` hostname of `*.<apex>` is refused only indirectly: the overlap rule rejects it while `tlsPassthroughServices` carries its defaults, because the wildcard covers api.<apex>, so emptying that list is enough to let it through, and the terminate listener then goes from every published name a TLSRoute claims under the wildcard. The README states this; refusing the apex and its wildcard on their own terms is a one-line rule worth adding when the field is wired up to chart values. A wildcard hostname on an attached HTTPRoute used to render a listener name the shipped Gateway CRD refuses, `https-*-<hash>`, which failed the whole Gateway write while the README said the shape worked. Listener and Certificate names now spell a leading `*` as `wildcard`, with the hash still covering the whole hostname, and a test reads both the listener-name and the hostname patterns off the shipped CRD bundle and renders wildcards through them; every name that was already admissible is pinned byte for byte, so nothing is renamed on upgrade. Under http01 an HTTPRoute wildcard claim is refused before the race and the overlap are read, with `NoMatchingListenerHostname`, because the HTTP-01 challenge issues no wildcard certificate and a terminate listener could never have carried the name. A route status write that fails no longer leaves the route carrying its previous condition: `updateRouteStatuses` and the whole-apex pass collect every failed write and return them after the loop, so the reconcile fails, the TenantGateway reports `Ready=False` with `ReconcileError` and controller-runtime requeues, while the loop still runs to the end so one route's conflict does not hold the others back. Conditions this controller wrote under http01 are retracted when the tenant moves to dns01, existingSecret or edge, for HTTPRoutes and hostname-less TLSRoutes alike, so a refusal does not outlive the mode that produced it. And a TLSRoute claiming a wildcard passthrough entry itself leaves the names beneath the wildcard alone, since Envoy ranks an exact server name above the wildcard and nothing collides. One thing this does not cover yet: Accepted is computed from the hostname race and the withdrawals above, so a route is still told it is fine when an HTTPRoute holds the name and the TLSRoute claiming it cannot attach to the terminate listener that answers it. That one predates this change. Two neighbours of it no longer fall in the gap. A `parentRef` port that names no listener answering the hostname is compared alongside the `sectionName`, since Gateway API requires every key a route pins itself with to match the listener it selects. An HTTPRoute's `parentRef.port` is judged the same way: the terminate listener sits on 443, so any other pinned port earns no listener and no certificate through that parentRef and reads the port-mismatch cause; port 80, or `sectionName: http`, from a namespace the plain-HTTP listener admits (the publishing tenant and cozy-cert-manager) is served plain with no terminate listener and no refusal, from any other namespace it is refused with `NotAllowedByListeners`, and a `sectionName` and a port that select different listeners select none and read `NoMatchingParent`. A plain-served route stays in the hostname race on the same footing as a terminating one, in every recount as well as the first count. And a `sectionName` naming a `tls-<name>` listener this Gateway does not render now reports `Accepted=False` with `NoMatchingParent`, Gateway API's reason for a parentRef matching no listener: the controller renders every listener in that namespace, so an absent name there is a fact rather than a guess, while a `sectionName` naming the plain-HTTP listener is judged like port 80 below, and any other name outside the prefix is written Accepted like an attached route, with the class controller's own status entry answering for the listener it named. The namespace case is off that list. Native-port listeners admit the publishing tenant alone, and a TLSRoute elsewhere on one of their hostnames is refused with `NotAllowedByListeners`. An HTTPRoute that pins itself by `sectionName` to a rendered `tls-<name>` passthrough listener is refused with the same reason: that listener forwards the stream it matched by SNI and terminates nothing, so it takes TLSRoute alone by protocol and cannot serve the route whatever answers its hostname. Its claim also leaves the hostname race before ownership is ranked, so the TLSRoute that can be served is not told it lost to a route that could never attach. Cilium does not report this refusal at the pin. The port-443 passthrough listeners name HTTPRoute in `allowedRoutes.kinds` to keep the port-443 set uniform (cilium#45559), the Gateway reports that entry back as `ResolvedRefs=False/InvalidRouteKinds` on the listener, and Cilium's route-side kind check reads a listener's kinds against every route on the Gateway rather than against the routes that named it, so it reports the route Accepted and the condition this controller writes is the only refusal the route gets. A hostname whose every claimant is refused this way earns no HTTPS-terminate listener and no certificate, where before it earned both. The refusal is decided per route rather than per hostname, which matters once a claim overlaps more than one reserved name: a wildcard covering both a port-443 service hostname and a native-port one is attachable through the first and refused through the second, so the answer comes from the listener the `parentRef` names, its own hostname and its own attach set. The row that pinned a `sectionName` naming no rendered listener as a known gap now pins the verdict instead. On the port-443 listeners the two namespace sets agree by construction, since `collectHostnameClaims` and the listeners' allowedRoutes selector run off the same list. And the justification for the pin is written out at every site that depends on it rather than once, so the day the pin moves they all have to move together and the one that does not is a false statement no diff can catch. That is not hypothetical here: three claims of exactly that shape were wrong in this branch and were caught by reading the source they described rather than by any diff, so the volume of restated reasoning is itself what produces them. One canonical block with references to it would carry the same knowledge with one place to update, and rewriting the narrative under review would be a documentation change rather than a fix, so it is tracked on #3842 instead. The gateway chainsaw suite now covers the withdrawal, which it did not before: a tenant owning its own Gateway renders the shipped passthrough entries with nothing routed to them, an HTTPRoute on api.<apex> keeps its terminate listener, its certificate and Accepted=True there, and attaching a TLSRoute to the same listener, pointed at a Service in the tenant's own namespace, since a route that forwards nowhere is the case the first phase already measures, withdraws all three and moves the route to NoMatchingListenerHostname. A third phase patches in a native-port entry and reads the per-listener readiness flag, which is the claim no test against a fake client can settle. The first two phases are one measurement: withdrawing on demand is what dates the earlier assertion to settled state rather than to a withdrawal that had not arrived yet. The `Require API owner review for sizeable API changes` check is red on purpose and does not clear with a code change. A new field plus tightened validation on already-shipped fields counts as a sizeable API change, so `.github/workflows/api-review-gate.yaml` holds the PR until an API owner approves it. The gate counts each owner's latest review on the current head commit, so the approval has to land on the head as pushed and any later push invalidates it; the job re-runs on review submission, so no push is needed to flip it. Reading that red as "not ready" is the mistake this paragraph exists to prevent. Part of #2815. ### Release note ```release-note feat(gateway): TenantGateway supports layer-4 TLS-passthrough listeners on native database ports for SNI-based external exposure. On HTTP-01 gateways a hostname a TLS-passthrough listener answers, including any name beneath a wildcard entry, no longer earns an HTTPS-terminate listener or a Gateway-issued certificate once a TLSRoute claiming that hostname can be served on the listener and forwards to a backend that resolves; a hostname claimed only by a TLSRoute stops earning one too. A TLSRoute with no spec.hostnames counts as claiming the hostname of each passthrough listener its parentRef selects, and one whose hostname covers a listener's counts as claiming that listener's name, both being the name Cilium serves it on. A declared passthrough listener with no route on it changes nothing, because it emits no filter chain and the terminate listener is what answers the name, and neither does a route whose backendRefs resolve to nothing, a Service that does not exist, or one in another namespace with no ReferenceGrant, since such a route emits no filter chain either. The controller watches Services and ReferenceGrants, so the withdrawal it defers lands once the backend appears. So the tenant whose Gateway the platform TLSRoutes name, tenant-root unless the cluster's expose-ingress says otherwise, sheds the terminate listeners and certificates for api, vm-exportproxy and cdi-uploadproxy on upgrade (those TLSRoutes ship with the cozystack-api, kubevirt and kubevirt-cdi packages, each rendered only where the Gateway integration is enabled and the service is among the platform's exposed services), and a tenant that renders the same three entries with no route on them keeps what it has. The shed certificates leave their TLS Secrets behind: cert-manager here ships enableCertificateOwnerRef: false, so each Secret stays in the namespace with no owner, nothing renewing it and nothing deleting it. That cuts both ways, as stale Secrets to remove by hand and as a cheap way back, since a re-created Certificate adopts a still-valid Secret instead of placing an order. A route the controller renders no listener it can attach to now carries Accepted=False from this controller instead of Accepted=True. Five shapes report NoMatchingListenerHostname and the message tells them apart: the HTTPRoute whose hostname a passthrough listener answers, the TLSRoute with no passthrough listener behind it, the route whose parentRef names a listener that answers a different hostname, the route whose parentRef names a port no listener answering its hostname is published on, and the HTTPRoute claiming a wildcard hostname under http01, which the HTTP-01 challenge cannot certify. A route pinned by sectionName to a tls-<name> listener the Gateway does not render carries NoMatchingParent instead, which is what Gateway API calls a parentRef matching no listener, and so does an HTTPRoute whose sectionName and parentRef port select different listeners. A TLSRoute outside the publishing tenant's namespace claiming a native-port hostname carries NotAllowedByListeners instead, since those listeners admit the tenant alone. So does an HTTPRoute pinned by sectionName to a rendered tls-<name> passthrough listener, which cannot serve its kind, and an HTTPRoute selecting the plain-HTTP listener from a namespace it does not admit; a refused route also stops counting in the hostname race, so a hostname whose every claimant is refused earns no terminate listener and no certificate. Listener and Certificate names derived from a wildcard hostname are admissible now, names that were already admissible do not change, a failed route-status write fails the reconcile instead of being logged away, and conditions written under http01 are retracted when certMode leaves it. The dns01 and existingSecret cert modes shed nothing, because neither renders per-hostname listeners at all, and tlsPassthroughListeners is accepted under http01 alone: those two serve the whole apex from one wildcard terminate listener, and edge terminates TLS upstream and renders no TLS listener a passthrough entry could sit beside. The withdrawal does not take a working path away: on the pinned Cilium both port-443 filter chains already carry these hostnames and nothing on the objects says which one answers a connection, so a race that was previously undetermined and unobservable now resolves the same way every time. Treat this as a change of serving path rather than the removal of something idle: today each of those three hostnames has a terminate listener and a passthrough listener on port 443, both filter chains carry the SNI, and nothing on the objects says which one answers a connection. Gateway API calls for both listeners to report Conflicted; the pinned v1.19.5 has no Conflicted condition at all, so there is nothing to read, and you cannot tell from the cluster whether api.<apex> is terminated by the Gateway or passed through. Afterwards it is passed through, always. Read the Gateway's listener conditions before upgrading to see what those listeners were serving. Before upgrading, confirm the management apiserver serving certificate carries api.<apex> in its SANs, because once the ACME certificate is withdrawn that is the only certificate left on the name; a cluster that came from the ingress-nginx path already meets that, a cluster served by the Gateway's ACME certificate since it moved does not necessarily and loses the name until the SAN is added. If something depends on the terminate path for one of those names, the package README lists three ways forward and what each costs. Moving the consumer to another hostname under the apex is the only one that gives up nothing. ``` ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against this diff: `api/gateway/v1alpha1/{tenantgateway_types.go,zz_generated.deepcopy.go}`, `internal/controller/tenantgateway/{reconciler,renderers,conflict,mapper,status,backends,reconciler_test,crdcel_test,mapper_test,backends_test}.go`, `internal/controller/tenantgateway/rbac_test.go`, `cmd/cozystack-controller/main.go`, `hack/e2e-chainsaw/gateway/chainsaw-test.yaml`, `packages/system/cozystack-controller/definitions/gateway.cozystack.io_tenantgateways.yaml`, `packages/system/cozystack-controller/templates/rbac.yaml`, `packages/core/platform/values.yaml`, and `packages/extra/gateway/README.md`. No trigger fires, and the two repositories worth ruling out explicitly are below rather than waved off. `cozystack/website`: no package under `packages/apps/` or `packages/extra/` is added, renamed or removed. `packages/extra/gateway/README.md` is edited in place. The docs generator regenerates a reference page from a package README only for the apps hardcoded in the website `Makefile`, and `gateway` is in none of `APPS`, `K8S`, `VMS`, `NETWORKING` or `SERVICES`, so there is no generated page for this package to drift out of step with. That gap is the same shape the trigger map already records for `packages/apps/opensearch` and `packages/apps/bucket`; it predates this change and is neither created nor widened here. None of the other website triggers apply: `packages/core/platform/values.yaml` changes only in a comment, which now says that api, vm-exportproxy and cdi-uploadproxy render a TLSRoute rather than an HTTPRoute, with no key added, moved or given a new default, no platform variant or component moves, no release asset is renamed, `ApplicationDefinition` semantics are unchanged, there is no Talos bump and no developer-tooling change. `cozystack/terraform-provider-cozystack`: this adds an API field, so the field-level triggers are the ones to check, and they are keyed to app charts rather than to this CRD. No app is added under `packages/apps/`; no `values.schema.json` gains, loses or renames a field; no version enum, `values.yaml` default, `kind`, `plural`, `release.prefix` or output Secret/Service name changes. `TenantGateway` is not among the CRDs the provider types by hand. Those are `Package` under `cozystack.io` and `Plan`/`RestoreJob` under `backups.cozystack.io`, and the provider carries no reference to `gateway.cozystack.io` at all. It therefore falls under the map's last bullet: the spec travels as opaque JSON, so the provider keeps working and simply offers no typed access to `tlsPassthroughListeners`, which is the position it already holds for every other field it does not model, and not a break. The provider's new-top-level-field test guard is scoped to `api/apps/v1alpha1`, which this diff does not touch. `cozystack/ansible-cozystack`: nothing in reach. `packages/core/installer/values.yaml` and the `cozy-system`/PSA contract are untouched, no platform variant is added or renamed, none of the `packages/core/platform/values.yaml` keys the role sets are involved (the file changes in a comment alone), no object the role waits on is renamed, and `hack/e2e-prepare-cluster.bats` node prerequisites are unchanged. The schema constraints this diff adds to already-shipped fields all reach objects that already exist, and every one of them is a kind the apiserver ratchets. `spec.apex` gains a 253-character limit (no pattern: a mixed-case apex is admitted and refused by the controller only when passthrough listeners are declared), and `spec.tlsPassthroughServices` gains a DNS-1123 item pattern and a length bound. A stored value that violates one still passes any write that leaves it unchanged, so the rule bites only once the value itself changes. A repeated `tlsPassthroughServices` entry is reported by the controller as `Ready=False` rather than refused at admission, and the reason is not the one this section used to give. Ratcheting does reach a list-type error: `customresource.ValidateUpdate` validates an incoming object's list-type semantics only when the stored object passes them, so a stored duplicate is ratcheted through rather than blocking every write. What makes the marker the wrong tool is the grain of that ratchet, which is the whole object rather than the field, so one stored duplicate would switch list-type checking off for every list on the `TenantGateway` at once and say nothing about it. Documented in `packages/extra/gateway/README.md`, and pinned by `TestTightenedConstraintsOnExistingObjects` together with `TestListTypeSetWouldNotRefuseAStoredDuplicate`, which compiles a copy of the shipped schema carrying the marker to show what the apiserver actually does with one. Shipped defaults conform and the chart already fails on an empty host, so this is a no-op in practice. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added layer-4 TLS passthrough listeners (`tlsPassthroughListeners`) with native port + SNI hostname, rendered as passthrough listeners and limited to TLSRoute attachment. * **Bug Fixes** * Added stricter admission-time and controller validation (reserved ports 80/443, distinct listener ports, tenant-apex hostname rules, and collision prevention with passthrough services). * Improved listener rendering safety: prevents exceeding the Gateway listener cap and ensures AllowedRoutes are isolated per listener. * **Documentation** * Updated gateway docs with listener-cap considerations and layer-4 passthrough usage details. * **Tests** * Added CRD/CEL and reconciliation coverage for valid/invalid configurations and rendering behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
cca3b08733 |
test(backups): pin the Kafka restore path and fail fast on a missing artifact
The restore half of the driver - the half that mutates a live Kafka cluster - had no test reaching its happy path, so three reversions of it left the suite green. - Add restore reconcile tests mirroring the Rabbitmq driver this was copied from: ToCopy (pins the targetApplicationRef.Name merge - a to-copy restore must resolve the TARGET, not write the source's topics back into the source), NoRecordedArtifactURI, TargetClusterNotReady and TargetNotFound. Verified non-vacuous: neutralising the name merge reddens ToCopy, and reverting the fail-fast ordering below reddens NoRecordedArtifactURI. - Fail fast on a Backup with no status.artifact.uri: move the check ahead of the readiness gate so it terminates immediately instead of waiting out the whole deadline and then reporting a misleading "timed out waiting for Kafka cluster to become Ready" (reconcileRabbitmqRestore checks it first; same order here). - Add bats cases for the fail-closed guards that no test reddened: MODE validation, the s3:// URI prefix, cleanup treating 404 as success, an unparsable backup/live describe, a kafka-configs describe error, and an unrecognised record kind (plus a CFG_FAIL knob in the configs stub). The describe-parse guards abort via `set -euo pipefail` at the `X=$(describe_field ...)` assignment, so these pin the fail-closed behaviour; the explicit guards remain as belt-and-braces. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
38d6c4cf58 |
fix(backups): bound Kafka metadata run at the Job layer, not a controller wall clock
A controller-side running-Job deadline was enforced in reconcileKafka / reconcileKafkaRestore: at kafkaDefaultBackupDeadline from StartedAt the BackupJob/RestoreJob was marked Failed. But the batch Job carried no activeDeadlineSeconds and markBackupJobFailed only writes status, so the Job kept running, finished, and its final `s3 PUT` uploaded the object — while the terminal phase meant no Backup was ever created and cleanup (keyed on Backup deletion) never ran. Every timed-out run thus orphaned a uniquely-keyed object, and a cluster whose export legitimately exceeds the deadline could never produce a usable backup. The shipped docs also claimed the run "keeps the BackupJob Running ... rather than failing on a wall clock", which the deadline made false. Move the bound onto the Job as activeDeadlineSeconds (setKafkaJobDeadline): Kubernetes fails an unschedulable or wedged Job as DeadlineExceeded and kills its pod before the upload, so a timed-out run lands in the JobFailed branch cleanly and leaves no object behind. Make it operator-tunable with the `backupTimeout` BackupClass parameter (a Go duration), which round-trips through the Backup so a restore is bounded by the same value; the readiness-precondition wait honours it too. Correct the docs. Also requeue on a transient broker-image List error instead of failing the whole run: resolveKafkaBrokerImage now wraps a sentinel (errNoKafkaBroker) so only a Ready cluster with no listable broker is terminal. Tests: pin the deadline on the built Job, the backupTimeout override, a JobFailed run marking the owner Failed, and the transient-List requeue. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
4925a84c26 |
fix(backups): pin the restore-side guards and bound a running Kafka Job
- delete_topic waited with `grep -qx` (BRE), so deleting `audit.events` matched the live `audit-events` and spun to its timeout — the e2e failed at step 25 on every run that reached it. Use `grep -qxF`. - The two -k/--connect-to guards were written as `! grep`, which cannot fail under hack/cozytest.sh (it wraps each @test in `set -e` + a trailing `return 0`, and bash exempts a `!`-negated command from errexit). Rewrite them as positive `if grep ...; then return 1; fi`; verified the -k reinstatement now reddens the suite under cozytest. - Cover the restore-side literal-topic guard that was unpinned at every level: a bats case runs the existing-topic branch with a colliding sibling and asserts the `--alter` argv carries `\Q…\E`, and run-all.sh now leaves `audit.events` live so the e2e exercises that branch instead of only `--create`. Add bats cases for the partition-shrink refusal, the `-f` upload guard, and `--exclude-internal`. - Bound a running Job: reconcileKafka / reconcileKafkaRestore now fail the BackupJob/RestoreJob once kafkaDefaultBackupDeadline elapses in the Job-running branch (buildJobStrategyBatchJob sets no activeDeadlineSeconds), so an unschedulable pod no longer pins it in Running forever. Pinned by TestReconcileKafka_FailsWhenJobNeverCompletes. - Fail closed on an unknown record kind and an unknown MODE, and validate the live partition/RF parsed on restore (an empty value silently skipped reconciliation). Correct the stale "image pinned in the strategy" comment in the example helpers and the chainsaw test (the controller now resolves the target broker's image as .ClientImage). Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
0d918e1f91 |
fix(backups): resolve Kafka backup client image from the target broker
The Kafka metadata strategy pinned kafkaBackupClientImage to an upstream Strimzi tag with no producer: nothing stamped it and Renovate does not manage helm-values, so it drifted from the broker version the operator provisions. A dedicated small client image (the sibling curl-only drivers' follow-up) is not viable here because the driver runs the Kafka Admin-API CLI, which only the broker image ships. Resolve the client image from a live broker pod of the target Strimzi Kafka at reconcile time instead, so the CLI always matches the broker (any version the operator picks, not only the chart default), and record it on the Backup so cleanup can reuse it once the source cluster is gone. The chart no longer pins the image; the strategy template renders it as the .ClientImage placeholder the controller fills. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
14c6af122e |
fix(backups): restore Kafka S3 port workaround and hold the script with a test
- Regression fix: an earlier change dropped the curl --connect-to port workaround along with -k, but curl 7.76.1 (the image's) 403s on ANY ported endpoint, which is the shape backupStorage.endpoint ships (:8333) and what provisionBucket:false and s3.ingress.enabled:false produce. Restore --connect-to (sign a portless URL, redirect to the real port), keeping -k off so TLS is still verified. IPv6-safe host/port split, path prefix preserved. The port-less ingress passed; the ported case is the one that regressed. - Correct the S3 envelope claim in values.yaml and backup-classes.md: the 403 is decided by the port, not a body-hash the backend recomputes, and --connect-to handles it on SeaweedFS and MinIO alike. - Hold the script fixes at the layer the contract lives at. Externalise the driver to files/kafka-backup.sh (embedded verbatim via .Files.Get, render byte-identical) and add hack/kafka-backup-script.bats, which EXECUTES it against stubbed kafka CLIs + curl and asserts behaviour: \Q literal match on the colliding pair, --connect-to with no -k on a ported endpoint, fail-closed --list, comma-value bracket grouping, RF-divergence abort, and the cleanup delete's HTTP handling. Not a template grep. - Pin the Backup-delete dispatch and the readiness deadline in Go: TestCleanupOnDelete_Kafka_RoutesToKafkaCleanup and TestReconcileKafka_FailsAfterReadinessDeadline. - Docs: config restore is additive (no --delete-config); export time scales with topic count and the Job has no wall clock (as its siblings). Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
ed3cb34cbf |
fix(backups): adopt-guard Kafka Backup artifact and correct URI-template doc
- createKafkaBackupArtifact now rejects adopting a same-named Backup that describes a different application, so a retained Backup reused for a different app of the same name is not reported as this run's success. Mirrors the RabbitMQ driver's guard; pinned by a Go test. - ArtifactURITemplate godoc said empty leaves status.artifact unset, but the driver hard-fails an empty template. Reword to state it is required, and regenerate the CRD description to match. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
0270650237 |
fix(backups): harden Kafka metadata S3 handling and own the artifact
- Drop `-k`: mirror the RabbitMQ S3 layer (ARTIFACT_URI as the single source of truth, forcePathStyle honoured, whole base URL kept) so curl --aws-sigv4 verifies TLS against the ACME endpoint the helper returns. The derived endpoint is default-port, so the curl 7.76.1 --connect-to port workaround is gone too; this also fixes the IPv6 and path-prefix mangling in the old host-splitting. - Own the artifact: a cleanup mode in the script plus cleanupKafkaBackup wired into BackupReconciler.cleanupOnDelete delete a deleted Backup's object (best-effort, waited on) instead of orphaning it, matching the RabbitMQ precedent. Kafka has the identical own-your-object property. - Restore probes existence with --list (fail closed), so an unreachable broker aborts instead of being misread as "topic absent" and created. - Hold the fixes with tests at the right level: a colliding topic pair (audit.events / audit-events, distinct partition counts) in the e2e so a revert of the \Q...\E literal-match breaks the round-trip, and Go tests pinning the scope stamp and the cleanup Job. - Correct the docs/comments: endpoint-shape row, the "cached on nodes" claim, and the narrower curl 7.76.1 S3 envelope (empty-body payload hash) that makes a small modern-curl client image the right follow-up. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
537aadf389 |
fix(backups): fix Kafka backup S3 signing and literal topic names
Reproduced end-to-end against a real Kafka 3.9.1 broker: - [CRITICAL] The Strimzi image ships curl 7.76.1, whose --aws-sigv4 omits a non-default port from the SigV4 canonical Host while sending it in the Host header, so every S3 PUT/GET 403s and the driver never uploads or downloads. Sign/send the bare host and redirect the connection to the real port with --connect-to (the same fix the topic-data example already carries), and accept the in-cluster self-signed cert with -k. - [CRITICAL] kafka-topics --topic is a Java regex, not a literal: a backup entry for "audit.events" also matched a live "audit-events", so restore repartitioned an unrelated topic irreversibly and backup recorded the wrong partition/replication values. Wrap the name in \Q...\E on every filter call (--describe on both paths, --alter on restore) so it matches exactly one topic; --create and kafka-configs --entity-name are already literal. Applied in the example helpers too. - [MAJOR] cozy-default's Kafka route looked like a data backup at runtime. Stamp scope=topic-metadata on the produced Backup's DriverMetadata and say "topic definitions and configs only; no message data" in the Ready condition, so kubectl describe backup is honest. - [MAJOR] Add reconcile tests for the two Go behaviours a mutation could silently drop: the Strimzi-Ready precondition on backup (BackupJob stays out of Succeeded and creates no Job) and the applicationRef Kind gate on the restore target. - [MINOR] Fail closed on the config-capture grep (exit != 1 aborts) and clarify the image pin has no automated producer. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
405eb25095 |
fix(backups): harden Kafka metadata backup
- Fail closed on topic listing: set -o pipefail, capture kafka-topics --list --exclude-internal exit separately, and abort on failure so an unreachable broker can no longer produce an empty backup reported as Succeeded. An empty list after a successful call stays legitimate. - Check each kafka-topics --describe exit and validate the parsed partition count / replication factor before writing a record, so a timed-out describe cannot write a corrupt orders\t\t\t line. - Store topic configs one per line with key and value in separate tab-delimited fields, sourced from kafka-configs --describe (one config per line), so a value containing a comma (cleanup.policy=compact,delete) round-trips; on restore each config is applied with the value bracketed only when it contains a comma. - Detect topic divergence on restore instead of a silent --if-not-exists no-op: grow partitions with --alter when the backup asks for more, and fail loudly (naming the topic and both values) when it asks for fewer partitions or a different replication factor. A restore that cannot reach the recorded state no longer reports Succeeded. - Add KafkaStrategyKind to the RestoreJob cleanup-dispatch no-op group so deleting a Kafka RestoreJob does not run a pointless Velero cleanup; add a regression test alongside the siblings, plus a reconcile Kind-gate test and a rendering test that actually renders the strategy template. - Wire the kafka-metadata Chainsaw suite into both selectors (select-install suite_to_source, select-e2e src_to_suites) so editing the kafka app or the suite selects it and the bats unit tests pass. - Pin the client image to the operator's default broker version (kafka 3.9.1) with a digest, and align the example default; fix the values comment. Point the RabbitMQ comment references (which shipped in the CRD description) at the Job strategy actually mirrored, and trim the in-pod comments that rendered into the manifest. - Document the Strimzi Topic Operator caveat: for a topic owned by a KafkaTopic CR the operator reconciles its config from the CR, so an in-place config restore is durable only for out-of-band topics. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
a410a3f48b |
feat(backups): Kafka topic-metadata backup via a dedicated strategy driver
Add a strategy.backups.cozystack.io/Kafka driver that backs up the topic metadata of a Cozystack-managed Kafka application — partition counts, replication factors and non-default topic configs — through the Kafka Admin API, and restores it with kafka-topics --create + kafka-configs --alter. Modelled on the Job/Rabbitmq strategies: the driver renders a one-shot batch Job, gates on the Strimzi Kafka cluster reporting Ready, and keys each backup by .BackupName so runs do not overwrite each other. Scope is deliberately narrow: message payloads, consumer-group offsets, ACLs, quotas and KafkaUsers are out of scope, and there is no PITR. Restored topics are created directly (unmanaged on the target) — the accepted tradeoff of an Admin-API-only driver. Both restore flows are implemented: in-place recreates dropped topics into the source, to-copy applies the source's topics onto a freshly-bootstrapped empty cluster. Wire it into cozy-default (strategy-kafka-default.yaml gated on the bucket name, a BackupClass binding, read RBAC on kafka.strimzi.io/kafkas for the readiness gate, and the Strimzi Kafka client image), document it in backup-classes.md, and add an honest round-trip example plus Chainsaw e2e (examples/backups/kafka-metadata, kafka-3-metadata-roundtrip) that proves a per-run retention.ms sentinel survives both restores. Unit tests cover the driver's pure helpers (application-ref validation, readiness verdict, parameter round-trip, per-run artifact-URI keying). Not yet verified on a live cluster. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
9121456ba8 |
chore(cozystack-controller): name the Service and ReferenceGrant reads
Deciding whether a TLSRoute forwards anywhere reads the backend Service and the ReferenceGrant admitting a cross-namespace reference to it, and both are watched rather than fetched on demand. The read-everything rule at the foot of the ClusterRole covers them today, so nothing is broken. Naming them keeps the feature working the day that rule is narrowed, where an informer starting without a grant fails with Forbidden and takes the manager with it. The kubebuilder markers in the package generate nothing: controller-gen runs over ./api/... alone and writes no rbac artifact, so the shipped role is hand-written and only a test connects the two. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
2bf9ae4a27 |
feat(gateway)!: serve passthrough hostnames and stop terminating them
Each tlsPassthroughListeners entry renders a mode=Passthrough listener on its own port, answering the entry's hostname by SNI and admitting TLSRoutes from the publishing tenant's namespace alone. It declares no allowedRoutes.kinds: at the pinned Cilium v1.19.5 a listener that spells its kinds out has them applied to every route on the Gateway rather than to itself, so naming TLSRoute here would reject every HTTPRoute on the Gateway. A hostname a passthrough listener answers no longer earns an HTTPS-terminate listener or a Gateway-issued certificate. That is the break, and it is keyed on service rather than on declaration: the same Cilium builds a passthrough filter chain per route out of the backends that survived resolution, so a listener no TLSRoute attaches to, and a route whose backendRefs resolve to nothing, both match no ClientHello and leave the terminate listener holding the name. Eligibility asks what v1.19.5 asks and no more, the reference allowed and the Service present, and fails closed on a read it cannot complete. Services and ReferenceGrants are watched, so a withdrawal deferred on a missing backend lands on the pass that sees it arrive, and the Service watch takes only the creates and deletes, existence being all that is read. Which hostname a route claims is answered the same way. spec.hostnames is optional on a TLSRoute, and ComputeHosts substitutes the hostname of each TLS listener the route's parentRefs select for a route that declares none, so such a route carries the SNI and takes the name over exactly as one that spells it out. A declared hostname broader than the listener's is that substitution from the other side: the route is served on the listener's name and not on the wildcard it wrote, so a claim covering a reserved name counts as a claim on it. The substitution reaches only the listeners that would admit the route. A native-port listener takes the publishing tenant's namespace alone, so Cilium never puts a route from elsewhere on it and never serves that route on its hostname; borrowing the name anyway refuses the route for a namespace, over a hostname its own object does not carry, in the same pass that sheds a terminate listener because that same route is serving a name it borrowed legitimately. A route that names the listener in its sectionName keeps the borrow, since there the refusal is the answer to what it asked for. A route the Gateway renders no listener for hears it from this controller instead of from Cilium alone: the hostname a passthrough listener took, the hostname nothing answers, the sectionName or port naming a listener that answers something else, the sectionName naming no rendered passthrough listener, and the native-port listener that admits the tenant alone, each with its own Gateway API reason and a message naming the hostnames that lost. An HTTPRoute pinning a rendered passthrough listener by sectionName is refused there too, since such a listener forwards a stream it never terminates and takes TLSRoute alone by protocol. Nothing else tells that route: the port-443 listeners name HTTPRoute in their kinds to keep every port-443 set uniform, which the Gateway reports back as InvalidRouteKinds rather than as a grant, and the pin reads a listener's kinds against every route on the Gateway, so it reports the route accepted. The field is confined to certMode http01, stated as the modes that can carry it so one added later is refused until someone rules on it. The wildcard modes cover the whole apex with one terminate listener the pinned Cilium cannot keep apart from a passthrough listener beneath it, and edge terminates TLS at the class provider and renders no TLS listener at all. An HTTPRoute claiming a wildcard is refused under http01 before the race and the overlap are read. The HTTP-01 challenge issues no wildcard certificate, so the terminate listener rendered for the claim could never carry the name, and the listener name built from the hostname's first label carried the asterisk, which the Gateway CRD admits in a hostname and refuses in a name, so the apiserver refused the whole Gateway write and every other listener on it froze where it stood. The renderer now writes that label as "wildcard", so every hostname the CRD admits yields a name it admits, and no name that was already admissible moves. That refusal is a statement about the terminate listener, so it is judged where that listener is decided rather than ahead of everything else. The port-80 listener carries no hostname and serves a wildcard claim as readily as any other, so a route that selects it, by pinning the port or by naming the listener, never needed the listener HTTP-01 cannot supply and is served rather than refused. Judged first, the refusal reached that route too and reported it unserved while Cilium was serving it in plaintext. A parentRef port on an HTTPRoute is read the way servableOn reads it for a TLSRoute. The terminate listener sits on 443, so a route pinning another port attaches to no listener the render would give it, and it used to earn a listener and an ACME certificate for a route that attached to nothing. Port 80 is the plain-HTTP listener, which admits the publishing tenant alone: a route from there pinning it is served plain and earns neither a listener nor a refusal. A route status write that fails is told apart by kind. A conflict or a route that vanished between the list and the write is a race between the two status writers this Gateway has: the reconcile requeues and retries the write without pinning Ready=False on the tenant, and the status pass runs after the http-to-https redirect, so such a race cannot withhold the one object that guard exists to provide. A write the apiserver will refuse on every pass still fails the reconcile, after every other route has been written, because the condition is the only object saying why a hostname is not served, and a pass that reported success over a lost write left the route reading Accepted=True until the resync. The retryable half is logged on the way out. Declining to pin Ready=False leaves the requeue as the only thing acting on it, and the requeue says nothing: the tenant goes on reading Ready, no event is recorded, and returning no error opts out of the backoff that would widen the interval, so a write failing on every pass retries at a fixed period indefinitely with nothing anywhere to show for it. A TLSRoute claiming a hostname broader than the listener's is pulled into the eligibility question for each name it covers, because the pinned Cilium serves it on the listener's own name. That is right only where the listener answering a name is named exactly that name. Under a wildcard entry the substitution hands such a route the wildcard, so its chain carries "*.db.<apex>" while the terminate chain carries the exact name beneath, and Envoy matches an exact server name ahead of a wildcard, so nothing collides; pulling the route in there withdrew a served endpoint and handed its name to the database behind the passthrough listener. The pull-in is now keyed on an exact listener match. Every condition this controller writes on an HTTPRoute, and the one it writes on a TLSRoute declaring no hostnames, is written under http01, and the whole-apex modes collect no claims, so a switch of certMode left those verdicts standing over listeners the new mode does not render. The whole-apex pass now drops this controller's entry from HTTPRoutes and from hostname-less TLSRoutes; the edge refusal stays confined to routes that declare hostnames, since a refusal written on a lent hostname would have no path back. Within http01 the same verdicts outlive their listeners for a second reason, and no mode change is needed to reach it. The status pass walks the routes that attached rather than the ones that claimed a hostname, and retracts its entry from the difference, because a claim set is not a route's own property once a hostname can be borrowed: a TLSRoute declaring none takes the hostname of every passthrough listener its parentRef selects, so removing that listener empties the claim set while the route stands unchanged, and an HTTPRoute reaches the same state by dropping its hostnames. Judged on claims alone such a route keeps whatever the last pass that did judge it left behind, describing a listener the Gateway no longer has. An HTTPRoute pinned by sectionName to a passthrough listener this Gateway does not render is refused with NoMatchingParent, the reading a TLSRoute in the same shape already got, instead of earning a terminate listener it could never attach to. A route served on the port-80 listener, by parentRef port or by naming it, earns no terminate listener and is refused nothing, but it publishes the name from its namespace all the same, so it stays in the hostname race. Every recount now ranks it with the field it recounts, because a recount that left it out crowned the namespace that had lost to it and dropped a loss the first count recorded: a third route refused in a third namespace, or a passthrough entry declared for the name, flipped the loser to Accepted=True. Selecting the port-80 listener by name is judged as selecting it by port: from a namespace the listener admits the route is served plain, from anywhere else the listener turns it away and the route reads NotAllowedByListeners, and a sectionName and a parentRef port that select different listeners select none and read NoMatchingParent, which is what Gateway API requires when both are given. Before this a route naming the listener from a refused namespace earned a terminate listener and a certificate it could never attach to. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
1f04e31ed3 |
feat(backups): add Redis backup/restore strategy driver (#3992)
## What this PR does Adds backup/restore support for the `Redis` app (spotahome `RedisFailover`) via a dedicated `strategy.backups.cozystack.io/Redis` strategy controller. The spotahome operator ships no backup CRD, so the driver follows the generic-script-Job pattern (a specialisation of the Job/Altinity drivers): it runs a one-shot Job that dumps an RDB snapshot from the current master to the system bucket and replays it on restore. - **Backup**: discover the current master via the operator's sentinel Service (`rfs-<name>`, master `mymaster`) → `redis-cli --rdb` → `curl --aws-sigv4` PUT to `s3://<bucket>/<namespace>/<app>/dump.rdb`. - **Restore**: GET the object → convert RDB → RESP (`hdt3213/rdb`) → `FLUSHALL` + `redis-cli --pipe` into the target's master. Both **in-place** and **to-copy** are implemented; a to-copy restore reads the source's prefix via `.Backup.ApplicationRef.Name` while writing into a differently-named target. - **PITR**: not supported — RDB is a discrete snapshot with no continuous log to replay to an arbitrary time (the etcd-like case). The driver is a specialisation of the Job/Altinity script-Job flow with a `Redis` applicationRef gate and an app-readiness precondition (surfaces a precise `Ready=False` instead of a silent hang; blocks only on an explicit `Ready=False` so a missing condition never wedges backups). Strategy parameters round-trip through the Backup's `DriverMetadata`; there is no `underlyingResources` snapshot (the Job pattern re-renders on restore). ### System bucket Coordinates and credentials reach the Job as env `secretKeyRef` from the projected `cozy-backups-creds` (the ClickHouse/Altinity delivery model), so **the app chart is untouched** — no `backup.useSystemBucket`, no new RBAC (`batch/jobs` and `apps.cozystack.io/*` are already granted). The bare-host `endpoint` gets `https://` prepended when unscheme'd; against a self-signed endpoint the strategy verifies with the CA from `backupStorage.endpointCASecretName` when set, otherwise against the image's system CA store; skip-verify is a separate explicit `backupStorage.insecureSkipTLSVerify` opt-in, never a fallback. The `cozy-default-redis` strategy is gated on `$bucketName` (CRD-ordering) and bound in the `cozy-default` BackupClass. ### Changes - `api/backups/strategy/v1alpha1/redis_types.go` + generated CRD and deepcopy. - `internal/backupcontroller/redisstrategy_controller.go` (+ unit tests) wired into both dispatch switches, `supportedBackupStrategyKinds`, and the no-op restore-cleanup case. - `cozy-default-redis` strategy template + `cozy-default` BackupClass route + helm rendering-gate coverage. - `images/redis-backup` (redis-cli + curl + RDB→RESP converter) + Makefile target + `backupStrategyController.redisBackupClientImage`. - `examples/backups/redis/` — an honest marker round-trip for both restore flows. ### Validation `go build`/`vet`, `go test ./internal/backupcontroller/... ./api/backups/...`, `make generate` (clean), and `helm unittest` (10/10) all pass. Not yet cluster-verified. ## Downstream None. ```release-note Add backup and restore support for the Redis application (spotahome RedisFailover) via a new Redis backup strategy, wired into the cozy-default BackupClass. Supports in-place and to-copy restore; PITR is not supported. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Redis backup and restore support using RDB snapshots in S3-compatible storage. * Supports restoring data in place or to a separate Redis instance. * Added configuration for Redis backup images, custom HTTPS certificate authorities, and TLS verification. * Added a complete Redis backup and restore demonstration with automated setup, verification, and cleanup. * **Documentation** * Documented Redis storage, credentials, endpoints, TLS, and restore behavior. * Added administrator and tenant scenario guides. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
afc003c464 |
feat(postgres): database read-replica autoscaling via KEDA (#3954)
## What this PR does Adds automatic horizontal autoscaling of a PostgreSQL database's **read replicas** in response to load, implementing the merged design proposal [Database Horizontal Autoscaler](https://github.com/cozystack/community/tree/main/design-proposals/database-horizontal-autoscaling) (community#44). The mechanism is entirely stock — no bespoke operator, no new CRD: - **KEDA as an optional platform component** (`packages/system/keda`, wired into `core/platform` as a source + system-bundle entry). It claims the `external.metrics.k8s.io` APIService that nothing served before. - **`cozy-lib.keda.scaledObject` helper** (`packages/library/cozy-lib/templates/_keda.tpl`) renders a KEDA `ScaledObject`, fail-closed on missing/invalid arguments. - **`packages/apps/postgres` autoscaling** — an `autoscaling` values block (`enabled`, `minReplicas`, `maxReplicas`, `metric`, `target`, `maxReplicationLagSeconds`, `dryRun`, `transition`) validated by `values.schema.json`; under active autoscaling the chart renders a `ScaledObject` and seeds `spec.instances` to a constant floor (`max(replicas, effectiveMin)`), while the postgres release is forced to client-side (two-way) Helm apply so the seed is a merge no-op on upgrade and the KEDA-managed HPA — the runtime writer via the CNPG `Cluster` scale subresource — is not reverted. - **Single-value metric (§1)** — one PromQL query returns `Σ(read load over standbys) + target`, so a stock HPA computes `desired = ceil((Σ+target)/target) = 1 + ceil(Σ/target) = primary + desiredRead`, with no per-pod emission. - **The four static brakes (§5)** — quorum floor `max(minReplicas, maxSyncReplicas+1, 2)` (quorum wins over `maxReplicas`), scale-down pacing (`Pods/1/600s`), a replication-lag clamp folded into the query (freeze both directions while lag exceeds the threshold and the primary is writing, with `max_over_time` hysteresis), and dry-run (render paused). - **Two-phase enable/disable migration** via a `transition` sub-flag: phase 1 stages `spec.instances` to the live count and brings KEDA up with its HPA paused in both directions; flipping to phase 2 hands the runtime count to KEDA, and because the seed re-renders the same count the flip is a merge no-op with no dip. - **Observability** — a database-autoscaling Grafana dashboard and PrometheusRule alerts. **Live validation (dev cluster, CNPG 1.30.0).** The full path works end-to-end: the CNPG `Cluster` scale subresource serves `status.selector`, KEDA renders the HPA, the metric reaches it (`ScalingActive=ValidMetricFound`, `TARGETS 151/150`, `desiredReplicas=2 = 1 primary + 1 replica` — the §1 arithmetic), and the lag-clamp query returns the base value on live VictoriaMetrics. Two packaging fixes were needed and are included: KEDA must run on the cozystack `cozy.local` cluster domain (the upstream `cluster.local` default makes the metrics gRPC target NXDOMAIN and every HPA hangs on `FailedGetExternalMetric`), and the operator's client-go throttle is raised so a single-replica operator does not lose its leader lease under scale-subresource polling. Implements #3953. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. Two repositories are affected; both are follow-ups to open once this lands (and after its dependency #3951 merges and the values surface is final), left unticked here for a maintainer to confirm rather than claimed with a speculative link: - **cozystack/terraform-provider-cozystack** — this adds the nested `autoscaling` block to `packages/apps/postgres/values.schema.json`. The provider hand-models each app's fields (schema + model + expand/flatten), so it needs the new block to expose it in Terraform. - **cozystack/website** — KEDA is a new platform component (`guides/platform-stack`), and the postgres reference page gains the `autoscaling` params. The managed-app reference is regenerated from `README.md` by the release bot; the platform-stack component list is hand-maintained. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: to open (new platform component + postgres autoscaling params) - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: to open (new `autoscaling` block in postgres values.schema.json) ### Depends on - #3951 — bump CloudNativePG to 1.30.0. The HPA can only drive the CNPG `Cluster` scale subresource on CNPG ≥ 1.28.4, where it serves `status.selector`; without it the HPA fails with `InvalidSelector`. This PR should not merge before #3951. ### Release note ```release-note feat(postgres): add optional horizontal autoscaling of read replicas via KEDA, configured through the new `autoscaling` values block; KEDA ships as a new optional platform component. Requires CloudNativePG >= 1.28.4 for the Cluster scale subresource selector. Note: every Postgres release now applies client-side (`release.cozystack.io/helm-server-side-apply: "false"`) so the autoscaler's live `/scale` count is not reverted by Helm; a kind-wide side effect is that chart-rendered fields on Postgres releases are no longer re-asserted by server-side-apply drift correction (driftDetection is unset), a documented trade-off with a per-instance apply strategy as follow-up. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable PostgreSQL read-replica autoscaling with KEDA. * Added CPU and read-connection metrics, replica bounds, replication-lag safeguards, dry-run mode, and transition handling. * Added optional KEDA platform support, Grafana autoscaling dashboard, and Prometheus alerts. * **Documentation** * Clarified integer thresholds, prerequisites, configuration, and transition guidance. * **Bug Fixes** * Improved validation and actionable errors for unavailable KEDA or invalid settings. * **Tests** * Expanded coverage for autoscaling, observability, package selection, and validation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
b4d621f4f7 |
fix(backups): delete Redis backup objects on cleanup and size the restore Pod
Delete the backup object when its Backup is removed. The Redis driver
owns its S3 object (one per BackupJob, no engine retention prunes it), so
it now spawns a cleanup-mode Job that deletes the object and holds the
Backup's finalizer until it completes, mirroring the Rabbitmq driver.
Without this a retention-pruned Plan grew the bucket unboundedly and the
key was lost with the CR.
Size the strategy Pod from backupStrategyController.redisBackupResources
so an admin can raise the restore ceiling (the throwaway loader holds the
whole dataset in memory) via chart values, which a direct edit of the
cluster-scoped, Helm-managed Strategy CR could not survive. The
loader-not-ready diagnostic now names the memory limit, the likeliest
cause, instead of only the image version.
Make the endpoint CA volume optional: nothing projects that Secret into
the app namespace, so a required mount wedged every backup Pod on
FailedMount; a missing ca.crt now falls through to the system CA store
and fails the handshake fast, which is not the same as skipping verify.
Promote the Pod-level activeDeadlineSeconds to the Job so a wedged run is
bounded once, not (backoffLimit+1) times. Use find, not ls, for the base
AOF so its no-match diagnostic is not dead code under pipefail.
Pin the object key reaching the rendered Job: both fixtures now render
--key={{ .ObjectKey }} and assert it, so zeroing the key (which left the
package green) is caught. Document .ObjectKey and the cleanup mode in the
CRD render-context, list cozy-default-redis among the Helm-managed CRs,
and mirror the CA/TLS and find fixes into the example strategy.
Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
|
||
|
|
919efa044f |
fix(backups): branch the Redis Job-existence check on IsNotFound
The readiness re-gating guard read `r.Get(...) == nil` to mean "the Job does not exist yet", folding any non-NotFound error (an apiserver hiccup, an RBAC denial) into that branch. On such a transient error the driver would re-enter the readiness precondition even though the Job may already exist and have completed, reintroducing the spurious terminal failure this guard removes. Switch on the Get error: nil means the Job exists (observe it), NotFound means run the pre-launch readiness gate, and any other error requeues without deciding. Same on the backup and restore paths. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
ff061a8eea |
fix(backups): stop re-gating Redis readiness after the backup Job exists
The Redis backup/restore reconcilers checked the application's Ready condition on every pass, ahead of the Job-state switch. Once the Job has been created that re-check is wrong: if the app's Ready flips False after launch (an unrelated HelmRelease blip) and stays so past the deadline, the driver marks a run Failed whose Job may already have completed, a spurious terminal failure of a successful backup or restore. Gate the readiness precondition on the Job not yet existing; once it does, observe the Job directly. The deadline still bounds the pre-launch wait. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
|
632f7b31a0 |
test(backups): cover Redis backup-path unmappable kind
The restore path already asserts a terminal failure for an applicationRef that passes the Redis gate but has no REST mapping; add the symmetric backup-path case so both dispatch sides are covered. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
|
b0eb104276 |
feat(backups): add Redis backup/restore strategy driver
The spotahome RedisFailover operator ships no backup CRD, so add a dedicated strategy.backups.cozystack.io/Redis driver that runs a one-shot Job: it dumps an RDB snapshot from the current master (discovered via the operator's sentinel Service) to the system bucket over S3, and on restore replays it (RDB -> RESP via redis-cli --pipe) into the target's master. Structurally a specialisation of the Job/Altinity script-Job pattern with a Redis applicationRef gate and an app-readiness precondition. Backup parameters round-trip through the Backup's DriverMetadata (no underlyingResources snapshot); restore drives from .Backup.ApplicationRef so a to-copy restore reads the source's object-storage prefix while writing into a differently-named target. Both in-place and to-copy are supported; PITR is not (RDB is a discrete snapshot). - api: Redis/RedisList strategy type + generated CRD and deepcopy - driver: reconcileRedis / reconcileRedisRestore + pure helpers, wired into both dispatch switches, supportedBackupStrategyKinds and the no-op restore cleanup case - default: gated cozy-default-redis strategy (system-bucket coords via cozy-backups-creds env secretKeyRef, https:// scheme + optional CA) and the Redis route in the cozy-default BackupClass - image: images/redis-backup bundling redis-cli + curl + an RDB->RESP converter, wired via backupStrategyController.redisBackupClientImage - tests: driver unit tests (gate, precondition, render, param round-trip, both reconcile paths) and the helm rendering-gate coverage - example: examples/backups/redis honest marker round-trip for both restore flows Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
|
54621bc046 |
feat(migration): tenant self-service VM import from VMware vSphere
Hidora's customers are leaving VMware now, and a tenant has to be able to import their own machines without a platform administrator. The chart approach could not express that: a tenant can neither create the Secret holding vCenter credentials nor name the VDDK image, so both had to come from somewhere they cannot reach. Two CRDs in a new `forklift.cozystack.io` group, split by lifecycle. A VMImportSource is a long-lived connection -- type, endpoint, credentials, readiness -- and a VMImportTask is the one-shot operation that names machines and produces VMDisks and VMInstances. The outputs carry no owner reference back to the task, so deleting a finished import leaves the machines alone. Konveyor Forklift does the transfer underneath; the controller renders its objects, mirrors its verdicts, and hands each finished volume into a VMDisk without copying it. What a live vCenter migration forced out, and no amount of source reading would have: - A guest copied verbatim keeps the drivers it had under VMware and no virtio, so on a virtio disk it does not fail the import -- it succeeds and then does not boot. Imported disks go on SATA. - VDDK connects straight to the ESXi host at the address vCenter advertises, which is routinely unreachable, and on the first cluster collided with the Service CIDR. `spec.hosts` redirects per host. - Firmware must be carried across or a UEFI guest imports as BIOS and never boots (stacked on #3002). - Deleting a task mid-transfer orphaned its DataVolume; a finalizer now removes the engine's volumes and leaves the outputs. The console gains a Migration section: connections are editable, and the VM field is a picker built from a machine list the controller publishes, since the aggregated API cannot reach the inventory and should not hold a source's credentials to try. Two general mechanisms fall out of that -- an option source may take an argument, and one named in a CRD annotation may reference a sibling field. Design: cozystack/community#62. Docs: cozystack/website#673. Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
|
d685fb30ec |
fix(gateway)!: require tenant routes to declare their hostnames (#3475)
## What this PR does The tenant route hostname policy skipped the shape it most needed to check. Its CEL opened with `!has(object.spec.hostnames) ||`, so an HTTPRoute or TLSRoute declaring no hostnames satisfied the first operand and got admitted with nothing compared against the tenant apex. An empty list passed too: `has()` is true on `[]`, and `all()` over an empty list is vacuously true. Such a route is not inert. Under Gateway API it inherits the hostnames of every listener it attaches to, and `parentRefs` can name a Gateway in another namespace, so it covers whatever that listener covers. Reported in #3467. The predicate now requires the field to be present and non-empty before validating it. Both policies rendered from that template share one CEL string, so HTTPRoute and TLSRoute get the change together, and both match `CREATE` and `UPDATE`, so an existing route that drops its hostnames is denied exactly as a new one is. The issue is titled after TLSRoute, but the reachable shape is mostly HTTPRoute. TLSRoute allows a hostname-less route only on the deprecated `v1alpha2`; at `v1` and `v1alpha3` the bundle this repo vendors marks `spec.hostnames` required and rejects an empty list on `minItems: 1`, while `v1alpha2` carries neither constraint. HTTPRoute `v1` carries neither either, so it accepts both shapes. All four are checkable in `packages/system/gateway-api-crds/templates/crds-experimental.yaml`. Checking that turned up a stale claim in the template header, which said TLSRoute had no `v1` in Gateway API 1.4 to 1.5 and used it to justify the rule naming only `v1alpha2`. Gateway API 1.5.1, which this repo vendors, serves TLSRoute at `v1` as the storage version. The rule still covers every served version because `matchConstraints.matchPolicy` defaults to `Equivalent`, and that default is now written down, because setting `matchPolicy: Exact` reads like a tightening and would silently stop the policy seeing `v1` and `v1alpha3` TLSRoutes. I corrected the text and left the coverage alone: the wrong sentence predates this branch, but my change is what makes the file contradict itself, so the text is mine to fix while widening version coverage is a separate job. A `ValidatingAdmissionPolicy` has no cross-object lookup, so it cannot resolve the parent Gateway to see which hostnames a route would inherit. Requiring the route to name its own is what keeps every admitted hostname comparable against the apex. That has a cost: a route relying on listener inheritance now has to spell its hostnames out, and a tenant that hand-writes one gets a denial where it used to get an admission. ### The controller change belongs in the same PR `renderHTTPRedirect` built the http-to-https redirect HTTPRoute with no hostnames on purpose, to catch every host on port 80. It lands in a `tenant-*` namespace, so the stricter policy denies it, `reconcileHTTPToHTTPSRedirect` returns that error, and since it runs before the status update the TenantGateway never reaches Ready. That redirect is what keeps app routes attaching by hostname from serving plaintext on port 80. Neither half of this change is safe alone. The redirect now names the tenant apex and its wildcard. Both entries are needed: a wildcard hostname is a suffix match over any number of labels, so `*.<apex>` covers subdomains at every depth but not the bare `<apex>`. `renderWildcardCertificate` already builds the same pair. Coverage narrows in one direction. A request on port 80 with a Host outside the tenant apex now falls through to a 404 instead of a 301. That narrowing costs no covered host: the port-80 listener admits routes only from the Gateway's own namespace and cert-manager's, and the tightened policy requires every route in the tenant namespace to declare its hostnames inside the apex, so any host published from this namespace already falls under the apex or its wildcard. An inheriting child tenant does not widen that set either. Its hostnames would reach port 443 through a listener on this same Gateway, and `cozystack-gateway-hostname-policy` compares every listener against the host label of the Gateway's own namespace, so a child apex outside that label never gets its listener: the Gateway write carrying it is refused at admission whole, in `reconcileGateway` and before the redirect is rendered. A child apex under this one passes that check and the wildcard already covers it. That holds at merge base as well, so it is not something this branch changes. Deriving hostnames from `spec.apex` makes that field load-bearing for admission, so it gains `MinLength=1`. An empty apex used to reconcile to Ready, since the redirect declared no hostnames at all; it would now render `""` and `"*."` and be rejected on the HTTPRoute hostname schema, surfacing as a validation error naming the route rather than the field. The renderer checks it too, because the CRD and the controller binary roll out separately and an older CRD in the cluster will not carry the bound. That schema bound is a breaking change to an existing field and the API review gate flags it as one, so here is what it actually invalidates. The chart that renders these CRs already fails on an empty `_namespace.host`, and in DNS-01 or existingSecret mode an empty apex already broke earlier, in `renderGateway`, on a listener with `hostname: "*."`. Nothing that works today becomes invalid. What the bound rejects is a TenantGateway written by hand, past the chart, with an empty apex, and I would rather that be refused at admission naming the field than at reconcile time in controller logs, which is why the bound sits in the CRD as well as the renderer. Naming those hostnames also meant excluding the redirect from `collectHostnameClaims`. A route that attaches to the Gateway and declares hostnames otherwise reads as an app publishing them, and each claimed hostname becomes an HTTPS listener named after its first label. The first label of `*.<apex>` is `*`, which the Gateway CRD's listener-name pattern rejects, so the Gateway write fails and the reconcile stops before anything downstream. The exclusion matches on namespace, name and controller ownership rather than on the labels the renderer stamps: those labels are writable by anyone who can create a route, and the claim set also drives conflict resolution, so a label-keyed exception would let any route opt out of being marked `Accepted=False` when it loses a hostname race. Name and ownership are each required and each is pinned by its own test, since ownership alone would exclude any other controller-owned route and name alone would extend the exception to a route planted under that name by someone else. ### Radius Only namespaces starting with `tenant-` are matched. The three first-party TLSRoutes the platform ships, `kubernetes-api` in `default` plus `cdi-uploadproxy` and `vm-exportproxy`, set hostnames explicitly and none sits in a `tenant-*` namespace, so nothing changes for them. I went through every producer of routes in the tree, in three places rather than one. No Helm template emits an absent or empty hostname list in any configuration a tenant can reach: the vendored openbao, harbor, opencost and flux-operator route templates can render one, but each is behind a flag that defaults off with no values path to turn it on, and openbao would fail CRD validation first because its default route apiVersion requires hostnames. Exactly one Go site builds a route, the redirect fixed here. And one e2e fixture did rely on the old behaviour: the Gateway suite's parentRef probe declared no hostnames and is applied into `tenant-test`, which the policy matches, so it now reads the apex from the namespace label and names a hostname under it, the same way the neighbouring cases in that suite already did. cert-manager also writes routes into tenant namespaces, since it creates the Challenge in the Certificate's own namespace and the HTTP-01 solver route beside it. That route declares the hostname it is validating, so the policy admits it. ### Related issues Two open issues have a text half that this PR happens to settle, and neither of them closes here. #3788 reports that the port-80 allow-list contains the namespace its own comment says it excludes; the comment and the README paragraph now describe what that filter actually does, while the behavioural exit it names stays open. #3789 reports that the TLSRoute rule names only the deprecated API version; that premise is now written down in the template header, including what has to change before a bundle drops `v1alpha2`, but the rule still names `v1alpha2` alone. #3787 covers the wildcard-hostname listener name that the apiserver rejects, which is what makes the claim-set exclusion here load-bearing rather than tidy. ### Screenshots N/A (no UI changes). ### Downstream repositories I walked the trigger map in `docs/agents/contributing.md` against the diff, file by file. No package is added, renamed or removed; no `values.yaml` key, `values.schema.json` field, version enum or default moves; no `kind` or `plural` changes; no namespace or release-asset name changes; nothing under `hack/` moves, is renamed, or changes what a make target does. One row deserves a sentence rather than a blanket denial, because the diff does change a CRD schema: `spec.apex` on `gateway.cozystack.io/v1alpha1 TenantGateway` gains a `minLength` bound, which lands in the generated `gateway.cozystack.io_tenantgateways.yaml`. The provider hand-types `Package` under `cozystack.io` and `Plan`/`RestoreJob` under `backups.cozystack.io`; `gateway.cozystack.io` is not among them, and a tightening on an existing field adds no new field for it to offer typed access to. The `packages/extra/gateway/README.md` edit sits in the `## Security model` section, below the `## Parameters` block that `cozyvalues-gen` regenerates, so it survives `make generate`. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(gateway)!: the tenant route hostname policy now requires an HTTPRoute or TLSRoute in a `tenant-*` namespace to declare a non-empty `spec.hostnames`. A route that omitted the field, or set it to an empty list, was previously admitted without any hostname being checked against the tenant apex, even though such a route inherits the hostnames of the listener it attaches to rather than matching nothing. A route in a `tenant-*` namespace relying on that inheritance is now rejected and has to name its hostnames. The controller-owned http-to-https redirect route names the tenant apex and its wildcard accordingly, so a request reaching port 80 with a Host outside the tenant apex now gets a 404 instead of a 301. `TenantGateway.spec.apex` additionally gains a `minLength: 1` bound, because the redirect derives its hostnames from that field; a TenantGateway written past the chart with an empty apex is now rejected at admission instead of failing later in the controller. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added edge-terminated gateway support with plain-HTTP listeners and TLS passthrough handling. - HTTP-to-HTTPS redirects now explicitly cover the tenant apex and subdomains. - Gateway configurations require a non-empty apex hostname. - HTTPRoute, TLSRoute, and GRPCRoute resources must declare hostnames. - **Bug Fixes** - Invalid hostname-less routes are now rejected consistently. - Controller-managed redirects no longer trigger unnecessary hostname claims, listeners, or certificates. - Switching certificate modes now cleans up obsolete ACME resources and route status. - **Documentation** - Updated security and gateway guidance for hostname validation, edge termination, route compatibility, and certificate handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8738063287 |
fix(operator): reject tar entries containing ".." before any path operation
safeJoin already rejected traversal and re-checked the joined path against destDir, but the guard sits behind a call boundary that CodeQL's go/zipslip barrier does not follow, so the extraction loop reported three high-severity arbitrary-write alerts. Reject the entry name up front instead, which is both the fail-closed spelling and the shape the analysis recognises. Extend the traversal test to cover a ".." buried mid-path, not just a leading one. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
15be9329a1 |
refactor(marketplace): source-centric Tap resource, label-based identity
The aggregated Tap resource no longer derives a community.<org>.<repo> name from the connect URL. A connected repository is identified by its Flux source (tap-<org>-<repo>), and the PackageSource(s) it materializes keep their own declared names. Community is computed from the marketplace-tap label, and disconnect is guarded by that label instead of a name prefix. List also surfaces tap sources with no materialized PackageSource yet (a connect in progress, or one blocked by a collision), carrying the materializer error message, so a blocked tap is visible in the dashboard rather than silently never appearing. The community. prefix constant is removed. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
1653da967b |
refactor(marketplace): drop artifact-prefix injection and reserved names
Without the community. rename, a tapped repository's cozyrds ApplicationDefinition references its component's artifact by the concrete <packagesource>-<variant>-<component> name again, so the operator no longer injects a cozystackArtifactPrefix value and the -rd chart renders its cozyrds asset with a plain Files.Get. cozypkg init scaffolds the concrete chartRef. Reserved name prefixes are removed: repositories choose their own PackageSource names, and a clash with a core component is caught on-cluster at tap time (the collision check) rather than by refusing prefixes offline. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
95badea1c8 |
refactor(marketplace): identify taps by marker label, drop community rename
Tapped repositories keep their own declared PackageSource names instead of being renamed under the community. prefix. Tap-ness is now marked by the marketplace-tap label (and source annotation) on the OCIRepository and every materialized PackageSource, so untap, the Flux-source cleanup, and idempotent re-tap key on the marker rather than a name prefix. A declared name that clashes with an existing core component (or another tap) is rejected: the CLI tap fails fast before applying anything, and the operator materializer refuses to overwrite the foreign object, records the reason on the source (annotation and Event), and does not stamp the revision so a corrected artifact retries. The shared check lives in internal/marketplace/collision. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |