Commit Graph
16 Commits
Author SHA1 Message Date
Myasnikov Daniil b83cc087cc docs: move the repo-only guides onto the website
Seven guides in docs/ were written next to the code and left in the
source tree, where no operator looks. The website carries them now, so
a second copy here only lets the two drift again: the Backup Classes
page had already fallen behind its original.

docs/release.md, docs/deprecating-applications.md and the two SeaweedFS
runbooks in docs/operations stay. A migration, a bats suite, a workflow
or AGENTS.md reads each of them by path, which is the test for keeping
a document in this repository.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-21 17:50:46 +05:00
Andrey Kolkov 46a8bd7363 fix(mongodb): apply the storage once per job, and pin the terms that decide ownership
The system-bucket storage was applied on every reconcile of a job, not
once per job: with the operator CR already minted nothing returned before
the apply, so a streaming dump saw an SSA patch every poll for up to a
day, and the only thing keeping that patch off a cluster that had
re-acquired the chart's tasks and pitr since the mint was a predicate
term no test held. The storage a CR names is fixed for its dump, so the
apply now runs only before the CR exists; the legacy-render hold covers
that window, and a test pins that an existing CR is never injected over.

The other predicate that decides whether a Backup owns an archive
compared the live entry's bucket and credentialsSecret against the
strategy, and only the bucket half had a fixture. The credential is what
tells a tenant's entry from the platform's when a bucket name collides,
so a case varying only that term joins the foreign-bucket one. The
cleanup shortcut's snapshot-kind term and the 24h running ceiling are
pinned the same way, the latter as an absolute boundary pair.

The generated field description said the schema rejects useSystemBucket
without backup.enabled. The aggregated apiserver does not evaluate CEL on
application schemas yet, so the render-time refusal is the one that
holds; the description and the runbook now say so. The runbook's restore
paragraph still described the unconditional credential swap this branch
narrowed, exempted MariaDB from a retention warning its shipped strategy
does not earn, and pointed at a Plan retention field that does not
exist; the example README and a dispatcher test's comment still carried
the pre-branch claim that the driver never injects and never deletes.

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-29 10:14:44 +04:00
Andrey Kolkov a6bb0556fc fix(mongodb): own archives by where the storage points, and bound the open loops
The prune finalizer on a driver-minted PerconaServerMongoDBBackup means
the platform owns the archive: cleanup deletes the operator CR on that
marker and the operator removes the object from storage. It was stamped
from the app's useSystemBucket flag, which says nothing about where the
storage the CR names actually points. With a strategy carrying no s3
block nothing was injected, the presence-only precondition passed on a
legacy tenant-bucket entry, and the CR went out finalized against the
tenant's own bucket; deleting the Cozystack Backup then deleted the
tenant's archive. The finalizer now follows the storage's coordinates as
read back from the live cluster after the injection, and a strategy
without s3 coordinates, or a storage pointing anywhere else, holds the
job instead of minting.

The injection keyed on the same desired value. The app CR is projected
from the HelmRelease's spec.values, so the flag reads true before
helm-controller renders the revision, while the tenant's own storage,
scheduled tasks and PITR stream stay on the cluster. Injecting over that
entry redirected the tenant's nightly dump and oplog into the shared
bucket under platform credentials. A cluster still carrying tasks or
pitr is now left alone and the job held until the release renders the
flag, and the schema carries a rule marking useSystemBucket without
backup.enabled invalid, the pair the chart already refuses at render.

The storage-race retry counted surviving operator CRs against its
budget, but an errored CR is reaped the moment it is deleted (the
operator drops delete-backup without pruning on a non-ready CR), so the
count never left one and the job minted a CR per poll for the whole
deadline. The count now lives on the BackupJob and survives the reap.

A system-bucket dump in running had no bound at all: a wedged pbm agent
left the job Running forever and a nightly Plan stacked another beside
it. It now fails after 24 hours with the operator CR left in place as
the handle to whatever pbm still writes. The deadline cancel pins its
delete to the resourceVersion the live read answered for, and the
artifact-retry failure names the window that was actually in force.

Tests pin the guards that were exercised only in the harmless direction:
a system-bucket snapshot reaching the prune, the resolved prefix reaching
the cluster, a missing live client deferring the cancel, and the
finalizer refused against foreign coordinates.

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-29 10:14:35 +04:00
Andrey Kolkov 5e3b8c217b fix(mongodb): refuse useSystemBucket without backup.enabled and pin the retry's flow
The pbm agents run only when backup.enabled is true, so useSystemBucket
with backups disabled renders a cluster that can never service a
BackupJob, and the tenant learned that thirty minutes into the first job
when the driver's precondition gave up. The chart already fails closed on
the bootstrap clash; do the same here so the combination is a render-time
error with the remedy in it.

The storage-race retry keyed on the app flag while everything else in the
state switch had moved to the flow recorded on the CR's own finalizer, and
that flag term was the only thing keeping a legacy tenant's operator CR
from being deleted and retried on the same operator error — with no test
holding it. Key it on the CR's flow like its neighbours and pin the legacy
side: the error is terminal and the CR survives.

The retry matches the operator's own error wording, which is correct at
the pinned crVersion and nothing else ties together: a bump that rewords
it turns the retry off with the suite still green. Name the coupling at
the pin and at the matcher so a bump re-checks it.

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-22 12:03:58 +04:00
Andrey Kolkov e653a4fa5d fix(mongodb): correct the finalizer-prune claims, pin the pre-read release CR, and fix the retention field text
The delete-backup finalizer prunes storage only for a `ready` backup at
v1.22.0; a timed-out backup never reaches `ready`, so the deadline-cancel
comment no longer claims the finalizer "takes anything partially written" —
nothing is written until the backup runs, which the cancel prevents. The
psmdbBackupTimedOut godoc no longer says the operator "has not started" the
backup, which is wrong for `requested` (the operator sets it as it dispatches);
it now describes the not-yet-running states it actually gates.

The pre-read CR handed to releaseMongoDBCleanup on the terminating path had no
test that failed when the argument was dropped, because the existing cases
either call the function directly or let the second Get succeed. Added a
cleanupMongoDBBackup case whose operator-CR Get fails after the first: the
release must strip from the passed object, not re-read, or the teardown wedges.

The useSystemBucket field said "retention move[s] to the Plan", but a Plan has
no retention field; corrected to "scheduling moves to a Plan; this flow has no
retention yet" and regenerated the schema, README and mongodb-rd. Softened the
"never pins a namespace in Terminating" wording in the cleanup godoc and RBAC
comment: the driver only bounds its own finalizer, the operator's release-lock
finalizer can still hold the CR while the operator is down.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-22 12:03:58 +04:00
Andrey Kolkov cc51e040ae fix(mongodb): keep the cleanup escape hatch off legacy operator CRs, and surface a stuck finalizer
The skip-artifact-cleanup annotation reaches releaseMongoDBCleanup before the
ownership check, and the operator-CR delete sat outside the finalizer gate, so
a legacy backup (no delete-backup finalizer) lost its PerconaServerMongoDBBackup
record when a tenant set the annotation to release a stuck Backup or applied it
across a namespace during teardown. That record is what a psmdb restore-by-name
resolves against, and both the RBAC comment and the docs promise it is never
touched. Gate every write in the release on the finalizer being present: an
unowned (legacy) CR or one that cannot be read at all is left intact, and the
Cozystack Backup still releases.

Surface a failed finalizer strip. The release must proceed even against an
unreachable CR, but the swallowed patch error left the operator CR holding
percona.com/delete-backup — pinning its namespace in Terminating, the outcome
the branch exists to prevent — while the Event reported the success case. Emit
a FinalizerNotStripped Warning naming the object so an operator can find it.

Mark the sibling backup fields (schedule, retentionPolicy, destinationPath,
endpointURL, insecureSkipTLSVerify, s3AccessKey, s3SecretKey) as ignored under
useSystemBucket, so a tenant editing one instead of the flag itself is not
misled by a description that describes only the legacy flow; regenerate the
schema, README and mongodb-rd. Pair the insecureSkipTLSVerify rendering
assertion with a case that flips the platform knob on, so the two together
discriminate the rendered value from a hardcoded default.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-22 12:03:58 +04:00
Andrey Kolkov f27a02f1b6 docs(mongodb): correct the inlined-storage story and surface the TLS knob
The strategy godoc still said the storage config is NOT inlined and the
strategy only names the storage — which controller-gen copies verbatim into
the CRD, so kubectl explain served the new s3 field and its own denial from
one object. Rewrite it (and the StorageName field doc) to describe both
flows, and regenerate the CRD.

Render insecureSkipTLSVerify on the default MongoDB strategy: the field is
plumbed end to end into the driver-injected storage entry and backed by a
platform knob, but the template set only forcePathStyle, pinning it to
false so a self-signed external S3 that Redis backups honour would fail
MongoDB x509 verification. Pin it in the rendering test alongside the other
s3 coordinates.

Fix the useSystemBucket field description: angle brackets rendered as empty
HTML tags in the README table and the dashboard schema, so wrap the path in
backticks (as the postgres field already does) and note that schedule,
retentionPolicy and PITR are ignored on this flow. Regenerate the chart
schema, README and mongodb-rd.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-22 12:03:58 +04:00
Andrey Kolkov 9545bee233 fix(mongodb): correct system-bucket prose, pin strategy rendering, make restore credential swap flow-aware
Several comments and the useSystemBucket field description had drifted from
what the code does.

The strategy header still said the $bucketName gate was "not because it
needs the value" and that the sibling strategies needed the bucket name
while this one only asked for ordering, but the strategy now renders
bucket: {{ $bucketName }} for the injected coordinates, so it needs the
value; the precondition comment claimed the s3-storage entry exists on
backup.enabled=true, which is only the legacy flow. The rendering-test
comment carried the same "needs no bucket coordinates" claim. The
useSystemBucket field description said injection happens "at first BackupJob
time" (it now runs on every BackupJob) and that backups scope to
<namespace>/<release> (the prefix templates the application name, not the
release), and it ran ~700 chars of implementation detail into the CRD that
backup-classes.md already carries; trimmed and corrected, then regenerated
schema/README/types/cozyrds. Same <release> slip fixed in the driver
comment and the docs.

Nothing asserted the new strategy rendering. Added the MongoDB row to the
endpoint-form suite (s3.endpointUrl, scheme preserved both ways) and field
assertions to the rendering suite — bucket, endpointUrl, region,
credentialsSecret, forcePathStyle, and prefix, which must reach the CR as a
literal go-template string so the driver can template it per-application;
a stray brace would empty it with every doc-count test still green.

The restore credential swap broke across flows. It replaced the source
credentialsSecret with whatever the target declared, comparing no buckets,
so a system-bucket backup restored into a legacy target kept the platform
bucket against the tenant's own -s3-creds, and a legacy backup restored
into a system-bucket target got cozy-backups-creds against the tenant
bucket - both AccessDenied. Skip the swap when the source already names the
platform-projected cozy-backups-creds (it outlives the source app, nothing
to repair), and only adopt a target credential whose storage is on the
same bucket as the archive. psmdbStorageS3 now reads .s3.bucket alongside
.s3.credentialsSecret; TestPsmdbTargetCredentialsSecret covers the match,
the cross-bucket refusal, and the empty-bucket no-op.

Also drop the post-apply re-Get: the typed SSA Patch decodes the server's
merged object back into the passed struct, so the precondition reads the
injected storage from it instead of a follow-up Get that hit the informer
cache and briefly reported Ready=False on the happy path.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
2026-09-22 12:03:58 +04:00
Andrey Kolkov 730592f323 feat(mongodb): support the system-bucket backup flow (useSystemBucket)
MongoDB was the only cozy-default app with no useSystemBucket flow: its
backup storage was always static on the app CR (bucket/endpoint/creds
from backup.* values + a chart-emitted <release>-s3-creds Secret), so a
tenant could not back up to the platform system bucket out of the box the
way postgres/clickhouse/etc. do.

PBM takes bucket/endpointUrl/region/prefix as static CR fields (not
secret refs) and the platform bucket name/endpoint are only known at
BackupJob time, so - unlike ClickHouse's runtime env secretKeyRef - the
chart cannot render them. The fix mirrors the CNPG driver: the chart adds
a useSystemBucket flag that keeps spec.backup.enabled (so the pbm agents
run) but omits the static storages/tasks/pitr and the creds Secret; the
cozy-default MongoDB strategy carries the coordinates (endpointUrl already
scheme'd by the endpoint helper, credentialsSecret cozy-backups-creds);
and the psmdb driver SSA-injects spec.backup.storages[s3-storage] onto the
live PerconaServerMongoDB at BackupJob time under its own field manager
with ForceOwnership, so a Flux re-render never reverts it. Injection is
gated on the storage being absent, so a legacy app that ships its own
static storage is left untouched. The backup snapshot falls back to the
strategy coordinates when the operator status echo is empty, so the
restore path rebuilds backupSource (credentialsSecret cozy-backups-creds
is projected into the restore namespace before dispatch).

RBAC grows patch/update on perconaservermongodbs for the injection. A
bootstrap.enabled + useSystemBucket render guard mirrors postgres.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
2026-09-22 12:03:58 +04:00
Andrey KolkovandClaude Opus 4.8 dff56aa998 feat(backups): add a native MongoDB (psmdb) backup-strategy driver
Adds a MongoDB (Percona Server for MongoDB / psmdb) backup-strategy driver so
`apps.cozystack.io/MongoDB` reaches feature parity with the CNPG, MariaDB and
ClickHouse native drivers — logical backups, restore-to-differently-named
instances, and point-in-time recovery through the unified
BackupJob/RestoreJob/BackupClass/Plan interface (closes #3545).

The driver orchestrates the operator's PerconaServerMongoDBBackup /
PerconaServerMongoDBRestore CRDs:

- BackupJob: creates a one-shot PerconaServerMongoDBBackup, polls
  status.state, and persists the S3 destination + storage descriptor
  (by-reference, no credentials) onto the Cozystack Backup artifact.
- RestoreJob: drives PerconaServerMongoDBRestore via backupSource (read back
  from the source backup status, snapshot fallback) so the same artifact
  restores in-place or into a differently-named instance; PITR via
  spec.options.pitr (date/latest).

Because psmdb resolves the backup storage by name against the cluster's
spec.backup.storages, the driver names the chart's storage (s3-storage) via the
cozy-default-mongodb Strategy rather than injecting it, and surfaces a clear
Ready=False precondition when a target has backups disabled.

Wiring:
- api/backups/strategy/v1alpha1: MongoDB strategy type + generated CRD/deepcopy.
- internal/backupcontroller: psmdbstrategy_controller + psmdbtypes/mongodbapp
  partial-schema mirrors; dispatch, supported-kinds and scheme registration.
- backupstrategy-controller: cozy-default MongoDB route, cozy-default-mongodb
  Strategy CR, and psmdb.percona.com RBAC.
- apps/mongodb: backup.insecureSkipTLSVerify knob (psmdb s3 storage has no
  CA-bundle field) so backups can target Cozystack's self-signed seaweedfs.

Tests + e2e:
- unit tests for the driver (validation, ensure-idempotency, precondition,
  backupSource resolution, snapshot round-trip, PITR option parsing).
- examples/backups/mongodb + a mongodb-2-backup-roundtrip Chainsaw suite that
  drives run-all.sh end-to-end (sentinel document round-trips through S3 into a
  restored copy while the source stays untouched), per docs/agents/e2e-testing.md.
- docs/operations/backup-classes.md updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-08-12 12:00:33 +04:00
Aleksei Sviridkin 5af5962edf feat(apps): mark stateful-app storageClass fields as immutable
Annotate storageClass as immutable across every stateful app whose
chart exposes one — clickhouse, foundationdb, harbor, http-cache,
kafka (Kafka + ZooKeeper), kubernetes (top-level only), mariadb,
mongodb, nats, openbao, opensearch, postgres, qdrant, rabbitmq,
redis, vm-disk. Without coverage across the catalogue the UI would
render storageClass read-only on some apps and editable on others —
a worse mental model than uniform behaviour.

The annotation lands at the schema layer (CEL XValidation rule
'self == oldSelf' on values.schema.json / types.go / embedded
openAPISchema in cozyrds/<app>.yaml). The cozystack aggregated
apiserver does not yet evaluate CEL on application-backed resources;
that is tracked at cozystack/cozystack#2657. The UI
(cozystack/cozystack-ui#6) consumes the marker today.

The PVC-pinning argument is identical for every chart: Kubernetes
fixes a PVC's storageClassName at creation time, so editing this
field on an existing resource never migrated data. The schema
annotation makes that contract explicit.

kubernetes.nodeGroups[name].storageClass is intentionally NOT
annotated: the field is optional and undefaulted, so a strict
'self == oldSelf' rule would block any future attempt to set it on
an existing node group. A richer ratcheting form is needed there —
see docs/storage-immutability.md.

A refactor in pkg/registry/apps/application/rest.go extracts the
inline schema-build path into buildSpecSchema so it can be
unit-tested. Three regression tests in rest_spec_schema_test.go pin
the contract that x-kubernetes-validations survives the
v1->internal->structural conversion (loaded from the actual MariaDB
ApplicationDefinition fixture on disk, not a hand-edited mock).

Each per-app README gains a one-line blockquote pointing to a new
docs/storage-immutability.md that explains the contract and which
consumers enforce it today.

Requires cozystack/cozyvalues-gen with @immutable support — bump the
pre-commit pin in .github/workflows/pre-commit.yml once the release
exists.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-26 23:51:11 +03:00
Aleksei SviridkinandMatthieu ROBIN fdffedbd4b feat(apps): instance-type resourcesPreset across apps and extras
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:

  nano    -> t1.nano       micro   -> t1.micro
  small   -> t1.small      medium  -> c1.small
  large   -> c1.medium     xlarge  -> c1.large
  2xlarge -> c1.xlarge

Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.

Three hardcoded legacy literals in chart templates migrated:
  - packages/apps/harbor/templates/harbor.yaml:121 (portal)
  - packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
  - packages/system/keycloak/templates/sts.yaml:115 (heap fallback)

packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.

Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.

Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-11 21:08:41 +03:00
Myasnikov Daniil e0ab4d0639 [docs] Fixed controller-gen markers
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-03-25 15:57:25 +05:00
Myasnikov Daniil 9e55552910 [docs] Updated app go types
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-03-25 15:57:25 +05:00
Myasnikov Daniil 733e87137f [docs] Added deepcopy methods to go types
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-03-25 15:57:25 +05:00
Myasnikov Daniil e090e2f317 [docs] Added go types codegeneration for managed apps
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-03-25 15:57:13 +05:00