mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-13 08:05:19 +09:00
cc51e040ae1677c3e643ad06d50bc6c5e5cce1b8
2285
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Andrey Kolkov
|
cc51e040ae |
fix(mongodb): keep the cleanup escape hatch off legacy operator CRs, and surface a stuck finalizer
The skip-artifact-cleanup annotation reaches releaseMongoDBCleanup before the ownership check, and the operator-CR delete sat outside the finalizer gate, so a legacy backup (no delete-backup finalizer) lost its PerconaServerMongoDBBackup record when a tenant set the annotation to release a stuck Backup or applied it across a namespace during teardown. That record is what a psmdb restore-by-name resolves against, and both the RBAC comment and the docs promise it is never touched. Gate every write in the release on the finalizer being present: an unowned (legacy) CR or one that cannot be read at all is left intact, and the Cozystack Backup still releases. Surface a failed finalizer strip. The release must proceed even against an unreachable CR, but the swallowed patch error left the operator CR holding percona.com/delete-backup — pinning its namespace in Terminating, the outcome the branch exists to prevent — while the Event reported the success case. Emit a FinalizerNotStripped Warning naming the object so an operator can find it. Mark the sibling backup fields (schedule, retentionPolicy, destinationPath, endpointURL, insecureSkipTLSVerify, s3AccessKey, s3SecretKey) as ignored under useSystemBucket, so a tenant editing one instead of the flag itself is not misled by a description that describes only the legacy flow; regenerate the schema, README and mongodb-rd. Pair the insecureSkipTLSVerify rendering assertion with a case that flips the platform knob on, so the two together discriminate the rendered value from a hardcoded default. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
542f42ef67 |
fix(mongodb): make the system-bucket cleanup teardown-safe and single-source its ownership
Four issues in the system-bucket cleanup path. Teardown wedge. The cleanup requeued forever waiting for the operator to prune, and the escape-hatch annotation released the Cozystack Backup without touching the delete-backup finalizer the driver stamps on the operator CR. So a namespace teardown, where the operator's delete-backup step cannot run (it execs against a pbm agent that is going away), left the operator CR — and the namespace — stuck in Terminating with no way out but hand-patching finalizers. Cleanup now detects a terminating namespace (as Redis/Rabbitmq do) and, on that or the annotation, strips its own finalizer and releases, leaving the object for a bucket lifecycle policy. Needs patch on perconaservermongodbbackups for the finalizer strip. Ownership was read from two places written at different times: the finalizer (stamped when the operator CR is created) and a driverMetadata flag (written when the artifact is created, a later reconcile). If the app's useSystemBucket flag flipped between them they disagreed. Read ownership from the finalizer on the one CR named in driverMetadata (the one the driver minted): resolving it by name, never by scan, keeps the operator's own cron-* task-minted backups (which do carry percona.com/delete-backup at v1.22.0) out of scope, so the finalizer on that named CR is an unambiguous single marker. Drops the driverMetadata flag. Injection error was terminal. A transient server-side apply — an apiserver hiccup, or a conflict while the operator writes the same cluster — failed the BackupJob outright. Requeue with backoff instead, bounded by the existing deadline, matching the cluster/app read errors alongside it. Injection ran before the enabled check, mutating a cluster that can never back up. Gate it on backup.enabled so a disabled cluster is left untouched. Also: reframe the restore S3 backfill as the defensive guard it is (the pinned operator always echoes status.s3, so the live path normally carries coordinates), and correct the psmdbtypes doc that still claimed the driver never patches the operator CRs. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
f27a02f1b6 |
docs(mongodb): correct the inlined-storage story and surface the TLS knob
The strategy godoc still said the storage config is NOT inlined and the strategy only names the storage — which controller-gen copies verbatim into the CRD, so kubectl explain served the new s3 field and its own denial from one object. Rewrite it (and the StorageName field doc) to describe both flows, and regenerate the CRD. Render insecureSkipTLSVerify on the default MongoDB strategy: the field is plumbed end to end into the driver-injected storage entry and backed by a platform knob, but the template set only forcePathStyle, pinning it to false so a self-signed external S3 that Redis backups honour would fail MongoDB x509 verification. Pin it in the rendering test alongside the other s3 coordinates. Fix the useSystemBucket field description: angle brackets rendered as empty HTML tags in the README table and the dashboard schema, so wrap the path in backticks (as the postgres field already does) and note that schedule, retentionPolicy and PITR are ignored on this flow. Regenerate the chart schema, README and mongodb-rd. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
1d05dafb2e |
fix(mongodb): own and prune the system-bucket archive, and heal the restore read path
Two correctness gaps on the useSystemBucket flow. Retention. Gating spec.backup.tasks out of the chart also dropped the keep that psmdb task retention used, and an on-demand PerconaServerMongoDBBackup is not covered by task retention, so nothing pruned the shared cozy-backups bucket — a retention-swept Plan grew it unboundedly for every tenant at once. Own the archive the way the Redis/Rabbitmq drivers own theirs: stamp the operator's percona.com/delete-backup finalizer onto the backups minted on this flow, and on Cozystack Backup deletion delete that operator CR and wait for the finalizer to free the object before releasing. A legacy backup (tenant's own bucket) keeps the pre-existing hands-off no-op. Adds the delete verb on perconaservermongodbbackups and a skip-artifact-cleanup escape hatch for a wedged operator, matching the sibling drivers. Also drops the unused update verb from the psmdb cluster RBAC (server-side apply is a patch, not an update). Restore. resolveMongoDBBackupSource returned early off the live operator Backup CR whenever it reported a destination, copying status.s3 as-is. On this flow the operator can echo a destination with status.s3 empty (the storage was injected, not declared), and the target has no declared storage to resolve from either, so the restore stranded with no coordinates. Backfill a missing live S3 block from the snapshot the BackupJob recorded. Also correct the precondition godoc: the pbm agents start on spec.backup.enabled alone; the declared storage is required by the backup CR's resolution, not by the agents. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
9545bee233 |
fix(mongodb): correct system-bucket prose, pin strategy rendering, make restore credential swap flow-aware
Several comments and the useSystemBucket field description had drifted from
what the code does.
The strategy header still said the $bucketName gate was "not because it
needs the value" and that the sibling strategies needed the bucket name
while this one only asked for ordering, but the strategy now renders
bucket: {{ $bucketName }} for the injected coordinates, so it needs the
value; the precondition comment claimed the s3-storage entry exists on
backup.enabled=true, which is only the legacy flow. The rendering-test
comment carried the same "needs no bucket coordinates" claim. The
useSystemBucket field description said injection happens "at first BackupJob
time" (it now runs on every BackupJob) and that backups scope to
<namespace>/<release> (the prefix templates the application name, not the
release), and it ran ~700 chars of implementation detail into the CRD that
backup-classes.md already carries; trimmed and corrected, then regenerated
schema/README/types/cozyrds. Same <release> slip fixed in the driver
comment and the docs.
Nothing asserted the new strategy rendering. Added the MongoDB row to the
endpoint-form suite (s3.endpointUrl, scheme preserved both ways) and field
assertions to the rendering suite — bucket, endpointUrl, region,
credentialsSecret, forcePathStyle, and prefix, which must reach the CR as a
literal go-template string so the driver can template it per-application;
a stray brace would empty it with every doc-count test still green.
The restore credential swap broke across flows. It replaced the source
credentialsSecret with whatever the target declared, comparing no buckets,
so a system-bucket backup restored into a legacy target kept the platform
bucket against the tenant's own -s3-creds, and a legacy backup restored
into a system-bucket target got cozy-backups-creds against the tenant
bucket - both AccessDenied. Skip the swap when the source already names the
platform-projected cozy-backups-creds (it outlives the source app, nothing
to repair), and only adopt a target credential whose storage is on the
same bucket as the archive. psmdbStorageS3 now reads .s3.bucket alongside
.s3.credentialsSecret; TestPsmdbTargetCredentialsSecret covers the match,
the cross-bucket refusal, and the empty-bucket no-op.
Also drop the post-apply re-Get: the typed SSA Patch decodes the server's
merged object back into the passed struct, so the precondition reads the
injected storage from it instead of a follow-up Get that hit the informer
cache and briefly reported Ready=False on the happy path.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
|
||
|
Andrey Kolkov
|
4ea4888685 |
fix(mongodb): key system-bucket injection off the app flag, not live storage
The MongoDB backup driver decided what to inject and what to record from the live cluster's storage set rather than from the app's opt-in flag. That had three consequences. The snapshot fallback fired for legacy apps too. Its gate was `snap.S3 == nil && rendered.S3 != nil`, but the coordinates moved onto the shared cozy-default strategy, so rendered.S3 is now non-nil for every MongoDB app. A legacy backup written to the tenant's own bucket got its snapshot rewritten to the platform bucket, sending restore to the wrong place. Gate the fallback on the app's backup.useSystemBucket flag instead, threaded through createMongoDBBackupArtifact, and pin the legacy case. The injected storage never caught up with the strategy. Injection was gated on the named storage being absent, so it ran once per cluster and froze the coordinates at the first backup; a later endpoint/region change or bucket re-provision was a silent no-op, and Helm could not repair it because the useSystemBucket chart renders no storages. Key injection off the app flag and re-apply on every BackupJob (the prefix is deterministic, so this never splits the archive the way CNPG's serverName would). Add UseSystemBucket to mongodbapp.MongoDBSpec so a legacy app is protected by the flag, not by whatever happens to be on the cluster. The docs said the opposite of what the driver does. backup-classes.md claimed "the MongoDB driver cannot inject its S3 target per-backup"; correct it and add an opt-in-to-the-system-bucket section that names which values stop being read and that scheduled backups + PITR must move to a Plan. Fix the endpoint table row, the tuning reference, the RBAC comment that said the driver never patches, the strategy header/inline comments, and the example comment. Also assert notExists on spec.backup.pitr in the helm test, and lift the duplicated cozy-backups-creds default into a const. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
Andrey Kolkov
|
730592f323 |
feat(mongodb): support the system-bucket backup flow (useSystemBucket)
MongoDB was the only cozy-default app with no useSystemBucket flow: its backup storage was always static on the app CR (bucket/endpoint/creds from backup.* values + a chart-emitted <release>-s3-creds Secret), so a tenant could not back up to the platform system bucket out of the box the way postgres/clickhouse/etc. do. PBM takes bucket/endpointUrl/region/prefix as static CR fields (not secret refs) and the platform bucket name/endpoint are only known at BackupJob time, so - unlike ClickHouse's runtime env secretKeyRef - the chart cannot render them. The fix mirrors the CNPG driver: the chart adds a useSystemBucket flag that keeps spec.backup.enabled (so the pbm agents run) but omits the static storages/tasks/pitr and the creds Secret; the cozy-default MongoDB strategy carries the coordinates (endpointUrl already scheme'd by the endpoint helper, credentialsSecret cozy-backups-creds); and the psmdb driver SSA-injects spec.backup.storages[s3-storage] onto the live PerconaServerMongoDB at BackupJob time under its own field manager with ForceOwnership, so a Flux re-render never reverts it. Injection is gated on the storage being absent, so a legacy app that ships its own static storage is left untouched. The backup snapshot falls back to the strategy coordinates when the operator status echo is empty, so the restore path rebuilds backupSource (credentialsSecret cozy-backups-creds is projected into the restore namespace before dispatch). RBAC grows patch/update on perconaservermongodbs for the injection. A bootstrap.enabled + useSystemBucket render guard mirrors postgres. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-By: LLM |
||
|
IvanHunters
|
aa20495df3 |
feat(marketplace): register a tapped repository's apps on connect
Connecting a marketplace repository materialized its PackageSource but created no Package, so its ApplicationDefinitions never rendered and its apps never appeared in the dashboard catalog until a manual `cozypkg add` the dashboard cannot drive (the package list is computed from the ApplicationDefinitions, which do not exist yet). The tap materializer now creates a tap-managed registration Package for each materialized PackageSource, the way the platform ships a Package per built-in app, so the ApplicationDefinition registrations deploy on connect and the apps become browsable; the app components carry no install block and are still instantiated per user resource. Privileged install components of an unconfirmed tap auto-registration (a Package that still carries the marketplace-tap label) are refused on several best-effort layers: the Package reconciler skips their HelmRelease and prunes a lingering one at the install site, and the PackageSource reconciler withholds their ExternalArtifact so the helm-controller has no privileged content to deploy, re-running on confirmation via a Package watch. `cozypkg add --allow-privileged` sheds the label as an explicit ownership handover, which is the confirmation; a platform Package never carries the label. A benign component flipped to privileged in a new revision can still be materialized by source-watcher before these layers observe the flip; closing that race fully requires decoupling the catalog registration from install and is tracked in issue #4359. `cozypkg untap` and the dashboard disconnect remove the registration Package (de-registering the apps) under a UID + ResourceVersion precondition, so a concurrent `cozypkg add` handover is never clobbered; running instances are left but unmanaged. `cozypkg add` re-opens a tap auto-registration for variant selection and hands it over. The cozystack-api ClusterRole gains delete on packages for the disconnect path. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Alexey Artamonov
|
949096694b |
fix(platform): reject non-string realm names and shell-quote the hook realm
A realmName that YAML resolves to an int or a bool (a bare 123, or a reserved word like on/off) was coerced with toString, silently renaming the realm to something the operator never typed (on -> "true"). Fail the render with a quoting hint at the source instead of surfacing it later as a broken issuer URL. The update-email hook Job interpolated the realm into its shell script unquoted, so a name carrying a space or a shell metacharacter could break or inject into the script; single-quote it. The Grafana and managed-cluster issuer helpers build the URL with printf %s straight off the channel value, so guard them with toString for the same non-string reason. Name the configured realm in the tenant kube-apiserver oidc.mode description and the Grafana identity-mode comment, which still read cozy as if it were fixed. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
80d32b3ab7 |
fix(keycloak): complete realm-name coverage and guard non-string names
The enable-update-email hook Job in keycloak-configure still pinned REALM=cozy, so on a custom realmName it waited on a realm the chart never provisions and the cold install timed out — the one seventh consumer left behind, and the only value nobody could set without breaking the install. A bare-integer or YAML-reserved realm name reached the printf %s that builds the derived keycloakInternalUrl as a Go int, rendering a "%!s(...)" format error into the URL; coerce the name to a string once at the source. Document that renaming the realm on a running cluster provisions a fresh empty realm and orphans the accounts in the old one, and note in the OIDC tenant and grafana guides that the realm name is now the configurable default. info/Makefile carried NAME=etcd, so cozyhr drove the wrong release from that package; correct it while touching the file. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
44a9a8e735 |
feat(platform): make the Keycloak realm name configurable
The realm name "cozy" was hardcoded in keycloak-configure (which owns the realm, its clients and the cozystack-cluster-admin roles), in the dashboard and linstor-gui gatekeepers, in the info kubeconfig, and in the Grafana and managed-Kubernetes apiserver OIDC issuers. Operators could not rename the realm without patching vendored templates, which `make update` reverts. The apiserver OIDC issuer is already configurable via talm oidcIssuerUrl, so pointing it at a custom realm left every SSO component on "cozy" and broke login with an issuer mismatch. authentication.oidc.realmName (default "cozy") now rides the _cluster channel so every consumer resolves the same realm, and the default keycloakInternalUrl is derived from it so the realm is named in one place. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Aleksei Sviridkin
|
8799a4e4aa |
fix(cluster-api): backport live migration before host eviction (#4291)
## What this PR does CAPK v0.1.10 handles host eviction by draining and deleting the guest VMI. It also looks up the physical host name in the guest cluster, and a recreated guest can remain cordoned. Backport CAPK #374, #389 and #392 onto the current provider API and runtime, with separate bounded submission, active migration and fallback windows, and ordinary VMIM objects without a priority, a field the pinned `kubevirt.io/api` v1.5.0 does not carry. The submitted VMIM deadline survives reconciliation; expired, invalid or legacy marker values enter the existing fallback path. Add the manager build to the package pipeline, read its digest pin through the provider imageUrl override, and ship VMIM create permission in the compressed provider manifest. Source tests and four packaging checks pass, including failed-build pin preservation and the rendered ConfigMap payload. The regression tests include unprocessed submissions, legacy and invalid deadline values, repeated reconciles, and a fresh active migration window after submission expiry. The updated manager also cross-compiles for linux/amd64 with Go1.26.4. The previous OCI build predates the submission-deadline correction and does not validate an image containing this change. The updated source still needs the maintainer-owned image build described below. The checked-in pin still identifies the original upstream image, so installing this source chart directly does not enable the backport. The regular main and release builds now include this package, stamp the built image digest, and package that exact artifact; the release workflow also commits the generated pin. Pre-merge E2E validation needs a maintainer-owned build because fork publication only allows image repositories already present in the base tree. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(cluster-api): attempt live migration before drain/delete on External eviction, use the guest node name for fallback drain, and uncordon recreated guests. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a Cluster API Provider KubeVirt image pinned to version 0.1.10. - Added migration-first workload eviction with fallback draining and node recovery. - Added permission to create virtual machine migration resources. - Included the provider image in builds and deployment templates. - **Bug Fixes** - Preserved migration deadlines across retries and improved recovery after interrupted or completed migrations. - Added a bounded wait for migrations before falling back to draining. - **Documentation** - Documented the provider image version, patches, build process, and release considerations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
3219f7ee9a |
fix(backups): give the barman-cloud sidecar its own resources (#4220)
## What this PR does The barman-cloud plugin injects its sidecar with no resource requests or limits. What that means depends on the namespace. A tenant with `resourceQuotas` set ships a `LimitRange` that defaults containers to 128Mi (`packages/apps/tenant/templates/quota.yaml`), so there the sidecar inherits that default and is OOMKilled mid-backup. A namespace without a `LimitRange`, which is `cozy-keycloak` and any tenant that leaves `resourceQuotas` empty, gave the sidecar no requests and no limit at all, so nothing reserved memory for it and nothing bounded it. The failure on the tenant path is quiet from the control plane: the `ObjectStore` stays healthy, the `Cluster` stays `Ready`, and only the backup fails. Observed on a 1.6 cluster while backing up a 38 MB database: `lastState.terminated.reason: OOMKilled`, `exit 137`, four restarts on the sidecar, and a cgroup high-water mark of 254 MiB. `container_memory_working_set_bytes` peaked at only 64 MiB over the same window — it is sampled, and it misses the spike that actually triggers the kill, which is why the working-set number does not explain the failure on its own. Both paths that build an ObjectStore now carry resources: - `cozy-lib.barman.sidecarConfiguration`, used by `packages/apps/postgres` for the backup and recovery ObjectStores and by `packages/system/keycloak`; - `barmanSidecarConfiguration()` in `internal/backupcontroller/cnpgstrategy_controller.go`, which builds the platform's own ObjectStore on the `useSystemBucket=true` path. Requests are 100m/256Mi with a 1Gi memory limit on every caller: 256Mi holds the measured working set and 1Gi is four times the measurement, which is also the ceiling a caller without a LimitRange gets where it had none. No CPU limit is set, following the `entityOperator` precedent in `packages/apps/kafka/templates/kafka.yaml`: a throttled sidecar stalls WAL archiving instead of failing it, which is harder to notice than an outright failure. Measured CPU peak was 50 mCPU. Inside a tenant the limit is charged against the `ResourceQuota` `limits.memory` budget, 1Gi per instance pod; a 4Gi tenant spends a quarter of it per Postgres instance. The helper is renamed from `checksumSidecarConfiguration` to `sidecarConfiguration`, because it no longer carries only the checksum pin and its name and doc comment would otherwise be wrong. Tests cover both paths and both render sites: a new helm suite in `packages/apps/postgres`, extra assertions in the existing keycloak suite, an assertion in the controller test, and a deepcopy test for the new field. Each of them fails against the unfixed sources. One thing this PR deliberately does not do. This is the fifth time the 128Mi tenant default has been worked around per component — kafka and zookeeper presets (#2537), the kafka entity-operator (#2934), the cert-manager cainjector (#3199), and the note carried in the etcd-operator values. Whether the default itself should move looks like your call rather than something to fold into a fix for one sidecar, so it is left alone here. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff, entry by entry. The change touches one cozy-lib helper, the two chart templates that include it, one Go function, and tests. It adds, renames or removes no package; changes no `values.yaml`, `values.schema.json`, `Chart.yaml` or `README.md`, so no version enum, default or generated artifact moves; changes no `ApplicationDefinition` semantics, no `release.prefix`, no output Secret or Service name; touches no CRD, no namespace, no `hack/` file, no telemetry metric or label, no `cozy-proxy` annotation, no node or network requirement, and no commit or PR convention. No entry matches. - [x] No downstream repository is affected by this change ### Release note ```release-note fix(backups): give the barman-cloud sidecar its own resources The barman-cloud sidecar was injected without resource requests or limits, so in tenant namespaces it inherited the 128Mi `LimitRange` default and was OOMKilled during backups while the `ObjectStore` and the `Cluster` both stayed healthy. It now requests 100m/256Mi and caps memory at 1Gi on the chart path and on the platform's Go path alike; in namespaces without a LimitRange the sidecar had no limit before and gets the same 1Gi ceiling. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added explicit CPU and memory resource settings for barman-cloud backup and recovery sidecars. - Backup sidecars request 100m CPU and 256Mi memory, with a 1Gi memory limit. - Recovery sidecars receive a 1Gi memory limit. - Preserved the S3 checksum configuration. - **Bug Fixes** - Prevented sidecars from inheriting unsuitable tenant resource limits. - Ensured resource settings are preserved when backup configuration is copied. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d71e78a74a |
fix(linstor): wait for temporary probe devices (#4100)
## What this PR does Backports [LINBIT/linstor-server#528](https://github.com/LINBIT/linstor-server/pull/528) to the LINSTOR 1.33.3 build used by piraeus-server. A ZFS probe volume can be created before udev publishes its device path, leaving the pool without I/O properties and blocking replica placement on 4K pools. Wait for the probe device before reading its properties, retry up to three times and report failures. Skip null paths in the existing volume list and fall back to a probe when ZFS has no usable path. Serialize probe creation and cleanup, stop after cleanup failures and preserve interrupts. Inactive LVM volumes retain the existing no-probe behavior. Validation: all six patches apply together to v1.33.3. All 1026 tests and `assemble` pass on Linux amd64 with JDK 11, including 17 regression tests; 14 of those fail without this patch. `checkstyleMain` reports the same existing return-inside-for error at `DrbdAdm.java:970` from `retry-secondary-after-mkfs.diff`, reproduced without this patch; no new findings. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(linstor): wait for ZFS probe devices before reading pool I/O properties to avoid incorrect block sizes and failed replica placement ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved storage device probing by waiting for temporary devices to become available and retrying when necessary. - Improved compatibility with ZFS volumes when standard device paths are unavailable. - Prevented unnecessary probing of inactive shared LVM volumes. - Improved handling of interrupted probes and cleanup failures. - **Documentation** - Added documentation describing storage probing behavior and compatibility updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Yan Bondarenko
|
4dc6065ec9 |
fix(backups): give the sidecar sizing its reason per caller
The Keycloak comment and the helper doc claimed the tenant LimitRange as the reason for the resources, but cozy-keycloak has no LimitRange and neither does a tenant that leaves resourceQuotas empty: there the sidecar had no requests and no limit at all, and the change gives it a 1Gi ceiling where there was none. The comments and both doc blocks now say what applies to each caller. No rendered object changes. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
9214430bd8 |
fix(backups): give the barman-cloud sidecar its own resources
The barman-cloud plugin injects its sidecar without resource requests or limits. Tenant namespaces ship a LimitRange that defaults containers to 128Mi, so the sidecar inherits that default and is OOMKilled mid-backup: the ObjectStore stays healthy, the Cluster stays Ready, and only the backup fails. A cgroup high-water mark of 254 MiB on a 38 MB database shows the default cannot hold it. The shared cozy-lib helper and the Go path that builds the platform's own ObjectStore now request 100m/256Mi and cap memory at 1Gi. No CPU limit is set, matching the entityOperator precedent in the kafka chart: a throttled sidecar stalls WAL archiving instead of failing it. The helper is renamed from checksumSidecarConfiguration to sidecarConfiguration, since it no longer carries only the checksum pin. Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
bc2ba106e1 |
docs(cluster-api): explain the omitted migration priority by the pinned API
CAPK v0.1.10 pins kubevirt.io/api v1.5.0, whose VirtualMachineInstanceMigrationSpec carries only vmiName; upstream #392 sets priority against v1.9.0. The typed migration does not compile against the pinned API, which is why the patch builds it unstructured. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
c85f35070c |
fix(cluster-api): bound pending migration submission
A migration that KubeVirt never processes must not block host eviction indefinitely. Keep its submission deadline separate from active migration and fallback drain windows. Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
bb3a2fa220 |
docs(cluster-api): clarify trusted backport image delivery
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
c8f4906e93 |
fix(cluster-api): backport live migration before host eviction
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
8ec76a3614 |
fix(kafka): make topics[].config optional
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
cee9065589 |
fix(linstor): wait for temporary probe devices
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Aleksei Sviridkin
|
88a7ea0e3c |
fix(kubeovn): retry VM migration setup after scheduling (#4253)
## What this PR does Kube-OVN can drop VM migration setup when a migration enters `Scheduling` before the target Pod is visible or the source or target node is known. The phase may remain unchanged afterward, leaving the target Pod waiting for network setup. Backport the final fix from kubeovn/kube-ovn#6834 to the pinned v1.15.10 image: return an error for the transient missing-node case so the existing rate-limited queue retries it. Keep the same-node behavior unchanged. That backport can be removed when updating to v1.15.14 or later. Add a separate retry guard for the target-Pod creation race. It returns an error when the informer cache has no target Pod yet, instead of forgetting the migration event. This guard is not part of #6834 and must remain until an equivalent fix is verified in the selected upstream version. Also include the existing kubeovn image target in the root build list. Without it, a kubeovn patch produces an empty PR build matrix and the image is omitted from the regular build and release paths. Validation: - All six regression cases pass on Linux/amd64 using the real controller workqueue. The new missing-target-Pod case fails without its guard and passes after the Pod appears in the informer cache, without changing the VMIM phase. The existing deferred-node, already-scheduled, and same-node cases continue to pass. - All 19 build-matrix tests pass; the full matrix includes kubeovn exactly once. - Both lint steps, pinned golangci-lint v2.10.1 and upstream `modernize -test -skipgenerated ./...`, pass for the Linux/amd64 target using Go 1.26.4. On macOS, modernize is built for the host and run with the Linux target environment, since `go tool` otherwise tries to execute a cross-compiled Linux binary. - All three packaged patches apply cleanly to pinned v1.15.10 with `git apply --check --whitespace=error`; the resulting source matches the tested tree. Formatting and source/build whitespace checks pass. The regression test covers controller queue behavior; the standard container E2E environment does not cover DRBD-backed live migration. The regular image build applies the backport and stamps the resulting digest before packaging. ### Screenshots Not applicable; no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The change adds an existing image target to the build list; package layout, make target interfaces, chart values, APIs, and published image-reference formats stay the same. CCP discovers package image targets dynamically and does not carry the root build list. The website has no documented workaround for kubeovn/kube-ovn#6823 or #6834 to remove. ### Release note ```release-note fix(kubeovn): retry VM migration network setup when the target Pod or source/target scheduling information is not yet available, and include the Kube-OVN image in the regular build matrix. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved VM migration handling when source or destination node scheduling information is temporarily unavailable. - Migration events are now retried after scheduling completes instead of being dropped. - Preserved existing behavior when migration source and destination nodes are identical. - **Tests** - Added coverage for migration scheduling and retry scenarios, including partially assigned and same-node cases. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timofei Larkin
|
8f19910988 |
chore(platform)!: remove the HTTPCache application (#4200)
## What this PR does Removes the HTTPCache application. Telemetry says it is the least installed app we ship: 3 instances in June, July and August 2026, 4 so far in September, out of 1162-1418 clusters reporting per month. That is not zero, so I am not going to claim nobody uses it. The maintenance side is what does not hold up. `make update` in this package is broken. All eight sed lines in the `update` target patch `images/nginx/Dockerfile`, but the image lives in `images/nginx-cache/Dockerfile`, so the target fails on the first line. It is the only package under `packages/apps/` pointing at a Dockerfile that does not exist. The pins show what that cost us: ``` nginx 1.25.3 -> 1.31.5 ngx_cache_purge 2.5.3 -> 3.0.2 nginx-module-vts 0.2.2 -> 0.2.7 ip2proxy-c 4.1.2 -> 4.3.0 IP2Location-C 8.6.1 -> 8.7.0 ip2location-nginx 8.6.0 -> 8.7.0 ip2proxy-nginx 8.1.1 -> 8.2.0 51Degrees 3.2.21.1 -> 3.2.21.1 (frozen upstream) ``` So we ship an old nginx with third party C modules compiled into it, and nothing tests it: no chainsaw suite, no `helm unittest` target. I don't think we should keep carrying that for 4 instances. Removed: the chart in `packages/apps/http-cache`, its ResourceDefinition in `packages/system/http-cache-rd`, the values type in `api/apps/v1alpha1/httpcache`, the `cozystack.http-cache-application` PackageSource, its line in the `naas` bundle, the `httpcaches` entry in `cozy:tenant:admin:base`, the dashboard icon, and the entry in the root `Makefile` build list. `cozy:tenant:admin:base` enumerates the `apps.cozystack.io` resources a tenant owner may write, so `httpcaches` comes out of it with the rest of the application. A cluster that freezes gets the grant back from the migration instead, which keeps the chart's list tracking the catalog and leaves no entry behind for a later sweep to read as dead config. ### Existing installations A cluster that still runs an HTTPCache keeps it, served from a frozen source. A cluster that never created one keeps nothing. Without a migration an upgrade would break both. `templates/sources.yaml` is a bare glob with no keep annotation, so Helm prunes the http-cache PackageSource; the ArtifactGenerator is owned by it and goes too, and both ExternalArtifacts go with the generator. Those are the `chartRef` targets of `hr/http-cache-rd` and of every tenant `http-cache-<name>` release, so all of them would sit `Ready=False` for good and `HelmReleaseNotReady` would fire on every cluster with the naas bundle. Migration 58 runs as a pre-upgrade hook, before that prune, and branches on whether any instance exists. Its header comment carries the mechanism in full; this is the summary. If one does, it copies the packages `OCIRepository` and pins the copy to the manifest digest that object resolves at hook time, which is the release the instances are rendering from right then. `spec.ref.digest` is used when the installer pinned one; a tag or semver ref goes through `status.artifact.revision`, whose digest half is the manifest digest. `status.artifact.digest` is the checksum of the downloaded tarball and cannot stand in, and a source with neither aborts the migration rather than publishing a source that points nowhere. Reading the digest live rather than baking one into the script makes the freeze a no-op by construction and asks no question about which release was the last to ship the chart. The copy takes the url, the pull secret, the interval and any verification block from the live object, so an air-gapped or mirrored registry keeps serving exactly what it already served. It is written with no owner references and no Helm metadata, and with the `platform.cozystack.io/no-delete` label the platform chart puts on the original, because it becomes the only chart source those instances have. The PackageSource is then repointed at the copy, annotated `helm.sh/resource-policy: keep` and stripped of its release metadata, so neither Helm nor a later release adopts or prunes it. Helm re-reads that annotation off the live object when it prunes, which is what makes disowning from inside a pre-upgrade hook work. Names stay as they are, deliberately. The ApplicationDefinition inside that artifact hardcodes its `chartRef`, so a renamed PackageSource would leave every release pointing at an artifact nobody produces. Because the names hold, `hr/http-cache-rd` and every tenant release keep the `chartRef` they already have and nothing goes not ready. What changes is where the chart comes from: a pinned digest that no later release moves. The migration also applies `cozy:tenant:admin:http-cache-frozen`, a ClusterRole labelled `rbac.cozystack.io/aggregate-to-tenant-admin`, granting the verbs `cozy:tenant:admin:base` used to carry on `httpcaches`. The hook applies it before the upgrade drops the rule from the chart, so an owner's access never lapses and nothing changes for them. Without it a frozen instance would be visible and undeletable by the person who owns it: the wildcard over `apps.cozystack.io` lives in `cozy:tenant:base`, which aggregates into `cozy:tenant` and not into `cozy:tenant:admin`, `cozy:tenant:view:base` is read only, and no role in the tenant chain carries a verb on `helm.toolkit.fluxcd.io`. If no instance exists, the migration removes the platform side instead: the Package first and waited on, because the operator reconciles off an informer cache and re-creates the release it owns while that Package is still there, then the release, its ApplicationDefinition and its Helm storage. Nothing is pinned or granted on a cluster that never used the application. Every path is fail-closed. A failed fleet scan, a failed read of the source OCIRepository, a source with no digest to freeze on, a failed patch or a failed delete stops the migration before it stamps the version. The migration is numbered 58, because main already carries a 57 from the SeaweedFS volume-size-limit change and its `targetVersion` is already 58. #4287 wants 58 as well, so whichever of the two lands second renumbers and bumps `targetVersion` with it. main is merged in rather than rebased, because the branch carries commits that are not mine to rewrite; the one conflict was `migrations.targetVersion`, resolved as 59, one past this branch's migration number. ### The procedure, written down `docs/deprecating-applications.md` is new, and is the point of doing this one carefully. It covers how to choose between migrating instances onto a successor and freezing them, the chain that makes a bare chart deletion strand every surviving instance, how the freeze reads its digest off the cluster's own source, what the migration owes a cluster on each branch, and where the tenant RBAC grant goes. `AGENTS.md` points at it so the next retirement starts there rather than rediscovering it. ### If you run HTTPCache Nothing changes on upgrade. Your instance keeps running and stays manageable through the Cozystack API and the dashboard, and `kubectl get httpcaches` keeps working. What changes is that it is frozen. The chart now comes from the packages artifact your cluster was already running, pinned by digest, and gets no further updates from us, including security updates to the nginx build. There is no replacement in the catalog. Delete the HTTPCache object when you no longer need it, the way you always did; the migration keeps the tenant permission that lets you. ### Verification `helm template` on `packages/core/platform` before and after, for `isp-full`, `isp-full-generic` and `isp-hosted`. In all three the differences are the `http-cache` Package and PackageSource going away and `migrations.targetVersion` moving from 57 to 58, which on a cluster below 58 also renders the migration hook Job and its RBAC. The migration suite `hack/migration-58-http-cache-freeze.bats` covers both branches, the order of every step and each fail-closed path. Freezing on the tarball checksum instead of the manifest digest, dropping the `no-delete` label, dropping the `managed-by` null, dropping the tenant grant, returning from the Package delete without waiting, freezing unconditionally and granting on a cluster with no instance each turn the matching test red. `packages/system/cozystack-basics/tests/clusterroles-tenant-admin-aggregation_test.yaml` pins the aggregation label the migration's ClusterRole reaches tenant admins through, so renaming it fails in the chart rather than silently on the clusters that need the grant. Green: `helm unittest` for `packages/core/platform` and `packages/system/cozystack-basics`, `make migrations-target-check`, `hack/cozystack-version-stamp.bats`, `go test ./internal/controller/cacert/...`, `go build ./...` in the `api/apps/v1alpha1` module, the console `ServicesTab` and `sidebar-icons` suites with `tsc --noEmit`, `make rd-presets-check`, and the bats suites that enumerate packages and image refs. ### Screenshots The catalog entry and its nginx sidebar icon are gone. Nothing new to show. ### Downstream repositories Two repos are affected. No follow-up is open yet, so no box is ticked. `cozystack/website` carries `content/en/docs/*/networking/http-cache.md` in eight doc versions and lists `http-cache` in the hardcoded `NETWORKING` list in its Makefile. The `next` page and the Makefile entry are what need to go, released versions stay as history. `cozystack/terraform-provider-cozystack` has a `cozystack_httpcache` resource and data source. The resource keeps working after this: it is hand written against its own terraform-plugin-framework model and imports nothing from here. What breaks, once that repo bumps its `api/apps/v1alpha1` pin past this release, is `internal/provider/httpcache_model_test.go`, the per kind schema guard, which is the only file over there importing this module. The resource and the guard should go together. The other repos have no reference to the package. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note chore(http-cache)!: remove the HTTPCache application from the catalog. New instances can no longer be created from the catalog and the `HTTPCache` values type is gone from the `api/apps/v1alpha1` Go module. A platform migration keeps existing instances working: on a cluster that runs one, the application is frozen on the packages artifact that cluster was already rendering from, pinned by digest, and stays manageable through the Cozystack API and the dashboard, but receives no further updates, including security updates to the nginx build. On a cluster with no instance the application is removed outright. There is no replacement in the catalog; delete the HTTPCache object when you no longer need it. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Removed** * Removed the HTTP Cache application and its deployment, configuration, dashboard metadata, and packaging from the platform. * The HTTP Cache application is no longer included in the NaaS bundle. * HTTP Cache resources and related API definitions are no longer available. * **Migration** * Existing HTTP Cache installations are frozen during migration; environments without instances are cleaned up. * Platform migration target advanced to version 58. * **Documentation** * Updated storage immutability documentation to reflect Valkey instead of HTTP Cache. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
000e2e3121 |
build(opensearch-operator): run the patched tree's tests in the image build
The patches under images/opensearch-operator/patches carry their own tests and
nothing runs them. The build compiles main.go, and go-unit-tests in the root
Makefile covers this repository's own Go tree, so a test file that had stopped
compiling still produced an image:
printf 'func Broken() { this is not go }' >> restart_gate_test.go
go build -a -o manager main.go # exit 0
Run go test over the two package trees the patches touch before building the
binary. Both run without a cluster, and the stage takes about eight minutes on
an arm64 builder, most of it the module download the build needs anyway.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
|
||
|
IvanHunters
|
28195d3ef8 |
fix(opensearch-operator): stop excluding a node whose drain cannot finish
The yellow restart gate carried in the previous patch lets a permanently yellow cluster through, and with three or more nodes packages/apps/opensearch turns drainDataNodes on. The restart then reaches PreparePodForDelete, which excludes the candidate from allocation before it can know whether the drain will finish. On the cluster that made the gate wait it never finishes: a shard on that node has nowhere else to go, which is the same condition holding the cluster yellow. The exclusion is withdrawn only after a pod is really deleted, so a data node stays out of allocation for as long as the restart is stuck, shedding its shards onto the rest of the cluster and buying nothing. Ask the allocator before excluding anything, and only while nothing is moving, so a drain that is making progress costs nothing. A started shard answering can_move_to_other_node: no means the drain cannot finish, and then no exclusion is written and one left by an earlier pass is withdrawn. The two-data-node path keeps its behaviour, because it never waits for a full drain. Checking after the fact instead would flap: the next reconcile, ten seconds later, would exclude the node again. Upstream as opensearch-project/opensearch-k8s-operator#1585, fixed there by #1586; the patch goes away with the first release carrying it. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Timofei Larkin
|
c398e0e116 |
Merge branch 'main' into chore/drop-http-cache
main took migration 57 and moved migrations.targetVersion to 58 while this branch was open, so that line is the one conflict: 57 at the merge base, 58 on main, 59 here. Resolved as 59, which is what this branch's migration 58 requires, since run-migrations.sh loops up to targetVersion - 1. Merged rather than rebased because the branch carries commits that are not mine to rewrite. Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Timofei Larkin
|
2cbc02bd10 |
fix(platform): freeze HTTPCache on the artifact the cluster runs
Pinning the frozen source to a digest written into the migration picks one release for every cluster, and v1.6.0 is not even the last one that ships http-cache: v1.6.1 through v1.6.3 and v1.7.0-alpha.1 all do, and 1.6 patch releases may keep doing so. A cluster on any of those would be rolled back to an older chart by the upgrade that was meant to leave its instances alone, and an air-gapped mirror resolves the constant only if someone mirrored that exact manifest. The hook runs before the upgrade re-points cozystack-packages, so the live object still names the release the instances render from, in whatever registry the operator serves it from and with whatever pull secret and verification they configured. Reading the digest off it makes the freeze a no-op by construction and needs no knowledge of which tag was last. spec.ref.digest is used when the installer pinned one; a tag or semver ref goes through status.artifact.revision, whose digest half is the manifest digest. status.artifact.digest is the checksum of the downloaded tarball and cannot stand in. A source with neither has nothing to freeze on and the migration stops there. The copy also carries the no-delete label the platform chart puts on the original, because it is the only chart source those instances have left. The tenant grant travels with the freeze as well. cozy:tenant:admin:base enumerates the apps.cozystack.io resources a tenant owner may write, and an application that has left the catalog has no business in that list: the next sweep over it takes the entry as dead config, and the failure is silent and remote, a Forbidden for the one person who still needs to remove what they have. The chart drops httpcaches, and a cluster that freezes gets a ClusterRole under the tenant admin aggregation label instead, applied before the upgrade drops the rule so the access never lapses and carrying the verb set the chart had so nothing changes for the owner. A cluster with no instance is granted nothing. The Package delete on the no-instance path has to block. The operator reconciles off an informer cache, so a reconcile in flight against the pre-delete entry re-creates the release it owns, and with it the ApplicationDefinition and the Helm storage the following lines remove. Returning immediately made the ordering a property of the command log rather than of the cluster. The preset check's SIGPIPE note named http-cache-rd as its example of a large enum, and that chart goes away here; kubernetes-rd carries the same enums. Numbered 58 rather than 57: main already carries a migration 57, the SeaweedFS volume-size-limit one, and its targetVersion is already 58. The whole shape of this retirement was worked out here rather than anywhere reusable, so docs/deprecating-applications.md writes it down as a procedure: how to pick between migrating onto a successor and freezing, what the prune takes with it, and what the migration owes a cluster on each branch. The next application to go should not have to rediscover it. Assisted-by: LLM Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Aleksei Sviridkin
|
a04887db3f |
fix(linstor): add opt-in graceful satellite shutdown on Talos (#4292)
## What this PR does Cozystack's Talos configuration removes the systemd DRBD shutdown guard without a replacement. Add `talos.gracefulShutdown.enabled`, disabled by default, to release unused Secondary DRBD resources before the satellite stops so their backing devices, including ZFS zvols, can be released during normal shutdown. The hook skips Primary resources and open devices, rechecks each candidate, and aborts further actions on ambiguous status, command failure or timeout. It uses no force, demotion, Kubernetes API calls or additional permissions. The hook is available only when both Talos and DRBD are enabled. Keep the value in `spec.components.linstor.values.talos.gracefulShutdown.enabled` on the `cozystack.linstor` package. The chart embeds the script in its existing `cozystack-talos` satellite configuration, which Piraeus merges into the generated DaemonSets. No custom satellite image is required. Default renders remain unchanged for Talos, generic Linux and DRBD-less Talos. This is opt-in because preStop also runs on ordinary satellite restarts. Unused Secondary replicas disconnect until their replacement satellite reattaches them, and Piraeus has one DaemonSet per node without a cluster-wide rollout barrier. Simultaneous restarts can therefore pause I/O. Enable this only with sequential satellite/node maintenance and replication checks between nodes. The README covers the 60-second hook budget, pod versus node shutdown grace periods, rollback behavior and the unsupported forced-reboot/broken-kubelet paths. Related upstream discussion: https://github.com/piraeusdatastore/piraeus-operator/issues/860. The reported outage from unconditional forced demotion in https://github.com/piraeusdatastore/piraeus-operator/issues/860#issuecomment-5635881307 is why this hook never demotes a Primary, even when its devices appear closed. Validation: 22 Helm tests and 20 behavioral Python tests pass. The same 20 tests pass inside the existing Cozystack satellite image. Two of them pin the rendered command form: `python3 -c <script> --execute` reaches `main()` with `["--execute"]` because CPython sets `sys.argv[0]` to `-c`, and a script-name placeholder before `--execute` is rejected. Failure-path tests cover nonzero status/down exits, bounded stderr, partial timeout diagnostics, and stopping before the next resource. Full default renders match the base commit for all three supported substrate variants. The rendered script survives strategic merging against Piraeus 2.10.2 and 2.11.0 satellite manifests; the base chart fails the new feature's regression tests. Runtime validation on Talos 1.13.6 used the script from commit `b8c86f2e70d029166939ff29eab8a45ac191e34b` through per-node satellite configurations: a normal node reboot with drain completed without force or an external reset, and the node returned with its ZFS pool online and DRBD replicas UpToDate. That version is installed on all three nodes of the tested cluster after sequential satellite rollouts and replication checks. The later diagnostic-only update in `527570d8079cba7c91b3c345050cab1b73a290a8` is covered by the local and exact-image tests above but has not been rolled out to that cluster. This does not validate forced reboot, a failed kubelet, or simultaneous satellite restarts. GitHub Actions for this fork require maintainer approval; local checks do not replace that CI. ### Screenshots Not applicable. ### Downstream repositories The diff adds an opt-in system-package value and its implementation/tests. It does not change app schemas, platform variants, node prerequisites, shared tooling contracts or an existing documented workaround. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(linstor): add an opt-in Talos satellite preStop hook that releases unused Secondary DRBD resources without force. Enable it with talos.gracefulShutdown.enabled for sequential node maintenance; existing installations keep their current behavior by default. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional graceful shutdown support for LINSTOR satellites running on Talos. * When enabled, unused Secondary DRBD resources are released before termination, with safeguards for active resources and bounded shutdown timing. * Added a 90-second termination grace period and configuration validation for the new setting. * Added documentation covering configuration, operational requirements, and shutdown behavior. * **Tests** * Added Helm and Python coverage for shutdown behavior, validation, timeouts, and failure scenarios. * The test target now runs both Helm and Python unit tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a4bdddf030 |
feat(kubevirt): expose migration configuration through platform values (#4254)
## What this PR does
Cluster-wide migration settings could only be applied by patching the
KubeVirt CR: the kubevirt chart did not render `migrations`, and the
platform bundle did not forward it. Add the optional
`kubevirt.migrations` platform value and pass it through the generated
Package to `spec.configuration.migrations`.
For example, set the following under `spec.components.platform.values`
on the `cozystack.cozystack-platform` Package:
```yaml
kubevirt:
migrations:
bandwidthPerMigration: 625M
parallelMigrationsPerCluster: 2
parallelOutboundMigrationsPerNode: 1
```
`bandwidthPerMigration` is a quantity in bytes per second. Empty or null
settings leave the field unset; existing KubeVirt defaults are
unchanged. The map is forwarded intact, including explicit `false` and
`0` values. `make update` restores the new parameterization
independently after a hand-merge.
Validation:
- All 207 Helm tests in `packages/system/kubevirt` and
`packages/core/platform` passed (42 suites).
- Before the template changes, the new limit and false/zero cases failed
at both forwarding stages (four expected regression failures).
- The GNU sed / make update-idempotency suite passed, including
restoration of a missing migrations block, repeated runs, and rejection
of a missing insertion anchor.
- Standalone KubeVirt rendering and `helm lint` passed. Neither modified
chart defines a `make generate` target. Full platform rendering requires
its live source repository lookup; the platform template behavior was
checked through its existing Helm unit-test fixtures.
This is separate from #3989, which exposes `vmStateStorageClass` in the
same chart and bundle files.
### Screenshots
Not applicable; no UI changes.
### Downstream repositories
- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: [#697](https://github.com/cozystack/website/pull/697)
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
The website follow-up
[#697](https://github.com/cozystack/website/pull/697) documents this new
platform value in the unreleased `next` reference and should be merged
after this implementation. The merged website PR
[#678](https://github.com/cozystack/website/pull/678) documents
`kubevirt.disabledFeatureGates` only. The other listed repositories do
not need a coordinated change: this adds an optional platform value
without changing app schemas, required installer values, CRDs, package
layout, source kinds, shared tooling contracts, or node prerequisites.
### Release note
```release-note
feat(kubevirt): Configure cluster-wide KubeVirt migration settings through the platform Package value kubevirt.migrations. Empty settings preserve existing defaults.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added optional KubeVirt migration configuration.
* Supports migration bandwidth, concurrency limits, auto-convergence,
and completion timeout settings.
* Migration bandwidth can be specified in bytes per second.
* Empty or unset migration settings preserve KubeVirt defaults.
* **Bug Fixes**
* Migration values now preserve valid `false` and `0` settings when
provided.
* **Tests**
* Added coverage for default, empty, null, configured, and
repeated-update scenarios.
* Added validation to prevent incomplete or duplicated migration
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
325a4e3cee |
fix(opensearch-operator): let a permanently yellow cluster restart (#4317)
## What this PR does A single-replica OpenSearch release stops rolling once any index on it asks for a replica. The copy has nowhere to go on one data node, so the cluster stays yellow for good. The operator's restart gate at v2.8.0 opens on green, or on yellow only when `.opensearch-observability` is the yellow index, and at OpenSearch 2.11.1 that index is `auto_expand_replicas: 0-2`, green on one node. So the gate never opens. Every release gets such an index by itself. The operator sets `plugins.security.audit.type: internal_opensearch`, and the first audited event creates a daily `security-auditlog-*` index with one replica. After that nothing that needs a restart is rolled out, a version upgrade or the move of the HTTP listener onto the cert-manager chain included. This is what timed out the `opensearch` e2e suite on some runs. Both failed runs I read show the same thing in the OpenSearch log: the bootstrap node leaves, `Connection reset` on the transport layer, and `[security-auditlog-<date>] creating index ... shards [1]/[1]` within 40 ms. The security plugin audits that exception (`SecuritySSLNettyTransport` hands it to `AuditLogSslExceptionHandler`), so the reset is what creates the index. Then the operator logs `waiting for health to be green` every ten seconds until the assert gives up. Not sure why the green runs had no audit index before the roll, a green run keeps no snapshot. The fix carries opensearch-project/opensearch-k8s-operator#1569 back to v2.8.0 as a second patch on the in-tree operator image, with the upstream tests. The new gate lets a yellow cluster through once allocation is enabled, every data node has joined and no shard is moving, and `PreparePodForDelete` refuses to restart a node that holds the only active copy of a shard meant to have others. A lone data node is exempt. I left out the third commit of that PR (tiered data roles in `HasDataRole`), the chart renders only the `data` role. A bump would not help: the only operator build after v2.8.0 is `3.0.0-alpha` from January, and its gate holds this case the same way. The patch goes away on the first release that includes #1569. The suite now creates an index with an unassignable replica before it moves the listener, so the roll always runs against a yellow cluster. The catch block also reads cluster health and the shard table from the node, because the operator logs that health held a restart back and never what the health was. The image digest in `values.yaml` stays as it is. Closes #4259 ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff: an e2e suite, the operator image's Dockerfile header and patch directory, one comment in the operator package's `values.yaml`. No values key, schema, make target or documented workaround changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(opensearch-operator): a single-replica OpenSearch release no longer gets stuck ahead of every rolling restart once an index with a replica, such as the security audit log or any index created with default settings, leaves the cluster permanently yellow ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved rolling restart handling for clusters that remain yellow, allowing restarts to proceed safely once shard activity has settled. - Prevented restart operations from becoming blocked by temporary shard-allocation settings. - Added safeguards to ensure data is not disrupted while replicas are relocating or initializing. - **Diagnostics** - Added automatic cluster health and shard diagnostics when restart-related test operations time out or encounter issues. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Yan Bondarenko
|
6e90447091 |
fix(kubeovn): retry migration setup until target pod exists
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
55a2918fcb |
fix(kubeovn): retry VM migration setup after scheduling
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
dcad74d8c0 |
test(linstor): pin the chart command form of the shutdown hook
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
6be4acad28 |
fix(linstor): retain shutdown command failure diagnostics
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
ebacd2ea72 |
fix(linstor): add opt-in graceful satellite shutdown on Talos
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
f7f4eb6656 |
fix(kubevirt): reject incomplete migrations template blocks
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Yan Bondarenko
|
5ed076cff8 |
feat(kubevirt): expose migration configuration through platform values
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Aleksei Sviridkin
|
d9fbeab33c |
fix(opensearch-operator): let a permanently yellow cluster restart
The operator restarts a node only when its health gate opens, and the gate wanted green, or yellow caused by the observability index alone. A cluster whose replicas can never be assigned reaches neither: on a single data node any index with a replica is yellow for good, and the operator itself turns on the audit log that creates one on the first audited event. From then on nothing that needs a restart was ever rolled out, a version upgrade and the move of the HTTP listener onto the cert-manager chain included. Upstream rewrote the gate to let a yellow cluster through once allocation is enabled, every data node has joined and no shard is moving, and to refuse instead to restart a node that holds the only active copy of a shard meant to have others. No released operator carries that change, so it is carried here as a patch against the pinned version, together with the upstream tests for it. Signed-off-by: Aleksei Sviridkin <f@lex.la> Assisted-by: LLM |
||
|
Yan Bondarenko
|
8c649df7de |
fix(linstor): reconnect only the stalled peer in the satellite plunger
drbd_stall_act disconnects one peer with "drbdadm disconnect res:peer" but then runs "drbdadm connect res", which walks every connection of the resource. The peers that are still connected make drbdsetup fail with "Device has a net-config (use disconnect first)" (exit code 10), the helper only logs "WARN: action failed" and the stalled peer stays disconnected. Reconnect the same res:peer pair that was just disconnected. Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Aleksei Sviridkin
|
355630a5e7 |
fix(kubernetes): pin KubeVirt CSI sidecars to SIG Storage releases (#3934)
## What this PR does Replaces five mutable OpenShift `latest` image references (four unique sidecar images) in the KubeVirt CSI controller and tenant-node DaemonSet with versioned Kubernetes SIG Storage releases. This addresses the affected references from #3578 without claiming the other references in that issue. The selected controller versions match the sidecars already used by Cozystack's LINSTOR CSI deployment. Render tests pin every changed container name and image so a mutable tag cannot return silently. Validation: - `helm unittest packages/apps/kubernetes`: 247/247 passed. - `helm unittest packages/system/kubevirt-csi-node`: 6/6 passed. - Live MSK-prod guest-cluster canaries on 2026-08-20 and 2026-08-21 reproduced the failure with the stock chart: the `latest` sidecars exited on the old CPU with the x86-64-v3 glibc error and the controller stayed in CrashLoopBackOff. A disposable guest cluster with the pinned images reached a 7/7 Ready controller on 2026-08-19 and was removed after the check. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the four changed template/test files. No package, schema, API, default, CRD, tooling contract, or downstream hardcoded value changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(kubernetes): pin KubeVirt CSI sidecars to versioned SIG Storage images that run on pre-x86-64-v3 CPUs ``` |
||
|
Andrey Kolkov
|
c9aa64cbda |
fix(dashboard): let dashboard users read backup CRD schemas (#4236)
## What this PR does
The dashboard's backup build/edit forms (`BackupPlanCreatePage`,
`BackupJobCreatePage`, `BackupRestoreJobCreatePage`, `BackupCreatePage`,
`BackupClassCreatePage`, `BackupClassEditPage`,
`BackupResourceEditPage`) build their form from the CRD's OpenAPI
schema, which `useCRDSchema` fetches directly from
`apiextensions.k8s.io` with the logged-in user's token — a
cluster-scoped `get` on the CRD by name.
No role a tenant user carries granted `customresourcedefinitions`, so
every tenant user — tenant super-admin included — hit a **403 on the
schema lookup and the form never rendered**, even though they already
hold write access to the backup CRs through `cozy:backups:admin`.
`kubectl` kept working because that only touches the CRs, not their
CRDs.
The tenant personas (`cozy:tenant:{view,use,admin,super-admin}`) are
bound only by RoleBindings, which authorize namespaced requests but
never a cluster-scoped one, so the grant belongs on
`cozystack-dashboard-readonly` in
`packages/system/dashboard/templates/rbac.yaml`: it is bound to
`system:authenticated` by a ClusterRoleBinding and already carries the
dashboard's other cluster-scoped reads (`storageclasses`,
`applicationdefinitions`, `backupclasses`). This adds a `get` on the
five `backups.cozystack.io` CRDs, scoped by `resourceNames` to those
schemas only — exactly the by-name `get` `useCRDSchema` performs, no
wildcard and no collection listing.
Verification: `helm template` renders `cozystack-dashboard-readonly`
with the new rule cleanly; the real contract is `kubectl auth can-i get
customresourcedefinitions.apiextensions.k8s.io/plans.backups.cozystack.io
--as-group=<tenant>-super-admin`, which belongs in e2e.
Fixes: #4053
### Downstream repositories
- [x] No downstream repository is affected by this change
### Release note
```release-note
fix(dashboard): grant dashboard users read access to the backup CRD schemas so the backup create/edit forms stop returning 403
```
|
||
|
Andrey Kolkov
|
3a8a06b44f |
fix(dashboard): let dashboard users read backup CRD schemas
The dashboard backup build/edit forms build each form from the CRD's OpenAPI schema, which useCRDSchema fetches directly from apiextensions.k8s.io with the logged-in user's token — a cluster-scoped get on the CRD by name. No role a tenant user carries granted customresourcedefinitions, so every tenant user hit a 403 on the schema lookup and the form never rendered, while kubectl kept working through the existing backup CR grants. The tenant personas are bound only by RoleBindings, which cannot authorize a cluster-scoped request, so the grant belongs on cozystack-dashboard-readonly: it is bound to system:authenticated by a ClusterRoleBinding and already carries the dashboard's other cluster-scoped reads. Add a get on the five backups.cozystack.io CRDs, scoped by resourceNames to those schemas only. Fixes: cozystack/cozystack#4053 Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrei Kvapil
|
5692297f94 |
feat(dashboard): move Repositories from the Marketplace to Admin
Connecting an External-Apps repository creates a cluster-scoped Tap and runs its charts in the management cluster. That is an operator action, not something a tenant does while browsing the catalog, so the entry belongs next to the other cluster-wide areas in the Admin portal rather than in the Marketplace sidebar. The page moves to /admin/taps under Administration. /marketplace/taps redirects there, since the :appName catch-all would otherwise render an order page for an application called "taps". Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Aleksei Sviridkin
|
a2db2aeb8c |
fix(seaweedfs): raise the number of buckets a volume server can serve
A volume server started with -max 0 takes a slot count derived from disk rather than a fixed one, replication 001 spends a slot on each of the two servers per logical volume, and a collection claims COPY_2 volumes whenever it grows. So the bucket ceiling is set by the product of the size limit and COPY_2, and moving either number alone moves it. The mechanism is upstream in MaybeAdjustVolumeMax (weed/storage/store.go) and findVolumeCount (weed/topology/volume_growth.go). Do not divide these out of the nominal PVC size, because the count follows the space the filesystem makes available and not the volume's advertised size: the shipped 10Gi volume PVC was seen to give nine slots per server at the upstream 1000 and 19 at 500, which puts 400 at 23 to 24 by the same division, or eight collections against two. The span is the observation's own, since the free space behind 19 at 500 fits more than one filesystem geometry and those divide differently at 400; eight either way, because a collection that cannot get a whole batch of COPY_2 keeps the volumes it did get and writes into them. The division itself is measured: on a 1 GB image the master reports max as the filesystem's available space over the limit in every run, 25 slots and nine collections at the 40 and 3 that scale to this pair. Two is what the platform's own cozy-backups bucket and a single tenant bucket already use. Past that every S3 PUT to a new bucket times out while the disk still reads as nearly empty, and no COSI object reports anything wrong, because the claim and the access are both granted. 400 and 3 make the budget 1200 MB per collection. Splitting the reduction across both numbers rather than taking it out of the size limit alone keeps the cost off the curve that grows: open descriptors track stored bytes divided by the limit, so a 10 TiB server keeps roughly 52k of them at this limit against 21k upstream, where moving the size limit alone would have put it at 105k. A .dat and an .idx stay open per volume; the .vif beside them is written and closed. The ceiling reached is the same either way. Both figures describe the shipped replicationFactor of 2. The master picks its growth count from the replication the filer asks for, so at 1 and 3 copies it reads COPY_1 and COPY_3 instead and this COPY_2 does not apply; lowering the size limit still raises the ceiling there. The size limit was out of an operator's reach: it lived in the data-plane chart, which is pulled by ExternalArtifact from a component that declares no install section, so a Package-level values override for it is dropped before any merge. The app chart now declares master.volumeSizeLimitMB and sends it on every release, and 400 is set there. The data-plane chart keeps upstream's 1000 as its fallback on purpose. A platform upgrade can re-render that release from its artifact before the parent has sent anything new, and a lower fallback would then reach the master first and drop the limit ahead of an operator's own setting. COPY_2 is sent by nothing and lives in the data-plane chart, so it does land in that window, where a grow costs 3000 MB against the 6000 MB it cost before. The app chart also writes maxVolumes into dataDirs on every release, and because dataDirs is a list Helm replaces the data-plane chart's copy wholesale, so the zero that makes the slot count auto-derived at all comes from there. Losing it drops the server to the subchart's fixed seven without failing any render. The field is bounded at both ends, because either end takes an instance down. Zero is not a smaller limit but an off switch: the volume server leaves its slot calculation untouched at zero, keeps reporting no capacity, and nothing is ever grown. Past 30000 the master does not start at all. The bound sits in the values schema rather than in a template guard, so Helm refuses the value before any template runs. The apps API does not: it reads that schema only for defaults, so it accepts a value out of range, and the release then fails to render. An explicit null gets past both bounds, because Helm drops the key before it validates, and would render an empty -volumeSizeLimitMB= that the master exits on at start, measured on the shipped image, so the template requires the value. No bound against the disk goes with it, though the field invites one. A limit at or above the volume's own size does not break an instance: MaybeAdjustVolumeMax floors the slot count at one, measured on 4.31 with a 255 MiB disk against a 1000 MB limit, which reported Max 1 and served a write it then read back. Such a server is capped at a single slot rather than dead, so refusing it at render would be a capacity policy that stops a working release on every reconcile, not only on the upgrade. The field's description carries the relation instead, where the value is chosen. The growth count is deliberately not exposed beside the limit. The master picks which of its four growth counts to apply from the replication the filer requests, and the filer takes that from replicationFactor, so a field wired to COPY_2 would do nothing at 1 or 3 copies while still appearing to be a setting. A knob that is silently inert for two of the three supported values is worse than no knob. Two further prices. A collection absorbs the budget before it needs another grow, so growth requests arrive five times as often as at the upstream pair. A collection also starts with half as many writable volumes, and a volume server takes the writes into each volume one at a time (syncWrite holds that volume's dataFileAccessLock), so a heavily parallel writer spreads over fewer volumes until the collection grows again; that holds for instances kept at 1000 too. And lowering the limit on an instance that already holds data carries the other one: its volumes between the new limit and the old go oversized on the next heartbeat and leave the master's writable set, and each affected collection needs one successful grow before its next write lands. Nothing is lost and the volumes stay readable; the room left inside them for further writes is given up and does not return as capacity. Those grows compete for one pool, and the pool is smaller than it looks. An oversized volume is charged no headroom, while one below the limit is charged all of its unfilled room; a grow then claims a whole COPY_2 batch and takes as much of it as is there, and the volumes it creates are empty and charge their headroom straight back. Measured on a local SeaweedFS 4.31 setup: two oversized collections with two free slots between them, the first to write took both, and the second stayed refused with the disk still reporting a quarter of itself free. Collections that need no grow are not free either, since their volumes below the new limit were created six at a time and each can hold back a whole limit. Also measured there, with the limit and batch scaled down: a server with free space for one batch per collection refused the grow its one affected collection needed, because a second, nearly empty collection held the room, and the same free space without it let the grow through. The minFreeSpacePercent reserve only comes into it on a server whose writable volumes have all passed the limit, where nothing is charged headroom: from 8000 MiB of filesystem upward, a 10Gi PVC measuring 9979, five percent is at least 400 MB, and such a server then refuses a grow only once its disk is down to the reserve. Everywhere else the headroom charge decides, and a refusal can arrive with the disk far from full whatever the reserve. Deleting a collection gives the room back a slot per volume, however little it stored, since the headroom charged to those volumes leaves with them. So, on a variant that runs platform migrations, the upgrade does not change the limit on an existing instance unless its operator already set one, which the previous chart accepted and ignored. Platform migration 57 writes master.volumeSizeLimitMB: 1000 into every existing SeaweedFS instance that has not chosen a limit, and 400 applies only to instances created from here on. It leaves alone only an instance whose master is not an object, which no v1 chart renders, and names that one in its log. The default variant runs no platform migrations, so a cluster there that installed SeaweedFS by hand gets no pin, and the document below gives the same patch to apply before upgrading. A chosen limit is a whole number the new chart accepts; anything else counts as not chosen and is replaced with 1000, which is what the previous chart ran in its place, since it ignored the key and the new chart would refuse to render it. A valid value already set is kept, and since the previous chart ignored it, the upgrade is when it starts to apply; the document below lists the ones that change the limit ahead of it. Raised values are listed as well: existing buckets keep writing, but every volume below the new limit is charged up to it, and on a local SeaweedFS 4.31 setup seven nearly empty volumes left no free slot once the limit went up tenfold, so a new bucket was refused. The value goes into the application's HelmRelease, whose spec.values is the application spec, for an instance created through the API and for one the tenant module renders alike. The tenant module never renders spec.values on that release, and a Helm upgrade leaves a key alone that neither manifest carries: measured with Helm 4.3.0 on client-side, server-side and forced server-side apply, and read in createPatch at the v4.1.1 that helm-controller v1.5.0 runs. Where the migration runs, it is a pre-upgrade hook of the platform release, and the cozystack-packages source every application chart comes from is a manifest of that same release, applied only after its hooks; so no instance meets the new chart unstamped, and a failed migration stops the platform upgrade before that source moves. That holds while the platform source is pinned by digest, as it is out of the box; one that follows a tag, a semver range or a branch lets source-controller move cozystack-packages on its own schedule. Until cozystack-packages moves, each patched release keeps its old chart, which ignores the key, since no v1 release reads it or closes the master object in its schema. An instance kept at 1000 still gains, because at a replication factor of 2 a grow now claims three volumes rather than six, so it has room for roughly twice as many new buckets. Lowering an existing instance is left to its operator, and docs/operations/seaweedfs-lowering-the-volume-size-limit.md is the check for it. It lists each server's volume files and adds one limit for every volume below the limit and a batch for every collection with no volume at or below 90 percent of it, since the master grows such a collection on its own schedule without a write; measured the same way, a collection with every volume in that band and no writes got two new volumes per server within one pass. Where each disk type is served by as many volume servers as the replication factor, as the shipped two at a factor of 2 are, each collection's volumes sit on every server of that type, and that bounds the claim from above without assuming how the collections grew. The master places a new volume on any server of the requested disk type with room rather than where its collection already lives, whichever group or pool that server belongs to, and it reads an empty disk type as hdd; under MultiZone it also grows each zone on its own. So the document keeps zoned instances, and any disk type served by more servers than copies, at 1000. BREAKING CHANGE: on the default variant, which runs no platform migrations, set master.volumeSizeLimitMB to 1000 before upgrading on every SeaweedFS instance that has no valid value and whose master is an object, with the step in docs/operations/seaweedfs-lowering-the-volume-size-limit.md. On the variants that run platform migrations, migration 57 does this. If the step is skipped, every such instance drops to 400 on the upgrade: each of its volumes at or above 400 MB leaves the master's writable set, and a bucket whose volumes have all crossed that line takes no writes until it can grow. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Timofei Larkin
|
ae7a7115f5 |
fix(harbor-rd): carry the release prefix in the resource selectors (#4170)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
`harbor` declares `release.prefix: "harbor-"` and then selects its
Secret, Service and Ingress without it, so all three `resourceNames`
selectors match nothing. This corrects the three names and adds a guard
so the next omission fails a test instead of passing silently.
### Why they never matched
Release names are `<prefix><app name>`, but the lineage webhook
evaluates selectors against `{kind, name, namespace}`, where `name` is
the custom resource's name and carries no prefix. Every definition
therefore spells the prefix out itself, and the others do — `postgres-{{
.name }}-credentials`, `bucket-{{ .name }}-credentials`, and so on.
`harbor` is the only one that does not.
A selector that matches nothing raises no error: the webhook labels the
object `internal.cozystack.io/tenantresource=false` and continues, which
is indistinguishable from a selector that legitimately has nothing to
match. That is why this survived.
### What actually changes at runtime
Close to nothing, and that is the point — the manifest was wrong, not
the behaviour:
- the Secret already reaches tenants, because #2541 worked around this
exact mismatch by stamping `apps.cozystack.io/user-secret: "true"` on it
and matching by label. That workaround is deliberately left in place;
- the Service and Ingress labels flip from `false` to `true`, which
nothing consumes: `spec.services` and `spec.ingresses` are read only by
the webhook that computes the label, and the label's only consumer is
the TenantSecret registry, which handles Secrets. The dashboard reaches
both through the per-release Role, which names them correctly.
So this closes a latent defect before it bites, rather than fixing an
outage.
### The guard
`hack/check-rd-release-prefix.bats` asserts that every `resourceNames`
entry in `packages/system/*-rd/cozyrds/*.yaml` contains `<prefix>{{
.name }}`. `hack/*.bats` is auto-discovered by `BATS_UNIT_FILES`, so it
runs under `make unit-tests` with no registration.
The assertion is "contains", not "starts with", because operator-created
objects legitimately carry leading words of their own: mongodb excludes
`internal-mongodb-{{ .name }}-users`, which is correct and stays green.
Definitions with an empty prefix are skipped, since `hack/update-crd.sh`
generates the definitions for `packages/extra` charts with `PREFIX=""`.
A selector that is correct and still fails the check can opt out on its
own with a trailing `# cozy-lint:ignore rd-release-prefix -- <reason>`
comment. A directive without a reason does not count.
Verified by mutation: the guard fails on the three harbor selectors
before the fix and passes after. Across the rest of the tree `harbor`
was the only violation.
### What this PR does not do
It does not settle whether the prefix belongs in the template context
instead. That would remove the duplication, but it requires migrating
the definitions that spell the prefix out today, or they would double
it. The guard is the cheaper half and closes the class either way.
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
Trigger map walked against the diff, which touches
`packages/system/harbor-rd/cozyrds/harbor.yaml` and adds
`hack/check-rd-release-prefix.bats`. The two nearest entries were
checked and neither fires:
- **terraform-provider-cozystack** lists "change `release.prefix`, or
rename an output Secret or Service". Neither happens here:
`release.prefix` is untouched and no object is renamed. The provider
concatenates the prefix itself to derive names, so it was already
computing the correct ones — it is this definition's selectors that were
wrong.
- **ccp** lists "rename `packages/system/<name>-rd/cozyrds/<name>.yaml`"
and "move or rename anything under `hack/`". The file is edited, not
renamed, and the new bats file is an addition; no make target changes
behaviour.
**website** would fire on a change to `ApplicationDefinition` semantics.
Semantics are unchanged — one definition's data is corrected — and
`content/en/docs/next/cozystack-api/application-definitions.md`
documents neither `resourceNames` nor the prefix rule, so there is
nothing there to bring in step.
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix(harbor-rd): the ApplicationDefinition selectors for Harbor's Secret, Service and Ingress now carry the release prefix, so they match the objects the chart creates. No user-visible change — the Secret already reached tenants through a label-based workaround, and nothing consumes the Service and Ingress labels.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Updated Harbor resource naming patterns to consistently include the
`harbor-` prefix for credentials, services, and ingress resources.
- Improved alignment between Harbor resource names and their configured
release prefixes.
- **Tests**
- Added validation to detect missing release prefixes in secret,
service, and ingress resource selectors across system application
definitions.
- Validation reports all affected selectors and supports documented
exceptions where necessary.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
IvanHunters
|
b222cb6580 |
fix(opensearch-operator): keep a single-replica cluster electable after bootstrap
The operator deletes the bootstrap pod as soon as initialization completes and never looks at the voting configuration first. With one cluster_manager replica there are exactly two eligible nodes, OpenSearch keeps the configuration at an odd size of one, and the voter it keeps is the bootstrap node that formed the cluster. Deleting that pod takes away the only vote there was, and the survivor can never hold an election again, because changing the configuration itself needs a quorum. The cluster is unusable about a minute after it installs, and a node with no cluster-manager still answers the readiness probe, so nothing says so until something restarts the pod. Build the operator in-tree, the way kamaji and redis-operator already are, and carry a patch that excludes the bootstrap node from the voting configuration and waits for that to commit before deleting the pod. Reaching the cluster is a precondition rather than a nicety: every failure keeps the pod and requeues, because a bootstrap pod that outlives its purpose costs a pod, while a cluster that cannot elect does not recover without a human. Deleting a pod is a request rather than a wait, so clearing the exclusion afterwards can find the node still in the cluster and fail. That path requeues and is retried once the pod is really gone, which keeps a stale entry out of the capped exclusion list; clearing stays at the API default wait_for_removal=true so it cannot re-admit a node some other operation is still draining. Upstream carries the same defect, tracked as opensearch-project/opensearch-k8s-operator#1448. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Aleksei Sviridkin
|
f9ea0da3de |
feat(backups): Kafka topic-metadata backup via a dedicated strategy driver (#4000)
## What this PR does Adds a dedicated `strategy.backups.cozystack.io/Kafka` Go driver that backs up the **topic metadata** of a Cozystack-managed `Kafka` application and wires it into the platform `cozy-default` flow — the counterpart to the app-specific drivers (CNPG, MariaDB, Altinity, MongoDB). It is a typed variant of the generic Job strategy: the controller renders a one-shot batch Job that talks to the Kafka Admin API, and adds on top an applicationRef Kind gate, a `Ready` precondition on the Strimzi `Kafka` cluster, and a per-run object key via `.BackupName` so distinct runs never overwrite each other. What it captures: every non-internal topic's partition count, replication factor and non-default configs (via `kafka-topics --describe`), stored as one small object per run in the system bucket; restore recreates them with `kafka-topics --create` + `kafka-configs --alter`. Deliberately **out of scope**: message payloads (a volume-snapshot strategy's job), consumer-group offsets (a log position, only meaningful with a byte-identical data restore), ACLs, quotas, and `KafkaUser`s. No PITR. Restored topics are created directly through the Admin API, so on the target they are unmanaged (no `KafkaTopic` CR) — the accepted tradeoff of an Admin-API-only driver. Both restore flows are implemented: **in-place** recreates dropped topics into the source, and **to-copy** applies the source's topics onto a freshly-bootstrapped empty `Kafka` (the "bootstrap a cluster, then apply the metadata state" flow). Contents: - Driver: `api/backups/strategy/v1alpha1/kafka_types.go`, `internal/backupcontroller/kafkastrategy_controller.go` (+ unit tests for its pure helpers), `internal/backupcontroller/kafkatypes/` (Strimzi `Kafka` readiness gate), dispatch cases in both controllers + `supportedBackupStrategyKinds()`, scheme registration, generated CRD + deepcopy. - Chart: `strategy-kafka-default.yaml` (gated on the bucket name), a `cozy-default` binding, read RBAC on `kafka.strimzi.io/kafkas`, and the Strimzi Kafka client image value. - Docs: a Kafka section in `docs/operations/backup-classes.md`. - Example + e2e: `examples/backups/kafka-metadata/` and `hack/e2e-chainsaw/kafka-metadata/` (`kafka-3-metadata-roundtrip`), an honest round-trip that proves a per-run `retention.ms` sentinel survives both restores. The original `packages/apps/kafka` chart is left untouched (it has no legacy backup mechanism). Validated: `go build ./...`, `go vet`, `go test ./internal/backupcontroller/...` (pure helpers plus a reconcile Kind-gate test and the RestoreJob cleanup regression test), `helm unittest` on the package (which now renders `strategy-kafka-default.yaml` and asserts its kind/name/`artifactURITemplate`), the `select-install`/`select-e2e` bats suites, `helm template` of the strategy CR, `bash -n` of the rendered script, and `make generate` (clean). The end-to-end round-trip (`kafka-3-metadata-roundtrip`) is enabled and mapped like the sibling backup suites, so CI builds this branch's controller and runs it live against a fresh sandbox; it has not been merged-and-run before, so that first live run must go green before merge. ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. This PR adds a new `strategy.backups.cozystack.io/Kafka` CRD, a Go driver, chart wiring, docs and examples. It does not add or rename a `packages/apps`/`packages/extra` package, does not change any `ApplicationDefinition`, `values.schema.json`, version enum, `release.prefix`, output Secret/Service, the `Package`/`Plan`/`RestoreJob` CRDs the provider types by hand, any `hack/*.mk` anchor or make-target behaviour, node prerequisites, labels/annotations, or telemetry metrics. A brand-new `strategy.backups.cozystack.io` CRD travels to the Terraform provider as opaque JSON (it types only `Plan`/`RestoreJob`/`Package`), so it keeps working untouched. None of the downstream triggers match. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(backups): add a Kafka topic-metadata backup strategy bound by cozy-default — backs up topic definitions and configs (partitions, replication factor, non-default configs) via the Kafka Admin API, with in-place and to-copy restore; message data, consumer offsets, ACLs, quotas and KafkaUsers are out of scope ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added Kafka topic-metadata backup and restore support, excluding message data. - Supports ad-hoc and scheduled backups, in-place restoration, and restoration to a separate Kafka cluster. - Recreates missing topics and restores compatible configurations while validating partition and replication-factor constraints. - Waits for Kafka readiness and cleans up backup artifacts after completion. - Backup and restore jobs now enforce configurable deadlines to prevent stalled operations. - **Documentation** - Added guidance covering Kafka metadata backups, restoration limitations, and S3 configuration. - **Tests** - Added end-to-end coverage for backup, restoration, validation, and cleanup scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
01967b56d3 |
docs(backups): correct the Kafka .Mode doc and the embedded-script templating note
The KafkaSpec.Template doc listed .Mode as "backup" or "restore", but the driver also renders it as "cleanup" when reaping a removed Backup's object; the comment is copied verbatim into the CRD, so kubectl explain and the dashboard showed two values while a custom strategy's restore branch would run on the cleanup pass. List all three, matching the Redis sibling, and regenerate the CRD. The strategy template also claimed the embedded script "reads only env vars, never Helm/controller values, so no tpl is applied". The controller renders every string leaf of the PodTemplateSpec through internal/template, args[0] included, so the script is templated on every run. State that instead, without literal template braces that Helm would itself expand. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |
||
|
Andrey Kolkov
|
02bf3e9970 |
fix(backups): probe live Kafka topics without a pipe and isolate the scratch file
The restore existence check piped the live topic list into `grep -qxF`. grep exits at the first match, so once the matched topic is far enough from the end of a list larger than the 64 KiB pipe buffer, the producing printf takes SIGPIPE, pipefail turns the pipeline into a failure, the `if` reads false, and a topic that exists is sent to --create, which fails with TopicExistsException. Read the list from a here-string instead: no pipe, no SIGPIPE, decided by content rather than list size. Also make the local metadata scratch file overridable (METADATA_FILE), like BIN already is, so the bats suite can point each test at its own path instead of sharing /tmp/kafka-metadata.txt across runs. The Job never sets it and keeps the default. Signed-off-by: Andrey Kolkov <androndo@gmail.com> Assisted-by: LLM |