mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-25 12:37:06 +09:00
docs/agents-dev-principles
7518
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
ccfe394ffc |
docs(agents): bring the agent docs in line with their markdown rules
overview.md now requires a language on every code fence and a blank line around every list and after every heading. It broke these rules itself, and AGENTS.md broke the list and heading ones. Label the directory-tree fences as text, separate headings and lead-in lines from the lists under them, and wrap the one bare URL. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
3293d12532 |
docs(agents): note that e2e-fork runs from the default branch
e2e-fork.yaml is triggered by workflow_run, which always executes the default branch's copy of the workflow. An edit to it cannot be observed on any fork PR, its own included, until it is merged. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
29ee36d916 |
docs(agents): define commit scope, reference trailers and PR body
Commits are what a backport cherry-picks and a revert undoes, so each must hold one logical change and be rewritten, not patched on top, when review changes it. Fixes: is reserved for a kernel-style commit reference and issues use Closes:, so the trailer always means one thing. PR bodies tended to repeat the commits while the commits lacked the rationale a future git log reader needs. Say which facts belong in the permanent record and which in the PR body, and add the review etiquette agents most often get wrong. The commit and push examples now use long flag names. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
edab402b0b |
docs(agents): require test-first with the failure path covered first
The testing section listed runners but no discipline, so agents wrote the happy-path test after the code, or none, and turned a red test green by weakening it. Require a red run first, failure paths before the happy path, and fixes in the code rather than the test. Also record the Helm numeric-scalar trap (len of an unquoted number aborts the render before any fail message), Containerfile for new first-party images, linter and markdownlint expectations, and the reserved ranges for example addresses. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
2d837de9e3 |
docs(agents): add working principles that apply to every task
The routed documents cover processes; nothing told an agent how to reason about a failure or what may leave the session. Agents tie a red run to whatever changed last, trust remembered flags and versions, run cluster-touching suites against whatever kubeconfig context is current, and leak private cluster or tool names into public text. State each rule once at the top level, where every session reads it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1d813c9445 |
fix(backups): refuse concurrent restores and skip the CNPG purge only for this restore's Cluster (#4754)
## What this PR does Fixes #3322. An in-place Postgres (CNPG) restore decided whether to skip deleting the live `Cluster` by comparing its `creationTimestamp` with the RestoreJob's `StartedAt`. The timestamp says the Cluster is new, not which restore rendered it, so two cases let a restore skip its purge and report `Succeeded` on data from another restore: 1. Two RestoreJobs into the same application at once. The second sees the Cluster the first produced, created after its own `StartedAt`, and skips its purge. 2. The live Cluster is deleted while the restore waits for the WAL archive. The still-active HelmRelease re-renders it from the previous restore's values, again created after `StartedAt`. ### Refuse a restore into an application another restore holds The RestoreJob controller now refuses, for every driver, a RestoreJob that has not started yet when another unfinished RestoreJob in the namespace writes into the same application and goes first. The refused job fails with reason `ConcurrentRestore`, and its message names the job that goes first. It fails instead of waiting, so the application is never restored a second time later, after the first restore, by a job its author may no longer expect to run. A started restore goes before one that has not started. Between two that have not, the older goes first, by creation time and then by name; creation times have one-second precision, so two jobs created in the same second are ordered by name. A running restore is never failed by a newer one. The target application is resolved by `restoreJobTarget`, the same helper the BackupJob hold uses, so a Velero restore into another namespace does not collide with an in-place one. The gate lists RestoreJobs through the manager's API reader, not the informer cache. A job reconciled before the cache holds its peer's `StartedAt` would otherwise see that peer as not started and, on a same-second name tie, proceed alongside it. The controller runs a single worker, so a peer's `StartedAt` written by an earlier reconcile is already on the API server when the next job reads it; `SetupWithManager` now sets `MaxConcurrentReconciles: 1` explicitly, since the guarantee depends on it. ### Skip the CNPG purge only for this restore's Cluster The restore patch already sets `bootstrap.newServerName` to `restoredServerName(cluster, RestoreJob UID)`, a value unique to each RestoreJob. The postgres chart now annotates the `Cluster` with it as `postgres.cozystack.io/restored-server-name`. The annotation is rendered even when the barman plugin is not; the plugin's `serverName` also carries the value, but the plugin renders only with chart-level S3 coordinates. The driver skips the purge only when the annotation names this RestoreJob. A Cluster produced by a concurrent restore, or re-rendered from an earlier restore's values, carries another value and is purged. The mark lives on the rendered Cluster rather than in RestoreJob status. That keeps it valid in the case the skip exists for: the purge succeeded, but the `TargetPurged` status write failed. A UID recorded in status was rejected in the code for exactly that reason. A Cluster rendered by a chart that predates the annotation has none and keeps the creation-time check, so a restore in flight across an upgrade behaves as before. ### Tests - `TestRestoreAhead` covers order, a started job going first, the name tie-break, finished and terminating jobs, other applications, retargeted restores and a Velero restore into another namespace. - `TestReconcileRestore_ConcurrentRestoreIsRefused` checks that the newer job fails with `ConcurrentRestore` and the running one is left alone. - `TestReconcileRestore_ConcurrentRestoreReadsPastTheCache` gives the cache a stale, not-started peer and the API reader the started one, and requires the job to fail with `ConcurrentRestore`. - `TestCNPGClusterFreshlyRecovered` adds both cases from the issue: a Cluster marked by another restore and created after `StartedAt` must be purged. The timestamp-only cases are kept for the unmarked fallback. - `TestReconcileCNPGRestore_RepeatInPlacePurgesStaleRecoveryCluster` gains two reconcile-level cases. One purges a Cluster that another restore marked, even though it was created after `StartedAt`. The other runs a full purge, takes the mark from the `bootstrap.newServerName` the restore patch wrote, drops `TargetPurged` as a lost status write would, and requires a Cluster carrying that mark to survive, although it predates `StartedAt`. That pins the purge decision to the value the chart actually renders. - `packages/apps/postgres/tests/restore_mark_test.yaml` checks that the annotation is rendered with and without the barman plugin, and is absent on a Cluster that was never restored. Each new test was checked against the code it guards: disabling the gate call, listing through the cache, flipping the started-job precedence, ignoring the annotation, or deriving the expected mark from the app name instead of the cluster name each turns one red. `go test ./internal/backupcontroller/...` and `helm unittest packages/apps/postgres` (152 tests) pass. The restore flow was not run against a live cluster. `docs/operations/backup-classes.md` documents the one-restore-at-a-time rule and the annotation. ### Screenshots N/A — no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff: Go changes in `internal/backupcontroller` (no CRD or API type change), one annotation added to the postgres chart's Cluster (no `values.yaml` or `values.schema.json` change), and a docs paragraph. No package is added or renamed, no platform values, ApplicationDefinition, RestoreJob CRD, metric, label or namespace changes, so no trigger is reached. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backups): a RestoreJob now fails with reason ConcurrentRestore when another unfinished RestoreJob is already restoring the same application, instead of running alongside it; create it again once the first restore has finished. An in-place Postgres restore now skips deleting the live Cluster only when that Cluster was rendered for this RestoreJob, identified by the new postgres.cozystack.io/restored-server-name annotation, so it no longer reports Succeeded on data from a concurrent or earlier restore. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Restores targeting the same application are now handled one at a time. A competing restore is refused rather than queued, with started restores taking priority. * PostgreSQL now calculates a default `shared_buffers` value when the memory limit is below 512 MiB, while respecting explicitly configured values. * **Bug Fixes** * Recovery clusters are more reliably distinguished from manual recoveries and clusters created by other restores, reducing the risk of purging the wrong cluster. * **Documentation** * Clarified restore ordering and how recovery-cluster cleanup is determined. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
05cfb63044 |
feat(keycloak): expose zone placement for Keycloak and the database proxy (#4606)
## What this PR does Neither the Keycloak StatefulSet nor the `keycloak-kms-proxy` Deployment exposes any placement, so the scheduler is free to put both replicas of each in one failure domain. For Keycloak this is worse than it looks: a StatefulSet does not replace a pod stuck on an unreachable node, so losing that domain takes every Keycloak replica with it for as long as the domain stays lost. This exposes placement for both, with empty defaults so nothing changes unless an installation opts in: - `affinity` and `topologySpreadConstraints` for the Keycloak StatefulSet; - `encryption.affinity` and `encryption.topologySpreadConstraints` for the proxy; - a `PodDisruptionBudget` (`maxUnavailable: 1`) for the proxy when `encryption.replicas > 1`, so a drain cannot take both replicas at once. The `values.yaml` example spreads one replica per zone with `whenUnsatisfiable: DoNotSchedule` and `nodeTaintsPolicy: Honor`. The taint policy matters for a zone loss: kube-scheduler already ignores terminating pods when it counts the spread (`countPodsMatchSelector` in `pkg/scheduler/framework/plugins/podtopologyspread/common.go`, "Bypass terminating Pod"), but with the default `nodeTaintsPolicy: Ignore` the lost zone's nodes still count as a domain holding zero pods, so a replacement pod could not be placed on the surviving zone without violating `maxSkew`. With `Honor`, nodes carrying the `node.kubernetes.io/unreachable` NoSchedule taint drop out of the count and the replacement schedules. `ScheduleAnyway` was tried first in a live deployment and did not spread the proxy replicas on rollout. Found during a site-isolation drill in which both Keycloak pods and both proxy replicas had drifted into one zone. Tests: new `tests/placement_test.yaml` (defaults render no placement; set values render as given; PDB only for more than one proxy replica); `helm unittest` passes 87/87; the default render is unchanged apart from a randomly generated secret. ### Screenshots Not applicable, no UI changes. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff: it touches only `packages/system/keycloak` (templates, values, tests) — no app under `packages/apps` or `packages/extra`, no platform values, CRD, `hack/` tooling, commit convention or PR template. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(keycloak): expose affinity and topologySpreadConstraints for Keycloak and the keycloak-kms-proxy, and add a PodDisruptionBudget for the proxy when it runs more than one replica. Defaults are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added optional node affinity and topology spread settings for Keycloak, its KMS proxy, and the database to control pod placement. * Database-specific topology spread settings take precedence over cluster-wide settings; constraints without a selector are scoped to the database cluster. * Added a disruption budget for the KMS proxy when running multiple replicas, allowing up to one pod to be unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
f093273736 |
feat(redis): size the Sentinel pods separately from Redis (#4832)
## What this PR does
The Redis application renders `spec.sentinel.resources` from the same
`resources`/`resourcesPreset` pair as the Redis pods. Every instance
runs three Sentinels, so it reserves three extra copies of a Redis
replica's memory for processes that hold only their configuration and
the failover state. The dataset that drives the Redis sizing never
reaches a Sentinel.
On our Cozystack v1.6.4 staging cluster, an HA Redis on `t1.nano`
reserves 3 x 128Mi for its Sentinels. The VPA recommender
(recommendation mode only) gives an uncapped target of 16Mi for the
`sentinel` container. On a `u1.small` instance the same three Sentinels
would reserve 3 x 4Gi.
This PR adds an optional `sentinel` block with the same two keys the top
level has:
```yaml
resourcesPreset: u1.small
sentinel:
resources:
cpu: 100m
memory: 64Mi
# resourcesPreset: t1.nano
```
Resolution rules:
- `sentinel` empty (the default): the Sentinels keep the Redis sizing
(`resources` + `resourcesPreset`). Existing instances therefore render
exactly as before and an upgrade does not roll their Sentinels.
- `sentinel.resourcesPreset` set: the Sentinels use that preset.
Explicit top-level `resources` are not carried over, since they were
sized for the dataset.
- `sentinel.resources` set: it wins. Any key it leaves out is filled
from `sentinel.resourcesPreset`, which falls back to `resourcesPreset`.
- The Sentinel `WorkloadMonitor` label
`workloads.cozystack.io/resource-preset` now carries the preset the
Sentinels actually use. The Redis `WorkloadMonitor` is unchanged.
Changes:
- `values.yaml`: new `Sentinel parameters` section (`@typedef Sentinel`,
`@param {Sentinel} [sentinel]`, default `{}`), reusing the existing
`Resources` and `ResourcesPreset` types so the enum and quantity
validation is the same as the top-level pair.
- `templates/redisfailover.yaml`: computes the Sentinel preset and
resources once and uses them for `spec.sentinel.resources` and the
Sentinel `WorkloadMonitor` label.
- `README.md`: generated table rows plus a short hand-written `sentinel`
section next to `resources and resourcesPreset`.
- Generated: `values.schema.json`, `api/apps/v1alpha1/redis/types.go`
(new `Sentinel` struct, `ConfigSpec.Sentinel`),
`api/apps/v1alpha1/redis/zz_generated.deepcopy.go`,
`packages/system/redis-rd/cozyrds/redis.yaml`.
- `tests/sentinel_resources_test.yaml`: 10 helm-unittest cases covering
the rules above and an unknown preset being refused.
I chose not to change the default sizing. A smaller default (for example
a fixed small preset for Sentinel) would save more for everyone, but it
would also roll every instance's Sentinels and change what tenants are
billed for on upgrade. If maintainers prefer that, it is a one-line
follow-up on top of this change.
Testing done:
- `make generate` in `packages/apps/redis` with cozyvalues-gen v1.7.0
(the version `pre-commit.yml` installs); regenerated files committed.
- Deepcopy: `controller-gen object` v0.16.4 on `api/apps/v1alpha1` (the
last step of `hack/update-codegen.sh`), only the redis package changed;
`go build ./...` and `go vet ./redis/` pass in that module, `go mod
tidy` leaves no diff. I could not run the full root `make generate` on
my machine: code-generator v0.37.1 fails to parse the Go 1.27.1 standard
library (`math/rand/v2: method must have no type parameters`), which is
a host toolchain problem unrelated to this change. CI's `codegen-drift`
job should confirm.
- `make test` in `packages/apps/redis` (helm-unittest v1.1.2): 58
passed. With the template change reverted, 4 of the new cases fail (the
ones that set `sentinel.*`); the cases that pin today's behaviour pass
both ways.
- Rendered output with no `sentinel` block is identical to `main` (all
documents except the random-password Secret) for the defaults,
`resourcesPreset=u1.small`, and explicit `resources` (cpu 1, memory
2Gi).
Backwards compatibility: purely additive. The field is optional with an
empty default, existing instances render unchanged, and nothing rolls on
upgrade. Clients that do not know the field are unaffected.
### Screenshots
Not a UI change. The dashboard form gets a `sentinel` object from the
schema like other nested parameters (e.g. `tls`).
### Downstream repositories
- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
Trigger map walked against the diff: this adds a field to
`values.schema.json`, so `cozystack/terraform-provider-cozystack` needs
to model `sentinel.resources` / `sentinel.resourcesPreset` in its Redis
resource. I will open that issue once this PR has a number and link it
here; a draft is in `FOLLOWUP-terraform-provider-redis-sentinel.md`. The
website page for `redis` is regenerated from `README.md` by the release
bot (`redis` is in the website `APPS` list), so no manual follow-up
there. Nothing else in the map is touched.
### Release note
```release-note
feat(redis): new optional `sentinel.resources` and `sentinel.resourcesPreset` parameters size the three Sentinel pods independently of the Redis pods. When unset, Sentinels keep the Redis sizing as before, so existing instances are unchanged.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added independent CPU and memory resource settings and sizing presets
for Redis Sentinel pods.
* When Sentinel-specific settings aren’t provided, Sentinel pods inherit
Redis resource settings. Unspecified Sentinel resource values can use
the Sentinel preset, which defaults to the Redis preset.
* Added configuration documentation and validation for Sentinel resource
settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
bd8264fbda |
docs(api): clarify kind-wide Helm timeout overrides (#4493)
## What this PR does Clarifies that Helm install and upgrade timeout annotations are configured on an ApplicationDefinition and apply to every Application of that kind. It also documents that the upgrade-specific timeout takes precedence over the shared install timeout and the global upgrade default. Fixes #4382 ### Screenshots N/A — this change only updates comments and CLI help text. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified that install and upgrade timeout overrides are configured for an Application kind through its `ApplicationDefinition`, rather than per Application. * Specified that the install-timeout setting applies to both installs and upgrades, while the upgrade-timeout setting takes precedence for upgrades. * Updated descriptions of global timeout defaults and override precedence. Timeout behavior is unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
08ee470f6a |
fix(postgres)!: refuse plaintext sessions on external instances (#4847)
## What this PR does CloudNativePG offers TLS on an external Postgres but does not require it. Its built-in last `pg_hba` rule is `host all all all`, which matches TLS and non-TLS connections alike, so a client outside the cluster that connects with `sslmode=disable` gets an unencrypted session. The chart's `tls.enabled` only decides whether the external hostname goes into the certificate, so it cannot prevent this. This is #4619. When `external: true`, the chart now renders `spec.postgresql.pg_hba`: ```yaml pg_hba: - hostssl all all all scram-sha-256 # md5 on PostgreSQL 13, matching CloudNativePG's default rule - host all all all reject ``` CloudNativePG puts user rules after its fixed rules (`local` peer, `streaming_replica` and pooler `cert`) and before its default `host all all all` rule, and PostgreSQL uses the first matching rule, so the two lines above make remote sessions TLS-only without touching replication or the metrics exporter. See the CloudNativePG [`pg_hba` section](https://cloudnative-pg.io/documentation/current/postgresql_conf/#the-pg_hba-section) and the [template](https://github.com/cloudnative-pg/cloudnative-pg/blob/v1.30.0/pkg/postgres/configuration.go) for v1.30.0, the operator version the platform ships. A new tri-state `tls.required` follows the same pattern as `tls.enabled`: when omitted it follows `external`, `false` opts out, `true` requires TLS on an in-cluster instance too. The issue is silent on a switch; I added one because the change is breaking and a tenant with a client that cannot do TLS needs a way back that does not mean patching the operator's resources. In-cluster clients: the pod network cannot be told apart from an external client (the external Service uses `externalTrafficPolicy: Local`, so the source IP is the client's), so the rules apply to every remote client. The chart's init job connects to `<release>-rw` with libpq's default `sslmode=prefer`, which already negotiates TLS, as do the backup and monitoring paths that run inside the pod over the Unix socket. Only clients that disable TLS explicitly are refused. In-cluster instances with `external: false` are unchanged. Changes: - `templates/_tls.tpl`: `postgres.tls.required` helper. - `templates/db.yaml`: renders `spec.postgresql.pg_hba` when required. - `values.yaml`: `tls.required`, with the generated `values.schema.json`, `README.md`, `api/apps/v1alpha1/postgresql/types.go` and `packages/system/postgres-rd/cozyrds/postgres.yaml`. The `Required` deepcopy block in `zz_generated.deepcopy.go` is controller-gen v0.16.4 output. - `README.md`: new "Refusing plaintext sessions" paragraph. - `tests/pg_hba_test.yaml`: 10 helm-unittest cases. Backwards compatibility: breaking for existing external instances whose clients connect without TLS. The rules apply on the next reconcile. They are refused until the client enables TLS or the instance sets `tls.required: false`. Testing done: - `make generate` in `packages/apps/postgres` with cozyvalues-gen v1.7.0; regenerated files committed. - `helm unittest .` in `packages/apps/postgres`: 206 passed (25 suites), 10 of them new. I did not run it against a live cluster, so the `pg_hba` ordering is taken from the operator source and docs rather than observed. ### Screenshots Not a UI change. The dashboard form gets a `tls.required` toggle from the schema. ### Downstream repositories I walked the trigger map against the diff and left every box empty so a human decides. The nearest hit is the website's managed-apps reference page for postgres, which a stable-tag bot regenerates from `README.md`; the Terraform provider's postgres schema may also want the new `tls.required` field. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(postgres)!: an external Postgres now refuses sessions that are not encrypted. The chart renders `hostssl` rules for all users and databases with scram-sha-256 (md5 on PostgreSQL 13, matching CloudNativePG's default rule) and rejects plain `host` connections, so a client using `sslmode=disable` is refused. The new `tls.required` value follows `external` by default; set it to `false` to keep accepting plaintext sessions. ``` Fixes #4619 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a `tls.required` setting for PostgreSQL instances. By default, TLS is required for external instances and optional for others. You can override this behavior to require encrypted connections for both external and in-cluster clients, or allow plaintext connections for external instances. Rendering fails if TLS is required while PostgreSQL SSL is disabled. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
f0e3e17493 |
feat(kubernetes-nodes): let CDI trust a self-hosted Talos image factory (#3797)
## What this PR does Adds `talos.imageFactoryCA` to the `kubernetes-nodes` chart, so CDI can verify a Talos Image Factory served with a certificate from an internal CA. Fixes #3796. ### The gap `talos.imageFactoryURL` is documented for self-hosted factories and mirrors in air-gapped, rate-limited or flaky-egress environments. Such an endpoint is normally served by the organisation's own CA, and CDI, which trusts only its system store, refuses the import with `x509: certificate signed by unknown authority`. The DataVolume never completes and no worker joins. `DataVolume.spec.source.http.certConfigMap` exists for exactly this, but the worker DataVolume in `templates/nodegroup.yaml` renders `url` alone and no value reaches that field. ### The change One optional value on the existing `talos` group of a `KubernetesNodes` pool: ```yaml talos: imageFactoryURL: "https://factory.example.com" imageFactoryCA: | -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- ``` When set on an HTTP import path (the default and `osImage.factory`), the chart renders a ConfigMap `<release>-talos-factory-ca` in the release namespace and points the DataVolume's `source.http.certConfigMap` at it. A pool on `osImage.builtin` clones a golden image, builds no HTTP source and gets no ConfigMap. Empty, the default, renders nothing: `helm template` with the field unset is byte-identical to `main`, so existing pools keep their KubevirtMachineTemplate name. Setting the CA changes that name, so the pool rolls once. CDI appends the bundle to the system pool rather than replacing it (`x509.SystemCertPool()` followed by `AppendCertsFromPEM` in `pkg/importer/http-datasource.go`), so a publicly trusted endpoint keeps verifying. The PEM is taken inline rather than as a ConfigMap name because `certConfigMap` must live in the DataVolume's namespace, and the dashboard offers no way to create that object by hand. ### Fail-closed guards The value lands verbatim in a ConfigMap that anything able to read the namespace can read, so it must be complete PEM certificate blocks and nothing else, and must never carry private key material. That includes the realistic accident of `cat ca.pem key.pem` pasted into the field. The rules now live in one shared helper, `cozy-lib.tls.validateCACert`, which `cozy-lib.tls.caCertSecret` also calls instead of keeping its own copy. The regexes are unchanged; the failure message names the caller's field. ### Testing - `packages/apps/kubernetes-nodes/tests/talos_image_factory_ca_test.yaml`, 13 cases: unset renders nothing, set renders the ConfigMap and a matching `certConfigMap`, whitespace-only counts as unset, multi-certificate chains, the builtin path, and each guard on the input that trips it. - `packages/tests/cozy-lib-tests/tests/tls_validatecacert_test.yaml`, 6 cases for the shared helper, plus the existing `caCertSecret` tests updated for the new message. - `make generate` in the package: regenerated `values.schema.json`, `README.md`, `api/apps/v1alpha1/kubernetesnodes/types.go` and the `kubernetes-nodes-rd` definition are committed. Not yet exercised end to end against a live internal factory. ### Not in this PR `packages/apps/vm-disk/templates/dv.yaml` renders `source.http.url` with the same gap. Left for a follow-up if this shape is accepted. ### Screenshots Not a UI change. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: cozystack/terraform-provider-cozystack#31 (the field belongs to the `kubernetes_nodes` model) - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: Walked the trigger map against the diff: - **terraform-provider-cozystack**: ticked. The diff adds a field to `packages/apps/kubernetes-nodes/values.schema.json`, which the trigger map names directly. Filed as an issue there. - **website**: not ticked. No package is added, renamed or removed, and the reference page is regenerated from the package `README.md`, which is regenerated and committed here. - Every other repository: no path in the trigger map is touched. ### Release note ```release-note feat(kubernetes-nodes): add `talos.imageFactoryCA` so CDI can verify a self-hosted Talos Image Factory served with a certificate from an internal CA. Previously the worker image import failed with `x509: certificate signed by unknown authority`. Set the CA in PEM form and the chart wires it into the worker DataVolume; leave it empty and nothing changes. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an optional setting for providing PEM certificate authorities trusted when importing worker OS images over HTTPS. This supports the default image source and factory images, allowing imports from endpoints that use certificates not already trusted by the system. * Certificate authorities are provided as PEM certificate blocks; when configured, they are used for HTTPS image imports. The setting defaults to empty and has no effect on pools using built-in OS images. * **Bug Fixes** * Invalid certificate input, including private keys, is rejected. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
dde75d6cf5 |
feat(keycloak): expose zone placement for the keycloak-db instances
The Keycloak StatefulSet and the database proxy can already be spread across failure domains, but the CloudNativePG Cluster behind them could only follow the cluster-wide topology constraint, with no way to pick zones for this database alone. With both instances in one zone, losing that zone loses the database and every login with it. Add database.affinity (CloudNativePG shape) and database.topologySpreadConstraints, with empty defaults so nothing changes unless an installation opts in. The cnpg.io/cluster label selector is added to constraints that carry none, and the constraints replace the cluster-wide ones for this database when set. Assisted-by: LLM Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Aleksei Sviridkin
|
faf8a250ba |
fix: typo in monitoring-agents values.yaml (#4799)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Fixes a typo in monitoring-agents values.yaml. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Corrected the namespace metadata field used for event streams. Event metadata is now labeled consistently, helping downstream tools identify the namespace associated with each event. This fixes an incorrect field name in the event output configuration without changing other event stream behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
6a75883013 |
fix(backups): keep the CNPG purge for unmarked Clusters this restore did not render
A recovery Cluster without the restored-server-name annotation fell back to the creation-time comparison. On the current chart the annotation is absent when bootstrap.newServerName is empty, i.e. a manually configured recovery: re-rendered during the WAL-archive wait it is created after StartedAt, the purge was skipped and the restore could report success on the manual recovery source. Use the creation-time fallback only when the application's bootstrap.newServerName already carries this restore's server name, which keeps it for Clusters rendered by a chart that predates the annotation. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
35ca217839 |
fix(backups): name the restore that goes first in ConcurrentRestore
restoreAhead returned the alphabetically smallest of the jobs ahead, so with more than one ahead the ConcurrentRestore message could name a job that is not the one holding the application. Return the first job in restoreGoesFirst order, and pin the creation-time tie-break with cases where creation order and name order disagree. Assisted-By: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
e4e991e837 |
fix(backups): skip the CNPG restore purge only for this restore's Cluster
The in-place CNPG restore skips deleting the live Cluster when it carries bootstrap.recovery and was created after the RestoreJob started, taking that as "this restore's own purge and re-render produced it". The timestamp does not say which restore rendered the Cluster. A Cluster deleted while the restore waits for the WAL archive is re-rendered by the still-active HelmRelease from the previous restore's values and is created after StartedAt too; the restore then skipped its purge and reported Succeeded on the previous restore's data. The restore patch already sets bootstrap.newServerName to restoredServerName(cluster, RestoreJob UID). The chart now annotates the Cluster with it as postgres.cozystack.io/restored-server-name, whether or not it renders the barman plugin that also carries the value, and the purge is skipped only when the annotation names this RestoreJob. The mark lives on the rendered Cluster rather than in RestoreJob status, so it still covers the failed status write the skip exists for. A Cluster rendered by a chart that predates the annotation keeps the creation-time check. Fixes #3322 Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
3daf942dce |
fix(backups): refuse a restore into an application another restore holds
Nothing stopped two RestoreJobs from rewriting one application at once. Each restores the target from its own Backup, so the application ends up on whichever finishes last, and the CNPG driver's purge decision cannot tell which of the two produced the live Cluster: the second job could skip its purge and report Succeeded on the first job's data. A RestoreJob that has not started now fails with reason ConcurrentRestore when another unfinished RestoreJob in the namespace writes into the same application and goes first. A started restore goes before one that has not; between two that have not, the older one goes first, by creation time and then name, so a running restore is never failed by a newer one. The refused job fails instead of waiting, so the application is not restored a second time later, after the first restore, by a job its author may no longer expect to run. The gate lists RestoreJobs through the manager's API reader rather than the informer cache. Creation times have one-second precision, so two jobs created in the same second are ordered by name, and a job reconciled before the cache held its peer's StartedAt would see that peer as not started and proceed alongside it. The controller runs one worker, so a peer's StartedAt written by an earlier reconcile is on the API server by the time the next job reads it; SetupWithManager now states that. The target is resolved as the BackupJob hold resolves it, so a Velero restore into another namespace does not collide with one in place. Refs #3322 Assisted-by: LLM Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Aleksei Sviridkin
|
22e0fdbc63 |
test(tests): stop unit tests from reaching a cluster (#4804)
## What this PR does One unit test was reading from whatever cluster the caller's kubeconfig selected. The knob test in `hack/run-kubernetes-node-join_test.bats` runs `cozy_report_node_join_failure` in a fresh `bash -c` and stubs only `kubectl`, as a shell function. `cozy_diag_read` runs every read through `timeout`, and `timeout` execs the real `kubectl` from PATH, so the stub never ran. With a recording `kubectl` first on PATH, one run of that file logged 170 calls such as `kubectl -n tenant-test describe virtualmachineinstances.kubevirt.io`. All of them were reads, but a unit suite should not touch a cluster at all. The PR has three parts: 1. Both `bash -c` blocks in that file now stub `timeout`, as the post-source test already did. They also put recording `kubectl` and `talosctl` executables first on the subshell's PATH and fail if either one is executed. The declined-path block made no calls in my runs, because a zero budget declines every read. It gets the same guard because it has the same shape. 2. The unit and controller targets now run with no cluster to reach. The root `Makefile` sets `KUBECONFIG=/dev/null` and an empty `KUBERNETES_SERVICE_HOST` for `unit-tests`, each of its prerequisites, and `test-controllers`, so the CI command and each target run on its own are covered. The second variable is needed because client-go falls back to in-cluster credentials when the kubeconfig is empty and that variable plus a service-account token are present. The new `hack/unit-lane-no-cluster.bats` runs every prerequisite of `unit-tests` as its own goal with a sentinel kubeconfig and service host in the environment, and fails if any target receives them. 3. One unit file was missing the strict-mode helper. `hack/bats-strict-setup.bats` requires a `load test_helper` line in every unit file, and `hack/fork-new-images.bats` is the only one without it, so the bats lane is red on main and would stay red here. #4805 carries the same one-line fix on its own; if it lands first, this commit drops out on rebase. I ran the whole lane (`make unit-tests test-controllers`) with recording fakes for 20 cluster clients first on PATH. It made no `kubectl` call. `helm` was called 444 times, all `template` or `unittest`, none with `--validate`. `flux` was called once, for `push artifact --help`. Deleting either export line from the Makefile, or removing one target from the list, turns the new test red. Removing the `timeout` stubs turns the knob test red, and the failure lists the reads that escaped. Closes #4793 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note test(tests): unit and controller test targets run with no kubeconfig and no in-cluster service host, so a unit test can no longer reach a cluster. ``` |
||
|
Aleksei Sviridkin
|
bcde187858 |
fix(dashboard): small Console fixes from the #3828 triage (#3837)
## What this PR does Six small Console fixes from the #3828 triage. Each one is small and most of the code sits in `apps/console/src`, so they go together in one PR. `Fixes #3105`: sizes between 1Ki and 1Mi printed as a raw byte count. `humanizeBytes` had branches for Ti, Gi and Mi and then fell through to bytes. It now has a Ki branch with the same `toFixed(0)` as the Mi branch. The `1023B` case still holds, and the test covers 1Ki and 512Ki. `Fixes #3106`: the tenant picker showed on admin pages where the tenant changes nothing. It is now hidden on the cluster-scoped admin pages only: Capacity, Backup classes, External IPs and Modules. The rest of `/admin` mounts the same tenant-scoped pages as `/console`, and those take their namespace from the tenant context, so the picker stays there. `/admin/tenants` keeps it too, because its rows come from the picker-filtered tenant list. `Fixes #3107`: both capacity drill-downs showed one generic error, so a permission failure looked like a broken page. Both now use the `error instanceof K8sApiError && error.status === 403` check that `ClusterStorageSection` already uses, with a test for the forbidden case and one for a generic error on each page. `Fixes #3102`: an immutable leaf was not restored when the YAML edit dropped its parent object. `overlayPath` stopped as soon as the submitted body had nothing at a path segment. It now creates `{}` for the missing parent, but only when the target there is undefined or null and the original really has the leaf to restore. So a scalar or an explicit null the user put there survives. A wildcard with more path under it does not rebuild a deleted parent, since that would put back entries the user removed. The test uses foundationdb's `storage.storageClass`, which is a shipped path of this shape. `Fixes #3135`: a blocked submit did not scroll to the field that blocked it. Plain `focusOnFirstError` crashes here, because RJSF's built-in handler reads `form.elements` and this form is `tagName="div"`. So the form passes a small handler that finds the field by its generated id and scrolls it into view. The ids come from one shared definition, and the widgets that had none now carry it: the `DynamicOptionsWidget` select, which every `x-cozystack-options` field uses, and the `SourceField` controls. The nested form that `AdditionalPropertiesField` mounts per map entry continues the outer id namespace instead of restarting at `root`, so fields like `nodeGroups.*.instanceType` resolve. A map key with a dot in it, such as a mariadb user `john.doe`, resolves by trying both readings of the dotted error path. `Fixes #3108`: `cilium`, `coredns` and `verticalPodAutoscaler` have only `valuesOverride` and no `enabled`, so they rendered as plain groups next to the toggleable addons and looked like a switch that failed to appear. Setting `<addon>.enabled: true` in the YAML editor is accepted, because the schema sets no `additionalProperties`, and the API stores the field and echoes it back. Nothing reads it. The three addons are not the same case. `cilium` and `coredns` are always installed. `verticalPodAutoscaler` has no switch of its own and is installed and removed together with `addons.monitoringAgents.enabled`. The three are now marked in `values.yaml` with a new `x-cozystack-no-enable-switch` schema keyword, and the form renders them with a note that there is no enable switch and an `enabled` field in YAML does nothing. The per-addon reason goes into the schema description, which the form renders and which regenerates into the README, the Go types and the ApplicationDefinition. The generated files are regenerated with cozyvalues-gen v1.7.0, the version CI pins, and the pre-commit check passes. The console typecheck and tests pass in CI. ### Screenshots ##### Ki formatting <img width="1182" height="184" alt="image" src="https://github.com/user-attachments/assets/62662265-4f77-455d-b90e-85e417684bb9" /> ##### Admin without tenant picker <img width="1440" height="245" alt="image" src="https://github.com/user-attachments/assets/99aff4f3-fda9-4c4c-8cf8-0c1eb01fb973" /> ##### Capacity 403 error <img width="1232" height="270" alt="image" src="https://github.com/user-attachments/assets/f4e4b328-ee5e-4516-b78f-9b5084b6bbc8" /> <img width="1232" height="944" alt="image" src="https://github.com/user-attachments/assets/142e438e-f538-436e-aae8-dfb79ec747e8" /> ##### Validation focus <img width="1166" height="135" alt="image" src="https://github.com/user-attachments/assets/89abb73b-a624-406f-9736-3bf38bb8a923" /> ##### Addon lifecycle <img width="1166" height="591" alt="image" src="https://github.com/user-attachments/assets/14a95891-84d1-4073-a009-bc0eab94f114" /> ### Downstream repositories I walked the trigger map in `docs/agents/contributing.md` against the diff, file by file. This PR changes field descriptions, adds the `x-cozystack-no-enable-switch` schema keyword on three addon objects, and changes the console. No field is added, removed or renamed, and no default, version enum, `kind`, `plural`, `release.prefix`, output Secret or Service changes, so none of the terraform provider triggers fire. No package is added or removed, and nothing changes in `packages/core/platform/values.yaml`, the variants, `hack/` or the node prerequisites. cozyvalues-gen itself is unchanged, since the keyword goes through its generic `@x-*` passthrough. The kubernetes reference page on the website regenerates from `README.md` on a stable tag, which is automated rather than a follow-up. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(dashboard): Ki-range sizes now render as Ki instead of raw bytes, the tenant picker is hidden on cluster-scoped admin pages, capacity drill-downs distinguish a permission error from a broken page, an immutable field whose parent object is absent is now applied, a blocked submit scrolls to the field that blocked it including fields inside maps and the source and dropdown widgets, and the cluster addons with no enable switch now say so instead of showing a toggle that never appears ``` |
||
|
Aleksei Sviridkin
|
1b0a5a2829 |
fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2 (#4834)
## What this PR does Bumps the `velero-plugin-for-aws` init container in `packages/system/velero/values.yaml` from `v1.14.1` to `v1.14.4`, and updates the pin in `tests/velero_test.yaml`. `v1.14.1` sets `x-amz-tagging` on every `PutObject`, even when the BackupStorageLocation configures no tags, in which case the header is empty. Cloudflare R2 rejects that with `501 NotImplemented: Header 'x-amz-tagging' with value '' not implemented`. Velero then cannot upload backup metadata, so every Velero backup to an R2-backed location fails. That includes Cozystack's own `cozy-default` BackupStorageLocation when the platform backup bucket is on R2. Upstream fixed this in v1.14.3: velero-plugin-for-aws#309 (a backport of #299, "Only set PutObject Tagging when tags are configured"). `v1.14.4` is the current patch release of the 1.14 line. Its only change on top of v1.14.3 is CVE dependency bumps (#336). v1.14.3 itself also carries #310 (SSE-C presigned URLs), #313 (no amz-checksum encapsulation when `checksumAlgorithm=""`) and CVE/Go bumps. The plugin's compatibility table lists `v1.14.x` for Velero `v1.18.x`, which is what the chart ships (`velero:v1.18.1`), so this stays inside the supported pairing. Testing done: - `make test` in `packages/system/velero` (helm-unittest v1.1.2): 7 passed, with the existing init-container pin assertion moved to `v1.14.4`. - `helm template` renders `velero/velero-plugin-for-aws:v1.14.4` as `initContainers[0]` of the `velero` Deployment. - On a Cozystack v1.6.4 staging cluster with the backup bucket on Cloudflare R2: before the bump, Velero backups failed on the empty `x-amz-tagging` header. After pinning the same plugin image through a Package values override (`docker.io/velero/velero-plugin-for-aws:v1.14.4@sha256:c85a7fa803569e3128c33e4f5766f84ab156c8d171c9f1dd2715581d45ce98fa`), all of our volume backup and restore drills pass. That covers CSI snapshots with the Kopia data mover to R2, and incremental backups confirmed. Backwards compatibility: same plugin minor line and the same Velero version. The init container image changes, so the `velero` Deployment rolls once on upgrade. Users who never set `tagging` on a location now get no tagging header at all; users who do set it get the same header as before. The reference stays in the existing shape (tag only, no digest), like the current `v1.14.1` line. If you'd rather pin third-party init containers by digest, the v1.14.4 multi-arch index digest is `sha256:c85a7fa803569e3128c33e4f5766f84ab156c8d171c9f1dd2715581d45ce98fa`. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Trigger map walked against the diff: a third-party image tag in a system package values file and its test; no app schema, platform values, names or tooling. Fixes #4852 ### Release note ```release-note fix(velero): velero-plugin-for-aws is bumped from v1.14.1 to v1.14.4, which no longer sends an empty x-amz-tagging header, so Velero backups work against S3-compatible stores that reject it, such as Cloudflare R2. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Updates** * Updated the AWS backup plugin used by Velero to version 1.14.4. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
cd1345e661 |
fix(redis): re-send the Sentinel auth-pass when the operator resets a Sentinel (#4833)
Fixes #4851 ## What this PR does Adds a downstream patch to the redis-operator image (`packages/system/redis-operator/images/redis-operator/patches/unauthenticated-sentinel.diff`) so that a Sentinel which lost its `auth-pass` is repaired by the operator instead of being reset forever. ### The failure After a node power cycle on a Cozystack v1.6.4 cluster (3 nodes, Talos, HA Redis with `authEnabled: true`), the Sentinel pod on that node came back monitoring the correct master address, but with no `auth-pass`. The master answered it with `NOAUTH`, it reported the master as `s_down,disconnected`, and `SENTINEL CKQUORUM mymaster` on it returned `NOQUORUM 1 usable Sentinels`. The operator logged `mismatch number of sentinels in memory. resetting` on every resync and sent `SENTINEL RESET *`. That does not touch `auth-pass`, so the Sentinel never recovered on its own: the instance keeps running with two working Sentinels out of three until someone deletes that pod. One more failure then leaves Sentinel without a majority. The mechanism, which I reproduced locally with Redis 8.6.2 (script and output in #4851): 1. The pod restarts in place (same pod, same IP), so the `sentinel-config-copy` init container writes the operator's initial `sentinel.conf` again: `sentinel monitor mymaster 127.0.0.1 6379 2` and no `auth-pass`. 2. The other Sentinels still know that address and keep sending their hello messages to it directly. A failover has raised the config epoch, so the restarted Sentinel adopts the current master address from a peer (`+config-update-from sentinel ... +switch-master mymaster 127.0.0.1 ... <current master>`) before the operator looks at it. 3. `CheckAndHeal` then finds the right address in `CheckSentinelMonitor` and skips `NewSentinelMonitor`, the only call that sends `SENTINEL SET <master> auth-pass`. 4. `INFO sentinel` reports `status=sdown`, so `CheckSentinelNumberInMemory` fails and `RestoreSentinel` sends `SENTINEL RESET *`. The reset keeps the empty `auth-pass`, and the cycle repeats. The same code is on `main`: freshworks-oss v3.3.5 sets `auth-pass` only in `MonitorRedisWithPort`, and none of the three patches Cozystack carries touch this path. ### The change `RestoreSentinel` sends `SENTINEL SET <master> auth-pass <password>` right after `SENTINEL RESET *` when the RedisFailover has auth enabled. It uses the same password `NewSentinelMonitor` uses (`k8s.GetRedisPassword`), the same TLS config, and the same master name (`rf.MasterName()`, so `disableMyMaster` is respected). - It only runs on the unhealthy path. `RestoreSentinel` is called only when a Sentinel's view is already wrong: its sentinel or replica count does not match, or its status is not `ok`. A healthy Sentinel is never touched. - It does not disturb a working Sentinel. `SENTINEL SET auth-pass` drops the Sentinel's links to the monitored instances, but the `RESET` just before it has already done that. - Without auth, `GetRedisPassword` returns an empty string and nothing is sent, so the behaviour is exactly as before. - Bootstrap mode goes through the same function and gets the same password `NewSentinelMonitorWithPort` already sends there. Patch contents: a `SetSentinelAuthPass` method on `service/redis.Client`, its mockery-style mock, the two-step `RestoreSentinel`, and `TestRestoreSentinel` (5 cases: auth off, auth on, reset error, unreadable Secret, `SET` error). The file name sorts after `tls.diff` on purpose: the patch is written on top of it (same functions, new `tlsConfig` parameters), and the Dockerfile applies `patches/*.diff` in glob order. If you prefer numbered patch names, I can rename it. Like `tls.diff`, this belongs upstream too: the same fix is open as freshworks-oss/redis-operator#90. This carried patch can be dropped once a release containing it is vendored here. ### Why not a quorum-checking liveness or readiness probe The RedisFailover CRD does allow `spec.sentinel.customLivenessProbe` / `customReadinessProbe`, so a `SENTINEL CKQUORUM` probe could be set from the chart. I looked at that first and decided against it: - A liveness restart does not repair this. It restarts the container, not the pod. `sentinel.conf` lives on an `emptyDir` that survives a container restart, and only the init container rewrites it, on pod start. The restarted Sentinel reads the same file without `auth-pass` and fails again, so the probe would restart it in a loop. - `CKQUORUM` does not reliably detect it. In the local reproduction the broken Sentinel still answered `OK 3 usable Sentinels`, because its peers reach it directly. On the cluster it answered `NOQUORUM`. What does show the problem every time is `status=sdown` in `INFO sentinel`, which the operator already reads. - It hurts in real outages. With two of three nodes down, a quorum-based liveness probe would restart the last Sentinel, and a quorum-based readiness probe would empty the `rfs-` Service, at the moment clients most need it. The operator already detects this state and already runs a repair for it. The repair just did not include the password. ### Testing done - Applied the four patches to a fresh v3.3.5 tarball in Dockerfile order (`git apply patches/*.diff`). Then ran the Dockerfile's test step: `go test .` and `go vet -tags integration .` in every directory the patches touch, all 11 pass. Also ran `GOOS=linux GOARCH=amd64 go build ./cmd/redisoperator`. I could not run `make image` because Docker was not available on my machine. - `TestRestoreSentinel` fails on 3 of its 5 cases without the `heal.go` change and passes with it. The existing `operator/redisfailover/...` and `service/redis` tests pass unchanged. - End-to-end against real Sentinels (Redis 8.6.2, local): master + replica with `requirepass`, three Sentinels configured the way the operator configures them, a `SENTINEL FAILOVER`, then one Sentinel restarted on the same port with the operator's initial config. Result: that Sentinel switched to the new master from a peer's hello and sat in `status=sdown`, `flags s_down,master,disconnected`. `SENTINEL RESET *` alone left it in `sdown`. The patched `service/redis` client (`ResetSentinel` then `SetSentinelAuthPass`) brought it to `status=ok,...,slaves=1,sentinels=3`. - The mock was written in the format mockery v2.45.0 generates; v2.45.0 itself does not build with Go 1.26+ (`golang.org/x/tools` v0.17.0), so I could not regenerate it mechanically. Backwards compatibility: no API or chart change. The operator image has to be rebuilt to pick up the patch, like the existing patches (the image digest in `packages/system/redis-operator/values.yaml` is stamped by the build). Instances without auth are unaffected. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: Trigger map walked against the diff: one new file under `packages/system/redis-operator/images/redis-operator/patches/`; no values, schema, CRD, name or tooling change. ### Release note ```release-note fix(redis): the redis-operator now re-sends the Redis password to a Sentinel it resets, so a Sentinel that restarted in place (for example after a node reboot) and lost its auth-pass recovers on the next check instead of staying down until its pod is deleted. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Sentinel recovery now restores the Redis authentication password after a successful reset when a password is configured. * Recovery stops if the reset fails and reports errors encountered while retrieving or applying the password. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
gireassen
|
525814d399 |
fix: typo in monitoring-agents values.yaml
Signed-off-by: gireassen <80508012+gireassen@users.noreply.github.com> |
||
|
frolov.so
|
9b35fa0b48 |
docs(api): clarify kind-wide Helm timeout overrides
Timeout annotations are read from an ApplicationDefinition and therefore affect every Application of that kind. Describing them as per-Application hides their actual scope and the upgrade override precedence. Fixes cozystack/cozystack#4382 Assisted-by: LLM Signed-off-by: frolov.so <opsmon.sh@yandex.com> |
||
|
Fedor Batonogov
|
68cd1028bd |
feat(kubernetes-nodes): let CDI trust a self-hosted Talos image factory
talos.imageFactoryURL is documented for self-hosted factories and mirrors in air-gapped, rate-limited or flaky-egress environments. Such an endpoint is normally served with a certificate from an internal CA, and CDI — which trusts only its system store — refuses the import with "x509: certificate signed by unknown authority". The DataVolume never completes, no worker joins, and the release sits in install until it times out. DataVolume.spec.source.http.certConfigMap exists for exactly this, but the chart rendered source.http.url alone and no value reached the field. Add an optional talos.imageFactoryCA taking the CA in PEM form. When set, the chart renders it into a ConfigMap in the release namespace — CDI requires certConfigMap to sit alongside the DataVolume — and references it from the worker DataVolume on the HTTP import path; a pool cloning a golden image via osImage.builtin builds no source.http and renders neither. When empty, nothing is rendered and the output is byte-for-byte what it was. CDI appends the bundle to the system pool rather than replacing it, so a release pulling from a publicly-trusted endpoint keeps verifying either way. The value is emitted verbatim into a ConfigMap readable by anything that can read the namespace, so the render fails closed on private key material and on anything that is not complete PEM certificate blocks. Those guards live in a new validate-only cozy-lib.tls.validateCACert helper that takes the caller's field name for its messages, and cozy-lib.tls.caCertSecret is refactored onto it, so the check exists in exactly one place. Assisted-by: LLM Signed-off-by: Fedor Batonogov <fedor@batonogov.ru> |
||
|
Yasar Rahman
|
8580a9d46d |
feat(redis): size the Sentinel pods separately from Redis
Every Sentinel pod gets the Redis pods' resources, so the three Sentinels of an instance reserve three times a Redis replica's memory while holding only their configuration and the failover state. On a t1.nano instance that is 3 x 128Mi while the VPA recommender's uncapped target for a Sentinel container is 16Mi, and the waste grows with every larger preset, because the dataset that drives the Redis sizing never reaches a Sentinel. sentinel.resources and sentinel.resourcesPreset follow the convention of the top-level pair, and sentinel.resourcesPreset falls back to resourcesPreset. Explicit Redis resources are not carried over once the sentinel block sets either key, since they were sized for the dataset. An empty sentinel.resources map counts as unset, as an empty top-level resources does. An empty sentinel block keeps today's sizing, so existing instances render unchanged and an upgrade does not roll their Sentinels. The Sentinel WorkloadMonitor reports the preset the Sentinels actually use. Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> Assisted-by: LLM |
||
|
Yasar Rahman
|
f6f879cf38 |
fix(postgres)!: refuse plaintext sessions on external instances
CloudNativePG offers TLS but its built-in last pg_hba rule, host all all all, also accepts plaintext, so a client outside the cluster that connects with sslmode=disable gets an unencrypted session. tls.enabled only decides whether the external hostname goes into the certificate and cannot prevent this. When external is true the chart now renders spec.postgresql.pg_hba with hostssl all all all scram-sha-256 followed by host all all all reject. CloudNativePG places user rules after its fixed rules and before its default rule, and the first match wins, so only TLS sessions are accepted. The pod network cannot be told apart from an external client, so in-cluster clients are held to the same rule; libpq's default sslmode=prefer already negotiates TLS. The new tri-state tls.required follows external when omitted. Setting it to false keeps accepting plaintext on an external instance and true requires TLS on an in-cluster one. The hostssl rule uses the same password method as CloudNativePG's own default rule (md5 before PostgreSQL 14, scram-sha-256 from 14), so existing md5 verifiers keep working on PostgreSQL 13. Rendering fails when postgresql.parameters.ssl is off while TLS is required, since hostssl would never match and every remote session would be refused. BREAKING CHANGE: external clients that connect without TLS are refused after the upgrade. Set tls.required to false to keep accepting them. Assisted-by: LLM Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> |
||
|
Aleksei Sviridkin
|
f4bde9e50d |
fix(backups): delete the original PVC of an in-place VM restore without keepOriginalPVC (#4672)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
An in-place VMInstance restore with `keepOriginalPVC: false` reports
`Succeeded` without restoring the disks, and leaves every disk it
touched unable to come back. `prepareForRestore` deletes each DataVolume
with Orphan propagation and leaves the PVC in place, on the assumption,
stated in the `deleteDataVolume` comment, that Velero overwrites it.
Velero does not overwrite a PVC. Its CSI restore action returns early
for a claim that already exists, so no DataDownload is ever created, and
with `existingResourcePolicy: update` the merge patch it attempts
instead is rejected because a bound claim's spec is immutable. All that
reaches the claim is the labels-only fallback patch, which also drops
the `cdi.kubevirt.io/allowClaimAdoption` annotation the resource
modifier added.
The claim left behind then blocks the disk for good. Orphan propagation
has removed its owner reference to the DataVolume, and it lacks the
`cdi.kubevirt.io/storage.populatedFor` annotation that
kubevirt-velero-plugin writes on the backed-up copy, so when the
HelmRelease resumes, the CDI webhook refuses the DataVolume it creates
with `Destination PVC <ns>/<name> already exists`, on every reconcile.
Deleting the disk through the API leaves the PVC behind, and a new disk
of the same name fails the same way.
Observed on a v1.6.3 cluster, for a VMInstance restore with
`keepOriginalPVC: false` whose disk was attached to the VM. The Velero
log for the restore, trimmed:
```text
level=info msg="Retrieved per-restore resource modifiers from configmap cozy-velero/restore-modifiers-..."
level=warning msg="PVC already exists and ExistingVolumeDataPolicy is not in-place restore. Skip restore this PVC." Action=PVCCSIRestoreItemAction PVC=tenant-datalab/vm-disk-e2edisk
level=info msg="Applying resource modifier patch on tenant-datalab/vm-disk-e2edisk"
level=warning msg="patch attempt failed for /v1, Kind=PersistentVolumeClaim tenant-datalab/vm-disk-e2edisk: PersistentVolumeClaim \"vm-disk-e2edisk\" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests and volumeAttributesClassName for bound claims"
level=info msg="backup/restore labels successfully updated for PersistentVolumeClaim tenant-datalab/vm-disk-e2edisk"
```
`velero restore describe --details` lists the PVC as `(failed)` and the
Restore is `Completed` with two warnings. Afterwards the claim had its
original creation timestamp and UID, no owner reference, no
`allowClaimAdoption`, no `populatedFor`, and the disk's HelmRelease was
`Ready=False` with the webhook error above. That cluster runs a Velero
build from `main`, whose wording is quoted above; the released v1.17.1
and v1.18.3 return early at the same point with `PVC already exists.
Skip restore this PVC.` (`pkg/restore/actions/csi/pvc_action.go`).
The fix deletes the PVC when `keepOriginalPVC` is false, only once its
DataVolume is confirmed gone so that CDI cannot recreate it, and holds
the Velero Restore until the claim has actually disappeared. A claim
still mounted stays `Terminating` behind `kubernetes.io/pvc-protection`,
and creating the Restore then would hit the same existing-claim path.
Velero then creates the PVC from the backup, the data mover fills it,
and the recreated DataVolume adopts it through `populatedFor`. A failed
deletion now fails the RestoreJob instead of letting it report
`Succeeded` over the old data. Before
|
||
|
Aleksei Sviridkin
|
c290b75923 |
docs: link the documentation of the current release (#4761)
## What this PR does Points the documentation links in the README and GOVERNANCE.md at the current release. The unversioned `/docs/...` paths on cozystack.io are Hugo aliases owned by the v0 docs. Each one is a meta-refresh to the v0 page, so these links sent readers to the v0 documentation, a major version behind the current release: | Link | Went to | Now | |---|---|---| | README `/docs/guides/use-cases/public-cloud/` | `/docs/v0/guides/use-cases/public-cloud/` | `/docs/v1.6/guides/use-cases/public-cloud/` | | README `/docs/guides/use-cases/private-cloud/` | `/docs/v0/guides/use-cases/private-cloud/` | `/docs/v1.6/guides/use-cases/private-cloud/` | | README `/docs/guides/use-cases/kubernetes-distribution/` | `/docs/v0/guides/use-cases/kubernetes-distribution/` | `/docs/v1.6/guides/use-cases/kubernetes-distribution/` | | README `/docs/getting-started/` | `/docs/v0/getting-started/` | `/docs/v1.6/getting-started/` | | README `/docs/operations/troubleshooting/` | `/docs/v0/operations/troubleshooting/` | `/docs/v1.6/operations/troubleshooting/` | | GOVERNANCE.md `/docs/operations/configuration/telemetry/` | `/docs/v0/operations/configuration/telemetry/` | `/docs/v1.6/operations/configuration/telemetry/` | In the v1 docs the kubernetes-distribution path holds Build Your Own Platform (BYOP), installing only the packages you need onto an existing cluster that already has networking and storage, rather than the v0 "Kubernetes distribution for bare metal" page. The README entry is rewritten to match: "Using Cozystack to build your own platform", with "You can install only the Cozystack components you need on an existing Kubernetes cluster". The full version path follows the other versioned docs links in the repository, which all name `/docs/v1.6/`. The short `/docs/v1/...` path is not an option: only the site's 404 page redirects it, in JavaScript. All six new URLs return 200 without a redirect, and the same pages exist under the website's `content/en/docs/next/`, so the next release keeps them. The same alias problem in the website's own `llms.txt` is fixed in cozystack/website#737. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note docs: link the v1.6 documentation from the README and GOVERNANCE.md instead of the v0 pages the unversioned cozystack.io paths redirect to, and describe the kubernetes-distribution use case as Build Your Own Platform, which is what that page documents in v1. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated README use-case and documentation links to point to the Cozystack v1.6 documentation, including the Telemetry link in the governance guide. * Renamed the Kubernetes-distribution use case to “Build your own platform” and clarified that selected Cozystack components can be installed on an existing Kubernetes cluster. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
b1192bfcbc |
fix(backups): create no driver object for a job that already finished (#4673)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
The BackupJob and RestoreJob reconcilers decide a job is still active
from the informer cache, and the cache can trail their own writes. A
driver's first pass stamps `StartedAt`, which requeues the job, and may
then fail the job in the same pass on a transient error, for instance a
503 from the aggregated `apps.cozystack.io` API while
`createVeleroBackup` reads the application. The requeued pass can start
before the informer has seen `Failed`: it retries the step that failed
and, once the error has cleared, creates the driver object anyway. Its
own status write then conflicts on the stale resourceVersion, the next
pass sees `Failed` and skips the job, and a Velero Backup is left
running that no `Backup` artifact will reference and nothing tracks.
Reproduced against a real apiserver (envtest 1.34) with the real manager
cache and the reconciler wired through `SetupWithManager`, nothing else
writing the BackupJobs. The only fault injected is one 503 on the first
GET of each application through `apps.cozystack.io`. Out of 40
BackupJobs, every one ends `Failed` with `failed to create Velero
Backup: the server is currently unable to handle the request`, and on
`main` (
|
||
|
Aleksei Sviridkin
|
a2e10312de |
fix(platform): stop treating a failed ConfigMap read as a fresh cluster (#4792)
## What this PR does A single failed API read at the start of the platform migration Job could skip every pending migration and still end green. `run-migrations.sh` checked whether the `cozystack-version` ConfigMap exists with a silenced `kubectl get` and looked only at the exit code. A timeout, a dropped connection or an apiserver restart during the upgrade all counted as "no ConfigMap", so the script stamped the target version and exited 0. The hook Job is rendered only when that ConfigMap already exists, so in practice an error was the only way into this branch. Now the script creates the ConfigMap only when it is really missing. The check uses `--ignore-not-found --output name`: kubectl drops NotFound and nothing else, so empty output means a genuine NotFound. Any other error keeps a non-zero exit, `set -e` aborts the script, and the Job retries the pod. The rest of the script already works this way: the read of the stamped version aborts on error, and the stamp runs under `pipefail`. The tests cover both branches. A new case in `hack/run-migrations.bats` makes the existence check fail with a connection error and expects a non-zero exit, no migration run and the stamp left where it was. It fails against the old check. The fake kubectl returns empty output for a missing ConfigMap only when `--ignore-not-found` is passed, as real kubectl does, so dropping the flag breaks the existing "no ConfigMap" case. No migration is added, so `targetVersion` stays. The script ships in the migrations image, and the release build restamps that image's digest as usual. Several older migrations have the same problem in a smaller form. They check for an object with `kubectl get ... >/dev/null 2>&1` and skip their own work when the check fails, for example migrations 31, 32, 36, 48, 50 and 51. A transient error there skips one migration's work, not all of them. Editing migrations that already shipped has its own risk, so I'd track that separately. Closes #4579 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(platform): a transient API error while the platform migration Job checks for the cozystack-version ConfigMap now fails the pod so the Job retries, instead of stamping the target version and silently skipping every pending migration. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Migration checks now distinguish a missing version record from an error while checking for it. When the check fails, the pod stops instead of treating the record as missing and proceeding with migrations. * Added coverage for TLS-handshake timeouts during the check, confirming that migrations do not run and the stamped version remains unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Marian Koreniuk
|
c61fcf0ee4 |
docs(governance): link the Telemetry page of the current release
The unversioned /docs/operations/configuration/telemetry/ path is an alias owned by the v0 docs, so GOVERNANCE.md sent readers to the v0 Telemetry page. It now names the v1.6 page, like the Licenses link a few lines below. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Marian Koreniuk
|
b39f623338 |
docs(readme): describe the use case the kubernetes-distribution page documents now
In the v1 docs the kubernetes-distribution use-case path holds Build Your Own Platform: installing only the packages you need onto an existing cluster that already has networking and storage. The README entry still described the v0 page, Cozystack as a Kubernetes distribution for bare metal, so its label and description no longer matched the page it links to. Both now describe BYOP. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Marian Koreniuk
|
59af194b73 |
docs(readme): link the documentation of the current release
The unversioned /docs/... paths on cozystack.io are aliases owned by the v0 docs, so the use-case, getting-started and troubleshooting links sent readers to the v0 documentation, a major version behind the current release. They now name the v1.6 pages, in line with the other versioned links in the repository. Assisted-by: LLM Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> |
||
|
Aleksei Sviridkin
|
be2387323b |
feat(gateway)!: reserve a passthrough listener's name by the spec alone (#4638)
## What this PR does This keeps the TLS-passthrough services on port 443 of the tenant Gateway and makes the passthrough rules hold on Cilium 1.20.2. It is stacked on #3343 and has to land with it (stack #4717). On 1.20, upstream Cilium marks an HTTPS-terminate listener and a TLS-passthrough listener on one port `Conflicted` when their hostnames overlap, and drops both from Envoy. That took `*.<apex>` and the `api`, `vm-exportproxy` and `cdi-uploadproxy` passthrough listeners down together on every tenant Gateway in `dns01` and `existingSecret` mode (#3912). #3343 now ships a patched operator that conflicts the pair only when the hostnames are equal, so the layout stays as it is on main: one Gateway, one address, everything on 443. What remains is the equal-hostname case and the cleanup that 1.20 allows. Under `http01`, a name a passthrough listener answers is now reserved by the spec alone. Before, the controller dropped the terminate listener for such a name only once a TLSRoute was attached to the passthrough listener and its backend Service resolved, which copied how Cilium builds filter chains. Without a working TLSRoute, an HTTPRoute on `api.<apex>` kept its terminate listener next to `tls-api`, and on 1.20 that equal-hostname pair takes both listeners down. Now such an HTTPRoute gets no listener and no certificate, TLSRoute or not. If all its hostnames are reserved it reads `Accepted=False` with `NoMatchingListenerHostname`. If it also claims a free hostname it stays `Accepted` and is served there. The backend lookup, its Service and ReferenceGrant watches and their RBAC rules are gone. Other changes: - `tlsPassthroughListeners` is accepted under `dns01` and `existingSecret`. The old refusal relied on 1.19.5 serving all ports from one Envoy listener; 1.20 splits ports whose SNI overlaps. - A TenantGateway reads `Ready=False` with reason `GatewayClassUnsupported` when its GatewayClass lists features in `status.supportedFeatures` but not `TLSRoute`, while it renders passthrough listeners. A class that lists nothing is not judged. Cilium 1.20.2 lists `TLSRoute`. - Comments, the README and one error message no longer describe Cilium 1.19.5 as the shipped release. - The gateway chart writes `tlsPassthroughServices` on the publishing tenant only, behind the same `gateway.enabled` switch and `publishing.exposedServices` filter as the platform TLSRoutes. Before, every tenant Gateway got the three passthrough listeners with nothing routed to them, and the reservation above would have taken those names from a child tenant's own HTTPRoutes. The uniform `allowedRoutes.kinds` on port 443 from #3087 stays. The Cilium bug behind it is fixed since 1.19.6, but several tests pin it, so removing it is a separate cleanup. ### Upgrade notes Only the publishing tenant keeps the `api`, `vm-exportproxy` and `cdi-uploadproxy` passthrough listeners. That is `tenant-root` unless `publishing.ingressName` says otherwise, and only for names in `publishing.exposedServices` while `gateway.enabled` is on. Other tenants lose these listeners on upgrade. Nothing shipped was attached to them, so a child tenant's own HTTPRoute on `api.<child-apex>` keeps its listener and certificate. Two narrow cases do change. On the publishing tenant under `http01`, an HTTPRoute on an exposed passthrough name loses its listener and certificate when the platform TLSRoute for that name forwards nowhere (the owning package is not installed, or its backends do not resolve). A TLSRoute attached by hand to a child tenant's `tls-*` listener loses its parent. To keep that listener, merge-patch `tlsPassthroughServices` into the tenant's `TenantGateway` after the upgrade. From then on the chart does not render the key there, and later upgrades leave the patch alone. `dns01` and `existingSecret` do not change: the wildcard listener still covers such a name, and the passthrough listener answers it by its exact name. Tested locally: Go tests for the controller, `go vet`, helm-unittest for `extra/gateway` and `cozystack-controller`, `make generate` and `make manifests` with no diff. The gateway e2e scenario checks that a child tenant renders no passthrough listener. It then declares `tls-api` and a passthrough listener on a native port on that tenant, checks that the tenant stays Ready with no route behind either, and checks the reservation against `tls-api`. Unit tests catch a regression in either change. In e2e, dropping the reservation turns the gateway suite red, and dropping the chart scoping does not, because CI never sets `gateway.enabled` (#4846). The controller was not run on a live cluster; the operator patch it depends on was, see #3343. ### Downstream repositories - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/issues/725 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(gateway)!: the tenant Gateway renders the api, vm-exportproxy and cdi-uploadproxy passthrough listeners only on the publishing tenant, for the names publishing.exposedServices lists while gateway.enabled is on. Under certMode http01 a name a TLS-passthrough listener answers no longer gets an HTTPS listener on 443, with or without a TLSRoute behind it. On the publishing tenant, an HTTPRoute on such a name whose TLSRoute forwards nowhere loses its listener and certificate. A TLSRoute attached by hand to a tls-* listener of another tenant loses its parent. An HTTPRoute with some reserved and some free hostnames used to be rejected whole; it is now Accepted and its free hostnames get a listener and a certificate. tlsPassthroughListeners is accepted under dns01 and existingSecret. A TenantGateway whose GatewayClass does not list TLSRoute in its supported features reads Ready=False while it renders passthrough listeners. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * TLS passthrough listeners are supported with HTTP-01, DNS-01, and existing-secret certificate modes. Edge-terminated Gateways remain unsupported. * Declaring a passthrough hostname reserves it from HTTPS termination and certificate creation, even when no route is attached. * Passthrough listeners are rendered only for exposed services on the enabled publishing Gateway. * **Bug Fixes** * Gateways that declare passthrough listeners but explicitly lack TLSRoute support are now reported as not ready. * Routes targeting a hostname served by a passthrough listener on a different Gateway are correctly withdrawn. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d9689bd6e7 |
fix(gateway): stop describing Cilium 1.19.5 as the shipped release
With Cilium at 1.20.2 the tenant gateway comments, README and one status message still explained behaviour as that of "the pin" at 1.19.5. Several of those claims no longer hold: a Gateway with two routed passthrough ports now gets an Envoy listener per port, Cilium judges a route's kinds against the listener the route names, and the shipped operator accepts an HTTPS and a passthrough listener on one port when the passthrough hostname is strictly narrower. Behaviour that is unchanged is re-anchored on the 1.20.2 source, behaviour that is gone is stated as history, and the refusal of one hostname on two passthrough ports no longer blames a Cilium limitation the shipped release does not have. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
ad3f15d4f3 |
feat(gateway)!: reserve a passthrough listener's name by the spec alone
The TenantGateway controller withheld the HTTPS-terminate listener for a name a passthrough listener answers only once a TLSRoute was attached there and resolved a backend, mirroring how Cilium builds its filter chains. That kept a served endpoint up on Cilium 1.19.5, but it tied every route's status to one implementation's translator and needed Service and ReferenceGrant watches to land the deferred decision. The shipped operator accepts a wildcard terminate listener beside a narrower passthrough listener on 443 and keeps conflicting every other overlap. A terminate listener for the passthrough name itself is not narrower, so on 443 the rule now protects the dataplane outright; on a native port Gateway API would serve the pair, and the rule holds as a policy that keeps a platform name with its passthrough listener. Reading it off the spec makes the outcome the same on every implementation. Under http01 an HTTPRoute claiming such a name gets no listener and no certificate whether or not a TLSRoute is attached. A route whose every hostname is reserved reads Accepted=False with NoMatchingListenerHostname; one that also claims a free hostname stays Accepted and is served there, as Gateway API accepts a route whose hostnames intersect at least one listener. dns01 and existingSecret are unchanged: their wildcard listener still covers such a name, and the passthrough listener answers it by its exact server name. The chart writes the passthrough services on the publishing tenant alone, so the names reserved by default are the ones the platform publishes there through its own TLSRoutes. The shape an upgrade changes is an HTTPRoute on such a name whose platform TLSRoute forwards nowhere, because it never attached (the package owning the service is not installed) or because none of its backends resolve: that route kept a terminate listener and now gets none. The backend resolution, its watches and the RBAC rules that served them go away. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6144805bea |
fix(mariadb): mount the chart's my.cnf under a .cnf name (#4843)
## What this PR does The MariaDB chart's my.cnf has not been read by the server since the mariadb-operator bump to v0.28.1. Up to v0.0.27 the operator mounted the referenced ConfigMap key as `/etc/mysql/conf.d/my.cnf` whatever the key was called. Since v0.0.28 the file takes the key's own name, so the chart's key `config` became `/etc/mysql/conf.d/config`. The server's `!includedir` reads only `*.cnf` files, so it skipped it, and every instance has been running on stock MariaDB defaults. On the default `t1.nano` preset that means a 128M buffer pool inside a 128Mi memory limit. This renames the key to `my.cnf` in the ConfigMap and in the MariaDB CR. The file content is unchanged. I started all four shipped versions (10.6.24, 10.11.15, 11.4.9, 11.8.5) with the rendered file under a 128m memory limit, and each one came up with the chart's values. The MySQL-only options in the file log a warning and are otherwise ignored. They and the deprecated `innodb_flush_method` are tracked in #4842. Tests: a helm-unittest suite pins the key on both sides, and the mariadb chainsaw suite gets a step that compares `@@innodb_buffer_pool_size` on the running server with the value in the rendered my.cnf. This changes behaviour on every existing instance, see the release note. Each instance rolls its pods once because the projected volume in the pod template changes: replicas first, then the primary with a switchover. After that the server runs with every setting in the chart's file. Instances created before the operator bump (May 2024) ran with these settings before. Newer ones never did. Closes #4836 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(mariadb): the chart's my.cnf is applied again. Since mariadb-operator v0.28.1 it was mounted under a name the server ignores, so MariaDB instances ran on stock defaults. Every existing instance rolls its pods once on upgrade (replicas first, then the primary with a switchover) and then runs with the chart's settings. That changes behaviour: buffer pool, redo log, connection limits and caches are sized from the instance's resources; sql_mode becomes NO_ENGINE_SUBSTITUTION, so strict mode is off and invalid values are truncated with a warning instead of rejected; innodb_flush_log_at_trx_commit=2 can lose up to about a second of committed transactions if the node crashes; innodb_fast_shutdown=0 makes every pod shutdown slower. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * MariaDB configuration is now stored and read using the `my.cnf` key. A verification check confirms the running server applies the configured buffer-pool size. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
4346a30f4b |
ci(e2e): run nightly and release-tag e2e on the container lane (#4828)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Nightly and the release-tag e2e (`e2e-tag.yaml`, the rc gate) now run on the Talos-container substrate the PR lanes moved to in #4020. This is the interim step from #4642, not the whole issue, so it does not close it. On QEMU the nightly full suite has been cancelled at its 215-minute timeout on every night since 2026-10-01 except 2026-10-07. From 2026-10-02 on, kubernetes-latest and kubernetes-previous failed their tenant node-join deadline almost every night, each failure took about 35 minutes, and the suites after them alphabetically never ran. In the 18 PR runs since 2026-10-03 that I checked and that ran them, they passed every time, at 7 to 10 minutes each. A nightly dispatched from this branch (run 37934155231) passed all 62 tests in 193 minutes. The kubernetes suites took 9 to 11 minutes there. Changes: - `nightly.yaml` and `e2e-tag.yaml` get the same host module step, `prepare-env-container`, endpoint and storage-class env and two-part teardown as `pull-requests.yaml`. Nightly's e2e no longer waits for the nocloud disk, which it does not boot. The disk is still built and published. - `e2e-tag.yaml` still requires the release to carry its disk but no longer downloads it. A dispatch against a tag cut before the container lane (v1.6.3 and older, v1.7.0-alpha.1) fails early and says to dispatch with `--ref <tag>`. When `tags.yaml` calls it, it runs the copy from the tag's own commit, so the guard does not fire there. - The comment above `timeout-minutes: 215` in all four e2e workflows claimed the budget covers two failing tenant Kubernetes suites. It does not, and the comment now says so. The value is unchanged. - `platform-packages_test.bats` now checks the container-lane env on all four workflows, and `collect-images_test.bats` covers the new guard. What loses coverage: DRBD and the `replicated` StorageClass (Immediate binding, tenant StorageClass propagation, RWX), the Cozystack Talos node image with its extensions, and the stock 600M CDI importer limit. None of them can run on containers, so no CI lane tests them until #4642 gets a lane with a kernel per node. The docs and comments that pointed at nightly and rc as that coverage now say this. Release lines keep their own `e2e-tag.yaml`, so rc tags on release-1.6 stay on QEMU until this is backported. That is why `kind/backport` is on this PR. Leftover QEMU-only text and diagnostics are collected in #4825. A dispatch of `e2e-tag.yaml` from this branch against `v1.7.0-alpha.3` is running as run 37958852914. I'll post the result here. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ci(e2e): nightly and release-tag e2e run on Talos containers instead of QEMU guests. DRBD, the replicated StorageClass and the Cozystack Talos node image are no longer covered by any e2e lane. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **CI and Testing** * Release-tag and nightly end-to-end tests now run against container-based environments rather than booting disk images. * Release-tag checks still verify that the published release includes its disk image. * CI no longer covers QEMU-based DRBD scenarios or certain storage and Talos extension behaviors; these coverage gaps are documented. * **Documentation** * Updated E2E and release guidance to reflect the container-based test coverage and its limitations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
8ab6d4948f |
test(mariadb): assert the server runs with the chart's my.cnf
The chart's unit tests can pin the ConfigMap key and the CR reference, but not what mariadb-operator does with them or which files the server then reads from /etc/mysql/conf.d. Compare innodb_buffer_pool_size as the live server reports it against the value in the rendered my.cnf, which differs from the server default on this fixture's preset. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d76bd1f263 |
fix(mariadb): mount the chart's my.cnf under a .cnf name
mariadb-operator up to v0.0.27 mounted the referenced ConfigMap key as
/etc/mysql/conf.d/my.cnf whatever the key was called. Since v0.0.28 the
file takes the key's own name, so the chart's key "config" became
/etc/mysql/conf.d/config, and the server's !includedir reads only *.cnf
files from that directory. Every setting in the chart's my.cnf has been
ignored since that operator bump: instances run on stock defaults,
including a 128M InnoDB buffer pool inside the 128Mi default limit.
Renaming the key restores the file as intended. Existing instances roll
their pods once, since the projected volume in the pod template changes,
and come back with the chart's settings in effect, including sql-mode
and innodb_flush_log_at_trx_commit, not only the resource sizing.
Fixes:
|
||
|
Yasar Rahman
|
83aaa65ed1 |
fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2
velero-plugin-for-aws v1.14.1 sets x-amz-tagging on every PutObject,
empty when the BackupStorageLocation configures no tags. Cloudflare R2
answers that with 501 NotImplemented ("Header 'x-amz-tagging' with value
'' not implemented"), so Velero cannot write backup metadata and every
backup to an R2 location fails, the cozy-default location included when
the platform bucket is on R2.
The plugin sends the header only when tags are configured since v1.14.3
(velero-plugin-for-aws#309, a backport of #299). v1.14.4 is the current
patch release of the 1.14 line, which upstream lists as the one
compatible with Velero 1.18, and adds only CVE dependency bumps on top
of v1.14.3.
Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com>
Assisted-by: LLM
|
||
|
Yasar Rahman
|
8bfd4e5f5e |
fix(redis): re-send the Sentinel auth-pass when the operator resets a Sentinel
A Sentinel that restarts on the same pod IP, as after a node reboot, comes back with the operator's initial sentinel.conf, which monitors a 127.0.0.1 placeholder and carries no auth-pass. Its peers still know that address and keep publishing hello messages to it, and once a failover has raised the config epoch the restarted Sentinel adopts the current master address from them before the operator looks at it. The operator then finds the right master address and skips NewSentinelMonitor, the only call that sends SENTINEL SET auth-pass. The master answers that Sentinel with NOAUTH, it reports the master as s_down, and the operator's resulting SENTINEL RESET on every check keeps the empty auth-pass, so the instance runs with one Sentinel fewer until someone deletes the pod. RestoreSentinel now sends the RedisFailover's password with SENTINEL SET auth-pass after the reset. It runs only when the Sentinel already looks unhealthy, and the reset has just dropped its links, so a Sentinel that had the password loses nothing. Without auth there is nothing to send. The patch file sorts after tls.diff because it is written on top of it; the Dockerfile applies patches/*.diff in glob order. Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com> Assisted-by: LLM |
||
|
Aleksei Sviridkin
|
f61e13fdef |
fix(cilium): size the agent memory limit for its BPF maps (#4803)
## What this PR does This raises the cilium-agent memory limit from 4Gi to 24Gi. The 4Gi limit from #3495 has no room for the agent's BPF maps. On nodes with roughly a terabyte of RAM or more, the agent is OOM-killed on every start after a reboot, and the node loses pod networking. The maps grow with node RAM. Cilium sizes its CT, NAT, neighbor and socket rev-NAT maps as a share of total RAM (`bpf-map-dynamic-size-ratio`, 0.0025 by default), and caps each map at 16Mi entries. These are LRU hash maps, so the kernel preallocates them in full. Since Linux 5.11 that memory is charged to the memcg of the process that creates the maps, which is the agent container. This is also why the upgrade itself works. A map reused from bpffs stays charged to the cgroup that created it. The failure shows up on the first reboot after the upgrade, when the maps are recreated inside the limited cgroup. Policy maps don't belong on that list. The issue and the upstream chart's configmap comment both say the ratio sizes them, but at this version they are LPM tries sized by `bpf.policyMapMax`, and they allocate per entry. One DaemonSet serves every node, so the limit can't follow node RAM. The map sizes are capped, though, so a fixed limit can cover any node. With every map at 16Mi entries for both IPv4 and IPv6, the maps take just under 20Gi of kernel memory. Add the 4Gi the agent already had and you get 24Gi. The request stays at 256Mi. The limit doesn't reserve memory, and the pod cgroup still gets `memory.max`, which is what #3495 was after. What it costs is the leak backstop. The agent is now stopped at 24Gi minus its own map memory, so on most nodes a leak in its heap is no longer caught before the node feels it. If someone prefers a tighter backstop over IPv6 headroom, they can lower `cilium.resources.limits.memory` in the networking component values. About 13Gi still covers the IPv4-only map ceiling. I didn't pin the map sizes (`bpf.ctTcpMax`, `bpf.natMax` and so on). No single value works for every node. A value small enough for a 16Gi node cuts the conntrack capacity of a 1.5TiB node many times over. A pinned size also differs from what almost every node above ~14Gi of RAM runs today, so the agent would recreate those maps on upgrade and drop their conntrack and NAT state. A lower ratio has the same problem and doesn't lower the ceiling. The new test keeps the ratio at its default and the sizes unpinned, so changing either has to be deliberate. The figures are computed, not measured. I read the sizing code at cilium 1.19.5 (`pkg/option/config.go`), and 1.20.2, which #3343 moves to, has the same code. Memory comes from that formula plus the kernel hash map layout: a 48-byte element header, the key and value rounded up to 8 bytes, and a 16-byte bucket per power of two of `max_entries`. By that math the IPv4 maps alone pass 4Gi at roughly a terabyte of RAM. Bucket rounding moves the exact point, so I wouldn't read more precision into it. For the 1.5TiB node in the report, the issue's 3.9GiB is the agent's own budget (0.25% of RAM), and the kernel's actual charge comes out higher. Follow-ups, not in this PR: - Operators who want smaller maps on large nodes can size them per node with a `CiliumNodeConfig` selected by node label. This needs a documentation note. - The reporter's `bpftool` measurement of the real map memory is still pending. That is the measured check for the numbers above. The cilium helm-unittest suite passes on this branch, and also with this change applied over the 1.20.2 chart from #3343. Each new assertion fails when its value changes: the limit, the CT, NAT and neighbor sizes, the socket rev-NAT size set through `extraConfig`, and the ratio. Closes #4800 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff changes only `packages/system/cilium/values.yaml` and its test. None of the trigger-map entries apply: no app, schema, platform value, variant, release asset or tooling changes, and no website page documents the agent limit. ### Release note ```release-note fix(cilium): raise the cilium-agent memory limit from 4Gi to 24Gi so it covers the agent's BPF maps, which cilium sizes from node RAM. On nodes with around a terabyte of RAM or more, the 4Gi limit got the agent OOM-killed on every start after a reboot, leaving the node without pod networking. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Increased the Cilium agent memory limit to 24 GiB. * Updated validation for BPF map sizing: checks the dynamic size ratio and confirms that specific map-size settings are absent. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
52f09d1aed |
fix(gateway)!: render passthrough services only where platform routes attach
The chart wrote the tlsPassthroughServices default onto every non-edge tenant's TenantGateway, so every tenant Gateway carried tls-api, tls-vm-exportproxy and tls-cdi-uploadproxy. The TLSRoutes that serve those names are platform routes: the packages owning the services render one per name in publishing.exposedServices while gateway.enabled is on, parented to the publishing tenant's Gateway alone. Everywhere else no shipped route attached to the listeners, and a passthrough listener holds its name whether or not anything is routed through it. The chart now writes the field only on the publishing tenant, with the same gateway.enabled switch and the same exposedServices filter those routes use, so a listener exists exactly where a platform route can attach. A TLSRoute attached by hand to one of those listeners on another tenant loses its parent. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
bdca165f4c |
feat(gateway): report a GatewayClass that does not serve TLSRoute
A GatewayClass lists the features it serves in status.supportedFeatures, and TLS passthrough is TLSRoute. On a class that leaves it out the passthrough listeners are accepted and serve nothing, so the TenantGateway now reads Ready=False/GatewayClassUnsupported naming the class instead of Ready. A class that lists no features at all is not judged: implementations predating the field leave it empty. The class writes that list after it is accepted and may change it when its controller is upgraded, so a GatewayClass event requeues the TenantGateways on it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
d624049a3a |
feat(gateway): allow tlsPassthroughListeners under dns01 and existingSecret
The refusal existed because Cilium 1.19.5 served every Gateway port from one Envoy listener matched by SNI alone, so the "*.<apex>" terminate listener of the wildcard modes and a passthrough listener under the apex could reach each other's backend. The shipped Cilium 1.20.2 gives each port its own Envoy listener when SNI overlaps across ports, and Gateway API treats listeners on different ports as compatible, so the pair is served as declared. edge still refuses the field: it renders no TLS listener for an entry to bind to. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e1319b5a6c |
docs(cilium): say the SNAT backport is dropped by hand
The patch header cited a commit of cilium/cilium#49232, an open PR whose branch is force-pushed, so the commit is reachable from no ref. Cite the PR alone. It also read as if the build would flag the patch once upstream carries the fix. It does not: a fix that lands in another place leaves this patch applying cleanly as a second early return, so the drop is a manual step. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
e816522366 |
chore(renovate): leave the cilium-operator image pins to make update
The operator rebuild compiles one Cilium release with the Go toolchain upstream's builder image pins at that tag, and make update re-pins the builder line from there, along with the operator-generic base, its digest and the source archive checksum. A Renovate bump of that line, by tag or by digest, would build the release with a toolchain upstream never shipped it with, and the next make update would revert it without notice. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
1bfa64f83a |
fix(cilium): keep per-type xDS while the 1.20 upgrade rolls
The 1.20 chart renders envoy-xds-mode "ads" and an adsConfig bootstrap for cilium-envoy unless xdsMode or upgradeCompatibility says otherwise. A 1.19 agent never registers the aggregated discovery service, and a 1.20 agent in ADS mode registers nothing else. The agent and envoy DaemonSets roll independently, so on every node where their versions differ envoy gets no configuration until both pods are replaced, and Gateway traffic stops on all nodes at once during the upgrade. Pin xdsMode to split, the per-type xDS both versions serve and the agent's own default when the key is absent. Moving to ADS is a separate step once every node runs 1.20. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5b92ca3fc8 |
fix(cilium): serve HTTPS and passthrough on one port from one listener
When a Gateway needs per-port Envoy listeners, the operator emits one listener per HTTPS port and, with two or more passthrough ports, one per passthrough port, all named listener-<port>. A port carrying both gets two listeners with one name, and the agent skips the whole CiliumEnvoyConfig, so every listener of the Gateway stops serving while its status stays green. With one passthrough port the port is instead mapped to two listeners in the CEC services. A tenant Gateway with HTTPS and passthrough on 443 plus a passthrough listener on a native port reaches the first case. The patch puts the passthrough filter chains of such a port on its HTTPS listener and leaves the port out of the passthrough listeners and service mappings. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |