Commit Graph
2798 Commits
Author SHA1 Message Date
Mattia Eleuteri c368c48fd8 fix(controller, platform): give unbounded pool members the root's LimitRange
The tenant chart renders the tenant-range-limits LimitRange only next to
the tenant-quota of a tenant that declares resourceQuotas. Since quotas
became hierarchical, an unbounded sub-tenant shares its parent's pool
and gets a controller-owned tenant-quota-allocated ResourceQuota, but
still no LimitRange. A ResourceQuota on limits.cpu or limits.memory
makes admission reject any container that omits that limit, and with
no LimitRange nothing fills it in. Pods that ran before the quota was
introduced fail as soon as they are recreated; the barman-cloud plugin
sidecar, which sets no CPU limit, takes every backed-up postgres
instance in such a namespace down on its next restart.

The controller already knows which namespaces it clamps, so it copies
the pool root's chart LimitRange into each unbounded member, under its
own name so that Helm can still create the chart object if the tenant
later declares a quota of its own, and removes the copy once the
namespace leaves the pool.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-11-10 09:03:55 +01:00
Aleksei Sviridkin da7f2c2f50 fix(tests): keep two unit tests green after the release image restamp (#5180)
## What this PR does

Same change as #5179, on `main`. Both tests pass here today, because
`main` still carries the pre-release image pins; they fail on every
release line as soon as the first rc is cut, since the `Prepare release`
commit rewrites `packages/*/values.yaml` and two unit tests read exactly
the fields it rewrites. On `release-1.7` this has kept `Unit &
controller tests` red on every PR since v1.7.0-rc.3
([job](https://github.com/cozystack/cozystack/actions/runs/41162009892/job/124421449910)
on #5172), and `release-1.8` would repeat it.

| test | reads | before the release build | after (release-1.7 today) |
|---|---|---|---|
| `hack/cni-plugins-staging-contract.bats` "the kube-ovn release the CNI
plugin pin was checked against is the one vendored" |
`.global.images.kubeovn.tag` in `packages/system/kubeovn/values.yaml` |
`v1.15.26@sha256:…` | `v1.7.0@sha256:…` |
| `packages/system/monitoring/tests/vtraces_collector_test.yaml` "pins
the collector image through the cozy-lib.image helper" |
`tracingCollector.image` |
`docker.io/otel/opentelemetry-collector-contrib:0.136.0@sha256:…` |
`ghcr.io/cozystack/cozystack/otelcol-tracing:v0.136.0-cozystack.0@sha256:…`
|

Changes:

- The bats test takes the vendored kube-ovn release from `appVersion` in
`packages/system/kubeovn/charts/kube-ovn/Chart.yaml`. That is the value
`packages/system/kubeovn/Makefile` builds the image from, and a release
build leaves it alone. The hint printed on a mismatch names the new
file.
- The helm unittest case no longer names the image at all. It sets
`_cluster.images-registry` and asserts that the registry lands in front
of the rendered image and that the image stays digest-pinned (the
`grafana_upstream_image_plugins_test.yaml` pattern). Removing the
`cozy-lib.image` call from the template makes it fail; the old regex did
not.

Same single commit as #5179; the only difference is the unrelated `DOC=`
line next to the edit, kept as `main` has it.

Verified locally on this branch: `bats
hack/cni-plugins-staging-contract.bats` 10/10, `helm unittest
packages/system/monitoring -f tests/vtraces_collector_test.yaml` 28/28.

Refs #5178 (the release-1.7 report), #5179, #5172.

### Downstream repositories

- [x] No downstream repository is affected by this change

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Tests**
* Version validation now compares the checked version with the chart’s
application version and reports missing or mismatched versions with the
chart path.
* Monitoring image checks now verify use of the configured mirror
registry and require a 64-character lowercase SHA-256 digest, without
requiring a specific image name.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-09 12:19:21 +03:00
Aleksei Sviridkin 23461e065d fix(redis): run one Sentinel when Redis runs a single pod (#5151)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

The chart pins `spec.sentinel.replicas: 3` whatever `replicas` says, so
an instance created with `replicas: 1` runs one Redis pod and three
Sentinel pods that have nothing to fail over to. This makes the Sentinel
count follow the data pods: one Sentinel when `replicas` is 1, three
otherwise.

The operator cannot run zero Sentinels (freshworks `Validate()` resets
`sentinel.replicas <= 0` to 3), and it needs at least one to locate the
master. With one Sentinel its quorum is `replicas/2+1` = 1, so the
single Sentinel works on its own.

The operator also gives the Sentinel Deployment a PodDisruptionBudget
with `minAvailable: 1` whenever Redis has two replicas or fewer
(`ensurePodDisruptionBudget` in
`operator/redisfailover/service/client.go`). On a single Sentinel that
budget refuses every eviction and would block node drains, so the chart
sets `sentinel.disablePodDisruptionBudget: true` in that case only. The
Sentinel `WorkloadMonitor` now expects the rendered count instead of a
fixed 3/2.

The operator only stops ensuring that PDB once the flag is set; it never
deletes the one it created while three Sentinels ran
(`EnsureSentinelDeployment` in freshworks v3.3.5). An existing instance
scaled to one replica would keep it, so
`templates/hooks/sentinel-pdb.yaml` adds a post-upgrade hook that
deletes `rfs-<release>`, with a Role scoped to that one name. It renders
on upgrade only, since a fresh one-replica install never gets that PDB.

From `replicas: 2` up nothing changes. Scaling an existing instance
between 1 and 2 replicas scales the Sentinel Deployment between 1 and 3
accordingly.

Not in scope: with `replicas: 1` the Redis data pod keeps its own
operator PDB (`minAvailable: 1`), which already blocks drains of its
node today; that is unchanged here.

Tests: `tests/sentinel_replicas_test.yaml` covers 1, 2, 3 and the
default, the PDB switch, and the `WorkloadMonitor`. Three of its cases
fail on `main` without the template change.
`tests/sentinel_pdb_hook_test.yaml` covers the hook. `helm unittest
packages/apps/redis`: 79 passed. Open PR #4990 also edits
`redisfailover.yaml` (Sentinel spreading); the two touch different keys.

Backport: `release-1.7` ships the same freshworks operator and CRD, so
`kind/backport` applies. `release-1.6` is not backportable as is: it
still runs spotahome `v1.3.0-rc1`, whose vendored CRD (chart 3.3.0) does
not declare `sentinel.disablePodDisruptionBudget`, and the operator's
CRDs are installed with Flux's default `crds: Create` policy, so they
are never updated on an existing cluster. The API server would prune the
field and the single Sentinel would keep a `minAvailable: 1` PDB that
blocks node drains. A 1.6 fix would need a CRD update path first.


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change (walked the
map: the diff changes a template, a test, and the wording of two
descriptions; no field, default, enum or name changes)
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(redis): an instance with `replicas: 1` now runs one Sentinel instead of three, without a Sentinel PodDisruptionBudget; on upgrade, the PDB the operator created earlier is deleted. Instances with two replicas or more are unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Redis deployments configured with one replica now use one Sentinel;
other replica settings continue to use three.
* Workload monitoring now reflects the configured Sentinel count. When
changing to one replica, upgrades remove the Sentinel
PodDisruptionBudget.
* **Documentation**
* Updated Redis configuration guidance to explain how Sentinel count
varies with the replica setting and clarify Sentinel resource defaults.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-09 12:17:32 +03:00
Aleksei Sviridkin d61db784a6 fix(mariadb-operator): derive the cluster name from the platform domain (#5123)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

Fixes #2348.

`packages/system/mariadb-operator/values.yaml` pins
`mariadb-operator.clusterName: cozy.local`, and the vendored chart turns
it into the operator's `CLUSTER_NAME` and into a SAN of the webhook
certificate. On a platform installed with any other
`networking.clusterDomain`, the operator resolves its own pods as
`<name>.<ns>.svc.cozy.local` and gets NXDOMAIN. Observed on a cluster
with `clusterDomain: lab.local`: a 2-replica MariaDB never gets
replication configured (`status.replication` stays `{}`, the secondary
runs as a standalone semi-sync master), and User, Database and Grant
fail with `lookup mariadb-<name>-primary.<ns>.svc.cozy.local ... no such
host`, while the operator only retries at verbosity 1.

`clusterName` is a value of the vendored subchart, which the `_cluster`
lookup used by package templates cannot reach. The platform now carries
`networking.clusterDomain` under the `mariadb-operator` subchart key in
the shared `cozystack-values` Secret, the same route
`keda.clusterDomain` and `opensearch-operator.manager.dnsBase` already
take. The literal in the package stays as the fallback for a render
without that Secret, so a cluster on `cozy.local` renders exactly as
before.

Tests cover both ends of the path:
`packages/core/platform/tests/apps_mariadb_operator_cluster_name_test.yaml`
pins that the Secret carries the domain under the anchored subchart key
and tracks a non-default value (it fails with the `apps.yaml` change
reverted), and
`packages/system/mariadb-operator/tests/cluster_name_test.yaml` pins
that a domain set on that key reaches the operator's `CLUSTER_NAME` and
the webhook certificate, with `cozy.local` kept as the standalone
default. `make helm-unit-tests` passes locally.


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(mariadb-operator): the MariaDB operator now uses the platform's networking.clusterDomain instead of a hard-coded cozy.local, so replication and User/Database/Grant reconciliation work on clusters with a non-default cluster domain.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* MariaDB now uses the configured networking domain as its cluster name
and webhook certificate address. If no custom domain is configured, it
continues to use the default cluster name and address.
* **Tests**
* Added coverage for default and custom domain settings, including their
effect on the MariaDB cluster name and webhook certificate address.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-09 12:07:37 +03:00
Aleksei Sviridkin 9e8e10c307 fix(backups): re-render the default backup objects when the S3 endpoint changes (#5230)
## What this PR does

Refs #5205.

The S3 endpoint the platform backups use is copied twice, and neither
copy is refreshed after `publishing.host` changes. The COSI sidecar
writes `BucketInfo` once per `BucketAccess`, so the credentials Secret
`bucket-cozy-backups-system-credentials` rendered from it keeps the old
endpoint. The Velero BSL and the default Strategy CRs keep whatever
endpoint the last render of `backupstrategy-controller` saw. The default
objects gate only acted on missing objects, so backups kept going to the
old host and nothing reported it.

The gate now also forces a render when a copy is stale:

- the bucket's `<bucket>-system` release, when the credentials Secret's
endpoint differs from the COSI Secret it is rendered from;
- `backupstrategy-controller`, when the endpoint an object was rendered
with differs from the credentials Secret's. The chart stamps the
rendered endpoint on the BSL and on every default Strategy as
`backups.cozystack.io/endpoint`, because the endpoint itself sits at a
different path in every kind. Objects without the annotation count as
stale until a render stamps it; `MinForceInterval` bounds the forces.

Stale credentials are forced before the BackupClass and the routed
objects are read, so an absent BackupClass or an unreadable Strategy
still fails the check but does not keep the old endpoint in the
credentials Secret. A `BucketInfo` without an endpoint is not compared,
since rendering from it would write an empty endpoint or fail the bucket
release. Repairing missing objects keeps working while the credentials
comparison is stale or fails, and a `systemSecretName` that is not
derived from a COSI Secret (no `-credentials` suffix, or the raw COSI
Secret itself) is not compared at all. RBAC is unchanged: the controller
already holds cluster-wide `get` on Secrets. No new metric: nothing
would consume it.

What this does not do: the COSI Secret still holds the endpoint the
`BucketAccess` was granted with, so after a host change the endpoint
follows only once an operator deletes the `BucketAccess` and forces
`bucket-cozy-backups` to recreate it (this issues new S3 keys). From
there the gate replaces the remaining manual steps of the recovery
described in the issue. Automatic re-grant is left for a separate issue,
and the operator runbook goes to the website, hence `Refs` rather than
`Fixes`.

Known limitation: the credentials comparison runs only when
`backupStorage.systemSecretName` is
`bucket-<bucketName>-system-credentials`, the Secret the forced
`bucket-<bucketName>-system` release renders. If an operator sets
`bucketName` and `systemSecretName` inconsistently, the comparison is
switched off rather than forcing a release that never rewrites the
compared Secret, so a stale endpoint in that Secret is not repaired.
Repairing a missing credentials Secret still forces the release derived
from `bucketName`, as before this PR.

Tested with unit tests for the gate (including the comparison
normalisation shared with the chart helpers), helm-unittest for the
annotation and the wiring, and a bats guard that every rendered Strategy
CR and the BSL carry the annotation. On a live cluster, a server-side
render of the chart produced exactly the annotation the gate derives
from the real credentials Secret, so the steady state does not force
anything.

### Screenshots

Not a UI change.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up: an operator runbook for changing `publishing.host` with
platform backups enabled (re-grant the `BucketAccess`, then let the gate
converge; restart ClickHouse pods that read the credentials from
environment variables). Not filed yet, pending approval of the text.
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(backups): the backup controller now re-renders the Velero BSL, the default backup strategies and the platform bucket's credentials Secret when their S3 endpoint is stale, so after the platform bucket's BucketAccess is re-granted for a new publishing.host, backups move to the new endpoint without forcing releases by hand.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Backup strategies and storage locations now record their storage
endpoint, enabling detection and repair of outdated endpoint settings
after bucket access changes.
* Bucket credentials and backup objects are refreshed independently when
their endpoints are missing or out of date.
* **Documentation**
* Clarified when endpoint changes trigger updates and how suspended
releases are handled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-09 12:02:42 +03:00
Marian Koreniuk e92bc8664b fix(tests): keep two unit tests green after the release image restamp
The release build rewrites packages/*/values.yaml: the kube-ovn image tag
becomes the Cozystack version and the traces collector becomes the
first-party otelcol-tracing build. Two unit tests read exactly those
fields, so every PR against release-1.7 has failed Unit & controller
tests since the first rc, and release-1.8 would repeat it.

Same change as #5179 on release-1.7.

Assisted-by: LLM
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
2026-11-08 19:37:03 +01:00
Alexey Artamonov c6c107fd08 docs(backups): state when the gate compares endpoints in the values reference
The values reference read as if the credentials endpoint were always
compared and an object without the endpoint annotation never forced a
render. Operators sizing the force counter or choosing a custom
systemSecretName need both conditions spelled out.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-08 20:18:29 +03:00
Alexey Artamonov dc909c778c fix(backups): compare the credentials endpoint only for the forced bucket's Secret
The gate forces bucket-<bucketName>-system, which renders the
credentials Secret of that bucket's system user and nothing else. The
COSI Secret name was derived from systemSecretName instead, so a value
naming another bucket's Secret made a lagging endpoint there force the
platform bucket release every throttle window without ever refreshing
the Secret that was compared.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-08 20:18:29 +03:00
Alexey Artamonov d0f47f7dd5 fix(backups): compare the credentials endpoint only for a Secret derived from COSI
backupStorage.systemSecretName may name a Secret the bucket chart does
not render from a COSI Secret: the raw COSI Secret itself, which the
projector accepts, or any Secret without the -credentials suffix. The
COSI Secret name was then the source name unchanged, so the gate
compared a Secret with itself and either reported it stale on every
tick or failed the check, and stopped repairing missing default objects
as well. Skip the comparison when there is no distinct COSI Secret or
no endpoint key to compare.

The gate compares the annotation with what the chart's endpoint
helper renders, and the helper renders a present but blank endpoint as
a bare https:// instead of falling back, so the gate has to compute
the same value or its expectation and the chart disagree for that
Secret. The gate's docs, metric help and log lines named only missing
objects, so they did not cover a stale endpoint they now report.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-08 20:18:29 +03:00
myasnikovdaniil 970191dc8a fix(clickhouse): restore the archive a Backup was taken as (#5208)
## What this PR does

A ClickHouse `RestoreJob` restored the newest archive under the app
prefix, whichever `Backup` it named. Strategy pod generated the archive
name with `date -u` and nothing was recorded on the `Backup`, so restore
could only list the remote archives and take the last one. Restoring an
older `Backup` brought back the latest data.

Now the controller names the archive before it creates the backup Job
(`clickhouse-<app>-<startedAt>` from persisted `status.startedAt`, so
retried pods write the same one), passes it to the pod as `.ArchiveName`
and records it in the `Backup` `driverMetadata`. Restore takes exactly
that archive and fails with its name when storage has no such archive.
The name is recorded only when the Job's pod got it, so a Job left from
the previous controller during an upgrade, or a custom strategy with its
own names, leaves no name instead of a wrong one.

A retried pod reuses the name, so it waits for an attempt the sidecar
still runs (no 423) and asks for `resume`.

A `Backup` outlives its `BackupJob` and a Plan names `BackupJob`s after
the schedule slot, so a `BackupJob` created again under a used name met
the earlier run's `Backup` when its Job completed, took it as its result
and left the archive it just wrote recorded nowhere. Now it fails if
that `Backup` records another archive, and the message names both. A
retry of the same run still gets its own `Backup` back.

Old `Backup`s have no recorded name and stay restorable: the pod logs a
`WARNING` and takes the newest archive stamped no later than `takenAt`,
and fails when nothing qualifies. While the chart rolls out, previous
controller pod can still render the new template and pass no `takenAt`,
so a restore of a `Backup` without recorded name fails before it touches
any table, and works when created again after the rollout.

Fixes https://github.com/cozystack/cozystack/issues/5199

Tests are go unit tests for naming, recording, a reused `BackupJob` name
and restore render, and `hack/altinity-backup-strategy.bats`, which runs
the shipped script against a stubbed clickhouse-backup API (the restore
tests go red on the old template). Checked on a live ClickHouse with two
backups of 1500 and 2000 rows: restore of the older one by recorded name
ends with 1500 rows, fallback for an unnamed older backup picks the same
archive with the warning, a missing archive fails without touching the
table, a retry over an existing local backup completes under the same
name. The controller binary itself was not run on the cluster, only its
rendered output in the pod.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(clickhouse): a RestoreJob now restores the archive its Backup was taken as, not the newest one. New Backups record the archive name, Backups taken before this release are restored best effort from the newest archive stamped no later than their takenAt, with a warning in the restore pod log. A ClickHouse BackupJob created again under a name whose Backup still exists now fails instead of reporting that Backup as its result.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Backups now use a consistent archive name tied to the backup job, and
completed backups retain that name for future restores.
* Restores can select the recorded archive and backup timestamp. If no
archive was recorded, the system selects the newest eligible archive for
the application.
* **Bug Fixes**
* A restore now fails when its recorded archive is missing instead of
silently selecting a different one.
* Backup requests avoid starting a duplicate action when the matching
action is already in progress.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-06 11:54:38 +05:00
myasnikovdaniil 18779dddfc fix(backups): report sizeBytes for MariaDB and CNPG backup artifacts (#5223)
## What this PR does

Fixes https://github.com/cozystack/cozystack/issues/4316.

Continues https://github.com/cozystack/cozystack/pull/5015 by androndo
on a separate branch. All ten of their commits are kept as they were
(authorship, messages and order, only rebased on current `main`).
MariaDB and CNPG backups stored in S3 report
`Backup.status.artifact.sizeBytes`, the size of the dump
mariadb-operator uploaded or the total of the barman base backup, and
the field stays unset when the size cannot be attributed to that one
backup.

Four commits are added on top:

- `test(backups)`: operator Backup fixtures carry the condition
mariadb-operator writes on success, `Complete=True` with reason
`JobComplete` and message `Success`. Operator sets `Complete=True` for a
failed Job too (reason `JobFailed`), so a fixture without reason
describes a state that never happens.
https://github.com/cozystack/cozystack/pull/5207 makes the driver check
that reason, the tests pass with and without it.
- `refactor(backups)`: the MariaDB matcher returns the matched dump
(key, file name, timestamp from the name, size, LastModified) and tells
"not found" from "ambiguous", and the Secret lookup and S3 client live
on a small value any reconciler can build from its own client. A MariaDB
restore needs this to restore the dump of the Backup it was given and
not the newest one under the prefix. `sizeBytes` does not change here.
- `fix(backups)`: the probe reads bucket, prefix, endpoint, credentials
and TLS from the operator Backup's own `spec.storage.s3`, not from the
strategy as it renders now, so editing the strategy during a backup does
not send the probe somewhere else.
- `fix(backups)`: a dump is attributed only when its ownership is
proven. Before, a backup pod with a clock more than a minute behind,
right after a previous run, got that run's dump size (666 bytes for a
123-byte dump in a test). Now the dump has to be the only one that
either clock places in the operator Backup's lifetime, both clocks have
to agree, and neither may place it in the lifetime of another finished
operator Backup with the same bucket and prefix. Otherwise the size
stays unset. Two cases are still not covered. First, this run's own dump
is missing from the listing while the operator Backup of a neighbour was
deleted or has not finished, then the neighbour's dump can be taken.
Second, backup pod clock and object store clock are both off the same
way by more than the minute of slack, which moves a finished neighbour's
dump into this run's window, and nothing the clocks record tells them
apart. A run whose Job failed has no dump of its own and reaches the
probe only because the driver takes `Complete=True` for success,
https://github.com/cozystack/cozystack/pull/5207 fixes that.

Of the open threads on the original PR, the cache reads, the dump file
name layout and the default AWS endpoint test were already handled by
its later commits, the clock skew one is the last commit here and the
failed Job case is https://github.com/cozystack/cozystack/pull/5207.

`minio-go` is new to this tree, this PR adds it as a direct dependency
and `internal/backupcontroller/artifact_size.go` is its only importer.
The open Bucket strategy PR
https://github.com/cozystack/cozystack/pull/4235 picks the same client
at the same version.

Each clock skew case, and a run whose own dump is gone while the
neighbour's operator Backup still exists and has finished, have a unit
test that fails on the old matcher, and every new guard has a mutant
that turns a test red. I also ran the matcher against a real bucket
after four MariaDB backups in a row. Runs that finished more than a
minute apart mapped to the dump their operator Job logged, the two back
to back came back unset instead of taking each other's dump, and with a
run's own dump hidden from the listing the matcher refused the
neighbour's dump that the old one returned.

### Screenshots

N/A, no UI changes.

### Downstream repositories

The only API change is the description of the existing
`Backup.status.artifact.sizeBytes` field, no new field, value, default
or enum, no package, values or tooling change, and neither
cozystack/website nor cozystack/terraform-provider-cozystack references
`sizeBytes`.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(backups): MariaDB and CNPG (Postgres) backups stored in S3 now report `status.artifact.sizeBytes`: the size of the dump mariadb-operator uploaded, or the total size of the barman base backup (WAL archive excluded). A MariaDB backup takes the size of the one dump that falls inside its run on both the backup pod's and the object store's clock and inside no other finished run, and the size is left unset rather than set to 0 whenever it cannot be measured or attributed. Velero-based backups intentionally keep it unset.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Backup artifacts now report the stored size attributable to an
individual MariaDB or CNPG backup when it can be determined. Shared or
ambiguous storage may leave the size unset.
* **Bug Fixes**
* Improved handling of backup artifact creation and lookup errors,
including retrying when an existing backup cannot be read and failing
jobs when artifact creation is rejected.
* **Documentation**
* Clarified that an unset size means unknown; zero does not indicate an
unknown size.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-06 11:54:28 +05:00
myasnikovdaniil ab13f9f72c fix(cilium): pass the CiliumNode list to jq on stdin in the IPAM hook (#5212)
## What this PR does

Fixes https://github.com/cozystack/cozystack/issues/4968. The cilium
IPAM migration hook passed the whole `kubectl get ciliumnodes -o json`
to `jq` as one argument, and Linux allows 128 KiB for a single argument.
Cilium copies Node labels onto every CiliumNode (22 to 30 KiB per node
with Talos, NFD or GPU labels), so the list is over the limit within six
nodes. `jq` can't start (`Argument list too long`), the hook Job fails
before it moves any node, `cilium` HelmRelease loops on `UpgradeFailed`
and everything that depends on networking stays in `DependencyNotReady`.

The hook is rendered on every upgrade of cilium in `cluster-pool` mode,
so a cluster that already finished the migration hits the same on its
next values change. The failing step only logs the pods that still hold
an old Ingress IP, but it runs before the node loop, so on a cluster
that is not migrated yet it blocks the migration itself.

Now the list goes to `jq` on stdin together with the IP objects, same as
the other `jq` calls in the script get their input.

The new test in `hack/cilium-ipam-migration.bats` makes six CiliumNodes
over the limit (one 24 KiB annotation each instead of labels, and it
checks the list is over 128 KiB even printed compact). It fails on the
old script with status 126 and passes now. It also checks that each pod
is named with the node whose Ingress IP it holds and that a pod on an
unrelated address is not named, because a join that always names the
first node, or reports every IP object, passed the old tests.

I did not take two suggestions from the issue. Skipping the hook when
every CiliumNode is in the pool changes what one existing test pins (the
hook adds the pool to `excludeIps` in that case) and doesn't help
clusters that are not migrated yet. A value to disable the hook leaves
the default broken. `cozystack-operator` rewriting `disableHooks` and
helm-controller rolling the DaemonSet on every retry are other
components. The bug is in release-1.6 and release-1.7 as well.

### Downstream repositories

I walked the trigger map against the diff (one hook script and its bats
test): no package, values, schema, default or output name changed.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(cilium): the IPAM migration hook no longer fails with "Argument list too long" on clusters with more than a few nodes, which blocked the cilium upgrade and every package that depends on networking.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved IP address migration handling for environments with large
node lists, avoiding failures caused by operating system argument-size
limits.
* Migration continues to identify affected pods by node and distinguish
unrelated pods.
* **Tests**
* Added coverage for migrations involving large node lists, multiple IP
objects, and unrelated pods.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-05 19:37:05 +05:00
Alexey Artamonov cd91138f33 fix(backups): re-render the default backup objects when the S3 endpoint changes
The S3 endpoint the platform backups use is copied twice and neither
copy is refreshed. The COSI sidecar writes BucketInfo once per
BucketAccess, so the credentials Secret rendered from it keeps its
endpoint, and the Velero BSL and the default Strategy CRs keep whatever
endpoint the last render of backupstrategy-controller saw. The default
objects gate only acted on objects that were missing, so after a
publishing.host change every backup kept going to the old host with
nothing reporting it.

The gate now also forces a render when a copy is stale: the bucket's
<bucket>-system release when the credentials Secret's endpoint differs
from the COSI Secret it is rendered from, and backupstrategy-controller
when the endpoint an object was rendered with differs from the
credentials Secret's. That endpoint sits at a different path in every
kind, in some only as a bare host, so the chart stamps the rendered
value on each object as backups.cozystack.io/endpoint and the gate
compares that. Objects rendered before the annotation existed count as
stale until one forced render has gone through, which stamps it.

The COSI Secret itself still holds the endpoint the BucketAccess was
granted with; it follows a host change only once the BucketAccess is
re-granted.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-04 19:13:55 +03:00
Aleksei Sviridkin c1ba05574f feat(monitoring-agents): keep application identity in container log streams (#5217)
## What this PR does

Makes Fluent Bit keep the pod lineage labels
`apps.cozystack.io/application.{group,kind,name}` on container log
lines, as the stream fields
`apps_cozystack_io_application_{group,kind,name}`, for both the default
container-log output and the per-tenant routed output.

Why: the per-application logs scope in the design proposal [Tenant
application logs](https://github.com/cozystack/community/pull/95)
(section 6) filters by these stream fields. Today the kubernetes filter
fetches pod labels, but they are dropped together with the rest of the
`kubernetes_*` keys before output.

What changes in the pipeline:

- A second `nest` lift of the pod labels, then a rename of the three
lineage keys to their final names before the existing `Remove_wildcard
kubernetes_` runs. All other labels are still removed. Every lifted key
is prefixed, so no other label can collide into the identity names.
- The three keys are added to `_stream_fields` of both container-log
outputs. Events and audit pipelines are unchanged.
- A parsed JSON log line is kept under `log_processed` (already on
`main`), so a line's content cannot create the identity keys.
- Guest Kubernetes clusters are not affected: their monitoring-agents
release replaces the filters and outputs wholesale. A regression test in
`packages/apps/kubernetes` pins that guest streams carry no identity
fields.

Effect on existing streams: VictoriaLogs treats a missing stream field
like an empty one, and the rename writes nothing when the label is
absent. Pods without lineage labels keep exactly the same stream. Pods
with lineage labels get a new stream once after the rollout; their
earlier lines stay reachable by namespace.

#### How it was tested

- helm-unittest for `monitoring-agents` (lift order, rename before
`Remove_wildcard`, stream fields on both outputs, events and audit
untouched) and for `kubernetes` (guest streams unchanged). The
events/audit guard was mutation-checked.
- Verified on a test cluster: real pod log lines arrive, Redis pods'
lines carry the three `apps_cozystack_io_application_*` stream fields,
and Fluent Bit logs show no errors.

#### Known limitation

Guest clusters write to the parent log store without authentication, so
a guest administrator can forge these stream fields as well as the
namespace field, within the store their tenant shares. This is the
write-path trust risk already documented in the proposal.

### Screenshots

No UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

Downstream: no trigger matched. Candidate: none affected, because the
diff changes `packages/system/monitoring-agents/values.yaml` (a system
package with no schema) and adds tests; the file under
`packages/apps/kubernetes` is a test only, so the app's `values.yaml`
and `values.schema.json` are unchanged.

### Release note

```release-note
feat(monitoring-agents): container log lines of pods that belong to a Cozystack application now carry `apps_cozystack_io_application_{group,kind,name}` stream fields in VictoriaLogs. Streams of pods without application labels are unchanged; streams of application pods start a new stream once after the upgrade.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Container-log streams now include application group, kind, and name
metadata for root and tenant stores.
* Guest monitoring streams include pod name, container name, and
namespace fields.
* **Bug Fixes**
* Application metadata is added to container-log streams without
changing the existing stream fields for event and audit outputs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-04 18:27:20 +03:00
Andrey Kolkov 523d74a503 fix(backups): retry instead of failing when an existing Backup cannot be read
When Create collides with a Backup an earlier reconcile already made and
the read that follows fails, the drivers failed the BackupJob, although
the backup and its Backup both exist. Return a distinct error instead,
which the drivers hand back to the controller so the reconcile is
retried with backoff and then finds the Backup.

minio.New already refuses an endpoint with a path prefix and accepts a
trailing slash, so drop the duplicate check in the shared client; the
CNPG probe keeps its own check because it parses the URL itself and
would otherwise drop the path before minio sees it.

Say in the sizeBytes API doc what the value covers, such as the base
backup without the shared WAL archive for CNPG, and that strategies
which cannot attribute stored bytes to one backup leave it unset.

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-11-04 13:04:22 +00:00
Andrey Kolkov d4ad5612d3 fix(backups): report sizeBytes for MariaDB backup artifacts
Backup.status.artifact.sizeBytes was only ever set by the etcd strategy,
so MariaDB backups always arrived without a size.

mariadb-operator writes each logical dump as
<prefix>/backup.<RFC3339>[.<compression>].sql under a prefix shared by
every backup of the application, and the URI the driver records is
synthetic, so there is no single object to HEAD. On completion the driver
now lists the prefix with the tenant credentials the operator Backup was
rendered with and takes the size of the one dump whose timestamp falls
inside the operator Backup's lifetime. When the bucket cannot be read or
the window does not match exactly one dump, the field stays unset.

The API doc now states that an absent sizeBytes means "unknown", and the
Velero strategy documents why it leaves the field unset: kopia stores
data content-addressed across backups, so there is no per-backup stored
size.

Refs #4316

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-11-04 13:04:22 +00:00
Myasnikov Daniil 47b497194e fix(backups): restore the ClickHouse archive a Backup was taken as
The Altinity strategy named each archive inside its Pod with date -u
and recorded nothing on the Backup, so a restore could only guess: it
listed the remote archives under the application's prefix and took the
newest. A RestoreJob of an older Backup therefore restored whichever
backup had run last.

The controller now names the archive before it creates the backup Job,
from the BackupJob's persisted StartedAt, so every reconcile and every
retried Pod of the Job writes the same one. The name is recorded in the
Backup's driverMetadata next to the strategy parameters, and a restore
renders it into the strategy Pod, which restores exactly that archive
and fails, naming it, when remote storage no longer holds it. The name
is recorded only when the Job's Pod was given it, so a Job created by
an earlier controller, or a strategy that names its own archives,
leaves the Backup without one instead of pointing it at an archive
that does not exist.

Because a retried Pod reuses the name, it waits on an attempt the
sidecar is still running instead of being refused with 423, and asks
for resume, so a local or partial remote backup an earlier attempt left
is completed rather than refused as existing.

Backups taken before names were recorded stay restorable: the Pod picks
the newest archive whose name is stamped no later than the Backup's
takenAt, which is set only after the backup Job finished, and logs a
warning that the choice is a best effort.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-11-04 11:10:42 +00:00
Myasnikov Daniil a2d0e4cb50 fix(cilium): pass the CiliumNode list to jq on stdin in the IPAM hook
The hook fed the whole `kubectl get ciliumnodes -o json` to jq as one
argument. Cilium copies every Node label onto its CiliumNode, 22 to 30
KiB per node on Talos or with NFD and GPU labels, so the list passes the
128 KiB Linux allows a single argument within six nodes. jq then fails
to start with "Argument list too long", the hook Job fails before it
moves any node, and the cilium HelmRelease loops on UpgradeFailed with
the HelmReleases behind it stuck in DependencyNotReady.

The hook renders on every upgrade of cilium in cluster-pool mode, so a
cluster that already finished the migration hits the same failure on its
next values change.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-11-04 06:57:03 +00:00
ZverGuy e5f1714836 feat(monitoring): keep application identity on container log lines
A reader that scopes logs to one application needs the owning
application on every line, and the lineage webhook already puts it on
the pod as apps.cozystack.io/application.{group,kind,name}. Fluent Bit
fetched those labels and then dropped them with the rest of the
kubernetes map, so the store could only be scoped by namespace.

Keep the three labels as apps_cozystack_io_application_{group,kind,name},
the names the metrics side uses, and make them stream fields on every
container-log output so the scope is an indexed stream filter. A pod
without the labels produces no such fields, and VictoriaLogs builds the
stream from the fields present, so existing streams keep their
identity; a labelled pod starts a new stream once.

Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-11-04 04:18:09 +07:00
ZverGuy 7de70ac2fc feat(monitoring-agents): export PVC labels from kube-state-metrics
The tenant metrics API scopes a query to one application by looking up
its pods and PVCs through their apps.cozystack.io/application.* lineage
labels. kube-state-metrics emits kube_persistentvolumeclaim_labels only
for resources on --metric-labels-allowlist, so without PVCs there the
PVC half of that lookup always comes back empty and volume series of
an application fall out of its scope.

All labels are allowed, as for pods and deployments: kube-state-metrics
emits one series per PVC whatever keys are listed, and PVCs are far
fewer than pods, which already export every label.

Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-11-04 04:12:15 +07:00
myasnikovdaniil 4087218a02 docs: move the repo-only guides onto the website (#5034)
## What this PR does

Seven guides under `docs/` were operator facing and written alongside
the code, and left in the source tree where no user looks. They are on
the website now, so this deletes them here and repoints every reference.

```
docs/operations/backup-classes.md                  -> operations/services/backup-classes
docs/operations/resource-presets.md                -> guides/resource-management
docs/storage-immutability.md                       -> operations/storage-class-immutability
docs/operations/system-memory-limits.md            -> operations/configuration/system-memory-limits
docs/gateway-classes.md                            -> networking/gateway-classes
docs/observability/distributed-tracing.md          -> operations/services/monitoring/distributed-tracing
docs/operations/seaweedfs-capacity-for-backups.md  -> operations/services/object-storage/capacity-for-backups
```

`docs/release.md`, `docs/deprecating-applications.md` and the two
SeaweedFS runbooks in `docs/operations/` stay. A workflow, `AGENTS.md`,
a migration and a bats suite read them by path, which is the test from
#3777.

References point at `https://cozystack.io/docs/v1/...`. The site
rewrites that short prefix to the latest version in `layouts/404.html`,
so the links follow releases and pin none. Backup Classes, resource
management and `storageClass` immutability exist in the latest version
already. The other four pages are only in `next/` until 1.7.0 is
promoted, and their links start to resolve then. The generated mongodb
files come from `values.yaml` through `make generate` with
cozyvalues-gen v1.7.0, the version pinned in
`.github/workflows/pre-commit.yml`.

Tests: helm unit tests for mongodb (151), postgres (209), vm-disk (114)
and kubernetes-nodes (223), go tests for backupcontroller, presets and
the operator, rd-presets-check, the migration-50 bats suite.

### Downstream repositories

- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: cozystack/website#763

Walked the rest of the trigger map against the diff. No provider
follow-up: the mongodb `values.schema.json` diff is one `description`
string, with no field added, removed or renamed, no enum, no default,
and no `kind`, `plural` or `release.prefix` change.
`packages/core/platform/values.yaml` changed in comments only and no
key, so the hand written platform-package table on the site is
unaffected.

Please merge after cozystack/website#763, the links in the code only
resolve once those pages deploy.

### Release note

```release-note
docs: seven guides move from the docs/ directory of this repository to cozystack.io (Backup Classes, resource presets, storageClass immutability, system component memory limits, GatewayClass selection, distributed tracing, SeaweedFS capacity for backups), and in-repo references point there.
```
2026-11-03 23:09:46 +05:00
Aleksei Sviridkin a2cc9a3016 fix(dashboard): link a tenant row to its detail page (#4004)
## What this PR does

Fixes #3822

Tenant list showed name as plain text and the only link in a row was
Edit button, so tenant detail page with its tabs and Delete was
reachable only by opening edit form and cancelling out of it. Name cell
is a link to that page now, it appears exactly where Edit appears and
points to the same CR. Plain click switches selected tenant to the row's
parent first, same as Edit does, because detail page takes namespace
from selected tenant. Ctrl, cmd and shift click don't run that switch,
so the tab you open a tenant from stays where it was.

First commit fixes what the link depends on. Console recovers a tenant's
parent from namespace name (`tenant-a-b` sits under `tenant-a`), but
root is the one level that naming skips: tenant `acme` ordered in
`tenant-root` owns `tenant-acme`, not `tenant-root-acme`. So
`realParentNamespace` returned undefined for every top level tenant,
which is most tenants on a normal cluster, and Edit button has been
missing there for as long as it exists. Ancestor labels carry the parent
that the name does not. Root level tenant named `root-x` resolves to CR
`root-x` now, not `x`.

### Notes

I first tried to make links carry their tenant in the url (`?tenant=`),
so a detail page opened in new tab resolves on its own. That turned into
questions about tenant context itself: what to do with a tenant in a
link the user can't see, what localStorage may hold, and TenantNamespace
LIST ignoring `labelSelector`. Detail route addresses a Tenant CR by
relative name and takes namespace from selected tenant, and making that
robust is a refactor of tenant context and routing, which should start
with study and planning, not with patches on patches. So this PR only
closes the UX gap (no way from tenant list into a tenant's page), going
deeper is out of scope.

Known limit that comes with it: link has no tenant in it, so opening it
in a new tab (ctrl/cmd/middle click, open in new tab, copied url)
resolves against the tenant last selected in any tab, because selection
lives in shared localStorage. For a row directly under that tenant it
opens the right page. For a row deeper in the tree the page says
`Application <name> not found.` and there is no Delete. If that tenant
also has a direct child tenant with the same relative name, new tab
opens that other tenant's page with working Edit and Delete, and delete
confirm says only `Delete Tenant "<name>"?`. Plain click in the same tab
is not affected. Detail page's own tabs and Edit link already behave
this way, it's how detail route resolves its namespace today and fixing
it properly is the refactor above.

A row links, and offers Edit, exactly when namespace holding its Tenant
CR is one the apiserver returns to this user. It returns a tenant
namespace only if the user is subject of a RoleBinding in it, so
sub-tenant admin that cant act in `tenant-root` sees no link there.

No row level Delete button. Issue allows either that or a route to
detail page, I took the route so confirm and mutation stay in one place.

Covered by tests, every source change here fails a test when reverted:

- top level tenant, nested one, root level tenant named `root-x`, forest
root without ancestor labels
- bridged row whose real parent is invisible gets neither link nor Edit
- plain click switches to the row's parent and opens detail page
- ctrl click leaves selection and location alone

### Screenshots

Tenant list under root on a dev cluster. Every row name is a link to
tenant page now (`pr4078` is hovered), top level tenants had neither
link nor Edit before. Host column redacted.

<!-- SCREENSHOT: drag in 01-tenant-list-links.png here -->
<img width="1440" height="900" alt="01-tenant-list-links"
src="https://github.com/user-attachments/assets/39cd723a-1ac5-4d3b-8c36-b70ddc626961"
/>


### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

All changed files are under `packages/system/dashboard/images/console`,
console has no downstream copy (standalone UI repo is archived).

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(dashboard): tenant name in the tenant list links to its detail page now (where Delete is), and top level tenants get their Edit button back.
```



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Root-level tenant names are now displayed without incorrectly removing
prefixes, and tenants ordered directly under the root are correctly
recognized in the hierarchy.
* Tenant rows link to their detail pages when the parent tenant is
accessible. Same-tab navigation updates the active tenant, while opening
a link in another tab leaves the current selection unchanged.
  * Edit links and navigation now use consistent tenant routes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-02 16:33:51 +03:00
Myasnikov Daniil 7f55ff5d85 fix(dashboard): link a tenant row to its detail page
The tenant list rendered the name as plain text and put the row's only
link on an Edit button, so nothing in the list reached the detail page
that every other kind gets, tabs and a Delete action included. The only
way there was to open the edit form and cancel out of it.

The name cell becomes a link wherever Edit is offered, to the same CR.
The detail page reads the namespace from the tenant context, so a plain
click aligns the context to the row's parent first, the way Edit
already does. React runs that handler for a ctrl-, cmd- or shift-click
too, which the browser turns into a new tab or window, so it runs only
for the click that navigates its own tab; otherwise opening a tenant
beside the current one would move the tab left behind as well.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-30 16:38:53 +05:00
Myasnikov Daniil e44e41d307 fix(dashboard): resolve the Tenant CR of a tenant ordered in the root
A tenant's CR lives in its parent's namespace, and the console recovered
the parent from the namespace name: `tenant-a-b` sits under `tenant-a`.
The root is the one level that naming skips. A tenant `acme` ordered in
`tenant-root` owns `tenant-acme`, not `tenant-root-acme`, so the prefix
test never matched a top-level tenant and `realParentNamespace` returned
undefined for it. That is most tenants on a normal cluster, and the Edit
button, offered only where the CR's namespace resolves, was missing for
all of them.

The ancestor labels carry the parent the name does not, so the lookup
falls back to `tenant-root` when the chain names it. Deriving the CR name
keys on the parent being the root rather than on the child's name
happening to start with it, which keeps a root-level tenant called
`root-x` from resolving to the CR `x`, and the row label agrees with it
rather than showing `x`.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-30 16:23:10 +05:00
IvanHunters 8d96c46994 fix(monitoring-agents): isolate merged log payload under log_processed
Fluent Bit's kubernetes filter runs with Merge_Log On but without
Merge_Log_Key, so a parsed JSON log line is merged into the record root.
An application log line can thereby forge the service fields that later
pipeline stages rely on for routing and tenant isolation
(kubernetes_namespace_name, kubernetes.labels, tenant, cluster,
log_source). Setting Merge_Log_Key log_processed nests the parsed
payload under a dedicated key so application content can no longer
override top-level service fields.

Applies to both the platform (system/monitoring-agents) and guest
(apps/kubernetes) Fluent Bit configurations.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-29 17:08:49 +03:00
Mattia Eleuteri 48779cc03b fix(redis): run one Sentinel when Redis runs a single pod
The chart pinned three Sentinels whatever the replica count, so a
single-pod instance paid for three pods that have nothing to fail over
to. One Sentinel is enough for the operator to locate the master: its
quorum is replicas/2+1, which is 1.

The operator gives the Sentinel Deployment a PodDisruptionBudget with
minAvailable 1 whenever Redis has two replicas or fewer. On a single
Sentinel that budget refuses every eviction and would block node
drains, so it is disabled in that case. From two Redis pods up nothing
changes: three Sentinels, budget kept.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-10-29 10:00:55 +01:00
Aleksei Sviridkin 60d1f53e82 fix(kubernetes-nodes): plan autoscaler capacity on the cores the VM actually gets (#4974)
On the kubevirt substrate `resources.cpu` is written into
`domain.cpu.cores` as spelled and decoded as YAML, so `"010"` gives the
VM eight cores, while the cluster-autoscaler capacity annotation parsed
the same string as decimal ten
(`capacity.cluster-autoscaler.kubernetes.io/cpu: "9000m"` on a node that
offers 7000m). This is the inconsistency reported in #4554 and in the
comment on #4546.

- The YAML-decoded value the range check already computes is kept, and
on kubevirt the capacity annotation and the reserved-CPU check start
from it.
- The kubelet reservations are not touched. The worker machineconfig
(`_talosconfigtemplate.tpl`) sizes them from the raw string read as a
decimal, and the TalosConfigTemplate name is a hash of that config, so
reading the decoded value there would roll every leading-zero pool once.
Instead the annotation subtracts the reservations the machineconfig
really carries from the cores the VM really has: for `"010"` that is
8000m − 2 × 500m = `"7000m"`, the node's real allocatable, while `"8"`
keeps `"7200m"` (2 × 400m). Tests pin both sides: the annotation and the
bootstrap `configRef` name in `resources_cpu_test.yaml`, and
`systemReserved.cpu` / `kubeReserved.cpu` in the machineconfig in
`talos_worker_config_test.yaml`.
- No pool rolls. Checked by rendering `nodegroup.yaml` on main and on
this branch for nine spellings (`"2"`, `2`, `2.0`, `"2.0"`, `"+2"`,
`"1e1"`, `"010"`, `999999`, `4294967295`): the only difference is the
capacity annotation of the `"010"` case, so the
`KubevirtMachineTemplate` name and the TalosConfigTemplate `configRef`
stay as they are. The machineconfig template itself is not changed.
- Proxmox parses the quantity as decimal for the machine template and
for the reservations alike, so it never had the inconsistency and is
unchanged (pinned by a test).
- The `resources` description says the value is decoded as YAML, what
that means for a leading zero, and that the reservations still read it
as a decimal. README, schema, the RD and the Go type are regenerated
from it.

This closes #4554 without the migration and the render-time refusal it
proposes. A refusal at render breaks a stored pool on its next upgrade,
a pattern in the OpenAPI schema breaks every update of an existing CR,
and a platform-wide migration over every `KubernetesNodes` and parent
`nodeGroups` entry is a lot of blast radius for a spelling nobody has
reported in a live cluster. With this change the autoscaler plans for
the node the VM really is. What is left of a leading zero is the octal
reading itself and reservations sized from the decimal value, and both
are documented. The unquoted zero of the issue is already harmless: it
is falsy, counts as unset, and renders the same bytes as an absent
field. If a migration is still wanted, please reopen.

Verified with `helm unittest .` in `packages/apps/kubernetes-nodes` (38
suites, 233 tests, 12 snapshots; pointing the annotation back at the
decimal string turns the `"010"` case red with `9000m`), `make generate`
(no diff), and a trial merge with #4973 (clean, 243 tests green).

Closes #4554


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* KubeVirt VM sizing and autoscaler capacity now reflect YAML-decoded
CPU values. For example, `"010"` represents eight VM cores, while
kubelet reservations interpret it as decimal ten. `"8"` and `"10"` yield
`7200m` and `9000m` of autoscaler capacity, respectively. Proxmox
continues to interpret `"010"` as ten cores, yielding `9000m` of
capacity.
* **Documentation**
* Clarified explicit worker CPU and memory sizing behavior, including
how leading-zero CPU values affect VM sizing, reservations, and capacity
calculations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-29 11:22:40 +03:00
Aleksei Sviridkin 44848b9c9d fix(kubernetes): run Proxmox CCM v0.16.1 and pin the Proxmox images by digest (#4917)
## What this PR does

`v1.7.0-rc.1` and `release-1.7` ship the Proxmox
cloud-controller-manager as
`ghcr.io/sergelogvinov/proxmox-cloud-controller-manager:v0.15.0`, a bare
tag. That tag was re-pushed upstream on 2026-09-27: it used to name the
2026-08-06 build (`sha256:fbc553921145…`) and now names
`sha256:d81702998ae1…`, a binary whose gitVersion is `0.15.0` without
the leading `v`. component-base v0.37 parses that string in an `init()`
and panics:

```
panic: version string "0.15.0" doesn't match expected regular expression: "^v(\d+\.\d+\.\d+)"
```

The CCM Deployment crash-loops, no new Node loses
`node.cloudprovider.kubernetes.io/uninitialized`, and workloads on new
workers stay Pending. On a live Proxmox cluster this went unnoticed for
five days: the existing Nodes were already initialized, and the failure
only showed when the next new worker came up. Every fresh install pulls
the broken build, so as it stands v1.7.0 would be the first stable
release with the Proxmox substrate, shipping a CCM that cannot start.

Two commits, both cherry-pick onto `release-1.7` unchanged:

1. **`fix(kubernetes)`**: run v0.16.1, pinned as tag plus digest in
`images/proxmox-cloud-controller-manager.tag`. That is the same shape
the chart already uses for the KubeVirt cloud provider. v0.16 builds
with goreleaser and stamps the version with its `v`. The config the init
container writes still parses: the cluster entry moved to
go-proxmox-pool's `ClusterConfig` with the same `url`, `insecure`,
`token_id`, `token_secret` and `region` keys, and `features.provider:
capmox` still writes `proxmox://<SMBIOS UUID>`, the providerID capmox
puts on the Machine.
2. **`fix(kubernetes, proxmox-csi)`**: pin the CSI driver from the same
registry at its current v0.20.0, with no version change. The
management-side controller goes through
`images/proxmox-csi-controller.tag`. The tenant-side node plugin goes
through `repository` + `tag` in
`packages/system/proxmox-csi/values.yaml`, which also puts it in
`hack/lib/image-refs.sh` and so in front of the release multi-arch gate.
Pinning takes the version out of the vendored chart's hands, so
`hack/proxmox-csi-driver-pin_test.bats` now fails unless the chart's
`appVersion` and both pins name the same release.

All three digests are OCI indexes for `linux/amd64` and `linux/arm64`.
The sig-storage sidecars stay on tags, as they do for the KubeVirt CSI
driver: registry.k8s.io publishes through image promotion, which does
not move a tag.

### Testing

- `helm unittest`: `packages/apps/kubernetes` 31 suites / 346 tests,
`packages/apps/kubernetes-nodes` 37 / 223, all passing.
- bats: the new test, plus `image-pin-consistency`,
`promote-rewrite-tags_test`, `overlay-main-images_test`,
`nightly-mirror_test`, `promote-retag_test`, `stitch-multiarch_test`,
`bats-strict-setup`, `bats-runner-coverage`: 107/107. The new test was
checked to fail on a node tag at another version.
- `git cherry-pick -x` of both commits onto `release-1.7` (315b335cf)
applies cleanly, and the Proxmox unit suites and the new bats test pass
there.
- v0.16.1 compatibility was checked against its source, as described
above. It has also run on a live Proxmox cluster, including the
initialisation of a new worker; the results are in the comment below.

### Screenshots

Not a UI change.

### Downstream repositories

The diff touches no app `values.schema.json` or defaults, adds or
renames no package, and moves nothing under `hack/`.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(kubernetes): the Proxmox cloud-controller-manager now runs v0.16.1, and the Proxmox CCM and CSI driver images are pinned by digest. The v0.15.0 tag referenced before was re-pushed upstream to a build that panics on start, which left every new Proxmox worker with the uninitialized taint.
```
2026-10-29 11:20:22 +03:00
Marian Koreniuk f7913329b2 fix(kubernetes-nodes): plan autoscaler capacity on the cores the VM actually gets
On the kubevirt substrate resources.cpu is written into domain.cpu.cores
as spelled and decoded as YAML, so "010" gives the VM eight cores, while
the cluster-autoscaler capacity annotation parsed the same string as
decimal ten. The autoscaler then planned 9000m on a node that offers
7000m.

Keep the YAML-decoded value the range check already computes and start
the capacity annotation and the reserved-CPU check from it on kubevirt.
The kubelet reservations stay as they are: the worker machineconfig
sizes them from the decimal string, and the TalosConfigTemplate name is
a hash of that config, so changing them would roll every leading-zero
pool once. The annotation therefore subtracts the reservations the
machineconfig really carries (2 x 500m for "010") from the cores the VM
really has, which is the node's real allocatable. Neither template name
changes for any spelling, so no pool rolls. Proxmox parses the quantity
as decimal on both sides and is unchanged. The field description says
the value is decoded as YAML and that the reservations still read it as
a decimal.

This closes #4554 without the migration and the render-time refusal it
proposes: a refusal breaks stored pools on their next upgrade, a schema
pattern breaks updates of existing CRs, and a platform-wide migration of
every KubernetesNodes and nodeGroups entry is a lot of blast radius for a
spelling nobody has reported in a live cluster.

Closes #4554

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-10-28 21:04:27 +01:00
Marian Koreniuk 780b068df0 fix(kubernetes, proxmox-csi): pin the Proxmox CSI driver images by digest
The CSI controller in the management cluster was referenced by the bare
tag ghcr.io/sergelogvinov/proxmox-csi-controller:v0.20.0, and the node
plugin in the tenant by the vendored chart's default, its appVersion as
a bare tag. That registry is the one whose proxmox-cloud-controller-
manager:v0.15.0 was re-pushed to a build that panics, so a tag there is
not a statement of what runs.

Both now carry the v0.20.0 digest: the controller through
images/proxmox-csi-controller.tag, like the cloud-controller-manager, and
the node plugin through repository and tag in the proxmox-csi values,
which also makes it visible to hack/lib/image-refs.sh and therefore to
the release multi-arch gate. Both digests are indexes for linux/amd64
and linux/arm64.

Pinning takes the version out of the vendored chart's hands: a chart
bump would bring a new appVersion while both pins kept naming the old
release, and the controller and the node plugin would drift apart from
the templates and from each other without anything noticing.
hack/proxmox-csi-driver-pin_test.bats fails unless the chart's
appVersion and both pins name the same release.

The sig-storage sidecars stay on tags, as they do for the KubeVirt CSI
driver: registry.k8s.io publishes them through image promotion, which
does not overwrite a tag.

Assisted-by: LLM
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
2026-10-28 18:38:35 +01:00
Aleksei Sviridkin 0b3f5dd7e1 fix(keycloak)!: make the KMS proxy keep its keys across restarts (#4818)
## What this PR does

Two fixes for the optional KMS proxy of the platform Keycloak
(`encryption.*`).

The default static backend never started. The chart stored 32
alphanumeric characters as the KEK, but the proxy wants base64 of 32
bytes and exits with `config: KKP_KEK must decode to 32 bytes (got 24)`.
The KEK is now generated with `randBytes`, and a stored key of the old
form is replaced. The proxy never ran with such a key, so nothing was
encrypted under it. Any other stored key is kept if the proxy accepts it
and fails the render otherwise, so the chart never overwrites key
material. The KEK Secret now carries `helm.sh/resource-policy: keep`:
the DEK set is wrapped with it, so it has to survive turning encryption
off or reinstalling the package.

Without `deksetSecretName` the proxy mints a new DEK set on every start
(`buildCipher` in `cmd/proxy/main.go` of keycloak-kms-proxy) and keeps
it only in memory. After the first restart of the proxy pod it cannot
decrypt anything it wrote before. The values comment said this is safe
on a fresh DB, it is not. The chart now refuses to render the proxy
without a DEK set unless `fields: disabled`. Passthrough still works
without one, so the static backend can get its KEK Secret created before
the DEK set is minted with it.

Breaking: a release with encryption on and no `deksetSecretName` fails
to render on upgrade until a DEK set is configured. Such a setup loses
its data on the next proxy restart anyway, so failing loudly is the
point. A stored static KEK the proxy cannot decode fails the render too,
instead of being replaced.

Checked with `helm unittest` (100 tests, the new ones fail on the old
template) and by starting keycloak-kms-proxy v0.2.3 with the KEK the
chart renders now. With the KEK from the old template it fails as quoted
above.

### Screenshots

No UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/pull/742 documents the
setup with a DEK set
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The other repositories do not touch the keycloak chart values.

### Release note

```release-note
fix(keycloak)!: The KMS proxy of the platform Keycloak now requires `encryption.deksetSecretName` unless `encryption.fields` is `disabled`. Without it the proxy generated a new data key on every start and could not read data encrypted before a restart. The static backend now generates a KEK the proxy accepts; before, the proxy could not start with it. The KEK Secret is no longer deleted when encryption is turned off or the package is removed.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Configuration**
* Enabling field encryption requires a persistent DEK set. Chart
rendering reports an error if it is missing; setting fields to
`disabled` does not require one.
* For the static backend, a valid stored KEK is reused, and a missing or
legacy-format KEK is replaced. An unusable stored KEK causes rendering
to fail.
* The KEK Secret is retained when encryption is disabled or the package
is removed.
* **Documentation**
* Clarified DEK set requirements and sharing across starts and replicas,
and updated guidance on configuring multiple proxy replicas.
* Warns against deleting the retained KEK while encrypted values remain.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-27 11:52:17 +03:00
Mattia Eleuteri f21abe0d35 test(mariadb-operator): run the package suite from make test
The helm-unittest sweep reaches a package only through its make test
target, so the suite added with the cluster-name fix would otherwise
never run in CI.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-10-27 09:12:25 +01:00
myasnikovdaniil b36628661f fix(ingress-nginx): drop the admission-webhook policy a rebase brought back (#5088)
## What this PR does

Rebase behind [#3931](https://github.com/cozystack/cozystack/pull/3931)
brought back namespaced `CiliumNetworkPolicy` for admission webhook,
which [#3997](https://github.com/cozystack/cozystack/pull/3997) deleted
in favour of clusterwide policy. It renders only on
`admissionWebhookPolicy.enabled` (true by default) and never checks that
Cilium API exists, so on cluster without Cilium every tenant with
`ingress: true` fails to install with `no matches for kind
"CiliumNetworkPolicy"`.

On cluster with Cilium it repeats deny that clusterwide policy already
has, and also renders for child tenants where webhook is disabled.

This PR removes the template with its test and adds suite that renders
whole package without Cilium API and fails if any Cilium resource comes
out. Existing capability suite selects only clusterwide template, so it
could not catch this file coming back. Clusterwide policy and
`admissionWebhookPolicy` value are unchanged.

Fixes #4706

On cluster without Cilium nothing in this chart restricts webhook port.
Removed policy could not do it either because it needs Cilium CRD to
install at all, and clusterwide policy already renders only where Cilium
API is served, so no protection is lost, such cluster now just installs
ingress instead of failing. Only `tenant-root` runs webhook, child
tenants set `admissionWebhooks.enabled: false`. Restricting webhook port
on non-Cilium clusters is separate design question, not part of this PR.

Checked `helm template` with and without
`cilium.io/v2/CiliumClusterwideNetworkPolicy` in api versions: without
it no Cilium object renders, with it only clusterwide policy, same as
before. New suite fails on tree before this change.

### Screenshots

Not a UI change.

### Downstream repositories

Only template and tests under `packages/system/ingress-nginx` changed,
`admissionWebhookPolicy` value kept, nothing from trigger map in
`docs/agents/contributing.md` matches.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(ingress-nginx): remove the namespaced admission-webhook CiliumNetworkPolicy that was re-added by mistake. It was rendered without checking that the Cilium API exists, so on clusters without Cilium every tenant ingress failed to install with `no matches for kind "CiliumNetworkPolicy"`. The clusterwide admission-webhook policy is unchanged and still renders only where the Cilium API is served.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Configuration**
* The ingress-nginx chart no longer generates an admission webhook
network policy. Deployments that relied on this policy to restrict
access to the webhook will need to configure equivalent traffic controls
separately.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-27 11:47:30 +05:00
Mattia Eleuteri c70f83b375 fix(mariadb-operator): derive the cluster name from the platform domain
The package pinned clusterName to cozy.local, so on a cluster installed
with any other networking.clusterDomain the operator resolved its own
MariaDB pods as <name>.<ns>.svc.cozy.local and got NXDOMAIN. Replication
between replicas was never configured, and User, Database and Grant
resources failed to reconcile, with the operator only retrying at a
verbose log level.

clusterName is a vendored subchart value, out of reach of the _cluster
lookup that package templates use, so the platform now carries it under
the subchart key in the shared values Secret, the same route KEDA and
the OpenSearch operator already take. The literal in the package stays
as the default for a render without that Secret.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-10-26 22:11:09 +01:00
myasnikovdaniil 7dc5c0b5e6 fix(marketplace): do not install a connected repository's components on connect (#5071)
## What this PR does

Connecting a marketplace repository materialized its `PackageSource`s
and then auto-created a tap-managed `Package` per source. The Package
reconciler installs a `HelmRelease` for every install-marked component
of the default variant, and Flux runs those under its `flux` service
account, which is bound to `cluster-admin` with no impersonation
(`--default-service-account` is not set, and neither `internal/operator`
nor `pkg/registry/apps` set `serviceAccountName` on the HelmRelease). A
publisher's install chart therefore ran as `cluster-admin` on connect,
with no operator step, contradicting the documented behaviour that
tapping installs nothing. The `privileged` flag is a prompt to the admin
and does not protect against a publisher acting in bad faith.

This removes the on-connect auto-registration (`ensureRegistration`).
Connecting now only materializes `PackageSource`s. Installing an app is
an explicit `cozypkg add`, which becomes the sole creator of a
`Package`; every other `Package` reference in the operator is a read or
a watch. Removing the call also fixes the second effect of the same
path: a `Package` the admin removed with `cozypkg del` is no longer
re-created on the next source revision.

A second commit reconciles the texts this behaviour change made wrong:
they promised that tapping registers a repository's apps in the catalog
and runs its charts. Reworded the `cozypkg tap`/`untap` help, the tap
success output (it now prints a `cozypkg add <name>` line per
materialized PackageSource), the `cozypkg init` scaffold README, the
packages-index trust model, and the dashboard Repositories banner and
per-repository empty state.

Accepted consequence: a connected repository's apps appear in the
catalog only after an admin runs `cozypkg add`, since the dashboard
catalog is computed from `ApplicationDefinition`s. Install from a
connected repository is CLI-only for now; the console connects and
disconnects repositories but does not create a `Package`. Populating the
catalog from `PackageSource` metadata without an install is the
long-term shape in #4359 and is out of scope here. The
privileged-confirmation gaps #4358 and #4359 stay open.

Upgrade note: a `Package` that Cozystack 1.7.0 auto-created on connect
stays after the upgrade, and the privileged-confirmation gate still
applies to it, so this is not a regression for existing installs. For a
newly connected repository no tap-labelled `Package` is created, so the
gate does not come into play on that path. An operator who does not want
a 1.7.0 auto-created package can remove it with `cozypkg del`.

Addresses #4978.

### Screenshots

Dashboard text-only change (Repositories banner and per-repository empty
state). No layout change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff. The diff is operator logic plus CLI/dashboard/doc text; it matches
no concrete trigger (no package, API, schema, tooling, annotation,
metric or commit-convention change). The dashboard console is vendored
in this repo (`packages/system/dashboard`), so its text change travels
in this PR. Leaving every box empty rather than ticking "No downstream
repository is affected".

### Release note

```release-note
fix(marketplace): connecting a marketplace repository no longer installs its components. A connected repository's apps are materialized as PackageSources but not installed until an explicit `cozypkg add`; install from a connected repository is CLI-only for now. Previously, connecting auto-created a Package whose install-marked components were deployed by Flux under its cluster-admin service account. A Package that Cozystack 1.7.0 auto-created on connect stays after upgrade and can be removed with `cozypkg del`.
```
2026-10-26 21:10:45 +05:00
myasnikovdaniil 7215b02d38 feat(keycloak): load additional CA files from Secret or ConfigMap keys (#5039)
## What this PR does

Closes #3823.

With the stock Cozystack chart, Keycloak cannot connect to an LDAPS
server, external identity provider or SMTP server signed by a private CA
because the chart cannot deliver a CA file to its truststore. Add
`truststores`, a list of existing Secret/ConfigMap references in the
release namespace, with one selected key per entry (`ca.crt` by
default).

```yaml
truststores:
  - configMapName: freeipa-ca
    key: ca.crt
  - secretName: partner-ca
    key: certificates.pem
```

Each key is mounted read-only in a separate `cozystack-<index>`
subdirectory of `/opt/keycloak/conf/truststores`. Keycloak discovers it
automatically; the chart adds no environment override, Kubernetes API
permission or arbitrary volume path. Existing image certificates, Java
default roots, themes and `extraEnv` keep their behavior. References are
mandatory: a missing object or selected key prevents the pod from
starting. Helm validates the reference shape and safe key paths, but
cannot inspect external certificate contents.

The trust decision belongs to platform administrators and applies to the
entire Keycloak server, across all realms. Write access to a referenced
object therefore permits changing that server's trust. If HTTPS client
authentication is set to `request` or `required` without a separate
`https-trust-store-file`, these CAs also verify incoming client
certificates; configure that file explicitly when incoming mTLS requires
a different trust boundary. Use public certificates only (PEM bundles or
unencrypted PKCS12 without private keys). This does not disable TLS
hostname verification. Updating an object's contents does not restart
Keycloak or reload its truststore: restart both pods after rotation, or
use a versioned object name to trigger a rollout.

Validation:

- All 113 Keycloak Helm tests pass, including 21 new cases for reference
wiring, read-only mounts, selected-key isolation, themes, PKCS12
filenames, defaults and invalid input.
- Boundary tests reject source names and keys of 254 characters and the
single-dot key, and accept a source name and key of exactly 253
characters. Removing any of the three guards makes its regression test
fail; changing either length comparison to reject 253 also fails the
boundary test.
- Empty-default render matches a clean upstream/main worktree after
normalizing the existing randomly generated bootstrap password. Four
mutations (wrong mount path, writable mount, wrong selected key and
removed validation) each turn the new suite red.
- Enabled and default manifests render successfully; enabled YAML parses
with yq. `git diff --check` passes.
- Runtime check with official Keycloak 26.6.3 in isolated Docker:
synthetic LDAPS succeeds with a CA loaded automatically through an
emulated Kubernetes AtomicWriter symlink layout (HTTP 204 from the LDAP
connection-test Admin API); no custom CA fails with PKIX, and a trusted
CA with a mismatched SAN fails hostname verification (both HTTP 400).
KC_TRUSTSTORE_PATHS was unset. This checks TLS establishment, not a
complete federation bind/failover or deployment to a Kubernetes cluster.
- `make generate` has no target in this system chart; no generated
schema/API files are involved. `make show` requires the unavailable
cozyhr CLI and a live release, so rendering uses `helm template`. `helm
lint` reports the same pre-existing undeclared cozy-lib dependency on
both this branch and clean main.


### Screenshots

Not applicable; no UI changes.

### Downstream repositories

The website guide currently says CA mounts are unavailable, so its
`next` documentation needs updating once this ships. Other downstream
trigger-map entries do not apply: this only adds an opt-in value to the
system Keycloak chart, with no app schema, platform configuration, CRD,
tooling, naming or node-contract changes.

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/issues/765
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(keycloak): load additional CA files from existing Secret or ConfigMap keys through the new truststores value, enabling TLS to private LDAPS, identity-provider and SMTP endpoints.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

- **New Features**
- Added support for configuring Keycloak truststores from existing
Secrets or ConfigMaps.
- Supports PEM bundles and unencrypted PKCS12 certificates, with
optional key selection and read-only mounts.
- Truststore certificates apply server-wide, including incoming
client-certificate validation when no separate HTTPS truststore is
configured.
- Invalid truststore configurations produce clear errors. Missing
objects or keys prevent startup; certificate rotations require pod
restarts unless the referenced object name changes.
- **Tests**
- Added coverage for default behavior, multiple truststore sources,
theme mounts, explicit paths, and invalid configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-26 19:57:18 +05:00
IvanHunters 2f39e304f0 fix(marketplace): reconcile tap texts with no-install-on-connect
Removing on-connect auto-registration made several user-facing and doc texts
wrong: they promised that tapping registers a repository's apps in the catalog
and runs its charts. Tapping now installs nothing, so the texts told operators to
wait for apps that never appear.

Reword the `cozypkg tap`/`untap` help, the tap success output (now printing a
`cozypkg add <name>` line per materialized PackageSource), the `cozypkg init`
scaffold README, the packages-index trust model, the dashboard Repositories
banner and per-repository empty state, and the stale re-registration code
comments. No behaviour changes beyond the added `cozypkg add` hint in the tap
output.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-26 15:18:14 +03:00
Andrei Kvapil 448b12657b fix(foundationdb): stop offering the 7.1 line
The FoundationDB operator no longer carries the 7.1 client, so it can
neither manage nor upgrade a 7.1 cluster. The version map offered v7.1
all the same; it now lists the lines the operator carries, 7.3 and 7.4,
which is also what hack/update-versions.sh reads from the operator
chart.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil d62056ef03 fix(foundationdb)!: pick the version from a fixed list
cluster.version took any string and passed it to the operator as the
FoundationDB version, so a tenant chose the image tag rather than a
supported release line. The application now takes version (v7.4, v7.3,
v7.1) and maps it to a tag through files/versions.yaml, the same way
postgres and mariadb do.

The lines are the ones the pinned operator ships client libraries for,
which hack/update-versions.sh reads from the operator chart. The map
keeps v7.3 on 7.3.63, the previous default, so a cluster that never set
the version renders exactly as before.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil 941b50c675 fix(kubernetes-nodes)!: drop the kubectl image override
images.kubectl picked the image of the talos-reconcile and pre-delete
unpin Jobs. Both run in the management cluster under service accounts
that read the pool's Talos and cluster CA secrets and patch its CAPI
objects, so a tenant-chosen image ran with access the apps API never
grants. The image now always comes from images/kubectl.tag.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:14 +05:00
Andrei Kvapil af20694359 fix(kubernetes)!: drop the helper image overrides
images.waitForKubeconfig, images.kubectl and images.talosCsrSigner
ran whatever image a tenant named inside the management cluster: the
csr signer as a sidecar of the tenant control plane with the Talos CA
key mounted, the other two in pods holding service account tokens for
CAPI, KubeVirt and Cilium objects in the tenant namespace. A tenant
never gets that access through the apps API, and the platform should
not hand it over through an image field.

The images now always come from the bundled tag files. A release that
still carries the keys renders exactly as one without them.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:14 +05:00
Andrei Kvapil 6f8a04c11a fix(opensearch)!: drop the custom image override
images.opensearch put any image a tenant named into the
OpenSearchCluster, so the platform ran that binary in its own cluster
and the version map stood behind nothing. The application offers a
choice of version; the image now always comes from that version.

A release that still carries the key renders as if it were absent: the
operator rolls the nodes onto the image of the selected version.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:14 +05:00
Федор Батоногов d607b68a0f test(keycloak): pin truststore name and key constraints
Boundary cases must catch removal of the constraints and the error at
253 characters.

Assisted-by: LLM
Signed-off-by: Федор Батоногов <f.batonogov@yandex.ru>
2026-10-23 04:58:26 +03:00
Myasnikov Daniil 8c1d2dbee2 fix(ingress-nginx): drop the admission-webhook policy a rebase brought back
A rebase re-applied the namespaced CiliumNetworkPolicy that the move to
source-side clusterwide policies had deleted, so two policies now guard
the same webhook port and the older one is the worse of the two.

It is gated only on admissionWebhookPolicy.enabled, which defaults to
true, and never checks that the Cilium API exists. On a cluster without
Cilium every tenant ingress release therefore fails to install with
`no matches for kind "CiliumNetworkPolicy"`.

On a Cilium cluster it repeats the target-side deny the clusterwide
policy already carries. It also renders for child tenants, whose
controller runs with the admission webhook disabled and for which the
clusterwide policy renders nothing, so there it denies a port nothing
listens on and is the only thing that puts a cluster-spanning peer onto
the controller endpoints. That peer is enumerated per pod identity, the
coupling to namespace churn the clusterwide policy was written to
contain.

The clusterwide policy and the admissionWebhookPolicy value it reads are
left as they are. The existing capability suite selects only the
clusterwide template, so it could not have noticed this file coming
back; a suite with no template filter now fails if any Cilium kind
renders while the Cilium API is absent.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-22 19:33:06 +00:00
Федор Батоногов 04a3f4be56 docs(keycloak): clarify incoming mTLS trust scope
The system truststore also verifies HTTPS client certificates unless a
separate HTTPS truststore is configured. Operators adding an LDAP CA
need to know that this can extend inbound TLS trust as well.

Assisted-by: LLM
Signed-off-by: Федор Батоногов <f.batonogov@yandex.ru>
2026-10-21 16:04:18 +03:00
Myasnikov Daniil b83cc087cc docs: move the repo-only guides onto the website
Seven guides in docs/ were written next to the code and left in the
source tree, where no operator looks. The website carries them now, so
a second copy here only lets the two drift again: the Backup Classes
page had already fallen behind its original.

docs/release.md, docs/deprecating-applications.md and the two SeaweedFS
runbooks in docs/operations stay. A migration, a bats suite, a workflow
or AGENTS.md reads each of them by path, which is the test for keeping
a document in this repository.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-21 17:50:46 +05:00
Федор Батоногов 8bc8e429e6 feat(keycloak): trust additional CA files from existing objects
Private LDAPS, identity-provider and SMTP endpoints need their CA in the
Keycloak truststore. Environment variables alone cannot supply that file,
and manual StatefulSet patches are overwritten by Flux reconciliation.

Assisted-by: LLM
Signed-off-by: Федор Батоногов <f.batonogov@yandex.ru>
2026-10-21 15:50:35 +03:00
myasnikovdaniil d78897f842 fix(cluster-api): resync capi-operator every minute (#4939)
## What this PR does

After an upgrade restarts capi-operator, a Cluster API provider can keep
`Ready=False` for about 20 minutes while its controller is already
running, and `capi-providers-core` times out waiting for the
CoreProvider. The bundled operator v0.19.0 drops create events of
provider Deployments it finds running after a restart
(kubernetes-sigs/cluster-api-operator#1031), so it looks at them again
only when one changes or on informer resync, and the first resync after
a restart comes two sync periods later, about 20 minutes with the
default 10m.

This sets the operator `--sync-period` to 1m. On a test cluster a
provider left `Ready=False` across an operator restart recovered 2m02s
after the operator took its lease, the operator with the default period
had not recovered after more than 4 minutes. A resync is served from the
operator cache and writes nothing while providers are in sync: no
provider object was updated over five resyncs, about 15 log lines a
minute, 2m CPU and 26Mi memory.

The vendored chart has no value for this flag, so a small package patch
(reapplied by `make update`) adds one, and a test holds the rendered
argument. The upstream fix is in cluster-api-operator v0.27.0, bumping
to it is a bigger change and is left for a later release.

Mitigates #4926.

### Downstream repositories

- [x] No downstream repository is affected by this change

### Release note

```release-note
fix(cluster-api): resync capi-operator every minute so a provider left not ready across an operator restart recovers in about two minutes instead of about twenty
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* The Cluster API operator now refreshes its cached state every minute.
After an operator restart, provider readiness status can recover sooner
instead of remaining stale for a longer period.
* Providers that are already in sync are not written back to the API
during these cache refreshes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-21 15:48:14 +05:00
myasnikovdaniil 3cd261277e fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit (#4941)
## What this PR does

Fixes a size regression from #4929. Helm stores each release revision in
one Secret and the apiserver refuses a Secret over 1 MiB. With all
default Strategy CRs rendered the `backupstrategy-controller` release
crosses that limit, so the forced upgrade that creates them fails with
`Secret "sh.helm.release.v1.backupstrategy-controller.vN" is invalid:
data: Too long: may not be more than 1048576 bytes`, the CRDs stay and
the default strategies never appear. The release was already within
about 1 KB of the limit before #4929, which added tests to the chart.

The strategy CRDs are 2.2 MB of YAML and Helm stores them twice per
revision, as chart files and in the rendered manifest. Almost all of it
is the upstream `PodTemplateSpec` schema inlined by the Altinity, Job
and FoundationDB strategies. Kafka, RabbitMQ and Redis already mark
their `Template corev1.PodTemplateSpec` as `Schemaless` and
`PreserveUnknownFields`, this PR does the same for
`AltinitySpec.Template`, `JobSpec.Template` and
`FoundationDBTemplate.BackupDeploymentPodTemplateSpec`. Go types are
unchanged and every description is kept. The CRDs go from 2.2 MB to 317
KB and the largest release revision from over the limit to about 250 KB,
24% of it. The size rationale now lives once in the package doc, the
field docs of the six schemaless templates share one short line.

What changes for users: the apiserver no longer validates these three
fields and `kubectl explain` no longer lists children under them
(`kubectl explain pod.spec` still documents them), same as for Kafka,
RabbitMQ and Redis today. Unknown keys are stored instead of pruned, the
typed decode in the drivers drops them either way. Labels and
annotations under `template.metadata` were rejected by the strict v1.6
schema and are now accepted and kept. A Go template in a non-string slot
never worked, it was rejected on apply, now the apiserver accepts it and
the typed decode in the controller is what rejects it, as for the other
three kinds (I did not run that case live).

The drivers read these fields through the non-strict client and
`internal/template` rewrites only string leaves, nothing under
`internal/backupcontroller` relies on pruning. Go tests for
`internal/backupcontroller`, `internal/template` and `api` pass.

`hack/strategy-crd-schemaless.bats` fails on any strategy CRD that
inlines a pod schema and prints the property path and the two markers to
add. `hack/helm-release-size.bats` estimates the stored release of every
chart under `packages/` the way Helm encodes it (release JSON, gzip,
base64) and fails above 90% of the limit, plus the worst case of this
chart. The estimate runs 1 to 2 percent above real Secrets. A chart has
to render offline to be measured with its manifest: the 36 charts that
do not are listed in `NO_OFFLINE_RENDER` and measured on files and
templates only, any other render failure fails the test, and so does a
listed chart that starts rendering.

Checked on a throwaway kind cluster: v1.6.4 upgrades to this branch and
the forced follow-up creates all default strategies, a fresh install
works, strategy objects created under the old schema stay valid, a Job
strategy with templated args and pod template labels is accepted.

### Downstream repositories

- [x] No downstream repository is affected by this change

### Release note

```release-note
fix(backups): the backupstrategy-controller release no longer fails with "Secret ... Too long" on the upgrade that creates the default backup strategies, the Altinity, Job and FoundationDB strategies store their pod templates schemaless like Kafka, RabbitMQ and Redis
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Backup strategy templates for Altinity, FoundationDB, Jobs, Kafka,
RabbitMQ, and Redis now preserve fields not defined in the API schema.
The API server no longer validates these templates against the pod
schema; validation occurs when the driver creates the workload. This
lets strategies include pod-template fields without requiring those
fields to be listed in the strategy schema.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-21 14:27:15 +05:00
myasnikovdaniil f69958b15d docs: move the operator guides out of docs/ to where readers find them (#3780)
## What this PR does

Five documents under `docs/` were operator facing guides written
alongside the feature PRs that added the code, and left in the source
tree where no user looks. They are on the website now, so this deletes
them here and repoints every reference.

```
docs/oidc-tenant.md          -> kubernetes/oidc-authentication
docs/oidc-grafana.md         -> operations/services/monitoring/oidc-authentication
docs/gpu-vgpu.md             -> virtualization/vgpu
                              + operations/troubleshooting/gpu-operator-host-driver
docs/vm-external-vlan.md     -> networking/vm-external-vlan
docs/hubble-observability.md -> networking/hubble
```

`docs/vm-external-vlan.md` needed splitting, not moving. Its "Staging
the reference CNI plugins" section is a contract,
`hack/cni-plugins-staging-contract.bats` greps six literal sentences out
of it and asserts them against the multus Dockerfile, the daemonset
manifest and the platform values. That half is package documentation so
it moves verbatim into a new `packages/system/multus/README.md` and the
suite reads it there, the operator guide goes to the website. The bats
messages name the README now, so a future failure points at the file
that has to change.

References repointed in the `@param` blocks, template comments and test
comments that named the guides. The generated files come from `make
generate` with cozyvalues-gen v1.7.0, the version pinned in
`.github/workflows/pre-commit.yml`. Where a reference points at text
that exists only on `next/` of the website (the aggregated API servers
section of the tenant OIDC guide), the comment names the page and the
section instead of a `v1.6` URL, which would not contain it.

Two changelog links in `docs/changelogs/v1.6.0.md` repointed as well.
The wording is untouched, only the location of the documents it names
has changed.

One bit of process framing dropped on the way past, per #3777:
`bundles/system.yaml` said GrafanaAdmin promotion was "out of scope for
Phase 1" when it is simply not implemented.

Tests: 10 cni-plugins contract, 47 install-wait, 346 kubernetes chart,
22 multus chart, 301 monitoring chart, rd-presets-check.

### Downstream repositories

- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: cozystack/website#648 (the four guides),
cozystack/website#754 (edits made to the originals in `docs/` after the
import), cozystack/website#647 (grafana guide made reachable in v1.6),
cozystack/website#596 (tenant kubectl guide, rebased)

Walked the rest of the trigger map against the diff. No provider
follow-up: the kubernetes `values.schema.json` diff is two `description`
strings and the monitoring one is one, with no field added, removed or
renamed, no enum, no default, and no `kind`, `plural` or
`release.prefix` change. `packages/core/platform/values.yaml` changed in
a comment only and no key, so the hand written platform-package table on
the site is unaffected.

Please merge after cozystack/website#754 and cozystack/website#647. The
`@param` URLs only resolve once those pages deploy, and
cozystack/website#754 carries the edits made to the guides since the
import, which deleting them here would lose.

### Release note

```release-note
docs: operator guides for tenant Kubernetes OIDC, Grafana OIDC, NVIDIA vGPU, VM external VLAN and Hubble now live on cozystack.io instead of the docs/ directory of this repository, and in-repo references point there. The CNI plugin staging contract that shared the VLAN guide moved to packages/system/multus/README.md.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated configuration and test references to link to the published
Cozystack documentation.
* Added Multus guidance covering staged CNI plugins, configuration,
troubleshooting, and rollout behavior.
* Removed outdated local guides for GPU/vGPU, Hubble observability, OIDC
authentication, and external VM VLANs.
* Clarified that server-level Grafana administrator promotion is not
currently supported.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-21 12:36:29 +05:00