Commit Graph
7777 Commits
Author SHA1 Message Date
Marian Koreniuk 4d7d336efc fix(e2e): tell a failed pool existence read from an absent pool release
The failure-path report checked the pool release with `get hr -o name`
and discarded the exit status and stderr, so a timeout, a kill, a
Forbidden or a refused connection all printed that the run ended before
the worker pool was applied. That is the conflation the guard wrapper
exists to refuse, on the path where a slow apiserver is most likely.

Read it with --ignore-not-found instead: exit 0 with empty output is an
absent release, exit 0 with a name is one to classify, and any non-zero
exit is a failed read named through _cozy_guard_read_failure.

The pool reads also skipped the report's own deadline and checked only
the phase clock, which says yes on every path that opens no phase.
Check the deadline before the existence read, like the listing and the
addons do, and name what was not read.

The stub's ABSENT marker now answers the flag the way kubectl does. New
cases pin the failed read and the deadline; the two budget cases are
re-timed for the extra clock reading.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-10-28 18:45:50 +01:00
Marian Koreniuk 28d9d352e1 test(kubernetes): name the Proxmox fixture's node pool <cluster>-<pool>
kubernetes-nodes names its release kubernetes-nodes-<CR name> and refuses
to render one that does not start with kubernetes-nodes-<spec.cluster>-
(_helpers.tpl, since fd14126be). The Proxmox fixture named its
KubernetesNodes "md0", so helm install of the pool failed, no Machine was
ever created, and the first real run of the lane on a self-hosted stand
timed out on the Machine assert after 20 minutes.

Name the pool k8s-e2e-md0 and add a fixture test that fails on any name
the chart would refuse.

Assisted-by: LLM
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
2026-10-28 18:44:04 +01:00
Marian Koreniuk 780b068df0 fix(kubernetes, proxmox-csi): pin the Proxmox CSI driver images by digest
The CSI controller in the management cluster was referenced by the bare
tag ghcr.io/sergelogvinov/proxmox-csi-controller:v0.20.0, and the node
plugin in the tenant by the vendored chart's default, its appVersion as
a bare tag. That registry is the one whose proxmox-cloud-controller-
manager:v0.15.0 was re-pushed to a build that panics, so a tag there is
not a statement of what runs.

Both now carry the v0.20.0 digest: the controller through
images/proxmox-csi-controller.tag, like the cloud-controller-manager, and
the node plugin through repository and tag in the proxmox-csi values,
which also makes it visible to hack/lib/image-refs.sh and therefore to
the release multi-arch gate. Both digests are indexes for linux/amd64
and linux/arm64.

Pinning takes the version out of the vendored chart's hands: a chart
bump would bring a new appVersion while both pins kept naming the old
release, and the controller and the node plugin would drift apart from
the templates and from each other without anything noticing.
hack/proxmox-csi-driver-pin_test.bats fails unless the chart's
appVersion and both pins name the same release.

The sig-storage sidecars stay on tags, as they do for the KubeVirt CSI
driver: registry.k8s.io publishes them through image promotion, which
does not overwrite a tag.

Assisted-by: LLM
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
2026-10-28 18:38:35 +01:00
Marian Koreniuk 05a347e6f6 fix(kubernetes): run proxmox-cloud-controller-manager v0.16.1, pinned by digest
The Proxmox CCM was referenced by its bare v0.15.0 tag, and that tag no
longer names the image it named when the template was written. It was
re-pushed on 2026-09-27 (index sha256:d81702998ae1, replacing the
2026-08-06 build sha256:fbc553921145) to a binary whose gitVersion is
"0.15.0" without the leading v. component-base v0.37 parses that string
in the init() of its legacy metrics registry and panics:

  panic: version string "0.15.0" doesn't match expected regular
  expression: "^v(\d+\.\d+\.\d+)"

so the Deployment crash-loops, no Node ever loses the
node.cloudprovider.kubernetes.io/uninitialized taint, and workloads on a
new worker stay Pending. Any node that pulls the tag fresh gets it, which
is every fresh install.

v0.16 builds with goreleaser and stamps gitVersion as v{{.Version}}. The
configuration this template writes is read unchanged: the cluster entry
moved to go-proxmox-pool's ClusterConfig with the same url, insecure,
token_id, token_secret and region keys, and features.provider: capmox
still writes proxmox://<SMBIOS UUID>, the providerID capmox puts on the
Machine.

The reference now lives in images/proxmox-cloud-controller-manager.tag
as tag plus digest, the shape the chart already uses for the KubeVirt
cloud provider, so a moving tag can no longer change what runs.

Assisted-by: LLM
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
2026-10-28 18:38:35 +01:00
Marian Koreniuk 0e77e2b10f fix(kubernetes-nodes): name instanceType in the partial-spec refusal
On a pool with no instanceType, "set both resources.cpu and
resources.memory, or neither" sent the user straight into the next
refusal, since neither sizing source is refused as well. The message now
ends in "or neither and set instanceType" on such a pool; a pool sized by
an instancetype keeps the old wording.

The tests anchor both spellings, so the suffix cannot leak onto, or drop
off, either kind of pool.

Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
2026-10-28 17:06:39 +01:00
Aleksei Sviridkin ae9e7c4c64 test(kubernetes): pin RetryOnFailure on the Proxmox csi release (#4969)
The KubeVirt csi and cilium releases carry a helm-unittest pin on
`strategy.name: RetryOnFailure` for both actions, and the e2e
HelmRelease guard in `hack/e2e-chainsaw/_lib/run-kubernetes.sh` names
those two tests as load-bearing. `helmreleases/csi-proxmox.yaml` sets
the same strategy for the same reason and had no pin.

This adds
`packages/apps/kubernetes/tests/csi_proxmox_install_retry_test.yaml`, a
mirror of `csi_install_retry_test.yaml` rendered on the proxmox
substrate, and updates the guard comment that pointed at the gap. No
template change.

Verified with `helm unittest .` in `packages/apps/kubernetes` (32
suites, 349 tests).

Closes #4731


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Expanded automated coverage for Proxmox CSI installation and upgrade
retry behavior, including the retry interval and remediation retry
limits. The checks verify that retries continue according to the
configured settings during installation and upgrades. This update does
not change user-facing functionality.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-28 18:30:06 +03:00
Aleksei Sviridkin 30fc100e65 ci(e2e): end the Proxmox lane's SSH key with a newline (#5090)
## What this PR does

The Proxmox lane's Stage step writes `PROXMOX_SSH_KEY` to disk with
`printf '%s'`, so the key file ends exactly where the secret value does.
`gh secret set` strips the trailing newline from a value it reads from
stdin, which is the usual way to set a key (`gh secret set
PROXMOX_SSH_KEY < key`). OpenSSH refuses a private key whose last line
has no newline:

```
Load key ".../.ssh-pve": error in libcrypto
e2e-ro@10.60.0.1: Permission denied (publickey).
[20:45:58] FAILED: VM 124 has no scsi0
```

That is `vm-shape-on-the-hypervisor`, the first test that reaches the
hypervisor over SSH. Every run so far stopped before it, so nothing had
exercised the key. It showed up on a self-hosted stand once the CCM fix
in #4917 let the lane past `tenant-and-cluster`.

The step now writes the key with `printf '%s\n'`. If the secret kept its
own newline, the file ends in a blank line, and OpenSSH accepts that.

### Testing

- `hack/proxmox-teardown-contract.bats` gets a case that runs Stage with
a freshly generated key, once without a trailing newline and once with
one, and loads the result with `ssh-keygen -y`. It fails against the old
step on the first form. The whole file passes: 9/9.
- `bats-no-negated-assert`, `bats-strict-setup`, `bats-no-exit-trap`,
`bats-runner-coverage`: 63/63.
- Reproduced by hand on OpenSSH 9.6p1: the same key loads with one or
two trailing newlines and fails with `error in libcrypto` with none.

### Screenshots

Not a UI change.

### Downstream repositories

The diff changes one workflow step and adds a case to an existing bats
file. It moves or renames nothing under `hack/` and changes no make
target or `tags.yaml`.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Proxmox workflows now correctly stage SSH keys whether or not the key
ends with a newline.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-28 18:17:36 +03:00
Aleksei Sviridkin 3fe3028fa6 feat(kubernetes-nodes): trust a factory CA in the Proxmox template script (#4860)
## What this PR does

#3797 added `talos.imageFactoryCA`, so a pool's CDI import can verify a
self-hosted Image Factory served by an internal CA.
`hack/proxmox-talos-template.sh` (#4742) downloads the same image with
curl on the Proxmox node. curl there trusts only the system store, so
with that kind of factory it fails with `curl: (60) SSL certificate
problem: unable to get local issuer certificate`, which is exactly the
setup #3797 was written for.

`COZY_TALOS_FACTORY_CA` names a PEM file that holds those certificates,
the same blocks a pool puts in `talos.imageFactoryCA`. The program
writes the node's system bundle into its scratch directory, appends the
certificates after it, and passes the result to `curl --cacert`. They
are therefore trusted on top of the system store, the same way CDI
appends them to its pool, and a redirect to a publicly trusted host
still verifies. Nothing is installed into the node's own trust store.

The certificates are pasted into the program that runs as root on the
hypervisor, so the file is held to the rule
`cozy-lib.tls.validateCACert` applies to the chart value: it must
contain complete certificate blocks and nothing else. The script refuses
a file with a private key alongside the certificates, with trailing
text, or with an unclosed block, and it does so before printing
anything. It also refuses a CA given for a plain-http factory. On the
same principle, curl gets `--proto-redir =https` whenever a CA is set,
so a redirect cannot take the download off verified TLS halfway. Without
the variable, the program is byte-for-byte what it was.

Follows #3797 and #4742.

### Testing

- `bats hack/proxmox-talos-template_test.bats` (Bats 1.14.0) passes
15/15. Three tests are new: the program without a CA is unchanged; with
a CA, curl receives `--proto-redir =https` and the system bundle
followed by the certificates, and CRLF line endings are stripped;
malformed files and a CA for an http factory are refused. The new tests
go red against the script from #4742, against a validator that accepts
any line, and against a bundle that leaves out the system store.
- The same file passes 15/15 under mawk and under busybox awk.
- `bats-strict-setup`, `bats-no-negated-assert` and `bats-no-exit-trap`
are clean, and so is `shellcheck -s sh`.
- The bundle the program writes was checked against a local `openssl
s_server` with a certificate from a throwaway CA: curl verifies with the
bundle and fails with `(60)` without it.
- Against a local HTTPS server that redirects to http, curl follows the
redirect without the flag and stops with `(1) Protocol "http" not
supported or disabled` with it.
- Re-run after the rebase onto main 11b19635e, with #4742 merged: the
bats file is 15/15 under Bats 1.14.0, mawk and busybox awk, the three
Bats audits pass, `shellcheck -s sh` is clean and
`hack/check-commit-trailers.sh` passes.
- Run live on a Proxmox VE 9.2 node (curl 8.14.1) against a stand-in
factory: an HTTPS server with a certificate from a throwaway CA, reached
from the node through an ssh tunnel, given as `COZY_TALOS_FACTORY_URL`
with `COZY_TALOS_FACTORY_CA` naming that CA. Each run used a test VMID
and tags no pool selects, and the template was destroyed afterwards.
- Without the CA, the program stops at curl with `(60)` before `qm
create`.
- With the CA and a factory that redirects to http, curl stops with `(1)
Protocol "http" disabled (in redirect)`, again before `qm create`.
- With the CA and a factory that serves the image itself, the template
is built (76 s): `agent: enabled=1`, `template: 1`, the given tags.
- With the CA and a factory that redirects to
`https://factory.talos.dev`, the template is built (53 s), so the system
store is still trusted after the redirect.
- Neither a CA file with a private key appended nor a CA for an http
factory gets past the local checks.
- After every run the node's `/etc/ssl/certs/ca-certificates.crt` has
the same sha256, and no scratch directory is left in `/var/tmp`.

### Screenshots

Not applicable, no UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change

The change is a `hack/` script and its test. No chart value, schema or
generated file changes.

### Release note

```release-note
feat(kubernetes-nodes): `hack/proxmox-talos-template.sh` accepts `COZY_TALOS_FACTORY_CA`, a PEM file with the CA of a self-hosted Image Factory, and trusts it on top of the Proxmox node's system store for the image download. A factory that needs `talos.imageFactoryCA` for CDI now works for Proxmox templates too.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* HTTPS downloads of Talos images can now use certificates from a
specified private CA file, alongside the Proxmox node’s trusted
certificates.
* Private CA files are validated, and downloads are restricted to HTTPS.
Without a specified CA file, downloads continue using the existing
system trust behavior.
* **Tests**
* Added coverage for private CA handling, certificate-file validation,
and HTTPS requirements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-28 18:17:05 +03:00
Andrey Kolkov 07301c562c fix(workloadmonitor): stop doubling the bucket- prefix in bucket Workload names (#5083)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

Fixes #5014.

A bucket's `Workload` was named `"bucket-" + <BucketClaim name>`, and a
Bucket application names its claim after its release, which already
starts with `bucket-`. A bucket an operator knows as `cozy-backups`
therefore appeared as `bucket-bucket-cozy-backups`, and that string is
the `workload` of the bucket's consumer in every billing usage report.

- **Naming.** The `bucket-` prefix is now added only when the claim name
does not already carry it. A Bucket application's claim
`bucket-cozy-backups` gives the `Workload` `bucket-cozy-backups`; a
claim without the prefix, such as `my-bucket`, still gives
`bucket-my-bucket`. In the tree today every monitored BucketClaim comes
from the Bucket application (Harbor's registry claim carries no labels
its monitors select, so it has no `Workload` on either side of this
change).
- **Migration of existing Workloads.** Nothing garbage-collects bucket
`Workload`s by name: they are owned by their BucketClaim and live as
long as it does. Without a migration the old `bucket-bucket-…` object
would stay next to the new one and the bucket would be metered twice.
When the new `Workload` is reconciled, the controller looks up the
legacy name and, only if the same BucketClaim (matched by UID) is its
sole owner, uses its recorded sizes as the starting point and then
deletes it, with UID and resourceVersion preconditions so a `Workload`
that took over the name in the meantime survives. If the legacy
`Workload` has other owners as well, it is another bucket's renamed
`Workload` that a rolled-back controller attached this claim to: the
controller takes neither its sizes nor the object and only drops its own
owner reference. A legacy `Workload` with no owners left (a claim
deleted with `--cascade=orphan` and recreated) that still carries this
monitor's label is deleted the same way, without taking its sizes. Sizes
already recorded at the new name win over the legacy ones only when the
claim already owned that `Workload`. Carrying the sizes over matters
because the controller deliberately keeps the last known size when the
object-store metrics are unavailable; a fresh `Workload` would otherwise
report no size until the next successful poll, which billing reads as an
empty bucket.
- **Never taking over another claim's Workload.** With Bucket
applications `x` and `bucket-x` in one namespace, the legacy name of the
first (`bucket-bucket-x`) is the new name of the second. The controller
never takes over a `Workload` that has owners when the claim is not one
of them (in practice another BucketClaim). While the new name is held
that way, it keeps metering the bucket under its legacy name, without
migrating, and emits a `BucketWorkloadNameTaken` warning on the monitor.
For Bucket applications the other claim renames its `Workload` away on
its own reconcile and the migration completes on the next pass. Only
when the legacy name is held as well — which for a claim without the
prefix, whose two names coincide, means a claim `x` outside the Bucket
application next to the application `x` — is the bucket skipped, and the
warning says it is not metered. Without this, whichever claim reconciled
first decided whether the second bucket ended up with the first one's
size or with no size at all. The same rule keeps two claims that map to
one name (`x` and `bucket-x`) from sharing a `Workload`.

**Rollback.** The migration is one-way. A controller rolled back to a
previous version recreates `bucket-bucket-<name>` next to the renamed
`bucket-<name>`, which its monitor keeps alive, so the bucket is metered
twice until the `bucket-<name>` Workloads of Bucket applications are
deleted by hand. Upgrading again after such a rollback converges without
deleting another bucket's `Workload` (see above).

**Known limitation.** Taking over an ownerless `Workload` keeps the
sizes recorded on it as the last known ones, as the controller always
has. If such an object was left behind by another bucket (for example
`kubectl delete bucketclaim --cascade=orphan`), its sizes count for the
new claim until the next successful size poll.

**Effect on billing, worth knowing before the upgrade:** `Workload`
names are the identity billing integrates over. A bucket's consumer
changes name once, at the upgrade, so a usage report whose window spans
the upgrade lists the bucket twice: under the old name up to the switch
and under the new one after it. The totals are right; the split is
cosmetic. Reports for windows entirely before or after the upgrade are
unaffected.

Tests (`internal/controller/workloadmonitor_controller_test.go`):

- `TestBucketWorkloadName`: the naming rule for a Bucket application
claim, a plain claim and a Harbor claim.
- `TestReconcileBucketApp_NoDoublePrefix`: a Bucket application gets
`bucket-cozy-backups` and no `bucket-bucket-cozy-backups`.
- `TestReconcileBucketApp_ReplacesLegacyWorkload`: an existing
`bucket-bucket-cozy-backups` owned by the claim is deleted, and its
recorded logical and physical sizes (no metrics endpoint in the test)
are carried over to the new `Workload`.
- `TestReconcileBucketApp_KeepsLegacyNameOwnedByAnotherClaim`: a
`Workload` under the legacy name owned by a different claim is left
alone and its size is not taken.
- `TestReconcileBucketApps_LegacyNameIsAnotherAppsNewName`: Bucket
applications `x` and `bucket-x` with their legacy `Workload`s,
reconciled in both orders, end with `bucket-x` and `bucket-bucket-x`,
each with one owner and its own size, and the legacy
`bucket-bucket-bucket-x` gone.
- `TestReconcileBucketClaim_DoesNotAdoptAnotherClaimsWorkload`: a claim
whose name is held by another claim's `Workload` leaves it untouched,
emits `BucketWorkloadNameTaken` and is metered under its legacy name.
-
`TestReconcileBucketClaim_KeepsMeteringUnderLegacyNameWhileNewNameIsHeld`:
the legacy `Workload` keeps being updated, with its size and owner,
across repeated reconciles while the new name is held.
- `TestReconcileBucketClaim_UnprefixedClaimWithHeldNameIsSkipped` and
`TestReconcileBucketClaim_BothNamesHeldIsSkipped`: when no free name is
left, nothing is taken over and the event says the bucket is not
metered.
- `TestReconcileBucketClaim_UnprefixedClaimKeepsItsWorkload`: for a
claim without the prefix, where the legacy and the new name coincide,
its own `Workload` and size survive repeated reconciles.
- `TestReconcileBucketApp_RecordedSizeWinsOverLegacy`: a size already
recorded on the new `Workload` takes precedence over the legacy one.
- `TestReconcileBucketClaim_UpdatesItsOwnWorkload`: a claim's own
`Workload`, with its owner reference carrying the `BucketClaim` kind as
a live cluster's cache records it, keeps being updated and raises no
warning (the fake client leaves that kind empty on objects the
controller creates, so the other tests do not exercise this).
- `TestReconcileBucketClaim_AdoptsOwnerlessWorkload`: a `Workload` that
lost all owner references is taken over by the claim whose name it
carries, rather than blocking that name forever.
- `TestReconcileBucketApp_ReleasesSharedLegacyWorkload`: a legacy
`Workload` that another claim also owns (the rollback case) keeps its
size and its other owner; only this claim's reference is dropped, and
its size is not carried over.
- `TestReconcileBucketApp_KeepsWorkloadThatReplacedLegacy`: a `Workload`
that replaces the legacy one under the same name between the lookup and
the delete survives.
- `TestReconcileBucketApp_RemovesOrphanedLegacyWorkload`,
`TestReconcileBucketApp_KeepsOrphanOfAnotherMonitor`: an ownerless
legacy `Workload` with this monitor's label is removed without carrying
its sizes; one labelled for another monitor is left alone.
- `TestReconcileBucketApp_LegacySizeWinsOverOwnerlessNewWorkload`: a
stale size on an ownerless `Workload` at the new name loses to the
migrating legacy `Workload`'s size.
- `TestReconcileBucketApp_LegacyAlreadyGone`: a legacy `Workload` that
disappears between the lookup and the delete (NotFound) does not fail
the per-claim reconcile, and the size is still carried over.

Reverting the naming rule, the legacy delete, the size carry-over, the
owner check, the refusal to adopt, the same-name guard, the size
precedence ignoring NotFound on the legacy delete, the own-claim match
in the adoption check, the ownerless case, the physical size carry-over,
the sole-owner rule for legacy Workloads, the delete preconditions, the
legacy-name fallback or its own held check, the ownerless-legacy rule or
the size precedence on takeover each fails one of these. `go test
./internal/... -count=1` and `go vet ./internal/controller/` pass.

### Screenshots

N/A — no UI changes.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff, which touches only
`internal/controller/workloadmonitor_controller.go` and its test: no
package, values, schema, CRD, variant, `hack/` tooling, telemetry metric
or cozy-proxy contract changes.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(workloadmonitor): Workloads of Bucket applications are no longer named with a doubled "bucket-" prefix (bucket-bucket-<name> becomes bucket-<name>). Existing Workloads are replaced on upgrade, keeping their recorded size; billing usage reports whose window spans the upgrade list such a bucket once under each name. Rolling the controller back afterwards requires deleting the bucket-<name> Workloads of Bucket applications, or the buckets are metered twice.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Bucket workloads now use a consistent name without duplicating the
`bucket-` prefix.
* Existing workload size data is retained when workloads are renamed or
updated.
* Workloads owned by another claim are preserved. When a naming conflict
occurs, metering continues under the legacy name when possible; if both
names are unavailable, metering is skipped and a warning is emitted.
* Legacy workloads are cleaned up or updated according to their
ownership, helping prevent duplicate or orphaned workloads.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-28 13:43:42 +04:00
Myasnikov Daniil b502a9211f fix(release): keep release bodies within GitHub's size limit
Finalize sets the release body after the stable tag, the
api/apps/v1alpha1 tag and the release-X.Y fast-forward already exist,
so a body GitHub refuses fails the step with the release still a
draft, and the image retag and chart publish behind it never run.
GitHub's web UI cut the v1.7.0-rc.2 changelog at about 125000
characters, and the v1.7.0 changelog is 259,026.

A release can now carry a short hand-written docs/releasenotes/<tag>.md
that becomes its body, while the changelog stays the full record. Both
finalize and update-releasenotes take the release notes when they have
content and the changelog otherwise, and hold the body to 100000
bytes. Bytes are never fewer than characters, and the margin makes
GitHub's exact limit, and its answer to an oversized body, irrelevant.
Over the limit, finalize publishes a one-line link to the changelog on
the tag instead, and the sync leaves the published body alone: the
tags.yaml backstop brings a changelog to main without its release
notes, and the sync would otherwise replace them.

A release without release notes whose changelog is under the limit
still gets its changelog byte for byte.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-27 18:38:15 +00:00
Aleksei Sviridkin 0b3f5dd7e1 fix(keycloak)!: make the KMS proxy keep its keys across restarts (#4818)
## What this PR does

Two fixes for the optional KMS proxy of the platform Keycloak
(`encryption.*`).

The default static backend never started. The chart stored 32
alphanumeric characters as the KEK, but the proxy wants base64 of 32
bytes and exits with `config: KKP_KEK must decode to 32 bytes (got 24)`.
The KEK is now generated with `randBytes`, and a stored key of the old
form is replaced. The proxy never ran with such a key, so nothing was
encrypted under it. Any other stored key is kept if the proxy accepts it
and fails the render otherwise, so the chart never overwrites key
material. The KEK Secret now carries `helm.sh/resource-policy: keep`:
the DEK set is wrapped with it, so it has to survive turning encryption
off or reinstalling the package.

Without `deksetSecretName` the proxy mints a new DEK set on every start
(`buildCipher` in `cmd/proxy/main.go` of keycloak-kms-proxy) and keeps
it only in memory. After the first restart of the proxy pod it cannot
decrypt anything it wrote before. The values comment said this is safe
on a fresh DB, it is not. The chart now refuses to render the proxy
without a DEK set unless `fields: disabled`. Passthrough still works
without one, so the static backend can get its KEK Secret created before
the DEK set is minted with it.

Breaking: a release with encryption on and no `deksetSecretName` fails
to render on upgrade until a DEK set is configured. Such a setup loses
its data on the next proxy restart anyway, so failing loudly is the
point. A stored static KEK the proxy cannot decode fails the render too,
instead of being replaced.

Checked with `helm unittest` (100 tests, the new ones fail on the old
template) and by starting keycloak-kms-proxy v0.2.3 with the KEK the
chart renders now. With the KEK from the old template it fails as quoted
above.

### Screenshots

No UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/pull/742 documents the
setup with a DEK set
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The other repositories do not touch the keycloak chart values.

### Release note

```release-note
fix(keycloak)!: The KMS proxy of the platform Keycloak now requires `encryption.deksetSecretName` unless `encryption.fields` is `disabled`. Without it the proxy generated a new data key on every start and could not read data encrypted before a restart. The static backend now generates a KEK the proxy accepts; before, the proxy could not start with it. The KEK Secret is no longer deleted when encryption is turned off or the package is removed.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Configuration**
* Enabling field encryption requires a persistent DEK set. Chart
rendering reports an error if it is missing; setting fields to
`disabled` does not require one.
* For the static backend, a valid stored KEK is reused, and a missing or
legacy-format KEK is replaced. An unusable stored KEK causes rendering
to fail.
* The KEK Secret is retained when encryption is disabled or the package
is removed.
* **Documentation**
* Clarified DEK set requirements and sharing across starts and replicas,
and updated guidance on configuring multiple proxy replicas.
* Warns against deleting the retained KEK while encrypted values remain.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-27 11:52:17 +03:00
myasnikovdaniil b36628661f fix(ingress-nginx): drop the admission-webhook policy a rebase brought back (#5088)
## What this PR does

Rebase behind [#3931](https://github.com/cozystack/cozystack/pull/3931)
brought back namespaced `CiliumNetworkPolicy` for admission webhook,
which [#3997](https://github.com/cozystack/cozystack/pull/3997) deleted
in favour of clusterwide policy. It renders only on
`admissionWebhookPolicy.enabled` (true by default) and never checks that
Cilium API exists, so on cluster without Cilium every tenant with
`ingress: true` fails to install with `no matches for kind
"CiliumNetworkPolicy"`.

On cluster with Cilium it repeats deny that clusterwide policy already
has, and also renders for child tenants where webhook is disabled.

This PR removes the template with its test and adds suite that renders
whole package without Cilium API and fails if any Cilium resource comes
out. Existing capability suite selects only clusterwide template, so it
could not catch this file coming back. Clusterwide policy and
`admissionWebhookPolicy` value are unchanged.

Fixes #4706

On cluster without Cilium nothing in this chart restricts webhook port.
Removed policy could not do it either because it needs Cilium CRD to
install at all, and clusterwide policy already renders only where Cilium
API is served, so no protection is lost, such cluster now just installs
ingress instead of failing. Only `tenant-root` runs webhook, child
tenants set `admissionWebhooks.enabled: false`. Restricting webhook port
on non-Cilium clusters is separate design question, not part of this PR.

Checked `helm template` with and without
`cilium.io/v2/CiliumClusterwideNetworkPolicy` in api versions: without
it no Cilium object renders, with it only clusterwide policy, same as
before. New suite fails on tree before this change.

### Screenshots

Not a UI change.

### Downstream repositories

Only template and tests under `packages/system/ingress-nginx` changed,
`admissionWebhookPolicy` value kept, nothing from trigger map in
`docs/agents/contributing.md` matches.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(ingress-nginx): remove the namespaced admission-webhook CiliumNetworkPolicy that was re-added by mistake. It was rendered without checking that the Cilium API exists, so on clusters without Cilium every tenant ingress failed to install with `no matches for kind "CiliumNetworkPolicy"`. The clusterwide admission-webhook policy is unchanged and still renders only where the Cilium API is served.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Configuration**
* The ingress-nginx chart no longer generates an admission webhook
network policy. Deployments that relied on this policy to restrict
access to the webhook will need to configure equivalent traffic controls
separately.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-27 11:47:30 +05:00
myasnikovdaniil 372c85575f fix(controller): carry healthCheckExprs and waitStrategy to existing releases (#5087)
## What this PR does

This PR syncs `healthCheckExprs` and `waitStrategy` from
ApplicationDefinition to HelmReleases that already exist. cozystack-api
writes both fields only when it builds a release, and the definition
reconciler copied chartRef, valuesFrom, disable-wait setting and labels
but not these two, so release created before its definition got
expressions keeps waiting on objects the definition already exempts.

Most visible case is vm-disk. Every disk created before the DataVolume
`current: "true"` expression was added still waits on its DataVolume, so
after platform upgrade an existing disk with import still in progress
fails the upgrade after 10 minutes with `timeout waiting for:
[DataVolume/<namespace>/vm-disk-<name> status: 'InProgress']`, then
retries forever and holds one of 5 workers of its helm-controller shard.

Reconciler now derives both fields same way the API does
(`config.ResolveWaitStrategy`) and syncs them in both directions, so
existing release ends up same as freshly written one. On current
definitions only vm-disk (DataVolume) and postgres (KEDA ScaledObject)
set expressions, both `current: "true"`, so this can only relax waits.
Releases of other kinds have nothing to sync and are not rewritten.

HelmReleases rendered by another chart (tenant modules like Etcd,
Ingress, Monitoring) carry Flux origin label
`helm.toolkit.fluxcd.io/name` and are skipped, their gate comes from
their own chart. Their chartRef is still synced as before.

Failed update was logged and dropped before, and nothing requeues the
definition for hours after that. Now errors are returned, so a conflict
is retried.

Unit tests cover both sync directions, `waitStrategy: legacy` without
expressions, no rewrite of in-sync release (nil vs empty slice too),
chart-rendered and other-kind releases left alone, and update failure
returned as error.

Fixes #4409

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(controller): existing HelmReleases now get `healthCheckExprs` and `waitStrategy` from their ApplicationDefinition. vm-disk and postgres releases created before these expressions were added stop waiting on DataVolume and KEDA ScaledObject during upgrades.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Helm releases now synchronize health-check expressions and wait
strategies configured in their application definition, including when
those values are removed or no expressions are specified.
* Releases with chart-rendered health-check settings retain those
settings, and releases outside the matching application kind are
unaffected.
* Update failures are now reported during reconciliation, while updates
to other releases continue. Reconciliation can retry after a failed
update.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-27 11:47:21 +05:00
myasnikovdaniil 1ac83aebd9 fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates (#5086)
## What this PR does

After platform upgrade the tenant `gateway` HelmRelease fails with
`timeout waiting for: [TenantGateway/<namespace>/cozystack status:
'InProgress']`, retries every 30s without end (`RetryOnFailure`) while
holding a helm-controller worker, and parent tenant release that waits
on it fails too.

helm-controller waits on every object of the release with kstatus on
install and on every upgrade, and kstatus reads TenantGateway
`Ready=False/ListenersNotReady` as InProgress. TenantGateway stays there
while any listener certificate is pending, which depends on ACME or DNS
and on routes owned by other releases, so waiting longer doesn't help.
It passed before only because the wait runs inside a Helm action and the
install happened before those listeners showed up.

I added `healthCheckExprs` for `TenantGateway` with `current: "true"`
and pinned `waitStrategy: poller`, because expressions are evaluated
only under poller. Wait stays on. Flux returns InProgress while
`status.observedGeneration` lags `metadata.generation` before it reads
any expression, so release still waits until controller observed the new
TenantGateway spec, but not for listeners or certificates. tenant-root
renders same template and gets the change too. Gateway release that is
already stuck picks this up on its next attempt, after the current one
times out.

New helm-unittest suite
(`packages/apps/tenant/tests/gateway_wait_test.yaml`) pins the strategy
and the whole expression list, checks there is no `disableWait` on
install or upgrade, and that no gateway release is rendered when
`gateway` is unset.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(tenant): the tenant gateway release no longer times out on upgrade while TenantGateway listener certificates are pending, it waits only until the controller has observed the current TenantGateway spec.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Gateway releases now determine readiness from the gateway resource’s
current status, rather than waiting for its listeners and certificates
to become ready. Releases still wait for the latest resource update to
be observed.
* **Tests**
* Added checks for gateway release readiness behavior and for tenants
configured without a gateway.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-27 11:47:15 +05:00
myasnikovdaniil 7dc5c0b5e6 fix(marketplace): do not install a connected repository's components on connect (#5071)
## What this PR does

Connecting a marketplace repository materialized its `PackageSource`s
and then auto-created a tap-managed `Package` per source. The Package
reconciler installs a `HelmRelease` for every install-marked component
of the default variant, and Flux runs those under its `flux` service
account, which is bound to `cluster-admin` with no impersonation
(`--default-service-account` is not set, and neither `internal/operator`
nor `pkg/registry/apps` set `serviceAccountName` on the HelmRelease). A
publisher's install chart therefore ran as `cluster-admin` on connect,
with no operator step, contradicting the documented behaviour that
tapping installs nothing. The `privileged` flag is a prompt to the admin
and does not protect against a publisher acting in bad faith.

This removes the on-connect auto-registration (`ensureRegistration`).
Connecting now only materializes `PackageSource`s. Installing an app is
an explicit `cozypkg add`, which becomes the sole creator of a
`Package`; every other `Package` reference in the operator is a read or
a watch. Removing the call also fixes the second effect of the same
path: a `Package` the admin removed with `cozypkg del` is no longer
re-created on the next source revision.

A second commit reconciles the texts this behaviour change made wrong:
they promised that tapping registers a repository's apps in the catalog
and runs its charts. Reworded the `cozypkg tap`/`untap` help, the tap
success output (it now prints a `cozypkg add <name>` line per
materialized PackageSource), the `cozypkg init` scaffold README, the
packages-index trust model, and the dashboard Repositories banner and
per-repository empty state.

Accepted consequence: a connected repository's apps appear in the
catalog only after an admin runs `cozypkg add`, since the dashboard
catalog is computed from `ApplicationDefinition`s. Install from a
connected repository is CLI-only for now; the console connects and
disconnects repositories but does not create a `Package`. Populating the
catalog from `PackageSource` metadata without an install is the
long-term shape in #4359 and is out of scope here. The
privileged-confirmation gaps #4358 and #4359 stay open.

Upgrade note: a `Package` that Cozystack 1.7.0 auto-created on connect
stays after the upgrade, and the privileged-confirmation gate still
applies to it, so this is not a regression for existing installs. For a
newly connected repository no tap-labelled `Package` is created, so the
gate does not come into play on that path. An operator who does not want
a 1.7.0 auto-created package can remove it with `cozypkg del`.

Addresses #4978.

### Screenshots

Dashboard text-only change (Repositories banner and per-repository empty
state). No layout change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff. The diff is operator logic plus CLI/dashboard/doc text; it matches
no concrete trigger (no package, API, schema, tooling, annotation,
metric or commit-convention change). The dashboard console is vendored
in this repo (`packages/system/dashboard`), so its text change travels
in this PR. Leaving every box empty rather than ticking "No downstream
repository is affected".

### Release note

```release-note
fix(marketplace): connecting a marketplace repository no longer installs its components. A connected repository's apps are materialized as PackageSources but not installed until an explicit `cozypkg add`; install from a connected repository is CLI-only for now. Previously, connecting auto-created a Package whose install-marked components were deployed by Flux under its cluster-admin service account. A Package that Cozystack 1.7.0 auto-created on connect stays after upgrade and can be removed with `cozypkg del`.
```
2026-10-26 21:10:45 +05:00
myasnikovdaniil 7215b02d38 feat(keycloak): load additional CA files from Secret or ConfigMap keys (#5039)
## What this PR does

Closes #3823.

With the stock Cozystack chart, Keycloak cannot connect to an LDAPS
server, external identity provider or SMTP server signed by a private CA
because the chart cannot deliver a CA file to its truststore. Add
`truststores`, a list of existing Secret/ConfigMap references in the
release namespace, with one selected key per entry (`ca.crt` by
default).

```yaml
truststores:
  - configMapName: freeipa-ca
    key: ca.crt
  - secretName: partner-ca
    key: certificates.pem
```

Each key is mounted read-only in a separate `cozystack-<index>`
subdirectory of `/opt/keycloak/conf/truststores`. Keycloak discovers it
automatically; the chart adds no environment override, Kubernetes API
permission or arbitrary volume path. Existing image certificates, Java
default roots, themes and `extraEnv` keep their behavior. References are
mandatory: a missing object or selected key prevents the pod from
starting. Helm validates the reference shape and safe key paths, but
cannot inspect external certificate contents.

The trust decision belongs to platform administrators and applies to the
entire Keycloak server, across all realms. Write access to a referenced
object therefore permits changing that server's trust. If HTTPS client
authentication is set to `request` or `required` without a separate
`https-trust-store-file`, these CAs also verify incoming client
certificates; configure that file explicitly when incoming mTLS requires
a different trust boundary. Use public certificates only (PEM bundles or
unencrypted PKCS12 without private keys). This does not disable TLS
hostname verification. Updating an object's contents does not restart
Keycloak or reload its truststore: restart both pods after rotation, or
use a versioned object name to trigger a rollout.

Validation:

- All 113 Keycloak Helm tests pass, including 21 new cases for reference
wiring, read-only mounts, selected-key isolation, themes, PKCS12
filenames, defaults and invalid input.
- Boundary tests reject source names and keys of 254 characters and the
single-dot key, and accept a source name and key of exactly 253
characters. Removing any of the three guards makes its regression test
fail; changing either length comparison to reject 253 also fails the
boundary test.
- Empty-default render matches a clean upstream/main worktree after
normalizing the existing randomly generated bootstrap password. Four
mutations (wrong mount path, writable mount, wrong selected key and
removed validation) each turn the new suite red.
- Enabled and default manifests render successfully; enabled YAML parses
with yq. `git diff --check` passes.
- Runtime check with official Keycloak 26.6.3 in isolated Docker:
synthetic LDAPS succeeds with a CA loaded automatically through an
emulated Kubernetes AtomicWriter symlink layout (HTTP 204 from the LDAP
connection-test Admin API); no custom CA fails with PKIX, and a trusted
CA with a mismatched SAN fails hostname verification (both HTTP 400).
KC_TRUSTSTORE_PATHS was unset. This checks TLS establishment, not a
complete federation bind/failover or deployment to a Kubernetes cluster.
- `make generate` has no target in this system chart; no generated
schema/API files are involved. `make show` requires the unavailable
cozyhr CLI and a live release, so rendering uses `helm template`. `helm
lint` reports the same pre-existing undeclared cozy-lib dependency on
both this branch and clean main.


### Screenshots

Not applicable; no UI changes.

### Downstream repositories

The website guide currently says CA mounts are unavailable, so its
`next` documentation needs updating once this ships. Other downstream
trigger-map entries do not apply: this only adds an opt-in value to the
system Keycloak chart, with no app schema, platform configuration, CRD,
tooling, naming or node-contract changes.

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: https://github.com/cozystack/website/issues/765
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
feat(keycloak): load additional CA files from existing Secret or ConfigMap keys through the new truststores value, enabling TLS to private LDAPS, identity-provider and SMTP endpoints.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

- **New Features**
- Added support for configuring Keycloak truststores from existing
Secrets or ConfigMaps.
- Supports PEM bundles and unencrypted PKCS12 certificates, with
optional key selection and read-only mounts.
- Truststore certificates apply server-wide, including incoming
client-certificate validation when no separate HTTPS truststore is
configured.
- Invalid truststore configurations produce clear errors. Missing
objects or keys prevent startup; certificate rotations require pod
restarts unless the referenced object name changes.
- **Tests**
- Added coverage for default behavior, multiple truststore sources,
theme mounts, explicit paths, and invalid configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-26 19:57:18 +05:00
IvanHunters 09b0688abd fix(marketplace): fail closed on an unparseable verified digest, and cover the generation gate
A verified cosign signature whose manifest digest could not be parsed fell back
to an unpinned tag pull, which reopens the tag-mutation window the verification
was meant to close. Return an error instead.

Add a test that isolates the status.observedGeneration == metadata.generation
conjunct of artifactReady; the existing tests exercised only the tag-prefix
conjunct, so a mutation of the generation check went uncaught.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-26 17:11:52 +03:00
IvanHunters 0195e73d38 fix(marketplace): verify the signature before pulling, and anchor the identity match
With --certificate-identity and --certificate-oidc-issuer, `cozypkg tap` pulled by
tag without verifying and applied the artifact's PackageSources with
ForceOwnership; only the Flux source carried the verify policy. A tag moved by
someone without the signing identity could land unsigned content in the cluster.
Verify the signature at tap time and pin the verified digest, so the content
pulled and applied is what a trusted identity signed.

Flux evaluates a cosign matchOIDCIdentity issuer and subject as Go regexps, and
they were written unanchored. An exact identity such as alice@example.com would
also accept malice@example.com, and the dots would match any character. Anchor
and quote both values at the API and the CLI.

Reject --tag together with a digest reference instead of silently ignoring the
tag.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-26 16:55:47 +03:00
IvanHunters 7494d1fad2 fix(marketplace): pin a tap digest only from a current-generation artifact
The materializer pinned whatever digest status.artifact reported, even when the
artifact still described the previous reference. On a re-connect to a new tag,
source-controller has not re-resolved yet, so the operator materialized and
pinned the old tag's content (issue #4979). Gate materialization and pinning on
the artifact reflecting the current spec: status.observedGeneration equals
metadata.generation, and for a tag-only source a revision that belongs to that
tag (<tag>@<digest>).

A tap connected before digest pinning existed already has its
materialized-revision equal to the artifact revision, so Reconcile short-circuited
and never pinned, letting it follow the tag one more time. Pin the current digest
before that early return.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-26 16:55:40 +03:00
IvanHunters 96758b9e3d fix(marketplace): drop the index-entry digest from the tap pin change
The index-entry `digest` field, and the gate and README claims around it, assume
a published index that does not exist yet: decision 0002 in the marketplace
proposal defers it, and #4981 tracks that work. The gate also never compared a
recorded digest against the artifact it verified, so the field added surface
without the check that would make it trustworthy.

Drop it from this PR. Digest pinning still works from an explicit
oci://host/org/repo@sha256:... reference and from the digest a tag resolves to;
only the index-entry path is removed. The tag-based pinning and the verify-flag
wording stay in the index README.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-26 16:55:36 +03:00
IvanHunters 2f39e304f0 fix(marketplace): reconcile tap texts with no-install-on-connect
Removing on-connect auto-registration made several user-facing and doc texts
wrong: they promised that tapping registers a repository's apps in the catalog
and runs its charts. Tapping now installs nothing, so the texts told operators to
wait for apps that never appear.

Reword the `cozypkg tap`/`untap` help, the tap success output (now printing a
`cozypkg add <name>` line per materialized PackageSource), the `cozypkg init`
scaffold README, the packages-index trust model, the dashboard Repositories
banner and per-repository empty state, and the stale re-registration code
comments. No behaviour changes beyond the added `cozypkg add` hint in the tap
output.

Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-10-26 15:18:14 +03:00
Aleksei Sviridkin 749d401899 fix(kubernetes, opensearch, foundationdb)!: restrict managed application images (#5085)
## What this PR does

Image overrides let tenants run arbitrary containers in management
cluster, including helpers with access to Talos CA secrets and
Kubernetes API credentials. This PR removes those overrides and uses
images shipped by the platform.

Replaces [the original
PR](https://github.com/cozystack/cozystack/pull/4287) for 1.7. Andrei
Kvapil's commits keep his authorship.

- Kubernetes and KubernetesNodes helpers use the bundled image
references through `cozy-lib.image`. OpenSearch uses its version map.
Talos installer settings for tenant worker VMs stay available.
- FoundationDB uses `version: v7.3` or `v7.4`, mapped to `7.3.63` and
`7.4.1`. Migration 60 carries the old `cluster.version` over to its
release line, records releases needing attention and preserves the
legacy value while the old chart can still render it.
- The FoundationDB chart refuses a render when `cluster.version` and
`version` name different release lines. This also covers variants
without platform migrations.
- OpenSearch and FoundationDB get version-map update scripts. They
retain the current default while it remains supported.

Application types and schemas are still generated from the charts.

### Upgrade notes

Stored image overrides are accepted without new admission warnings and
ignored by the charts. On the first reconcile, OpenSearch nodes move to
the image for their selected version, and Kubernetes components whose
images changed restart. Check custom OpenSearch images before upgrading:
if one runs a newer version than the chart's map, switching back can
leave nodes unable to read their data. Air-gapped installations need
`docker.io` and `ghcr.io` mirrored on management-cluster nodes.

FoundationDB preserves the release line, not an arbitrary patch: 7.3.x
moves to 7.3.63 and 7.4.x to 7.4.1. Check these versions before
upgrading. A cluster on 7.1 needs an upgrade to 7.3 first. On variants
without migrations, set `version` to the line the cluster runs. To
change release lines later, remove the leftover `cluster.version` as
directed by the chart error.

Migration 60 records affected releases in the
`cozystack.io/migration-60-needs-attention` annotation on the
`cozystack-version` ConfigMap. Dry-run reports patch changes without
patching, annotating or advancing the migration stamp. Migration 56 no
longer carries the removed kubectl override into adopted node pools.

An explicit OpenSearch `version: null` now fails rendering instead of
falling back to `v2`.

### Validation

- `make unit-tests` passed, including migrations 56 and 60.
- Four-chart regression run: 750 tests passed.
- API Create/Update, admission, defaulting and existing warning
regression tests passed. `cmd/`, `internal/` and `pkg/` match main.
- Shell test lanes passed with `/bin/sh` as dash throughout the process
tree.
- `make generate manifests` passed with no generated drift, using
cozyvalues-gen v1.7.0.

E2E passed on e694b05b4.

### Screenshots

Not a UI change.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
existing generated parameter tables are refreshed by the release docs
job.
- [x]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up: [schema
updates](https://github.com/cozystack/terraform-provider-cozystack/issues/74).
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The provider follow-up removes the Kubernetes and KubernetesNodes image
blocks and adds FoundationDB version selection. No other downstream
follow-up is needed for these fields.

### Release note

```release-note
fix(kubernetes, opensearch, foundationdb)!: Remove tenant image overrides from Kubernetes, KubernetesNodes and OpenSearch. Existing overrides switch to platform images on reconciliation, which can roll OpenSearch nodes and restart Kubernetes components. Air-gapped installations must mirror the upstream registries on management-cluster nodes. FoundationDB replaces cluster.version with version (v7.3 or v7.4); migration 60 preserves the release line and records patch changes. Check running image versions before upgrading. Variants without migrations must set the FoundationDB version before upgrading, and the chart refuses mismatched legacy release lines. OpenSearch rejects an explicit version: null.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* FoundationDB now supports selecting version lines `v7.3` and `v7.4`,
with `v7.3` as the default.
* Added automatic migration of existing FoundationDB version settings,
with warnings for settings that need attention.

* **Changes**
* Removed per-application image overrides for Kubernetes, Kubernetes
Nodes, and OpenSearch. These charts now use their bundled images, which
can still be routed through the configured image registry.
* OpenSearch now reports an error when its version is missing or
unsupported.
* Existing FoundationDB settings with conflicting version lines are
reported for review.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-26 13:00:10 +03:00
Myasnikov Daniil 7fceb92575 ci(renovate): track the SBOM scanner pin in hack/common-envs.mk
The dockerfile manager does not read Makefiles, so the digest-pinned
scanner would stay on the syft it was cut with, where the floating tag
it replaced moved on its own. A regex manager bumps the tag and the
digest together, and a test holds its regexp to the SBOM_SCANNER line,
so a reformatted pin cannot drop out of it unnoticed.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-by: LLM
2026-10-26 06:17:01 +00:00
Myasnikov Daniil 91f3b07a70 docs(release): drop the nightly mirror from the image SBOM section
The mirror copies main's builds from OCIR to GHCR, and only tags.yaml
sets SBOM, so the mirror never sees an image that carries an
attestation. Promotion is the only copy that moves one.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-by: LLM
2026-10-26 06:16:57 +00:00
Marian Koreniuk 70ec326666 ci(e2e): end the Proxmox lane's SSH key with a newline
Stage wrote PROXMOX_SSH_KEY with printf '%s', so the key file ended
exactly where the secret did. `gh secret set` strips the trailing
newline from a value read from stdin, and OpenSSH refuses a private key
whose last line has none: ssh reports `Load key ...: error in libcrypto`
and the first hypervisor check fails with `Permission denied
(publickey)`. Write it with printf '%s\n'. When the secret kept its own
newline the file ends in a blank line, which OpenSSH accepts.

The contract test now runs Stage with the key both ways and loads the
result with ssh-keygen; against the old step it fails on the first.

Assisted-by: LLM
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
2026-10-23 22:53:21 +02:00
myasnikovdaniil 55aad6f914 fix(tcp-balancer): cap HAProxy fds and threads by the pod's own limits (#5005)
## What this PR does

On a v1.6.5 cluster a TCPBalancer created with the chart defaults
(`t1.nano`, limits 250m CPU and 128Mi memory, `haproxy:3.4.4`)
crash-looped. Both pods were OOMKilled with a memory limit of 134217728,
7 restarts each in about 15 minutes, and HAProxy logged `[NOTICE] (8) :
Automatically setting global.maxconn to 524126.` On another cluster the
same chart picked maxconn 524251 and the pods kept running.

`haproxy.cfg` has no `global` section, so HAProxy sizes itself from the
node. maxconn comes from the nofile limit capped at the built-in
`fd-hard-limit` of 1048576 (`DEFAULT_MAXFD`), and the thread count is
the number of CPUs the process may run on, whatever its CPU limit is.
Both are paid for at startup, before any traffic, and the thread part
grows with the node.

This PR adds a `global` section sized from the replica's own limits,
read through the same `cozy-lib.resources.defaultingSanitize` helper the
Deployment uses, so `resources` and `resourcesPreset` resolve the same
way. For `t1.nano` it renders

```
global
    fd-hard-limit 4096
    thread-hard-limit 1
```

`fd-hard-limit` is 32 per MiB of the memory limit and HAProxy derives
maxconn from it, 2035 for `t1.nano`, which is about 64 KiB per
connection. For comparison `haproxy -m 128` picks maxconn 3600 on 3.4.4,
about 35 KiB per connection, and the difference is left for the process
itself and for the kernel socket buffers the cgroup also charges. An
explicit `maxconn` would read simpler, but `strict-limits` is on by
default and HAProxy refuses to start when it cannot raise nofile to
about twice that value. With 1024 soft and 4096 hard nofile `maxconn
4096` exits with `Cannot raise FD limit to 8251`, while `fd-hard-limit
8192` starts with maxconn 2018. `thread-hard-limit` is the CPU limit
rounded up, at least 1. It caps the detected thread count and never
forces more threads than the node has.

Reproduced with Docker Desktop on arm64 (12 vCPU, kernel 6.12) and the
same image, running the rendered config the way the Deployment mounts
it. A larger node is stood in for by adding `nbthread 96` or `nbthread
128` to the `global` section. Memory is `memory.peak` of the container
cgroup.

```sh
cd packages/apps/tcp-balancer && mkdir -p /tmp/cfg
helm template tb . --show-only templates/configmap.yaml | yq '.data["haproxy.cfg"]' > /tmp/cfg/haproxy.cfg
docker run --rm --memory 128m --memory-swap 128m --ulimit nofile=1048576:1048576 -v /tmp/cfg:/usr/local/etc/haproxy:ro haproxy:3.4.4
```

| run | main | this PR |
| --- | --- | --- |
| idle, 12 threads | maxconn 524258, peak 89.6 MiB | maxconn 2035, 1
thread, peak 18.7 MiB |
| idle, `nbthread 96` | maxconn 524127, worker exited with code 137,
OOMKilled | 1 thread, peak 20.8 MiB |
| idle, `nbthread 128` | worker exited with code 137, OOMKilled | 1
thread, peak 22.9 MiB |
| 2035 clients echoing 16 KiB through a backend for 25 s, 12 threads |
worker exited with code 137 about a second after the clients started |
running, peak 83.7 MiB |

One case this PR does not cover is clients and servers that both stop
reading. They fill kernel socket buffers that the cgroup charges, and
with this PR the 128Mi container was still OOMKilled after 1225 such
connections, which maxconn does not bound.

`helm unittest .` passes 13 of 13 with helm-unittest 1.1.2 as CI pins
it. Against the template on main the four new cases fail and the nine
existing ones pass, and each of six template mutations fails at least
one case. `haproxy -c` accepts the render for `t1.nano`, `t1.4xlarge`,
`m1.4xlarge`, a zero CPU limit, the whitelist, `kubernetesAndTalos` and
`tcp-with-proxy` inputs. A full `helm template` differs from main only
in the `global` section and the `checksum/config` annotation, so the
pods roll once on upgrade. I have not run the change on a live cluster.

The template is the same on release-1.6, and that backport is a separate
PR.

### Screenshots

Not a UI change.

### Downstream repositories

One chart template and its unittest suite in
`packages/apps/tcp-balancer`. No values key, schema, default, README or
package change, so no trigger map entry applies.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(tcp-balancer): HAProxy now takes its fd and thread caps from the replica's memory and CPU limits instead of the node, so a t1.nano balancer is no longer OOMKilled on nodes with many CPUs. maxconn follows the memory limit at about 16 connections per MiB, 2035 on t1.nano, and the pods roll once on upgrade.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- TCP Balancer now automatically configures HAProxy file-descriptor and
thread limits based on the selected or explicitly defined resource
settings.
- CPU-based thread limits are rounded up and always maintain a minimum
of one thread.
- Memory-based file-descriptor limits scale automatically with the
configured memory capacity.

- **Tests**
- Added coverage for default, preset-based, explicit resource, rounding,
and minimum-limit scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-23 23:53:39 +05:00
myasnikovdaniil 432d02e20c fix(etcd): stop counting canceled watch streams as failed gRPC requests (#4985)
## What this PR does

The tenant etcd chart has its own copy of
`etcdHighNumberOfFailedGRPCRequests`, and both of its rules count every
`grpc_code` other than `OK` as a failure. `grpc_server_handled_total`
counts a Watch stream once, when it ends, and the streams of a
kube-apiserver that goes away end with `Canceled`. So whenever an API
server on a tenant etcd stops, the Watch method of every member reads as
100 percent failed and the critical rule fires on all of them.

Both rules now count only
`Unknown|FailedPrecondition|ResourceExhausted|Internal|Unavailable|DataLoss|DeadlineExceeded`,
the list the etcd mixin has used since etcd-io/etcd#13127 and the one
the platform copy in
`packages/system/monitoring-agents/alerts/etcd.yaml` already uses. The
grouping `BY (job, instance, grpc_service, grpc_method)`, the
thresholds, `for` and the annotations stay as they are, so the alerts
keep their labels.

Seen on a v1.6.5 cluster while a Kubernetes cluster on a tenant etcd was
created and deleted a few minutes later. vmalert had the critical alert
firing for `grpc_method="Watch"` on all three members with the value
100. At that point
`rate(grpc_server_handled_total{grpc_code!="OK"}[10m])` was 0.63 for
Watch with `Canceled`, and the only other non-OK codes were
`FailedPrecondition` on `RoleGet` and `UserGet` at 0.0017 each, which is
one call each. The same query an hour earlier returned nothing. The
platform copy runs in the same vmalert over the same series, and none of
the firing alerts came from it. Each alert also showed up seven times
there, once per tenant copy of these rules, because the root tenant's
vmalert loads the rules of every tenant namespace. This PR does not
change that.

`hack/etcd-grpc-alerts.bats` renders the chart's PrometheusRule with
`helm template` and replays it through `vmalert-tool` v1.136.0 pinned by
digest, the same image #4963 uses.
`hack/testdata/etcd-grpc-alerts/unittest.yaml` has three cases with
seven checks. Watch streams that end only with `Canceled` must not fire
at 6, 12 and 30 minutes. `Range` calls failing with `Unavailable` at 25
percent must fire the critical rule at 7 minutes and both rules at 12,
with the labels and the summary pinned. Both together must fire only for
`Range`. Against the rules on main the suite fails 4 of the 7 checks,
all of them on Watch alerts with the value 100. Dropping `Unavailable`
from the list, or grouping by `job` and `instance` only, fails 3 checks
each.

```console
$ bats hack/etcd-grpc-alerts.bats
1..1
ok 1 tenant etcd failed gRPC request alerts pass their vmalert-tool cases
```

The same run on the rules from main prints `not ok 1` and the four
failing checks. With Bats 1.14.0 at the commit CI pins, the suite passes
together with `bats-strict-setup`, `bats-no-negated-assert`,
`bats-no-exit-trap`, `bats-runner-coverage`, `bats-runner-contract`,
`alert-severity-contract` and `image-pin-consistency`, 81 of 81. Like
the migration suites it needs docker, and only the image pull touches
the network. Helm 3.17.3 and 4.2.4 render the same rules. I have not run
the new rules in vmalert on a live cluster.

The rules file is the same on release-1.6 and release-1.7. The commit
cherry-picks onto release-1.6 without conflicts, and that backport is a
separate PR with one more commit that drops the `load test_helper` line,
because release-1.6 runs `hack/*.bats` through `hack/cozytest.sh`.

### Screenshots

Not a UI change.

### Downstream repositories

A rule template in `packages/extra/etcd`, a Bats suite under `hack/` and
its test data. No values, schema, README, image or
`ApplicationDefinition` change, and the website does not mention these
rules, so no trigger map entry applies.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(etcd): the tenant etcd alert etcdHighNumberOfFailedGRPCRequests now counts only the gRPC codes that the etcd mixin and the platform copy of the alert count, so the Canceled watch streams of a Kubernetes API server that stops no longer fire it on every etcd member.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Refined etcd gRPC failure alerts to target specific failure status
codes, reducing alerts for non-failure responses while preserving
existing thresholds and timing.
- Added coverage for canceled watch streams, unavailable range requests,
and combined failure scenarios, including warning and critical alert
evaluations.

- **Tests**
- Added integration validation for rendering etcd alert rules and
running them through the alert-testing tool, including failure handling
when prerequisites are unavailable.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-23 22:40:33 +05:00
myasnikovdaniil 614847976f test(harbor): pin the resources of the nginx proxy and the portal (#5025)
## What this PR does

#4095 gave the Harbor nginx proxy the `t1.small` preset and moved the
portal from `t1.nano` to `t1.micro`, so neither falls back to the
LimitRange default of a tenant namespace, 250m CPU and 128Mi. It came
without a test. This adds a helm-unittest suite that pins both presets
in the nested HelmRelease and checks that they stay the same when the
other components are resized, so dropping them or tying them to another
component's values fails the suite.

Both containers run nginx with `worker_processes auto`, which starts one
worker per CPU of the node whatever the container's CPU limit is. With
the proxy config the chart renders and `goharbor/nginx-photon:v2.15.1`,
96 workers idle at about 205 MiB, so on a node with many CPUs the proxy
is OOMKilled at 128Mi before it serves a request. That is what happened
on a v1.6.5 cluster, where both nginx pods of a default Harbor restarted
5 and 6 times within minutes, because `release-1.6` never got #4095. The
backport, with this suite and the measurements, is a separate PR.

`helm unittest .` in `packages/apps/harbor` passes 47 of 47 with
helm-unittest 1.1.2. With the #4095 hunk reverted the three new cases
fail and the 44 existing ones pass. Each of seven template mutations
fails at least one new case. They drop the nginx block, give nginx
`t1.micro` or the portal `t1.nano`, tie nginx to the core values or the
portal to the jobservice values, misspell the `resources` key under
nginx, and make core ignore its preset, which the resize case checks so
that it cannot pass on overrides that never reach the release.

### Screenshots

Not a UI change.

### Downstream repositories

One new helm-unittest suite in `packages/apps/harbor/tests`. No
template, values key, schema, default, README or package change, so no
trigger map entry applies.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
NONE
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Tests**
* Added Helm coverage verifying default resource settings for Harbor’s
nginx and portal components.
* Added coverage confirming nginx and portal memory allocations remain
consistent when related service resources are customized.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-23 22:40:28 +05:00
Myasnikov Daniil 8d8179f20a fix(controller): carry healthCheckExprs and waitStrategy to existing releases
cozystack-api writes healthCheckExprs and the resolved wait strategy
from the ApplicationDefinition only when it builds a HelmRelease, and
the definition reconciler that keeps existing releases in step copied
neither. A release created before its definition exempted an object
kept waiting on that object at every upgrade. For vm-disk that is
every disk created before the DataVolume expression was added: a disk
whose import is still running after a platform upgrade fails the
upgrade at the timeout and, under RetryOnFailure, retries forever
while holding a helm-controller worker.

The reconciler now derives both fields the way the API does, in both
directions, so an existing release ends up with what a fresh write
would produce. On the shipped definitions this only relaxes waits: the
vm-disk DataVolume and the postgres KEDA ScaledObject. A release that
another chart renders carries Flux's origin label and keeps the gate
its chart sets.

A failed update used to be logged and dropped, and nothing else
re-enqueues the definition for hours, so that release stayed out of
sync. The errors are now returned and the definition is retried.

Fixes https://github.com/cozystack/cozystack/issues/4409

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-23 14:35:34 +00:00
Myasnikov Daniil 5265ee662c fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates
helm-controller waits on every object of a release with the kstatus
poller on install and on every upgrade. kstatus reads a TenantGateway at
Ready=False/ListenersNotReady as InProgress, and a TenantGateway sits
there whenever one listener certificate is pending, which depends on
ACME or DNS and, under HTTP-01, on routes that other releases own. After
a platform upgrade the gateway release therefore times out, retries
without end under RetryOnFailure, holds a helm-controller worker
meanwhile, and fails the tenant release that waits on it. It used to
pass because health is judged only inside a Helm action: the release
was installed while the TenantGateway had no status yet or was still
Ready, and the listeners whose certificates do not issue came later
with other releases' routes.

An always-true expression replaces kstatus for TenantGateway and keeps
the wait on. Flux checks status.observedGeneration against the
generation before it evaluates any expression, so the release still
waits until the controller has observed the new spec, but not for
listener or certificate health. The poller strategy is set explicitly
because expressions are only evaluated under it.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-23 14:34:55 +00:00
Myasnikov Daniil e694b05b4e refactor(api): keep application image policy in charts
The charts enforce image selection without extending the API server's
write path. Keep the existing admission behavior while the application
schemas expose the supported fields.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-23 19:25:04 +05:00
Myasnikov Daniil 59932e5098 fix(platform): report FoundationDB patch drift in migration dry runs
The carry path skipped diagnostics in dry-run mode, so its summary could
report no releases needing attention even when the upgrade would lower a
FoundationDB patch version. Skip only the write and share the warning and
summary bookkeeping with a real run.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-23 19:14:17 +05:00
Myasnikov Daniil 639ede20be fix(api): leave FoundationDB version checks in the chart
The chart still reads cluster.version to guard release-line changes,
so calling it ignored in an admission warning contradicts the render
error. Keep that decision in the chart, which can explain the version
mismatch, and limit removed-field warnings to inert values.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-23 19:14:17 +05:00
Andrei Kvapil 465158f4a0 test(platform): load the strict-mode helper in the migration 60 suite
Every unit bats file loads test_helper so its tests run with nounset,
and hack/bats-strict-setup.bats fails the unit job on a file that
does not. The migration 60 suite passes under it unchanged.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:17 +05:00
Andrei Kvapil b43196e549 fix(kubernetes): pin the Proxmox cloud-config builders to the bundled image
The Proxmox substrate added two init containers that build the cloud
config from the hypervisor token Secret, and both still took their
image from images.waitForKubeconfig. With that key gone the render
failed on a nil map; with it set, a tenant chose the image that reads
the token. They now run the bundled busybox through cozy-lib.image, like
the wait-for-kubeconfig container next to them.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:17 +05:00
Andrei Kvapil 4cb505a512 fix(platform): record 7.1 FoundationDB clusters instead of carrying them
The operator no longer carries the 7.1 client, and the chart no longer
offers v7.1, so a 7.1 cluster has no line to be carried to. Migration
59 now records it, together with any release whose version already
names a line that is no longer offered; both have to be upgraded or
repointed before the chart will render them.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil 9775628cc8 fix(foundationdb): refuse a render that would move the release line
Migration 60 carries cluster.version over to version, but platform
migrations run only on the variants that enable them. On any other the
new chart ignored cluster.version and rendered the v7.3 default, so a
7.4 cluster was downgraded and a 7.1 one moved onto a line nothing had
upgraded it to.

The render now fails when a stored cluster.version names another
release line than version selects, and says how to proceed, so the
release stays on the revision it runs until someone picks the line.
Where the migration ran the two agree and nothing changes.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil 448b12657b fix(foundationdb): stop offering the 7.1 line
The FoundationDB operator no longer carries the 7.1 client, so it can
neither manage nor upgrade a 7.1 cluster. The version map offered v7.1
all the same; it now lists the lines the operator carries, 7.3 and 7.4,
which is also what hack/update-versions.sh reads from the operator
chart.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil d5ebb62632 fix(platform): keep migration 60's record whole across a retry
A failed patch stops the migration before the stamp, and the retry
finds version already written on the releases it carried, so it never
reached the step that records a patch-level move. The move is now
worked out from the values alone, so a retried run records it too.

An OpenSearch release storing version: null rendered the old chart's
v2 default and is refused by the new one; it now goes on the record
before it turns up as a stuck release.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil 1636349a63 fix(api): point the removed image overrides at node registry mirrors
The warning for a removed image override told an air-gapped install to
use the platform-wide registry, but nothing in the platform writes
_cluster.images-registry, so there is nothing to set. What serves these
images offline is what serves every other platform image: the registry
mirrors configured on the nodes, as the air-gapped install guide sets
up. The tests that hold the cozy-lib.image routing now say what they
check rather than promising a platform knob.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil dfff5c2dac test(kubernetes): hold the routing of the busybox and unpin images
The busybox helper shared by four workloads and the pool's pre-delete
Job had no test holding their route through cozy-lib.image, so
reverting either line left both suites green.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil 8c2e20539d fix(platform): record every release migration 60 moves
A FoundationDB release carried to its line still moves to the one tag
the chart ships for it, which can be a step below what it runs, and
that showed up only in the Job log. Kubernetes and KubernetesNodes
releases pointing the removed image overrides at a mirror were not
listed at all. Both now go into the needs-attention record next to the
OpenSearch ones.

The migrations image carries no charts, so the tag map is repeated in
the migration, and the bats suite fails when it drifts from the
chart's versions.yaml.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil 044162583b fix(api): skip the removed-field warning on an empty default
The image overrides and the FoundationDB cluster.version were defaulted
to empty strings, the read path fills those defaults in, and a client
that writes back what it read stores them. The warning then fired on
releases that never set an image. For those keys only a value counts;
the Phase 2 worker-pool keys still warn on presence.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:16 +05:00
Andrei Kvapil a41015e309 fix(api): warn when a release still carries a field nothing reads
The apps API accepts and stores a values key the chart has stopped
reading, so a kubectl apply or a terraform apply carrying one succeeds
and changes nothing. The Phase 2 warning for the Kubernetes worker-pool
keys now covers the image overrides and the FoundationDB
cluster.version as well, one table per kind.

The version helpers name the value and the versions on offer when the
release values carry an explicit null, instead of failing on a template
type error that names neither.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil 033bec74fb fix(platform): keep one bad release from stopping migration 60
spec.values on a HelmRelease takes any shape, so a release holding a
scalar there, or a cluster.version the apiserver stored as a number,
crashed jq and stopped the carry-over for the whole management cluster:
every healthy FoundationDB release then went into the upgrade without
its version. Those releases are now reported and recorded like any
other value that cannot be carried over.

The same record covers OpenSearch releases that still set
images.opensearch. Nothing can be carried over for them, so what the
operator needs is the list.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil a887125ee5 build(apps): keep the version default out of the tag refresh
Both generators wrote the newest line as the default, so a run made to
pick up a patch tag also moved every release that never set a version
onto another major line. The default now stays where it is while it is
still supported, and a line that leaves the supported set is reported
instead of quietly leaving the enum.

FoundationDB takes the patch from the operator chart rather than from
the newest tag in the registry: the operator only carries the client
library of that build, and the generator now reproduces the committed
map exactly.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil d99a1b9f2f fix(kubernetes): route the helper images through cozy-lib.image
These images now go through cozy-lib.image like the harbor, kafka and
mariadb hook images, so they follow _cluster.images-registry wherever
that is set rather than any per-application value. An air-gapped
install serves them the way it serves every other platform image,
through the registry mirrors configured on its nodes.

The talos-csr-signer sidecar and the OIDC Jobs had no test holding
their image, which is where an override could come back unnoticed.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil f3058d2353 fix(platform): guard the FoundationDB version carry-over on all values
Migration 60 tested only cluster.version before adding version, so a
version set on the release between the scan and the patch would have
been overwritten. The patch now tests the whole values object it read.

resourceVersion is not used for this: helm-controller keeps writing the
release status, and a mismatch there would abort the platform upgrade
for no change to the values.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil 930051280b fix(platform): carry FoundationDB cluster.version over to version
The foundationdb chart no longer reads cluster.version, so a cluster
that set it to 7.1 or 7.4 would move to the v7.3 default on the first
reconcile after the upgrade: a full-bounce upgrade from 7.1, or a
downgrade from 7.4.

Migration 60 sets version to the release line of each cluster.version
it finds. It keeps cluster.version, because the old chart still renders
the release between the migration and the chart upgrade and would fall
back to its own default without it. A value outside the supported lines
is recorded on the cozystack-version ConfigMap and does not block the
upgrade.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00
Andrei Kvapil 46a9e12578 build(opensearch): generate the version map the way postgres does
hack/update-versions.sh listed the majors by hand and only rewrote
files/versions.yaml, so the Version enum and the default in values.yaml
drifted from the map on every refresh. It now takes the tested range
from the compatibility table of the pinned operator release, picks the
newest tag per major that has a matching Dashboards image, and rewrites
the enum and default along with the map, like postgres and mariadb.

The committed map keeps its current tags; the helper follows the
postgres idiom and no longer falls back to v2 on an empty version.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-10-23 19:14:15 +05:00