mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-22 04:33:36 +09:00
e53904e5646a78baffcba57fae581fec30e5c0f3
6797
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e53904e564 |
feat(cozy-lib): add ns-host-required apex guard helper
Add cozy-lib.ns-host-required: it returns the tenant apex like cozy-lib.ns-host, but fails the render when the platform has injected _namespace and its host is empty AND the app sets no host override (.Values.host). Charts that compose a service hostname as <sub>.<apex> otherwise emit a malformed trailing-dot host silently when the apex is unset. The guard is scoped to the injected state, so a standalone render (helm lint / helm template with no _namespace, or an empty _namespace map) stays renderable. This mirrors the gateway path, which fails on an empty apex only when a Gateway is actually rendered. Callers use it only at a hostname-emitting site, so a tenant with no apex and no public exposure is unaffected. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
784d290c42 |
docs(tenant): correct gateway param description to match behavior
The gateway parameter description claimed the chart auto-enables a Gateway for tenants whose apex is derived from the parent (empty host) and leaves it off for custom-apex tenants. The resolver tenant.gatewayEffective never inspects host: an unset gateway key always means the tenant owns no Gateway and publishes through the nearest ancestor that owns one, regardless of host. The unit tests in gateway_default_test.yaml already pin this real behavior. Rewrite the description to match the code and regenerate the schema, README, Go types and the tenant-rd CRD. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
29932ddbb0 |
test(tests): add a render smoke check over every app chart (#4018)
## What this PR does I added an offline render check for application charts because helm-unittest suites can leave templates outside their `templates:` lists unchecked. It runs in `make unit-tests` with fixtures for fresh, configured and wildcard certificate states. The fixtures cover injected cluster and namespace values, with checks for PostgreSQL scheduling, tenant OIDC groups, shared services and Harbor certificate settings. The two charts that require instance-type lookup are skipped only while every fixture fails with the expected lookup error. I also added ClickHouse tests for generating the backup API password and preserving an existing one. The render check uses empty lookup results and checks YAML rendering only. Closes https://github.com/cozystack/cozystack/issues/4057 ### Validation I ran `make unit-tests` and `make test-controllers`. All 16 render-matrix tests passed under bats and the shell runner with dash. Deliberately broken rendering, fixture validation and password generation or reuse made the tests fail. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added automated validation that charts render successfully across fresh, configured, and wildcard cluster states, including checks for conditional resources and scheduling settings. * Added coverage confirming ClickHouse backup API passwords are retained when a Secret already exists. * Expanded checks for invalid fixtures, skipped charts, and consistent rendered output. * **Documentation** * Documented the render scenarios and how fixture values represent cluster configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
56ceb3b427 |
docs(release): add changelog for v1.6.4 (#4534)
This PR adds the changelog for release `v1.6.4`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.6.4.md`.
|
||
|
|
59faddc98a |
docs: add changelog for v1.6.4
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com> |
||
|
|
cafc8204a5 |
fix(ci): let the fork e2e lane check out the pull request again (#4514)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Every fork pull request has a red "E2E Tests" status since the Renovate roll-up in #4447 moved `e2e-fork.yaml` to actions/checkout v4.4.0. That release refuses to check out a fork's pull request ref from a `workflow_run` workflow unless the step sets `allow-unsafe-pr-checkout: true`, so the publish job stops at its first step and the e2e run never starts. This sets the opt-in on both pull-request checkouts in the workflow, which gives back the v4.3.1 behaviour. The lane already takes fork code on purpose and its header describes the boundary: `publish` runs no fork-authored script and `e2e` holds no push token. A new unit test, `hack/fork-pr-checkout_test.bats`, fails if either checkout loses the opt-in. Removing it from one step turns the test red under both bats and `hack/cozytest.sh`. Closes #4513 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> Not applicable, CI-only change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(ci): run the fork e2e lane again after the actions/checkout v4.4.0 update ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Updated pull request checkouts in the fork workflow to allow the required checkout behavior for both publishing and end-to-end checks. - **Tests** - Added a check to verify that pull request checkouts enable this setting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
1bc7effadb |
fix(ci): let the fork e2e lane check out the pull request again
actions/checkout v4.4.0 refuses to check out a fork's pull request ref from a workflow_run workflow unless the step opts in. The Renovate roll-up that moved e2e-fork.yaml to v4.4.0 made the publish job fail at checkout for every fork PR, so the required "E2E Tests" status now goes red on all of them regardless of the change. The lane was built to take fork code this way: publish runs no fork-authored script and e2e holds no push token. Opting in restores the behaviour it had on v4.3.1, and a unit test fails if either pull-request checkout loses the opt-in. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
101d1d9861 |
chore(seaweedfs): bump chart to 4.45 (#4079)
## What this PR does Refreshes the vendored SeaweedFS chart from 4.31 to 4.45 through `make update`. This supersedes #2918, which stopped at 4.34: both things that bump was after (SIGHUP hot-reload of the JWT signing keys, and native `expires_after_seconds` under `global.seaweedfs.securityConfig.jwtSigning`) are in 4.45 too, along with three months of chart fixes. What changes in a rendered cozystack install: - Every SeaweedFS server moves from `chrislusf/seaweedfs:4.31` to `4.45`. The chart takes its image tag from `appVersion`, so the `master`, `volume` and `filer` StatefulSets and the `s3` Deployment all get a new image and roll. None of them sets an `updateStrategy`, so they fall back to RollingUpdate and restart one pod at a time, in every tenant that runs SeaweedFS. Selectors, `serviceName` and `volumeClaimTemplates` are byte-identical across the bump, so no immutable field is touched and no volume is recreated. - The mysql Secret and the `WEED_MYSQL_*` env block now render only for the mysql filer store, so the dummy `seaweedfs-db-secret` and its env vars drop out of a postgres2 install. That Secret is a `pre-install` hook resource and Helm never deletes hook resources on upgrade, so an existing one stays behind unused; the tenant cleanup hook still removes it when the tenant goes away. - The filer ServiceMonitor selects on a `monitoring: "true"` label that upstream moved from the filer-client Service onto the filer Service, so metrics come from one Service instead of two. See the upgrade note below. - The filer db-init ConfigMap volume is marked optional. - S3 gained a Lance catalog port (9101), rendered on the deployment and published on the Service. This package pins it off, see below. - The new per-component NetworkPolicy templates are opt-in and off by default. The rendered s3 port set is therefore unchanged from 4.31: `swfs-s3` and `metrics`, same as before the bump. ## Catalog listeners `weed s3` ships two table-catalog listeners that default to on: Lance on 9101, new in this bump and published on the s3 Service, and Iceberg on 8181, which predates it and stays on the pod IP. Both are served over plain HTTP with no way to supply a certificate, while every other port on this gateway is TLS by way of `s3-tls-main-port.patch`, and both expose writable namespace and table routes. Nothing in cozystack consumes either. - Lance is pinned off in `values.yaml`, which drops the container port and the Service port with it. - Iceberg could not be pinned off at all: the chart wraps `-port.iceberg` in an `if` and the value defaults to null, so the flag was dropped and weed served its own 8181. A new carried patch renders the flag the way 4.45 already renders `-port.lance`, so null reaches weed as 0. An explicitly configured port still wins in both cases. ## Volume resize hook The hook decides twice: one gate emits the `--cascade=orphan` delete of the volume StatefulSet, the other emits the PVC patches. Both were comparing sizes as strings, in different ways, and the two disagreed about what a size change is. The PVC gate reached its operands through an `include`, which returns a string, so byte counts were compared lexicographically. A grow that crossed a decimal exponent and lowered the leading digit (5Gi to 10Gi, 8Gi to 16Gi, 20Gi to 100Gi) read as a shrink: the patches were skipped while the delete above them still ran, leaving the StatefulSet recreated and the claims at the old size, hook exit 0 and nothing logged. The same comparison read a real shrink across an exponent as a grow and emitted a patch the API server rejects, failing the pre-upgrade hook after the delete had landed. The delete gate compared the live `volumeClaimTemplates` size against the desired one raw. Since the API server re-emits a quantity in canonical form, that never converges for a non-canonical spelling: values saying `10240Mi` come back from the cluster as `10Gi`, so the volume StatefulSet is deleted and recreated on every upgrade for a size that never moved, and no patch follows because the numeric gate correctly sees no growth. A third defect sits in the patch command itself, which formatted the desired size with `%s`. Helm decodes values through JSON, so a size written without quotes arrives as a float64 and the rendered command carried a Go format error where the quantity belongs. The hook fails on it, and it fails after the orphan delete has already run. It is `%v` now: `21474836480` renders as `2.147483648e+10`, which `resource.ParseQuantity` accepts and the API server stores as `21474836480`. The tenant app always quotes `size`, so no tenant could reach this. Carried as one patch normalising both gates through `seaweedfs.resource-quantity`, with unit tests on the exponent-crossing grow, a grow that always worked, a shrink, a respelling, and a size set as a number. ## Upgrade notes Two things to know before upgrading. `filer.ingress` is gone from the chart's values. Upstream replaced it with `filer.ingresses.http.*` and added `filer.ingresses.grpc.*`; those six removed keys are the only contract break in the whole values file across the bump. Nothing in tree sets the old key and the tenant-facing SeaweedFS resource has no filer-ingress field, so a default install is unaffected. An operator who set `filer.ingress.enabled: true` through a platform-level `spec.components.seaweedfs-system.values` override is not: Helm accepts the unknown key without complaint and the Ingress simply stops rendering, with nothing in an event or a log to read. Filer metrics stop being double-counted. Before this bump both `seaweedfs-filer` and `seaweedfs-filer-client` matched the filer ServiceMonitor, so Prometheus scraped two targets per filer pod and every filer series was duplicated. One target is the correct number, but panels built on `rate(SeaweedFS_filer_request_total{...})` and `rate(SeaweedFS_filerStore_request_total{...})` in the shipped dashboard step down by half at upgrade time. That step is the duplicate going away, not a throughput loss. Quantile panels are ratios and are unaffected. ## Patches Ten carried patches, all applying cleanly from a fresh download. `resize-api-server-annotation.diff` needed a rebase: upstream now renders pod-template labels on the resize hook, so the patch appends the cozystack apiserver label to them instead of creating the block. Two are new and are described above: `s3-iceberg-port-explicit.patch` and `volume-resize-quantity.diff`. Both belong upstream and are carried only until they land there. ## Verification Checked locally: a fresh download of 4.45 with all ten patches applied in Makefile order reproduces the committed tree byte for byte; `helm unittest` passes 73 in `packages/system/seaweedfs` and 35 in `packages/extra/seaweedfs`; the naming-guard canary still refuses a blind upgrade; and a rendered diff against 4.31 comes back identical for object names, StatefulSet selectors, `serviceName` and `volumeClaimTemplates`, so a running install keeps its workloads and volumes across the upgrade. Each fix above has a unit test that is red without it. Not checked locally, worth an eye in review or e2e: the live upgrade path, since `helm template` cannot exercise the `lookup` that preserves the security.toml JWT keys, and server-side apply convergence when the `monitoring` label moves between the two filer Services. ### Screenshots N/A, no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note chore(seaweedfs): bump the vendored SeaweedFS chart to 4.45, with the Lance and Iceberg table catalogs kept off and both gates of the volume resize hook comparing sizes numerically. Every SeaweedFS server moves to the 4.45 image, so the master, volume and filer StatefulSets and the s3 Deployment roll one pod at a time on upgrade. Two operator-visible notes: filer metrics stop being double-counted, so filer rate panels step down by half at upgrade time, and the chart's `filer.ingress` values key is replaced by `filer.ingresses.http`, so an override still using the old key is silently inert. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Updated SeaweedFS to version 4.45, with expanded chart configuration for enterprise licensing, NetworkPolicies, custom scheduling, certificate-based SFTP authentication, and existing-secret credentials. * Added separate HTTP and gRPC filer ingress options, S3 Lance ingress and listener settings, and Lance worker support. * Added a Grafana dashboard view for filer object-size distributions. * **Improvements** * Added configurable node ports and load-balancer settings across services, plus custom certificate DNS names. * Improved volume-resize handling for equivalent storage quantities and added support for resizing PVCs. * **Documentation** * Expanded chart guidance for persistence, credentials, networking, licensing, and MySQL configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
9a92537b19 |
ci(workflows): pin flux version across CI toolchain steps (#4442)
## What this PR does The flux CLI was installed across the build, PR, tag, nightly and e2e-fork workflows and in the e2e-sandbox image through the vendor `install.sh` with no version pin, so each install resolved the installer's floating `latest`. `latest` had advanced to v2.9.5, and a matrix job on #4429 failed when that release's checksums transiently 4xx'd during install. Pinning does not stop the checksum host from returning a transient error, so it is not a fix for that flake; what it fixes is the uncontrolled version. This sets `FLUX_VERSION` to 2.8.6 and guards each flux install on the installed client version (workflows, with a fixed-string match) or passes it to `install.sh` (the e2e-sandbox image), matching the already-pinned "Set up promotion toolchain" steps, so the flux CLI is fixed and reproducible across CI with no flux install resolving floating `latest`. This is scoped to the flux CLI only: other tools some of these steps install (yq, helm, mc) are not touched here, and the yq floating-latest is tracked separately in #3640. ### Screenshots <!-- REQUIRED for UI changes. Not applicable: this PR changes CI workflows and a test image only. --> Not applicable — CI and test-image change, no UI impact. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note ci(workflows): pin the flux CLI version across CI toolchain steps and the e2e-sandbox image so the flux CLI is fixed and reproducible instead of following the installer's floating latest ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Build, release, and end-to-end test environments now use Flux CLI version 2.8.6. * Flux is installed when it is missing or does not match version 2.8.6; an existing matching installation is reused. * The end-to-end sandbox image now installs Flux version 2.8.6 by default. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6e0c998b97 |
fix(seaweedfs): compare volume sizes as numbers in the resize hook
Both sides of the size comparison reach it through an include, which returns a string, so the hook compared byte counts lexicographically. A grow that crossed a decimal exponent and lowered the leading digit (5Gi to 10Gi, 8Gi to 16Gi, 20Gi to 100Gi) read as a shrink: the PVC patches were skipped while the --cascade=orphan delete above them still ran, so the volume StatefulSet was recreated and the claims stayed at the old size, hook exit 0 and nothing logged. The same comparison read a real shrink across an exponent as a grow and emitted a patch the API server rejects, failing the pre-upgrade hook after the delete had landed. The gate that emits the --cascade=orphan delete compared the live volumeClaimTemplates size against the desired one as raw strings, so it disagreed with the numeric gate below it about what a size change is. The API server re-emits a quantity in canonical form, which makes this permanent rather than one-off: values spelling 10Gi as 10240Mi come back from the cluster as 10Gi, the strings never converge, and the volume StatefulSet is deleted and recreated on every upgrade for a size that never moved. Normalise both sides through seaweedfs.resource-quantity. The hook also formatted the desired size with %s. Helm decodes values through JSON, so a size written without quotes reaches the template as a float64 and the patch command carried a Go format error in place of the quantity; it now renders with %v. Carry the comparison as a patch until upstream takes it, and cover both directions, the respelling and the unquoted size. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
ba46b9fc60 |
ci(testing): download the flux installer with --fail in the e2e-sandbox image
The flux install piped `curl -sSL` into bash, so on a 4xx/5xx the error page was piped into the shell and curl's exit code was lost. Download the installer to a file with `curl -fsSL` first, then run it, so an HTTP error fails the build step instead of feeding bash a bad script. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
95e68bad86 |
ci(workflows): use a literal match in the flux version guards
The guard ran `grep -qx "flux: v${FLUX_VERSION}"`, where the dots in the
version are regex metacharacters, so a client printing `flux: v2X8X6`
would satisfy it. Use `grep -Fqx` for a fixed-string match in the build
toolchain guards this change added.
Also correct the step comments that still read "install if absent
(idempotent)": the step now installs whenever the pinned version is not
the one present, not only when flux is missing.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
|
||
|
|
03c6c13443 |
ci(testing): pin flux version in the e2e-sandbox image
The e2e-sandbox image fetched flux through `install.sh` with no version, so the build resolved the installer's floating "latest". Add an ARG FLUX_VERSION, matching the neighbouring COZYHR_VERSION and CRUST_GATHER_VERSION pins, and pass it to install.sh so the sandbox's flux CLI is fixed and reproducible like the rest of the image's tools. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
8c0f91ff00 |
ci(workflows): pin flux version in remaining install-latest steps
The nightly toolchain steps still fetched flux through the installer's floating "latest". Pin them to FLUX_VERSION 2.8.6 with the same version-gated guard so nightly's flux push artifact step uses a fixed, reproducible CLI rather than whatever latest resolves to. Also switch e2e-fork's already-pinned step from a presence check to the same version gate, so a runner with a pre-cached older flux is corrected rather than left as is, and fix that step's comment to point at the flux distro cozystack actually ships (internal/fluxinstall/manifests/fluxcd.yaml, v2.8.0). Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
edbc4f1b3e |
ci(workflows): pin flux version in build toolchain steps
The "Set up build toolchain" steps installed flux with a bare `curl https://fluxcd.io/install.sh | sudo bash`, letting the installer resolve its floating "latest". latest had advanced to v2.9.5, and a matrix job on PR #4429 failed when that release's checksums transiently 4xx'd during install. Pin FLUX_VERSION to 2.8.6 and guard on the installed client version, the same shape the "Set up promotion toolchain" steps already use, so the flux CLI is fixed and reproducible across CI instead of following latest. This does not remove the checksum host's transient errors; it removes the uncontrolled version bump. Related to the yq floating-latest pin in #3640. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
b113cc77d5 |
fix(kafka): delete the release's node pools on uninstall (#4490)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Deleting a Kafka application left its brokers running. The migration hook creates the KafkaNodePools with `kubectl apply`, so they are not in the release manifest and `helm uninstall` does not remove them, and each pool owns the StrimziPodSet that runs its brokers. The pre-delete hook now deletes the release's pools, selected by `strimzi.io/cluster=<release>`, which is the label the migration hook stamps. Its Role gains `list` and `delete` on `kafkanodepools`, which is what a label-selected delete issues. Fixes #4488. The pools are deleted after the KafkaTopics, also when the topic wait timed out. The topic operator clears topic finalizers through the brokers, so the brokers have to outlive it. Deleting the pools before Helm deletes the Kafka CR does not bring brokers back. For the few seconds in between, Strimzi 0.45.1-rc1 (the version the operator chart pins) fails the reconcile with `KafkaNodePools are enabled, but no KafkaNodePools found` before it builds any StrimziPodSet. A regular pool is the ownerReference of its StrimziPodSet (`NodePoolUtils.createKafkaPools` at that tag), so the pods go away with the pool by garbage collection. If the Kafka CR is already gone when the hook runs, the pools are still found by label. An absent CRD exits 0, and so does a release with no pools, because a label-selected delete that matches nothing succeeds. Any other failure exits 1. The call uses `--wait=false` and no request timeout, so the #4276 invariants and the 270s deadline arithmetic in `delete.yaml` stay as they were. The cleanup lives in the pre-delete hook because only there a failure gets retried. In Helm v4.1.1, which helm-controller v1.5.0 vendors, a pre-delete hook error returns from `Uninstall.Run` before anything is deleted, so the next reconcile runs the hook again. A post-delete error is only collected: the release is still purged and Helm never runs its hooks again, so a failure there would leave the pools for good. An ownerReference from each pool to the Kafka CR would also work. I did not take it because it changes what the migration hook applies on every install and upgrade, including the ZooKeeper migration path that adopts a namespace-wide `kafka` pool. PVCs are out of scope. The pools set `deleteClaim: false`, so Strimzi leaves the broker and controller volumes in place when a pool is deleted, before and after this change. Not covered by this change: - Pools leaked by releases deleted before this fix stay where they are. Find them with `kubectl get kafkanodepools.kafka.strimzi.io -A -L strimzi.io/cluster`; a pool whose cluster has no Kafka CR in its namespace (`kubectl get kafkas.kafka.strimzi.io -A`) is leaked, and `kubectl -n <namespace> delete kafkanodepools.kafka.strimzi.io -l strimzi.io/cluster=<cluster>` removes it and its brokers. - `examples/backups/kafka-metadata/cleanup.sh` returns without waiting for the Kafka uninstalls, unlike `examples/backups/kafka/cleanup.sh`, which is why the metadata suite's new checks carry a 12m budget. The new step already holds the suite until the pools and pods are gone, so the next suite no longer starts into a namespace that is still uninstalling. Adding the same `wait_deleted` calls to the script would let those checks drop to the default timeout; left as a follow-up because the example is outside this change. - The "Deletion and PVC retention" section of `packages/apps/kafka/README.md` does not yet say that the pre-delete hook now removes the node pools while `deleteClaim: false` keeps the PVCs. That one sentence is left as a follow-up. - Helm runs the hooks stored in the release's latest revision, so a release uninstalled before it has been upgraded onto this chart still runs the old hook and still leaks its pools. Tests: - `packages/apps/kafka/tests/delete_test.yaml`: a new case pins, in one pattern, that the pool delete follows the topic delete, is scoped to this release, and does not wait, and pins the exact Role rule. - `hack/kafka-pre-delete-hook.bats` runs the rendered hook against a fake kubectl. Its fake now answers the pool call separately, and four new cases pin the exit codes and ordering: pools are deleted after the topics, still deleted after a topic-wait timeout, an absent pool CRD exits 0, and a failed pool delete exits non-zero. - Each new pin was checked by a mutation that turned it red: widening the selector, moving the pool delete before the topics, waiting on it, exiting on an absent CRD, not exiting on a failure, skipping the pools on the timeout branch, and removing or widening the Role rule. - E2E: the `kafka`, `kafka-2-backup-roundtrip` and `kafka-3-metadata-roundtrip` tests each gain a step, `verify-uninstall-removes-node-pools`, that checks, after the apps are deleted, that no KafkaNodePool and no pod labelled `strimzi.io/cluster=<cluster>` remains. The pod check covers every Strimzi pod of the cluster, not only the brokers, on purpose. The `kafka` test now deletes its app itself so it can look afterwards. E2E success criterion, registered before the run: in this PR's E2E run, every `verify-uninstall-removes-node-pools` step (one in each of the tests `kafka`, `kafka-2-backup-roundtrip` and `kafka-3-metadata-roundtrip`) passes on the first attempt. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches one chart template and its unit tests, one hack unit test, and two e2e suites. No values, schema, README, package list, hack tooling, CRD or node requirement changes, so no entry of the trigger map applies. ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(kafka): deleting a Kafka application now also deletes its KafkaNodePools, so its brokers stop instead of running on after the application is gone. Volumes are kept, as before. Node pools left behind by applications deleted before this fix are not removed automatically: delete them with `kubectl -n <namespace> delete kafkanodepools.kafka.strimzi.io -l strimzi.io/cluster=<cluster>`. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Kafka app uninstall now removes associated Kafka node pools after attempting to remove Kafka topics. If topic cleanup times out, uninstall continues; node pool cleanup failures are reported. * End-to-end checks now verify that Kafka node pools and pods are removed after uninstall and backup cleanup. If resources remain, diagnostic details are provided. * **Tests** * Added coverage for node pool cleanup outcomes, including missing resource definitions and deletion failures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2c2c0301f7 |
chore(deps): roll up pending Renovate updates and move to Go 1.27.1 (#4447)
## What this PR does This replaces the open Renovate PRs with one rollup. Go modules move to their newest releases that keep the Kubernetes libraries and controller-runtime where they are, the toolchain and builder images move to Go 1.27.1, and the first-party code gets the Go 1.27 modernizers. Tests for the code the bumps touch come first, in their own commits, and pass on both the old and the new dependencies. Before the bumps: - The e2e sandbox image builds again: dl.min.io answers 410 for mc since MinIO archived it, so mc now comes from its last GitHub release with a pinned checksum. - Renovate stops offering tags of the apimachinery fork. They mirror upstream, so #2999 is a downgrade that drops our patch. - `make unit-tests` now runs the tests of the nested Go modules (kubevirt-csi-driver, token-proxy, kubeovn-webhook, api/apps/v1alpha1). They never ran in CI before. - New tests: JWT verification in the dashboard token proxy, the kube-ovn plunger metrics, cert-manager object idempotency in the tenant gateway, values-file layering in the operator, and the kubevirt CSI driver wrapper. - One real fix. The lineage webhook decoded admitted objects with encoding/json, so an int64 above 2^53 came back rounded. jsonpatch v2.5 compares numbers exactly and would emit a replace that rewrites such a field in the pod. The webhook now decodes with apimachinery's json. Held back on purpose, with the reasons in the commit messages: - Kubernetes libraries stay on v0.35. #3001 needs the apimachinery fork rebased onto release-1.37 first. - cel-go v0.29.2 and etcd v3.6.15, the lines k8s.io/apiserver v0.35 is built and tested with. - grpc v1.83.2. v1.84.0 is affected by GO-2026-6443 and no v1.84 patch release carries the fix yet. - kubevirt.io/api and the CDI API stay on what upstream csi-driver builds against. From CDI v1.60 it no longer compiles. - api/apps/v1alpha1 keeps go 1.26.4, so consumers of the published types are not forced onto Go 1.27. - go fix runs without embedlit (Go 1.27-only syntax, breaks backports to release branches on Go 1.26) and omitzero (would make metadata, spec and status required in the published OpenAPI schemas), and one errorsastype rewrite that did not compile is left out. govulncheck reports nothing reachable in any of the five modules, standard library included. Supersedes #4311, #4310, #4309, #3452, #3993, #3927, #3902, #3805, #3451, #3415, #3414, #3413, #3302, #4050 and #3363. #3001 stays open. Not checked locally: the images that build upstream projects (kube-ovn, kamaji, cilium, linstor-csi and others) with the Go 1.27 builder. The image build jobs of this PR cover them. Closes #4487. ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note chore(deps): move the toolchain and builder images to Go 1.27.1 and raise Go dependencies to their latest compatible releases, closing the published advisories; the lineage webhook no longer rewrites int64 fields above 2^53 in admitted objects. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a restore resource policy option for Velero strategies. * Added a test target to run Go tests across tracked modules. * **Bug Fixes** * Updated the end-to-end test environment to download and verify the MinIO client from its current release location. * Added coverage for token validation, webhook patching, controller behavior, and reconciliation. * **Maintenance** * Updated the Go toolchain, dependencies, and container image versions. * Refreshed pinned workflow actions and base image digests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
7002397e40 |
test(kafka): assert that uninstall leaves no node pool or broker behind
The Kafka suites passed while the node pools of the apps they had deleted, and the brokers those pools own, kept running into the suites that came after. Each suite now checks, after its apps are gone, that no KafkaNodePool and no pod labelled strimzi.io/cluster=<cluster> remains in the namespace. The kafka suite deletes its app explicitly so it can look afterwards. The metadata roundtrip's cleanup does not wait for the uninstalls, so its checks carry a budget sized from the pre-delete Job's deadline. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c64fec66ff |
fix(kafka): delete the release's node pools on uninstall
The migration hook creates the KafkaNodePools with kubectl apply, so they are not in the release manifest and helm uninstall leaves them. Each pool owns the StrimziPodSet that runs its brokers, so after the application is deleted every broker keeps running, and keeps its memory request, with nothing left to remove it. The pre-delete hook now deletes the pools selected by strimzi.io/cluster=<release> once the KafkaTopic delete has finished, including when that wait timed out: the topic operator clears topic finalizers through the brokers, so the brokers have to outlive it. The call does not wait, so the Job's deadline arithmetic is unchanged. An absent CRD is not a failure. Any other failure exits non-zero: once the uninstall completes Helm never runs the hook again, so a retry is the chart's last chance to remove the pools. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
4a2dea2d08 |
feat(cluster-api): add the Proxmox infrastructure provider and in-cluster IPAM as optional packages (#4087)
## What this PR does **Series behind #3481** — three separable pull requests: 1. **#4087 — Cluster API providers** (capmox + in-cluster IPAM) 2. #4088 — the `substrate` switch in `kubernetes` and `kubernetes-nodes` 3. #4089 — the Proxmox CSI driver and cloud-controller-manager #4087 and #4088 share no files and can be reviewed in parallel. #4089 builds on #4088. --- This is the provisioning layer #3481 asks to specify first: the part that creates the VMs. Neither of the other two is needed to review it. ### What it adds Two Cluster API providers, packaged the way the existing KubeVirt provider is — vendored components, `gzip -9 -n` for a reproducible artifact, a `make components` target that regenerates it from the upstream release and checks the downloads against the SHA-256 sums the Makefile pins: | Package | Version | Contract | |---|---|---| | `capi-providers-infraprovider-proxmox` | capmox v0.7.7 | v1beta1 | | `capi-providers-ipam-in-cluster` | CAIP v1.0.3 | v1beta1 | capmox v0.7.7 rather than the current v0.9.x on purpose: from v0.8 it declares contract `v1beta2`, which the CAPI core shipped here (v1.10.1) does not serve. Both join the `iaas` bundle as **opt-in** packages, where the KubeVirt infrastructure provider is unconditional. The asymmetry is deliberate: capmox drives a hypervisor outside the cluster and needs a Secret holding a Proxmox API token, which capi-operator reads before it installs anything — with no Secret the provider is never deployed at all. Enabling it by default would ask every Cozystack install that has no Proxmox to supply hypervisor credentials. ```yaml bundles: enabledPackages: - cozystack.capi-provider-infra-proxmox # brings IPAM with it - cozystack.capi-provider-ipam-in-cluster # also enableable on its own ``` Enabling capmox emits the IPAM provider automatically — capmox allocates worker addresses from an `InClusterIPPool` and has no DHCP mode, so on its own it never gets a `ProxmoxCluster` to `Ready`. IPAM stays independently enableable, since assigning addresses is a generic Cluster API concern. Naming capmox in `enabledPackages` while disabling IPAM stops the render. The cluster would catch that combination on its own — the capmox `PackageSource` depends on the IPAM `Package`, so with that `Package` absent capmox goes `Ready=False` with `DependenciesNotReady` and no `HelmRelease` is created — but it would not say why. Failing the render names the two conflicting list entries at the moment someone writes them, instead of leaving a condition to decode. ### Why the earlier attempt did not produce a VM #1927's blocker — "ProxmoxMachine does not produce a VM" — was not a capmox incompatibility. It was four independent causes, each of which only became visible once the previous one was cleared: 1. **`PROXMOX_URL` included `/api2/json`.** capmox appends that itself, so every call went to `/api2/json/api2/json/…` and returned 501. 2. **No IPAM provider.** `no matches for kind "InClusterIPPool"`; the `ProxmoxCluster` never reached `Ready` and every `ProxmoxMachine` sat in `WaitingForClusterInfrastructure`. 3. **`templateSelector.matchTags` is set equality, not containment** (`slices.Equal` after sort). A template carrying four tags is not selected by naming one of them: `found 0 VM templates with tags`. 4. **capmox waits for the qemu-guest-agent, which Talos does not ship.** `error waiting for agent: timed out` — cleared with `checks.skipQemuGuestAgent` and `skipCloudInitStatus`. None of these are in the provider. They are packaging and configuration, which is why the earlier work could look complete and still never boot a machine. ### Evidence Run on Proxmox VE 9.2.11 with Talos v1.13.6 and Cozystack's CAPI stack (core v1.10.1, CABPT v0.6.12, Kamaji control-plane provider v0.19.0): ``` $ kubectl get proxmoxmachine,machine -n tenant-proxmox proxmoxmachine/kubernetes-px-md0-… Ready=True VMProvisioned=True machine/kubernetes-px-md0-… Running proxmox://0ef88cc1-6aea-4563-af64-344d8060d5c0 $ qm list 106 kubernetes-px-md0-jmzds-h9npx running ``` The machine boots Talos from the nocloud ISO, reads the machineconfig capmox writes, and joins. That is the layer #69 called a hard blocker. ### What this PR does not claim - One Proxmox node, one tenant cluster, manual runs — there is no e2e coverage for Proxmox in CI, and this PR does not add any. - It does not make Proxmox a supported platform. It makes the provisioning layer reviewable, which #3481 names as the precondition for that conversation. - It does not touch `capi-providers-core`. The same drift check, pointed at that package, shows that the `startupProbe` from #2946 never reached the shipped blob; that fix changes what every new install runs and is #4483, with its own release note. - Nothing here exposes `ProxmoxCluster.spec.credentialsRef` to a tenant, and nothing should: `capmox-manager-role` grants get/list/watch/patch on Secrets cluster-wide and that field takes a namespace. Whatever renders a `ProxmoxCluster` for a tenant in the rest of the series has to keep the field out of tenant hands. ### Testing `make unit-tests` — exit 0. New suites: `packages/core/platform/tests/sources_capi_provider_proxmox_test.yaml` and `bundles_proxmox_wiring_test.yaml` (both bundle paths, the enable/disable precedence, and the refusal above). Every negated `containsDocument` in the wiring suite carries `any: true`; without it the negation passes whether or not the document is in the render. Three mutations, each reverted afterwards — emitting the capmox `Package` unconditionally, emitting the IPAM `Package` unconditionally, and dropping the `disabledPackages` check from `$proxmoxEnabled` — each turn red the tests named after the property they break, and the unmutated chart stays green (44 suites, 198 tests). `hack/capi-provider-components_test.bats` covers both new provider packages: the blob and the YAML are the same bytes; the Makefile version, the ConfigMap name and `spec.version` are one value — `applied-spec-hash` covers the provider spec and the config Secret but not the ConfigMap, so a bump that regenerates the files while `spec.version` stays behind installs the old provider silently; and the vendored files match the SHA-256 sums the Makefile pins, so the pin holds even when nobody runs `make components`. The render was compared against `main` for five configurations (defaults, `isp-full`, `isp-full-generic` with paas, an unrelated `enabledPackages`, and `disabledPackages` naming IPAM). No line is removed in any of them and the `Package` count is unchanged; the only additions are the two `PackageSource` objects, which the chart renders for every package, optional or not. ### Screenshots Not applicable — no UI change. ### Downstream repositories I walked the trigger map in `docs/agents/contributing.md` against this diff, file by file. What it matches is below. **No box is ticked**, because every follow-up it points at depends on a decision that is not mine to make: #3481 asks whether Proxmox becomes a supported platform at all, and writing its documentation or its Terraform schema before that is answered would be work built on an assumption. I have not filed speculative PRs or issues in those repositories for the same reason. If the direction is accepted, I open them; a human should decide which. **cozystack/website — matches, no follow-up yet.** - `packages/core/platform/values.yaml` changed. `content/en/docs/next/operations/configuration/platform-package.md` is a hand-written table of the `spec.components.platform.values.*` keys; this PR does not add a key, but it documents two new values that `bundles.enabledPackages` now accepts, and the table is where a reader would look for them. - Two platform components are added (`capi-providers-infraprovider-proxmox`, `capi-providers-ipam-in-cluster`) → `guides/platform-stack/_index.md`, `operations/configuration/licenses.md`. - The `iaas` bundle gains two optional packages → `operations/configuration/variants.md`. **Everything else: no match.** Nothing under `packages/apps/` or `packages/extra/` (terraform-provider, website app lists), no `packages/core/installer/values.yaml` or `networking.*`/`publishing.*` key the Ansible role sets, no `hack/` file or package-layout change (ccp, external-apps-example), no `ApplicationDefinition` semantics, no telemetry metric, no node prerequisites (talm, ansible), no commit-convention change (community). - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(cluster-api): add optional Cluster API packages for Proxmox — the capmox infrastructure provider and the in-cluster IPAM provider it requires. Neither is deployed unless named in `bundles.enabledPackages`, since capmox drives a hypervisor outside the cluster and does nothing without Proxmox API credentials. Enabling the Proxmox provider brings the IPAM provider with it. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added optional Proxmox infrastructure provider support for IaaS clusters. - Added in-cluster IP address management, usable independently or alongside Proxmox. - Proxmox deployments require a pre-created credentials Secret and automatically include in-cluster IPAM. - Added validation for incompatible Proxmox and IPAM settings. - **Tests** - Added coverage for provider dependencies, package enablement, disabling behavior, invalid configurations, and packaged provider components. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
e883f6f7fc |
fix(capi): ship the core provider manifests the repository actually holds (#4483)
## What this PR does `capi-providers-core` ships the Cluster API core manifests twice: as `files/core-components.yaml`, which a reviewer reads, and as `files/components.gz`, which the chart mounts into the cluster. #2946 added a `startupProbe` to `capi-controller-manager` and edited only the YAML; the blob was never regenerated, so the probe never reached a cluster. This regenerates the blob from the YAML (`gzip -9 -n -c files/core-components.yaml > files/components.gz`), so the decompressed blob matches the YAML byte for byte, and the only change it carries is the six lines of #2946. ### What this reaches, and what it does not New installs only. capi-operator re-applies a provider when the hash of its spec and its config Secret changes (`applied-spec-hash`); the ConfigMap holding the components is in neither, so a cluster that already runs `capi-providers-core` keeps its manager without the probe until the provider's version moves. Whether existing clusters should get the probe, and how, is left open on purpose: moving `spec.version` and the ConfigMap name together would force the re-apply, but that is a version decision for the package, not a side effect of regenerating a blob. ### Also in this diff - `.helmignore`: `files/.*-components.yaml` is a glob, so it matched only names beginning with a dot, and `helm package` shipped the readable YAML alongside the blob that duplicates it. Now `files/*-components.yaml`. - `hack/capi-core-components_test.bats` holds the two files to the same bytes from now on; it goes red on a YAML edited without regenerating the blob. Found by the drift check written for the Proxmox provider packages in #4087, pointed at the package that was already here; split out because it changes what every new install runs and deserves its own release note. ### Testing `bats hack/capi-core-components_test.bats`: 1/1; red with a line appended to the YAML, green again after `gzip -9 -n -c`. `helm package` of the chart now contains `files/components.gz` and `files/metadata.yaml` only. ### Screenshots Not applicable, no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note fix(cluster-api): ship the `startupProbe` from #2946 in the packaged core provider components. New installs get it; a cluster that already runs the core provider keeps its manager without the probe until the provider's version moves, because capi-operator re-applies components only when the provider spec or its config Secret changes. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added a check that the compressed core components match the YAML source, with a diff provided when they differ. * **Chores** * Updated chart packaging exclusions to cover component YAML files regardless of whether their names begin with a dot. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
a285a5ef36 |
feat(kubernetes): add a proxmox substrate for managed clusters and worker pools (#4088)
## What this PR does **Series behind #3481** — three separable pull requests: 1. #4087 — the Cluster API providers (capmox + in-cluster IPAM) 2. **#4088 — the `substrate` switch** in `kubernetes` and `kubernetes-nodes` 3. #4089 — the Proxmox CSI driver and cloud-controller-manager #4087 and #4088 share no files and can be reviewed in parallel. #4089 builds on this one. --- #4087 adds the Cluster API providers; this PR teaches the two managed-cluster charts to point at them. ### The shape A single value, `substrate: kubevirt | proxmox`, on both charts — not a parallel set of charts. The control plane is Kamaji either way; what changes is the infrastructure half: | | `kubevirt` (default) | `proxmox` | |---|---|---| | `Cluster.infrastructureRef` | `KubevirtCluster` | `ProxmoxCluster` | | worker template | `KubevirtMachineTemplate` | `ProxmoxMachineTemplate` | | apiserver Service | `ClusterIP` | `LoadBalancer` | | tenant CCM/CSI | in-cluster (`kccm`, kubevirt-csi) | rendered off; the Proxmox pair is #4089 | The KubeVirt path renders byte-for-byte what it rendered before. Checked by rendering `kubernetes-nodes` with default values at `main` and at this head and diffing the output, not only by the snapshots — the snapshots cover `nodegroup.yaml` alone, and an earlier revision of this branch changed the reconcile Job while they stayed green. The parent chart's default render differs only in the random fields of `talos-secrets`, which differ between two renders of `main` as well. `osImage` (#4171) is refused on the proxmox substrate. Only half of it could be honoured there — `factory.version` reaches the machineconfig's installer image while the worker disk keeps coming from the Proxmox template — and a field that half-works is worse than one that says no. `talos.version` and `talos.schematicID` still set the installer target. ### Three things that only show up once workers are off-cluster **The apiserver endpoint.** A KubeVirt worker runs inside this cluster and reaches the tenant apiserver by ClusterIP. A Proxmox worker does not: it lives on the hypervisor's L2 and cannot route to `10.96.0.0/16`. The machine provisions fine, `VMProvisioned=True`, `providerID` set — and then never joins, with no CSR to show for it. On the proxmox substrate the Kamaji Service is therefore a `LoadBalancer`, and the reconcile Job reads `.status.loadBalancer.ingress[0].ip` instead of `.spec.clusterIP`. **And DNS, for the same reason.** The worker machineconfig carried `nameservers: ${COREDNS_IP}` — the management cluster's CoreDNS ClusterIP, equally unreachable. The proxmox path takes its own resolvers (`proxmox.dnsServers`) and the chart refuses to render without them, rather than writing an address that cannot answer. **`externalManagedControlPlane` and the managed-by annotation.** capmox's webhook nil-derefs on a `ProxmoxCluster` whose control-plane endpoint is empty unless `spec.externalManagedControlPlane` is set, which is correct here — Kamaji owns the endpoint. Separately, the `cluster.x-k8s.io/managed-by: kamaji` annotation that the KubeVirt path carries must **not** be set on the Proxmox object: with it, CAPI never sets an ownerRef and capmox stops at `Waiting for Cluster Controller to set OwnerRef`, leaving the address pool uncreated and every worker waiting on infrastructure. ### Linked clones by default `proxmox.full: false` — worker disks are linked clones of the template rather than full copies. Measured on a ZFS-backed store: **8 KiB per worker instead of 10.2 GiB**. Operators who want independent disks set `full: true`; the value is plumbed straight through to `ProxmoxMachineTemplate`. `proxmox.storage` is a full-clone parameter and is refused with a linked clone: a linked clone always lives on the template's storage, and both Proxmox (`parameter 'storage' not allowed for linked clones`) and capmox's own CRD (`Must set full=true when specifying storage`) reject the pair. The chart says so at render time, naming both values, instead of failing at apply time with an error about a `ProxmoxMachineTemplate` nobody wrote. ### Sizing `ProxmoxMachineTemplate` takes `numCores` and `memoryMiB`, not Kubernetes quantities, so the pool's `resources.cpu`/`memory` are converted (millicores → cores, bytes → MiB, both rounded up). The rounding is pinned in `tests/substrate_branches_test.yaml` with `1500m` and `8G`, where rounding up and down disagree; `tests/substrate_proxmox_sizing_test.yaml` covers the refusal to size a pool that gives neither. ### Known upstream behaviour this surfaces `TalosConfigTemplate.spec` is immutable by schema, so the reconcile Job cannot rewrite an endpoint it has already published — `kubectl apply` fails with `TalosConfigTemplate.Spec is immutable`, and the template has to be deleted for the Job to recreate it. A fresh install never meets it; it appears when the apiserver address changes on an existing cluster. Handling it changes the KubeVirt path too, so it is not in this PR: it is a separate change on `proxmox/tct-replace`, with its own upgrade note. ### Evidence ``` $ KUBECONFIG=<tenant> kubectl get nodes -o wide kubernetes-px-md0-jmzds-ndnl6 Ready <none> 75s v1.35.6 10.0.0.160 Talos (v1.13.6) $ KUBECONFIG=<tenant> kubectl get pods -A cozy-cilium cilium-… Running kube-system konnectivity-agent Running kube-system coredns-… Running ``` No management-cluster address remains in the rendered `TalosConfigTemplate`. ### Testing `make unit-tests` — exit 0. `apps/kubernetes` 26 suites / 314 tests; `apps/kubernetes-nodes` 29 suites / 160 tests / 12 snapshots, all unchanged on the default path. Each of the ten commits passes both suites on its own. Every substrate branch is asserted on both sides, and every assert goes red on the mutation it exists for: inverting the `SVC_IP` condition, inverting the `nameservers` condition, `ceil`→`floor` in the sizing, removing the proxmox gate from each of the kccm and csi templates (the two RBAC bindings included), pinning the retention lookup back to `KubevirtMachineTemplate`, widening the IPv6 class of the `dnsServers` validation, dropping the parent chart's `dnsServers` validation, the storage-with-linked-clone refusal, and the `osImage` refusal. Negated `containsDocument` asserts carry `any: true`. `hack/talos-reconcile-heredoc_test.bats` additionally runs the rendered heredoc through a real shell: a `dnsServers` entry with a substitution is refused and named, an IPv6-shaped one with a substitution is refused too, and a real address survives the shell unchanged. ### Screenshots Not applicable — no UI change. ### Downstream repositories I walked the trigger map in `docs/agents/contributing.md` against this diff, file by file. What it matches is below. **No box is ticked**, because every follow-up it points at depends on a decision that is not mine to make: #3481 asks whether Proxmox becomes a supported platform at all, and writing its documentation or its Terraform schema before that is answered would be work built on an assumption. I have not filed speculative PRs or issues in those repositories for the same reason. If the direction is accepted, I open them; a human should decide which. **cozystack/terraform-provider-cozystack — matches, no follow-up yet.** - `packages/apps/kubernetes/values.schema.json` and `packages/apps/kubernetes-nodes/values.schema.json` both gain fields: `substrate`, and a `proxmox` object (`allowedNodes`, `dnsServers`, `ipv4Config`, and on the pool side `templateTags`, `network`, `storage`, `full`). The provider hand-maintains a schema, a model and an expand/flatten pair per Kind, with no codegen from these files. - `values.yaml` gains defaults for those fields (`substrate: kubevirt`, `proxmox.full: false`, `prefix: 24`). The provider restates defaults it models and sends its own, so a stale one wins silently rather than showing as a diff. **Everything else: no match.** No app added, renamed or removed (website app lists); no version enum, `kind`, `plural`, `release.prefix` or output Secret/Service name changed; `packages/core/platform/values.yaml` untouched; the `*-rd` files are regenerated, not renamed (ccp); no `hack/` change; no node prerequisites (talm, ansible); no telemetry metric; no commit-convention change (community). - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(kubernetes): add a `substrate` value to the `kubernetes` and `kubernetes-nodes` applications, selecting whether worker VMs run on KubeVirt in this cluster (`kubevirt`, the default and unchanged) or on an external Proxmox VE cluster through capmox (`proxmox`). On the Proxmox substrate the tenant apiserver is published through a LoadBalancer and worker disks default to linked clones. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for choosing KubeVirt or Proxmox for Kubernetes clusters and node pools. * Added Proxmox configuration for nodes, networking, storage, DNS, templates, cloning, and IPv4 allocation. * Proxmox deployments now use the appropriate cluster, machine, networking, and control-plane resources. * **Validation** * Added checks for supported substrates, required Proxmox settings, sizing, and incompatible options. * **Documentation & Tests** * Documented the new settings and expanded coverage for rendering, validation, cloning, sizing, and template replacement. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
09d02fdc6c |
docs(apps): correct the shared resources and preset paragraphs (#4394)
## What this PR does Corrects two sentences of the boilerplate shared by the app READMEs, both raised in the review of #3936. - "When left empty, the preset defined in `resourcesPreset` is applied" describes an all-or-nothing rule, but `cozy-lib.resources.defaultingSanitize` merges per key: every resource `resources` leaves unset is taken from the preset. Twelve READMEs carried the sentence. - The legacy flat names were called aliases of their "1:1 instance-type equivalents", which reads as the `c1` series listed just before it. `nano` through `small` equal the `t1` sizes, and `medium` equals `c1.small` rather than `c1.medium`, the trap `docs/operations/resource-presets.md` already warns about. Ten READMEs carried the sentence. Only the hand-written section changes. `make generate` for all twelve packages leaves the diff at exactly those 22 lines. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff: README prose only, no values, schema, default or package change. The website regenerates the app reference pages from these READMEs on release, so nothing to open there. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified how resource settings work across several app READMEs: any unset resource field now inherits its value from the preset, rather than requiring the entire resource block to be empty. * Updated legacy preset name guidance to explain that deprecated flat names keep their original sizes, including `nano`–`small` matching `t1` sizes and `medium` matching `c1.small`. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
b16edec3f3 |
fix(tcp-balancer): rename backends and pin HAProxy so the chart starts on 3.3+ (#4366)
## What this PR does `haproxy:latest` now resolves to 3.3 and newer. Since 3.3 a backend named like a frontend is a fatal config error, so every tcp-balancer pod fails at config parse (#1966). We hit it on Cozystack 1.6.1 with `haproxy:3.4.4` behind the tag. - backends get a `_backend` suffix and every `default_backend` follows; the rename is the one from #2321 by @officialasishkumar - the image is `haproxy:{{ .Chart.AppVersion }}` with `appVersion: 3.4.4`, the current 3.4 LTS tag - the whitelist `acl`, `accept` and `reject` lines of each frontend render inside one guard - `tests/` holds helm-unittest suites, and the package `Makefile` gets the `test` target the sweep looks for - the `whitelistHTTP` description says it needs a non-empty `whitelist` Two inputs change behaviour on upgrade. On `main` both fail at config parse with `no such ACL : 'whitelist'` on every HAProxy version, because the `acl` line sits inside the `with` and `accept if whitelist` does not: - `whitelistHTTP: true` with an empty `whitelist` now fails the render with `whitelistHTTP is enabled but whitelist is empty`. Its HelmRelease upgrade fails and the release stays on its old revision, which is already crash-looping. - `kubernetesAndTalos` with an empty `whitelist` now starts, and ports 6443 and 50000 accept any source. With `external: true` the Service is a LoadBalancer and carries both ports. `whitelist` is the only switch those frontends have, so an empty list means no filtering. Every other input renders the same config apart from the backend names. The image and the `checksum/config` annotation change, so the Deployment rolls, and for releases already crash-looping on `latest` the rollout is the fix. Validation: renders of `main` and of this branch for the default, whitelist and empty-whitelist inputs were checked with `haproxy -c` on 2.9.7 and 3.4.4. `helm unittest .` passes 9/9, and removing the `reject` or `accept` line from any single frontend turns a test red. Fixes #1966 ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff: chart templates, `Chart.yaml`, the package `Makefile`, `tests/`, and a description-only change in `values.yaml` with its generated carriers. No field is added, removed or renamed and no default changes, so nothing in the map fires. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(tcp-balancer): rename HAProxy backends and pin the image to the chart appVersion so the balancer starts on HAProxy 3.3+. whitelistHTTP with an empty whitelist now fails the render; kubernetesAndTalos with an empty whitelist leaves ports 6443 and 50000 unfiltered. ``` |
||
|
|
8722462066 |
fix(kubernetes-nodes): refuse osImage on proxmox instead of honouring half of it
#4171 gave a worker pool an osImage, choosing where its system disk comes from: a CDI clone of a shared golden image, or an HTTP import from an Image Factory. A Proxmox worker's disk comes from neither. capmox clones a VM template that already carries Talos, and nothing in this chart can change what that template holds. Half of the field would still appear to work, which is why it is refused rather than ignored. osImage.factory.version reaches the machineconfig's installer image and changes the reconcile Job, so a pool could set it, watch the Job roll, and end up running one Talos while booting another: v1.12.6 moves the installer to .../installer/<schematic>:v1.12.6 and leaves the ProxmoxMachineTemplate untouched. osImage.builtin has no meaning here at all. Nothing is lost by refusing it: the installer target on this substrate is settable through talos.version and talos.schematicID, which do reach it. The branches suite gains the refusal case. The substrate parameter's description said changing the value "rolls the pool". The chart refuses the change on a live pool, and the description says so; it reaches the README, the values schema and the CRD. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
4bc6621a5c |
test(kubernetes,kubernetes-nodes): assert both sides of every substrate branch
Every `if substrate` in both charts is asserted on both sides, so an inverted condition cannot pass: - the reconcile Job reads SVC_IP from the LoadBalancer status on proxmox and from the ClusterIP on kubevirt; - the machineconfig carries the pool's nameservers on proxmox and none on kubevirt; - the proxmox gate on each kubevirt-only template in the parent chart, the two RBAC bindings included, so a proxmox cluster renders no binding to a Role, ClusterRole or ServiceAccount it does not render; - the sizing conversion at 1500m and 8G, where ceil and floor disagree (2 cores and 7630 MiB against 1 and 7629); - the old-template retention loop keeps a ProxmoxMachineTemplate a MachineSet still references; - the proxmox worker monitor carries no Pod threshold while the kubevirt one does; - a live cluster and a live pool refuse a substrate switch while a first install still renders; - both charts refuse a dnsServers entry that is not an address, the parent chart included, where the value reaches ProxmoxCluster.spec.dnsServers. The negations carry `any: true`: without it a negated containsDocument passes whether or not the document is in the render. hack/talos-reconcile-heredoc_test.bats gains the dnsServers cases: the render refuses a command substitution and names the entry; an entry with a colon in it is refused too, which pins the IPv6 branch of the validation and not only the IPv4 one; and a real address survives the Job's shell unchanged. The heredoc is extracted by shape — `cat <<EOF` at the start of a line or right after `=$(` / `="$(` — so the same case keeps working whichever way the Job writes it. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
517baf945d |
fix(kubernetes,kubernetes-nodes): validate dnsServers, refuse a substrate switch, drop the Pod threshold on proxmox
Three faults in the proxmox substrate, none of which the render refused. 1. proxmox.dnsServers reached the reconcile Job's shell unescaped. The Job applies the worker TalosConfigTemplate through an unquoted heredoc, so every byte of its body is subject to parameter expansion and command substitution at Job runtime. The INVARIANT comment above that block says each tenant-controlled value must be shell-escaped or pattern-validated; the new nameserver list was neither. `quote` there is YAML quoting. Rendered with `$(id -u)` and a backtick pair in dnsServers, extracted the way hack/talos-reconcile-heredoc_test.bats does and run under sh, the shell executed both and put their output in the worker's resolver list. The Job's pod holds the parent cluster's Talos token, cluster id, cluster secret and bootstrap token in its environment, and its Role can read the Talos CA and patch the parent KamajiControlPlane. Each entry is now validated as an IPv4 or IPv6 address at render time, in both charts. Validation rather than escaping because Talos takes addresses here: an address cannot carry a metacharacter, so the property holds by construction, and the ordinary mistake of writing a hostname is caught in the same move. IPv6 is matched on its character set rather than its full grammar — what matters is which bytes may reach the shell, and the apiserver rejects a malformed address anyway. 2. Nothing stopped a live cluster or pool from being switched between substrates. `substrate` had no immutability of any kind, and the `## @immutable` marker this adds is dashboard-enforced only, so both charts now read the live object. Switching a cluster removes the KubevirtCluster, the cloud-controller-manager, the kubevirt-csi controller and the in-tenant -csi HelmRelease with all their RBAC from the render. Flux then uninstalls the CSI driver behind every PVC the tenant has. CAPI does not prevent it — its Cluster webhook forbids removing spec.infrastructureRef but admits a change of kind. Switching a pool rolls every worker to the other hypervisor and drops the machine template the draining MachineSet still references, because the retention loop looks up only the kind the new substrate uses. A pool whose substrate differs from its parent's was not rejected either, though the README says the two must match. Both guards are inert offline, like assertNoForeignPool and assertParentVersion beside them: with no apiserver the lookup is empty and a first install renders. 3. The worker WorkloadMonitor reported a proxmox pool as broken. The controller lists Pods matching the monitor's selector and sets Operational=false while fewer than minReplicas are ready. On kubevirt those Pods are the virt-launcher Pods. A proxmox worker is a VM on another machine with no Pod here, so the count is always zero and any threshold pinned the pool — and through the lineage labels its whole application — to WorkloadsReady=False while every node was healthy. The monitor still renders for the dashboard and the lineage; what it no longer carries on proxmox is a threshold counted in Pods. Two smaller things. The wait-loop log line is written per substrate, so a default kubevirt render of kubernetes-nodes is byte-identical to main and no existing pool gets a new reconcile Job on upgrade. And the comment claiming a rename of $kubevirtmachinetemplate would roll every live worker VM is replaced: the hash is taken over the rendered template, not the variable name, so renaming it leaves the render byte-identical. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
33e1de52d2 |
test(kubernetes-nodes): pin the linked-clone default and three silent guards
Nothing failed when `full` was wrong: the pool came up, the workers joined, and each one quietly cost the whole template disk instead of the increment — 10.2 GiB against 8 KiB on a ZFS store. A default whose only symptom is disk usage is exactly the kind that drifts, so both it and its override are now asserted. The bridge and dnsServers guards were two of the guards on this path with no coverage. Both fire on absence, so the suite omits each and the tests supply the other one back; declaring either at suite level would make it impossible to take away, which is why the sizing guard already lives alone. The storage guard is the third: `storage` with a linked clone is refused and the message names both values, while `storage` with `full: true` reaches the ProxmoxMachineTemplate. Each assert goes red on the edit it exists for: flipping the default reds the clone suite, deleting a guard reds the guard suite. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
ceb7119a1b |
fix(api): regenerate deepcopy for the proxmox value structs
The proxmox blocks added to the Kubernetes and KubernetesNodes values carry string slices, and the generated deepcopy never learned about them: a copy shared allowedNodes, dnsServers and templateTags with its original instead of owning them. Cause was procedural rather than conceptual — the per-package `make generate` under packages/apps regenerates the chart schema and the Go types, but the deepcopy functions come from the repository-root target. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
c5996c1d0f |
fix(kubernetes,kubernetes-nodes): give Proxmox workers an endpoint they can reach
A worker on the proxmox substrate is a VM on the hypervisor network, not a Pod in this cluster. It was handed two addresses only an in-cluster consumer can route to: the apiserver as a ClusterIP, and the management cluster's CoreDNS ClusterIP as its resolver. The VM booted, the Machine reported Ready with a providerID, the disk was where it should be — and no CSR ever arrived. Nothing in that picture points at an address. The control-plane chart now publishes the tenant apiserver as a LoadBalancer on the proxmox path, putting the endpoint on the network the workers share, and the pool's reconcile Job reads the Service's LoadBalancer address instead of its ClusterIP for the host entry the workers dial. The nameservers follow the same reasoning: proxmox pools carry their own resolvers rather than pointing at a CoreDNS they cannot reach, and the chart refuses to render a pool that has not named any. KubeVirt keeps the ClusterIP on both counts. Its workers are in this cluster, where a ClusterIP is the right answer and a LoadBalancer address would be an unnecessary one. Verified end to end on a real tenant cluster: the node joins, its CSR is approved, and cilium, konnectivity and coredns come up in the tenant. One operational note this surfaced, not caused by it: TalosConfigTemplate.spec is immutable, so the reconcile Job cannot rewrite an endpoint that has already been published — the template has to be deleted for the Job to recreate it. That matters only when an existing cluster's apiserver address changes. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
d78af928fc |
feat(kubernetes-nodes): default Proxmox worker disks to linked clones
A full clone on a ZFS pool configured with `sparse 0` reserves the whole volsize the moment the VM is created: 10.2G for a worker whose Talos image occupies 337M. A linked clone writes only its own increment — measured at 8K for the same worker on the same host — because it shares the template's base snapshot until it diverges. The cost is that the clone holds that snapshot, so the template cannot be rotated while any linked clone still references it. That is bounded rather than permanent: consolidation detaches each worker 24 hours after creation, so template rotation is blocked for at most a day. Consolidation is not a chart concern, but two of its constraints are worth recording where the flag is set, because both were discovered by testing on a live host rather than from the docs. A linked clone can only be detached by moving its disks to a *different* storage — Proxmox refuses a same-storage move outright. And both disks must move: leaving efidisk0 behind keeps the template pinned just as effectively as the system disk would. Set full: true for a pool that must be independent of the template from the first second. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
135eb6bac4 |
fix(kubernetes): make the Proxmox cluster object reconcilable
Two corrections found by building a real tenant cluster on Proxmox rather than by rendering the chart. Each produced a symptom that pointed away from its cause. capmox skips control-plane endpoint validation only when spec.externalManagedControlPlane is set. Kamaji owns the endpoint here, so the field was semantically true all along and merely absent — and on an absent controlPlaneEndpoint the webhook does not return a validation error, it panics (internal/webhook/proxmoxcluster_webhook.go:105). The install failed with a nil-pointer stack trace out of an admission webhook, which reads like a broken provider rather than a missing field. The cluster.x-k8s.io/managed-by annotation now renders only on the KubeVirt path. It tells Cluster API the infrastructure object is owned by something outside CAPI, so the Cluster controller never sets an ownerRef on it — and capmox's reconciler exits at "Waiting for Cluster Controller to set OwnerRef on ProxmoxCluster" without ever creating the InClusterIPPool its workers draw addresses from. Harmless on KubevirtCluster, which nothing reconciles; load-bearing on ProxmoxCluster, which capmox must. With both in place the object reaches Ready=True, the pool is created, and the Cluster reaches Provisioned. The proxmox suite's ProxmoxCluster case asserts both: the field is set and the annotation is absent. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
db839d1198 |
feat(kubernetes): add a proxmox substrate for managed clusters
Companion to the same switch in kubernetes-nodes. The control plane is Kamaji on either substrate, so this changes exactly one object — the one Cluster.infrastructureRef points at — plus the two tenant addons that exist only because the workers are KubeVirt VMs living in this cluster. A KubevirtCluster needs no spec: its workers are in the same cluster and reach the apiserver over the pod network. A ProxmoxCluster addresses an external hypervisor, so it carries what capmox cannot infer — which Proxmox nodes may host VMs, which addresses they may take, and who resolves names for them. controlPlaneEndpoint stays absent on both: the cluster.x-k8s.io/managed-by annotation hands that to Kamaji. The address pool is required rather than defaulted. capmox has no DHCP mode; it allocates from an InClusterIPPool built out of ProxmoxCluster.spec.ipv4Config, so an unset pool is not a smaller configuration but a ProxmoxCluster that never reaches Ready and worker Machines that wait on it forever. Refusing at render time turns that into a message instead of a silence. The KubeVirt cloud-controller-manager, its cloud-config and the kubevirt-csi driver are gated off on the proxmox path. They reach the workers' VirtualMachines and DataVolumes in this cluster; on Proxmox those objects do not exist, and leaving them rendered would put controllers in the tenant reconciling nothing. Their Proxmox equivalents are separate packages and land separately. The default path is unchanged: rendering it before and after this commit differs only in the per-render random bootstrap tokens, and all 19 existing suites pass untouched. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
85f576756b |
feat(kubernetes-nodes): add a proxmox substrate for worker pools
Worker pools are already Talos machines created by Cluster API; only the
substrate underneath them is fixed to KubeVirt. This adds `substrate:
kubevirt|proxmox` and swaps exactly two things: the machine template object
and every reference to it. Autoscaler capacity annotations, kubelet
reservations, MachineDeployment, MachineHealthCheck, WorkloadMonitor and the
talos-reconcile Job are substrate-agnostic and rendered once, which is why
this is a branch in the existing chart rather than a second copy of it.
The default path is unchanged down to the bytes: the render snapshot and all
15 existing suites pass untouched, and the hash-named template keeps its
variable name so no live worker VM rolls on upgrade.
Five things the Proxmox path must do differently, each learned from a
failure that pointed somewhere else:
- Skip capmox's guest-agent and cloud-init-status probes. Talos ships
neither, and left on they leave the VM booted and the Machine stuck on
"error waiting for agent: the operation has timed out". This is the same
class of problem `virtualMachineBootstrapCheck.checkStrategy: none` solves
for CAPK's SSH probe; capmox's CRD names TalOS in the skipCloudInitStatus
description.
- Clone a template carrying the nocloud image, not openstack. capmox injects
bootstrap data as a nocloud ISO, and an openstack image ignores it exactly
as silently as a nocloud image ignores the configdrive the KubeVirt path
depends on.
- Pass templateTags through verbatim. capmox compares the template's tag set
with slices.Equal after sorting, so a selector listing a subset matches
nothing and reports "found 0 VM templates with tags ..." while the tags are
plainly present.
- Require resources.cpu and resources.memory. Proxmox has no
VirtualMachineClusterInstancetype, so the lookup that normally supplies
effective CPU and memory has nothing to read — and those two numbers feed
the kubelet reservations and the autoscaler capacity annotations. A pool
that cannot state them would be scheduled against invented capacity, so it
is refused at render time. KubeVirt-only knobs (gpus, logSerialConsole,
podCpuLimit/Request) are refused too rather than accepted and ignored.
- Refuse proxmox.storage unless full is true. A target storage is a full-clone
parameter: a linked clone always lives on the template's storage, and both
Proxmox ("parameter 'storage' not allowed for linked clones") and capmox's
own CRD ("Must set full=true when specifying storage") reject the pair.
Left to them, a pool with storage set and a linked clone fails at apply time
with an error about a ProxmoxMachineTemplate the operator never wrote;
failing the render names the two values instead.
The Talos image is not a chart input on this path: capmox clones a template
that already exists on the hypervisor, so building and rotating that template
is an operator responsibility with no equivalent on the KubeVirt side.
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
|
||
|
|
9934014908 |
fix(capi): ship the core provider manifests the repository actually holds
#2946 added a startupProbe to capi-controller-manager so it survives the certificate provisioning delay. It edited capi-providers-core/files/core-components.yaml, which is the copy a reviewer reads. The chart mounts files/components.gz, and that blob was never regenerated, so the probe never reached a cluster. The blob is regenerated from the YAML it is supposed to carry: gzip -9 -n -c files/core-components.yaml > files/components.gz The decompressed blob now matches the YAML byte for byte, and the only change it carries is the six lines of #2946. This reaches new installs only. capi-operator re-applies a provider when the hash of its spec and its config Secret changes; the ConfigMap holding the components is in neither, so a cluster that already runs capi-providers-core keeps its manager without the probe until the provider's version moves. Whether existing clusters should get the probe, and how, is left open here on purpose: moving spec.version and the ConfigMap name together would force the re-apply, but that is a version decision for the package, not a side effect of regenerating a blob. .helmignore is corrected in the same change: `files/.*-components.yaml` is a glob, so it matched only names beginning with a dot, and `helm package` shipped the readable YAML alongside the blob that duplicates it. hack/capi-core-components_test.bats holds the two files to the same bytes from now on, so the next edit of one without the other fails in CI. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
7843ec98b5 |
test(capi): catch a vendored provider whose blob and YAML have drifted
Each provider package ships the manifests twice: the YAML a reviewer reads and the gzip blob the chart mounts. `make components` writes both from one download and nothing keeps them together afterwards, so editing the YAML alone produces a diff that reviews cleanly while the cluster runs the version nobody read. Both new provider packages are covered. A second pair of tests holds the version together: capi-operator hashes the provider spec and the config Secret into applied-spec-hash, and the ConfigMap is in neither, so a bump that regenerates the vendored files while the ConfigMap name and spec.version stay behind installs the old provider and reports the old version, with nothing reporting an error. The Makefile variable, the ConfigMap name and spec.version have to be one value. A third pair checks the vendored files against the SHA-256 sums the Makefile pins, so the pin holds even when nobody runs `make components`. Written for dash rather than bash, since the repo's runner executes these under sh — process substitution is not available there. Each pair goes red on the edit it exists for: a YAML edited without regenerating the blob, a Makefile version bumped on its own, a ConfigMap renamed without spec.version, and a vendored file that no longer matches its pinned sum. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
18b83a72b3 |
feat(platform): make the Proxmox substrate installable from a bundle
Until now the two Cluster API packages behind the Proxmox substrate existed but no bundle emitted them, so a Cozystack install shipped neither and the substrate was reachable only by applying Package CRs by hand. They join the iaas bundle as opt-in packages, where the KubeVirt infrastructure provider is unconditional. The asymmetry is the point: capmox drives a hypervisor outside this cluster and needs a Secret holding a Proxmox API token, which capi-operator reads before it installs anything — with no Secret the provider is never deployed at all. Enabling it by default would ask every Cozystack install that has no Proxmox to supply hypervisor credentials. Enabling capmox brings the in-cluster IPAM provider with it — capmox allocates worker addresses from an InClusterIPPool and has no DHCP mode, so on its own it never gets a ProxmoxCluster to Ready. IPAM stays independently enableable, since assigning addresses is a generic Cluster API concern rather than a Proxmox one. Naming capmox in enabledPackages while disabling IPAM stops the render. The cluster would catch that combination on its own: the capmox PackageSource depends on the IPAM Package, so with that Package absent capmox goes Ready=False with DependenciesNotReady and no HelmRelease is ever created. What it would not do is say why. Failing the render names the two conflicting list entries at the moment someone writes them, instead of leaving a condition to decode later. The negated containsDocument asserts in the wiring suite carry `any: true`. Without it a negated containsDocument passes whether the document is in the render or not, so the two properties the suite exists for — off by default, and disabledPackages winning — could not fail. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
67db200530 |
feat(cluster-api): add Proxmox infrastructure provider package
Packages ionos-cloud/cluster-api-provider-proxmox (capmox) the same way capi-providers-infraprovider packages CAPK: vendored components gzipped into a ConfigMap, an InfrastructureProvider that fetches them by label. Two details differ from the KubeVirt package on purpose. The fetchConfig label is `infraprovider-provider: proxmox`, not the shared `infraprovider-components: cozy`. A shared label would put both providers in one match. capi-operator would still install the right one — it keys every matched ConfigMap by its name as the version and takes the one equal to spec.version — but it builds that repository in a single pass and returns an error for the whole list as soon as one match is malformed: a name that does not parse as semver, or missing metadata. Sharing the label would let a broken ConfigMap belonging to one provider stop the other from being installed. The bootstrap packages already separate on a `bootstrap-provider` key; this follows that. components.gz is produced with `gzip -n`. Without it the header carries a timestamp, so regenerating from identical input yields a different blob and the committed artifact appears to change when nothing has. `make components` records the version pin, the source URLs and the flag, so refreshing the vendored manifests is reproducible rather than remembered. It also checks both downloads against the SHA-256 sums recorded in the Makefile before packaging them: a release tag is mutable, so the pin is on the bytes, and bumping CAPMOX_VERSION means recording the new sums next to it. Pinned to the v0.7 series: from v0.8 capmox declares the v1beta2 Cluster API contract, while capi-providers-core here is v1.10.1, which serves v1beta1. The package source depends on the in-cluster IPAM provider and deliberately not on cozystack.kubevirt: the KubeVirt provider needs it because its VMs run inside this cluster, whereas a Proxmox hypervisor is external and reached over its API with the credentials named by `credentialsSecretName`. That Secret is a precondition rather than an optional extra, and the chart does not create it — a hypervisor token belongs to the platform's secret management, not to a Helm release. capi-operator reads the Secret before it installs the provider, so while it is missing nothing is deployed at all. values.yaml carries what an operator needs at that point: that PROXMOX_URL is the bare https://<host>:8006 — capmox appends /api2/json itself, and a URL that already carries it sends every call to /api2/json/api2/json/... and gets 501 back — the full user@realm!tokenid form of PROXMOX_TOKEN, and the fact that a rotated token reaches the manager only after the next provider reconcile and a restart of its pod. Both edges are asserted in tests/sources_capi_provider_proxmox_test.yaml; losing the IPAM edge reproduces a failure mode whose symptom points nowhere near its cause. Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua> Assisted-by: LLM |
||
|
|
86834d30a9 |
feat(cluster-api): add in-cluster IPAM provider package
Cluster API ships address management as a separate provider, and Cozystack
has never needed one: KubeVirt workers lease their addresses from the
cluster network's DHCP. An infrastructure provider that allocates addresses
instead of leasing them cannot work without it.
capmox is such a provider. It builds an InClusterIPPool out of
ProxmoxCluster.spec.ipv4Config and hands every VM a static address from it.
With no IPAM provider installed, the pool kind does not resolve:
Reconciler error: no matches for kind "InClusterIPPool"
in version "ipam.cluster.x-k8s.io/v1alpha2"
ProxmoxCluster then never reaches Ready and every ProxmoxMachine sits in
WaitingForClusterInfrastructure indefinitely — indistinguishable, from the
outside, from a provider that is simply broken.
Pinned to the v1.0 series rather than v1.1.0: both serve InClusterIPPool
v1alpha2, but v1.0 declares the v1beta1 contract that matches
capi-providers-core v1.10.1.
`make components` checks both downloads against the SHA-256 sums recorded
in the Makefile before packaging them. A release tag is mutable, so the pin
is on the bytes rather than on the tag; bumping CAIP_VERSION means recording
the new sums next to it.
The package carries no edge to any infrastructure provider. The dependency
belongs on the consumer's side, which keeps this reusable for any future
provider with the same requirement.
Signed-off-by: Marian Koreniuk <moriarti@cp.if.ua>
Assisted-by: LLM
|
||
|
|
cffe509dcf |
feat(monitoring): per-tenant OTLP collector (tracing phase 2) (#3763)
## What this PR does Phase 2 of distributed tracing (design cozystack/community#38, epic #3761), built on the phase-1 backend (#3762, merged): a per-tenant **OpenTelemetry Collector** — the app-facing OTLP ingest gateway. It lives in the tenant namespace (like a per-tenant vmagent), so app→collector traffic is intra-namespace and never crosses the tenant boundary. No OTel operator exists in the platform, so this is a plain Deployment + ConfigMap + Service (the fluent-bit pattern). ## Changes - **`templates/vtraces/collector.yaml`** — OTLP receivers (gRPC `:4317`, HTTP `:4318`) → `memory_limiter` + head sampling (`probabilistic_sampler`, default 10%) + optional `attributes/redact` (PII/secret control point) + `batch` → `otlphttp` exporter to the `tracingStorages` backend (`vtinsert:10481` cluster / `vtsingle:10428` single, endpoints per upstream docs). PSS-restricted securityContext, digest-pinned image via `cozy-lib.image`, config-checksum roll. - **app-facing Service `otel-traces`** (4317/4318). - **`tracingCollector` values** — `enabled` / `replicas` / `samplingPercentage` (schema-bounded 0-100) / `redactAttributes` / `resources`. A **fail-guard** rejects an enabled collector fronting more than one `tracingStorages` backend (it exports to a single backend by design). - **`otel-traces` WorkloadMonitor** and dashboard resource-map entry, rendered under the collector's own gate. The same resource-map Role also gets the pre-existing `alermanager` typo corrected to `alertmanager`, so the alertmanager WorkloadMonitor is readable from the dashboard. - helm-unittest coverage for the collector and its WorkloadMonitor; regenerated `values.schema.json` / `README.md` / `monitoring-rd` ApplicationDefinition. ## Verification - `helm unittest` — `system/monitoring` 110, `extra/monitoring` 15, all pass. - `helm template` renders cleanly; `make generate` idempotent. - Live check on a test cluster (single-mode backend): the collector starts with `redactAttributes` containing a numeric, a colon-bearing and an empty key; an OTLP/HTTP span sent to `otel-traces:4318` is accepted and is queryable from the backend through the Jaeger API, with the redacted attributes removed and the others kept. - Local `/branch-review` — **LGTM** (one blocker + four recommendations fixed, each with a test: multi-backend guard, memory_limiter cgroup limit, configurable replicas, sampling bounds, trust-boundary wording). ## Scope / deferred (epic #3761) `AccountID`/`ProjectID` header injection for the shared-central topology (phase 4) and engine integrations (phase 3) are follow-up PRs. ### Screenshots N/A, no UI change. ### Downstream repositories The diff touches only `packages/extra/monitoring`, `packages/system/monitoring` and `packages/system/monitoring-rd`. The website's monitoring reference page is regenerated from the package README by the release bot, and the Terraform provider models only the tenant's `monitoring` flag, not these values, so no follow-up is needed. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(monitoring): deploy a per-tenant OpenTelemetry (OTLP) collector for traces once a VictoriaTraces backend is configured. Applications send OTLP to the in-namespace `otel-traces` Service (gRPC `:4317`, HTTP `:4318`); the collector applies head sampling (`samplingPercentage`, default 10%) and optional attribute redaction (`redactAttributes`) before exporting to the backend. Off by default (no backend configured). ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable trace storage in cluster or single-instance modes, with retention settings and Grafana datasources. * Added an enabled-by-default OpenTelemetry collector with configurable replicas, sampling, attribute redaction, and resource limits. * Added workload monitoring and automatic resource recommendations for configured trace components. * Added readiness health checks for both tracing backend modes. * **Bug Fixes** * Corrected dashboard permissions for alert monitoring and trace collector resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
368d00b5d4 |
test(e2e): remove unused monitoring suites and audit BATS runners (#4019)
## What this PR does I removed the two monitoring OIDC BATS suites that had no runner after the Chainsaw migration, and brought the runner audit and wrapper OIDC test from [#3778](https://github.com/cozystack/cozystack/pull/3778). Also added checks for audience mapper and `OIDC_MODE=System`. The named `KeycloakRealmGroup` test documents coverage the document-count and kind assertions already provided. Runner audit reports unused BATS suites, missing files in runner paths, and unresolved Make variables (including `$(E2E_SUITE)`). Live Monitoring API admission, Grafana operator reconciliation and mode transitions are not tested here. ### Validation - `make -j4 unit-tests SHELL=/usr/bin/dash` passed. - Monitoring Helm tests: 53 system-chart tests and 10 wrapper-chart tests passed. - Runner audit: 12 cases passed under dash and Bats 1.14.0. The full shell syntax and BATS sweep passed under dash too. - Three chart mutations and seven runner mutations failed as expected. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note NONE ``` |
||
|
|
03e2da5805 |
feat(postgres,mariadb)!: drop plaintext user passwords (#4078)
## What this PR does Removes the plaintext `users[].password` field from the `apps/postgres` and `apps/mariadb` charts (and the mariadb `root` account), so managed-database passwords can no longer be pinned in tenant values or committed to git. Passwords are always chart-generated into the `<release>-credentials` Secret and preserved across reconciles via `lookup`, matching the model `redis`/`valkey` already use. This closes the "User passwords" item of the portal upstream epic (aenix-org/cozyportal#1060). The generated password never lands in the init-script Secret or the SQL it runs: the postgres init-job reads each user's password from the mounted `<release>-credentials` Secret at run time and applies it through a psql variable (`ALTER ROLE … WITH PASSWORD :'pw'`), so it does not linger in the init-script manifest a tenant can read. It does live in cleartext `stringData` in the `<release>-credentials` Secret itself, which is a chart manifest and therefore kept in Helm release history for `MaxHistory` revisions like any chart-generated secret — that residue is exactly why rotation cannot be built on the chart (see below). A `password` or a `passwordRotation` key left over in a tenant's values is accepted-but-inert and draws an admission warning, so an operator editing it is told it has no effect rather than getting a silent 200. Backup/restore into a copy keeps the target's chart-managed credentials working: the CNPG driver clears `bootstrap.enabled` once recovery is healthy so the init-job re-applies the generated passwords onto the recovered roles, and the MariaDB driver excludes `mysql.global_priv` from the dump (`ignoreGlobalPriv`) for backups captured from this release onward, so restoring one of those into a copy does not clobber the target's chart-managed users or `root`. Excluding the grant table has a flip side, and it applies to every MariaDB restore, not only a to-copy one: only accounts declared through `User` CRs (the chart's `users` map) are captured, so an account created out of band directly in MySQL is in no backup and a restore brings back its data and `mysql.db` grants but not the account itself. This is also a backup-side fix only: a restore from a backup taken **before** this change still replays the source's `mysql.global_priv` and overwrites the target's users and `root`, and `root` then needs an out-of-band reset (the operator applies `rootPasswordSecretKeyRef` only at datadir bootstrap, so there is no in-band path). See `docs/operations/backup-classes.md` for the convergence window, the out-of-band-account limitation, and this old-backup caveat. Password **rotation** is deliberately **not** in this PR. A chart-rendered credentials Secret keeps its cleartext in Helm release history for `MaxHistory` revisions, so a rotation built on it cannot revoke a leaked password; rotation is being reworked as a controller that owns the Secret — design proposal in cozystack/community#72. Migration is seamless: existing releases keep their current passwords on upgrade (the value already lives in `<release>-credentials`, which `lookup` re-reads and preserves); a `password` still present in a tenant's values is ignored, so no existing HelmRelease fails validation. Verification: `helm unittest` passes for both charts (postgres 87, mariadb 126, including render-time username guards that reject any name k8s would later reject as an invalid Secret key or User resource name); `go build ./...`, `go vet`, the backup-controller tests and the apps-apiserver tests are green; `make generate` was run with the CI-pinned `cozyvalues-gen` v1.6.0 so `values.schema.json`, the READMEs, the API types, and the `cozyrds` ApplicationDefinitions are all in sync (the branch was rebased onto `main`, which reconciled cleanly with the read-replica autoscaling feature that landed there). ### Screenshots N/A — no UI change. ### Downstream repositories - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: affected — `content/en/docs/next/cozystack-api/go-types.md` is hand-written and still shows `Password:` in the Go examples, and the managed-app reference pages regenerate from these chart READMEs, so both document `users[name].password` until refreshed (not yet filed). - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: affected — the provider hand-maintains the postgres/mariadb schemas and per-field defaults; this PR removes `users[].password`. A follow-up issue/PR is needed there (not yet filed). `terraform-provider-cozystack` and `website` are reached by this diff; both boxes are left unticked because the follow-ups are not yet filed. Same-repo future work this change suggests: the controller-owned password rotation (design in cozystack/community#72), and extending the "passwords only in the Secret" model to the other engines that still accept plaintext (`clickhouse`, `mongodb`, `nats`, `opensearch`, `rabbitmq`, `vpn`). ### Release note ```release-note feat(postgres,mariadb)!: passwords for managed PostgreSQL and MariaDB users can no longer be set via `users[].password` — they are always auto-generated and stored only in the `<release>-credentials` Secret, never rendered into the init-script or the SQL it runs. Existing releases keep their current passwords on upgrade; a `password` (or a `passwordRotation`) left in values is ignored with an admission warning. Password rotation is deferred to a follow-up controller design (cozystack/community#72). MariaDB backups now exclude the `mysql.global_priv` grant table (`ignoreGlobalPriv`), so a restore recreates only accounts declared through the chart's `users` map — an account created out of band directly in MySQL is not captured and must be recreated after a restore. ``` |
||
|
|
0cb37dba9e |
docs(release): add changelog for v1.6.3 (#4077)
This PR adds the changelog for release `v1.6.3`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.6.3.md`.
|
||
|
|
9d6fea7017 |
fix(monitoring): correct the alertmanager name in the dashboard Role
The Role listed the WorkloadMonitor as 'alermanager', so the real 'alertmanager' monitor was never granted: a tenant dashboard is forbidden from reading its status and the alertmanager health tile breaks. The test pins the correct name and the absence of the typo. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
60b1b05434 |
feat(monitoring): add per-tenant OTLP collector for distributed tracing
Applications need one in-namespace OTLP endpoint that also enforces
sampling and redaction before spans reach the VictoriaTraces backend.
No OpenTelemetry operator ships with the platform, so the collector is
a plain Deployment, ConfigMap and Service ('otel-traces', gRPC 4317,
HTTP 4318) in the tenant namespace; app-to-collector traffic therefore
never leaves the tenant, and isolation comes from the existing tenant
network policies.
The collector renders only when it is enabled and a tracingStorages
backend is configured. Tracing is off by default, and an empty backend
list is a clean off-state rather than a render failure; an app exporter
that finds no Service simply retries. It writes to the first backend
only and rejects a list with several, instead of silently dropping
spans for the others.
redactAttributes is the PII and secret control point, so its keys are
rendered quoted and empty keys are dropped: an unquoted key with YAML
structural characters, or an empty one, makes the collector fail at
startup, and a key YAML re-types (numbers, booleans, null) silently
escapes redaction. replicas has a lower bound of 1 so a bad value fails
schema validation instead of the whole release at apply time; a zeroed
collector is expressed with enabled: false.
The collector gets a WorkloadMonitor and a dashboard-resources Role
entry under the same render gate, so it is part of tenant workload
accounting like every other workload in the chart.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
|
||
|
|
2d2186d1f0 |
feat(monitoring): VictoriaTraces backend + Grafana datasource (tracing phase 1) (#3762)
## What this PR does Phase 1 of distributed tracing (design: cozystack/community#38; epic: #3761): a **VictoriaTraces backend and Grafana traces datasource** in the monitoring stack, mirroring the logs (`VLCluster`) implementation. No new operator: `VTCluster`/`VTSingle` already ship in victoria-metrics-operator. - **`tracingStorages`** values in `extra/monitoring` (annotated, schema-generating) and `system/monitoring` (defaults), per-entry `mode: cluster|single`. **Empty by default, so tracing is off out of the box**: nothing produces spans until the collector ships, so no backend is provisioned until an operator adds an entry. - **`templates/vtraces/vtraces.yaml`**: `VTCluster` (default) or `VTSingle` (edge/dev/small); `retentionPeriod` plus optional `retentionMaxDiskSpaceUsageBytes`; a release-scoped **PVC cleanup label**, reused by the existing post-delete hook, so no hook change is needed. - **Render-time validation** (`_helpers.tpl`): the installed VictoriaTraces CRDs are the trimmed ones, so a bad value would otherwise reach the operator and fail after admission. Each `tracingStorages` entry is checked before anything renders: `name` must be an RFC 1123 label of at most 42 characters (the `vtstorage-<name>-<hash>` pod revision label is capped at 63) and unique across the list, `mode` must be `cluster` or `single`, `storage` must be a positive quantity (a zero size makes the operator mount an EmptyDir in single mode, and a negative one fails when the PVC is created), and `retentionDiskUsageBytes` must be a VictoriaMetrics `BytesString` (the operator rejects the Kubernetes-quantity `8Gi` grammar that `storage` uses). The `storage` field is also typed as a quantity in the schema, and the rendered name is quoted so a numeric name stays a string. - **`templates/vtraces/grafana-datasource.yaml`**: a Jaeger-typed traces datasource per entry (built-in Grafana core datasource, no plugin). - **VPA**: a `VerticalPodAutoscaler` for each trace component (`vtinsert`/`vtselect` Deployments, `vtstorage` StatefulSet, `vtsingle` Deployment), matching the metrics/logs sizing, with `updateMode: Initial`. - **Readiness gate**: `VTCluster`/`VTSingle` `healthCheckExprs` on `status.updateStatus`, alongside the existing `VLCluster` gate. - **VictoriaLogs datasource fix** (`templates/vlogs/grafana-datasource.yaml`): the range over `logsStorages` emitted no `---` between documents, so two or more entries collapsed into one document and only the last datasource survived. Each entry is now its own document. - **Tests and schema**: helm-unittest suites for off-by-default, cluster vs single, datasource type and both query URLs, retention, PVC label, VPA targets, every validation guard with its boundaries, and per-entry vlogs datasource documents, plus regenerated `values.schema.json`, `README.md` and the `monitoring-rd` ApplicationDefinition. #### Upgrade impact With default values the only rendered object that changes is the `monitoring-system` HelmRelease: it gains the `VTCluster`/`VTSingle` `healthCheckExprs` and `tracingStorages: []`. Flux evaluates those expressions only against objects in the release inventory, so they are inert while tracing is off. No VictoriaTraces object is created on upgrade. Installs with two or more `logsStorages` entries additionally gain the `GrafanaDatasource` objects that were previously dropped (see the VictoriaLogs fix above); nothing else under VictoriaLogs or VictoriaMetrics changes. #### Verification - `helm unittest`: `system/monitoring` (89 tests) and `extra/monitoring` (11 tests) pass. Each validation guard and each branch of the zero-storage pattern was mutation-checked: reverting it turns its own test red. - `helm template` renders `VTCluster`/`VTSingle`/datasource/VPA cleanly. `tracingStorages: []` (the default) renders nothing. An invalid, duplicate or over-long `name`, a zero or negative `storage`, or a bad `retentionDiskUsageBytes` (`8Gi`) is refused at render with an actionable message; a numeric `name` renders as a string. `make generate` is idempotent. - **Datasource URLs verified against upstream docs** ([VictoriaTraces and Grafana](https://docs.victoriametrics.com/victoriatraces/querying/grafana/)): cluster `vtselect:10471/select/jaeger`, single `vtsingle:10428/select/jaeger`. `jaeger` is the stable, built-in Grafana datasource; the Tempo Query API (`/select/tempo`, `type: tempo`, VictoriaTraces v0.9.4+) has a richer TraceQL UX but is upstream-experimental, so Jaeger is the default. Both URLs are pinned by unit tests. - The `retentionDiskUsageBytes` pattern, the zero-storage check and the name limit were checked against victoria-metrics-operator v0.68.4 (`BytesString.UnmarshalJSON`, `isStorageEmpty`, the `vtstorage-<name>` StatefulSet name) and the Kubernetes `ControllerRevisionName` naming. #### Scope / deferred (tracked in #3761) Per-tenant OTLP Collector, engine integrations, shared-central topology, and docs are follow-up PRs. This PR intentionally does **not** enable the OTLP/gRPC listener: apps can use `vtinsert`'s native OTLP/HTTP, and the gRPC listener lands with the collector. Builds on the now-merged design proposal cozystack/community#38. ### Screenshots Not a UI change. ### Downstream repositories Walked against the diff, no box ticked yet: - cozystack/website: `monitoring` is in the website `Makefile` `SERVICES` list, so its reference page is regenerated from `packages/extra/monitoring/README.md` automatically on the next stable tag. A user guide for tracing belongs to the docs follow-up in #3761; no website issue or PR exists for it yet. - cozystack/terraform-provider-cozystack: the provider has no `Monitoring` resource (only the tenant's `monitoring` flag), so the new schema field does not reach it. - The remaining repositories in the trigger map are not reached by this diff: no platform values, variants, installer values, `hack/` tooling, node prerequisites, ApplicationDefinition semantics or telemetry names change. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(monitoring): add a VictoriaTraces backend and a Grafana traces datasource to the monitoring stack (distributed tracing, phase 1). Tracing is off by default; enable it per tenant by adding a `tracingStorages` entry (`cluster` or `single` mode). Also fix the VictoriaLogs Grafana datasources: with two or more `logsStorages` entries, every entry except the last was silently dropped, so existing installs with several log stores gain the missing datasources on upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable trace storage with cluster or single-node deployment modes. * Supports retention periods, persistent storage sizing, storage classes, and optional disk-usage limits. * Automatically configures Grafana tracing data sources for each enabled trace backend. * Monitoring readiness reflects the health of configured trace storage services. * Invalid trace storage settings are rejected during configuration. * **Bug Fixes** * Every VictoriaLogs storage now gets its own Grafana datasource; with several log storages, all but the last were previously dropped. * **Tests** * Added coverage for storage rendering, retention settings, validation guards, Grafana endpoints, health checks, and cleanup behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
47df27de57 |
fix(api): let one ApplicationDefinition own each application kind (#4424)
## What this PR does Nothing enforced that `spec.application.kind` is unique across ApplicationDefinitions, and every consumer that maps a kind back to a definition resolved a duplicate its own way. The chartRef reconciler selected HelmReleases by the kind label alone and let every definition rewrite every release of its kind; the lineage webhook and the CA projection reconciler took whichever definition came first in an unordered list; cozystack-api registered one resource per definition. A second definition declaring a kind already in use could therefore repoint that kind's tenant releases at its own artifact, whatever the definition was named. Fixes #4361. This adds one ownership rule, in `internal/shared/appdefowner`: the definition created first owns the kind, with ties broken by name. Every Go consumer goes through it: - the chartRef reconciler rewrites releases only for the owner of their kind, emits a `KindClaimed` Warning event on a definition that is not the owner, and re-reconciles the next owner when the current one is deleted or changes kind; - cozystack-api registers the owner only and logs the definitions it leaves out; its restart hash now moves when ownership moves, while a cluster without duplicate kinds hashes exactly as before, so the upgrade itself does not restart the API; - the lineage webhook and the CA projection reconciler resolve the owner, and the CA projection watch now also reacts to a kind change; - the lineage webhook gains a validating endpoint that refuses a new ApplicationDefinition whose kind another definition already declares. An update that keeps its kind is allowed, so a duplicate that predates the check keeps upgrading, and so is a second definition from the same Helm release, because a release that renames its definition object creates the new one before it deletes the old one. It is registered with `failurePolicy: Ignore`, and a failure to read the existing definitions lets the request through with a warning instead of an error answer, which Ignore would not cover: ApplicationDefinitions ship with the platform, so the check must never block an install or upgrade, and the ownership rule holds whatever it lets through. Two decisions differ from the wording of the issue, deliberately: - a duplicate kind is not a fail-stop at cozystack-api start-up. One malformed or duplicate definition taking the API down for every tenant is the problem #4359's design comment calls out, so the API serves the owner and logs the rest instead; - "created first owns" rather than "the platform's definition owns". It needs no notion of provenance, and a definition that arrives later can never take a kind over. The consequence worth knowing: deleting the owner hands its releases to the next definition declaring that kind. Known gaps left for follow-ups, both pre-existing: - the dashboard console (`packages/system/dashboard/images/console`) still picks the first definition matching a kind or plural in the list it receives. It switches no release, but with a duplicate present it can show the duplicate's form; - `plural` collisions are not checked: a definition with a new kind but an existing plural still takes the plural's endpoint at registration. Existing releases are not reachable through it, since List and Get filter by the kind label. Tests cover the rule itself, each consumer, the webhook (create, update keeping and changing kind) and the chart; every guard was mutation-checked term by term. ### Screenshots Not a UI change. ### Downstream repositories The change alters `ApplicationDefinition` semantics (kind uniqueness and ownership), which the trigger map routes to `cozystack/website` (`content/en/docs/next/cozystack-api/application-definitions.md`). The follow-up there is not opened yet; it needs the maintainers to settle the behaviour first. No other repository in the map is reached: no CRD, kind, plural, prefix or values change. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(api): an application kind is now owned by one ApplicationDefinition, the one created first. A second definition declaring a kind already in use is refused at admission, and one that predates the check no longer manages that kind's releases or its API registration. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Application kinds now have a consistent owner: the earliest-created definition takes precedence, with name used to break timestamp ties. * New definitions that claim a kind already owned by another definition are rejected. Existing duplicates can keep their current kind, and definitions from the same Helm release can be renamed. * **Bug Fixes** * Runtime configuration and Helm releases now use the owning definition, preventing duplicates from replacing its configuration. * Ownership changes trigger affected definitions to be reconciled. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
0687124b00 |
fix(seaweedfs): pass the Iceberg catalog port explicitly so null means off
The chart wraps -port.iceberg in an if, and icebergPort defaults to null, so the flag is dropped and weed falls back to its own 8181 default. The chart therefore cannot express "no Iceberg catalog": the value that reads as off is the one that leaves an unmanaged plaintext catalog listening on the pod IP, with the same writable routes and the same admin-when-no-identities fallback as the Lance one. Render the flag the way 4.45 already renders -port.lance, so null reaches weed as 0 and an explicit port still wins. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
ea04e671a7 |
fix(seaweedfs): keep the Lance catalog listener off by default
Chart 4.45 ships upstream's Lance Namespace catalog enabled (lancePort: 9101) and publishes it on the s3 Service in every tenant namespace. weed serves that catalog over plain HTTP with no way to supply a certificate, while every other port on this gateway is TLS by way of the s3-tls-main-port patch, and its namespace and table routes are writable. Nothing in Cozystack consumes the catalog, so pin the port off and keep the s3 port set the bump inherited. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
|
46b4b35270 |
chore(seaweedfs): bump chart to 4.45
Refreshes the vendored chart from 4.31 through `make update`. All carried patches still apply; the volume-resize-hook one is rebased onto the pod template labels upstream now renders, so it appends the cozystack apiserver label instead of creating the block. Rendered names, StatefulSet selectors and volumeClaimTemplates are identical to 4.31, so running workloads keep their volumes across the upgrade. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |