mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-22 04:33:36 +09:00
5956b459fb9e50122636bf89abdd1e443aca382b
7281
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5956b459fb |
fix(backups): give each Postgres instance its own default WAL prefix
The default CNPG strategy archived a Postgres app under <namespace>-<name>, inside a destinationPath that is also keyed by namespace and name, and the driver fell back to the Cluster name when a strategy set no serverName. Deleting the app does not empty the bucket, so an app recreated under the same name archived its WALs and base backups into its predecessor's stream: both start on timeline 1, and a later restore of the old archive could pick the new cluster's base backup or WAL segments. barman-cloud refuses a non-empty archive only while .check-empty-wal-archive exists in PGDATA. CNPG removes that file as soon as ContinuousArchiving turns True, and with no plugin attached archive_command is a successful no-op, so the file is gone minutes after initdb, before the first BackupJob attaches the plugin. Suffix the default serverName with the application UID, in the strategy template and in the driver's fallback. The template guards the UID with `with`, so an object without one renders the previous name rather than "<no value>". A Cluster that already has the plugin keeps its serverName, so only clusters never backed up take the new prefix, and restores keep reading the source prefix from driverMetadata. Signed-off-by: Mattia Eleuteri <mattia@hidora.io> Assisted-by: LLM |
||
|
|
9734a16692 |
feat(postgres): add a postgis image flavor (#4587)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Adds a `flavor: postgresql | postgis` enum to the `postgres` app. `postgis` runs the cluster on the CloudNativePG PostGIS operand image, so tenants can enable `postgis`, `postgis_raster`, `postgis_topology`, `postgis_sfcgal`, `postgis_tiger_geocoder` and `address_standardizer` through the existing `databases.<name>.extensions`. The default stays `postgresql` and renders exactly the image it rendered before this change. This picks up #2672 by @matthieu-robin, which the stale bot closed after a rebase request, with these changes: - The postgis tags in #2672 (`17.7-3.5`, `16.11-3.5`, ...) do not exist in `ghcr.io/cloudnative-pg/postgis`; the registry answers 404, so the flavor would have ended in `ImagePullBackOff`. The map now uses tags that exist, `<minor>-<postgis>-standard-trixie`, on the same PostgreSQL minor as `files/versions.yaml`. - v18 is supported: PostGIS images for 18 now exist upstream. - `standard` rather than `system`: the chart archives through the barman-cloud plugin since the backup rework, so the operand no longer needs the barman binaries of the deprecated `system` variant. - `hack/update-versions.sh` derives the postgis map from the minors it picks, and `hack/check-postgres-postgis-versions.bats` fails if the two maps drift to different minors or away from `standard-trixie`. - The chart refuses a flavor change on a live `Cluster`. The default images are built on Debian bullseye (glibc 2.31) and the postgis ones on trixie (glibc 2.41): CNPG would roll the swap out as a plain image update, under an initialised data directory, which silently invalidates every index on collatable text, and the postgis-to-postgresql direction also drops the libraries behind every PostGIS object. Only the image name is compared, so minor bumps and registry mirrors keep passing. - `pgrouting` is not claimed: it is absent from the `18.1-3.6.1` build the chart pins and only appears in later postgis builds. A general mechanism (CNPG ImageVolume extensions) was considered and left out: it needs PostgreSQL 18 and Kubernetes 1.35 or the `ImageVolume` gate, so it cannot serve v13-v17 nor most management clusters today, and PostGIS is the only extension CloudNativePG ships as a full operand image. No other extension request exists in this repository. Testing: - `helm unittest packages/apps/postgres`: 145/145, including 6 new cases in `tests/flavor_test.yaml` (default image, postgis on the Cluster and on the init Job, the guard in both directions with a mocked live Cluster, and a mirrored digest-pinned image taking a new minor). Removing the guard makes exactly the two guard cases fail. - `hack/cozytest.sh hack/check-postgres-postgis-versions.bats` passes, and fails on a shifted minor or a `system` tag. - `make -C packages/apps/postgres generate` leaves no diff. - The rendered `ghcr.io/cloudnative-pg/postgis:18.1-3.6.1-standard-trixie` was started locally: `CREATE EXTENSION` for citext, postgis, postgis_raster and postgis_topology succeeds and `ST_Buffer` answers. A three-instance cluster on the `system` sibling of that image has been running PostGIS in production since 2026-09-29. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: cozystack/terraform-provider-cozystack#63 (issue: the provider models every field by hand, and the default handling needs a decision there) - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(postgres): add a `flavor` field accepting `postgresql` (default) or `postgis`. `postgis` bootstraps the cluster from the CloudNativePG PostGIS image, so PostGIS extensions can be enabled per database through `databases.<name>.extensions`. The flavor is fixed at creation: changing it on an existing cluster is refused, and so is restoring a backup into a cluster of the other flavor, whether through a RestoreJob or `bootstrap.enabled`; move data across flavors with a logical dump. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added a `flavor` option for PostgreSQL clusters. The default `postgresql` flavor remains available, while `postgis` selects an image with PostGIS extensions. - PostGIS images are available for PostgreSQL versions 13–18. - Cluster flavor is fixed at creation. To switch flavors, create a new cluster and migrate data with a logical dump. - **Bug Fixes** - Restores are refused when the backup and target cluster use different flavors. - **Documentation** - Updated PostgreSQL configuration guidance to describe flavor options and limitations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
0e99ec012a |
docs(observability): add a distributed tracing guide (#3772)
## What this PR does Adds `docs/observability/distributed-tracing.md`, the operator and tenant guide for distributed tracing (epic #3761, design cozystack/community `design-proposals/distributed-tracing`). The guide covers the per-tenant backend and collector (#3762, #3763): the in-namespace OTLP endpoints, the standard `OTEL_*` variables to point an instrumented workload at them, sampling, redaction and tenancy, and a per-engine table of what each managed engine can emit itself. Verification against the engines (#3766) showed that none of them pushes OTLP spans server-side, so the guide's answer for every engine is client-side instrumentation, with the ClickHouse span table as the one partial exception. It also covers shared-central tracing (#3775, with the per-tenant write rate in #4648): hosting the store in tenant-root, opting a tenant in, the shared disk and the write rate with its limits (counted before sampling, a 3 MiB request cap, the allowance a pod start lets through, at most 10 collector replicas), the trust boundary with the collector's write-only and Grafana's read-only logins, the lifecycle windows on enabling and disabling, and the order to stop hosting in. Those details used to live in the `tracingCentral` field description, which #3775 cut to a few sentences because it ships as field help in the dashboard. Both #3775 and #4648 are merged, and every figure and refusal the guide quotes was checked against the charts on `main`. ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The guide is a file under `docs/` in this repository and changes no package, values key or API, so no trigger in the downstream map applies. ### Release note ```release-note docs(observability): add a distributed tracing guide covering the per-tenant collector, client-side instrumentation per managed engine, and shared-central tracing in tenant-root. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a distributed-tracing guide covering tenant-local VictoriaTraces storage, OTLP collectors, workload configuration, engine tracing limitations, sampling, and optional attribute redaction. * Documented shared-central tracing prerequisites, including tenant-root hosting and tenant opt-in, plus write-rate, request-size, and replica limits; credentials and network rules; tenant isolation and lifecycle behavior; and conditions that stop hosting. * Clarified that cross-signal correlation is not configured. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
b3103af693 |
docs(observability): add a distributed tracing guide
Tracing reaches a workload only if it is instrumented and pointed at the collector, and the managed engines push no OTLP spans of their own, so operators and tenants need one place that says what the platform provides and what is left to the application: the in-namespace OTLP endpoints and the standard OTEL_* variables, sampling and redaction, and what each engine can and cannot emit. Shared-central tracing has more to it than the Tenant field description can carry, since that text is shown as field help in the dashboard: hosting the store in tenant-root, the shared disk and the per-tenant write rate with its limits, the write-only and read-only logins and the trust boundary, the windows on turning it on and off, and the order to stop hosting in. The guide holds that detail. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
11499c30e1 |
feat(monitoring): hold each shared-central tenant to a write rate (#4648)
## What this PR does Closes the part of the distributed tracing design that #3775 left open: the per-tenant collector rate-limits its tenant, so that a noisy or hostile tenant cannot exhaust a shared backend (cozystack/community `design-proposals/distributed-tracing`, Security and §2). **Why the disk cap is not enough.** #3775 bounds every traces store by disk usage, but VictoriaTraces always keeps the last two days, so a tenant writing faster than the cap allows over two days fills the disk every tenant on the shared store writes to. Only a per-tenant write rate closes that. **The collector is built here.** No upstream OpenTelemetry distribution or processor limits throughput, and neither the open-source vmauth nor VictoriaTraces limits a tenant's rate. Elastic's EDOT Collector registers Elastic's `ratelimitprocessor` among its default components, but it ships under the Elastic License 2.0 and carries far more components than the chart uses. The collector is therefore built with the OpenTelemetry Collector Builder from the upstream components the chart already uses, at the same v0.136.0, plus that `ratelimitprocessor` (Apache-2.0). Its v0.8.2 is the release built against collector v0.135.0, the last one before v0.137.0, and it builds against v0.136.0 unchanged. The image is versioned by the collector release with a cozystack respin counter (`v0.136.0-cozystack.0`), runs as the upstream image's user, and is built with the other package images. **How the rate is set and enforced.** The platform sets the rate with `monitoring.tracingCentralTenantBytesPerSecond`, and it reaches every tenant in `_cluster` through the `cozystack-values` Secret. A change there is a change in the values every monitoring HelmRelease reads, so Flux upgrades each of them; a rate read with `lookup` at render time would never reach a tenant whose release is already in sync. The platform writes the key only for a positive rate, because every release in every tenant reads that Secret, and refuses to render on a value that is not a whole number of bytes per second in plain decimal digits, up to 1 TiB/s ("1Mi", 1.5, "0100"), rather than reading it as the default. Without it, or on a value that is not a positive number, the collector holds the tenant to the default of about 97 KiB/s, at which one tenant writes at most the disk cap of a default 10Gi store over two days, counted uncompressed; the store keeps spans compressed, so they take less than that. A larger store needs the rate raised with it. Each central tenant's collector splits the rate across its replicas and refuses data above it before the batch processor, so the application sees the refusal, and right after the memory limiter, ahead of the sampler, which adds a tracestate to every span it keeps, and of the tenant attribute added to every resource, so it counts a request as the receiver admitted it. The rate is therefore counted before sampling, and the store keeps only the sampled share of it. Over OTLP/HTTP it gets a 429, which OTLP exporters retry. Over OTLP/gRPC the limiter answers RESOURCE_EXHAUSTED without RetryInfo, which the OTLP specification makes final, so a gRPC exporter drops those spans; the Tenant's `tracingCentral`, the Monitoring `tracingCollector` and the platform key tell central tenants to export over HTTP. The limiter sees the requests the receiver admits, so in central mode the receiver caps a request at 3 MiB over both OTLP/HTTP and gRPC, decompressed for HTTP. The limiter counts a request as the collector re-encodes it, which can be larger than it arrived (13% for a span carrying only its IDs, the smallest valid one, and up to 1.4x for spans with empty-valued attributes), so burst is at least 5 MiB, above 1.4 times the cap, since a request larger than burst never passes. Each collector pod starts with a full bucket, so a pod start lets through above the rate up to 5 MiB or one second of the pod's share of the rate, whichever is larger, and the chart refuses to render a central tenant's collector with more than 10 replicas. **Only the collector can write.** The rate holds only if the collector's login is the only way the tenant writes. Each central tenant now has two VMUsers on its one account: the collector's, which may only write, and its Grafana datasource's, which may only read, each with its own Secret. With one login for both, any user the tenant admits to its Grafana, a Viewer included, could POST through the datasource proxy to `../insert`, which vmauth cleans before it picks a route, and write into the shared store past the limiter. The tenant's Grafana pod may reach the vmauth too, so its image is the platform's, like the collector's, and only the `auth.ini` key is mounted from the Secret a tenant names in `oidc.customConfig.secretRef`: an image of the tenant's choosing with the collector's Secret mounted would otherwise write past the limiter as well. **The collector image is the platform's.** The apps API defaults a Monitoring spec from its schema but does not prune keys the schema leaves out, and the Monitoring chart hands its values to the inner chart as they are, so a tenant could set `tracingCollector.image` and run any collector it liked, one without the limiter included. That key is no longer passed through. **Trade-offs.** The limiter is a vendor component, not an OpenTelemetry one, and Elastic marks it as in development; it moves with Elastic's releases, so bumping the collector means picking the matching release of it (v0.74.0 builds against collector v1.67.0), and collector v0.136.0 is a year old. It logs each refused request at error level, so a tenant held at its limit produces steady log lines in its own collector. Several tenants writing at their limits together can still exceed the cap within two days; the rate bounds each tenant, not their sum. The limiter keeps its state in each pod's memory, so a tenant admin who restarts its collector pods over and over gains, per pod start, up to 5 MiB or one second of the pod's share of the rate, whichever is larger; closing that needs a limiter with shared state (the processor's gubernator mode), which is out of scope here. A sender whose connection stays on one replica gets that replica's share of the rate. In central mode the receiver refuses a request larger than 3 MiB: over OTLP/HTTP with a 400, which senders do not retry, where the receiver default is 20 MiB, and over gRPC, where the default is 4 MiB. Changing the rate upgrades every release in every tenant once, since it travels in the Secret they all read. Until CI builds the image, `tracingCollector.image` in `values.yaml` still names the upstream distribution, which has no rate limiter, so a central-mode collector from an unbuilt tree does not start; CI and releases stamp the built image, and the image repository in the build registry has to be public before a cluster can pull it. **Upgrade.** Collectors in both modes move to the built image, which carries the same components at the same version; central collectors also roll for the new processor. The replica cap fails the render of a central tenant's collector with more than 10 replicas, while a collector writing to the tenant's own store keeps any replica count; the collector has shipped only in the v1.7.0 alphas. Each central tenant's existing VMUser loses its read route, and a second VMUser and Secret appear for Grafana, whose datasource moves to the new login in the same release, so the tenant's trace reads may fail until vmauth loads the new user. Nothing else changes for clusters that do not set the rate: the platform writes no new key, so no other release upgrades. **Testing.** helm-unittest covers the platform writing the rate into `_cluster` only for a positive whole number and refusing a malformed one, numbers Helm reads as float64 included, the collector's limiter (the configured rate, the split across replicas, burst, the receiver's request cap in central mode and not in local mode, the default when unset and on a word, zero or a negative value, its place right after the memory limiter, with and without redaction, its absence in local mode), the replica cap for a central tenant's collector and its absence for a local one, the image key no longer passing through, the two logins (the collector's with only the write route, Grafana's with only the read route, separate Secrets and passwords), the Grafana image not passing through, and only `auth.ini` being mounted from a `secretRef` Secret for a central tenant; every term was checked by mutation. On a test cluster, with the chart's rendered VMAuth, VMUsers, VTCluster, Grafana datasource and collector: with one login for both, a Grafana Viewer's POST to `../insert` through the datasource proxy landed a span in the shared store; with the two logins vmauth answered `400 missing route` for every encoding of the traversal tried, reads went through, and the collector's login could write but not read. With the built collector at 100% sampling and the receiver capped at 4 MiB, a fresh pod accepted a request just under that cap made of 53-byte spans and one made of ID-only spans, refused a larger one with a 400, and answered 429 once the bucket was drained; the 53-byte spans it accepted were read back through the read-only login; the 3 MiB cap the chart renders has not been run on a cluster. A pod mounting a Secret through the chart's `secretRef` volume as rendered for a central tenant got only `auth.ini`, and one naming the collector's Secret, which has no such key, did not start. The design's Testing section also asks for a sustained span flood against a disk-capped store to show that the oldest spans are evicted and ingest is not blocked; there is no end-to-end lane for shared-central tracing yet, so that flood test is a follow-up, like the cross-tenant isolation test #3775 names. On a test cluster, under the network policies the tenant chart renders, the built image validated the rendered config in both modes and wrote and read through the shared store as before; a tenant given a small limit got 200 for its first OTLP/HTTP requests and 429 for the rest of a burst, with the processor logging each refusal, while another tenant under the default rate got 200 for all of the same requests. ### Screenshots No UI changes. ### Downstream repositories - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: cozystack/website#729 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The new platform key `monitoring.tracingCentralTenantBytesPerSecond` belongs in the hand-written `platform-package.md` reference on the website, tracked in cozystack/website#729. The Monitoring schema and the Tenant's `tracingCentral` change only descriptions; the provider models neither, so it needs nothing. The central-mode section #3772 adds to the tracing guide covers the rate. ### Release note ```release-note feat(monitoring): hold each shared-central tracing tenant to a write rate. The platform sets the rate with monitoring.tracingCentralTenantBytesPerSecond, and each central tenant's collector refuses spans above it, which bounds what one tenant writes into the store every tenant shares. The traces collector is now built by cozystack from the upstream components plus Elastic's rate-limit processor, and its image can no longer be set from a Monitoring application. A central tenant's Grafana reads the shared store with a login that cannot write, and its image can no longer be set from a Monitoring application. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Central tracing now applies a configurable per-tenant write-rate limit shared across collector replicas. The default is approximately 97 KiB per second, and collector replica counts are limited to 10. * Each starting replica can temporarily exceed its rate share by up to 5 MiB, or by one second of that share when it is larger. Tenants share central storage, so combined writes can fill the store and cause older traces to be evicted. * **Documentation** * OTLP/HTTP is recommended for central tracing: excess traffic is retried over HTTP, while OTLP/gRPC traffic may be dropped. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
e63b9e439c |
fix(monitoring): count a central request as the receiver admitted it
The limiter refuses for good any request it counts above burst, and burst was the receiver's 4 MiB cap. A request grows on its way to the limiter: the sampler adds a tracestate to every span it keeps, and the limiter counts a request as the collector re-encodes it, which writes the empty value of every attribute the wire may omit, so a valid request of spans with empty-valued attributes is counted 1.4x. A request the receiver admitted near its cap was therefore refused on every retry. The limiter now runs right after memory_limiter, so it counts the request before sampling. The receiver caps a request at 3 MiB and burst stays at 5 MiB, which holds 1.4x the cap; a larger burst would also be what each pod start lets through above the rate. The rate is counted before sampling, so the store keeps only the sampled share of it. The texts say that a request above the cap gets a 400, which senders do not retry, and that each collector rollout a tenant causes lets that burst through again on every pod. The replica bound that keeps that allowance small now applies only to a central tenant's collector, whose render fails above ten replicas: a collector writing to the tenant's own store has no rate, and the replica counts it shipped with stay valid. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
65f0caae2b |
fix(monitoring): keep a central tenant's write login out of its Grafana
A tenant's Grafana pod may reach the shared traces store's vmauth, and
the Secret a tenant names in oidc.customConfig.secretRef was mounted
into it whole. The apps API does not prune keys the Monitoring schema
leaves out, so a tenant could also set grafana.image: an image of its
choosing, with the collector's Secret mounted, writes past the rate
limit the collector enforces.
Grafana's image is the platform's, like the collector's, and for a
central tenant only the auth.ini key Grafana reads is mounted from the
named Secret. Other tenants have no collector login to keep out and
keep the whole Secret, which an auth.ini reading a sibling key with
$__file{} relies on.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
|
||
|
|
cc51cac63d |
fix(monitoring): give a central tenant's Grafana a login that cannot write
The collector holds a central tenant to its write rate, but the one login it writes with was also the login of the tenant's Grafana datasource, and it could both read and write. vmauth cleans the request path before it picks a route, so any user the tenant admits to its Grafana, a Viewer included, could POST through the datasource proxy to ../insert and write into the shared store past the limiter. Each central tenant now has two VMUsers on its one account: the collector's, which may only write, and Grafana's, which may only read, each with its own chart-owned Secret and password. A namespace name holds no dot, so the reader's username never matches another tenant's. On upgrade the datasource moves to the new login in the same release that creates it, so reads may fail until vmauth loads the new user. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
3497855e15 |
fix(platform): read the tracing rate only as plain decimal digits
Sprig's int64 reads a string with a leading zero as octal and turns a parse error into 0, so "0100" would reach the collectors as 64, and "08" or digits beyond int64 would leave the default in place without a word. A float beyond int64 converts differently on each CPU architecture, and the collector adds its replica count to the rate before splitting it. A string rate must now be plain decimal digits, and any rate above 1 TiB/s fails the render; the collector also falls back to the default on digits it cannot hold rather than start with a rate of zero, which the processor refuses. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
a0aea139fc |
fix(platform): refuse a malformed shared-central tracing rate
A rate written as a quantity or a fraction ("1Mi", "100KiB", 1.5) was
read as zero, so the platform wrote no key and every collector kept
the default without a word. The platform render now fails on anything
but a whole number of bytes per second. Helm reads a number from
values as a float64, which prints a million as 1e+06, so a whole
number is checked as a number and written as int64 digits.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
|
||
|
|
fc0db49187 |
fix(monitoring): bound what a central tenant writes above its rate
Each collector pod starts with a full token bucket, and burst was at least 64 MiB, the largest request vtinsert accepts. A tenant controls when its collector pods start, through any Monitoring change or by deleting them, and how many there are, so it could write tens of mebibytes above its rate per pod start and gigabytes per rollout of many replicas, which the two-day bound does not survive. The limiter only sees requests the receiver admits, so in central mode the receiver now caps them at 4 MiB, decompressed for HTTP, and burst drops to that floor; a pod start lets at most 4 MiB through above the rate. The replica count is capped at 10. The texts now say what the default rate guarantees: one tenant writes at most a default store's disk cap over two days counted uncompressed, rather than that it cannot fill it, and they name the per-pod-start allowance and that a sender pinned to one replica gets that replica's share. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
f8deb8e171 |
fix(platform): write the tracing rate only when it is positive
apps.yaml wrote the rate for any non-zero value, so a negative one reached every tenant's cozystack-values Secret and upgraded every release in every tenant, while the collectors ignored it and kept the default. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
177b207a27 |
feat(monitoring): hold each shared-central tenant to a write rate
The design has the per-tenant collector rate-limit its tenant, so that a noisy or hostile tenant cannot exhaust a shared backend. The disk cap alone does not do that: VictoriaTraces always keeps the last two days, so a tenant writing faster than the cap allows over two days fills the disk for every tenant on the store. The platform sets the rate with monitoring.tracingCentralTenantBytesPerSecond, and it reaches every tenant in _cluster through the cozystack-values Secret. A change there is a change in the values each monitoring HelmRelease reads, so Flux upgrades them all; a rate read with lookup at render time would never reach a tenant whose release is already in sync. The platform writes the key only when it is set, since every release in every tenant reads that Secret. Without it the collector holds the tenant to the rate of a default 10Gi store. Each central tenant's collector splits the rate across its replicas and refuses data above it before the batch processor, so the sender sees the refusal. An OTLP/HTTP sender gets a 429 and retries; the limiter answers gRPC with RESOURCE_EXHAUSTED and no RetryInfo, which the OTLP specification makes final, so central tenants should export over HTTP. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
32a6c71972 |
fix(monitoring): keep the collector image out of a tenant's reach
The apps API defaults a Monitoring spec from its schema but does not prune keys the schema leaves out, and the Monitoring chart hands its values to the inner chart as they are. A tenant could therefore set tracingCollector.image and run any collector it liked in its namespace; with a per-tenant rate limit in that collector, the same key would let a tenant drop the limit. The image is the platform's, so it is no longer passed through. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
474377e8ca |
build(monitoring): build the traces collector with a rate limiter
A shared-central traces store needs each tenant held to a write rate, or one tenant can fill the disk every tenant shares. No upstream OpenTelemetry distribution or processor limits throughput. Elastic's EDOT Collector registers Elastic's ratelimitprocessor among its default components, but it ships under the Elastic License 2.0 and carries far more components than the chart uses. The collector is therefore built here with the OpenTelemetry Collector Builder from the upstream components the chart already uses, at the same v0.136.0, plus that ratelimitprocessor (Apache-2.0). Its v0.8.2 is the release built against v0.135.0, the last before v0.137.0. The image is versioned by the collector release with a cozystack respin counter, runs as the upstream image's user, and is built with the other package images. The builder stage is alpine, which has no git, so a proxy error during its go install would have had neither the direct fallback nor a retry; the stage installs git. The module-fetch contract saw only go build, so it now checks go install too, and takes only git as cover for a versioned go install, which resolves outside the stage's module. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
a5d12e78d4 |
fix(ci): reconcile three unit-test contracts broken by adjacent merges (#4751)
## What this PR does Unit tests on main are red, and that blocks E2E on every open PR. Three contracts broke the same way. In each case a check from one PR and a line from another merged a few minutes apart on 2026-10-05, with no textual conflict. Any one of them alone keeps the job red, so they are fixed together. The first is the pin on how a suite reads HelmRelease history. #4689 added it. It accepts `$(kubectl ...` and, since #3594, the bounded `$(_cozy_guard_kubectl ...` wrapper. Nothing in front of kubectl is allowed. The monitoring suite from #4678 bounds that read with `$(timeout 30 kubectl ...`, and the bound is wanted. So the pin now accepts an optional `timeout [-k N] N` in front of kubectl. The prefix is spelled out on purpose. Any text would also let through `$(echo kubectl ...)`, which names the shared expansion without running it, and that form is now a refuse case. I checked it with mutations: the new accept cases fail against the old pattern, and a pin that takes any prefix fails a refuse case. The second is the flux install in workflows. #4645 forbids running fluxcd.io/install.sh and moved every workflow to the SHA-pinned fluxcd/flux2 action. The isp-hosted lane from #4712 still ran the script. It now uses the same action and version. The third is the negated-assertion audit. It reads the find operand in `\( ! -type d` in select-e2e_test.bats as a shell negation and fails. The line now says `-not -type d`. This is the same change as #4748, so this PR supersedes it. The audit bug itself is #4752. After the Go fix, the full log of the unit job had only these two bats failures left. Closes #4750 Closes #4755 ### Screenshots ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(ci): fix three unit-test contracts that went red on main after adjacent merges. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * History checks now recognize direct `kubectl` reads and reads through the supported wrapper, including timeout-bounded commands with an optional kill-grace setting. Commands that only include `kubectl` as an argument, use similarly named commands, or place a timeout before the wrapper are not treated as reads. * **Chores** * The hosted end-to-end workflow now uses a pinned Flux version, and suite selection handles directory exclusions consistently. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
f4a6eb809c |
fix(nats): enable JetStream for the generated account (#4675)
## What this PR does With `users` configured and `jetstream.enabled: true`, the application chart enables JetStream on the server but generates account `A` without it. Authentication and ordinary NATS messaging work, while `$JS.API.INFO` returns error 10039, "JetStream not enabled for account". The generated account now gets `jetstream: true` when persistent messaging is enabled and the account does not set JetStream itself. Tenant-supplied limits, explicit disablement, other accounts and the existing generated-user merge are kept. nats-server matches config keys case-insensitively and walks the account map in no fixed order, so a `JetStream` key in any casing counts as set. Otherwise the account would carry both keys and the server would apply either one, changing from restart to restart. Existing instances with `users` pick this up on upgrade. The rendered NATS config changes, so the StatefulSet rolls its pods once, and account `A` gets JetStream without account limits unless `config.merge.accounts.A.jetstream` sets them. ### Validation - `helm unittest` passes all 75 application-chart tests. Eight cases cover this change. Against the original template the generated-account and null-override cases fail. The other six pin what the change must keep, and the mixed-case one fails as soon as the check is case-sensitive. - Dropping the `jetstream.enabled` check, the case folding or the explicit-setting check turns at least one case red each time. - The application chart and the vendored system chart were rendered, and the resulting account configuration was run on NATS Server 2.11.10 in an isolated standalone setup. The original template reproduces error 10039. With this change a file-backed stream can be created, published to and read, and explicit disablement and a one-stream limit stay enforced. - `make generate` with cozyvalues-gen v1.7.0 leaves the generated files unchanged. ### Screenshots Not a UI change. ### Downstream repositories The diff changes an application template and chart tests only. Public values, schemas, output resource names and tooling contracts are unchanged. The website NATS documentation contains no account-level workaround to remove. No downstream trigger applies. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(nats): enable JetStream on the account generated for users when jetstream.enabled is true, keeping any explicit account-level JetStream setting. Instances with users roll their NATS pods once on upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Generated NATS accounts now enable JetStream when configured, without overriding an existing JetStream setting. Configured passwords are added while preserving existing account settings. * **Tests** * Added coverage for enabled and disabled JetStream, explicit limits and false or empty values, case-insensitive setting names, null overrides, and manually configured accounts when generated users are absent. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
0cae0e87ad |
test(e2e): spell the find negation as -not in the suite-list helper
The negated-assertion audit splits a line on `(` and reads the `!` after `\(` as a shell negation that errexit would exempt, so the helper's find predicate fails the audit and the unit tests on main. `-not` is the same predicate in GNU, BSD and busybox find and does not look like one. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
c854e054b9 |
ci(workflows): install flux in the isp-hosted lane through the pinned action
The isp-hosted lane fetched fluxcd.io/install.sh and ran it as root, which the CI tool-install contract forbids for every workflow: it runs whatever the site serves at that moment. The lane landed next to the contract and kept the old step, leaving the unit tests red on main. It now uses the same SHA-pinned fluxcd/flux2 action and version as the other workflows. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
f527ef17ee |
test(fluxcontract): accept a timeout-bounded history read
The monitoring suite bounds its HelmRelease history read with `timeout 30 kubectl`, which keeps a hung API call from eating the step's whole budget and the diagnostic with it. The shared-read pin admitted kubectl only bare or through the bounded wrapper, so the two landed side by side and left the unit tests red on main. The pin now takes an optional `timeout [-k N] N` in front of kubectl, the forms the suites already use for bounded reads. The bound is spelled out instead of allowing any prefix, since `$(echo kubectl ...)` would name the expansion without running it; that form, bare and behind a timeout, is now a refuse case. So is a timeout in front of _cozy_guard_kubectl: timeout runs a program, the wrapper is a shell function, and the read would fail every time. The command name has to end at a blank, so `kubectlx` or `_cozy_guard_kubectl_v2` is not taken for either one. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
|
7bf6906a80 |
fix(platform): keep the legacy etcd-operator stopped after etcd adoption (#4738)
## What this PR does Migration 50 scales the legacy etcd-operator to 0, adopts every legacy `etcd.aenix.io` cluster with `etcd-migrate`, and then scales the operator back to 1. At that point the Deployment still runs the legacy operator, and after a successful adoption it has nothing left to manage. Bringing it back undoes part of the adoption. Observed on a production cluster upgraded from 1.4.3 to 1.6.4. The API audit log and the Helm history of a tenant's `etcd` release show this sequence, in UTC on the same day. - 13:21:12 migration 50 scales `cozy-etcd-operator/etcd-operator-controller-manager` to 0. - 13:22:59 `etcd-migrate` updates `etcd-headless`, pointing its owner references at the adopted EtcdMembers, and orphan-deletes the legacy EtcdCluster and StatefulSet. - 13:23:04 migration 50 scales the operator back to 1. It is still the legacy operator, the v1alpha2 one replaced it only at 13:40. - 13:23:42 Flux upgrades the tenant's `etcd` release with the old chart (Helm revision 87), because the new chart artifact appeared only at 13:24:40. The old chart re-creates the legacy EtcdCluster at 13:23:43. - 13:23:43 to 13:23:45 the legacy operator reconciles it, updates `etcd-headless` twice and re-creates the legacy StatefulSet `etcd`. - 13:36:49 Flux upgrades the release to the new chart (revision 88), which prunes the legacy EtcdCluster. - 13:36:50 the garbage collector deletes `etcd-headless` and the re-created StatefulSet. The same happened in all five tenants that run an etcd. The pods survived because EtcdMembers own them, but they keep `spec.subdomain: etcd-headless`, so their `etcd-<i>.etcd-headless.<ns>.svc` names stopped resolving. From then on every member logged `no such host` for its peers, the v1alpha2 EtcdCluster had no status conditions, and the `etcd-defrag` CronJob failed every hour with `context deadline exceeded`. The transitional `etcd-headless` Service the etcd chart ships for this window (#3270) does not cover it. Helm patched the existing Service 202 ms before the garbage collector removed it, and did not create it again because it was already part of the release. With this change the operator stays at 0 after a successful adoption. The etcd-operator HelmRelease replaces the Deployment with the v1alpha2 operator later in the same upgrade, and no later migration waits for the operator. A failed or interrupted adoption still restores it to 1 through the existing EXIT trap, so a failed run does not leave the tenant etcds without an operator. If that HelmRelease cannot reconcile, the adopted clusters wait for it with or without this change, because the legacy operator does not reconcile v1alpha2 EtcdClusters. Clusters already upgraded with this bug can be repaired without restarting anything. Creating the Service from the tenant's own `etcd` release is enough. ``` helm -n <ns> get manifest etcd | awk 'BEGIN{RS="\n---\n"} /\nkind: Service\n/ && /\n name: etcd-headless\n/' | kubectl -n <ns> create -f - ``` On the cluster above the v1alpha2 EtcdCluster reported `Available` with reason `QuorumHealthy` within 11 seconds of the Service being created, in every tenant, and the members stopped logging `no such host`. ### Validation - `hack/cozytest.sh hack/migration-50-etcd-adopt.bats` passes all 29 tests. A new test asserts that a successful adoption scales the operator to 0 and never back to 1. Against the current script it fails with `FAIL: the legacy operator was scaled back up after a successful adoption`. - The existing test for a failed adoption still passes, so the EXIT trap still restores the operator to 1. - The new test checks the absence with `if grep ...; then echo "FAIL: ..."; false; fi`, the form the other suites use. A bare `! grep` assertion cannot fail a test under `hack/cozytest.sh`, so it could not have caught this. The other negated assertions in this file had the same weakness, and #3790 has rewritten them on main. With all of them made able to fail in a scratch copy, the suite still passes against this change. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. It changes a migration script and its bats test. No package, values, schema, CRD, make target or `hack/` layout changes, so no entry matches. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(platform): keep the legacy etcd-operator stopped after migration 50 adopts the legacy etcd clusters. Scaling it back up let it reconcile the legacy EtcdCluster that Flux re-created from a not yet upgraded etcd chart, re-create the legacy StatefulSet and take the legacy etcd-headless Service away from the adopted members, which the garbage collector then deleted. The adopted members lost their DNS names, the EtcdCluster never reported a status and etcd defrag failed every hour. Clusters already upgraded can recreate the Service from the etcd release manifest. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Migration Updates** * After successful etcd adoption, the legacy operator remains scaled to zero replicas for replacement by the etcd operator; it is no longer scaled back up before the migration is stamped. * If adoption fails or exits before completion, the legacy operator is restored to one replica. * The dry-run message now describes the zero-replica outcome of successful adoption. * Successful adoption continues to record the migration version stamp. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
415430bcc7 |
fix(e2e): fail on a failed release of cilium and csi as on the parent, and report addon remediation (#3594)
## What this PR does The history guard in the tenant Kubernetes e2e suite checked only the parent `kubernetes-<test>` HelmRelease. A failed action on one of the addon releases it installs left nothing the suite looked at, so a run showed only the downstream symptom, usually the node-join deadline, and never named the release behind it. The guard now reads every addon release of the parent. It selects them from the cluster by name prefix, so a release added to the chart later is covered too. The rule: - A `failed` or `uninstalled` Snapshot fails the run on any release whose install strategy is `RetryOnFailure`. That is the parent (the aggregated apiserver stamps it on every Application HelmRelease) and the tenant `cilium` and `csi`. - The same footprint on the other addons, which run the default strategy with `retries: -1`, is printed as a note and the run stays green. For the CRD addons that configuration is tracked in #3555. A transient `cilium` or `csi` failure that later recovers on a retry still turns the run red, by design, as long as its Snapshot still reads `failed`. One recovery leaves no such Snapshot: when an install fails and the very next attempt succeeds, Helm marks the failed install `superseded`, because there is no deployed release to supersede, and the history reads like an ordinary upgrade. The guard cannot see that case from the statuses alone. The code and the docs now say so, a test pins it, and detection is left to a follow-up. An earlier revision of this PR tried to tell a teardown apart from a retried failure and made only `uninstalled` fatal. That split does not hold. In helm-controller v1.5.0, the version the platform installs, a release with an active retry strategy never remediates. `atomic_release.go` returns an upgrade for a failed release before it reaches the remediation switch, and the two other paths that uninstall (a changed release target, and deletion) both clear history afterwards. So on these releases `uninstalled` never shows up, and `failed` is the only signal. The parent keeps the rule it already had on main, and `cilium` and `csi` now follow it too. The failure paths get the same reading. The guard runs after the assertions, while the deadlines a failed addon breaks end the script earlier, so the same classification is also printed, without a verdict, from the node-join collector and from the exit handler. That handler is now armed at the tenant CR apply, so the management-cluster waits before the tenant answers are covered too. Smaller changes that came with this: - Every guard read is bounded, and a failed read is never folded into "no history". - The parent must read back a non-empty strategy. An empty value there means the jsonpath stopped resolving, and without this check every footprint would quietly turn into a note. - The history read uses `HELMRELEASE_HISTORY_JSONPATH`, and the upstream-type pin in `internal/fluxcontract` now accepts the bounded wrapper the read goes through. `docs/agents/e2e-testing.md` §6 and checklist item 8 describe the new rule. On related issues: this adds the `local` that #3774 reports missing in `helmrelease_has_remediation_cycle`, but not the sweep over the other `_lib/` helpers that the issue also asks for. For #3773 it rewrites the guard comment the issue quotes. The waits without a backstop are not touched. Closes #3553. ### Screenshots Not a UI change. ### Downstream repositories I walked the trigger map against the diff. It touches `hack/e2e-chainsaw/_lib/`, `hack/*.bats`, one Go test under `internal/fluxcontract`, two spend-order comments in the kubernetes suites, and `docs/agents/e2e-testing.md`. No package, schema, CRD, make target or user-facing document changes. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(e2e): the HelmRelease history guard now also covers the tenant addon releases. A failed or uninstalled Snapshot fails the run on the parent, cilium and csi, which retry in place; other addons get a note. The same classification is printed on the failure paths where the guard itself cannot run. ``` |
||
|
|
0639ce39aa |
fix(kamaji): drop any subchart nested under the vendored kamaji chart (#4459)
## What this PR does A subchart nested under the vendored kamaji chart would deploy no matter what the values say, so this PR removes that directory in `make update` and adds a test that fails if anything renders from it. The vendored chart declares a `kamaji-etcd` dependency gated on `kamaji-etcd.deploy`, and the subchart itself is not vendored. The issue proposed pinning `kamaji-etcd.deploy: false` in the package values. I tested that, and it would not have helped: through this package the condition is never evaluated at all. Helm skips the condition because the umbrella chart declares no dependencies. `processDependencyEnabled` returns immediately when the chart it starts from has no `dependencies` in its `Chart.yaml`, and `cozy-kamaji` has none. The vendored `kamaji` chart is loaded from `charts/` without being declared, so Helm never recurses into it and never reads its `kamaji-etcd.deploy` condition. The same early return is in Helm v3 and v4. I put `kamaji-etcd-0.15.0.tgz` into `charts/kamaji/charts/` and rendered the package three ways: - default values: StatefulSet, Job, DataStore and four Certificates from `kamaji-etcd` - `kamaji.kamaji-etcd.deploy: false` in the package values: the same etcd - `deploy: false` edited into the vendored `charts/kamaji/values.yaml`: the same etcd Rendering `charts/kamaji` on its own with `kamaji-etcd.deploy=false` renders no etcd, which confirms the umbrella is what skips the condition. So any chart that ends up under `charts/kamaji/charts/` deploys whatever the values say, and no values change can close the hazard. This PR changes two things: - `make update` removes `charts/kamaji/charts` after extracting the upstream chart. Upstream does not ship that directory today, so the recipe still reproduces the current tree exactly. I checked that by running it on a copy of the package. I also ran it against a fake upstream tarball that carried the subchart: the directory was extracted and then removed. - The package gets its first helm-unittest suite. It asserts that no template of a subchart nested under `charts/kamaji/charts/` renders a document, and a `test` target lets `hack/helm-unit-tests.sh` pick it up in `make unit-tests`. With the subchart tgz dropped in, the suite fails. With the `rm` line removed from the recipe, the fake-upstream update leaves the tgz behind and the suite fails too. `/tests` is anchored in `.helmignore`, so the packaged chart's file list is unchanged. I left the `kamaji-etcd` block in the vendored values alone. It is not quite dead: `certmanager_issuer.yaml` reads it to decide whether to keep the self-signed issuer. That only matters when `certManager.issuerRef.name` is set, and this package does not set it. `helm template` output is byte-identical to main. Closes #4397 ### Screenshots Not a UI change. ### Downstream repositories The change touches only the kamaji package's update recipe, its test suite and its `.helmignore`. The closest trigger is package `Makefile`s → the website's `development.md`, but that page describes `make update` and `make test` in general terms that still hold, and the rendered chart is unchanged. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kamaji): `make update` now drops any subchart nested under the vendored kamaji chart, and a unit test fails if one appears, because such a subchart (for example the bundled kamaji-etcd) would deploy regardless of `kamaji-etcd.deploy`. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Helm chart updates now avoid including nested charts that could deploy unexpectedly. * Helm chart packages exclude test files, keeping them out of distributed chart artifacts. * **Tests** * Added a chart check to verify nested charts render no resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2188f16461 |
test(vminstance): cover VMDisk clones from an image and from a disk (#4586)
## What this PR does Closes #3482. The `vminstance` suite now covers both VMDisk clone paths, which had no e2e test at all. One disk is cloned from a golden image with `source.image`, and a second disk is cloned from the first with `source.disk`. For each clone the test checks that the DataVolume points at the expected source PVC and becomes `Ready`, that its PVC is `Bound`, and that the VMDisk reports `Ready` and `WorkloadsReady`, the same way the standalone VMDisk test does. The `Bound` check is separate because CDI derives `Ready` from the DataVolume phase alone, and can reach `Succeeded` from the claim's annotations before the claim binds; a `phase: Succeeded` assert would only repeat `Ready`. The golden image is a fixture the suite creates itself. `vm-default-images` is an opt-in package and the e2e install does not enable it, so `alpine-3.21` from the default collection is not there to clone. The suite imports the same Alpine 3.21 image into `cozy-public` under the name `e2e-alpine-3.21`, so it can never collide with an image the package owns. This is the only download in the new test, and the `source.disk` case reuses the first clone as its base disk. The long wait sits on the DataVolume, not on the HelmRelease. Since #4380 a vm-disk release goes Ready once the DataVolume exists and the clone's progress is reported through the VMDisk `WorkloadsReady` condition, so the test does not assert the HelmRelease at all. Chainsaw deletes the clones before the golden image at cleanup. The clone itself is only proven by this PR's E2E run, on the `local` class of the PR lane. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only `hack/e2e-chainsaw/vminstance/`, and nothing in the trigger map covers it. ### Release note ```release-note test(vminstance): e2e now covers VMDisk clones from a golden image (source.image) and from another disk (source.disk). ``` |
||
|
|
4b1a75a332 |
test(vminstance): dry-run evict the VM launcher pod (#4585)
## What this PR does Part of #4422. The e2e suite now asks KubeVirt whether it would let a VM launcher pod be evicted, without evicting it. Until now nothing in e2e evicted a launcher pod, so the drain behaviour set up in #4403 was known only from reading KubeVirt. Once the `vminstance` VM is running, the suite sends a dry-run Eviction (`dryRun=All`) for its virt-launcher pod and checks the answer. Nothing gets deleted. A dry run still goes through the KubeVirt eviction webhook, which is registered with `sideEffects: NoneOnDryRun` in v1.9.0. With `LiveMigrateIfPossible` its answer follows the VMI's `LiveMigratable` condition. That condition depends on storage, so the test reads it instead of assuming it: - `LiveMigratable=True`: the eviction must be refused with the webhook's `Eviction triggered evacuation of VMI "<ns>/<name>"` message (HTTP 429). A drain retries on that while the VM migrates away. - `LiveMigratable=False`: the eviction must be allowed. The old `LiveMigrate` value refused this case with "is not live-migratable". KubeVirt v1.9.0 creates no PodDisruptionBudget for a VMI, so no PDB can refuse it either. - Anything else, including a missing condition, fails the test. The allowed answer alone proves little, because the webhook also lets an eviction through under `None`. So the test also asserts `evictionStrategy: LiveMigrateIfPossible` in the VMI spec, where virt-api copies the cluster value at creation. After the dry run it checks that the launcher pod is the same object and not terminating, and that the VMI has no `evacuationNodeName`. On the PR lane only the `False` branch runs. Post-install prep pins `StorageProfile/local` to RWO because DRBD is off there, and an RWO disk makes the VMI non-migratable. Nightly and tag runs use `replicated`, where I expect the `True` branch, but I have not seen this suite run on that lane yet. A real drain and a workload update after a KubeVirt bump are still open in #4422. I ran the script against a stub `kubectl`. Both expected answers pass, and it fails on every wrong answer I tried, such as a refusal for a non-migratable VMI, a PDB-style 429, a missing condition, two launcher pods, or a replaced pod after the dry run. `chainsaw lint` (v0.2.15) and shellcheck pass. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only `hack/e2e-chainsaw/vminstance/` and the suite README, and nothing in the trigger map covers either. ### Release note ```release-note test(vminstance): the e2e suite now dry-run evicts the VM launcher pod and checks the KubeVirt eviction decision against the VMI's LiveMigratable condition. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added end-to-end coverage for dry-run eviction of a running virtual machine’s launcher pod. The test checks the VM’s live-migration policy and `LiveMigratable` condition, expecting eviction to be refused when the VM is live-migratable and accepted otherwise. * Verified that the dry run leaves the launcher pod unchanged and does not record an evacuation node on the VM. * **Documentation** * Documented the test’s behavior and the local `jq` requirement for running it. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6a98acb61d |
fix(api): leave failed golden images out of the image options (#4568)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The VM image picker no longer offers golden images whose import failed. A golden keeps its PVC in `cozy-public` after the import fails, so a picker built on the PVC list offered an image whose clone never completes. The `image` option source in cozystack-api now reads the `vm-default-images-*` DataVolumes next to the PVCs and drops names whose DataVolume is in phase `Failed`. I filter rather than mark them, because such an option can never be used and `OptionItem` has no disabled state. A golden that is still importing stays in the list, since CDI holds a clone until the source is populated, and so does a PVC with no DataVolume. When the DataVolumes cannot be read, the PVCs go out unfiltered. The Option list drops a source whose provider errors, and an empty picker would be worse than the old behaviour. cozystack-api gets a read grant on `cdi.kubevirt.io` DataVolumes for this. The console did not use that source at all. `SourceField` listed the PVCs itself, as the tenant, so the fix above would not reach the picker users see. It now reads the same Option list the other dynamic dropdowns use. With that, nothing in the console lists `cozy-public` PVCs, so I removed the read grant on them from the `cozy:tenant:dashboard` Role. That is a visible change: a tenant loses `kubectl get pvc --namespace cozy-public`, which was only ever granted for this picker in #2843. One limitation: in CDI v1.66.1, which cozystack ships, a golden import reaches `Failed` only on a checksum mismatch, and a broken URL keeps it in `ImportInProgress`. So this filter covers less than the issue assumed. I filed #4564 for that. Closes #4553 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> I have no screenshot. The picker looks the same; only the list behind it changes. `apps/console/src/components/SourceField.test.tsx` covers it: the picker shows the images the option source serves and not the raw PVC names. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(api): the VM image picker no longer offers golden images whose import failed. The console now reads images from the image option source, and tenants no longer get read access to PersistentVolumeClaims in cozy-public. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * The image picker now displays available images from the tenant’s image options. * **Bug Fixes** * Images whose imports have failed are excluded from the picker. Images remain available when import status is still in progress, has succeeded, or cannot be retrieved. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
86bf2ba8ec |
fix(kubernetes): let kubevirt-csi hot-unplug from an orphaned VMI (#4563)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does This lets the kubevirt-csi controller detach a volume from a VM instance that no VM owns. Right now that call is always denied. Closes #4452. The driver has a fallback for a volume that is still attached to the VMI but is no longer in the VM spec. It calls the VMI-level `removevolume` subresource. The `-kcsi` Role grants only the two VM-level subresources, so the call ends in 403. Nothing else reaches this service account with that grant. Only its own Role and ClusterRole name it as a subject, and the group bindings that cover every service account grant nothing on this subresource. At v1.9.0, `kubevirt.io:default` grants only `kubevirts`, `version` and `guestfs`. `kubevirt.io:edit` and `kubevirt.io:admin` do include the VMI subresource, but they aggregate only into the built-in `edit` and `admin` roles, and no tenant binding gives those to this account. The cozystack tenant roles aggregate on their own labels and don't list it. The fallback is still needed after the flag in #4453 goes to false. The flag only skips VMIs that a VM owns. When a VM is deleted with `propagationPolicy=Orphan`, the KubeVirt VM controller drops its finalizer and leaves the VMI running (`vm.go` at v1.9.0). The tenant node stays up with its volumes attached. When a pod moves off that node, the driver has to detach through the VMI, and on a 403 the hot-plug pod keeps holding the infra volume. Until #4453 lands, the fallback also runs for VM-owned VMIs and gets the same 403. The grant is `update` on `virtualmachineinstances/removevolume` and nothing more. The account can already remove a volume from any VM in the namespace, so it gets no new reach. virt-api also rejects the VMI-level call on a VM-owned VMI once `HotplugVolumes` is off. `virtualmachineinstances/addvolume` stays out because the driver only attaches through the VM. The new helm-unittest pins the Role's `subresources.kubevirt.io` rule to these three subresources with `update`. I haven't reproduced the 403 on a cluster. This shows it on a tenant namespace, before and after: ```bash kubectl auth can-i update virtualmachineinstances.subresources.kubevirt.io --subresource=removevolume \ --namespace <tenant-ns> --as system:serviceaccount:<tenant-ns>:<release>-kcsi ``` Upstream kubevirt/csi-driver has the same gap in `deploy/infra-cluster-service-account.yaml`. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(kubernetes): the kubevirt-csi controller can now hot-unplug a volume from a VMI that no VM owns, instead of failing with 403 Forbidden. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved Kubernetes CSI volume removal for virtual machine instances, including cases where a volume is not listed in the virtual machine specification. This helps the driver complete volume removal in scenarios where the Kubernetes virtualization API accepts the instance-level request. Existing permissions for adding and removing volumes from virtual machines remain unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
ec4d26df54 |
fix(kubernetes): accept a golden worker image that exists only as a PVC (#4578)
## What this PR does A worker pool on `osImage.builtin` refused a golden image that exists in `cozy-public` as a PVC without its DataVolume, and the whole KubernetesNodes release failed to render. CDI clones the PVC, not the DataVolume, so that golden is a valid source, and vm-disk already accepts it. This can happen once CDI garbage-collects DataVolumes (`dataVolumeTTLSeconds`) or when someone restores a golden PVC on its own. Now the render fails only when neither the DataVolume nor the PVC exists. The StorageClass and size checks already fall back to the PVC, so a PVC-only golden is still checked against the pool, and the new test suite pins that too. Closes #4565 ### Screenshots ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kubernetes): a worker pool on osImage.builtin now accepts a golden image that exists in cozy-public only as a PVC, matching vm-disk; it is refused only when neither the DataVolume nor the PVC exists. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Built-in golden images can now be used when available as either a DataVolume or a PVC. * **Bug Fixes** * Missing golden images now produce an error that identifies the missing image and suggests ways to resolve the issue. The error is suppressed when the pool already has a successful clone. * PVC-based golden images are checked against the pool’s storage class and minimum disk size requirements. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
f9f7562e56 |
fix(kamaji): resolve Gateway API availability once at startup (#4501)
## What this PR does Carries the fix proposed upstream in clastix/kamaji#1327 (not yet merged there; issue clastix/kamaji#1326) as a build patch on the 26.9.4-edge tag that cozystack builds Kamaji from. In Kamaji 26.9.4-edge, every TenantControlPlane reconcile checks whether the Gateway API is available. The discovery call ignores the reconcile context, so it is bounded only by the client's 32s default timeout, which is longer than the 30s reconcile budget. The watches, however, read availability only once, at startup. With the patch, Kamaji resolves availability once when the manager starts. The watches, the indexer, the reconciled resources and the admission webhook's Gateway API check all use that one value. A discovery error at startup now stops the manager, so it is no longer mistaken for "Gateway API not installed". A Gateway API installed after Kamaji starts needs a Kamaji restart. On this platform that case does not come up: the kamaji package depends on cert-manager, and cert-manager depends on gateway-api-crds. The webhook's separate TLSRoute check still runs discovery on each request, as before, and is listed as a follow-up in clastix/kamaji#1327. Kamaji's chart sets no `timeoutSeconds` on the webhook, so the apiserver's 10s default applies, and a discovery call that can take up to the client's 32s can make that admission time out. The patch is the upstream commit applied unchanged except for the dropped docs hunk, which is not part of the image. It applies cleanly to 26.9.4-edge with no offsets. It modifies different files than the existing `fix-kubelet-config-compat.diff`, so the two apply in either order. The upstream tests stay in the patch so that it matches the upstream commit; the image build does not run them. On the patched tree, `go build ./...` passes, and `go test -race` passes for `./controllers/...`, `./internal/webhook/...`, `./internal/utilities/...`, `./internal/kubeadm/...` and `./cmd/...`. I also checked that the new tests can fail: they go red when the availability check is inverted and when the old per-reconcile lookup is put back. If clastix/kamaji#1327 changes before it merges, this patch should be updated to match. It is dropped once Kamaji ships the change. Refs #4489. This does not close #4489, because this change does not explain the long stall reported there. ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kamaji): Kamaji now checks for the Gateway API once at startup instead of on every TenantControlPlane reconcile, so a slow discovery endpoint can no longer use up the reconcile timeout. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Gateway API availability is now checked at startup and consistently applied to Gateway-related setup and configuration validation. * Gateway configuration is rejected if the Gateway API was unavailable when Kamaji started. If you install the required CRDs afterward, restart Kamaji before configuring Gateway resources. * Discovery errors now stop startup rather than allowing Kamaji to continue with an uncertain Gateway API status. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
b0e07cadbb |
fix(operator): rebuild each changed ArtifactGenerator generation once (#4594)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does After an upgrade, an ArtifactGenerator can report Ready on its new spec while its artifacts were built from the old one, and source-watcher never rebuilds them. That is how the bucket-system chart in #4593 shipped without cozy-lib. The cause is upstream (fluxcd/source-watcher#384). source-watcher builds from the spec it read when the reconcile started, but the final status patch stamps Ready with the live generation. An upgrade changes the packages source and the copy rules within seconds, so a spec change often lands during a build. The generator status has nothing that tells a stale build from a good one. So the operator now rebuilds every changed generation once, after source-watcher first reports it Ready, through the force-drift path the operator already uses for fluxcd/pkg#934. An unchanged spec rebuilds to the same digest, so ExternalArtifacts and HelmReleases do not move. Generation 1 is skipped, since the race needs a spec change during a build. On the first rollout every generator past generation 1 gets one extra rebuild, and its PackageSource shows `Unknown/AwaitingSourceWatcherRecovery` until that rebuild lands. Nothing waits on that condition today. The same rebuild repairs clusters already stuck like #4593. The recovery messages on the PackageSource no longer blame fluxcd/pkg#934, because the same path now serves both cases. ### How it was tested Unit tests in `internal/operator/packagesource_reconciler_test.go` cover the forced rebuild on generation 2 and on a later generation, every case where no rebuild must happen, the rebuild settling without a second one, both failure orders (force fails, recording the generation fails), and a stale recovery counter. Each condition of the new predicate was mutated once to confirm a test fails. I also ran this commit's operator image on an arm64 test cluster running v1.7.0-alpha.3, the one where #4593 was found. Before that I had repaired its bucket generator by hand: deleting the stale ExternalArtifact and requesting a reconcile produced an artifact with `charts/cozy-lib` and both bucket HelmReleases went Ready. With the patched operator, the only generator past generation 1 got exactly one forced rebuild and the verified-generation annotation. source-watcher reported all its ExternalArtifacts up to date. The digests of all 214 ExternalArtifacts and the revisions of all 104 HelmReleases matched a snapshot taken before the swap, and every PackageSource and HelmRelease stayed Ready. The race itself was not reproduced on the cluster, since that needs a spec change landing during a build. Closes #4593 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(operator): rebuild an ArtifactGenerator once after each spec change, so an upgrade no longer leaves a component on artifacts built from the previous spec (for example a chart missing a library it now needs). ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved PackageSource recovery when an artifact generator reports ready: the system now verifies the generated output and triggers a rebuild when needed. * Failed rebuild or verification attempts can be retried during a later reconciliation, and recovery tracking is cleared once the rebuilt generation settles. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
50dec5d46a |
fix(platform): resume a retried migration Job from the stamped version (#4583)
## What this PR does A retried pod of the platform migration Job no longer runs migrations that already finished in the same upgrade. The Job gets `CURRENT_VERSION` once, at render time, and a failed pod is replaced by one with the same environment. By then every finished migration has stamped the `cozystack-version` ConfigMap, but the new pod ignored that and started from the rendered version again. `run-migrations.sh` now starts from the version stamped in the ConfigMap. If reading it fails, the pod fails and the Job retries it, because falling back to the rendered version is exactly the re-run this fixes. The rendered value is used only when the ConfigMap holds no integer version, which is also when the chart itself could not get a real version from it. Without that fallback a value like `v58` gave a green Job that ran nothing. The new `hack/run-migrations.bats` runs the script in the migrations image's alpine base against a fake kubectl that keeps the stamp in a file, so one pod's stamps are what the next pod reads. It needs docker, like the other migration suites. I also found that the ConfigMap existence check at the top of the script treats any API error as a missing ConfigMap and skips every pending migration. That is older than this change and is filed as #4579. Closes #4555 ### Screenshots ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(platform): a retried platform migration Job now resumes from the version stamped in the cozystack-version ConfigMap instead of re-running migrations that already finished in the same upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Migration startup now resumes from a valid stamped version, avoiding reruns of migrations that have already completed. * Empty or non-numeric stamps fall back to the rendered version. Out-of-range stamps or failed stamp reads stop migrations before they run. * If the migration ConfigMap is missing, migrations do not run and the target version is stamped. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
bb2946990d |
fix(vm-instance)!: refuse guest memory below KubeVirt's 1M floor (#4569)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The vm-instance resource guard now refuses guest memory below 1M, the same floor KubeVirt admission enforces. Before, a value such as `500m` or `1Ki` rendered and then failed when the VirtualMachine was applied. This is a breaking change. A memory above zero and below 1M rendered before and is now refused at render time. KubeVirt v1.9.0 would not admit such a VM either: `validateMemoryRequestsNegativeOrNull` refuses a non-zero request below 1M (`pkg/virt-api/webhooks/validating-webhook/admitters/vmi-create-admitter.go:1135`). The VirtualMachine admitter runs the same check on its template. The cpu, sockets and memory descriptions now list the values the guard refuses, so the schema states the same rule as the error. The error cannot quote an unquoted number as it was written, because YAML has decoded it before the chart sees it. A large one also printed in exponent form, so `4294967296` came out as `4.294967296e+09`. A string is still quoted exactly as written. A number is now printed in plain digits and marked "read as", which is the closest honest wording I found. Closes #4413 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(vm-instance)!: refuse `resources.memory` below 1M at render time, since KubeVirt does not admit a VM with less. A value above zero and below 1M rendered before and is now refused. The cpu, sockets and memory descriptions list the refused values, and the guard error marks an unquoted number as read rather than written. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * VM resource values are now checked when rendering: CPU cores and sockets must be positive whole numbers no greater than 4294967295. Unit-suffixed, fractional, zero, negative, and quoted values with leading zeros are rejected. * Memory values below `1M` are rejected, with clearer messages showing the value as entered or interpreted. * **Documentation** * Updated resource guidance to describe these limits and validation behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
ed20c53449 |
fix(vm-disk): refuse to clone a golden image that does not exist (#4567)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Stacked on #4392, so the base is `fix/vm-default-images-health-check`. Retarget to `main` once #4392 merges. A new vm-disk with `source.image` now fails to render when the golden image it names does not exist, and the error names the image and `cozy-public`. Before this the chart cloned `cozy-public/vm-default-images-<name>` without looking, and CDI waited for that PVC forever while the VMDisk stayed Ready. That happens on a typo, and on every cluster where the opt-in `cozystack.vm-default-images` package is off, so the message also says to add it to `bundles.enabledPackages`. The golden counts as missing only when neither its DataVolume nor its PVC exists. CDI clones the PVC, so a PVC with no DataVolume is still a usable source, and a DataVolume whose PVC is not created yet is an import about to start. Under `helm template` every lookup is empty, so the check runs only when the release namespace itself is visible. The same trick is already used in the kubernetes and seaweedfs charts. Existing disks are exempt the same way #4392 exempts them, because their DataVolume spec is immutable. A disk refused while the catalog is off renders by itself once the golden appears, since app HelmReleases retry on failure. Tests are in `packages/apps/vm-disk/tests/dv_golden_image_test.yaml`: refusal of a missing golden, a PVC without a DataVolume, a DataVolume before its PVC, rendering with no cluster, and an existing disk whose golden was removed. Closes #4548 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(vm-disk): a new disk with source.image now fails with a clear error when the golden image does not exist in cozy-public, instead of waiting forever for a clone that never starts. ``` |
||
|
|
c7b8009964 |
fix(kubernetes): log kubevirt-csi temp NFS mount dir removal failure (#4562)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The kubevirt-csi node plugin now logs a warning when it cannot remove the temp dir it uses for NFS publish. Before, the error was dropped, so a dir left behind after a failed unmount was never reported. The publish path temp-mounts the export root to create `/data`, then removes the temp dir in a defer. The failure is logged, the same way the unmount failure next to it already is, and not returned. A leftover temp dir does not break the volume, and failing a publish that otherwise worked only makes kubelet retry it. A new unit test publishes an NFS volume through the fake mounter, where the removal always fails, and checks that a warning naming the dir is logged. This is independent of #4451. That PR touches the same Go module but not `node.go`. A failed unmount still leaves the NFS root mounted. That is older than this change and is tracked in #4561. Closes #4466 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(kubernetes): the kubevirt-csi node plugin now logs a warning when it cannot remove its temporary NFS mount dir instead of dropping the error ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Failed cleanup of a temporary mount directory now produces a warning with the directory path, making the issue easier to identify. The volume publish operation continues to succeed when cleanup fails. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2ddd55ce5b |
fix(build): honour PUSH and LOAD from the environment and in image-talos (#4654)
## What this PR does A local build that asks to load images into docker no longer pushes them. `hack/common-envs.mk` set `PUSH` and `LOAD` with plain `:=`, and a makefile assignment beats the environment, so `LOAD=1 PUSH=0 make image` quietly pushed a multi-arch build. Both now take the environment value, and an empty one falls back to the default. I also made `PUSH` default to 0 when `LOAD=1`, so `LOAD=1` alone only loads. Before this, `make image LOAD=1` pushed and loaded at once. An explicit `PUSH` from the environment or the command line still wins, and `OCI_EXPORT_DIR` still forces both off. Every buildx target gets the values through `BUILDX_ARGS`, so the fix lives in that one file. The Talos installer was the one image that ignored `PUSH`. It is published with skopeo, not buildx, so it never saw `--push=$(PUSH)`. Its registry copies now run only when `PUSH=1`. A build that pushes nothing no longer writes `PUSHED_TAGS_LOG`. That log tells the multi-arch stitch which tags to move, so it must hold only tags that were really pushed. The cache helper lost an argument nobody used. No caller passes the second argument of `cache-args`, and an explicit tag there would bypass `CACHE_TAG`, so the amd64 and arm64 builds of that image would share one cache ref. CI sets neither `PUSH` nor `LOAD`, so CI builds do the same thing as before. Each fix has a `make --dry-run` test in `hack/common-envs_test.bats`. Closes #4499 Closes #4504 Closes #4551 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The ccp skills pass `PUSH=1` explicitly, which still works. The website docs do not mention `PUSH`, `LOAD` or `cache-args`. ### Release note ```release-note fix(build): `LOAD` and `PUSH` set in the environment are now honoured, `LOAD=1` no longer pushes unless `PUSH=1` is also set, and the Talos installer is no longer published when `PUSH=0`. ``` |
||
|
|
2af8b0c229 |
ci(build): fall back to direct, retry and cache Go module downloads (#4660)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does One transient error from proxy.golang.org is enough to fail an image build or the `Unit & controller tests` job today. The error is an HTTP/2 stream reset (`stream error ... INTERNAL_ERROR`), and nothing retries or falls back after it. This PR adds three things: a direct fallback, a bounded retry and a module cache. Image builds now fall back to fetching modules from their origin when the proxy fails. They get `GOPROXY=https://proxy.golang.org|direct` as a build arg from `hack/common-envs.mk`, and every Go builder stage declares `ARG GOPROXY`. With a pipe Go goes direct on any proxy error. With Go's default comma it does that only on a 404 or 410. The value comes from `IMAGE_GOPROXY` and not from the host's `GOPROXY`, because buildkit can't reach a mirror on localhost or a `file://` proxy. Every workflow job that runs Go on the runner sets the same value as env. Explicit module downloads in Dockerfiles are retried three times with a growing pause. The loop is inline, in one form that the contract test pins. I tried a shared script from a named build context first, but buildx refuses any `--build-context` for the four Dockerfiles still pinned to the `dockerfile:1.2` frontend. The images also build from different contexts, so no single file can be copied in. Raising those pins changes how those images build and needs a real build to check, so I left it for a follow-up. Two alpine stages, token-proxy and shadowbox's `ss-server`, left the download to `go build`. The alpine golang images have no git, so the direct fallback can't fetch for them, and they now get the same download step. The runner jobs use `hack/go-mod-download.sh` with the same bound. The unit test job now starts from a module cache instead of downloading everything. It restores `GOMODCACHE` with `actions/cache`, keyed on every `go.sum` in the tree, and then prefetches all tracked modules through the retry, so a cache miss still works. A cache saved by a PR run is visible only to that PR, but one saved on the default branch can be restored by any PR ([docs](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache)). That is why the build cache warmer on main writes the same key before its image build. This way the first run of a PR, which is the one that flakes, starts warm. The warmer restores only an exact key, so old module versions don't pile up, and its Go steps can't fail the image build. Both runner labels are Linux, so the key matches. Both also check out under `/home/ubuntu/_work`, so `go env GOMODCACHE` should give the same cache path, but I have not seen it on the warmer's runner yet. The release jobs in `tags.yaml` and `promote-rc.yaml` get the GOPROXY env only, no cache. `hack/go-module-fetch-contract.bats` covers all three. Each test is red on main, and I checked each guard with a mutation. Closes #4591 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ci(build): Go module downloads in image builds and CI jobs now fall back to direct fetching and retry on transient proxy.golang.org errors. The unit test job reuses a module cache written from main. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Reliability** - Go module downloads now fall back to direct access when the public proxy encounters errors. - Builds and automated checks retry failed module downloads, helping them recover from temporary network issues. - **Performance** - CI caches Go modules to reduce repeated downloads and speed up builds and tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
9c025fa2b1 |
fix(installer): skip the packages artifact push and stamp with PUSH=0 (#4663)
## What this PR does A local build with `PUSH=0` no longer publishes the packages artifact. Before this change, `make image-packages` in `packages/core/installer` ran `flux push artifact` no matter what `PUSH` was set to, and then wrote the pushed digest into `values.yaml`. The `chart` target in the same Makefile already respects `PUSH`. With `PUSH` set to anything other than 1, the target now skips both the push and the digest stamp. It prints one line saying so and leaves the committed `platformSourceUrl` and `platformSourceRef` as they are. It still writes `packages/core/platform/.build-revision`, which is gitignored. I chose to skip the stamp instead of computing a local digest, because every reader of that value needs a digest that exists in the registry. The chart passes `platformSourceRef` to the operator as `--platform-source-ref`, and Flux then pulls `cozystack-packages@<digest>` from the registry. The release scripts `verify-promoted-packages.sh`, `promote-packages-artifact.sh`, `stitch-multiarch.sh` and `nightly-mirror.sh` also treat it as a registry digest. A digest of an artifact that was never pushed points at nothing. On main an unpushed run would not even get that far, since the recipe dies at `test -n "$DIGEST"`. The gate is a make-level `ifeq ($(PUSH),1)`, so it reads whatever `PUSH` ends up as after `hack/common-envs.mk`. On main that means `PUSH=0` on the command line. I also applied the `common-envs.mk` change from #4654 locally, which takes `PUSH` from the environment and makes `LOAD=1` default it to 0. With that applied, a dry run shows no push and no stamp for `PUSH=0` in the environment, for `LOAD=1`, and for `PUSH=0` on the command line, while `LOAD=1 PUSH=1` and an empty `PUSH` still push. CI is not affected, because no workflow sets `PUSH` or `LOAD` and `PUSH` stays at 1. I dry-ran the three CI invocations of this Makefile before and after the change, on main and again with #4654 applied: - `tags.yaml`, release stitch step: `make -C packages/core/installer image-packages chart` with `REGISTRY`, a release tag as `IMAGE_TAG`, and `PUSHED_TAGS_LOG` - `pull-requests.yaml`, installer step of a same-repo PR: `make -C packages/core/installer image` with a `pr-N-sha` tag, `PLATFORM=linux/amd64`, `BUILDER`, and an empty `OCI_EXPORT_DIR` - `build-main.yaml` and `build-release.yaml`: `make build`, which runs the installer `image` target with defaults ```text $ diff --recursive before/ after/ # tags.txt pr.txt build.txt $ echo $? 0 ``` There is one side effect. `OCI_EXPORT_DIR` forces `PUSH := 0`, so running `image-packages` with it set no longer pushes, where before it would have tried an anonymous push. No CI path does that. The fork PR build already runs only `pre-checks image-operator` for the installer, and the `make -n ... image OCI_EXPORT_DIR=/oci` scan in `e2e-fork.yaml` greps only the `.oci.tar` names, which this change does not touch. The new test dry-runs the target in both modes. With `PUSH=0` there must be no `flux push artifact` and no `platformSourceRef` stamp; with `PUSH` at its default, both must be present. The first test fails on main. I mutated the guard three ways, and each time only the matching test went red: - condition forced true: the `PUSH=0` test fails - compared against 2: the default-path test fails - `endif` moved up so only the push is gated: the `PUSH=0` test fails on the stamp check One limitation is left as a follow-up. The test's `env -u PUSH -u LOAD` only isolates it once #4654 lands, because on main a file assignment wins over the environment anyway. It does not cover `PUSH=0` coming in through `MAKEFLAGS`, so `make bats-unit-tests PUSH=0` would fail the default-path test here and the ordering check in `hack/stitch-multiarch_test.bats`. Nothing runs the suite that way today. If that changes, passing an explicit `PUSH=1` on the default-path `make` lines, or unsetting `MAKEFLAGS` the way `hack/bats-unit-keep-going.bats` does, would fix it. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The website's `development.md` tells developers to run `make image-packages` with `REGISTRY` set to their own registry. That is the default `PUSH=1` path, and it works as before. Neither the website nor ccp mentions `PUSH=0` for this target. Closes #4656 ### Release note ```release-note fix(installer): `make image-packages` no longer pushes the cozystack-packages artifact or stamps its digest into values.yaml when PUSH=0. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * You can run `image-packages` with `PUSH=0` to skip publishing the artifact and keep the committed platform source reference unchanged. * When pushing is enabled, the command publishes the artifact and updates the platform source URL and digest-based reference. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
33cf6972bb |
test(qdrant): exercise the tenant trust anchor end to end (#4701)
## What this PR does The qdrant e2e suite now runs with TLS on and checks the tenant trust anchor end to end. Before this, the fixture left `tls` unset with `external: false`, so the chart rendered no CA and no `TenantProjection`, and nothing in e2e reached the controller or `tenantsecrets`. Setting `tls.enabled: true` turns TLS on without the LoadBalancer that `external: true` would allocate. The first new step checks that the anchor is published. The `qdrant-test-ca` sentinel has to report `Ready=True/Projected`, and `qdrant-test.tenant-ca` has to carry both the `tenant-ca` and `tenantresource=true` labels. The projection must hold only `ca.crt` with the same fingerprint as the chart's CA, and `tenantsecrets` must serve it. The key-bearing `qdrant-test-ca` and `qdrant-test-tls` must answer NotFound there, and any other error fails the step. The second step checks that the anchor is what decides trust. It reads the anchor through `tenantsecrets`, port-forwards the qdrant Service and runs `openssl s_client` with that file as the only CA, hostname check included. A CA with the same subject and another key must fail, and so must the runner's default trust store. A handshake failure counts only when the output shows a verification error, so a refused connection does not pass as a rejection. Plaintext HTTP must get no response. Curl exits 7 and 28 fail the step, and a second verified handshake afterwards proves the forward was still alive. TLS does not break the existing steps, because none of them talk to qdrant. `create-qdrant`, `wait-helmreleases-ready`, `verify-statefulset-and-storage` and `verify-pvc-reclaim-on-delete` only read Kubernetes objects: no `http://`, no port 6333 or 6334, no curl or exec, and the suite has no backup round trip. The release now also renders two Issuers, two Certificates and the sentinel. The chart switches its probes to HTTPS with `enable_tls`, so `readyReplicas: 1` now covers a pod serving TLS. Local checks: `chainsaw lint test` (v0.2.15) says the file is valid. `helm template` with the fixture's spec renders 2 Certificates, 2 Issuers and 1 TenantProjection, and none without TLS. I ran the handshake logic against a local `openssl s_server` with a CA and leaf shaped like the chart's. The anchor passes, the wrong CA and the default store fail on verification, a wrong hostname fails, plaintext gets `000`, and a closed port is not counted as a rejection. A sentinel with a missing source (`SourceNotFound`) is not tested here, the reconciler unit tests cover it. Review notes I left as they are: - Through `kubectl port-forward` the local socket is always accepted, so curl exits 7 and 28 only catch a dead forward process. A false pass would need qdrant to answer plaintext and a pod-side blip on that one request at the same time. - qdrant may reset the socket on the plaintext request, and some kubectl versions drop the whole port-forward on a pod-side error. Then the handshake after it turns the step red, not green. If CI shows `lost connection to pod` there, the plaintext check should get its own port-forward. - The suite's `catch` has no `describe` of the `TenantProjection`. The sentinel assert prints its conditions when it fails, but a red Secret assert or script after it would not. - The vendored chart's `servicemonitor.yaml` hardcodes `scheme: http`, so metrics scraping fails for any qdrant with TLS on. That was already the case before this PR, and the suite does not check metrics. Tracked in #4703. cert-manager does not set owner references here, so `qdrant-test-ca` and `qdrant-test-tls` stay in the namespace with their private keys after the app is deleted. That is tracked in #4704. The nats half of #3706 is tracked in #4702. Closes #3706 ### Screenshots No UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only `hack/e2e-chainsaw/qdrant/`, which no entry in the trigger map covers. ### Release note ```release-note test(qdrant): e2e now runs qdrant with TLS on and checks that a client trusting only the tenant-published CA reaches it, while a wrong or missing anchor and plaintext HTTP are refused. ``` |
||
|
|
7ea71042b7 |
ci(e2e): add a daily isp-hosted install lane on kind (#4712)
## What this PR does
This adds a daily CI lane that installs the `isp-hosted` platform
variant through the `hosted` installer variant. No lane ran either of
them before. The PR, nightly and tag lanes all install `isp-full` on
Talos, where Cozystack brings its own CNI and storage, and that is how
#3376 shipped unnoticed.
What `isp-hosted` changes is what it leaves to the host. The system
bundle swaps kube-ovn and Cilium for a `noop` networking Package and
drops multus, LINSTOR, MetalLB, cozy-proxy, snapshot-controller and the
rest of the shared data-plane set. The iaas bundle refuses to render.
Migrations are not a difference, `isp-full` enables them too.
So the lane runs on kind. kindnet, kube-proxy and local-path come from
the cluster, and there is no Cilium, no LINSTOR and no `replicated` or
`local` StorageClass. It installs the latest GHCR nightly instead of
building anything, at 05:00 UTC after the nightly mirror publishes, and
on `workflow_dispatch`. There is no pull request trigger, because a run
would install the nightly images and not the PR's.
A run passes when every HelmRelease is Ready, `cozystack-platform` and
`tenant-root` included. It also checks that the platform emitted the
`noop` networking Package and none of LINSTOR, MetalLB, multus or
KubeVirt, and that the cluster has no Cilium CRD and no `replicated` or
`local` class.
The lane reuses the e2e sandbox container for tooling and
`collect-report`, attached to the kind network.
`hack/isp-hosted-lane_test.bats` pins the wiring: the Package selects
`isp-hosted`, the install uses `variant=hosted`, the kind DNS domain
matches `networking.clusterDomain`, and kind is pinned by checksum and
digest. Six of its seven tests were red before the implementation
existed.
`hack/sandbox-runner-headroom.bats` gets a named exemption. That check
treats any workflow touching `packages/core/testing` as a lane that
boots the QEMU guests, and this one boots none. The exemption is checked
after the job match, and removing it turns the test red again.
The pins come from the kind v0.32.0 release: the binary sha256
`50030de2…` from its `kind-linux-amd64.sha256sum`, and
`kindest/node:v1.33.12@sha256:3f5c8443…` from its release notes.
v1.33.12 is the Kubernetes version the Talos lanes use, so the variant
is the only thing that differs. All three are plain `env` values in the
workflow, where a Renovate custom manager can match them.
### Real runs
The first real run on this branch,
[36961209755](https://github.com/cozystack/cozystack/actions/runs/36961209755),
went red on `tenant-root`:
```text
tenant-root False Helm install failed for release tenant-root/tenant-root with chart tenant@0.0.0+629b61300902: unable to build kubernetes objects from release manifest: resource mapping not found for name: "allow-external-communication" namespace: "tenant-root" from "": no matches for kind "CiliumNetworkPolicy" in version "cilium.io/v2"
```
This is #3387 reproduced live: `packages/apps/tenant` renders Cilium
policies on a cluster without Cilium. The other releases went Ready,
except `cozystack-basics`, which was still in an upgrade at the last
listing. The fix landed in #4710.
The run after it,
[37089699259](https://github.com/cozystack/cozystack/actions/runs/37089699259),
is green on the first attempt. It installed nightly
`0.0.0-nightly.20261003.37089458387.1`, built from
|
||
|
|
14eab3cfe3 |
fix(backups): restrict the kafka-metadata CLI Pod and correct demo personas (#4695)
## What this PR does
This fixes two problems in the Kafka and NATS backup demos under
`examples/backups/`.
The Kafka metadata demo ran its CLI Pod without any securityContext, so
a namespace that enforces the restricted Pod Security level would reject
it. The Pod now gets the same restricted context as the CLI Pod in the
topic-data demo: `runAsNonRoot`, the `RuntimeDefault` seccomp profile,
no privilege escalation, all capabilities dropped. The new bats case
reads the overrides passed to `kubectl run` and fails if any of those
fields is missing or the patch is not sent as `--override-type=json`. It
is red on main, and it goes red on each of seven single-field mutations
of the patch while the other five cases stay green.
One known limit: `runAsNonRoot` needs an image that runs as a non-root
user. The default Strimzi image runs as UID 1001 and is fine. If
`KAFKA_IMAGE` is overridden with an image that runs as root, the kubelet
refuses to start the container, and the helper fails only when its 5
minute `kubectl wait` times out.
The kafka and nats walkthroughs told a tenant to run scripts that the
tenant roles forbid. Steps 03, 04, 06, 07 and `cleanup.sh` were in the
tenant column, and the scenario docs and several script comments had the
tenant create the backup Secret. Those rows now say admin and name the
part a tenant admin can do on its own, and the scenario docs say the
same. After this change the persona column in both tables matches the
tenant roles. kafka-metadata has no persona table, so I didn't change
anything there.
These are the tenant rules on pods and backup objects, plus one example
of the per-app secret rules. Each app chart's dashboard Role allows
`get`, `list` and `watch` on a few named Secrets and no other verb on
secrets. Nothing aggregated into the tenant roles grants `pods/exec`,
`pods/log`, `helmreleases`, `objectstorage.k8s.io`, `statefulsets` or
`kafkas.kafka.strimzi.io`.
```yaml
# packages/system/cozystack-basics/templates/clusterroles.yaml:100-112 (cozy:tenant:view:base, every tenant level)
- apiGroups:
- ""
resources:
- pods
- services
- persistentvolumes
- endpoints
- events
- resourcequotas
verbs:
- get
- list
- watch
# packages/system/cozystack-basics/templates/clusterroles.yaml:238-246 (cozy:tenant:admin:base)
- apiGroups: [""]
resources:
- pods
- services
- persistentvolumes
- endpoints
- events
verbs:
- delete
# packages/system/backup-controller/templates/tenant-clusterroles.yaml:35-44 (cozy:backups:admin)
- apiGroups: ["backups.cozystack.io"]
resources:
- plans
- backupjobs
- restorejobs
verbs:
- create
- update
- patch
- delete
# packages/apps/bucket/templates/dashboard-resourcemap.yaml:6-14 (bound at the tenant "use" level)
- apiGroups:
- ""
resources:
- secrets
resourceNames:
{{- range $name, $user := .Values.users }}
- {{ $.Release.Name }}-{{ $name }}-credentials
{{- end }}
verbs: ["get", "list", "watch"]
```
How each changed row runs into those rules:
| Row | What the script needs | Why a tenant can't |
| --- | --- | --- |
| 03 (kafka, nats) | waits on the `HelmRelease` and COSI objects, `get`
on the `bucket-<name>-backup` Secret, kafka also reads the CA Secret in
`tenant-root` | no `helmreleases` or `objectstorage.k8s.io` rule; the
bucket Role covers only `<release>-<user>-credentials` |
| 04 (kafka, nats) | waits on the `HelmRelease` and the Strimzi `Kafka`
or the `StatefulSet`, `create` on `secrets`, `create` on `pods`,
`pods/exec` | pods are `get`/`list`/`watch`/`delete` only, and nothing
grants `create` on secrets |
| 06 (kafka, nats) | `create` on `pods` and `pods/exec` for the CLI Pod,
kafka also `get` on `pods/log` for the refused restore | same pod rules,
and nothing grants `pods/log` |
| 07 (kafka, nats) | same as 04 | same as 04 |
| `cleanup.sh` (kafka, nats) | `delete` on `secrets`, on `backups`, and
on the cluster-scoped `BackupClass` and strategy | `cozy:backups:admin`
deletes only plans, backupjobs and restorejobs, and no tenant rule
covers secrets or cluster-scoped objects |
Step 05 stays a tenant step. Its `BackupJob` comes from
`cozy:backups:admin`, so in practice that means a tenant admin. The nats
demo's CLI Pod has the same missing securityContext as the
kafka-metadata one, and I left it for a separate issue.
Closes #4625
Closes #4636
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
Not a UI change.
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix(backups): the kafka-metadata backup demo now runs its CLI Pod with the restricted Pod Security context, and the kafka and nats backup demos now list steps 03, 04, 06, 07 and cleanup as admin steps, because they need Pod, exec, Secret and cluster-scoped permissions that tenant roles do not grant.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Security**
* Kafka metadata backup CLI Pods are recreated with restricted security
settings when they are not running or lack the required non-root
configuration.
* **Documentation**
* Kafka and NATS guides clarify which setup, restore, and cleanup steps
require administrator access and which job actions tenants can perform.
* Updated examples clarify that the demo provides required storage
Secrets and certificates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
1621c471f2 |
ci(dashboard): run eslint on the console and clear its errors (#4692)
## What this PR does
The console now passes its own eslint config, and CI runs it. On main
`pnpm lint` exits 1 with 54 problems, 51 of them errors, and
`ui-test.yaml` never ran it. On this branch it exits 0 with no errors,
and a Lint step sits between Typecheck and Test.
How each rule was handled:
- `no-explicit-any` (38 errors): replaced with real types. These commits
build a vite bundle byte-identical to main's.
- `react-refresh/only-export-components` (6 errors): the hooks, context
objects, `resolveSource` and the RJSF registry moved out of component
files into their own modules. `useTenantContext` stays in
`tenant-context.tsx`, so its importers and the tests that mock that
module are untouched.
- `no-unused-vars` (1 error): dropped the unused `ad` binding in
`OverviewTab`.
- `react-hooks/set-state-in-effect` (6 errors): relaxed per line with a
reason, not in the config.
The last rule is the only relaxation. Each of these effects re-syncs
local state to something that changes outside the component. The rule
wants the state derived during render, which changes when the value is
committed or what the first render shows, so it would be a behaviour
change. Per file:
- `components/ResourceQuotasField.tsx` re-syncs the inputs after the
form value is reset from outside.
- `components/SourceWidget.tsx` re-syncs the selected option when the
value loads asynchronously in edit forms.
- `components/command-palette/command-palette.tsx` resets query and
level whenever the palette closes.
- `components/command-palette/use-keyboard-nav.ts` moves the highlight
back to the top when the item list changes.
- `hooks/use-is-mac.ts` would return `true` on the first render on macOS
with a lazy initial state. Today it returns `false`.
- `lib/tenant-provider.tsx` falls back to an existing tenant once the
tenant list has loaded.
The eslint config itself is unchanged, and `no-explicit-any` stays on
everywhere.
`pnpm typecheck` and `pnpm test` (442 tests) pass at every commit. After
the file moves the bundle has the same chunk set, and only the entry
chunk differs, by reordered and renamed minified functions.
Three `react-hooks/exhaustive-deps` warnings from main are left as they
are, because fixing them changes when the effects run. One is in
`SourceWidget.tsx`, where the effect reads `options` but depends only on
`value`. The other two come from `ads = adList?.items || []` in
`use-command-items.tsx`, which gives two `useMemo` hooks a new
dependency on every render.
Things I did not do here:
- The step has no `--max-warnings`, so a new warning will not fail CI.
Once the three warnings are fixed, `--max-warnings 0` can go in.
- `SourceField.tsx` and `SourceWidget.tsx` each declare the same small
`SourceOptionSchema` interface. I kept them local since the two widgets
share nothing else.
- The job name stays `Typecheck and test`, so a required check with that
name keeps matching.
- `pnpm -r lint` only reaches `apps/console`.
`packages/{k8s-client,ui,types}` have no `lint` script, same as on main.
Closes #3911
Closes #4566
### Screenshots
No visual change: types, module layout and CI only.
### Downstream repositories
The diff touches only the console workspace and `ui-test.yaml`. The
console has no downstream copy since `cozystack-ui` was archived, and no
trigger map entry matches.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
ci(dashboard): the console's eslint now runs in the UI test job, and the console passes it with no errors.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Tenant selection now uses available tenant namespaces, restores a
saved selection when possible, and falls back to the root tenant or the
first available tenant.
* Tenant options are sorted, and the selected tenant is kept in sync
with the available choices.
* **Chores**
* Automated checks now include linting alongside type checks and unit
tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
9239555e6a |
ci(workflows): pin flux, yq and helm-unittest installs (#4645)
## What this PR does Workflows no longer run the flux install script as root. Every step that needed the flux CLI downloaded `https://fluxcd.io/install.sh` and ran it with `sudo`, so whatever that host served at the moment ran on runners that later get registry credentials. PR #4442 pinned the version, but the script was still fetched and executed. All 12 of those steps now use `fluxcd/flux2/action`, pinned to the v2.8.6 commit with `version: "2.8.6"`. The action downloads the release tarball, checks it against that release's checksums file, and adds the binary to `PATH`. Two workflows still took yq from the floating latest release, nightly and the fork e2e lane. They now use the same install the promotion steps already have: yq v4.53.3 checked against a committed SHA-256. I verified that hash against the release's `checksums` file and against the binary itself. The helm-unittest plugin was installed at whatever version was newest. The plugin embeds its own Helm, so a plugin release could change chart test results on a PR that touched no chart. It's pinned to v1.1.2 now, the version CI already runs. A runner that carries a different version gets it replaced. A new contract test scans every workflow for these three tools, because an unpinned install usually comes back by copying an older step. On main it fails on the install script, the missing setup action, the latest-release yq and the unpinned plugin. I ran one mutation per guard, and each one failed only the test meant to catch it. The existing contract tests that checked `FLUX_VERSION` inside the old steps now check for the action step. The e2e sandbox image still runs `install.sh` in its Dockerfile. That's an image build, not a workflow step, so I left it for a separate change. Closes #3619 Closes #3640 Closes #4510 ### Screenshots Not applicable, no UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches only workflow tool-install steps and `hack/*.bats` contract tests. Nothing in the trigger map reaches them. ### Release note ```release-note ci(workflows): install flux through the commit-pinned upstream setup action instead of running fluxcd.io/install.sh as root, pin yq with checksum verification in the nightly and fork e2e workflows, and pin the helm-unittest plugin version. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Reliability** * CI workflows now use a consistent, pinned Flux version. The yq and Helm plugin setup also verifies or enforces specific versions. * **Tests** * Added checks to ensure workflow tool versions are pinned and consistent, and that downloaded yq releases are checksum-verified. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
40638429bd |
fix(release): collect refs stored as registry/image/tag/digest keys (#4697)
## What this PR does The shared image-ref enumeration skipped refs stored the way upstream ingress-nginx stores them: `registry`, `image`, `tag` and `digest` as separate keys, with the repository in `image` and a digest that has no `@`. Two refs in the tree use this layout, the ingress-nginx controller and its default backend, both in `packages/system/ingress-nginx/values.yaml`. Every tool that reads refs through `hack/lib/image-refs.sh` missed both of them: the promote retag, the nightly mirror, the multi-arch gate, the stitch step and the promoted-packages verifier. I added a sixth YAML shape that joins `<registry>/<image>@<digest>`. It fires only on a complete `sha256:` digest, so upstream's unpinned default (`digest: ""`) and image maps without a digest emit nothing. It checks that `digest` is a string before `test()`, same as shape 3 does. The library header and `docs/agents/image-refs.md` now say "the YAML sub-shapes listed" instead of a number, so the next shape doesn't make the count wrong again. The real tree now gives 93 refs instead of 91, and the two new ones are exactly the ingress-nginx pair. The retag and mirror output does not change today, because both refs sit outside the owned registry and the ownership filters drop them. Both digests are multi-arch indexes, so `hack/verify-multiarch.sh --refs-file` with just those two passes and the release gate stays green. To show the consumers really pick the shape up, I ran both dry-runs on a copy of the tree with the controller moved under the build registry, the host being the whole `registry` value. In that copy the controller is the only build-registry ref. Before: ```text $ REGISTRY=iad.ocir.io/idyksih5sir9/cozystack hack/promote-retag.sh 1.99.0 --dry-run No cozystack-owned digest-pinned image refs found under iad.ocir.io/idyksih5sir9/cozystack/ — is this the rc's baked tree? exit 1 $ hack/nightly-mirror.sh 1.99.0 <tree>/packages --dry-run No cozystack-owned digest-pinned image refs found under iad.ocir.io/idyksih5sir9/cozystack/ in '<tree>/packages' — is this the baked main tree? exit 1 ``` After: ```text $ REGISTRY=iad.ocir.io/idyksih5sir9/cozystack hack/promote-retag.sh 1.99.0 --dry-run ▸ iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller sha256:a8ddcee59b63f6cbf93f4d3c9ffa1d1dda3c1d39497748d4484655356c68b3f6 DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller@sha256:a8ddcee5… docker://iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller:1.99.0 exit 0 $ hack/nightly-mirror.sh 1.99.0 <tree>/packages --dry-run ▸ iad.ocir.io/idyksih5sir9/cozystack/ingress-nginx-controller -> ghcr.io/cozystack/cozystack/ingress-nginx-controller sha256:a8ddcee5… DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/…/ingress-nginx-controller@sha256:a8ddcee5… docker://ghcr.io/cozystack/cozystack/ingress-nginx-controller:1.99.0 DRY-RUN skopeo copy --multi-arch all docker://iad.ocir.io/…/ingress-nginx-controller@sha256:a8ddcee5… docker://ghcr.io/cozystack/cozystack/ingress-nginx-controller:nightly exit 0 ``` The red test is a new case in `collect_refs_from_file understands every YAML shape`. It has a fixture in the ingress-nginx layout, with a non-string digest in front of it, and three near-misses that must emit nothing: no digest, an empty digest, and a digest without `image`. On main the positive case fails with `got: <nothing>`. Removing the rule, the registry join, the full-digest check, or the type guard on `image` or on `digest` turns that test red, and only that one. The bats suites of all library consumers pass with both the old and the new library. Two gaps remain in the tests and I left them out on purpose. No fixture covers the new rule without a `registry` key, and none pins its type guard on `registry`. Shapes 2 and 3 have the same unpinned `registry` guard on main. The overlay script has no such gap. `hack/overlay-main-images.sh` does not use the library, and its line classifier already accepts `image:`, `registry:`, `tag:` and `digest:` lines. Closes #4518 ### Screenshots Not a UI change. ### Downstream repositories I walked the trigger map against the diff. None of the changed files are in it. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(release): the image-ref enumeration used by promotion, the nightly mirror and the multi-arch gate now reads refs stored as separate registry/image/tag/digest keys, the layout of the ingress-nginx controller and default backend. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Image references can now be read from configurations that store the image name and SHA-256 digest separately, with an optional registry. * **Bug Fixes** * Incomplete or invalid image and digest values are ignored rather than producing malformed references. * **Documentation** * Documented the additional image storage format. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
9196b9798e |
fix(e2e): bound the dataplane capture collector by wall clock (#4640)
## What this PR does The e2e data-plane collector limited itself by counting pods and LoadBalancers, but both of its callers stop it by the clock. `hack/cozytest.sh` gives it 600s and the Chainsaw catch gives it 300s. One affected pod could already run past the smaller limit, and the LoadBalancer capture, which runs last, was the part that got lost. I moved the collector to a wall-clock budget and fixed the other issues filed against it. - Every kubectl call now goes through `dp_run`. It cuts the call's timeout to what is left of `COZY_DATAPLANE_BUDGET` and starts nothing once the budget is spent. The default is 270s, and `cozytest.sh` passes 570s. The pod section stops 90s early, so the LoadBalancer section always gets time to start. A call the deadline cut says so, a call cut by its own bound names that bound, and a refused call says it was not started and names its section; exec blocks carry that line inside the block. The node capture and the LoadBalancer probe used to blame an unanswered lookup or a missing cni-server when the budget stopped a call. Now they name the budget (#3671). - A static test fails on any kubectl call that bypasses `dp_run` and lists the lines. It matches by command position, so a call inside a double-quoted `$(...)` is still checked (#3694). - Per-pod and per-LoadBalancer capture files join namespace and name with `_`, which neither can contain. A second run into the same directory appends, with the run separator `capture-notes.txt` already uses (#3679). - `capture_node` and `capture_lb_node` no longer write their callers' loop variables. A static test fails when a function assigns a name that a `read -r` loop or another scope owns (#3675). - The tests that shrank every timeout to one second now cut only the read under test. A `timeout` shim decides by argv, so a slow machine can't cut a read the test needs answered (#3685). - `cilium-dbg bpf lb list` runs unfiltered in every affected node's capture, not only for a failing LoadBalancer (#3689). The call-site comment in `.chainsaw.yaml` and the collector paragraph in `docs/agents/e2e-testing.md` now match the code. Every behaviour change has a test in `hack/capture-dataplane.bats` that failed on the code before it. Two parts of the issues live elsewhere. The runner stopping a bats file at its first failing test is #4620, and a conntrack read filtered by service VIP is #4623. From #4629 I fixed the items on lines this PR already changes: the node capture now gets the run separator, a failed speaker log read writes a note, the endpoint tcpdump says when it falls back to the geneve interface, and the per-read timeout knobs are validated (a zero there would otherwise look like a spent budget). The rest of #4629 stays open: `ovn-lflows.txt` and `lb-speaker-logs.txt` are still replaced on each run, and two comments still describe older behaviour. - Closes #3694 - Closes #3671 - Closes #3679 - Closes #3675 - Closes #3685 - Closes #3689 - Refs #4629 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(e2e): bound the failure-path data-plane collector by a wall-clock budget, keep its per-pod and per-LoadBalancer captures from overwriting each other, and capture the cilium service map for ClusterIP failures. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Data-plane diagnostics now follow a configurable time budget, reserving time for LoadBalancer captures and stopping operations when the budget is spent. * Capture reports distinguish failed, successful, unknown, and budget-limited checks. LoadBalancers with successful probes are skipped for heavy capture. * Repeated captures are preserved together, and limits help control LoadBalancer diagnostic collection. * Diagnostics include additional network details from relevant nodes during probe replay. * **Documentation** * Updated guidance covers capture time limits and how unanswered EndpointSlice reads are reported. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
f209c477e2 |
test(e2e): read the opensearch release history through the pinned name (#4689)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
The opensearch suite now reads HelmRelease history through the shared
jsonpath, and a unit test fails if any Chainsaw suite goes back to a
copied literal. Before this, the suite carried its own copy of the
expression, and the upstream contract test in `internal/fluxcontract`
only checks `HELMRELEASE_HISTORY_JSONPATH`. I checked the copy against
helm-controller `api` v1.5.1, the version go.mod pins: the history field
is `history` and each snapshot has `status`. So the read works today, it
was just not pinned.
The suite passes `-o "jsonpath=${HELMRELEASE_HISTORY_JSONPATH}"` now,
which gives kubectl exactly the same argument as before. The new test
reads every suite under `hack/e2e-chainsaw/`. A suite that calls
`helmrelease_has_remediation_cycle` must fill the local it hands the
guard from the shared name, and no suite may assign that name itself. It
reuses the matchers that already pin `run-kubernetes.sh`. The test is
red on main and green with the fix.
One known limit. The second-assignment check counts over the whole file,
which fits `run-kubernetes.sh` because it runs as one shell. A Chainsaw
suite runs each `script` step in its own shell, so an unrelated
`statuses=` in another opensearch step would turn the test red with
nothing broken. That fails toward red, never toward a silent pass, so I
left it.
Closes #4550
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
test(e2e): the opensearch suite reads HelmRelease history through the shared jsonpath, which is now pinned against the upstream Flux type for every Chainsaw suite that runs the remediation guard.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Expanded validation across Chainsaw end-to-end suites to check
consistent HelmRelease history handling in remediation checks, including
the order and isolation of script steps.
* The OpenSearch suite now uses the shared history lookup. Empty-history
checks and remediation-cycle failure handling are unchanged, and there
is no change to end-user behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
b4fbaf00a2 |
test(e2e): keep gateway and redis-tls op ceilings above their own waits (#4643)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does Two Chainsaw script ops had a timeout smaller than the waits inside them. When the op ceiling fires first, Chainsaw kills the shell: the script never prints its own error, and a cleanup trap never runs. The gateway listener-set step could poll for 660s under a 6m op. Its cleanup trap also deleted two tenants with no limit on the wait. Each delete now runs under an outer `timeout`, because kubectl's own `--timeout` does not cover discovery or a stalled request. A tenant delete gets 300s, the same 5m that `.chainsaw.yaml` gives a delete op. The parent is deleted only after the child is gone, because deleting a parent while its child is still uninstalling wedges the parent's cleanup. A failed tenant delete fails the step with kubectl's own error, so a leaked tenant shows up here and not in a later suite. The op is now 24m, which covers the polls plus those bounds. The redis-tls CA-projection step has 30s read caps that add up to well over its 5m op. Raising the op would not hold, because several reads run in loops. So all reads now share a 240s step budget. No read starts after the budget runs out, and a read that is still running gets cut off when it does. A read that hits its bound, or finds the budget already spent, says which read it was. Two exec checks in the same step blamed the 15s redis-cli cap for any failure, including a refused exec or a missing container. They now print the exit status and what the exec printed. The step also runs under bash now, for pipefail. The sandbox's `/bin/sh` is dash, which rejects `set -o pipefail`. Chainsaw 0.2.15, the version the sandbox image installs, supports a `shell:` field on script ops. Without pipefail, a failed Secret read took base64's zero exit status and was reported as a wrong source name. The private-key check read the projection a second time, so a failed second read made it pass. It now checks the value the step already fetched. I added `hack/chainsaw-op-budgets.bats` to guard all of this. It holds the gateway op above its polls plus its delete bounds, keeps the child-before-parent order, holds the redis budget under its op with every kubectl call going through the clamped helper, and runs the redis step's script against a stubbed kubectl to check each failure message and the clamp. On main, 11 of its 12 tests fail. The one that passes is the healthy-stub control. Closes #4114 Closes #4250 Closes #4246 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff touches two Chainsaw suites and adds one unit test file under `hack/`. It moves or renames nothing a downstream repository reads. ### Release note ```release-note test(e2e): the gateway listener-set and redis-tls projection steps now fail on their own errors instead of being killed by the op timeout, and redis-tls reports the real cause of a failed read or exec. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added automated checks to verify gateway operations have sufficient time to complete and cleanup deletions are bounded and correctly ordered. * Expanded gateway scenarios to cover cleanup failures: if child-tenant deletion fails or times out, the parent is left in place; deletion failures are reported. * Updated Redis TLS checks with bounded read times and a shared execution limit. Failure reports include command output and status, and checks cover private-key material in the decoded projection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
1ca603a7b2 |
test(e2e): wait for the S3 gateway to accept a key before using it (#4694)
## What this PR does The bucket e2e suite fails now and then because the S3 gateway rejects a key that COSI already reports as granted. The error is `The access key ID you provided does not exist in our records.` from `mc alias set`. In job 109130131391 the BucketAccess assert finished at 21:23:43.57 and the first `mc alias set` failed at 21:23:44.82. In job 109396174094 cap1 and cap2 were accepted, and cap3 was rejected two seconds into the step. Granted access is not the same as a usable key. In seaweedfs-cosi-driver v0.3.1, `DriverGrantBucketAccess` writes the new key into `/etc/iam/identity.json` on the filer and returns. The S3 gateway in SeaweedFS 4.45 reloads that file later, from a filer metadata subscription (`weed/s3api/auth_credentials_subscribe.go`). In our chart the driver writes through `seaweedfs-filer` and the two gateways follow `seaweedfs-filer-client`, so the change may have to pass through the other filer replica first. The suite now waits until the gateway accepts the key before it writes. Every alias in both write steps goes through `await_key`, which uses the same port-forward as the writes and so asks the same gateway pod. It waits only while the answer is the InvalidAccessKeyId text, for at most 60s. Any other error fails at once, and so does an attempt cut off at its 15s cap. When the window runs out, the step fails with a message that names the BucketAccess. One probe costs at most 77s, so both steps go from a 5m to an 8m op timeout. The inner budgets now sum to 409s and 441s, and the budget comments say so. I checked the probe locally against a fake `mc`: ```text case=ok rc=0 attempts=1 elapsed=0s Added `a` successfully. case=lag rc=0 attempts=3 elapsed=5s Added `a` successfully. case=other rc=1 attempts=1 elapsed=0s FAIL: mc alias set for BucketAccess bucket-test-admin failed (exit 1) with an error other than the key not being loaded yet case=never rc=1 attempts=31 elapsed=60s FAIL: S3 gateway still rejects the key of BucketAccess bucket-test-admin after 60s of probing (the gateway never loaded it, or the driver never wrote it) ``` A hanging `mc` fails on its first attempt after 15s with exit 124. `chainsaw lint test` from v0.2.15 accepts the file. The read-only upload check gets fixed along the way. Before, an unloaded `ro-user` key also made the upload fail, and that failure passed as the expected denial. One thing I left as it is: `await_key` is copied into both steps, because chainsaw steps share no shell, and moving it to `_lib/` would run every suite on each edit. Closes #4580 ### Screenshots Not applicable, no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note test(e2e): the bucket suite waits for the S3 gateway to accept a new key before writing with it, instead of trusting BucketAccess readiness. ``` |
||
|
|
858ca4d0a3 |
test(e2e): cover monitoring OIDC modes with a Chainsaw suite (#4678)
## What this PR does Monitoring OIDC gets live-cluster e2e coverage again. It had none since the two pre-Chainsaw BATS suites were removed without a port. This adds a Chainsaw suite in `hack/e2e-chainsaw/monitoring/` that ports what they asserted, plus the in-place switch from an inline config to a `secretRef` that nothing covered. The suite takes one Monitoring release in `tenant-test` through three modes in a row: System, CustomConfig with an inline config, and CustomConfig with a `secretRef`. It creates that release itself and never reads the tenant-root monitoring instance, so it keeps working if #4023 or #4479 turns root monitoring off. What each phase checks: - System mode: cozystack-api admits `spec.oidc` and both HelmReleases go Ready. The Grafana CR points `auth.generic_oauth` at the cozy realm, has no `role_attribute_path`, and has `skip_org_role_sync` and `oauth_allow_insecure_email_lookup` forced. The KeycloakClient is confidential and references the generated Secret, which holds 32 alphanumeric characters. The audience scope is bound to this release's clientId. There is no chart-owned `KeycloakRealmGroup`. The users-Job carries the users and the mode, and `grafana-deployment` reads the client secret from the Secret. - Switch to inline: the Grafana CR takes the tenant's map, keeps the forced keys and drops every System-only key. The users-Job reruns with the new list. The KeycloakClient, the audience scope and the client Secret are pruned, and the Deployment stops referencing that Secret. - Switch to `secretRef`: the inline `auth.generic_oauth` block is gone from the Grafana CR. The operator Secret is mounted at `/etc/grafana/oidc` on both the CR and the Deployment. This checks the mount only and does not prove Grafana loads the file, see #4679. - After the install and after each switch, neither release may show a remediation cycle. That means no `failed` or `uninstalled` snapshot, and the latest revision must be exactly 1, then 2, then 3. The revision check is there because helm-controller truncates history after the previous deployed snapshot. An upgrade that failed, rolled back and then passed on retry leaves no failed entry, only extra revisions. I did not assert `GF_PATHS_CUSTOM_INI`. I could not find anything in Grafana that reads it (#4679), and pinning it would make possibly dead wiring part of the contract. The coverage split with the render tests is this. `packages/system/monitoring/tests/oidc_test.yaml` keeps the fresh-install render of each mode, including the CustomConfig negatives (no Keycloak objects, no client Secret). This suite covers what a render cannot see: admission, the Deployment the Grafana operator builds, and what a switch between modes prunes or leaves behind. The Keycloak objects are checked as written, so creating the client in Keycloak is not asserted. Removing it is, because the operator's finalizer holds the KeycloakClient until the client is gone from Keycloak. Before the first apply the suite also waits until the tenant's `cozystack-values` carries `oidc-enabled: "true"`. The outer release copies `_cluster` into the inner release once, so a Monitoring applied right after the tenant is created can render from stale values and fail the System-mode guard for the full 15m. That is what the first CI run on this PR hit. The Keycloak CRDs are required, not probed for. The e2e install always enables platform OIDC, so a missing CRD is a broken install. The removed suites skipped in that case, and their probe could not tell an absent API group from a present one. The selector picks the suite up by name from `cozystack.monitoring-application` and `cozystack.monitoring`. A change to `system/monitoring`, `extra/monitoring`, `monitoring-rd` or `grafana-operator` now runs only this suite instead of the full run. The tenant-root Grafana bring-up stays covered by the install BATS while it exists. One selector unit test relied on `cozystack.monitoring` reaching no suite, so it now renames that source in its copy of the graph. The second commit keeps Keycloak changes on every suite that drives a Keycloak client. Before this PR a change to `system/keycloak` or `system/keycloak-operator` escalated to the full run. With the monitoring suite it would select monitoring alone, because `monitoring-application` is the only source that depends on `keycloak-operator`, and `kubernetes-latest` with its OIDC lifecycle would drop out. `src_to_suites()` now maps both to `kubernetes-latest monitoring`. The mapping has to be explicit because `kubernetes-application` has no `dependsOn` on `keycloak-operator` (#4680). A `postgres-operator` change now also reaches `kubernetes-latest`, through Keycloak's database. The selector comment and the e2e doc no longer use `system/postgres-operator` as the example of a path with a source that reaches no suite. Cost: one monitoring stack bring-up, two in-place upgrades and teardown. I expect 10 to 20 minutes on every full run. The selector change makes the CI run on this PR a full run, so the suite's own timing is in the chainsaw report. A red run can spend up to 16m on each of four waits, since each one sits a minute above the 15m timeout of the release it waits on. The suite cannot run locally, so I checked it this way: - `chainsaw lint test` at v0.2.15, the sandbox version, passes. - I ran the suite's own steps against a local envtest apiserver. Chainsaw applies the real manifests, and a stand-in for helm and the operators renders the chart from the stored Monitoring spec. With the chart intact all nine steps pass. Each of 30 single breakages of the chart or of the simulated controllers turns the expected step red. - `hack/select-e2e_test.bats` and `hack/select-install_test.bats` pass. The new Keycloak selection case fails without the mapping and when `kubernetes-latest` is dropped from it. A browser login through Keycloak is out of scope, same as in the removed suites. Left for later: - KeycloakClient and scope `status` after the operator creates them in Keycloak is not checked. - `hack/e2e-chainsaw/README.md` lists suites by hand. I added `monitoring`, but `backup`, `kafka-metadata` and `kubernetes-proxmox` were already missing. - The `operator diff selects all dependent app suites` selector test could also check `monitoring`. - The comment on the Keycloak mapping says both suites wait for the finalizer to remove the client. The monitoring suite does wait. `kubernetes-latest` only reads once after its upgrade. - The ready checks compare `observedGeneration` with `metadata.generation` through `contains()`. Both are fields of the same object, not a number literal, so the v0.2.15 literal gotcha should not apply, but no live run has passed this suite yet. - The inline fixture carries no `enabled: "true"`, so that phase checks a disabled OAuth block. The asserts do not depend on it. Closes #4051 Closes #3785 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added end-to-end coverage for Monitoring installations using system OIDC, inline custom configuration, and Secret-based configuration. The tests verify readiness, rendered resources, configuration cleanup, and upgrade history. * Updated test selection so Keycloak-related changes also run the Monitoring suite. * **Documentation** * Added Monitoring to the documented Chainsaw suite list and clarified how changed paths determine test coverage. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6e4c9c4b6a |
fix(ci): harden the PR labeler's title parsing and map recurring scopes (#4666)
## What this PR does Fixes three problems in `.github/workflows/pr-labeler.yaml`. Each one either killed the job or put a PR in the wrong area, and none of them showed up anywhere. - Both maps are now looked up with `Object.hasOwn`. A title typed or scoped `constructor`, `toString`, `hasOwnProperty` or `__proto__` used to resolve to an `Object.prototype` member and crash the step on `l.startsWith`, which left the PR with no labels at all. - Every leading `[Backport ...]` prefix is now stripped, not only the first one. With two prefixes, the second was read as a bracket scope, so the real scope and the `kind/*` label were both lost. The `area/release` label hid the problem by keeping `area/uncategorized` from firing. - Scopes that recur three or more times over the full PR history now map to an area. So does each area's own name (`networking`, `storage`, `database`, `virtualization`, `testing`, `dependencies`). Scope-to-area decisions: - `area/database`: `database`, `etcd-operator`, `opensearch`, `opensearch-operator` - `area/networking`: `networking`, `kubeovn`, `multus`, `kilo`, `hetzner-robotlb`, `ouroboros` (it rewrites the CoreDNS Corefile to fix hairpin NAT, so it is networking, not platform) - `area/virtualization`: `virtualization`, `vm`, `kubevirt-operator`, `kubevirt-cdi`, `kubevirt-instancetypes` - `area/platform`: `operator`, `cozystack-operator`, `controller`, `cozystack-controller`, `cozystack-basics`, `flux-shard-operator`, `migrations`, `scheduler`, `cozystack-scheduler`, `reloader`, `bootbox`, and `lineage`. I put `lineage` here on purpose: no existing label covers it more precisely, and it is too narrow for an area of its own. - `area/storage`: `storage`, `piraeus-operator`, `backupstrategy-controller` - `area/release`: `changelog`, `changelogs` - `area/ci`: `workflows`, `labeler` - `area/dependencies`: `dependencies`, `renovate` - `area/api`: `registry` (the cozystack-api storage layer) - `area/dashboard`: `dashboard-controller` - `area/monitoring`: `monitoring-agents` - `area/keycloak`: `keycloak-configure` - `area/testing`: `testing` - New `area/dx`, added to `.github/labels.yml`: `dx`, `hack`, `cozypkg`. These are developer tooling (hack/ scripts, package Makefiles, cozypkg). `area/build` covers image builds and `area/testing` covers tests, and neither describes this work. Left unmapped on purpose: - `http-cache`: the application was removed from the tree. - `openbao`: already mapped on main. - `apps`, `system`, `docs`: catch-all scopes that don't point to one area. - `fix`, `bugfix`: malformed titles. - `security`, `rbac`, `governance`, `codeowners`, `maintenance`, `images`, `readme`, `telemetry`, `cluster-autoscaler`: no single area fits any of them. I did not add spelling normalisation. `kubeovn` got its own row, since it is the only alternate spelling that recurs. A new contract test, `hack/pr-labeler-contract.bats`, pulls the workflow's own `script:` block out of the file and runs it under node with stubbed `context`, `core` and `github`, so it tests the shipped code, not a copy. On main it is red on the prototype titles (`THROW: l.startsWith is not a function`), on the double backport (only `area/release`), on the newly mapped scopes and on the area-name scopes. It also checks that every map target exists in `labels.yml` and that no key is written twice. The suite fails rather than skips when node is missing. The older node-based suites skip, so this is the first one that needs node on the unit-test runner. Some things stay out of this PR. The scope table in `docs/agents/contributing.md` is a summary. It has no row for `area/dx` and does not mention that an area's own name works as a scope. Separately, the singular `migration` still maps to `area/virtualization` as it did before, while the new `migrations` maps to `area/platform`. A backport PR whose scope is unmapped still never gets `area/uncategorized`, even with a single prefix. The `area/release` added by the strip satisfies the fallback check. This behaviour is the same on main and is not changed here. The test that checks map targets against `labels.yml` passes by itself if the map literal cannot be extracted. The duplicate-key and area-name tests go red in that case, so the suite as a whole still fails. Closes #4125, closes #4124, closes #4122 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(ci): the PR labeler no longer crashes on titles scoped like Object.prototype members, handles nested backport prefixes, and maps more recurring scopes, including a new area/dx. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added the `area/dx` label for issues and pull requests related to developer tooling. * **Improvements** * Pull requests can now be categorized using a broader set of scope aliases. * Repeated leading backport prefixes are recognized and route the pull request to the release area. * Label mapping now avoids treating inherited property names as valid mappings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
02fbe76418 |
fix(ci): close three blind spots in the e2e test selectors (#4658)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The e2e selectors miss Chainsaw suites that have no `chainsaw-test.yaml` or `.yml` directly in their directory. Chainsaw v0.2.15 also runs a directory that holds only numbered step files such as `01-install.yaml`, and its discovery walks every subdirectory of the path the runner hands it. A suite in either layout ran only when nothing was selected. No narrowed selection named it, the full-suite list left it out, and `select-install.sh --validate` never asked for its mapping. Now `hack/select-e2e.sh` and `hack/select-install.sh` list a top-level directory as a suite when Chainsaw would load a test from it or from anywhere below it. No suite in the current tree uses either layout, so today's suite list stays the same. A file placed directly in `examples/backups/` (a shared `common.sh`, for example) selected nothing, because it fell through to the inert `examples/` rule. It now escalates to the full suite, same as `examples/backups/_lib/`. `README.md` there stays inert through the `*.md` rule. The `test-chainsaw` comment in `packages/core/testing/Makefile` said Chainsaw discovers every `chainsaw-test.yaml`, which was too narrow. I fixed it, and the matching sentences in `docs/agents/e2e-testing.md` too. Every behaviour change has a bats test that fails against the scripts on main. Known gaps I left for a follow-up, none of them a wrong answer today: - The `full_suite_list` helper in `hack/select-e2e_test.bats` repeats the selector's `find`/`sed`, so a bug in that shared pattern would pass every full-suite assertion. The step-file and nested tests compare against literal lists, which covers the new layouts. - The selectors are a little wider than Chainsaw: a broken `chainsaw-test.y*ml` symlink, or one with no Test in it, still lists its suite. That errs toward running more. - A few test comments describe what used to happen rather than why the rule exists. Closes #4630 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(ci): the e2e test selectors now see Chainsaw suites made of numbered step files or nested below a suite directory, and run the full suite for an edit to a file placed directly in examples/backups/. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * E2E suite discovery recognizes supported manifest and numbered-step files at any depth and maps them to the top-level suite. * Changes to files directly under `examples/backups/` trigger full-suite selection. * Suite selection reports an error if discovery fails or finds no suites. * **Documentation** * Clarified supported suite layouts, disabled suite-file handling, discovery and escalation rules, and per-path coverage guidance. * **Tests** * Added coverage for nested suites, numbered steps, unmapped suite layouts, and backup-path selection. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |