Commit Graph
5937 Commits
Author SHA1 Message Date
Myasnikov Daniil ad00f35f2d test(e2e): correct two comments that describe the opposite of the code
pull-requests.yaml said the QEMU substrate "is not wired into any
workflow now" and that prepare-cluster and hack/e2e-prepare-cluster.bats
stay in the tree for a nightly lane that will carry the coverage. Both
halves are wrong in the direction that costs the most: nightly.yaml and
e2e-tag.yaml already run `make prepare-env`, which calls prepare-cluster,
so anyone trusting that comment deletes live code and takes nightly and
every release-candidate run with it. docs/agents/e2e-testing.md states it
correctly, so the tree disagreed with itself.

hack/e2e-platform-packages.sh said the lane replaces "only LINSTOR with
an otherwise-identical Package". It replaces two, and the kubevirt-cdi
one is not identical: it raises the importer memory limit from CDI's
600M to 4Gi and the request from 60M to 256Mi. The consequence belongs
next to the override -- the merge-gating lane does not exercise the
shipped CDI default, and whether a tenant worker disk imports at 600M is
answered by the QEMU lanes, which take no override.

Also drops the dead half of `resolve_assets`. It resolved
nocloud-amd64.raw.xz and exported disk_id; the container lane boots no
disk and nothing in this workflow reads that output any more, so all the
lookup could still do was fail a release-labelled PR over an asset it
does not use. The job stays -- e2e's release arm gates on its result --
and now proves what is still worth proving, that the draft release
exists. e2e-tag.yaml keeps resolving the asset, because it consumes it.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-09-05 14:51:06 +05:00
Myasnikov Daniil f73729937c docs(vm-disk): record what immediate binding costs on a node-pinned class
Requesting immediate binding for every source, not just `upload`, is
deliberate: a VMDisk is a standalone object populated before any VM
consumes it, so on a WaitForFirstConsumer StorageClass a disk with no VM
attached would never populate at all.

What that costs was recorded nowhere an operator reads. On a node-pinned
class the CDI worker pod chooses the node, so the volume binds where the
importer landed rather than where the VM will run, and a VM constrained
elsewhere -- a nodeSelector, a request only one node satisfies, or the
Windows affinity from _cluster.scheduling.dedicatedNodesForWindowsVMs --
stays unschedulable against a node-affinity conflict. One disk is enough
for that; it is not a multi-disk-only case. storageClass is immutable,
so the exit is delete and recreate. The chart default `replicated` binds
Immediate regardless and does not move.

Also pinned in tests, in both directions the annotation is emitted: per
source type (blank, image, disk) so a regression to the old upload-only
condition reddens, and on the existing-DataVolume path, where `helm
upgrade` retrofits the annotation onto disks created before it was
unconditional -- inert for a volume that has already bound, and now a
recorded decision rather than an unasserted side effect.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-09-05 14:51:05 +05:00
Myasnikov Daniil be55c68fbc docs(linstor): keep the strict-topology rationale out of the applied object
A `#` comment in a rendered template is output, not source: the eight
lines explaining --strict-topology were applied to the live
LinstorCluster and came back on every `kubectl get`. Moved into a
template comment, which is stripped at render.

The test comment beside the argument-count assertion claimed the count
was "a tripwire that forces a re-check" on a piraeus-operator bump. It
cannot be. The assertion reads the chart's own rendered args, and those
do not move when upstream adds a flag -- the count stays 12 and the
suite stays green. It catches an edit to this chart, which is the case
that needs it least. Say what it does catch, and say where the upstream
half actually has to be checked: the tree vendors the piraeus chart but
not the operator resources carrying those defaults, so there is nothing
in-tree to diff against and the re-check on a bump stays a human
obligation. The list is recorded as verified against v2.10.2.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-09-05 14:51:05 +05:00
Myasnikov Daniil f34a7f5ea1 fix(linstor): refuse drbd.enabled=false on a non-Talos substrate
drbd.enabled=false removes the drbd-logger sidecar, and that is all it
removes. The satellite DaemonSet also carries two DRBD-only
initContainers that piraeus-operator contributes unconditionally --
drbd-module-loader, which cannot load a module the substrate does not
have, and drbd-shutdown-guard -- and the only configuration that deletes
them is satellites-talos.yaml, gated on talos.enabled.

So with talos.enabled=false and drbd.enabled=false the chart rendered a
satellite that can never reach Ready: exactly the state the flag exists
to prevent, reached by setting the flag. Not hypothetical -- the
isp-full-generic bundle sets talos.enabled=false
(packages/core/platform/templates/bundles/system.yaml).

Fail at render time and name both flags. The alternative, moving the two
initContainer deletions and their volumes out from under talos.enabled,
is the real fix for generic Linux and is a larger change than this lane
needs; it is tracked in cozystack/cozystack#4092, and the message points
there. Refusing is what keeps an operator from getting no signal at all
in the meantime. values.yaml states the requirement beside the flag.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-09-05 14:51:05 +05:00
Myasnikov Daniil 12157a2f93 docs(agents): describe the container lane as the merged standard
The container lane moves out of "In-flight direction (not yet the merged
standard)" and into the body, because it is now what both merge-gating
lanes run. The node-join section is rewritten the other way round: it
described a deadline that failed the suite without blocking the lane,
and there is no longer any such carve-out to describe.

What the section gains is the part that was missing while the lane was a
spike -- what moves off the per-PR path with the substrate, and where it
still gets covered. DRBD and the replicated StorageClass were already
named; Immediate binding, tenant StorageClass propagation and the
Cozystack Talos node image were not, and all three now say nightly and
release-candidate rather than reading as lost. The host CPU and memory
visibility gap is stated too, since it is what a reader meets first on a
red run and it looks like a product bug.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:19:18 +05:00
Myasnikov Daniil 186547d835 test(e2e): run the merge-gating lanes on Talos containers
The srv1-srv3 nodes of both lanes that gate a merge -- the same-repo one
in pull-requests.yaml and the fork one in e2e-fork.yaml -- run as Talos
containers instead of QEMU guests, so a tenant worker sits at L2 rather
than L3. Measured in CI on commits where both substrates ran the same
tree: whole-job wall clock down 24-49 minutes, essentially all of it in
Chainsaw, while the two tenant-Kubernetes suites go from 1143-2040s to
466-809s and stop depending on the runner's vgif flag.

The node-join soft-red gate goes with the substrate, which is what it
was for. It existed because nested virtualisation on the shared runners
degraded far enough that a worker registering in minutes elsewhere could
miss any deadline this test could afford; #3513 established that the
discriminator was the host's kvm_amd vgif flag, and removing a nesting
level removes what it acted on. So hack/e2e-node-join-soft-red.sh, the
SOFT-RED-node-join.txt marker and the soft_red job outputs are gone from
all four workflows, and a missed deadline is an ordinary red again. The
::warning annotation survives, because the reason a run went red should
be legible without opening the log, and so does the 124-versus-everything
-else test, because "the workers were slow" and "the wait never ran"
send a reader to different places.

Done by swapping the substrate inside the existing e2e jobs rather than
by promoting a separate container job. Those jobs carry more than a
substrate -- the GitHub App token, the report-overrun guard, the images
list, the SSH breakpoint -- and promoting a job that never had them
would have dropped four features without saying so.

Three properties of container mode drive the design, and each fails
quietly rather than loudly. machine.kernel.modules is a silent no-op --
kernel_module_spec.go returns early on ModeContainer with no error and
no event -- so the host loads openvswitch and zfs before the nodes start
and the workflow step asserts both rather than trusting them; a missing
module surfaces an hour later looking like a CNI or a storage
regression. Docker gives a privileged container its own tmpfs /dev
seeded once at creation, so /dev/kvm works but ZFS zvols created later
never appear and linstor-csi fails ControllerPublishVolume; the compose
file binds the host devtmpfs, which is also why talosctl cluster create
docker can never carry LINSTOR. And there is no ARP VIP, so
192.168.123.11 stands in for the QEMU lane's .10.

The QEMU machinery stays in the tree and keeps running: nightly.yaml and
e2e-tag.yaml still call `make prepare-env`, so DRBD, the replicated
StorageClass, its Immediate binding mode, the tenant StorageClass
propagation that only applies to remotely-accessible classes, and the
Cozystack Talos node image with its extensions all keep nightly and
release-candidate coverage. None of them is exercised per pull request
any more. That is the trade.

Nothing on the PR path builds or downloads a nocloud disk now, so
build-talos drops the talos-nocloud step and the talos-image artifact.
build-talos itself stays: finalize consumes its digest fragment.

One property this does not fix, recorded beside the runner class because
it will be read off a red run eventually: a container node reports the
HOST's CPU and memory as capacity, and hack/e2e-container-up.sh counters
that with kubelet systemReserved. That corrects the scheduler and not
what a pod reads from /proc, so a workload sizing itself from
/proc/cpuinfo or /proc/meminfo is configured for the whole runner.

Also carried here because the same files carry them: the ghcr.io
pull-through mirror removal (see #4007 -- whichever lands first makes the
other a no-op), the authoritative HelmRelease readiness gate, the
backup-credential preflight, the prepull guard against an image-less
container fragment, and folding the two standalone OIDC suites into the
latest-version tenant cluster.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:19:17 +05:00
Myasnikov Daniil e49633208c ci(tests): parallelize unit test targets
The BATS half of `make unit-tests` ran as one shell loop over ~60 files,
which is a nine-minute serial tail no amount of runner CPU could touch.
Each file becomes its own Make prerequisite instead, so `-j4` schedules
them, and the workflow asks for `make unit-tests test-controllers -j4
--output-sync=target` in a single step: one make process owns all four
runner CPUs, the Helm and Go targets share the same bounded pool rather
than waiting for each other, and output stays grouped by target instead
of four logs interleaving.

The existence check moves into its own prerequisite rather than
disappearing. It guards against the wildcard matching nothing, which
would otherwise make the whole target succeed by having no work.

COZYTEST_TRACE rides per target. It arrived with the serial loop it now
replaces, and a target that dropped it would go back to streaming full
xtrace for every suite -- so the parallelism guard requires the prefix
rather than tolerating it.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:19:17 +05:00
Myasnikov Daniil 5eaf44b8ca fix(vm-disk): bind standalone disks immediately
A VMDisk is a standalone object that is populated before any VM consumes
it, so on a WaitForFirstConsumer StorageClass CDI needs an explicit
signal to schedule its worker and start provisioning. Without it the
DataVolume sits in PendingPopulation indefinitely and the disk is never
filled -- and nothing says so.

This is a consistency fix rather than a new behaviour. The chart default
`replicated` is volumeBindingMode: Immediate, so a disk on it already
bound and populated at creation; `local`, the cluster's default class,
is WaitForFirstConsumer, so the same VMDisk silently did nothing there.
The annotation was previously emitted only for source: upload, which is
the one source that could not work without it, leaving http, image, disk
and blank to differ by class.

Two consequences worth knowing. CDI checks the annotation by presence
and ignores its value, so "" and "true" are equivalent and this is not a
behaviour change on the upload path. And on a node-pinned class each
disk now binds at its own creation rather than at VM scheduling, so a
multi-disk VM on `local` can find its disks on different nodes -- that
is the existing limitation of a node-pinned class, which is why the
chart default is not one, but it is reached earlier now.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:19:17 +05:00
Myasnikov Daniil 0418577f55 fix(kubevirt-cdi): make importer resources configurable
podResourceRequirements applies to every worker pod CDI creates --
importer, uploader and host-assisted cloner alike, not only the importer
the parameter is named for -- so a value set here reaches the console's
disk-upload path as well as the imports. It has to be set on the CDI CR
and not on CDIConfig: the operator reconciles CDIConfig.spec from this
resource, so a direct patch is silently reverted and
status.defaultPodResourceRequirements stays where it was.

The default restates CDI's own built-in values exactly -- limits
750m/600M, requests 100m/60M, read off a live cluster -- so an install
that sets nothing behaves as it did before the parameter existed. That
is deliberate. A 4Gi ceiling was measured as necessary on one hand-built
stand, where the decompress+convert of a tenant worker disk was
OOM-killed at 99.8-100% and CDI then retried from scratch forever, but
the same 20Gi import completes at stock 600M on a QEMU-node cluster and
no CI run on either lane has recorded an OOMKilled importer. What has
been shown to need the headroom is that substrate, so the override lives
with the substrate in hack/e2e-platform-packages.sh rather than here.

The failure being answered is memory, and only memory. CDI already ships
a CPU limit of its own at 750m, nothing measured has asked for more, and
the default here restates that figure rather than raising it.

The package had no `test` target, and hack/helm-unit-tests.sh only runs
a package whose `make -n test` resolves, so a tests/ directory here was
collected by nothing and skipped in silence. Added with the suite.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
2026-09-05 13:19:17 +05:00
Myasnikov Daniil 499034e960 fix(linstor): constrain CSI provisioning to the scheduler's chosen node
Without --strict-topology, external-provisioner passes EVERY topology
segment as `requisite` and merely *prefers* the node the scheduler
picked, so when that node is out of space linstor-csi may legally
provision the volume somewhere else. For a node-pinned volume
(allowRemoteVolumeAccess: false, the `local` class that is the cluster
default) the result is a PV whose node affinity contradicts the pod's
other volumes, and the pod is then unschedulable forever against

  pv "pvc-..." node affinity doesn't match node "srv1": no matching NodeSelectorTerms

with nothing erroring anywhere.

It bites hardest on CDI imports, because an importer pod mounts two
volumes: the disk, and a ~21 GiB scratch volume CDI creates only AFTER
the pod is scheduled. The scheduler therefore sizes the node against the
disk alone, can place two importers where both disks plus both scratch
volumes do not fit, and the second scratch lands elsewhere. Capacity
tracking cannot help -- the scratch volume does not exist when placement
is decided, and CSIStorageCapacity agreed with LINSTOR to within 5% when
this was measured.

Reproduced on 2026-08-25 by cordoning two of three nodes so both
importers landed on one constrained to 45 GiB: enough for two 20 GiB
disks, not for their scratch. With the flag the scratch PVC stays
Pending and the pod reports `1 node(s) did not have enough free
storage`, then binds and completes once space frees. A silent permanent
deadlock becomes a legible, self-correcting wait.

The full argument list is restated because `args` has no patchMergeKey,
so a strategic merge replaces rather than appends. That makes the list a
drift hazard on every piraeus-operator bump, and the note beside it says
so -- the count assertion in the test guards this chart's own list and
cannot see a flag the operator adds upstream.

Verified for node-pinned volumes. For `replicated` the PV is remotely
accessible so the constraint is satisfiable wherever the pod lands, and
that case is reasoned rather than measured -- hence the switch.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:17:48 +05:00
Myasnikov Daniil 06ee08f636 feat(linstor): gate the DRBD sidecar behind drbd.enabled
On a substrate where the DRBD kernel module is unavailable the
drbd-logger sidecar exits immediately, the satellite pod is therefore
never Ready, and piraeus consequently registers zero nodes: LINSTOR
reports none, linstor-csi-node hangs in Init polling for a node that
does not exist, and every PVC stays Pending behind a cluster that looks
healthy. So this flag is the difference between working storage and no
storage at all there, not a cosmetic reduction in logging.

The concrete case is Talos nodes running as containers. DRBD's resource
registry is global per kernel, so three containers sharing one kernel
cannot represent three DRBD nodes -- the second new-resource for a given
name silently aliases the first instead of failing, which is worse than
an error because LINSTOR then records a replica it does not have. It
applies to kind equally; it is a property of one kernel, not of Talos.

Omitting the sidecar from Cozystack's own pod template is not enough,
because piraeus strategically merges satellite configurations and a
container another configuration contributes survives the omission. So a
LinstorSatelliteConfiguration deletes it from the effective DaemonSet as
well.

What the flag does NOT do is touch a StorageClass, and the comment says
so explicitly. Cozystack does not own them -- an operator writes them by
hand from the install guide -- so a class carrying layerList: "drbd
storage" still exists after setting this false and volumes on it will
never provision. The operator-facing half of that warning goes beside
the examples they copy, in cozystack/website#675.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-05 13:17:48 +05:00
myasnikovdaniil 7b0c35b9b3 feat(monitoring): add KubeVirt VM dashboard (#2198)
## Summary

- Adds `dashboards/vm/virtual-machines.json` — KubeVirt VM resource
metrics (CPU, memory, network, storage) with namespace and VM template
variables
- Adds `dashboards/vm/console-logs.json` — VM serial console log viewer
via VictoriaLogs with namespace filtering
- Adds `vm/virtual-machines` and `vm/console-logs` entries to
`dashboards.list`

## Problem

`dashboards.list` references `vm/virtual-machines` and `vm/console-logs`
but the corresponding JSON files don't exist.

The console-logs dashboard includes `kubernetes_namespace_name`
filtering to prevent cross-tenant log exposure.

Relates to #2194

## Test plan

- [ ] Verify JSON is valid for both dashboards
- [ ] Import dashboards in Grafana and verify panels render with
KubeVirt metrics
- [ ] Verify console-logs queries include namespace filter

```release-note
Add KubeVirt VM dashboards (virtual-machines and console-logs)
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* **VM Console Logs dashboard**: View live console output with namespace
and VM filtering, fast refresh, and improved log presentation (details,
wrapping, and sorting).
* **Virtual Machines dashboard**: Rich VM monitoring covering CPU,
memory, network, storage I/O, latency, swap/page faults, and filesystem
usage with thresholds and formatted visuals.
* **Dashboard catalog updated**: Added multiple new monitoring
dashboards for broader coverage and easier discovery, including the new
VM dashboards.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 16:59:24 +05:00
myasnikovdaniil 1b2beac3f8 [cilium] Set resources requests/limits on cilium-operator (#2618)
## Summary

Companion to #2617 (cilium-agent). The upstream cilium chart ships with
`operator.resources: {}`, which triggers a `no CPU limit` conformance
warning on the `cilium-operator` Deployment.

Override `packages/system/cilium/values.yaml` to set the sizing
recommended by the upstream `values.yaml` comment:

- `limits` : `cpu: 1000m`, `memory: 1Gi`
- `requests` : `cpu: 100m`, `memory: 128Mi`

## Test plan

- [ ] `helm template . --namespace kube-system` from
`packages/system/cilium/` renders a `resources` block on the
`cilium-operator` container of the Deployment (verified locally)
- [ ] Re-run the conformance/lint tool — the `cilium-operator has no CPU
limit` warning should be gone
- [ ] CI passes

### Release note

\`\`\`release-note
Set CPU/memory requests and limits on the cilium-operator Deployment to
clear the "no CPU limit" conformance warning. Values follow the upstream
chart's recommended sizing (limits 1000m/1Gi, requests 100m/128Mi).
\`\`\`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated Cilium system component resource configuration with CPU and
memory requests and limits.

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2618)

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 16:30:49 +05:00
Aleksei Sviridkin a2cb47c961 feat(gateway): update the CRD bundle to Gateway API v1.6.1 and gate GRPCRoute hostnames (#3872)
## What this PR does

Bumps the vendored Gateway API CRD bundle in
`packages/system/gateway-api-crds` from v1.5.1 to v1.6.1. The package
Makefile pins the upstream ref, so the change is that pin plus the
re-rendered `templates/crds-experimental.yaml`.

The bundle version is not only bookkeeping: it is a compatibility signal
consumers read at runtime. Every CRD in the bundle carries a
`gateway.networking.k8s.io/bundle-version` annotation, and a Gateway API
controller can read it off the `gatewayclasses` CRD to decide whether
the installed bundle matches the one it was built against. Controllers
that do this typically accept any patch release but require an exact
major and minor match, and report `SupportedVersion=False` with reason
`UnsupportedVersion` on their GatewayClass when the minor differs, a
status condition that surfaces the skew rather than a functional break.
The Cloudflare Tunnel Gateway API controller proposed for the platform
in #3858 is built against Gateway API v1.6.1 and performs exactly that
comparison, so against the v1.5.1 bundle shipped today its GatewayClass
reports the bundle unsupported, and with this bump it reports supported.
Neither change depends on the other landing first: this PR only moves
the bundle, and the check lives entirely on the consumer side.

One consequence worth stating plainly, since it will outlive this PR: an
exact-minor check makes the shipped bundle version a coupled pin for any
such controller. A future bump to v1.7.x flips the same condition back
to `UnsupportedVersion` until that controller is rebuilt against the
newer bundle, so the two want to move together.

The bundle still emits the same twelve CustomResourceDefinitions and the
same two `safe-upgrades.gateway.networking.k8s.io` admission objects,
and every CRD still carries the `helm.sh/resource-policy: keep` stamp
the Makefile's post-processing step adds. One of the policy's two CEL
rules is rewritten, though, and it changes what a cluster accepts. That
has its own section below.

Two kinds change their version surface: TCPRoute and UDPRoute gain a
served `v1` that takes over as the storage version, and their `v1alpha2`
stays served with a deprecation marker. Every other kind keeps exactly
the versions it served at v1.5.1. That matters because the Cilium
operator resolves GatewayClass, Gateway, HTTPRoute and GRPCRoute in
`v1`, ReferenceGrant in `v1beta1` and TLSRoute in `v1alpha2` before it
enables its Gateway API controller; all six are still served by the
v1.6.1 bundle. TLSRoute in particular keeps `v1alpha2`, which the tenant
gateway controller, the platform TLSRoute templates and the tenant
hostname admission policy all address.

Schema changes are small and land on fields nothing in this repository
sets. `sessionPersistence.idleTimeout` is gone from HTTPRoute, GRPCRoute
and XBackendTrafficPolicy, removed upstream under GEP-1619 because no
implementation ever implemented it and HTTP cookies have no native
idle-timeout mechanism, so no working behaviour is lost even though the
field disappears from the schema; HTTPRoute gains a lower bound on
`retry.attempts` and now rejects duplicate `retry.codes`; ReferenceGrant
requires `spec` at the root. In the other direction, Gateway
infrastructure annotations and frontend CA certificate references go
from a limit of 8 to 16, and TLSRoute hostnames from 16 to 1024. No CRD
schema loses or changes a CEL validation rule, and the six added belong
exclusively to the new TCPRoute and UDPRoute `v1` schemas.

### The safe-upgrades admission policy accepts a different set of
bundles

The experimental kustomization ships a `ValidatingAdmissionPolicy` and
its binding, both named `safe-upgrades.gateway.networking.k8s.io`, next
to the twelve CRDs. The binding is `validationActions: [Deny]` with
`failurePolicy: Fail`, matching CREATE and UPDATE on
`apiextensions.k8s.io/v1 customresourcedefinitions` cluster-wide, and
Cozystack installs it in two places: on the host through
`cozystack.gateway-api-crds`, and inside every guest cluster that sets
`addons.gatewayAPI.enabled`.

Its second rule reads the `gateway.networking.k8s.io/bundle-version`
annotation off an incoming CRD. At v1.5.1 that rule was a blacklist:
reject anything matching `v1.[0-4].\d+` or containing `v0`, allow the
rest. At v1.6.1 it is a whitelist: allow `v0.0.0-dev` outright,
otherwise require the annotation to start with `v1.` and not match
`^v1\.[0-4](\.|$)`. Evaluating both expressions over a spread of
annotation values:

| bundle-version | v1.5.1 policy | v1.6.1 policy |
| --- | --- | --- |
| `v1.6.1`, `v1.5.1`, `v1.5.0-dev`, `v1.10.0` | allow | allow |
| `v1.4.0`, `v1.4.10`, `v1.0.0` | reject | reject |
| `v0.0.0-dev` | reject | **allow** |
| `v2.0.0` | allow | **reject** |
| `1.6.1` (no leading `v`) | allow | **reject** |
| `v1.4` (no patch) | allow | **reject** |
| `main`, empty string | allow | **reject** |

Nothing in this repository trips the new rejections: every CRD the
bundle ships is stamped `v1.6.1`. What is exposed is an operator
applying a Gateway API CRD set from somewhere else on a cluster that
carries the policy. A fork, a vendor build or a nightly whose annotation
reads `1.6.1` or `v2.0.0` used to go in and is now denied with `reason:
Invalid`, naming a policy the operator may not know is installed, and
whatever HelmRelease owns those CRDs then sits un-Ready. Uninstalling
the policy is the escape hatch, and the denial message says so.

The same hunk fixes an upstream inconsistency: at v1.5.1 the policy pair
was stamped `v1.5.0-dev` while its twelve CRDs said `v1.5.1`. At v1.6.1
all fourteen documents agree.

The Gateway and HTTPRoute objects the tenant gateway controller renders
were serialized across all three certificate modes and checked against
both the v1.5.1 and the v1.6.1 schemas; the two runs are identical, so
the bump does not change what the controller can create. `go.mod` still
pins `sigs.k8s.io/gateway-api` v1.4.1 and this PR deliberately leaves it
there. The CRD bundle and the Go client types have been on separate
versions since before this change, and nothing the controller emits
depends on the gap.

One thing to know before the next bump: from v1.6 on, the two things
upstream calls the experimental bundle disagree. This package follows
`config/crd/experimental`, the kustomization, which emits 12 CRDs at
v1.6.1; the published `experimental-install.yaml` for the same tag emits
13, the extra one being `xbackends.gateway.networking.x-k8s.io`, new in
v1.6 and not listed in the kustomization. At v1.5.1 the two agreed, so
there was nothing to choose between. Nothing in this repository
references XBackend and no vendored CRD depends on it, so following the
kustomization costs nothing today, but whoever bumps next inherits the
choice, and it should be made deliberately rather than by whichever
command gets typed.

The second commit corrects a comment this bump invalidated. It claimed
TLSRoute stays at `v1alpha2` through Gateway API v1.5 and that the
promotion to `v1` was still upstream-only; TLSRoute has been served at
`v1` since v1.5.0, so the claim was already wrong before this bump and
the version range it named is now wrong too.

The third commit closes a gap that correction exposed.
`cozystack-route-hostname-policy-tls` named `v1alpha2` alone while the
bundle serves `tlsroutes` at `v1`, `v1alpha2` and `v1alpha3`: three
served versions against one named. The gap is not new and this bump
neither creates nor widens it: the v1.5.1 bundle and the v1.6.1 one
serve the same three versions, with the same two marked deprecated.
Coverage of the two unnamed versions rested entirely on request
conversion, because `matchPolicy` is unset on both rules in that file
and so defaults to `Equivalent`, which rewrites a request submitted
under any served version into the one the rule names before the policy
sees it. That is why there is no bypass to demonstrate, and equally why
the arrangement is brittle: the conversion has nothing left to convert
into once no named version is served, and a rule that selects nothing is
not caught by `failurePolicy: Fail`, because nothing failed. The rule
now names all three, which is the shape the HTTPRoute rule above it
already used, and `hack/route-hostname-policy-version-coverage.bats`
reads the served versions out of the vendored bundle and fails when a
policy does not cover every one of them, so a future bundle move cannot
outrun the list silently, in either direction.

That guard carries negative fixtures rather than only watching a correct
tree pass. Three of them drive `assert_covers` itself, with the two
readers redefined in a subshell, against an omitted served version, an
empty served set and an empty named set. Each requires the specific
complaint text rather than merely a non-zero exit, so a helper renamed
out from under it fails for the right reason instead of reading as a
successful rejection. A fourth drives the comparison helper directly.

The fourth commit pins the vendored bundle version in the CRD package's
unit suite. Nothing outside the package Makefile named a Gateway API
version, so re-running `make update` against a different ref landed
green: twelve CRDs, every one stamped, document count unchanged, suite
passing. The new case reads the
`gateway.networking.k8s.io/bundle-version` annotation off every rendered
CRD and fails when it is not the version the Makefile pins.

The fifth commit extends the version-coverage guard to
`cozystack-gateway-hostname-policy`, which names the `gateways` versions
literally the same way the two route policies do, and whose version set
lives in the vendored bundle and moves on `make update` there. It
matched the served set already; the guard now says so on every run
instead of leaving one of the three hostname policies unpinned.

Two notes for whoever reads the release: the TCPRoute and UDPRoute
storage version moves from `v1alpha2` to `v1`, and while nothing in this
repository ships either kind and the CRDs declare no conversion block,
an upgraded cluster accumulates both entries in `status.storedVersions`
— inert today, load-bearing only if a later bundle stops serving
`v1alpha2`.

The sixth commit gives GRPCRoute the same hostname rule the other two
route kinds have. GRPCRoute carries `spec.hostnames` and it can reach a
tenant Gateway: the port-443 listeners pin `allowedRoutes.kinds` to
HTTPRoute and TLSRoute, but the port-80 listener leaves `kinds` unset,
and an unset list resolves to the kinds the listener protocol supports,
which for HTTP includes GRPCRoute. A GRPCRoute in a tenant namespace
could claim any hostname on the plaintext listener with nothing checking
it. Tenants hold no `gateway.networking.k8s.io` RBAC in Cozystack, so
this is the same defence-in-depth against a chart bug that the other two
rules are, not a tenant-user control. The rule is the HTTPRoute rule
with `grpcroutes` in place of `httproutes` and the one version the
bundle serves for it, and the coverage guard pins that list to the
bundle like the others.

Several places in the tree enumerate which route kinds the hostname
policies cover, and all of them named two: the port-443 kinds invariant
in the tenant gateway controller and the test that pins it,
`packages/extra/gateway/README.md`, and the threat model together with
the self-assessment. They name three now. The port-443 kinds set also
carries an accurate reason now. cilium#45559 governs only that every
port-443 listener carries the same set, not what is in it, so the
contents are a least-privilege choice: nothing the platform ships needs
gRPC, TCP or UDP routing on port 443. The README had also described
HTTPS and TLS-passthrough listeners as carrying different kinds, where
the controller gives both the same one.

One more commit makes the suite's existing assertions bite. A
ValidatingAdmissionPolicyBinding whose `policyName` points at another
policy renders without complaint and leaves its own policy inert, and
the suite pinned `validationActions` on the bindings while never pinning
`policyName` at all, so pointing the GRPCRoute binding at the HTTPRoute
policy kept every case green with the GRPCRoute control doing nothing.
All three bindings now pin the policy they name. In the same file,
`uncovered()` assigned to the variable names its caller reads back when
printing a mismatch, so a call site with the arguments swapped would
have printed a diagnostic naming the wrong side.

Two things are named rather than made. The version-coverage guard
collects `apiVersions` from every `resourceRules` entry mentioning the
resource without also matching on `apiGroups` or `operations`, so a rule
in another group, or one scoped to a single verb, would donate its
versions to the set coverage is measured against; the chart's own suite
pins the group and the resource on the rule it checks, which closes that
a layer below. And
`packages/system/cozystack-api/templates/api-tlsroute.yaml` renders
`gateway.networking.k8s.io/v1alpha2` while the bundle serves `v1` as
storage and marks `v1alpha2` deprecated, but the identical pin sits in
the kubevirt and kubevirt-cdi TLSRoute templates, and the pinned
`sigs.k8s.io/gateway-api` carries no `apis/v1` TLSRoute type for the
controller side to follow, so moving one of the three here would leave
the tree less consistent than leaving all three.

Closes #3789.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff, file by file: neither `packages/system/gateway-api-crds/Makefile`
nor its rendered CRD template matches any entry. Leaving every box empty
rather than ticking the "no downstream repository" line, because the
docs site carries a Gateway API version statement that is already
inaccurate and needs a decision that is not mine:
`content/en/docs/next/networking/gateway-api.md` and its v1.5 and v1.6
copies say Gateway API v1.5 ships TLSRoute at `v1alpha2` and that the
graduation to `v1` is still upstream-only, when v1.5.0 already shipped
TLSRoute `v1` as the storage version.

### Release note

```release-note
chore(gateway-api-crds): update the vendored Gateway API CRD bundle to v1.6.1. TCPRoute and UDPRoute gain a served v1 that becomes their storage version, with v1alpha2 still served and now deprecated; every other kind keeps the versions it served at v1.5.1. Upstream removed `sessionPersistence.idleTimeout` from HTTPRoute, GRPCRoute and XBackendTrafficPolicy; the field is not set by anything Cozystack renders, but a hand-written route that carries it loses it silently on its next write, with no admission error.

The bundle's `safe-upgrades.gateway.networking.k8s.io` ValidatingAdmissionPolicy changes which CRDs it accepts. Cozystack installs it cluster-wide on the host and in every guest cluster with `addons.gatewayAPI.enabled`. Its `gateway.networking.k8s.io/bundle-version` rule was a blacklist on two patterns and is now a whitelist on the `v1.` prefix with a `v0.0.0-dev` escape hatch. A bundle annotated `v0.0.0-dev` is now accepted where it was denied. A bundle annotated `v2.0.0`, `1.6.1` without the leading `v`, `main`, `v1.4` without a patch, or carrying an empty annotation is now denied where it was accepted. Nothing Cozystack ships is affected, since every CRD in the bundle is stamped `v1.6.1`, but applying a Gateway API CRD set from a fork, a vendor build or a nightly on such a cluster can now fail admission with `reason: Invalid`. Uninstalling the policy remains the documented escape hatch.

fix(cozystack-basics): the tenant TLSRoute hostname admission policy now names every served TLSRoute version (`v1`, `v1alpha2`, `v1alpha3`) instead of `v1alpha2` alone. Coverage of the other two rested on request conversion, which holds only while a named version stays served, and a rule left selecting nothing is not caught by `failurePolicy: Fail`.

feat(cozystack-basics): GRPCRoute hostnames in `tenant-*` namespaces are now gated by the same admission policy that gates HTTPRoute and TLSRoute. A tenant Gateway's port-80 listener leaves `allowedRoutes.kinds` unset, which admits GRPCRoute by protocol default, so a GRPCRoute could claim any hostname on the plaintext listener unchecked. Tenants hold no `gateway.networking.k8s.io` RBAC, so this is defence-in-depth against a chart bug rather than a tenant-user control.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
  - Added hostname admission validation for `GRPCRoute` resources.
- Expanded `TLSRoute` hostname validation to support all served Gateway
API versions.
- Ensured route hostname policies consistently cover HTTP, TLS, and gRPC
routes.

- **Security**
- Updated listener policies to restrict port-443 routes to supported
HTTP and TLS route types.
- Clarified fail-closed hostname validation behavior across Gateway,
Route, and Ingress resources.

- **Documentation**
- Updated security and threat-model documentation to include GRPCRoute
handling and validation coverage.

- **Maintenance**
  - Updated the bundled Gateway API CRDs to version 1.6.1.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 15:23:36 +04:00
myasnikovdaniil b7e5100e73 fix(monitoring): strip infrastructure metadata from tenant logs (#2196)
## Summary

- Adds a `modify` filter in fluent-bit that removes
`kubernetes_node_name` and `kubernetes_host` from all container logs
- Prevents infrastructure topology (node names, hosts) from leaking to
tenant users

## Problem

`kubernetes_node_name` is exposed in log metadata, leaking node topology
to end-users. Other sensitive fields like `kubernetes_host` are also
visible.

Relates to #2194

## Test plan

- [ ] `helm template` monitoring-agents and verify `Remove
kubernetes_node_name` filter is present
- [ ] Deploy and verify `kubernetes_node_name` is absent from logs in
tenant Grafana

```release-note
Strip infrastructure metadata (node name, host) from tenant logs
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Improved Kubernetes monitoring log processing by filtering out most
`kubernetes_*` fields for `kube.*` events, while preserving the key
Kubernetes identity fields (such as pod, namespace, and
container-related names) for clearer, more consistent log output.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 16:16:24 +05:00
Andrey Kolkov 7786f72b9f feat(etcd-operator): bump to v0.5.5 and vendor EtcdDefragPolicy CRDs (#3968)
## What this PR does

Bumps the vendored `etcd-operator` from **v0.5.4** to
**[v0.5.5](https://github.com/cozystack/etcd-operator/releases/tag/v0.5.5)**
and adopts the new defrag API.

v0.5.5 ships the `EtcdDefrag` and `EtcdDefragPolicy` types — a
controller that runs a one-shot defrag against a target etcd cluster,
plus a cron-style policy that stamps scheduled `EtcdDefrag` runs
(`schedule`/`suspend`/`concurrencyPolicy`/`historyLimit`).

- **`etcd-operator-crds`**: vendor the two new CRDs
(`etcddefragpolicies`, `etcddefrags`) alongside the existing three; bump
`ETCD_OPERATOR_REF`. Re-vendoring also refreshed `etcdmembers.yaml` (a
doc-string change from the version-matched-`etcdutl` restore work in
v0.5.5). All five carry `helm.sh/resource-policy: keep`; the existing
resource-policy suite covers the new CRDs via its wildcard template
scope.
- **`etcd-operator`**: bump `appVersion`/image tag to v0.5.5 and
re-vendor the manager `ClusterRole` from the operator's
`charts/etcd-operator/files/manager-role-rules.yaml`. New grants match
what the defrag controllers need: full CRUD on `etcddefrags`,
read/patch/watch on `etcddefragpolicies`, the two new `/status`
subresources, and cluster-scoped `events` create/patch. Fixed the stale
`config/rbac/role.yaml` path in the Makefile/test comments (upstream
moved the role rules).

### Screenshots

n/a — no UI changes.

### Downstream repositories

Walked the trigger map against the diff. The change is confined to
`packages/system/etcd-operator{,-crds}`: etcd-operator is a system
component (not a `packages/apps`/`packages/extra` package the website
and TF provider track), the new CRDs are in group
`etcd-operator.cozystack.io` (not `cozystack.io`/`backups.cozystack.io`,
the only CRD groups the TF provider types), and nothing under `hack/`,
the installer values, or the platform values changed.

- [x] No downstream repository is affected by this change

### Release note

```release-note
feat(etcd-operator): bump to v0.5.5, adding the EtcdDefrag and EtcdDefragPolicy APIs for on-demand and scheduled etcd defragmentation
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added APIs for configuring recurring and one-time etcd defragmentation
tasks.
* Added status reporting for defragmentation progress, results, and
conditions.
* Added validation and lifecycle controls for defragmentation policies
and runs.

* **Enhancements**
  * Updated the etcd operator to version 0.5.5.
* Expanded permissions to manage defragmentation resources and report
events.
  * Improved restore documentation for etcd members.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-04 11:09:38 +04:00
Aleksei Sviridkin fb740db0b0 docs(security): separate the fail-open Gateway-listener policy from the fail-closed route policies
The self-assessment summarised the whole hostname policy set as
fail-closed, while its own table row states that the Gateway-listener
policy passes a Gateway whose namespace carries no host label. Say
which policies are which. The port-443 kinds comment in the reconciler
test also tied the GRPCRoute admission rule to the port-80 listener;
the rule matches the kind wherever it attaches, and the port-443
exclusion is a least-privilege choice, not the reason the rule exists.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-04 01:18:07 +03:00
Aleksei Sviridkin 19eaff6a30 test(cozystack-basics): pin which policy each route binding names
A ValidatingAdmissionPolicyBinding whose policyName points at another
policy renders without complaint and leaves its own policy inert. The
suite pinned validationActions on the bindings but never policyName, so
pointing the GRPCRoute binding at the HTTPRoute policy kept all 56 cases
green while the GRPCRoute control did nothing.

uncovered() assigned to the same variable names its caller reads back
when printing the mismatch, so a future call site with the arguments
swapped would print a diagnostic naming the wrong side.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-03 22:17:29 +03:00
Aleksei Sviridkin af19aa933c docs(security): record GRPCRoute in the route hostname policy set
The port-443 kinds set still excludes GRPCRoute, TCPRoute and UDPRoute.
cilium#45559 governs only that every port-443 listener carries the same
set, not what is in it, so the contents are a least-privilege choice:
nothing the platform ships needs gRPC, TCP or UDP routing on port 443.
TCPRoute and UDPRoute additionally carry no hostname and no admission
rule gates them.

The gateway README described HTTPS and TLS-passthrough listeners as
carrying different kinds. They carry the same one, which is what
cilium#45559 requires.

The threat model and the self-assessment enumerate which route kinds the
hostname policies cover, and both listed two.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-03 22:17:29 +03:00
Aleksei Sviridkin 70a675fd6b feat(cozystack-basics): gate GRPCRoute hostnames in tenant namespaces
The route hostname policy covered HTTPRoute and TLSRoute. GRPCRoute
carries spec.hostnames too, and it can reach a tenant Gateway: the
port-443 listeners pin allowedRoutes.kinds to HTTPRoute and TLSRoute,
but the port-80 listener leaves kinds unset, and an unset list resolves
to the kinds the listener protocol supports, which for HTTP includes
GRPCRoute. A GRPCRoute in a tenant namespace could therefore claim any
hostname on the plaintext listener with nothing checking it.

Tenants hold no gateway.networking.k8s.io RBAC, so this is the same
defence-in-depth against a chart bug that the other two rules are, not a
tenant-user control. The new rule is the HTTPRoute rule with grpcroutes
in place of httproutes and the single served version the bundle
provides; the coverage guard pins that list to the bundle like the
others.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-03 22:17:28 +03:00
Aleksei Sviridkin 6ddae65b45 test(cozystack-basics): cover the Gateway listener policy in the guard
cozystack-gateway-hostname-policy names the gateways versions literally,
the same way the two route policies do, and the set it must cover lives
in the vendored bundle and moves on `make update` there. It matched the
served set already; the guard now says so on every run instead of
leaving one of the three hostname policies unpinned.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-03 22:17:04 +03:00
Aleksei Sviridkin 72437d127f test(gateway-api-crds): pin the vendored bundle version in the suite
Nothing outside the package Makefile named the Gateway API version, so
re-running `make update` against a different ref landed with the whole
suite still green: twelve CRDs, every one stamped, document count
unchanged. The new case reads the bundle-version annotation off every
rendered CustomResourceDefinition and fails when it is not the version
the Makefile pins.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-03 22:17:04 +03:00
Aleksei Sviridkin 9311a48f6a test(cozystack-basics): make the version-coverage guard prove it still compares
The guard added alongside the TLSRoute version fix only ever ran against a
tree that already satisfied it. A passing check that is checking nothing
looks exactly like a passing check that is checking something, so a later
refactor could empty out the comparison and the suite would stay green
forever. That is the same shape as the negated assertions that cannot fail
elsewhere in this repository: the assertion survives, the guarantee does not.

Drive the checker with synthetic input instead of only with the live tree,
the way hack/bats-no-exit-trap.bats and hack/md-no-hardwrap.bats feed their
checkers a broken sample and require a complaint. Two layers need it
separately. The comparison itself moves into uncovered() so a fixture can
call it with a served set the named set does not cover. Above that,
assert_covers runs in a subshell with both readers replaced, so the wiring,
the empty-set guards and the exit status are exercised too -- covering only
the inner function leaves the outer one free to become a no-op while every
other case in the file stays green, because the live-tree cases are its only
callers and the tree they read is correct.

Each fixture requires the complaint it expects rather than any non-zero exit.
A renamed helper exits 127 and an unbound variable under set -u exits 1, and
either would read as a successful rejection if only the status were checked;
a rejection for the wrong reason is the same false comfort as no fixture, and
harder to notice because the red looks earned.

Each way of breaking the guard now fails at its own case. Emptying
uncovered(), reducing assert_covers to a no-op, and deleting the empty-set
guards each turn a different fixture red while the live-tree cases stay
green, and changing only the wording of assert_covers' complaint -- which
still rejects, just for an unstated reason -- turns the omission fixture red
on the message rather than passing on the exit status.

Select a resourceRules entry by comparing each resource name for equality,
and match versions as fixed strings rather than patterns. yq's contains()
compares array elements by substring, so a rule scoped to a subresource
would have satisfied a search for the parent: a rule naming tlsroutes/status
answers a query for tlsroutes and donates its apiVersions to the set the
guard checks coverage against. That direction is the wrong one for a
fail-closed check -- it makes coverage succeed where the real rule falls
short. No rule in this policy is scoped that way today.

Also pin apiGroups on the TLSRoute document in the chart suite. The HTTPRoute
document has always had that assertion; the asymmetry meant the case named
after both kinds only checked the group for one of them.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-03 22:17:04 +03:00
Aleksei Sviridkin 7ba22369d4 fix(cozystack-basics): match every served TLSRoute version in the hostname policy
The tenant route hostname policy names v1alpha2 alone while the vendored
Gateway API bundle serves tlsroutes at v1, v1alpha2 and v1alpha3: three
served versions against one named. The gap is not new. The v1.5.1 bundle
this package shipped before and the v1.6.1 one it ships now serve the
same three versions, with the same two marked deprecated, so nothing in
this change created or widened it -- a bundle bump is simply when
someone last opened the file.

Coverage of the two unnamed versions rested entirely on request
conversion. matchConstraints.matchPolicy is unset on both rules in this
file and so defaults to Equivalent, which rewrites a request submitted
under any served version into the one the rule names before the policy
sees it. That is why there is no bypass to demonstrate, and equally why
the arrangement is brittle: the conversion has nothing left to convert
into once no named version is served, and a rule that selects nothing is
not caught by failurePolicy: Fail, because nothing failed -- there was
simply no match.

Name all three served versions, which is the shape the HTTPRoute rule
directly above already uses. The defect is a version list drifting from
the bundle it must track, so a longer list on its own would not catch
the next drift: add a check that reads the served versions out of the
vendored bundle and fails when a policy does not cover every one of
them, in either direction. Both rules get an exact-value assertion in
the chart's own suite, so the coverage check and the pinned value fail
for different reasons and neither stands alone.

The header comment being replaced claimed TLSRoute stays at v1alpha2
through Gateway API v1.4-v1.5 and that its promotion to v1 was still
upstream-only. TLSRoute has been served at v1 since v1.5.0, so that was
already false, and false in the direction that hid this gap: a reader
who believed it had no reason to look at the version list at all.

Widening the rule then falsified every remaining place that restated
which versions it matches -- the template's own header, the
security-model layer list in packages/extra/gateway/README.md, and the
name of the test case that asserts the value. Rather than resync three
copies of a list that will move again, they now say which kinds are
gated and point at the rule and its guard for the versions.

A comment in the same test file now states only the behaviour it
guards: an earlier form of the CEL used `... ? true : (...)`, which
allowed routes in any tenant-* namespace whose host label was missing
or scrubbed.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-03 22:17:01 +03:00
Aleksei Sviridkin eb175aa5a9 chore(gateway-api-crds): vendor the Gateway API v1.6.1 CRD bundle
Re-renders the experimental kustomization at v1.6.1. The bundle still
carries the same twelve CRDs plus the safe-upgrades admission policy
pair, and every CRD keeps its helm.sh/resource-policy: keep stamp.

TCPRoute and UDPRoute gain a served v1 that takes over as their
storage version; their v1alpha2 stays served, now marked deprecated.
No other kind changes its served versions, so the group/version pairs
the Cilium operator and the tenant gateway controller resolve --
GatewayClass, Gateway, HTTPRoute and GRPCRoute in v1, ReferenceGrant
in v1beta1, TLSRoute in v1alpha2 -- are all still served.

Schema changes land on fields this repository does not set:
sessionPersistence.idleTimeout is dropped from HTTPRoute, GRPCRoute
and XBackendTrafficPolicy, HTTPRoute gains a lower bound on
retry.attempts and rejects duplicate retry.codes, and ReferenceGrant
now requires spec. Gateway infrastructure annotations, frontend CA
certificate refs and TLSRoute hostnames all get larger upper bounds.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-09-03 21:07:57 +03:00
Andrey Kolkov e94fcf8219 feat(backups): RabbitMQ definitions backup via a dedicated strategy driver (#3990)
## What this PR does

Adds backup/restore support for the managed `RabbitMQ` application.
RabbitMQ's cluster operator ships no backup CRD and RabbitMQ has no
consistent online export of message payloads, so this backs up broker
**definitions** (vhosts, users, permissions, queues, exchanges,
bindings, policies, parameters) over the management HTTP API
(`GET`/`POST /api/definitions`) and stores them in the system bucket.
Message payloads and point-in-time recovery are intentionally out of
scope; a full-data backup would be a Velero volume snapshot, which is a
separate strategy.

It is delivered as a **first-class
`strategy.backups.cozystack.io/Rabbitmq` driver**, following the
Altinity pattern (a dedicated driver that renders a one-shot Job for an
HTTP-API engine):

- `reconcileRabbitmq` / `reconcileRabbitmqRestore` render the strategy's
`PodTemplateSpec` — `curl` `GET`/`POST /api/definitions`,
pushing/pulling the JSON to S3 with `curl --aws-sigv4` — as a
`batch/v1.Job`, covering both restore flows: in-place (into the source
broker) and to-copy (into a separate, differently-named RabbitMQ in the
same namespace, authenticating with the target's own default-user
credentials and reading the source's object key).
- A **Go-read precondition** gates each run on the `RabbitmqCluster`
reporting `AllReplicasReady` (read through a small `rabbitmqtypes`
partial type), so a not-yet-ready broker surfaces a precise
`Ready=False` wait on the BackupJob/RestoreJob rather than a Job that
starts and fails against an unreachable management API.
- The produced `Backup` records the stored object's location on
`status.artifact.uri`, rendered from the strategy's
`spec.artifactURITemplate`. The `Backup` CRD has no status subresource,
so this persists on Create, mirroring the Etcd driver.

Dispatch is wired in both controllers (plus
`supportedBackupStrategyKinds`), `rabbitmqtypes` is registered in the
scheme, and RBAC grants read on `rabbitmqclusters.rabbitmq.com` (the Job
itself is already covered by the existing `batch/jobs` grant). It ships
`cozy-default-rabbitmq` (a `$bucketName`-gated `Rabbitmq` strategy) and
routes `apps.cozystack.io/RabbitMQ` to it in the `cozy-default`
BackupClass, which also enrols it in the `DefaultObjectsGate`. The
RabbitMQ application chart is left untouched — it had no prior backup
wiring, and the strategy carries the S3 coordinates itself.

Unit tests cover the driver's pure helpers (applicationRef validation,
cluster-name derivation, the readiness-message builder, artifact-URI
rendering, parameter round-trip) and the reconcile precondition gate
(defers with `Ready=False` and no Job until the cluster is ready, then
proceeds). A runnable example under `examples/backups/rabbitmq/` and an
honest-roundtrip Chainsaw e2e under `hack/e2e-chainsaw/rabbitmq/` seed a
sentinel vhost, back it up, drop it, and assert it round-trips through
object storage on both restore flows.

The Job pod reuses the existing `platform-migrations` image (curl +
ca-certificates) under a new
`backupStrategyController.rabbitmqBackupClientImage` key, stamped in
lockstep with `chBackupClientImage` in `packages/core/platform/Makefile`
so the repository stays pinned at a single digest
(`hack/image-pin-consistency.bats`). A dedicated slim image built by
this package is a follow-up.

**Validation:** `go build`/`go vet`/`gofmt`/`go test
./internal/backupcontroller/... ./internal/template/...`, deepcopy +
Rabbitmq CRD generation, `helm template` proving the CRD-ordering gate
both on and off, `helm lint`, and `helm unittest` (18/18). The backup +
in-place + to-copy roundtrip was verified end-to-end on a dev cluster (a
real management-API export and a real S3 `--aws-sigv4` upload/download
against SeaweedFS), so the driver's core contract is exercised on a real
cluster; the in-CI Chainsaw run is the remaining automated evidence. The
object-cleanup, terminating-namespace teardown and readiness-deadline
paths added during review are unit-tested (including `Reconcile`-level
finalizer retention and the terminating-namespace release) but not yet
cluster-exercised. (Note found while testing: RabbitMQ 4.2 is OOMKilled
on the `nano` preset, so the example/e2e use `c1.small`.)

### Screenshots

Not applicable — no UI changes.

### Downstream repositories

Walked the trigger map against the diff. This is an additive backup
strategy plus a new strategy CRD; no downstream repository is
mechanically forced by the change. A documentation page for RabbitMQ
backup on [cozystack/website](https://github.com/cozystack/website)
would be a reasonable follow-up, but it is not required by this diff —
leaving the boxes unticked for a human to decide rather than ticking "no
downstream" to clear the checklist.

### Release note

```release-note
feat(backups): add RabbitMQ definitions backup/restore via a dedicated strategy.backups.cozystack.io/Rabbitmq driver; it gates each run on the RabbitmqCluster being ready and records the stored object on Backup.status.artifact.uri.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added RabbitMQ backup and restore support for definitions, including
vhosts, users, queues, exchanges, bindings, policies, and related
settings.
  * Added in-place and restore-to-copy workflows with readiness checks.
  * Added scheduled and on-demand backup examples.
  * Integrated RabbitMQ backups with the default backup configuration.
* Backups store definitions only; message payloads and point-in-time
recovery are not included.
* **Documentation**
  * Added a complete RabbitMQ backup and restore demonstration guide.
* **Tests**
* Added end-to-end coverage for readiness and backup/restore round
trips.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-03 16:03:46 +04:00
Andrey KolkovandClaude Opus 4.8 6afe69b35b chore(etcd-operator): retrigger CI
The prior CI run stalled in the self-hosted runner queue for days and never
scheduled; an empty commit forces a fresh run on the same tree.

Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-03 15:55:17 +04:00
Andrey KolkovandClaude Opus 4.8 5896484c4a docs(etcd-operator): fix stale role-rules vendoring path in rbac.yaml comment
The manager ClusterRole header still pointed at the upstream config/rbac/role.yaml
path, which 404s at v0.5.5; the rules now live in
charts/etcd-operator/files/manager-role-rules.yaml (already corrected in the
Makefile and rbac_test.yaml).

Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-03 15:55:17 +04:00
Andrey KolkovandClaude Opus 4.8 85cbb3d7cb feat(etcd-operator): bump to v0.5.5 and vendor EtcdDefragPolicy CRDs
v0.5.5 ships the EtcdDefrag and EtcdDefragPolicy API types (a controller
plus a cron that stamps scheduled defrag runs). Vendor both CRDs into the
etcd-operator-crds package and grant the manager ClusterRole the verbs the
new controllers need: defragpolicies read/patch, defrags full CRUD, the two
new /status subresources, and cluster-scoped event emission.

Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-03 15:55:17 +04:00
Aleksei Sviridkin 88bae1fd98 docs(agents): require the neutral Assisted-by: LLM trailer (#4041) 2026-09-03 15:54:44 +04:00
myasnikovdaniil 2b99853e6d fix(networking): prevent webhook admission outages during namespace churn (#3997)
## What this PR does

Moves ingress-nginx and kubeovn-webhook pod deny rules to source-side
cilium policies, and keeps a target-side `ingressDeny` on the webhook
pods beside them, so enforcement does not depend on which component
performs the service DNAT. Pod access to both webhooks stays denied,
ingress-nginx keeps exception for konnectivity agent (now keyed on its
ServiceAccount instead of `k8s-app` label that any pod can set on
itself), and external `world` deny stays as static target rule.

Source-side rules deny only backend port. That covers ClusterIP traffic
too, because cilium translates service to its backends before it
enforces egress policy, and that ordering exists because
`packages/system/cilium/values.yaml` sets `kubeProxyReplacement: true`
against upstream default of `"false"`. The value is operator-reachable,
so the target-side rule is what keeps the deny working if it is ever
flipped, and two render-time beacons make a flip visible instead of
silent. Naming the frontend with `toServices` is not possible, and
templates now say why, because the construct looks correct and fails
silently: cilium resolves `toServices` only in allow rules
(`pkg/policy/k8s/service.go` walks `spec.Egress`, never
`spec.EgressDeny`), so such `egressDeny` gets non-nil empty L3 selector
from `pkg/policy/utils/parserules.go`, and
`pkg/policy/types/policyentry.go` treats that as wildcard when rule
carries `toPorts`.

This does not remove the namespace-churn map writes. The target-side
rule carries a cluster-spanning peer, so those writes come back. What
changes is that they stop being load-bearing, because whichever end is
degraded the other still denies. Cost is policy-map cardinality on the
two webhook endpoints, and residual risk is an endpoint near
`MaxEntries` getting the inverted add-before-delete order
(`pkg/endpoint/bpf.go:1168-1177`).

Both policies also change kind and name: namespaced
`CiliumNetworkPolicy/ingress-nginx-admission-webhook` and
`CiliumNetworkPolicy/kube-ovn-webhook` become cluster-scoped
`CiliumClusterwideNetworkPolicy/ingress-nginx-admission-webhook-c5d94713a379`
and `kube-ovn-webhook-c2bdbc146112`. Anything keyed on the old
namespaced objects goes quiet with no error.

## Tests

Package suites pin the deny shape per package, one rule per spec with a
destination selector cilium can actually resolve, and pin the peer shape
of the target-side rule against the two substitutions that would deny
the API server (`fromEntities: [all]`, and dropping the peer). Checked
by mutation, all of those go red. `matchRegexRaw` and `notMatchRegexRaw`
are not used, helm-unittest 1.0.3 ignores both.

`packages/system/kubeovn-webhook` had no `test` target in Makefile, and
`hack/helm-unit-tests.sh` finds suites by probing `make -n test` per
package, so all three of its suites never ran in CI, including port
agreement guard that exists to catch a drift failing open. Added the
target, plus `.PHONY: test` to it and to cilium.

E2e churn case proves webhooks stay available. Second case proves they
are still unreachable from pods, and that deny is confined to webhook
port. Both now gate every verdict on the admission endpoints having
ready addresses before, during and after, on controller pod identity and
restart count being equal across the window, and on a positive control
where the same pods' own 443 must stay reachable. So a dead or flapping
backend fails as setup failure instead of reporting a deny that was
never enforced. First run of these cases reported a false verdict for
exactly that reason.

## Not verified

Original outage in #3983 was not reproduced in isolation, and its
symptom argues against the mechanism this PR was written around.
`connect: no route to host` is `EHOSTUNREACH`, and a policy denial
cannot produce it, since a deny drops after `connect()` has already
returned and the caller sees a timeout. On a ClusterIP that errno is
what cilium's socket LB returns with no usable backend, and socket LB is
active in host netns because `pkg/kpr/kpr.go:47-49` forces it whenever
kube-proxy replacement is set. Namespace deletion churns EndpointSlices
and service backend maps as well as identities, which is directly on
that path. So this change is justified as removing a structural weakness
and adding a topology-independent deny, not as a confirmed root cause
fix. What would settle it: `cilium-dbg monitor --type drop` on the
controller's node during the window, agent log checked for the
map-overflow warning, and the admission Service's ready backend count
across those seconds.

`world` deny that issue named as its first suspect is kept, on `cilium
monitor` evidence recorded when kubeovn-webhook policy was written - api
server arrives with identity `host` on same node and `kube-apiserver`
from another, never as `world`. That is one topology, a kube-ovn-chained
management cluster, and it is not re-derivable on cilium-primary
variants or on a Kamaji guest cluster without a run.

Whether cilium without kube-proxy replacement is a supported
configuration is a maintainer decision and is open. If it is not
supported, a `fail()` in the cilium chart is the right shape and should
follow this PR.

Refs #3983.

### Screenshots

Not applicable.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(networking): enforce the Ingress and Pod admission webhook deny on both the caller and the webhook endpoint, so it no longer depends on Cilium performing service load-balancing. Both policies become cluster-scoped: CiliumNetworkPolicy/ingress-nginx-admission-webhook and CiliumNetworkPolicy/kube-ovn-webhook are replaced by CiliumClusterwideNetworkPolicy/ingress-nginx-admission-webhook-c5d94713a379 and kube-ovn-webhook-c2bdbc146112.
```

## Summary by CodeRabbit

* **Bug Fixes**
* Improved admission webhook reliability during namespace deletion and
churn.
* Preserved webhook protection while allowing unrelated secure traffic
to continue.
* Corrected webhook access controls for both ingress-nginx and kube-ovn.

* **New Features**
* Added cluster-wide policy support with safer backend-port enforcement
and automatic capability detection.
  * Added safeguards for policy naming and configuration overrides.

* **Tests**
* Expanded automated coverage for webhook availability, access
restrictions, policy rendering, and end-to-end suite selection.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-03 14:10:45 +05:00
Aleksei Sviridkin 61026b732c docs(agents): require the neutral Assisted-by: LLM trailer
The attribution trailer exists for disclosure, not to advertise a
model. A model name in the trailer tells a reviewer nothing they can
act on, and it goes stale with every tool change, so the accepted form
is a single `Assisted-by: LLM` line however many tools took part.

A link to an assistant session is worse still: it rots, and it carries
nothing about the change itself, so no session trailer and no
transcript URL belong in a commit, a PR description, or a comment.

The commit body explains why a change is made; the diff already shows
what changed.

The hygiene rules for messages, trailers and in-code comments are
written down as review blockers for the same reason: a NOT LGTM on one
of them is then a stated rule, not a reviewer's taste.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-03 01:14:58 +03:00
Andrey Kolkov 6eba4d9fa7 test(backups): use .svc FQDN for rabbitmq backup e2e S3 endpoint
The reworked rabbitmq backup roundtrip reached the upload step and failed
TLS verification: `curl (60) no alternative certificate subject name matches
target hostname 'seaweedfs-s3.tenant-root'`. The demo Bucket, in-cluster
endpoint resolution and the copied SeaweedFS CA all worked; the remaining
gap is the hostname. The seaweedfs filer serving cert's SANs are
`*.tenant-root.svc` and `*.tenant-root.svc.cluster.local`, which the 2-label
`seaweedfs-s3.tenant-root` matches neither; the 3-label `.svc` FQDN does (and
still resolves via search-domain expansion). Point S3_ENDPOINT at
`https://seaweedfs-s3.tenant-root.svc:8333`.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-02 21:25:21 +04:00
Myasnikov Daniil 686a81a491 docs(networking): stop claiming the churn coupling is removed
Restoring the target-side pod deny put a cluster-spanning peer selector back on
the webhook endpoint, which is the exact thing the source-side rules were added
to take off it. The comments still said the coupling was "structural and
removed", and the release note still said deleting a namespace no longer
rewrites that endpoint's policy map. Both were true of the previous revision
and are not true of this one.

What ships now is two denies that fail in different directions: the source-side
one keeps enforcing while the webhook endpoint's policy is being recomputed or
has overflowed its map, the target-side one keeps enforcing where Cilium is not
the service load balancer, and neither subsumes the other. So the coupling is
not eliminated, it is made non-load-bearing -- whichever end is degraded the
other still denies, and the map writes are writes rather than an outage.
Eliminating it outright would mean dropping the target-side rule and accepting
a silent fail-open, which is the worse trade.

The chainsaw case's header claimed the deletion "briefly made fail-closed
admission webhooks unreachable". That is the unconfirmed hypothesis, and the
errno in the report contradicts it: EHOSTUNREACH cannot come from a policy
denial, which drops after connect() has returned. It now says what the case
actually asserts -- both webhooks still admit across a namespace deletion --
rather than claiming to reproduce a cause.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 20:52:06 +05:00
Myasnikov Daniil 0335421769 test(e2e): check every webhook registration, and name the controller Service
Three narrowing fixes to the two webhook cases.

The availability precondition resolved only the FIRST registration of each
webhook name (awk ... exit) while every matching registration fires on the
applies below, so an unrelated flaky ingress-nginx release would have aborted
an apply and been reported as a churn-induced outage. It now waits on all of
them and says which one it is waiting for. Split into require_service_up plus a
loop, indexed with sed rather than piped into `while read`, because a
pipeline's body runs in a subshell under dash and a `return` from it would not
reach the caller.

That split immediately produced its own bug, caught by the scenario harness:
POSIX functions share one scope, so the callee's `label="$1"` overwrote the
caller's copy and the second registration reported itself as "(1/2) (2/2)".
The callee now uses distinct names.

The positive control's Service was picked with `grep -v -- '-admission$' |
head -1`, which relies on ingress-nginx-controller sorting before
ingress-nginx-controller-metrics. Deterministic but undocumented, and the
control has to be the Service that actually exposes the https port, so
-metrics is now excluded explicitly.

Also drops a stale sentence: the churn pod's comment claimed the old
target-side policies coupled the webhook endpoint to identity churn, which
overstates a map write as an outage and is moot now that the target-side rule
is back. The pod is there because an empty namespace allocates no identity, so
without it the case would exercise nothing it names.

Eleven scenarios through both scripts under dash with a stubbed kubectl, all as
specified, the new multi-registration case included.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 20:45:22 +05:00
Myasnikov Daniil b78405cede fix(networking): keep a target-side pod deny beside the source-side rules
Restores the deny that does not care who performs the service DNAT, which is
what the source-side rules on their own gave up. Both policies now carry it on
their webhook endpoint for the backend port, alongside the retained world deny,
so losing cilium.kubeProxyReplacement degrades enforcement rather than removing
it.

I argued against this and was wrong. The argument was that a target-side rule
needs a peer covering pods, that such a peer either reattaches namespace churn
to a fail-closed endpoint or, as a wildcard, denies the API server. The second
half does not hold: a CRD-sourced fromEndpoints is never a true wildcard,
because for a clusterwide policy Cilium rewrites every selector to also require
k8s:io.kubernetes.pod.namespace to Exist plus the cluster label (Cilium v1.19.5,
pkg/k8s/apis/cilium.io/utils/utils.go:120-154 getEndpointSelector, whose own
comment at :141 says a wildcard must not become a truly empty allow-all
selector), and no reserved identity carries either label. So this rule denies
pods and leaves host, remote-node and kube-apiserver allowed. `fromEntities:
[all]` and an omitted peer are the shapes that would reach them.

The first half was a cost dressed up as a correctness problem. A cluster-
spanning peer is enumerated per identity, so the rule does add policy-map
entries on the webhook endpoints in proportion to pod-identity count and a map
write per namespace event. It does not add a drop window:
pkg/endpoint/bpf.go:1168-1177 adds entries before deleting them expressly to
avoid transient drops, and inverts that order only when the addition would
overflow the map, warning when it does. The residual risk is therefore
cardinality, not churn -- an endpoint near MaxEntries gets the inverted order,
and cozystack leaves endpointLockdownOnMapOverflow false
(charts/cilium/values.yaml:1250) so an overflow degrades rather than locking the
endpoint down. Both templates now say that where the rule is, rather than
leaving the cost unstated.

The peer is the load-bearing part and both ways of losing it render as a
plausible ingressDeny, so each suite pins it: rule count, the exact peer
selectors, no fromEntities on the pod rule, no fromEndpoints on the world rule.
Mutation-proved by making exactly the two substitutions that would deny the API
server -- fromEntities: [all], and dropping the peer -- in both packages; all
four render cleanly and all four go red.

The ingress-nginx rule carries the konnectivity-agent ServiceAccount exemption
for the same reason the source-side rules do: a Kamaji-hosted tenant cluster
reaches this webhook through that pod, so without it guest-cluster Ingress
admission breaks.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 20:45:10 +05:00
Myasnikov Daniil 3daa83f4b5 feat(cilium): surface a kube-proxy-replacement override instead of nothing
The pin added earlier guards a cozystack developer changing the default. It
cannot observe an operator override in the field, and neither can the e2e,
which only ever runs the default configuration -- so the path Ivan's review
actually names, addons.cilium.valuesOverride on a tenant cluster and the
cozystack.networking Package's cilium component on the host, produced no signal
at all. Both webhook deny policies name their webhook's backend port only,
which covers a pod dialling the ClusterIP solely because Cilium performs the
service DNAT ahead of its own egress hook, so an override degrades that half
silently.

A status-beacon ConfigMap rather than a fail(), following the -awaiting-etcd
beacon in packages/apps/kubernetes/templates/cluster.yaml. Whether Cilium
without kube-proxy replacement is a supported cozystack configuration is a
platform decision, and refusing to render the CNI over a weakened
defence-in-depth control is not a call these charts should make. The host
beacon fires on a bool false, on the string "false", and on the umbrella key
being deleted, because the subchart's own "false" default coalesces into the
parent. The tenant beacon fires on an explicit non-true override only, since
absence means the chart default applies, and it renders on the management
cluster next to the HelmRelease carrying the override -- which is why the
mismatch is detectable there at all rather than inside the guest cluster.

Also withdraws the argument the suite header used to carry, that a target-side
ingressDeny could not be re-added as a fallback. It was wrong, and the next
commit adds that rule. The header now records what the two halves each buy.

Makefile gets .PHONY: test for the reason this branch already argued for
kubeovn-webhook, which applies with more force here now that the suite is
load-bearing for two other packages: hack/package.mk's .PHONY list deliberately
carries only the targets that file defines, so a path named `test` would
otherwise make the target up to date and skip the recipe. Proven both ways.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 20:44:53 +05:00
scooby87 3a70f91423 fix(vm-disk): fail on storageClass/source edit of existing disk (#3969)
## What this PR does

Fixes #2985. Editing a `VMDisk`'s `storageClass` or `source` after its
DataVolume already exists was silently ignored:
`packages/apps/vm-disk/templates/dv.yaml` looks up the existing
DataVolume and, when found, reused its `spec` verbatim (because
`DataVolume.spec` is immutable in CDI). The edit therefore rendered as a
no-op — the typed CR accepted it and the HelmRelease reported success,
while the DataVolume/PVC kept the original spec and the source of truth
silently diverged from the cluster.

The template now builds the desired `source` into a shared
`$desiredSource` variable (used both for rendering and comparison) and,
when a DataVolume already exists, calls `fail()` with an explicit
message if the requested `storageClass` or `source` differs from the
immutable existing spec — mirroring the explicit size handling already
done in `pvc-resize-hook.yaml`. Storage growth is unaffected and still
flows through the resize hook.

`source` is also marked `@immutable` in `values.yaml`, so the generated
schema now carries the same `x-kubernetes-validations` transition rule
(`self == oldSelf`) that `storageClass` already had; `types.go`,
`values.schema.json` and the generated `vm-disk-rd` CRD are regenerated
to match.

### Verification

- `helm unittest` covers both the immutability guard (via a mocked
existing DataVolume through `kubernetesProvider`, since `lookup` needs a
cluster) and initial-creation source rendering for every source type —
12/12 pass.
- Verified live on a dev cluster with `helm install --dry-run=server`
against a real DataVolume: an unchanged edit renders, a `storageClass`
change fails, a `source` change fails, and a storage grow still renders
the resize hook without tripping the guard.

### Scope note

This PR only makes the drift visible and marks `source` immutable. Two
deeper platform findings sit outside its scope: (1) #2657 (pre-existing)
— the `apps.cozystack.io` aggregated apiserver does not evaluate the
`x-kubernetes-validations` CEL rules on update, so `@immutable` is a
no-op at admission for every Application kind, which is exactly why the
chart-level `fail()` — not the CEL rule — is the substantive guard here;
(2) #3971 — that same apiserver does not honour `--dry-run=server` and
persists the change (surfaced while testing this fix).

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the diff
file by file. The diff touches only `packages/apps/vm-disk/` (template,
`values.yaml`, regenerated `values.schema.json`, tests), the regenerated
`packages/system/vm-disk-rd/cozyrds/vm-disk.yaml`, and
`api/apps/v1alpha1/vmdisk/types.go`. No package was added, renamed or
removed; no `values.schema.json` field was added, renamed or removed
(only a validation rule on the existing `source` field); no default
changed; `README.md` is unchanged; the `cozyrds` file was regenerated,
not renamed. The terraform provider models fields, defaults and enums
but not CEL immutability, so it is unaffected. No downstream repository
is reached by this change.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(vm-disk): editing storageClass or source on an existing VMDisk now fails with a clear message instead of being silently ignored; delete and recreate the disk to change these immutable fields.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for blank, HTTP, image, disk, and upload VM disk
sources.
* VM disk sources and storage classes are now protected from unintended
changes, while storage expansion remains supported.
* Existing disks preserve their configured source and storage settings.

* **Bug Fixes**
  * Invalid or conflicting source configurations are rejected.
* Prevented silent no-op updates and incorrect immutability errors for
existing disks.

* **Documentation**
* Documented source and storage class immutability, including the need
to recreate disks when changing either setting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-02 18:40:24 +03:00
Myasnikov Daniil cbd0e78cad test(e2e): stop the webhook cases passing for the wrong reason
Both new cases could report a verdict they had not observed, and the first CI
run of them did exactly that against a crash-looping ingress-nginx controller.

admission-webhooks-survive-namespace-churn died in setup: the admission Service
had no ready backend, so the first apply -- which passes through that same
fail-closed webhook -- failed with "no route to host" before any churn ran, and
the failure read as the outage under test. It now asserts both webhooks are up
BEFORE the churn, resolving their coordinates from the webhook registrations
themselves rather than from a guessed release name, and says so in its own
words when they are not. Proven against the pre-change script with a stubbed
kubectl: with a dead admission Service the old script proceeded into the churn
(delete namespace, two applies, delete namespace, two dry-runs); the new one
stops and names the cause without touching the churn namespace.

admission-webhook-deny-still-enforced measured a corpse. Its `kubectl wait` did
observe both pods Ready, but the controller was flapping on a ~100s cycle and
the probe ran against a dead backend 19 seconds later. "Denied by policy" and
"no backend to route to" are the same observation from inside the probe, so the
two deny assertions would have reported success against a cluster with no
policy installed at all, and the port-scoping assertion reported a policy bug
that did not exist. The deny verdicts are now gated on the admission Service
having ready backends before the probe, on every poll while it runs, and after
it finishes, and on the controller pods neither restarting nor being replaced
across that window -- EndpointSlice readiness rather than pod readiness,
because a `kubectl wait` is what a flapping pod satisfies transiently. The
probe reports reachable/unreachable per destination and always exits 0; this
script holds the endpoint evidence and is the only place that can attribute a
refused connect, so anything undecidable now fails as a setup failure instead
of as a verdict.

Ten scenarios driven through both scripts under dash with a stubbed kubectl,
all behaving as specified: a dead backend, a backend dying mid-probe, a
controller restarting mid-probe, a probe that never completed, an unregistered
webhook, the deny lost, the deny widened to every 443 destination, and the
healthy path. The restart-mid-probe case is the one that used to produce a
confident false pass.

Two more things the same rewrite fixes. The step budget was 6m against inner
budgets summing to 420s, so a slow-but-healthy cluster was killed with no
verdict and no describe output; both steps now carry a timeout above their own
sum, per the rule in hack/e2e-chainsaw/.chainsaw.yaml, and the `kubectl wait`
whose 180s dominated that sum is gone. The churn namespace now carries a
running pod, because a Cilium identity is a set of endpoint labels including
k8s:io.kubernetes.pod.namespace: an empty namespace allocates no identity and
its deletion drives none of the UpdateIdentities path the old target-side
policies coupled the webhook endpoint to. And both server-dry-run Pods carry a
restricted-compliant securityContext -- never scheduled, but a Pod that
PodSecurity would reject aborts the check under set -eu and reads as the
webhook being down.

Refs #3983.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:34:51 +05:00
Andrey Kolkov 86fe8e5a7e feat(valkey): add managed Valkey service (#3380)
## What this PR does

Adds a **managed Valkey** service to the catalog, alongside managed
Redis.

> **Stacked on #3406** (redis-operator migration to freshworks-oss).
This PR targets that branch and depends on it; review/merge #3406 first.

### Why

Redis relicensed at 8.x to a source-available family (AGPLv3 / SSPL /
RSALv2), and the managed Redis app currently ships `v8: 8.4.0` with no
BSD-licensed option ≤ 7.2. That is a licensing risk for CNCF Incubation.
[Valkey](https://valkey.io) is the BSD-3-Clause fork of Redis 7.2.4,
governed by the Linux Foundation, and a drop-in replacement for Redis.
This is the same move Argo CD made —
[`cncf/foundation#750`](https://github.com/cncf/foundation/issues/750)
was closed with "no need for exception" after they migrated to Valkey.

### How it works

Valkey runs on the same operator as managed Redis. #3406 migrates that
operator from the archived `spotahome/redis-operator` to the
actively-maintained **freshworks-oss/redis-operator** fork, which adds a
native `spec.engine: Valkey` switch. With it, the operator runs
`valkey-server` / `valkey-cli` natively (in the launch command and in
its liveness / readiness / shutdown probes), so the chart uses the
official `valkey/valkey` image directly — **no Redis-compatibility shim
image**.

The app mirrors managed Redis: same shape/values (`replicas`,
`resources`/`resourcesPreset`, `size`, `storageClass`, `external`,
`version`, `authEnabled`), Sentinel-based HA, metrics exporter,
`WorkloadMonitor`s, dashboard resource map, and `ApplicationDefinition`
(`kind: Valkey`).

Versions offered: `v8 → 8.1.8` (default) and `v7 → 7.2.13` (the
BSD-3-Clause Redis 7.2 line, most conservative drop-in).

### How it was verified

- `helm template` renders cleanly for `v8`, `v7`, and `external: true`;
the `RedisFailover` carries `spec.engine: Valkey` and `image:
valkey/valkey:<version>`. `helm lint` shows only the pre-existing
icon-URL / `cozy-lib` warnings that managed Redis emits too.
- `helm template` of `valkey-rd` renders a valid `ApplicationDefinition`
(`kind: Valkey`, `plural: valkeys`, `category: PaaS`).
- Generated artifacts (`values.schema.json`, `README.md`, `types.go`,
`zz_generated.deepcopy.go`, the ApplicationDefinition `openAPISchema`)
reproduce with the pinned `cozyvalues-gen` v1.6.0 and `controller-gen`
v0.16.4 — root `make generate` and per-app generate both produce no
drift; `go build ./valkey/...` passes.
- A live-cluster e2e was not run from this environment; the Chainsaw
suite `hack/e2e-chainsaw/valkey/` (mirroring the redis suite) is
included and `hack/select-e2e.sh` maps the `valkey-application` source
to it, so it runs in CI once the freshworks operator image is built.

### Notes

- The Valkey logo (`packages/apps/valkey/logos/valkey.svg`) is a
placeholder in Valkey's brand colors, pending the official brand asset.
- A Grafana `db/valkey` dashboard was intentionally left out of this
initial PoC; it can follow.

### Screenshots

N/A — no UI changes.

### Downstream repositories

This adds a new managed-app kind, which likely warrants follow-ups in
`cozystack/website` (user-facing docs for the new service) and possibly
`cozystack/terraform-provider-cozystack`. No follow-up PRs are opened
yet — leaving these for a maintainer to decide, per the template
guidance.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
feat(valkey): add managed Valkey service — a BSD-3-Clause, Linux Foundation-governed alternative to managed Redis, running on the freshworks-oss redis-operator via native spec.engine: Valkey (versions 8.1.8 and 7.2.13)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added managed Valkey service support with high availability and
configurable replicas.
* Supports Valkey versions 7 and 8, persistent storage, resource
presets, authentication, and optional external access.
  * Added monitoring metrics and dashboard integration.
  * Added platform package integration and tenant administrator access.
* **Documentation**
* Added Valkey configuration guidance and storage-class immutability
documentation.
* **Tests**
* Added end-to-end coverage for deployment readiness, storage, and
replicas.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-02 18:18:42 +04:00
Myasnikov Daniil 6a289cf6d0 test(cilium): pin the service-LB assumption both webhook denies rest on
Both admission-webhook deny policies name only their webhook's backend port.
That covers a pod dialling the ClusterIP because Cilium translates a service to
its backends before it enforces egress policy -- in v1.19.5, bpf/bpf_lxc.c runs
__per_packet_lb_svc_xlate_4 and lb4_local() and only then tail-calls into
handle_ipv4_from_lxc where policy_can_egress4 evaluates the rule.

That ordering exists only while Cilium performs the DNAT, and the only reason
it does is cilium's kubeProxyReplacement: true, set against an upstream chart
default of "false". Set it false and the DNAT moves downstream of Cilium's
egress hook: the packet arrives with dst = ClusterIP, matches no endpoint
identity, and with default-deny off it is allowed. Both webhooks regain a pod
-reachable ClusterIP path while the backend deny keeps working, so nothing
fails and nothing is logged. Same shape as the port disagreement
namespace-annotation-webhook.port already exists to prevent, and it had no pin.

The suite asserts the rendered cilium-config key rather than .Values, so an
upstream bump that renames it or stops honouring the override is caught too,
and it spells out each of the five variant valuesFiles combinations from
sources/networking.yaml so a future variant file that overrides the value reds
here. Proven both ways: flipping values.yaml reds all six cases, and adding the
override to values-kilo.yaml alone reds exactly the cilium-kilo case.

The suite also records why a target-side ingressDeny is not available as a
belt-and-braces fallback. A target-side rule needs a peer selector covering
pods, and the peer selector is what carries namespace churn -- the failure
being removed here. pkg/policy/selectorcache.go UpdateIdentities notifies a
selector only when the churned identities were in its own cached selections,
and pkg/policy/l4.go IdentitySelectionUpdated then writes the policy map of
every endpoint the rule's endpointSelector selects, which for a target-side
rule is the fail-closed webhook pods. The one churn-free peer is the wildcard,
which Cilium collapses to a single ANY-identity entry and skips identity
updates for -- but ANY covers reserved:host, remote-node and kube-apiserver,
so it would deny the API server. There is no third shape.

Refs #3983.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:17:24 +05:00
Myasnikov Daniil 042ac2715b test(kubeovn-webhook): declare the test target phony, fix the port comment
hack/package.mk's .PHONY line does not carry `test`, so a path named `test`
next to the Makefile makes the target up to date and make skips the recipe.
Proven both ways: with the declaration the three suites run with such a file
present, without it make reports "'test' is up to date" and asserts nothing.
That is the same fail-open shape the port agreement guard exists for, and
hack/helm-unit-tests.sh only catches it after the fact.

The port comment named a ValidatingWebhookConfiguration. This package ships a
MutatingWebhookConfiguration, whose clientConfig.service omits `port` so the
API server defaults it to 443. The assertion was right; the sentence explaining
it was not, and it is the only place a reader learns why 443 is pinned.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:46 +05:00
Myasnikov Daniil 9f485a18ad docs(ingress-nginx): keep the policy rationale out of the render
The rationale above the CiliumClusterwideNetworkPolicy was a `#` block, which
Helm copies verbatim into the manifest: 53 comment lines shipped with every
render. The sibling policy added alongside it already uses `{{/* */}}`. Convert
this one to match; the text is unchanged.

Also state in the values description what `enabled: true` actually promises.
The template renders only while cilium.io/v2/CiliumClusterwideNetworkPolicy is
discoverable, so on a cluster without it the default emits no policy at all and
leaves no Event or render-time message behind. `enabled: false` is what
distinguishes an opt-out from an absent API.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:46 +05:00
Myasnikov Daniil f73b54fee9 test(e2e): assert the webhook deny is enforced and scoped to its port
The churn case added alongside it proves the webhooks stay AVAILABLE. Nothing
proved they are still UNREACHABLE from pods, which is the property the
policies exist for, and no test anywhere checked that the deny is confined to
the webhook port -- which is why a rule that denied all TCP/443 egress
cluster-wide passed every existing assertion.

Probe from tenant-root, where the host cluster's ingress-nginx lives and where
allow-internal-communication permits pods to reach it, so the deny is the only
thing that can block the webhook port; from the isolated tenant-test the probe
would pass for the wrong reason. Three assertions against the same pods and
the same tenant egress allowances, differing only in port: the controller's
own HTTPS port must stay reachable, the admission Service ClusterIP must not
be, and the backend port must not be. The first is the one that separates a
scoped deny from a wildcard.

Uses busybox:musl, already staged on every node by hack/e2e-prepull-images.sh,
with a securityContext that satisfies the restricted PodSecurity profile.
Service names, ports and the backend port are discovered at runtime rather
than hardcoded, since the release name differs between installs.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:45 +05:00
Myasnikov Daniil bdc172a648 test(networking): pin the deny shape and run the kubeovn-webhook suites
The suites asserted the toServices rule that produced the cluster-wide 443
deny, so they pinned the defect: every rendered-YAML assertion passed on the
broken policy, and removing the rule would have read as the regression.

Replace those cases with the invariant that was actually missing -- one deny
rule per spec, and its destination must be a selector Cilium can resolve
(toEndpoints present, toServices absent, spec count fixed). Verified by
mutation: reintroducing the toServices rule fails these assertions in both
packages.

Not used: matchRegexRaw / notMatchRegexRaw. helm-unittest 1.0.3 silently
ignores both -- a notMatchRegexRaw against a pattern guaranteed to be present
still reports pass -- so a document-wide regex guard would have asserted
nothing.

packages/system/kubeovn-webhook had no Makefile test target, and
hack/helm-unit-tests.sh discovers suites by probing `make -n test` per package
and skips any package without one. All three of its suites -- including the
port agreement guard, whose whole purpose is to catch a drift that fails open
-- had therefore never run in CI. Add the target; the sweep now reports
"Running tests in packages/system/kubeovn-webhook".

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:45 +05:00
Myasnikov Daniil 6b2b68a685 fix(networking): drop unresolvable toServices deny from webhook policies
Cilium resolves toServices only inside ALLOW rules: pkg/policy/k8s/service.go
walks spec.Egress in processRule, hasMatchingToServices and specHasToServices,
and never spec.EgressDeny. An egressDeny whose only destination is a
toServices therefore never gains a ToEndpoints/ToCIDRSet, and the
EgressDeny -> PolicyEntry conversion in pkg/policy/utils/parserules.go does not
pass ToServices to mergeEndpointSelectors at all. Its nil-guard only fires for
a non-nil empty slice, so with every other destination field absent it returns
make(types.Selectors, 0, 0) -- non-nil and empty, which
pkg/policy/types/policyentry.go documents as an implicit WILDCARD whenever the
rule carries toPorts.

The rule therefore did not deny the webhook Service on 443; it denied every
destination on TCP/443 for every endpoint its spec selected -- all pods outside
kube-system for ingress-nginx, and with endpointSelector {} every pod in the
cluster for kubeovn-webhook. Cilium deny beats allow, so the tenant
allow-external-communication policy could not offset it. Validation accepts
the construct (l3DependentL4Support marks ToServices supported), which is why
it failed silently.

Deny the backend port only. That already covers ClusterIP-addressed traffic,
because Cilium translates a service to its backends before enforcing egress
policy -- the same reason its own toServices support resolves to backend
selectors rather than the frontend. The kubeovn-webhook servicePort helper
existed only to feed the removed rule, so the Service goes back to a literal
443.

Also correct the four comments that documented the frontend deny as working,
record why the world deny is kept (cilium monitor showed the API server
arriving as host / kube-apiserver, never world), and state plainly that the
namespace-churn coupling is structural rather than a reproduced root cause for
issue 3983.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:45 +05:00
Myasnikov Daniil f224baae46 docs(networking): document webhook policy scope
Explain which callers the source-side Cilium policies select and how
the Service and backend ports are kept in agreement.

Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:44 +05:00
Myasnikov Daniil ad961f6de1 test(e2e): cover webhook admission after namespace churn
Exercise ingress and pod admission immediately after deleting a tenant
namespace, and route ingress-nginx changes to the gateway suite.

Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:44 +05:00
Myasnikov Daniil fc5dc66c61 fix(networking): isolate webhook policies from namespace churn
Move pod denials to source-side Cilium policies so namespace churn
cannot regenerate fail-closed webhook endpoints. Keep the external
world deny as a static target rule.

Assisted-By: GPT-5 <noreply@openai.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-09-02 19:15:44 +05:00
Andrey Kolkov 329ef9cf3f test(backups): make rabbitmq backup e2e self-contained via demo bucket
The rabbitmq backup roundtrip was the only backup e2e routing through the
platform cozy-default system bucket, whose advertised endpoint in the CI
sandbox is the unroutable s3.example.org placeholder, so the backup Job
failed with "Could not resolve host: s3.example.org". Mirror the
clickhouse/mariadb backup examples: provision a demo Bucket, project its
access keys into rabbitmq-backup-creds and copy the seaweedfs CA into
rabbitmq-backup-ca, and render a self-contained Rabbitmq strategy +
rabbitmq-default BackupClass whose curl uploads to the in-cluster seaweedfs
endpoint (S3_ENDPOINT override, CURL_CA_BUNDLE for the self-signed cert).

The in-Pod script is kept identical to the shipped cozy-default-rabbitmq
strategy, which is left unchanged; the demo only reuses its client image.

Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-09-02 15:37:30 +04:00