List emits a pending Tap for a tap source (labeled OCIRepository) that has no
materialized PackageSource yet, but Get looked up only the PackageSource and
returned NotFound for the same object. A dashboard detail view (list then
detail) of a connecting or collision-blocked tap would 404 instead of showing
the block reason. Get now falls back to the pending tap source by name, the
same lookup Delete already uses, so Get, List, and Delete agree.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The aggregated Tap resource no longer derives a community.<org>.<repo> name
from the connect URL. A connected repository is identified by its Flux source
(tap-<org>-<repo>), and the PackageSource(s) it materializes keep their own
declared names. Community is computed from the marketplace-tap label, and
disconnect is guarded by that label instead of a name prefix.
List also surfaces tap sources with no materialized PackageSource yet (a
connect in progress, or one blocked by a collision), carrying the materializer
error message, so a blocked tap is visible in the dashboard rather than
silently never appearing. The community. prefix constant is removed.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
From @kvaps's review of the untrusted-artifact path:
- Refuse to silently retarget: two repositories that share an org/repo path on
different registry hosts derive the same tap name. The API and CLI now reject
a connect when an existing source points at a different spec.url, instead of
blindly repointing the first at the second.
- Fail closed on integrity: verifyAndExtract now errors on an absent digest
(not just a mismatch), and the reconciler waits for status.artifact.digest,
so a tap artifact is never materialized without a verified digest.
- Centralize the load-bearing community. prefix in internal/marketplace/tapconst
(removing two duplicate consts and several literals across the CLI, API,
materializer, and dashboard) so the disconnect guard, the community badge, and
the anti-shadowing guarantee cannot drift.
- Split multi-document YAML with the apimachinery reader so a PackageSource
whose values contain a separator-looking line is not fractured.
(The short-name version-pin drop was already fixed in an earlier commit.)
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Address the remaining review findings:
- The materializer now prunes PackageSources it materialized from an earlier
revision that the current artifact no longer contains (including a single/multi
rename), so a removed package leaves the catalog; installed Packages stay.
- A repeat connect updates the fetched OCIRepository's spec, label and name
annotation instead of replacing the object, so the operator's finalizer and
materialized-revision annotation survive; covered by a test.
- OCIRepository write verbs move from the cluster-wide cozystack-api ClusterRole
to a namespaced Role in cozy-system (get/list/watch stay cluster-wide for the
watch cache), since the API only manages tap sources there.
- 'cozypkg validate --require-signature' now pulls the exact manifest digest
cosign verified, closing the tag-mutation window between verify and pull.
- safeJoin rejects any '..' path segment outright (defense in depth and to
satisfy the Zip-Slip scanner) on top of the existing containment check.
- Document why the index gate's merge/label steps need a workflow_run split for
fork pull requests.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
- Extract the per-component artifact-name convention into
internal/marketplace/naming and have the validator and the Tap backend
delegate to it, so the component-to-ApplicationDefinition linkage cannot
drift across copies.
- cozypkg tap only rolls back resources it actually created, so an idempotent
re-tap no longer deletes a pre-existing OCIRepository/PackageSource when a
later apply fails.
- Pin cozypkg and yq in the index publication gate so the auto-merge decision
runs on a reproducible toolchain.
- Document the trust model explicitly: tap validates structure but does not
verify signatures; verification is enforced by the index CI gate (pinned to
the recorded cosign identity) and optionally by Flux OCIRepository
verification at pull time.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The marketplace Tap create/delete run with the cozystack-api ServiceAccount,
which was granted only get/watch/list on cozystack.io and nothing on
source.toolkit.fluxcd.io, so connecting or disconnecting a tap returned
Forbidden on every cluster. Grant the ServiceAccount delete on
cozystack.io/packagesources and full management of
source.toolkit.fluxcd.io/ocirepositories.
Also make a pending or failed connect recoverable: when a tap has an
OCIRepository but no materialized PackageSource yet, Delete now falls back to
removing the labeled OCIRepository matched by its tap-name annotation, instead
of returning NotFound and orphaning a finalized source.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add server-side connect to complete the marketplace write path. Creating a Tap
records the intent only: the API validates the oci:// reference and creates a
labeled Flux OCIRepository (with an optional pull-credential secretRef), never
blocking the request on a registry pull, and returns a Tap whose status shows
materialization pending. The connect inputs (url, tag, secretRef) are write-only
and are never echoed back or returned on read.
A new tap materializer reconciler in the operator watches community-tap
OCIRepositories, and once source-controller has pulled the artifact, fetches the
tarball over the Flux artifact URL (digest-verified, size-bounded, path-traversal
guarded), parses the PackageSource(s) it carries, and materializes them under the
community. prefix with their sourceRef repointed at the tap's source. A finalizer
cleans up the materialized PackageSources when the source is removed, leaving
installed Packages in place.
The shared label/annotation keys live in internal/marketplace/tapconst so the API
and the operator cannot drift on the connect-to-materialize handoff.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Implement Delete on the Tap resource so the dashboard can disconnect a
community tap, mirroring 'cozypkg untap'. It refuses official (non-community.*)
sources, removes the PackageSource, and removes the backing Flux OCIRepository
only when no other PackageSource still references it. Installed Packages are
left untouched. The delete verb is not granted to the tenant catalog-read
roles, so disconnecting stays cluster-admin only through the delegated RBAC.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Add a cluster-scoped, read-only Tap resource to the core.cozystack.io API
group that powers the dashboard marketplace view. Each Tap is computed on read
from a PackageSource and the ApplicationDefinitions attributable to it (matched
by the component's assembled-artifact name), exposing the connected repository,
whether it is a community tap, its readiness, and the packages it exposes with
their catalog metadata. It is served through the existing aggregation layer, so
the dashboard consumes it with the same list/watch client it already uses, and
authorization is the delegated RBAC the apiserver already enforces.
Taps carry catalog metadata only and never a pull-credential Secret, so
browsing cannot leak a private tap's credentials across the tenant boundary.
Read access is granted to the tenant catalog-read roles alongside options.
Modeling the marketplace as a virtual resource in an existing API group avoids
a new API group and needs no etcd or CRD: the object is computed from cluster
state via the apiserver privileged dynamic client, mirroring the options
resource.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
## What this PR does
Labels and annotations under `internal.cozystack.io/` belong to the
platform. The lineage webhook stamps
`internal.cozystack.io/tenantresource` to decide whether a Secret is
visible through this API at all, and the cacert controller stamps
`internal.cozystack.io/tenant-ca`, which an `ApplicationDefinition`
selects on in `spec.secrets` to hand a trust anchor to the tenant. The
registry copied every caller-supplied key onto the backing Secret as is,
so anyone with write access to `tenantsecrets` could plant either marker
on a Secret of their choice, or strip one to take an object out of the
platform's view.
`tenantToSecret` is the only place the write verbs go through, PATCH
included, because `rest.Patcher` is `Getter`+`Updater` and the endpoint
installer serves PATCH out of `Update`. It now skips keys with that
prefix. The update target starts as a copy of the stored Secret, so a
platform key cannot be removed either. The guard is on the whole prefix,
so a platform key added later is covered without anyone touching this
code again.
Reads are unchanged. A tenant can see the platform's verdicts, it just
cannot write them. Keys are dropped silently instead of rejected,
otherwise a GET, edit and PUT round trip would break `kubectl edit` for
no gain.
Nothing writes through this API today. Every `tenantsecrets` grant in
`cozystack-basics` is `get`/`list`/`watch`, and the console only lists
and watches. That read-only RBAC is what keeps the hole inert, and it
lives in a different package from the registry that implements the write
verbs, which is why I would rather close it here than keep relying on
it.
Two things I found next to the diff and did not fix. The `Patch` method
on this type is never reached: its signature is the `client-go` shape
and matches no interface the endpoint installer asserts, so no request
lands there. I gave it the same guard instead of leaving the weaker of
two doors, but deleting it is probably the better answer and I would
rather that call was yours. A repair after the write is weaker than the
conversion anyway: the forged key is live between the patch and the
repair, and forcing the object back to a pre-patch snapshot drops a key
that a controller stamped inside that window. Second one, and the reason
none of this is easy to spot: `Update` passes `nil` where the handler
hands the storage the current object, so `hasUID` reports false and the
patch mechanism takes its create-new-object branch. For JSON Patch,
merge patch and strategic merge patch that branch returns `NotFound`, so
all three get a 404 against an existing TenantSecret. Apply is the
exception. `applyPatcher.createNewObject` builds a fresh object of the
kind and applies onto that, so an apply does reach `Update`, carrying
only the fields the caller sent, and the merge in `tenantToSecret` is
what keeps the stored labels. Passing `secretToTenant(previous)` instead
of `nil` is the fix and it belongs in its own change, since it makes the
other three patch types live. Tell me if you want issues for either.
Closes#3464
### Screenshots
Not a UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
Walked the trigger map against the diff: two Go files under
`pkg/registry/core/tenantsecret/`, no package added, no chart, values
schema, CRD, `ApplicationDefinition` field, namespace, `hack/` target or
release asset touched. Nothing in the map fires.
### Release note
```release-note
fix(api): TenantSecret writes no longer accept caller-supplied `internal.cozystack.io/*` labels and annotations, so a principal with write access to the API can neither plant nor strip the platform markers that decide tenant visibility on the backing Secret.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Strengthened protection of platform-managed metadata during
tenant-secret creation and updates.
* Prevented tenant-provided platform-owned labels and annotations from
overwriting protected values.
* Preserved standard metadata while hiding sensitive resource markers.
* Improved patch-based update handling and protected metadata
consistency across changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Labels and annotations under internal.cozystack.io/ are platform
verdicts, not tenant metadata. The lineage admission webhook stamps
internal.cozystack.io/tenantresource to decide whether a Secret is
visible through this API at all, and the cacert controller stamps
internal.cozystack.io/tenant-ca, which an ApplicationDefinition
selects on in spec.secrets to promote a trust anchor to the tenant.
The registry copied every caller-supplied key onto the backing Secret
verbatim, so a principal holding write access to tenantsecrets could
plant either one on a Secret of its choosing, or strip one to take an
object out of the platform's view. Only read-only tenant RBAC, which
lives in a different package, stood between a caller and that path.
tenantToSecret is the choke point every write verb goes through,
PATCH included: rest.Patcher is Getter+Updater, so the endpoint
installer serves PATCH out of Update. It now skips the platform keys,
and since the update target starts as a copy of the stored Secret,
ignoring the caller there also means a platform key cannot be
removed. Guard on the whole prefix rather than on a list of keys, so
a platform key added later is protected without anyone remembering to
come back here.
Delete the Patch method on this type. Its signature matches no
interface the endpoint installer asserts, so no request has ever
reached it, and as a write path it was strictly weaker than the
conversion: the raw patch hit the backing Secret before any guard
ran, leaving a forged key live in etcd -- visible to every controller
and to List -- until a second write put the platform keys back, or
for good when that second write failed. One door is better than two,
and the surviving one commits once.
Keys stay visible on read. A tenant may see the platform's verdicts,
and failing a write that echoes them back would break a
read-modify-write round trip for no gain, so they are dropped
silently instead of rejected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
TestWarnRemovedKubernetesFields pinned the helper in isolation, so deleting
either call site (rest.go Create/Update) left it green while an operator editing
a Phase 2 removed field would silently get no warning. These drive Create and
Update end to end and assert the warning surfaces; removing a call site now
fails loudly (verified by mutation).
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The Kubernetes CR still accepts and stores spec.nodeGroups, spec.nodeHealthCheck
and spec.maxNodeProvisionTime after the Phase 2 split (the adoption migration
leaves them on an upgraded CR rather than rewriting parent values mid-upgrade),
but they no longer render anything, so an operator scaling via
spec.nodeGroups.<x>.minReplicas silently got nothing.
Emit a non-blocking client-facing admission warning on Create/Update when any
of these fields is present, pointing the operator at the KubernetesNodes
resources. A hard rejection is intentionally avoided: it would break edits to
already-upgraded clusters that still carry the fields. Also enumerate all the
Phase 2 breaking changes (three removed fields, two KubernetesNodes
immutability rules, the 32-char cluster-name cap) in the chart README.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
A Kubernetes cluster's worker pools are separate KubernetesNodes releases
named "kubernetes-nodes-<cluster>-<pool>". validateNameLength only checked
the parent's own "kubernetes-" prefix budget (42 chars), so a cluster name
of 33-42 chars was admitted while every child pool release overflowed the
53-char Helm limit and could not render -- the migration then pinned and
skipped the pool, leaving the cluster with no way to add workers. Pre-split
nodeGroups had no such constraint, so this was a regression.
Add a kind-specific 32-char cap for the Kubernetes kind so the overflow
fails at admission on the parent CR instead of at render time on an
un-creatable child.
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Nothing asserted that a published definition name contains no slash,
which is why the core and sdn groups shipped as Go import paths in
v1.6.0 and v1.6.1 without anyone noticing. The same omission on a
future group would be just as invisible.
TestDefinitionNamesAreDottedModelNames fails on any definition name
containing "/", names the offender and points at the fix (the
+k8s:openapi-model-package marker plus an OpenAPIModelName method).
TestDefinitionRefsResolve is the failure mode itself: it builds each
$ref the way kube-openapi's builder does, JSON pointer escaped, then
resolves it the way a client does, trimming "#/definitions/" with no
unescaping, and reports any that dangles. It also checks the declared
Dependencies against the published names, which catches a group whose
types disagree with each other rather than uniformly.
Both assert the invariant instead of listing today's 20 names, so a new
group that omits the marker is caught rather than a changed count, and
both guard against going vacuous if the definition map is ever emptied
or stops covering cozystack's own types.
Both fail on the pre-fix tree: the first names all 20 offending
definitions, and the second reproduces the reported error verbatim,
down to
"github.com~1cozystack~1cozystack~1pkg~1apis~1core~1v1alpha1.OptionSpec".
Refs: https://github.com/cozystack/cozystack/issues/3806
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The aggregated apiserver publishes the core.cozystack.io and
sdn.cozystack.io models under their Go import path, so a $ref to any of
them does not resolve and client-side validation fails on every
resource of those groups, not only the one named in the error:
error validating data: SchemaError(…core/v1alpha1.Option.spec):
unknown model in reference:
"github.com~1cozystack~1…v1alpha1.OptionSpec"
Since Kubernetes 0.35 the apiserver's DefinitionNamer.GetDefinitionName
returns the model name it is handed verbatim; it no longer converts the
Go import-path form into the "friendly" reversed-path form. Whatever
name the generated openapi map uses therefore becomes both the
published definition key and, JSON pointer escaped, the $ref pointing
at it. A name containing "/" ships as a definition keyed on the raw
path while every reference to it spells each slash "~1", and clients
resolve a $ref by trimming "#/definitions/" without unescaping
(kube-openapi pkg/util/proto/document.go). The two spellings never
meet, the reference dangles, and validation of the whole document
fails.
Mirror b916d7475, which fixed the same disagreement for apps: add the
+k8s:openapi-model-package marker to each doc.go and declare
OpenAPIModelName for all 11 core and 9 sdn types, so
GetCanonicalTypeName, Scheme.ToOpenAPIDefinitionName and the generated
map key all agree on a slash-free dotted name. That takes the number of
slash-bearing definition names in the published document from 20 to 0.
The methods are hand-written rather than emitted by openapi-gen's
--output-model-name-file because that flag also rewrites
zz_generated.model_name.go inside the read-only apimachinery and
apiextensions module-cache packages the shared gen_openapi helper
always passes as inputs, which fails on any consumer including CI that
vendors deps from the cache. With the marker in place openapi-gen emits
Type{}.OpenAPIModelName() for every type it references, so a future
type without a method fails the build instead of silently
reintroducing a Go-path name.
The generated openapi is the output of the root `make generate`;
nothing else in the generated tree moved.
kubectl apply --validate has been broken against any cozystack-api
built after 2026-07-21 on every resource, and this shipped in v1.6.0
and v1.6.1. In e2e it surfaces as the platform install hanging on
cozy-backup-controller/backupstrategy-controller, which takes
tenant-root and everything behind cozystack-basics with it.
Refs: https://github.com/cozystack/cozystack/issues/3806
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The field docs already say healthCheckExprs are only evaluated under the
poller wait strategy, but that is only half the precedence rule. Upstream
evaluates them only when the Helm action itself has wait enabled
(helm-controller/api@v1.5.1 v2/helmrelease_types.go:193-199), so the
release.cozystack.io/helm-install-disable-wait annotation makes them a
silent no-op whatever waitStrategy says — no error, no warning, the
release just reports Ready as before.
This is reachable today: packages/system/kubernetes-rd already sets that
annotation and is a plausible candidate for CEL expressions in phase 3.
Document it on the ApplicationDefinition field (so it reaches the CRD
description) and next to HelmInstallDisableWaitAnnotation. ComponentInstall
has no disableWait, so its field doc is unchanged.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
After the k8s 0.35 bump, server-side apply of any apps.cozystack.io kind
(e.g. Bucket) fails at field management with "no corresponding type for
apps.cozystack.io/v1alpha1, Kind=Bucket", blocking backupstrategy-controller
and the whole install.
The apps API multiplexes every catalog kind (Bucket, Postgres, …) onto the
single Application Go type via Scheme.AddKnownTypeWithName. The SSA field
manager attaches a kind's GroupVersionKind to the Application OpenAPI model
only when the model's name matches the name the apiserver's DefinitionNamer
computed for it. Since Kubernetes 0.35 the DefinitionNamer keys that lookup by
Scheme.ToOpenAPIDefinitionName (the "friendly" reversed-path form,
com.github.cozystack…Application) while the model builder still uses the Go
import-path form. With no OpenAPIModelName the two disagree, the GVK extension
is never attached, and the type converter ends up with no apps kinds at all.
Declare OpenAPIModelName on Application/ApplicationList/ApplicationStatus so
GetCanonicalTypeName, ToOpenAPIDefinitionName and the generated openapi map key
all agree on the friendly form. This is done via the +k8s:openapi-model-package
package tag plus hand-written methods rather than code-generator's
--output-model-name-file flag, because that flag also rewrites
zz_generated.model_name.go inside the read-only apimachinery / apiextensions
module-cache packages the shared gen_openapi helper always passes as inputs,
which fails on any consumer (including CI) that vendors deps from the cache.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This PR bumps the runtime k8s libraries to 0.35 (k8s.io/apiserver,
kube-openapi 20250910) but hack/update-codegen.sh still pinned the
openapi code-generator at v0.34.1.
The 0.34.1 openapi-gen keys the version.Info definition by its Go
import path ("k8s.io/apimachinery/pkg/version.Info"), whereas
kube-openapi 0.35 now resolves that model via the new OpenAPIModelNamer
interface — version.Info{}.OpenAPIModelName() ->
"io.k8s.apimachinery.pkg.version.Info". The key mismatch makes the
aggregated apiserver fatal while building its root OpenAPI spec
("cannot find model definition for io.k8s.apimachinery.pkg.version.Info"
-> klog.Fatalf, exit 255), crashlooping cozystack-api. That takes down
the apps.cozystack.io aggregated API, so the Bucket kind can no longer
be resolved and the backupstrategy-controller HelmRelease install fails,
reddening E2E.
Bump the code-generator pin to v0.35.0 (matching the runtime) and commit
the regenerated artifacts. No hand edits to generated files.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Add first-class fields so a package/app can declare the Flux helm-controller
v1.5.0 kstatus health gate for the custom resource it wraps, so
HelmRelease.Ready means "the CR is actually healthy" instead of "helm applied
the CR" (#2642). API-owner approved in #3263.
- api: WaitStrategy (scalar, maps to HelmReleaseSpec.WaitStrategy.Name — a
deliberate simplification) + HealthCheckExprs ([]kustomize.CustomHealthCheck,
the upstream type verbatim) on both ApplicationDefinition.spec.release (the
apiserver path) and ComponentInstall (the operator path).
- config.ResolveWaitStrategy: shared by both HelmRelease builders. Couples the
default — healthCheckExprs imply the poller strategy when none is set, since
exprs are only evaluated under poller. This removes the "must also set
waitStrategy: poller" footgun and does not depend on the controller's global
default.
- thread through buildHelmReleaseSpec (operator) and
convertApplicationToHelmRelease (apiserver); the apiserver loads the typed
spec.release fields in start.go.
- regenerate deepcopy, the ApplicationDefinition and PackageSource CRDs, and
openapi. The backupstrategy CRDs also regenerate — benign k8s-0.35 PodSpec
description drift, not a functional change.
disableWait is intentionally not added (YAGNI): the only known need (the
kubernetes tenant-addon deadlock) was resolved child-side, and waitStrategy:
legacy is not equivalent (legacy still waits Helm-v3-style; disableWait skips
waiting entirely). Add it later when a concrete leaf needs a true no-gate
escape hatch.
Refs #2642
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Address review feedback from coderabbitai on pkg/version/version_test.go: the
env-unset case only skipped t.Setenv, so a COZYSTACK_VERSION inherited from the
test runner would make it skip the baked-fallback path it is meant to cover.
Unset the variable (restoring it on cleanup) for that case.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The operator's platform version was baked into the image at build time
(`-ldflags -X pkg/version.Version` from `git describe`), so a
`vX.Y.Z-rc.N` image literally carried the rc string. That blocks promoting
a release-candidate to stable by retag: the bits would still self-report
the rc version.
Resolve the version from the COZYSTACK_VERSION env var at runtime, falling
back to the build-time value when unset (local builds still work). The
installer chart threads `cozystackOperator.platformVersion` into the
operator Deployment as that env var, and `make manifests` stamps it into
the kubectl-apply release artifacts. The same image bits can now report any
release name, so promotion needs no rebuild.
The only behavioral reader is the telemetry metric
`cozy_cluster_info{cozystack_version=...}`; cozypkg/check-readiness keep
their own build-time Version and are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
parentPoolUsage listed ResourceQuotas through the aggregated apiserver's cached
client, which lazily starts a cluster-wide ResourceQuota informer. cozystack-api
has no RBAC to list resourcequotas, so the informer never synced and tenant
create/update of a bounded child hung until the request timed out (observed on a
cluster). Read quota usage through the direct (uncached) watch client instead,
scoped to the pool's member namespaces, so no cluster-wide informer is started,
and grant cozystack-api get/watch/list on resourcequotas for those reads.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
The declaration-time gate only subtracted sibling carve-outs from a parent's
budget, not the parent pool's current usage. A tenant already consuming most of
its budget could therefore carve a full-budget child, letting the sub-tree
collectively exceed the root's allocation.
Sum the current usage of the parent pool (its own namespace plus its unbounded
descendants, read from each member's rendered ResourceQuota.status.used) and
subtract it from the remaining budget, mapping each shorthand quota key to its
rendered limit key (renderedLimitKey). Boundedness is decided by declared
resourceQuotas so the result is independent of whether Flux has rendered a
freshly-created tenant's quota yet. This is best-effort at declaration time (a
snapshot; concurrent writes can briefly overshoot and running pods are never
evicted) — the tenant-quota controller remains the runtime backstop.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
Add a declaration-time gate in the aggregated apiserver: a child tenant's
declared resourceQuotas are carved out of its parent tenant's remaining
quota and may not exceed it. A child without a quota shares the parent
pool and is allowed; a parent without a quota does not constrain children.
This blocks quota escalation via sub-tenant creation (declaring a bigger
or unbounded child) without re-implementing pod-level admission.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
## What this PR does
Introduces a tenant-facing `SecurityGroup` resource in a new
`sdn.cozystack.io/v1alpha1` API group, served by the Cozystack
aggregated API server as a 1:1 projection of a `CiliumNetworkPolicy`. A
SecurityGroup attaches to one of the tenant's managed applications by
reference (`spec.targetRef: {apiGroup, kind, name}`) and declares
allow-list ingress/egress for that application. Tenants manage network
policy for their own apps entirely through
`securitygroups.sdn.cozystack.io`, without being granted any access to
the `cilium.io` API group.
The translation is synchronous and stateless — the API server converts
each SecurityGroup into a CiliumNetworkPolicy in the same namespace on
write and back on read; there is no controller. The backing policy's
`endpointSelector` is **derived** from `targetRef` (matched on the
application's lineage labels
`apps.cozystack.io/application.{group,kind,name}`), never copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
Policies owned by this API carry an `sdn.cozystack.io/securitygroup`
marker label that the storage always re-asserts and hides from the view,
so it only ever surfaces or mutates its own policies and never touches
platform-managed network policies.
The value is self-service network policy on a Cozystack-native,
RBAC-scoped surface, without exposing Cilium's version-coupled CRDs to
tenants. Because the selector is derived from an authorized application
reference rather than free-form tenant input, the RBAC boundary is
structural, not selector-validated: a tenant cannot express network
policy against arbitrary or platform-owned pods. The full design —
selector derivation from lineage labels, the structural boundary, the
additive-policy reality (a SecurityGroup widens a managed application's
allowed traffic but cannot sever its platform-managed management plane,
and equally cannot tighten below the platform baseline), the in-tree
CiliumNetworkPolicy mirror that avoids a Cilium Go-module dependency,
RBAC, and deferred extensions (raw-pod targeting, cluster scope,
reusable groups) — is in `pkg/apis/sdn/DESIGN.md`.
Implements the attachment model from #1614.
### Release note
```release-note
feat(api): add the SecurityGroup resource (sdn.cozystack.io/v1alpha1) — a tenant-facing, namespace-scoped firewall that attaches to a managed application by reference and projects 1:1 onto a CiliumNetworkPolicy with a machine-derived endpoint selector, letting tenants manage their application's network policy without cilium.io access.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the `SecurityGroup` API resource (`sdn.cozystack.io` v1alpha1)
with a namespaced projection backed by Cilium network policies.
* Enabled the SDN `APIService` and wired SDN routing, scheme, and REST
support.
* Expanded RBAC to include SecurityGroup access and required Cilium
policy permissions.
* **Documentation**
* Added a design document describing projection behavior and rule
semantics.
* **Bug Fixes**
* Improved API rule validation coverage for missing list-type fields.
* **Tests**
* Added SDN round-trip, fuzzing, RBAC, and comprehensive REST
create/get/list/update/delete/watch/resource-version tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The talos-reconcile Job is the only thing that creates the
TalosConfigTemplate the worker MachineDeployment's bootstrap.configRef
points to; CAPI blocks MachineSet creation until it exists, so tenant
workers cannot scale until the Job has run.
The Job was a post-install/post-upgrade Helm hook, which runs only after
Helm has applied the release's main resources and, when Helm waits, after
they become Ready. But the same release emits in-tenant addon HelmReleases
— notably cilium — whose readiness needs worker nodes, and those nodes
need the TalosConfigTemplate this Job produces. So when the main-resource
wait is in effect the install deadlocks: it waits for cilium to be Ready,
cilium waits for workers, the workers wait for the template, and the
template waits for this hook — which never runs because the install never
finishes the wait that precedes the hook phase. The install times out and
the MachineDeployment never scales. The release sets
helm-install-disable-wait to skip that wait, but its effect is unreliable,
which makes the failure bimodal: the template lands in ~46s when the wait
is skipped, and the install is stranded when it is not.
Render the Job and its RBAC/ServiceAccount/egress policy as ordinary
main-phase resources. The Job is then created during the apply itself, not
after the wait, and runs on the management cluster where nodes exist
(unlike the tenant cilium-operator), so it produces the template, the
MachineDeployment scales, workers join, cilium goes Ready, and the wait
(if any) closes within the install budget — independent of whether
helm-install-disable-wait takes effect.
The Job is immutable, so it carries a content-hash name suffix over its
rendered spec (mirrors the KubevirtMachineTemplate idiom in cluster.yaml):
an identical render keeps the name and Helm no-ops it, while any spec
change yields a new name and a fresh Job, so no spec change ever lands on
the same name and triggers an immutable-field patch error. The applied
TalosConfigTemplate keeps its stable <release>-<group> name, so an
unchanged render re-applies a byte-identical template — a CAPI no-op that
never rolls existing workers.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The ingress fromApp/fromSG projection is exercised end to end, but the
symmetric egress toApp/toSG path — toApp to a lineage-label
endpointSelector and toSG to the referenced group's membership-label
endpointSelector — had no test. A wrong label key there would silently
point an egress allow at the wrong endpoints. Pin the projection and the
reverse reconstruction with a Create+Get round-trip, mirroring the
existing ingress coverage.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The backing CiliumNetworkPolicy carries the membership finalizer, so
deleting a SecurityGroup is virtually always asynchronous: the policy
gets a deletionTimestamp and survives until the securitygroup-controller
strips the membership labels off member pods and removes the finalizer.
REST.Delete nonetheless returned (nil, true), reporting the object as
instantly deleted, so the endpoint emitted a Success status claiming
completion instead of surfacing the terminating object — a
rest.GracefulDeleter contract violation for any client that distinguishes
the two to detect pending teardown.
After the delete, read the object's state back through the direct
(uncached) client and report deleted=true only when it is actually gone;
while a finalizer holds it, return deleted=false with the terminating
object. Using the direct client avoids a stale cache-backed read
misreporting an async delete as instant, and a dry-run (which leaves the
object in place) is resolved from the object's authoritative finalizers.
Implement rest.MayReturnFullObjectDeleter so the DELETE endpoint is
advertised and serialized as returning the SecurityGroup object, matching
the generic registry.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A SecurityGroup with attachments but no ingress or egress rules exists
only to stamp its membership label on the attached applications' pods, so
other SecurityGroups can reference it as a peer. Its backing
CiliumNetworkPolicy carried only an endpointSelector, which the
cilium.io/v2 CRD rejects synchronously: the spec anyOf requires at least
one of ingress/ingressDeny/egress/egressDeny, so a selector-only policy
fails with "spec.ingress: Required value". A membership-only group
could therefore never be created.
Always serialize the backing policy's ingress section, emitting an empty
list when the group has no ingress rules. An empty list is the CRD's
documented no-op ("if omitted or empty, this rule does not apply at
ingress") and enableDefaultDeny defaults to false for a direction with
no rules, so the policy is schema-valid yet datapath-inert: it neither
allows nor denies, matching the additive, non-isolating semantics of an
empty rule list. The projection guarantees the slice is non-nil so it
never marshals to null, which the CRD (no nullable fields) would also
reject. The empty section collapses back to nil on read, so a
membership-only group round-trips with no ingress or egress.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Ingress/Egress field docs claimed "an empty list denies all ingress/egress",
which is a false security guarantee. A SecurityGroup projects onto a
CiliumNetworkPolicy that only ADDS allow rules, and Cilium policies are additive
over the tenant's blanket-allow baseline, so an empty list does not isolate a
pod — effective connectivity stays open. Deny / default-deny enforcement depends
on the default-deny baseline, which is separate future work.
Rewrite the API field docs to the accurate semantics, regenerate the OpenAPI so
the served schema description matches, and fix the one DESIGN.md sentence that
still asserted "denies all traffic" (it already contradicted its own safety
section, which correctly states the API cannot deny anything today).
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Two watch-proxy defects in the SecurityGroup REST storage:
- watch.Error events from the backing CiliumNetworkPolicy watch (e.g. a 410 Gone
for an expired resourceVersion) carry a *metav1.Status, not a
CiliumNetworkPolicy. The CNP type assertion dropped them, so the client saw a
cleanly closed stream and never performed the required relist. Forward error
events verbatim before the type assertion.
- The DELETED-event bypass the label selector needs (a policy whose labels mutate
out of the selector is deleted carrying non-matching labels, so dropping it
would strand cached clients) was wrongly applied to the field selector too.
Name and namespace are immutable, so a field-selected watch (metadata.name=sg-a)
received other objects' deletions and could corrupt filtered client caches.
Apply the field filter to every event type.
Both paths are covered by new tests driving a fake backing watch.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
> **Supersedes #2610.** Same work, rebased cleanly onto current `main`
so it now sits on top of #2867 (the `kube-controller-manager` VAP
type-checker fix). That fix is what was killing the base install in
#2610's E2E *before* the `kubernetes` app test ever ran, leaving the
Talos worker-bootstrap rewrite verified only by code reading. #2610's
67-commit branch is squashed here into 8 logical commits with original
authorship preserved (@kvaps and @IvanHunters as authors); the tree is
**byte-identical** to #2610's head — no code changes, just the rebase
onto `main` (to pick up #2867) plus history cleanup. The detailed review
history lives on #2610.
Phase 1 of the Kubernetes-app split design (see cozystack/community#8):
replace the Ubuntu+kubeadm worker bootstrap path with Talos, driven by
`cluster-api-bootstrap-provider-talos` (CABPT) and a
`clastix/talos-csr-signer` sidecar embedded in the Kamaji control-plane
pod. Existing tenants keep working — old machines roll out and get
replaced by Talos workers without manual intervention.
Highlights:
- Add CABPT (v0.6.12) as a second `BootstrapProvider` alongside the
existing kubeadm one in `capi-providers-bootstrap`.
- Bump the Kamaji control-plane provider with an upstream-bound patch
that exposes `KamajiControlPlane.spec.network.additionalServicePorts`,
used to publish trustd (50001/TCP) on the apiserver service.
- Generate Talos PKI (Ed25519 CA + trustd TLS) via cert-manager and
stable random Talos secrets (`token`, `clusterId`, `clusterSecret`,
`bootstrapToken`) using the helm lookup-and-reuse pattern.
- Materialise a kubeadm-format `bootstrap-token-<id>` Secret inside the
tenant `kube-system` via a Helm post-install/upgrade Job.
- Render a `TalosConfigTemplate` (worker machineconfig, `generateType:
none`) and switch the `MachineDeployment.bootstrap` reference from
`KubeadmConfigTemplate` to `TalosConfigTemplate`, gated on Talos secrets
and the Kamaji apiserver service being ready.
- Boot workers from the Talos openstack image via a CDI `DataVolume`
(`source.http.url` → `factory.talos.dev`), expose the system disk as
virtio-blk with `blockSize.custom: logical=512, physical=4096` so
4Ki-native block storage backends (LINSTOR/DRBD) play nicely with QEMU's
O_DIRECT writes and SeaBIOS still boots, drop the separate kubelet disk
(Talos lays out EPHEMERAL itself).
- Pin pod/service CIDRs in the worker machineconfig to the tenant ranges
(`10.243.0.0/16` / `10.95.0.0/16`) to avoid Talos's address-overlap
diagnostic against the host pod CIDR.
- Set `cluster.controlPlane.endpoint` and `cilium.k8sServiceHost` to the
Kamaji apiserver Service ClusterIP (looked up at render time), so
bootstrap survives the chicken-and-egg moment where tenant DNS does not
exist yet and the host CoreDNS does not serve the tenant zone.
End-to-end on a dev cluster: a fresh tenant brings up a Talos worker,
kubelet CSRs get approved (apiserver-client by Kamaji, kubelet-serving
by the talos-csr-signer sidecar), Cilium initialises with hostNetwork
against the apiserver ClusterIP, CoreDNS comes up, and the node
transitions to `Ready`.
### Release note
```release-note
feat(kubernetes): bootstrap tenant workers with Talos Linux instead of Ubuntu+kubeadm. Existing clusters roll over to Talos workers automatically; new tenants come up on Talos from the start. Powered by cluster-api-bootstrap-provider-talos and the talos-csr-signer sidecar in the Kamaji control-plane pod. Note: the tenant `kubernetes` HelmRelease now reports Ready as soon as Helm completes (Install/Upgrade DisableWait) and no longer blocks on worker/addon readiness — worker-rollout health is tracked by WorkloadMonitor, not the HelmRelease.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
**New Features**
* Added Talos worker configuration for `talos.version` and
`talos.schematicID`
* Introduced `nodeHealthCheck` (`maxUnhealthy`, `nodeStartupTimeout`)
for worker remediation tuning
* Added image overrides for `kubectl` and `talosCsrSigner`
* Exposed additional tenant control-plane Service ports support
**Breaking Changes**
* Removed Kubernetes v1.30; supported versions start at v1.31
* Renamed `nodeGroups.ephemeralStorage` → `nodeGroups.diskSize` with
consolidated Talos system-disk semantics
**Removals**
* Removed Ubuntu container-disk images and related containerd patch
mechanisms
* Air-gapped tenant workers temporarily unsupported during the Talos
transition
**Documentation**
* Updated chart docs, parameters, and GitOps upgrade guidance for
version bumps
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Rewrite the SecurityGroup design for the membership model: each group's own
identity label, attachments stored in a backing-policy annotation, the
REST/controller split, live fromSG/toSG references, the controller's hardening
invariants and the eventual-consistency window contract. Spell out that the
default-deny baseline flip and the membership admission webhook are out of
scope and why, and that membership relocates rather than fixes the "tenant
firewalls its own managed application" concern.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Give each SecurityGroup its own membership label
(securitygroup.sdn.cozystack.io/<name>) and select it from the backing
CiliumNetworkPolicy, instead of deriving the endpointSelector from a single
app's lineage labels. spec.targetRef becomes spec.attachments, so one
SecurityGroup can apply to several applications at once.
Peers move from free-form fromEndpoints/toEndpoints selectors to
fromApp/toApp (resolved to lineage labels) and fromSG/toSG (resolved to
another group's membership label, a reference the Cilium dataplane resolves
live). Attachments are persisted in a storage-owned CiliumNetworkPolicy
annotation, round-tripped and hidden from the SecurityGroup view like the
marker label. Validation now covers attachments and app/SG peers and rejects
names that collide with reserved Cilium entities.
The membership labels are maintained by a separate controller added in a
follow-up commit; this change covers the API types and the
aggregated-apiserver projection only.
BREAKING CHANGE: SecurityGroup spec.targetRef is replaced by
spec.attachments, and ingress/egress peers use fromApp/fromSG/toApp/toSG
instead of fromEndpoints/toEndpoints. The feature is unreleased, so no
migration is required.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The access-check helpers returned plain fmt.Errorf values, which the
apiserver coerces to 500 InternalError regardless of the actual failure.
A request without user identity now surfaces as 401 Unauthorized, and a
RoleBinding list failure as 500 InternalError with a typed Status body.
Get also repackages a NotFound from the backing Namespace lookup so the
Status body reports the tenantnamespaces resource instead of namespaces.
The Watch goroutine already logs access-check failures and skips the
event, so no untyped error escapes to the client from that path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
- talos-reconcile-job: emit a `---` separator inside the nodeGroups
range so a tenant with multiple node groups renders valid
multi-document YAML instead of two Jobs concatenated into one doc
- talos-reconcile + bootstrap-token jobs: add a restricted
securityContext (drop ALL caps, no privilege escalation, read-only
root fs) and CPU/memory requests+limits; drop the unused management
SA token mount on the bootstrap-token job, which only talks to the
tenant apiserver via the mounted admin kubeconfig
- migration 44: list Helm-managed resources by the
app.kubernetes.io/managed-by=Helm label, not meta.helm.sh/release-name
(an annotation, so it never matched as a --selector — the keep
annotation was silently never applied)
- cluster.yaml: fail fast on a malformed nodeHealthCheck.maxUnhealthy
rather than silently coercing it to 0 via `| int`
- config: make helm-install-disable-wait parsing genuinely
case-insensitive (EqualFold + trim) to match its documented contract
- capi-providers bootstrap/cpprovider Makefiles: guard the
patches/*.diff loop against an empty glob
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Parse a new release.cozystack.io/helm-install-disable-wait annotation in start.go and apply it as Install/Upgrade.DisableWait in rest.go, so helm-controller no longer blocks post-install hooks on in-tenant addon readiness. This breaks the reconcile-hook chicken-and-egg where the hook that creates the worker TalosConfigTemplate was gated behind addons that need workers to schedule. Covered by a config table test.
Co-authored-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Rewrite the SecurityGroup design doc for the targetRef projection:
selector derivation from lineage labels, the structural (not
selector-validated) RBAC boundary, and the additive-policy reality —
a SecurityGroup can only widen a managed application's allowed traffic,
never sever its platform-managed management plane, and equally cannot
tighten below the platform baseline. Records the deliberate caveats:
raw non-application pods are not targetable, and a targetRef to a
non-existent application matches zero pods rather than erroring.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update the REST storage fixtures and e2e bats to the targetRef shape, and
add cases for the new behaviour: the derived endpointSelector matchLabels,
the apiGroup default to apps.cozystack.io, an empty-targetRef rejection,
and invalid-label-value rejection (over-length or malformed kind/name/
apiGroup). The e2e asserts the dotted-key lineage labels serialise and are
accepted by the real cilium.io/v2 CRD, and that the view reconstructs the
targetRef.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace the tenant-authored endpointSelector on SecurityGroup with a
targetRef that references a managed application by {apiGroup, kind, name}.
The backing CiliumNetworkPolicy's endpointSelector is now derived by the
API server from the application's lineage labels
(apps.cozystack.io/application.{group,kind,name}) instead of copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
The in-tree CiliumNetworkPolicy mirror gets its own spec type carrying a
concrete endpointSelector; the ingress/egress rule lists still project
1:1. Reads reconstruct the targetRef from the derived matchLabels, which
is lossless because validation caps each component at a valid label value
below the lineage webhook's truncation threshold.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
validateCIDRs rejected a bare IP (10.0.0.1) because net.ParseCIDR
requires a prefix, but Cilium's CIDR.sanitize accepts a bare IP as a
single-host CIDR. That made the validator stricter than Cilium and
blocked a legitimate single-host fromCIDR/toCIDR rule, contrary to the
design goal of rejecting only specs Cilium would silently discard.
Mirror Cilium by falling back to a plain-address parse, and add a
regression test for bare-IP CIDR acceptance.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The resourceVersion de-dup filter dropped ADDED events whose RV was <=
the requested startingRV. During a sendInitialEvents replay the backing
WatchList intentionally re-emits existing objects as initial ADDED
events with RV <= startingRV, so the filter broke the initial-state
replay. Skip the de-dup while sendInitialEvents is set; it still applies
when resuming a plain watch from a resourceVersion. Add a regression
test covering sendInitialEvents with a non-zero starting RV.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Enforce in the REST storage that Create persists into the request
namespace and that Update/force-create targets the request name,
rejecting a conflicting metadata.namespace or metadata.name with a
BadRequest. The aggregated apiserver already rejects these before the
storage is reached, but binding them here keeps the storage
self-defending against a cross-namespace or off-target write regardless
of the calling path. Add regression tests that drive the storage
directly to lock the contract.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Describe the implemented model: SecurityGroup as a synchronous 1:1
projection of a marker-labelled CiliumNetworkPolicy served by the
aggregated API server, the marker invariant the storage enforces on every
write, the RBAC boundary it provides, the in-tree mirror that avoids the
Cilium module dependency, and the deferred attachment / cluster-scope /
validation extensions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
Install the sdn.cozystack.io API group and serve the securitygroups
resource from the SecurityGroup REST storage, register the static
SecurityGroup kinds, add the group to the server codec, and register the
in-tree CiliumNetworkPolicy mirror with the client scheme.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
The aggregated API server translates each SecurityGroup into a single
CiliumNetworkPolicy in the same namespace and back. Policies owned by the
SecurityGroup API are marked with the sdn.cozystack.io/securitygroup label,
which the storage always re-asserts on every write and hides from the
SecurityGroup view, so it only ever surfaces or mutates its own policies,
leaves platform-managed policies untouched, and cannot be orphaned by a
tenant overwriting the marker through spec labels. Mutating verbs run the
genericapiserver admission hooks, and updates replace labels/annotations
rather than merging them, matching Kubernetes PUT semantics.
To avoid importing the full Cilium module — which pins a Kubernetes
version incompatible with this project's apimachinery fork — the backing
object is a minimal in-tree CiliumNetworkPolicy mirror that reuses
SecurityGroupSpec as its spec body, making the projection a near-identity
translation in both directions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
Introduce the sdn.cozystack.io/v1alpha1 API group with the SecurityGroup
resource. SecurityGroup is a namespace-scoped, tenant-facing firewall
object whose spec mirrors the subset of the CiliumNetworkPolicy rule that
the abstraction exposes (endpointSelector, ingress, egress, ports, CIDR,
FQDN), so the resource can be projected 1:1 onto a CiliumNetworkPolicy
without granting tenants direct access to the cilium.io API group.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
## What this PR does
Implements the tenant Flux sharding design from #2817: a new
`flux-shard-operator` system package that spreads tenant HelmReleases
across multiple helm-controller shards, so a noisy tenant (e.g. a
HelmRelease stuck in infinite remediation) cannot degrade the others.
- **Placement controller** owns the tenant→shard assignment: the unit of
placement is the tenant (all of its HelmReleases share one shard label),
distribution is greedy least-loaded weighted by HelmRelease count, with
pinned tenants, threshold-gated rebalance, paced moves, and never moving
deleting tenants. Watches are metadata-only, so the controller does not
decode the helm-controller status-patch firehose. Assignments are
recorded on tenant namespaces; HelmRelease labels remain the source of
truth on restarts.
- **Shard runtime** is provisioned by the operator: `shardCount`
helm-controller Deployments cloned from the flux-aio `flux` Deployment
and sanitised, so the image and feature-gates stay version-synced
automatically. flux-aio itself is unchanged and keeps owning everything
without a shard key.
- **CREATE-only mutating webhook** (`failurePolicy: Ignore`) stamps the
tenant's shard onto every HelmRelease at admission, so each one is born
on the correct shard regardless of creation path; any miss degrades
gracefully to the placement controller.
- The hand-rolled `flux-tenants` Deployment is retired: the operator
drains it (backfill `tenants` → `shard<i>`) and deletes it once empty;
migration 44 removes a leftover only under the same guard.
Ships with `shardCount: auto` (the default): the operator sizes the
shard count from the tenant HelmRelease count — `K = clamp(ceil(H/100),
1, min(16, T))` with hysteresis (scale up past 120 HR/shard, down under
60) — so small clusters stay at one shard (≈ today's behaviour) and
large fleets shard out automatically. An integer value pins the count
explicitly.
One deviation from the design doc: the webhook/controller scope is the
`sharding.fluxcd.io/key` label rather than
`internal.cozystack.io/tenantmodule` — every tenant HelmRelease is born
with `key=tenants` (stamped by ApplicationDefinition `release.labels`
and chart templates), so the webhook rewrites it and the placement
controller continuously relabels stragglers; `tenantmodule` covers only
the 6 tenant chart modules.
### Screenshots
Not a UI change.
### Release note
```release-note
feat(fluxcd): tenant HelmReleases are now reconciled by operator-managed helm-controller shards; the flux-shard-operator balances tenants across `shardCount` instances and retires the legacy flux-tenants deployment
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced **flux-shard-operator** to shard helm-controller workloads
with deterministic tenant placement, optional rebalance controls, and
autosized shard count.
* Added a **CREATE-only mutating webhook** to stamp the correct shard
label on tenant HelmReleases.
* **Bug Fixes**
* Application updates now preserve existing **flux-shard** label
assignments to prevent unwanted shard re-stamping.
* **Chores**
* Added platform packaging for the new component, including an image
build step.
* Removed the legacy tenants deployment and added a migration to retire
it safely.
* **Tests**
* Expanded unit and chart tests for placement, provisioning, webhook
behavior, and configuration parsing.
* **Documentation**
* Added operator documentation describing configuration and metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolve go.mod conflict: keep main's go 1.26.4 and golang.org/x/* bumps, and keep gomodules.xyz/jsonpatch/v2 v2.4.0 as a direct dependency (imported by the flux-shard-operator admission webhook) instead of main's indirect entry. go mod tidy leaves go.sum consistent.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
IvanHunters