Commit Graph
130 Commits
Author SHA1 Message Date
Mattia Eleuteri 0c62bee4da feat(bucket): add a ui flag to skip the s3manager web UI
Every bucket ships an s3manager Deployment with its Service and an
Ingress or HTTPRoute on <bucket>.<tenant host>. Tenants that only use
the S3 API pay for a pod per bucket and for one DNS name and TLS
certificate per bucket, none of which they need.

ui defaults to true, so existing buckets keep their UI. With ui set to
false the bucket no longer needs the tenant apex either, since the UI
hostname was its only consumer in the bucket-system chart.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-10-12 13:09:23 +02:00
Aleksei Sviridkin 2af8b0c229 ci(build): fall back to direct, retry and cache Go module downloads (#4660)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

One transient error from proxy.golang.org is enough to fail an image
build or the `Unit & controller tests` job today. The error is an HTTP/2
stream reset (`stream error ... INTERNAL_ERROR`), and nothing retries or
falls back after it. This PR adds three things: a direct fallback, a
bounded retry and a module cache.

Image builds now fall back to fetching modules from their origin when
the proxy fails. They get `GOPROXY=https://proxy.golang.org|direct` as a
build arg from `hack/common-envs.mk`, and every Go builder stage
declares `ARG GOPROXY`. With a pipe Go goes direct on any proxy error.
With Go's default comma it does that only on a 404 or 410. The value
comes from `IMAGE_GOPROXY` and not from the host's `GOPROXY`, because
buildkit can't reach a mirror on localhost or a `file://` proxy. Every
workflow job that runs Go on the runner sets the same value as env.

Explicit module downloads in Dockerfiles are retried three times with a
growing pause. The loop is inline, in one form that the contract test
pins. I tried a shared script from a named build context first, but
buildx refuses any `--build-context` for the four Dockerfiles still
pinned to the `dockerfile:1.2` frontend. The images also build from
different contexts, so no single file can be copied in. Raising those
pins changes how those images build and needs a real build to check, so
I left it for a follow-up. Two alpine stages, token-proxy and
shadowbox's `ss-server`, left the download to `go build`. The alpine
golang images have no git, so the direct fallback can't fetch for them,
and they now get the same download step. The runner jobs use
`hack/go-mod-download.sh` with the same bound.

The unit test job now starts from a module cache instead of downloading
everything. It restores `GOMODCACHE` with `actions/cache`, keyed on
every `go.sum` in the tree, and then prefetches all tracked modules
through the retry, so a cache miss still works. A cache saved by a PR
run is visible only to that PR, but one saved on the default branch can
be restored by any PR
([docs](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching#restrictions-for-accessing-a-cache)).
That is why the build cache warmer on main writes the same key before
its image build. This way the first run of a PR, which is the one that
flakes, starts warm. The warmer restores only an exact key, so old
module versions don't pile up, and its Go steps can't fail the image
build. Both runner labels are Linux, so the key matches. Both also check
out under `/home/ubuntu/_work`, so `go env GOMODCACHE` should give the
same cache path, but I have not seen it on the warmer's runner yet. The
release jobs in `tags.yaml` and `promote-rc.yaml` get the GOPROXY env
only, no cache.

`hack/go-module-fetch-contract.bats` covers all three. Each test is red
on main, and I checked each guard with a mutation.

Closes #4591

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
ci(build): Go module downloads in image builds and CI jobs now fall back to direct fetching and retry on transient proxy.golang.org errors. The unit test job reuses a module cache written from main.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Reliability**
- Go module downloads now fall back to direct access when the public
proxy encounters errors.
- Builds and automated checks retry failed module downloads, helping
them recover from temporary network issues.
- **Performance**
- CI caches Go modules to reduce repeated downloads and speed up builds
and tests.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-05 14:14:47 +03:00
Aleksei Sviridkin a73b91e232 build(bucket): run the tests of the packages the patches touch
The image build ran go build only, and go build never compiles a
_test.go, so nothing ran the tests of the code the patches change. Run
go test in the builder stage on the packages the patches touch. The list
is read from the patches themselves, so a new patch is covered without
an edit to the Dockerfile.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 22:00:58 +03:00
Aleksei Sviridkin 548d33f01f build(images): fall back to direct and retry Go module downloads
A transient proxy.golang.org HTTP/2 stream reset fails an image build
outright: Go's default GOPROXY falls back to direct only on a 404 or
410. Image builds now get GOPROXY=https://proxy.golang.org|direct as a
build arg from hack/common-envs.mk, declared by every Go builder stage,
so any proxy error falls back to fetching from the origin. The value
comes from IMAGE_GOPROXY rather than the host's GOPROXY, which may
name a mirror that buildkit cannot reach. The alpine golang images
carry no git, so in their stages a module hosted on GitHub cannot be
fetched directly and the retry is the cover.

Explicit `go mod download` steps retry three times with a growing
pause. The loop is inline, in one form the contract test pins: the
images build from different contexts, so no shared script can be
copied in, and a named build context in the shared buildx arguments
makes buildx refuse every Dockerfile pinned to a frontend older than
dockerfile:1.4.

The two alpine stages that left the download to `go build` get the
same download step, because without git neither the fallback nor a
retry covered them.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-01 19:37:00 +03:00
Aleksei Sviridkin b2ae6b773a fix(tenant): fail fast on an empty tenant apex and correct gateway docs (#4538)
Closes #4536

## What this PR does

Two fixes to how tenant charts handle the tenant apex host
(`_namespace.host`), from an audit of what happens when a tenant's
`host` is changed or left empty.

**1. Fail fast on an empty tenant apex (was: silent malformed
hostname).**
Charts that publish a service compose their hostname as
`<sub>.<_namespace.host>` (for example `harbor.<apex>`, `s3.<apex>`,
`grafana.<apex>`). When the injected apex was empty and no per-app
`host` override was set, they emitted a malformed trailing-dot hostname
(`harbor.`) into Ingresses, HTTPRoutes and Certificate dnsNames with no
error. Only the gateway path guarded against this.

A new library helper `cozy-lib.ns-host-required` now sources the apex at
each hostname-emitting site and fails the render with a clear message
when the apex is empty and no override is set. It is scoped to the
injected state, so standalone renders (`helm template` / `helm lint`
with no `_namespace`, or an empty `_namespace` map), no-exposure
tenants, and conditionally emitted resources (external=false, the
kubernetes no-etcd beacon path) are unaffected. The two post-delete
cleanup hooks are left unguarded so an empty apex cannot block
uninstall.

Charts covered: harbor, postgres (external), qdrant (external),
kubernetes (control-plane endpoint), vpn, seaweedfs, bootbox, monitoring
(grafana, alerta, grafana OIDC client), and system/bucket (which also
gains the `cozy-lib` library it was missing, both the local subchart
symlink and its package-source registration).

**2. Correct the tenant `gateway` parameter documentation.**
The `gateway` description (values.yaml, values.schema.json, README, the
Go type, the tenant-rd CRD, and an inline comment in namespace.yaml)
claimed the chart auto-enables a Gateway for tenants whose apex is
derived from the parent (empty `host`) and leaves it off for custom-apex
tenants. The resolver `tenant.gatewayEffective` never inspects `host`:
an unset `gateway` key always means the tenant owns no Gateway and
inherits from the nearest ancestor that owns one. The unit tests already
pin this behavior. The description is rewritten to match and
regenerated.

**Behavior change.** A tenant with an empty injected apex and no `host`
override that enables one of the covered components now fails its render
(the HelmRelease reports NotReady) instead of deploying a broken
hostname. In practice the apex is always injected (the root tenant has
`root-host`, children inherit it), so this only trips a genuinely
misconfigured tenant.

Tests: a new `cozy-lib.ns-host-required` unit test, plus per-chart guard
tests for harbor, qdrant, kubernetes, postgres, vpn, bootbox, seaweedfs,
monitoring and system/bucket. All affected chart suites pass.

### Screenshots

Not applicable (no UI change).

### Downstream repositories

I walked the trigger map against the diff. The only touchpoint is the
tenant `README.md` gateway description, which the website release bot
regenerates automatically from `README.md` on a stable tag (no manual
follow-up). The terraform provider is not affected: this is a
description-only change to the existing `gateway` field, with no field,
default or enum change, and the tenant-rd CRD schema reaches the
provider as opaque JSON. Boxes left for a maintainer to confirm.

- [ ] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(tenant): tenant charts now fail rendering with a clear error when a tenant's apex host is empty and no per-app host override is set, instead of silently emitting a malformed hostname; the tenant `gateway` parameter description is corrected to match actual behavior (an unset `gateway` never auto-enables a Gateway).
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Host-dependent services now show a clear rendering error when neither
a tenant host nor an app host is configured, instead of producing an
invalid hostname.
* Tenants without `gateway: true` use the nearest ancestor’s Gateway,
whether their host is inherited or custom. Enable `gateway: true` for a
tenant’s own Gateway, IP, and certificate; this is required to make a
custom apex publicly reachable.
* **Documentation**
* Clarified Gateway inheritance behavior and when a tenant needs its own
Gateway.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-28 21:32:36 +03:00
IvanHunters 2fa51bf587 test(charts): isolate guard coverage for every guarded template
Add per-template guard suites so each guarded hostname site is exercised
on its own: harbor harbor.yaml and httproute.yaml, bootbox machines.yaml,
seaweedfs cosi-deployment.yaml, bucket deployment.yaml and httproute.yaml,
monitoring alerta.yaml, grafana httproute.yaml and the grafanaHost helper
via oidc-keycloak.yaml. Each suite renders only its target template, so
removing that file's guard reddens it. Also add a postgres positive case
that renders the Certificate with an apex set (external=true).

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 18:32:49 +03:00
IvanHunters 0f4d39f1ef test(charts): add empty-apex guard tests for the remaining guarded charts
Add host_guard_test.yaml for postgres, vpn, bootbox, seaweedfs,
monitoring and bucket. Each renders its guarded template with a
non-empty injected _namespace whose host is empty and no per-app host
override, and asserts the ns-host-required failure, plus a positive
render case with the apex set. Completes guard coverage alongside the
existing harbor, qdrant and kubernetes tests.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 16:01:31 +03:00
IvanHunters 37f59369d3 fix(charts): fail fast on empty tenant apex in host composition
Charts that publish a service compose their hostname as
<sub>.<_namespace.host>. When the injected tenant apex is empty and no
per-app host override is set, they used to emit a malformed trailing-dot
hostname (e.g. "harbor.") into Ingresses, HTTPRoutes and Certificate
dnsNames silently. Source the apex through cozy-lib.ns-host-required at
each hostname-emitting site so the render fails with a clear message
instead, matching the gateway path.

Covered: harbor, postgres (external), qdrant (external), kubernetes
(control-plane endpoint), vpn, seaweedfs, bootbox, monitoring
(grafana, alerta, oidc client), and system/bucket (adds the cozy-lib
subchart symlink it lacked). The guard is placed at the point each
resource is actually emitted, so standalone renders, no-exposure
tenants and the kubernetes no-etcd beacon path are unaffected. The
post-delete cleanup hooks are left unguarded so an empty apex cannot
block uninstall.

Behavior change: a tenant with an empty injected apex and no host
override that enables one of these components now fails its render
(HelmRelease NotReady) instead of deploying a broken hostname. In
practice the apex is always injected (root has root-host, children
inherit it), so this only trips a genuinely misconfigured tenant.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-09-28 15:14:45 +03:00
Aleksei Sviridkin d9cbbec5ff fix(build): cross-compile image builders on the build platform
The remaining Go builder stages, plus the kube-ovn, console and
piraeus-server builders, ran on the target platform. A multi-platform
build therefore ran the whole compiler under QEMU: slow, and for Go
unreliable, as the migration-controller Dockerfile already records a
"found bad pointer in Go heap" crash from exactly that. The builders
now run on $BUILDPLATFORM and produce the target architecture from
TARGETARCH.

Two builders only produced the right architecture because they were
emulated. token-proxy ran `go build` with no GOOS/GOARCH, and kube-ovn
calls upstream `make build-go`, which hardcodes GOARCH=amd64 even on an
arm64 build. On a native builder both would have shipped host binaries
inside the other architecture's image, and nothing fails until a node
of that architecture starts the container. token-proxy now passes the
target explicitly; kube-ovn picks build-go or its arm64 twin
build-go-arm from TARGETARCH.

The console builder emits static JS and linstor-server's debian/control
declares every package Architecture: all, so both run natively without
a target.

A bats test requires every stage that compiles to sit on a BUILDPLATFORM
FROM and to read TARGETARCH or TARGETPLATFORM on a RUN or ENV line. A
bare ARG declaration does not count, which is what let token-proxy
through.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-26 15:05:06 +03:00
Aleksei Sviridkin ec7387109e build(go): move the toolchain and builder images to Go 1.27.1
Go 1.26.4 carries standard-library advisories in net/http, net/url,
crypto/tls, encoding/asn1 and html/template that govulncheck reports
as reachable from the first-party binaries. The shipped stdlib comes
from the builder image, so the builder tags move together with the
go directive; the digests are the multi-arch indexes that golang:1.27,
1.27-alpine and 1.27-bookworm resolve to at 1.27.1.

api/apps/v1alpha1 keeps go 1.26.4. It is a published types-only
module, no binary here links its stdlib, and raising its directive
would force every consumer onto Go 1.27.

linstor-csi v1.11.2 pins grpc v1.81.0, which no longer builds on
Go 1.27: golang.org/x/net drops http2.TrailerPrefix there. Its image
now raises grpc to v1.83.2 before building, as the kamaji image does
for its advisories; the upstream unit tests pass on it.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 17:23:18 +03:00
renovate[bot] a7dfbe6f8c chore(deps): update container image & action digests
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 16:55:04 +03:00
Aleksei Sviridkin eef5fe6908 fix(platform): declare the injected _cluster values in system charts
Ten system charts cannot be rendered on their own. They read the
platform-injected _cluster channel, and two of them (bucket,
monitoring) also read _namespace, without declaring either key in
values.yaml. On a cluster the platform supplies both through the
cozystack-values Secret, so installs are unaffected, but rendering a
chart by itself dies before a single template is emitted: on "index of
untyped nil" for a missing _cluster, and on "nil pointer evaluating
interface {}.host" for a missing _namespace.

Declare the keys with empty defaults. Helm merges chart defaults under
the supplied values, so an injected _cluster still wins and rendered
output does not change. Each of the ten also gets a suite that renders
with nothing injected; drop a declaration again and that chart's suite
goes red.

This makes `helm template` succeed for all ten. `helm lint` also
reports a missing cozy-lib dependency on keycloak and monitoring,
which is a separate pre-existing defect and survives this change.

etcd-operator reads _cluster too and stays undeclared here: its single
call site already guards with `| default dict`, so the chart renders
standalone as it is.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-07 02:23:47 +03:00
Mattia Eleuteri f859b710a4 fix(backupstrategy-controller): repair the bucket credentials Secret from the gate, not a chart fail
Addresses the four findings on #3524.

**The `fail` in the bucket chart (MAJOR).** The <bucket>-<user>-credentials
Secret was made to `fail` the render while its COSI lookup was empty, so the
release would retry and converge instead of skipping the Secret permanently.
Helm cannot render a partial set, so that fail aborts the WHOLE
<bucket>-system release: the other users' Secrets, and the bucket UI
Deployment/Service/Ingress/HTTPRoute with them. One declared user whose
BucketAccess never provisions takes down every other user of that bucket,
and on an already-installed release parks it in Failed, blocking every later
upgrade.

**No per-bucket escape (MAJOR).** Rather than add one, remove the need for
it. The gate already forces the release that renders the lookup-gated
Strategy CRs; the credentials Secret is the same trap one release earlier,
and the gate reads that Secret to resolve the bucket name in the first
place. So while it is absent, or carries no bucket name, force the bucket's
<bucket>-system release instead of doing nothing. That cannot deadlock its
own precondition: the BucketClaim and the BucketAccess whose COSI Secret the
lookup reads are rendered unconditionally by the parent release, not by the
one being forced.

The two releases are throttled independently — they are forced in sequence
on a bootstrap, so a shared timestamp would delay the second by a full
MinForceInterval for no reason. Coordinates come from the chart under
provisionBucket, and are omitted on external S3 where the Secret is
admin-managed and no release renders it. No RBAC delta: patch on
helmreleases is already cluster-scoped.

The chart goes back to a partial render, with the comment now explaining why
the skip is not self-healing and why failing is the wrong lever, and a test
that renders the UI with every user unresolvable so a reintroduced `fail`
fails the suite. Tenant buckets keep their pre-existing behaviour;
generalising the repair to every bucket needs a controller that owns them
and is tracked separately.

**Stale gauge (MINOR).** cozystack_backup_default_objects_missing was only
written on the happy path, so the state the gate exists to catch reported 0
and the documented alert never fired. It is now written on the unresolved
path too — the credentials Secret counts as one of the objects the default
backups depend on, which is what it always was. On an API error the gauge is
deliberately left alone rather than flapping, so a new
cozystack_backup_default_objects_check_errors_total marks it stale, and the
doc pairs the two instead of overstating the gauge.

**Suspended releases (MINOR).** forceHelmRelease now does a point Get and
skips a release with spec.suspend: true. helm-controller ignores forceAt and
requestedAt while suspended (`cozyhr suspend` sets exactly that), so the gate
was re-stamping every MinForceInterval for the whole suspension and climbing
the force counter — which the runbook attributes to a render that is not
producing the objects, the wrong diagnosis. The skip is logged and not
counted, so a climbing counter keeps its documented meaning.

Reported-by: IvanHunters <xorokhotnikov@gmail.com>
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
Assisted-By: Claude <noreply@anthropic.com>
2026-08-07 11:10:41 +02:00
Mattia Eleuteri 51ad226706 fix(backupstrategy-controller): repair lookup-gated backup objects
The default Strategy CRs and the Velero BackupStorageLocation are
Helm-templated behind a `lookup` of the BucketClaim the same chart
creates, and the <bucket>-<user>-credentials Secret behind a `lookup` of
the COSI Secret. When those lookups are empty at install time the objects
were silently skipped, and helm-controller does not re-render a release
whose chart and values did not change (drift detection is off on
operator-generated HelmReleases), so the skip was permanent: clusters ran
for months with only BackupClass cozy-default and no Strategy CRs at all.
That also fail-closes the pre-adoption snapshot in the v1.6.0 etcd
migration, which reads the projected credentials Secret.

Add a DefaultObjectsGate runnable to backupstrategy-controller. Once the
bucket name is resolvable from the projector's source Secret, it checks
that every object cozy-default routes to exists and forces one real Helm
upgrade (reconcile.fluxcd.io/forceAt + requestedAt) when any is missing.
Helm remains the objects' only author; the gate only makes sure the render
that produces them actually happens. It cannot be a render-time `fail` in
this chart: the chart is the producer of the Bucket its own lookup reads,
so a failed render would deadlock the condition.

The bucket chart's user-credentials template CAN fail safely — the
BucketAccess that produces the COSI Secret belongs to the parent release
and that release retries forever — so make it fail loudly instead of
skipping, with requireUserCredentials=false as the offline-render escape.

Also correct the chart and doc comments that claimed Flux re-renders on
its interval, and document the two-release manual recovery for clusters
already affected.

Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
2026-08-03 17:16:55 +02:00
cozystack-ci[bot] 235d38dae1 Prepare release v1.6.0 (promoted from v1.6.0-rc.4)
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-07-22 13:00:10 +00:00
cozystack-ci[bot] d95685c427 Prepare release v1.6.0-rc.4
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-07-21 10:31:23 +00:00
renovate[bot] 5b9b420ffe chore(deps): update container image & action digests 2026-07-10 05:29:23 +00:00
Matthieu c8b180d91a [bucket] Add resources requests/limits to s3manager container
The s3manager Deployment in the bucket package had no resources block,
triggering a 'no CPU limit' conformance warning. Set limits 500m/512Mi
and requests 100m/128Mi — comfortable headroom for the lightweight S3
browser UI. Bucket is a standalone chart (no vendored upstream), so
the values are set directly in the template, matching the existing
pattern in csi-resizer and capi-providers.

Signed-off-by: Matthieu <matthieu@hidora.com>
2026-07-03 19:14:09 +03:00
Myasnikov Daniil 56108eca83 ci(renovate): pin base images and GitHub Actions to digests
Enable Renovate's dockerfile and github-actions managers with
pinDigests so OpenSSF Scorecard Pinned-Dependencies (currently 1/10)
can pass and supply-chain refs stay reproducible.

PR #2849 SHA-pinned actions in 13 workflows, but the check also scores
container base images -- 58 unpinned FROM lines across 27 distinct
images -- and left build-main.yaml / release-e2e.yaml unpinned.
Nothing kept the new action SHAs updated either, so they would
silently stop receiving security patches.

Enabling the managers lets Renovate open the initial pin PR (and pin
the two missed workflows) and keep digests fresh thereafter.
Digest-only refreshes and the initial pin are batched into a single
weekly PR to avoid a trickle of rebuild-triggering PRs on the shared
CI runner; real tag bumps stay individual for review.

Also pin s3manager's floating tags (golang:1, alpine:latest) to
concrete versions first, otherwise digest tracking churns endlessly.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-24 12:07:49 +05:00
Aleksei Sviridkin 34563ce8c7 feat(platform): add opt-in shared wildcard issuance for ingress path
On the default ingress-nginx path (gateway.enabled=false) a DNS-01 solver
still minted a per-host ACME certificate for every system service, hitting
Let's Encrypt rate limits once a deployment exceeds ~50 endpoints. Add an
opt-in publishing.certificates.wildcard toggle (default false) and compute,
in apps.yaml, an effective wildcard secret name plus a wildcard-issue
signal: when enabled with solver=dns01 and no operator-provided
wildcardSecretName, the platform targets one shared wildcard cert
(*.<root-host> + <root-host>) instead of per-host certs.

Issuance is off by default because a single-label wildcard does not cover
a custom service host outside *.<root-host> (e.g. a keycloak ingress.host,
harbor host, or grafana host pointed at another domain); enabling it would
otherwise switch such a host to the controller's default certificate and
serve the wrong certificate. The operator-provided (BYO) name always wins
and never triggers issuance; http01 cannot issue wildcards; and the
Gateway API path issues its own wildcard via the controller, so all three
are excluded. This reuses the existing wildcard-secret-name consumption
path, so no per-service ingress template changes are needed.

Part of #2811.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-22 20:01:40 +03:00
cozystack-ci[bot] e1aaa90ed3 Prepare release v1.5.0
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-06-22 10:55:37 +00:00
myasnikovdaniil aeaedca4a4 ci: consolidated platform & e2e stabilization batch (#2948)
## What

Consolidated landing branch for a set of interdependent platform/CI
stabilization fixes. It began as the source PRs in the table below (in
dependency order) and has since grown with review-driven fixes and a few
production-behavior changes — the full current inventory is under
**Scope** below. Landing them as one unit lets CI run against the real
combined state instead of a fragile bottom-up merge train.

## Source PRs (dependency order)

| # | Commit | Source PR | Addresses |
|---|--------|-----------|-----------|
| 1 | fix(capi): startupProbe on capi-controller-manager | #2946 | capi
crashloop during cert provisioning |
| 2 | test(e2e): bucket.bats port-forward + S3 client reliability |
#2944 | flaky bucket test |
| 3 | fix(seaweedfs): restore -lock BucketClass, s3 svc name, drop
volumeSizeLimitMB | #2943 | missing bucket/access classes |
| 4 | test(e2e): in-cluster Cilium endpoint-leak healer (install + apps)
| #2874 | cilium "IP already in use" leak |
| 5 | refactor(build): mode=max registry cache + main-only warmer |
#2938 | build cache |
| 6 | ci(build): isolate each PR build on its own ephemeral runner VM |
#2939 | shared-buildkit contention |
| 7 | fix(e2e): LINSTOR post-install waits on a single 15m deadline |
#2928 | LINSTOR provisioning timeouts |
| 8 | test(metallb): assert digest-pinned image form, not version
literal | #2873 | brittle metallb assertion |

## Why consolidated

These form a dependency DAG (verified from CI logs): e.g. 2943 needs
2946, 2938 needs 2874, 2928 needs 2938/2939, 2873 needs 2928. Tested
bottom-up, each lower PR runs with none of the fixes above it and can't
go green alone. This branch carries all of them, so CI runs against the
real combined state.

## Verification status

CI is green on the latest head (`5653c30`): full-suite E2E passes
end-to-end and is reproducible (2 of 3 runs on this SHA green). The one
red run was a LINSTOR tie-breaker / DRBD-metadata infra flake on the
sandbox, unrelated to the diff — a different environmental subsystem
failed each run (details in the comments below). The earlier 3-hour
crust-gather snapshot hang is fixed in `5653c30`.

## Scope beyond the original 8

The branch has grown past the 8 source PRs above with review-driven
fixes and a few production-behavior changes. For reviewer transparency,
the full set:

**Review fixes (@lexfrei review):**
- **B1 (blocker)** — `8f41910` converts the SeaweedFS
`-lock`/`-readonly` BucketClasses and the s3 service-name override into
`patches/`, wired into `make update`, so a re-vendor no longer drops
them.
- **FU1** — `cc99230` grants the cilium leak-healer `delete` on pods
(both delete remedies were RBAC-forbidden).
- **FU3** — `014de3f` adds a unit test for the HelmRelease update
conflict-retry path.
- **FU4** — `6deddc6` makes the leak-healer refuse a disconnect for any
non-terminal owner phase, not just `Running`.
- **FU7** — `f3914d2` sets `persist-credentials: false` on the
build-cache checkout (SHA-pinning is handled repo-wide in #2849).

**Production-behavior changes (not in the table above):**
- VPA `updateMode: Auto → Initial` for etcd (`6e9ff90`) and monitoring
(`0997105`), plus `vmselect`/`vmstorage` `minAllowed` floors — stops
install-time eviction churn.
- cozystack-api: `retry.RetryOnConflict` on the Application→HelmRelease
update path (`728f676`).

**E2E robustness follow-ups:** per-test crust-gather snapshots
(`50e5f94`), EtcdBackupSchedule wait (`763b85e`), tenant-node
single-deadline wait (`9e43d45`), harbor BucketClaim 10m budget
(`d7f244f`), tenant API via LoadBalancer (`9beda4e`), crust-gather pin
(`95f42aa`).

**Not addressed (by decision):** FU2 — the s3 Service keeps
`seaweedfs.name`-based naming via patch (rendered output unchanged;
cozystack renders `fullname == seaweedfs`, so it matches the
`componentName`/`fullname` siblings in practice). FU6 — fork PR build
push is handled by the existing mirror-to-same-repo-branch workflow.

## Relates to

#2946 #2944 #2943 #2874 #2938 #2939 #2928 #2873 — if this lands, those
can be closed; otherwise they remain the granular per-PR review path.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added CI build cache warming for main branch builds.
* Added a best-effort in-cluster Cilium endpoint self-heal watchdog for
e2e installs.
* Extended SeaweedFS COSI with object-lock support plus separate
readonly access.

* **Improvements**
* Reduced e2e install flakiness with a shared readiness deadline, safer
waits, and improved cleanup/diagnostics capture.
* Prevented install-time churn by switching VPA update mode to
**Initial**.
  * Added container startup probes and updated SeaweedFS volume sizing.

* **Tests**
* Improved e2e robustness (etcd backup schedule waiting, digest-pinned
image checks, and corrected S3 port-forwarding).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-22 09:19:22 +05:00
Myasnikov DaniilandClaude Opus 4.8 d3958c5bcb refactor(build): mode=max registry cache with a main-only warmer (#2938)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-18 10:40:05 +05:00
Aleksei Sviridkin 9e3ae7447b fix(ingress): keep tls section in operator wildcard mode
Dropping the whole tls block in wildcard mode disabled the forced
HTTP-to-HTTPS redirect: ingress-nginx only issues the 308 redirect when
TLS is enabled for the Ingress, while a host without a tls section is
still served on 443 with the default certificate but stays reachable
over plain HTTP. Keep the tls section with the host list and omit only
the per-host secretName, so ingress-nginx falls back to the operator
wildcard configured as the default SSL certificate and the redirect
stays enforced, matching the non-wildcard behaviour.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-10 03:18:42 +03:00
Aleksei Sviridkin b1f5d9579c feat(ingress): serve operator wildcard via default SSL certificate
When publishing.certificates.wildcardSecretName is set, the root
tenant's ingress-nginx controller now serves the operator-supplied
wildcard Secret as its --default-ssl-certificate, and the platform
service Ingresses stop requesting their own per-host ACME certificates.

The flag is set only on the publishing controller (Release.Namespace ==
expose-ingress) because ingress-nginx loads the Secret from its own
namespace; child-tenant controllers cannot reach a Secret in another
namespace, so they keep the existing ACME path.

Each service Ingress (dashboard, keycloak, grafana, alerta, bucket,
linstor-gui, harbor, seaweedfs S3, bootbox) drops its cert-manager
cluster-issuer annotation, the http01 challenge-class annotation, and
its tls block in wildcard mode; nginx then falls back to the default
certificate for those hosts. Other annotations and routing are
unchanged. Each file keeps its existing render guard intact, and the
annotations block collapses cleanly where the cert annotations were its
only content.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-10 03:08:41 +03:00
Myasnikov Daniil 9868c43e53 fix(build): read --cache-from from ghcr :latest so PR builds warm-start
PR #2711 switched PR builds to push only :pr-<N>-<sha> tags (PUBLISH_FLOATING=0) to stop concurrent-push 409 conflicts, but left --cache-from pointing at $(REGISTRY)/<img>:latest. PR builds set REGISTRY to the per-CI registry (iad.ocir.io) and never publish :latest there, so every PR build's registry cache lookup 404s and rebuilds cold.

Add a CACHE_REGISTRY knob (default ghcr.io/cozystack/cozystack, where release builds do publish :latest) and point every --cache-from at it. Push targets ($(REGISTRY)) and the unique per-PR tags are unchanged, so the #2711 anti-conflict behaviour is preserved while PR builds regain a warm cache from the last release.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-09 14:24:36 +05:00
Andrey Kolkov f17ead3d26 feat(platform): add default backupclass
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-05-28 13:49:59 +04:00
Aleksei Sviridkin 3756f5c459 feat(networking): Gateway API support via Cilium (supersedes #2213) (#2470)
## What this PR does

Replaces ingress-nginx as the default ingress layer for cozystack-native
services with **Cilium Gateway API**, materialised per-tenant via a new
`TenantGateway` CRD reconciled by `cozystack-controller`. Supersedes
#2213 / #2208. Stacked on top of #2468.

## Architecture

The chart renders one `gateway.cozystack.io/v1alpha1 TenantGateway` CR
per tenant that owns its own Gateway. The cozystack-controller
reconciles the actual `Gateway`, per-tenant `Issuer`, `Certificate`
(wildcard or per-listener), and HTTP→HTTPS redirect `HTTPRoute` from
there. Helm does not render `Gateway` or `Certificate` directly — that
prevents the Helm-vs-controller race on `Gateway.spec.listeners` that
route-driven listener additions would otherwise cause.

```
┌── TenantGateway CR (rendered by extra/gateway chart for owning tenants)
│
└──▶ cozystack-controller reconciles:
        ├─ Gateway (with dynamic listeners, label-based allowedRoutes)
        ├─ Issuer (per-tenant ACME account)
        ├─ Certificate(s) — wildcard + per-child SANs in DNS-01, per-listener in HTTP-01
        ├─ HTTPRoute (controller-owned http→https redirect)
        ├─ Namespace label patching (cozy-* system namespaces from spec.AttachedNamespaces)
        └─ Watches HTTPRoute / TLSRoute attachments to materialise listeners
```

## Inheritance: tenants attach by default, opt into their own Gateway

A tenant gets a dedicated Gateway (own LB Service, own LB IP, own
Certificate) only when it explicitly asks via
`tenant.spec.gateway=true`. Every other tenant in the tree publishes
through the Gateway of the nearest ancestor that owns one — same shape
as the existing `_namespace.ingress` inheritance.

The `apps/tenant` chart writes a `namespace.cozystack.io/gateway` label
on each tenant namespace, carrying either the tenant's own name (when
owning a Gateway) or the inherited ancestor name (when inheriting). The
Gateway's listener `allowedRoutes.namespaces.selector` is keyed on that
label, so the same selector admits routes from every namespace pointing
at the owner — descendants and cozy-* system namespaces alike.
cozystack-controller patches the same label onto every namespace in
`tgw.Spec.AttachedNamespaces` and garbage-collects labels it wrote when
an entry is removed from the attach list (annotation-gated GC so labels
written by the apps/tenant chart are never stripped).

In DNS-01 mode, the controller extends the parent's wildcard Certificate
with `<child-apex>` + `*.<child-apex>` SANs per inheriting child, and
adds a `*.<child-apex>` HTTPS listener per child apex referencing the
same cert. Without this expansion the parent's single-level wildcard
cannot match a child route's hostname (`harbor.alice.example.org` is two
labels past the parent's `*.example.org`).

Opt-out into own Gateway is the right choice when: a tenant needs its
own LB IP (DNS pinned, firewall rule), the apex is not derived from the
parent (custom `host` value the ancestor's cert can't cover), or the
tenant wants its own ACME account / cert authority. Otherwise leave
`gateway` unset and inherit.

## End-to-end flow

1. Platform: `gateway.enabled=true` toggles cert-manager solver,
switches per-system-app templates from Ingress to HTTPRoute / TLSRoute.
2. Per tenant: explicit `tenant.spec.gateway=true` renders a
`TenantGateway` CR and triggers the controller to materialise the owner
Gateway. Tenants with the field unset inherit through
`_namespace.gateway` propagation in the tenant chart.
3. Adding a published app under any tenant (owner or inheriting) is just
deploying its HTTPRoute. The controller picks up the hostname, adds a
per-listener Certificate (HTTP-01) or extends the wildcard cert's SANs
(DNS-01), and updates the owner Gateway's listeners.

Defaults stay legacy: `gateway.enabled=false`,
`tenant.spec.gateway=false`. Existing clusters upgrade unchanged.

## External IP allocation — mechanism-agnostic

The per-tenant Gateway's auto-created `LoadBalancer` Service draws its
IP from whatever LB allocator the cluster admin has configured at the
platform layer — same shape as ingress-nginx today. The tenant API stays
mechanism-agnostic — no `gatewayIP` field, no allocator-specific
manifest in the tenant chart. Cozystack itself ships MetalLB installed
but does not render any `IPAddressPool` / `L2Advertisement` /
`BGPAdvertisement` from this chart; admins set up the allocator that
suits their environment (MetalLB pool with L2 / BGP, Cilium LB-IPAM with
announcer, robotlb against a cloud provider, or
`Service.spec.externalIPs` pinning).

If a tenant needs a specific address (DNS already pinned, firewall rule,
etc.), the operator pre-allocates it on the admin side: either
pre-create the Service with `loadBalancerIP` set, or hand the tenant a
reference to a named admin-managed pool. Per-Service IP uniqueness is
the allocator's responsibility — same as for any other LoadBalancer
Service.

## Cert mode: HTTP-01 (default) vs DNS-01 (opt-in)

`publishing.certificates.solver` selects how the controller sources TLS
certs.

**HTTP-01 (default)** — per-app HTTPS listener + per-app Certificate.
New apps require zero platform-side config: deploy the HTTPRoute,
controller does the rest. Inheriting children's hostnames pick up
per-listener certs the same way the owner's own apps do.

**DNS-01 (opt-in)** — single wildcard Certificate (`<apex>` +
`*.<apex>`) covering every published app under the apex, plus per-child
SANs (`<child-apex>` + `*.<child-apex>`) for every inheriting tenant.
Pick this for clusters where many apps share the apex and Let's Encrypt
rate limits matter. The DNS provider account must be able to write TXT
records under every apex level the parent serves — for deeply-nested
inherited children that requires zone delegation or a wide-scope
provider credential.

| `publishing.certificates.dns01.provider` | Required
`publishing.certificates.dns01.<provider>` keys |
| --- | --- |
| `cloudflare` (default) | `cloudflare.secretName`,
`cloudflare.secretKey` |
| `route53` | `route53.region`, `route53.secretName` (and `accessKeyID`
if not on IRSA) |
| `digitalocean` | `digitalocean.secretName` |
| `rfc2136` | `rfc2136.nameserver`, `rfc2136.tsigKeyName`,
`rfc2136.secretName` |

The platform chart wires every provider's keys into `_cluster.dns01-*`;
the per-tenant gateway chart and the cluster-wide `cluster-issuers.yaml`
both read them.

## Strengthens admission coverage on apps.cozystack.io/*

The `cozystack-tenant-host-policy` probe in `gateway.bats` initially
looked like a flake — the apply that was supposed to be VAP-rejected
just succeeded. Investigation traced it to the custom REST handler at
`pkg/registry/apps/application/rest.go`. genericapiserver hands every
storage method a callback into the validating admission chain:
`createValidation` for Create, `updateValidation` for Update,
`deleteValidation` for Delete. `Update` was correctly invoking its
callback since 23e399bd, but `Create` and `Delete` accepted the
parameter and never called it — admission was silently bypassed for both
verbs on every `apps.cozystack.io` resource (Tenant, MariaDB, Postgres,
Kubernetes, …).

This PR wires the missing `createValidation` and `deleteValidation`
calls into Create and Delete. Every ValidatingAdmissionPolicy and
ValidatingWebhook against `apps.cozystack.io/*` now fires on all three
verbs as the API contract requires. Per KEP-3488 aggregated APIServers
are responsible for enforcing admission on their own kinds;
cozystack-api is configured to do so via
`genericoptions.NewRecommendedOptions`, the only thing missing was the
explicit `*Validation` callbacks in the custom REST.

## Security — five guards, grouped by what they defend

Tenants in Cozystack interact with the platform exclusively through
`apps.cozystack.io/*` resources (Tenant, Bucket, Kubernetes, …) served
by `cozystack-api`. Tenant RBAC (`cozy:tenant:*` aggregated to a
RoleBinding in the tenant's own namespace) does not grant write access
to `gateway.networking.k8s.io/*`, core `Namespaces`, or
`cozystack.io/Package`. The protections below split into three groups by
who they defend against:

**Tenant-user-input gates.** Layer 4 (`cozystack-tenant-host-policy`).
`Tenant.spec.host` is the user-supplied field that surfaces as a
security boundary at the hostname layer; gated on every Create / Update
via `cozystack-api`'s admission chain.

**Defense-in-depth.** Layers 1, 2, 5, 7. These do not protect against
tenant-user input (tenants don't hold the relevant RBAC). They guard
against bugs in cozystack-controller / Flux, supply-chain compromise of
an app chart that emits Gateway API resources, and confused-deputy
mistakes by a cluster admin. Fail-closed via `failurePolicy: Fail` +
`validationActions: [Deny]`.

The five layers themselves:

1. **Listener `allowedRoutes` label selector** —
`namespace.cozystack.io/gateway = <owner-tenant-name>`
(controller-written for cozy-* via `AttachedNamespaces`, Helm-written
for tenant namespaces via the tenant chart, both feeding into the same
Gateway selector).
2. **`cozystack-gateway-hostname-policy`** — VAP on `Gateway`
CREATE/UPDATE. Restricts listener hostnames to the namespace's
`namespace.cozystack.io/host` apex.
4. **`cozystack-tenant-host-policy`** — VAP on `Tenant` CREATE/UPDATE.
5. **`cozystack-namespace-host-label-policy`** — VAP on core `Namespace`
CREATE/UPDATE. Immutability gate on the host label.
7. **`cozystack-route-hostname-policy`** — VAP on `HTTPRoute` and
`TLSRoute` CREATE/UPDATE. Closes the hostname-hijack vector across
namespaces under inheritance — a child cannot claim a hostname outside
its own apex.

The previous Layer 3 (`cozystack-gateway-attached-namespaces-policy`)
and Layer 6 (render-time `fail` in cozystack-basics) banned `tenant-*`
entries in `publishing.gateway.attachedNamespaces` on the assumption
that the attach list was the hijack vector. Under inheritance the attach
surface is the label selector, not the static list; the hijack vector is
closed by Layers 4/5/7 independently. Both Layer 3 and Layer 6 are
dropped — kept in numbering above for continuity with the earlier review
threads.

## Foreign-takeover guards

Five reconcile paths refuse to silently rewrite a pre-existing object
that shares the controller-derived name but carries no `OwnerReference`
back to the TenantGateway: `Gateway`, redirect `HTTPRoute`, per-tenant
`Issuer`, wildcard `Certificate`, per-listener `Certificate`. An
operator who hand-pinned a Certificate or Issuer at the controller's
derived name (private CA, manual cert pinning) gets an explicit
`Ready=False/ReconcileError` condition instead of having their config
silently destroyed and the resource re-issued from a different ACME
account.

The namespace-label patching path applies the same discipline:
cozystack-controller only writes / strips
`namespace.cozystack.io/gateway` on namespaces it annotates with
`cozystack.io/gateway-attached-by`. Labels written by the apps/tenant
chart (no annotation) are never touched, so inheritance for tenant
namespaces survives every reconcile.

## Tests

- **Go controller tests** — 50+ cases in
`internal/controller/tenantgateway/`: idempotency, OwnerReference
cascade, multi-parentRef status, HTTP-01↔DNS-01 mode-transition cert
lifecycle, foreign-takeover refusals (Gateway / HTTPRoute / Issuer /
Certificate / per-listener Certificate), DNS-01 provider matrix
(cloudflare / route53 / digitalocean / rfc2136), HTTPS Kinds=[HTTPRoute]
restriction, fail-closed CEL on missing host label, status condition
wiring. Plus, for inheritance: label-based allowedRoutes selector shape,
namespace label patch + garbage-collect via owner annotation, Helm-owned
label safety, DNS-01 wildcard SAN expansion per inheriting child,
per-child wildcard listener materialisation.
- **Admission-chain tests** —
`pkg/registry/apps/application/rest_admission_test.go` pins
`createValidation`, `updateValidation`, `deleteValidation` paths via
sentinel callbacks.
- **helm-unittest** across `extra/gateway` (TenantGateway CR rendering,
both modes), `core/platform` (DNS-01 wiring), `apps/tenant`
(gateway-effective decision matrix including inheritance),
`system/cozystack-basics` (three VAPs + Layer 7 VAP + tenant-root
namespace label including the conditional
`namespace.cozystack.io/gateway: tenant-root` + cozystack-values-secret
`_namespace.gateway` propagation), `system/cert-manager-issuers` (full
DNS-01 provider matrix), `system/cozystack-api` and `system/dashboard`
(Ingress↔TLSRoute/HTTPRoute toggle), `system/monitoring` (Grafana /
Alerta Ingress↔HTTPRoute toggle).
- **e2e bats** — `hack/e2e-apps/gateway.bats` (17 cases) covering
GatewayClass Accepted, Gateway Programmed + LB Service, HTTPRoute
Accepted with status, VAP rejects cross-tenant hostname hijack, VAP
rejects `tenant.spec.host` writes from non-trusted SA
(regression-detector for the admission-chain fix), VAP rejects
`namespace.cozystack.io/host` label changes from non-trusted SA,
route-hostname VAP rejects HTTPRoute hostnames outside the apex, and
three new cases for inheritance — Package admission accepts `tenant-*`
in `attachedNamespaces` (regression guard for the dropped ban), child
tenant without explicit gateway inherits `_namespace.gateway` from a
Gateway-owning parent in lockstep across the namespace label and the
cozystack-values Secret, and the cross-namespace attach end-to-end where
a child tenant's HTTPRoute reaches Accepted on the parent's Gateway via
the inheritance label.
- **CI wiring** — `make test-controllers` target invoked from
`pull-requests.yaml` runs `go test ./internal/...`.

## Backward compatibility

The only always-on change is the flip of `cilium.envoy.enabled` +
`cilium.gatewayAPI.enabled` to `true` (extra `cilium-envoy` DaemonSet,
~100 MB RAM/node idle). Everything else is gated behind
`gateway.enabled=false` / `tenant.spec.gateway=false`. Existing clusters
see no behavioural change until an operator explicitly opts in.

The cozystack-api admission-chain fix is unconditional. Existing
operators who do not define any VAP / admission webhook on
`apps.cozystack.io` resources see no observable change. Operators who DO
have their own webhooks/VAPs against these kinds will start seeing them
fire on Create and Delete (as intended) — review your policies before
merging if you maintain custom admission for `apps.cozystack.io/*`.

Operators who set `publishing.gateway.attachedNamespaces` to include
`tenant-*` entries used to hit a render-time fail or an admission
rejection — now the apply succeeds and the listed tenant namespaces
simply pick up the gateway-attach label alongside the cozy-* system
namespaces. This matches the inheritance model and was kvaps' explicit
ask in the May 25 review.

## Release note

```release-note
- Adds Gateway API support via Cilium, materialised per-tenant via the new `gateway.cozystack.io/v1alpha1 TenantGateway` CRD reconciled by `cozystack-controller`. Opt-in via `publishing.gateway.enabled=true` at the platform level; per-tenant opt-in via `tenant.spec.gateway=true` for dedicated Gateway / LB IP, otherwise the tenant inherits the nearest ancestor's Gateway via a label-based `allowedRoutes` selector (same shape as the existing `_namespace.ingress` inheritance). Includes HTTP-01 (default) and DNS-01 (opt-in, with cloudflare / route53 / digitalocean / rfc2136 providers) cert solvers; DNS-01 extends the parent's wildcard Certificate with per-child apex SANs and adds a `*.<child-apex>` listener per inheriting tenant. Tenant API is mechanism-agnostic for IP allocation — the LoadBalancer Service draws its IP from the cluster's admin-configured allocator (MetalLB / Cilium LB-IPAM / robotlb / externalIPs).
- Strengthens admission coverage on `apps.cozystack.io/*` resources: `cozystack-api` now invokes `createValidation` and `deleteValidation` callbacks on Create and Delete (Update was already wired). Operators with custom ValidatingAdmissionPolicies or webhooks targeting `apps.cozystack.io/*` will see them fire on all three verbs as the API contract requires; review your policies before merging if you maintain such admission.
```
2026-05-26 10:29:30 +03:00
Aleksei Sviridkin d2e851f63b feat: Gateway API support via Cilium
Replace ingress-nginx as the default ingress layer for cozystack-native
services with Cilium Gateway API, materialised per-tenant via a new
TenantGateway CRD reconciled by cozystack-controller.

The extra/gateway chart renders one gateway.cozystack.io/v1alpha1
TenantGateway per tenant; the controller owns the resulting Gateway,
per-tenant Issuer, Certificate(s), and the http->https redirect
HTTPRoute. This avoids the helm-vs-controller race on
Gateway.spec.listeners when route-driven listeners are added.

publishing.certificates.solver selects HTTP-01 (default; per-app HTTPS
listener + per-app Certificate) or DNS-01 (opt-in; single wildcard
Certificate per apex; providers: cloudflare, route53, digitalocean,
rfc2136). HTTP-01 keeps zero-config per-app onboarding; DNS-01 saves
Let's Encrypt budget on apex-shared clusters.

Security model -- seven independent layers, all admission-time gates
fail-closed:

  1. Listener allowedRoutes namespace whitelist via the
     kube-apiserver-written kubernetes.io/metadata.name label;
     plain-HTTP listener is strictly narrower than HTTPS, and HTTPS /
     passthrough listeners restrict allowedRoutes.kinds to
     HTTPRoute / TLSRoute respectively.
  2. cozystack-gateway-hostname-policy VAP on Gateway.
  3. cozystack-gateway-attached-namespaces-policy VAP on Package --
     blocks tenant-* in gateway.attachedNamespaces.
  4. cozystack-tenant-host-policy VAP on Tenant -- blocks spec.host
     writes from non-trusted callers.
  5. cozystack-namespace-host-label-policy VAP on Namespace -- blocks
     writes to namespace.cozystack.io/host from non-trusted callers.
  6. Render-time fail in cozystack-basics for tenant-* in
     gateway.attachedNamespaces.
  7. cozystack-route-hostname-policy VAP on HTTPRoute / TLSRoute,
     scoped to tenant-* namespaces.

Within-apex cross-namespace hostname conflict is resolved at reconcile
time by a HostnameConflict condition (cozy-* namespace wins).

Five reconcile paths refuse to rewrite pre-existing objects with the
controller-derived name but no OwnerReference back to the
TenantGateway: Gateway, redirect HTTPRoute, per-tenant Issuer, wildcard
Certificate, per-listener Certificate.

Derived-apex tenants (tenants whose host is a subdomain of a parent
tenant's) auto-enable Gateway when the platform-level gateway.enabled
is on.

Bonus fix: pkg/registry/apps/application/rest.go was ignoring the
createValidation and deleteValidation callbacks that genericapiserver
hands to every storage method, silently bypassing admission for Create
and Delete on every apps.cozystack.io resource (Tenant, MariaDB,
Postgres, Kubernetes, ...). Aligned with the existing updateValidation
pattern.

Always-on changes: cilium.envoy.enabled + cilium.gatewayAPI.enabled
flip to true (extra cilium-envoy DaemonSet, ~100 MB RAM/node idle),
and the cozystack-api admission-chain fix. Everything else is gated
behind gateway.enabled=false and tenant.spec.gateway=false; existing
clusters see no behavioural change until an operator opts in.

Tests: 60+ Go cases in internal/controller/tenantgateway/, admission-
chain regression tests in pkg/registry/apps/application/, helm-unittest
across 7 charts (extra/gateway, core/platform, apps/tenant,
system/cozystack-basics, system/cert-manager-issuers,
system/cozystack-api, system/dashboard), and 14 e2e bats scenarios in
hack/e2e-apps/gateway.bats. New make test-controllers target wired
into pull-requests.yaml.

Deps: sigs.k8s.io/gateway-api v1.4.1, cert-manager v1.17.4.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-05-25 17:07:02 +03:00
Myasnikov Daniil ed0297eb87 refactor(build): standardize image tagging to fix concurrent PR push conflicts
Every package Makefile tagged images via `settag` which, on a non-release
commit, collapsed to `:latest`. Concurrent PR builds therefore raced on every
image's `:latest` manifest in the OCIR registry. Two outliers (metallb,
redis-operator) used hardcoded version tags and collided across PRs even when
`settag` would not have.

Replace `settag` and the `TAG=latest` fallback with an `image-tags` macro
driven by three env vars:
  - IMAGE_TAG          — build-unique tag, always pushed
  - PUBLISH_VERSIONED  — also push :<component-version>
  - PUBLISH_FLOATING   — also push :latest

PR CI sets IMAGE_TAG=pr-<N>-<sha> with both PUBLISH_* off, so every push is
unique across runs. Release CI sets IMAGE_TAG=<git-tag> with both PUBLISH_*
on, preserving the previous release output and adding a uniform :<git-tag>
tag across every image. Local builds default to IMAGE_TAG=dev with PUSH=0
already off.

Image consumers reference digests pinned into values.yaml by each Makefile,
so the tag is decorative for cluster-side pulls; the build-unique tag is
sufficient for E2E.

Drops the redundant `:<component-version>-<git-tag>` form (cilium, linstor,
matchbox, ubuntu-container-disk, kubevirt-*, cluster-autoscaler) — nothing
in the repo or downstream values.yaml consumed it. The same information is
preserved via the new uniform `:<git-tag>` plus the existing
`:<component-version>` on release.

ubuntu-container-disk builds a separate image per Kubernetes version in a
loop. Switched its primary tag to `:<k8s-ver>-<IMAGE_TAG>` so loop
iterations within a single build do not push to the same tag.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-21 15:23:18 +05:00
cozystack-ci[bot] 9f655e29ff Prepare release v1.4.0-rc.2
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-05-13 21:48:39 +00:00
cozystack-ci[bot] 252c18f3e8 Prepare release v1.4.0-rc.1
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-05-13 12:29:01 +00:00
cozystack-ci[bot] b52e2801b4 Prepare release v1.3.0
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-22 07:28:09 +00:00
Myasnikov Daniil 2b6e20cc3f fix(platform): migrate ACME HTTP-01 to ingressClassName API
ClusterIssuer solver referenced IngressClass "nginx" which does not
exist on cozystack clusters — real classes are named after tenant
namespaces (e.g. tenant-root). Cert issuance only worked because every
requesting Ingress overrode the ClusterIssuer via the legacy
acme.cert-manager.io/http01-ingress-class annotation.

Switch both sides to the modern cert-manager API (available since
cert-manager 1.12; cozystack ships 1.19.3):

- ClusterIssuer: http01.ingress.ingressClassName, value parameterized
  from _cluster.expose-ingress (default "tenant-root")
- Ingress annotation: http01-ingress-ingressclassname

These must migrate together — mixing ingressClassName (ClusterIssuer)
with the old http01-ingress-class annotation triggers cert-manager's
"fields ingressClassName and class cannot be set at the same time"
validation and breaks issuance.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-21 17:00:31 +05:00
cozystack-ci[bot] 12bf6b0e26 Prepare release v1.3.0-rc.1
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:30 +00:00
cozystack-bot c652e1e0f9 Prepare release v1.2.0
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-03-27 14:00:46 +00:00
IvanHunters f647cfd7b9 [bucket] Fix s3manager endpoint to use actual S3 endpoint from BucketInfo
The deployment template was constructing the S3 endpoint from the tenant's
namespace host (e.g. s3.freedom.infra.example.com), while COSI credentials
are issued for the actual SeaweedFS endpoint (e.g. s3.infra.example.com).
This mismatch caused 'invalid credentials' errors when users tried to log
in with valid credentials from the bucket secret.

Now the endpoint is resolved from BucketInfo (same source as credentials),
with a fallback to the constructed namespace host for first-time deploys
before BucketAccess secrets are created.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-13 01:08:13 +03:00
cozystack-bot 3c971d0e1b Prepare release v1.1.0
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-03-06 12:03:56 +00:00
IvanHuntersandClaude fdb015458c build(bucket): rebuild s3manager image with login mode
Rebuild s3manager with auth.go login page support and push
to 999669/s3manager registry for testing.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-04 19:01:27 +03:00
IvanHuntersandClaude 84da47a2ce refactor(bucket): replace basic auth with s3manager login page
Remove nginx basic auth and credential secret injection from the
bucket Helm chart. s3manager now always starts in login mode and
handles authentication via its own login page with encrypted
session cookies. This eliminates the dependency on the -credentials
and -ui-auth secrets for the UI layer.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-04 18:47:51 +03:00
IvanHuntersandClaude 175b3badd2 feat(bucket): make credentials and basic auth conditional on users
deployment.yaml: use s3._namespace.host for ENDPOINT instead of
secret ref, inject ACCESS_KEY_ID/SECRET_ACCESS_KEY only when users
exist. Without users, s3manager starts in login mode.

ingress.yaml: nginx basic auth annotations only when users exist.
Without users, s3manager handles authentication via its login form.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-04 18:30:49 +03:00
IvanHuntersandClaude f777f659d5 feat(bucket): add login screen to s3manager for credentialless mode
When a bucket has no users configured, s3manager previously crashed
due to missing ACCESS_KEY_ID/SECRET_ACCESS_KEY env vars. This adds
a login mode where users enter their S3 credentials via a web form.

New Go code (via cozystack.patch):
- auth.go: session-based auth middleware, login/logout handlers,
  per-request S3 client from encrypted cookie session
- login.html.tmpl: Materialize CSS login form
- main.go: LoginMode toggle, conditional route setup
- Dependency: gorilla/sessions for AES-256 encrypted cookies

Dockerfile: add go mod tidy step for new dependency resolution.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-04 18:30:40 +03:00
IvanHuntersandClaude 9e6c240d6e fix(bucket): use per-user COSI secret for UI credentials
The default BucketAccess was removed in favor of per-user access.
Update secret.yaml to look up the first user's COSI secret instead
of the non-existent default one, with nil-check for race conditions.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-04 00:34:33 +03:00
IvanHuntersandClaude b4f1e96b98 feat(bucket): include COSI user credential secrets in dashboard
Create labeled secrets in the -system chart using lookup to copy
credentials from COSI-created secrets. The ApplicationDefinition
matchLabels selector exposes them in the dashboard.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-03-03 19:32:32 +03:00
cozystack-bot 65c6936e95 Prepare release v1.0.0-rc.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-25 16:09:50 +00:00
Myasnikov Daniil 356070615c feat(platform): Changed ingress annotation rendering for http01 solver
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-02-20 11:45:04 +05:00
Myasnikov Daniil 4aa1f03321 feat(platform): Added parameters to override ClusterIssuer
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-02-20 10:43:48 +05:00
cozystack-bot da359d558a Prepare release v1.0.0-beta.3
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-11 06:13:35 +00:00
cozystack-bot 8f3d686492 Prepare release v1.0.0-beta.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-02 12:34:58 +00:00
cozystack-bot 4ffe453351 Prepare release v1.0.0-alpha.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-01-20 07:50:59 +00:00