Commit Graph
239 Commits
Author SHA1 Message Date
myasnikovdaniil 7520967d8d feat(operator): expose HelmRelease generation knobs (interval, retry-interval, timeouts, max-history) (#2509)
## Why

The `cozystack-operator` Package reconciler builds every `HelmRelease`
it generates with a handful of hardcoded values. Three of those have
caused recurring pain across the project:

- **`Spec.Interval = 5m`** (line 220 before this PR) — too long for E2E
install where dependent HRs reconcile out of order; previously addressed
by a manual `flux reconcile --force` workaround in
`hack/e2e-install-cozystack.bats`.
- **`Install.Remediation{Retries: -1}` + `Upgrade.Remediation{Retries:
-1}`** — semantically "retry forever, never remediate", but the
retry-on-failure timing was tied to `Spec.Interval` (5m). Audit of the
last 30 successful PR runs found that **every** run hit `kubectl wait
hr/seaweedfs-system` timeouts at the 2-min mark, recovering only after
the inline `flux reconcile --force` workaround.
- **`Install.Timeout = 10m` / `Upgrade.Timeout = 10m`** — directly
contradicts the per-Application timeout work in `pkg/config/config.go` +
`pkg/registry/apps/application/rest.go` (commit `7b146cbe`).
`packages/apps/tenant/templates/etcd.yaml` hand-rolls its HR YAML to
bypass this for cert rotation; harbor has had multiple commits chasing
the same problem.
- **`Spec.MaxHistory`** unset → Helm default 5. Test clusters bounce HRs
frequently and accumulate release-history Secrets.

This PR exposes all four as operator flags + helm chart values, with
defaults that preserve current behavior.

## What

### New operator flags

| Flag | Default | HR field |
|---|---|---|
| `--helmrelease-interval` | `5m` | `Spec.Interval` |
| `--helmrelease-retry-interval` | `30s` |
`Spec.{Install,Upgrade}.Strategy.RetryInterval` |
| `--helmrelease-install-timeout` | `10m` | `Spec.Install.Timeout` |
| `--helmrelease-upgrade-timeout` | `10m` | `Spec.Upgrade.Timeout` |
| `--helmrelease-max-history` | `5` | `Spec.MaxHistory` |

### Strategy switch

`Install.Strategy.Name` and `Upgrade.Strategy.Name` are now set to
`RetryOnFailure` (instead of leaving the default `RemediateOnFailure`),
with `RetryInterval` set from the new flag. Functionally equivalent to
the previous `Remediation{Retries:-1}` ("retry forever") since
remediation never fired anyway, but **decouples retry-on-failure timing
from `Spec.Interval`** so failed releases recover at 30s without polling
healthy releases at the same fast cadence.

### Chart exposure

`packages/core/installer/values.yaml` gets five new optional values
(`cozystackOperator.helmRelease{Interval,RetryInterval,InstallTimeout,UpgradeTimeout,MaxHistory}`),
each defaulting to empty so the operator's own default applies. The
chart conditionally renders the corresponding flag only when the value
is set.

## Production safety

- All static defaults match the previous hardcoded values (5m / 10m /
10m / 5). No new code path runs until an admin explicitly sets a flag.
- **One intentional behavior change: failed-release retry cadence**
drops from ~5m (previously coupled to `Spec.Interval`) to 30s
(`RetryInterval` default). This is the whole motivation for the strategy
switch — the recurring `kubectl wait hr/seaweedfs-system` flakes
documented above were caused by the previous coupling. Healthy-release
reconcile cadence is unchanged at 5m in production; CI overrides this to
30s via `--set cozystackOperator.helmReleaseInterval=30s` in
`hack/e2e-install-cozystack.bats` for faster convergence on E2E
clusters.
- `RetryOnFailure` is functionally equivalent to `RemediateOnFailure +
Retries:-1` since neither ever triggers remediation.
- Empty value in the chart → flag not rendered → operator falls back to
its built-in default. (`helmReleaseMaxHistory: 0` — unlimited history
per Helm semantics — is also honored; the chart guards on `ne (toString
…) ""` rather than truthiness so int `0` is not silently dropped.)

## Verification

- `go build ./...` clean
- `go test ./internal/operator/...` passes (covers `Interval`,
`MaxHistory`, `Install/Upgrade.Timeout`, `Strategy.Name`,
`RetryInterval`, `ChartRef` propagation, plus a regression test for
`MaxHistory=0`)
- `helm template installer packages/core/installer` renders cleanly with
default and overridden values; `--set
cozystackOperator.helmReleaseMaxHistory=0` correctly emits
`--helmrelease-max-history=0`

## Followups (deliberately not bundled)

- **Per-component timeout overrides** on `ComponentInstall` (mirroring
the `UpgradeCRDs` pattern from #2427). Charts like etcd that need a 30m
timeout would set it on their `Component` rather than tuning a global
flag. Touches the Package CR API + codegen — bigger lift, separate PR.
- **`Remediation.Retries` as a flag.** No current pain; skip.
- **`createOrUpdateHelmRelease` field preservation.** Currently
preserves `Suspend` but not `Test`/`Rollback`/`Uninstall`. Would
silently overwrite an SRE's manual edit. Minor cleanliness, separate PR.

## Test plan

- [ ] CI passes E2E with `helmReleaseInterval=30s` set in install bats
(existing T1 test path)
- [ ] No regression on production install path (default values)
- [ ] HR objects in a healthy cluster render with
`Strategy.Name=RetryOnFailure` and `RetryInterval=30s`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Operator and Helm chart now accept CLI flags and Helm values to
configure HelmRelease reconciliation intervals, retry behavior,
install/upgrade timeouts, and history retention.

* **Bug Fixes**
* Operator now validates HelmRelease timing/history inputs on startup
and will exit if values are invalid or non-positive.

* **Tests**
* Added unit and e2e coverage to verify HelmRelease configuration and a
chart install path using a custom reconciliation interval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-05 14:35:00 +05:00
myasnikovdaniil 086e9bb35b fix(installer): bootstrap cozy-system via --create-namespace + label hook (#2508)
## Why

helm v3 has a chicken-and-egg with charts that ship their own Namespace
resource:

- **With `--create-namespace`** on the install: helm pre-creates the
namespace via plain kubectl-create (no helm meta annotations); the
chart's own Namespace apply then fails with `already exists`.
- **Without `--create-namespace`**: helm fails immediately because it
can't write its release-secret to a non-existent namespace.

The cozy-installer chart shipped a `Namespace cozy-system` resource
since the chart's introduction (December 2025). Every cold install since
has hit this conflict. It was masked by the 3× retry on `Install
Cozystack` in `.github/workflows/pull-requests.yaml`: attempt 1 fails
with the conflict, attempt 2 sees the existing failed release and takes
the upgrade code path which patch-merges instead of strict-create.

Surfaced when retries on the install step were dropped.

## What

- Remove `Namespace cozy-system` from
`packages/core/installer/templates/cozystack-operator.yaml`.
- Add `packages/core/installer/templates/cozy-system-labels.yaml`: a
pre-install/pre-upgrade hook (ServiceAccount + ClusterRole +
ClusterRoleBinding + Job) that patches `cozystack.io/system=true` and
`pod-security.kubernetes.io/enforce=privileged` onto the namespace.
- Hook RBAC is scoped to `verbs: [get, patch]` on `resourceNames:
[cozy-system]` only.
- `hook-delete-policy: before-hook-creation,hook-succeeded` so the hook
surface only exists during install/upgrade.

Install commands now use `helm upgrade --install --namespace cozy-system
--create-namespace …` — standard pattern, matches kube-prometheus-stack
/ argo-cd / cert-manager.

## Verification

End-to-end on a kind cluster:
- Cold install: 3.3s clean, no retries needed
- Upgrade (re-run): 3.0s, idempotent, namespace stays labeled
- Helm uninstall + re-install: clean

## Test plan

- [ ] CI on this PR passes E2E
- [ ] Existing clusters: namespace already labeled correctly, hook
re-asserts on next upgrade — no migration needed

```release-note
fix(installer): cold installs no longer fail on the chart's Namespace resource colliding with `helm --create-namespace`; cozy-system is now bootstrapped via a pre-install hook.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added an automated pre-install/pre-upgrade step that applies system
identity and pod-security labels to the system namespace during
deployment, run in a hardened one-shot task before normal workloads
start.

* **Chores**
* Separated namespace initialization from the operator install flow so
the operator no longer renders or retains the system namespace directly,
improving installation sequencing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-05 14:25:44 +05:00
Myasnikov Daniil 730037c85d fix(installer): address review feedback on cozy-system bootstrap
- Bump alpine/k8s to 1.33.4 (was 1.32.0, EOL); matches in-tree pin used
  by cozystack-api, seaweedfs, vm-disk, and vm-instance hooks.
- Skip hostNetwork=true and not-ready/CNI tolerations on the hosted
  variant: managed clusters provide CNI before install, and kube-system
  may carry strict PSA there that would reject hostNetwork pods.
- Add packages/core/installer/templates/cozy-system-namespace.yaml gated
  on .Values.bareNamespace (default false). The top-level Makefile's
  manifests target now passes --set bareNamespace=true and renders
  Namespace + operator into _out/assets/cozystack-operator-*.yaml so
  the kubectl-apply install path stays self-contained. Helm
  install/upgrade callers keep the default and bootstrap via
  --create-namespace + the pre-install labeler hook.
- Add a bats assertion in hack/e2e-install-cozystack.bats checking
  cozy-system carries pod-security.kubernetes.io/enforce=privileged and
  cozystack.io/system=true after helm install.
- Inline comment near hostNetwork=true documenting why dnsPolicy left
  at Default (cluster DNS not up at install time;
  ClusterFirstWithHostNet would not help).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 22:13:39 +05:00
Myasnikov Daniil 6ca5566391 fix(operator): handle MaxHistory=0 and cover HelmRelease spec build
Chart template now uses `ne (toString …) ""` for the five helmRelease*
knobs, so integer 0 (notably MaxHistory=0 = unlimited history per Helm
semantics, documented in values.yaml) is no longer silently dropped.

Extract buildHelmReleaseSpec from Reconcile and add unit tests
asserting Interval, MaxHistory, Install/Upgrade timeouts,
Strategy=RetryOnFailure, RetryInterval, and ChartRef propagate from
PackageReconciler config to the generated HelmRelease.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 11:14:38 +05:00
Myasnikov DaniilandClaude Opus 4.7 85ced0d729 fix(installer): harden cozy-system-labeler hook and add hosted variant
Address review feedback on PR #2508:

- Pin alpine/k8s by digest (matches operator image pinning style).
- Add resources requests/limits and a minimal securityContext to the
  labeler Job: capabilities drop ALL, allowPrivilegeEscalation: false,
  seccompProfile RuntimeDefault.
- Include hook-failed in helm.sh/hook-delete-policy across all four
  hook resources so a failed install doesn't leave SA / ClusterRole /
  ClusterRoleBinding / Job objects behind.
- Add explicit hosted variant case in the env block, mirroring the
  pattern in cozystack-operator.yaml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-04 10:59:36 +05:00
Myasnikov Daniil 4527320944 fix(installer): bootstrap cozy-system via --create-namespace + label hook
The chart used to ship `Namespace cozy-system` directly, which trips a
helm v3 chicken-and-egg whenever the install command uses
`--create-namespace`:

  - WITH `--create-namespace`, helm pre-creates the namespace via plain
    kubectl-create (no helm meta annotations); the chart's own Namespace
    apply then fails with `already exists`.
  - WITHOUT `--create-namespace`, helm fails immediately because it
    cannot write its release-secret to a non-existent namespace.

The bug was hidden by the 3x retry on the install step in
`.github/workflows/pull-requests.yaml`: attempt 1 fails with the
conflict, attempt 2 sees the failed release and takes the upgrade code
path (patch-merge instead of strict-create). Surfaces cleanly when the
retry is dropped; reproducible on every cold install.

Fix has two pieces:

1. Drop the chart-side `Namespace cozy-system` resource. Install
   commands now use the standard `helm upgrade --install --namespace
   cozy-system --create-namespace` (matches kube-prometheus-stack /
   argo-cd / cert-manager).

2. Add a pre-install/pre-upgrade Job hook (`cozy-system-labeler`) that
   patches `cozystack.io/system=true` and
   `pod-security.kubernetes.io/enforce=privileged` onto the namespace
   so the cozystack-operator pod (hostNetwork=true) is admitted.

The hook runs in `kube-system`, not `cozy-system`. Reason: the operator
pod needs hostNetwork=true (no CNI yet at install time), which violates
PodSecurity baseline. cozy-system therefore needs enforce=privileged
*before* the operator pod is admitted. A labeler pod inside cozy-system
would itself need hostNetwork=true and hit the same admission denial:

    pods "cozy-system-labeler-..." is forbidden: violates PodSecurity
    "baseline:latest": host namespaces (hostNetwork=true)

Job admission denial means no pod is ever created, .status.failed never
increments and backoffLimit never trips. helm's --wait waits for
Complete=True forever and surfaces only the generic "timed out waiting
for the condition" at its budget.

`kube-system` is PSA-exempt on Talos
(`defaults.exemptions.namespaces` in the apiserver
PodSecurityConfiguration) and unlabeled on vanilla kubeadm/kind/k3s, so
hostNetwork pods are admitted there. The hook's ServiceAccount is bound
to a ClusterRole scoped via `resourceNames: ["cozy-system"]` to the one
target namespace, keeping the cluster permission grant minimal.
`hook-delete-policy: before-hook-creation,hook-succeeded` ensures the
SA, RBAC, and Job are cleaned up after a successful install/upgrade.

Verified end-to-end in CI on the parallel consolidation branch
feat/e2e-optimization (PR #2500): cold install passes admission, the
operator pod comes up, all downstream E2E tests pass.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-05-02 17:48:27 +05:00
Myasnikov Daniil e410b02e1d feat(operator): expose Install/Upgrade timeouts and MaxHistory as flags
The HelmRelease generation in package_reconciler hardcoded three values
that have caused recurring pain:

- Install.Timeout = 10m (line 214)
- Upgrade.Timeout = 10m (line 220)
- MaxHistory = unset (Helm default 5)

The 10m timeout in particular contradicts the per-Application timeout
work done in `pkg/config/config.go` and
`pkg/registry/apps/application/rest.go` (commit 7b146cbe), which lets
individual Applications override the install/upgrade timeout via
annotation. The PackageSource side had no equivalent — etcd works around
it by hand-rolling its HR YAML in `packages/apps/tenant/templates/etcd.yaml`
to set a 30m timeout, harbor has had multiple commits chasing the same
problem from the other side. This closes that gap.

New operator flags (each maps to a chart value with empty default;
operator uses its own default when value is empty):

- `--helmrelease-install-timeout` (default 10m) → Spec.Install.Timeout
- `--helmrelease-upgrade-timeout` (default 10m) → Spec.Upgrade.Timeout
- `--helmrelease-max-history` (default 5)        → Spec.MaxHistory

Production behaviour unchanged. E2E and edge cases (cert rotation,
slow-installing charts) can override per-cluster without modifying
chart manifests.

Per-component overrides on `ComponentInstall` (mirroring the existing
`UpgradeCRDs` pattern) would be a strictly better fix for charts like
etcd that need an outlier timeout — left as a deliberate followup since
it touches the Package CR API.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-28 12:44:06 +05:00
Myasnikov Daniil c4c70e0e18 feat(operator): switch HelmRelease retries to Strategy=RetryOnFailure with configurable RetryInterval
The operator generated HelmReleases with `Install.Remediation{Retries: -1}`
and `Upgrade.Remediation{Retries: -1}`, which is the v2 way of saying
"retry forever, never remediate". Combined with the spec-level
`Interval: 5m`, this meant a failed install/upgrade waited a full 5
minutes before the next attempt — even when the underlying race that
caused the failure (e.g. a chart artifact not quite Ready, a HR
dependency reconciling out of order) cleared in seconds.

Audit of the last 30 successful PR runs found that every single run had
its `Install Cozystack` step's `kubectl wait hr/seaweedfs-system` time
out at the 2-minute mark and recover only after an inline
`flux reconcile --force` workaround. Same root cause: 5-minute retry
interval was longer than the test's wait window. That workaround can be
removed once this change lands.

Switch to `Strategy.Name=RetryOnFailure` with `RetryInterval` exposed via
a new `--helmrelease-retry-interval` operator flag (default 30s):

- Functionally equivalent to the previous configuration ("retry forever
  on failure") since `Retries: -1` meant remediation never fired anyway.
- Decouples retry-on-failure timing from `spec.Interval` so failed
  releases recover at 30s without polling healthy releases at the same
  cadence (which would multiply controller load by ~10x for no benefit).
- Helm chart exposes `cozystackOperator.helmReleaseRetryInterval`,
  defaulting to empty (operator uses its own 30s default).
- Same flag pattern as the existing `--helmrelease-interval` (5m default),
  also added in this PR via cherry-pick.

Verified: go build green, go test ./internal/operator/... pass, helm
template renders both flags conditionally on values.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-28 12:40:13 +05:00
Myasnikov Daniil e197aba625 feat(operator): make HelmRelease Interval configurable, override to 30s in E2E
Adds --helmrelease-interval (default 5m, matching today's behaviour) to the
operator and a corresponding cozystackOperator.helmReleaseInterval value
on the cozy-installer chart (default empty -> no flag rendered ->
operator default 5m applies).

E2E install sets it to 30s. Rationale: with the 5m default, dependency-
blocked HRs (waiting on cert-manager webhooks, CRDs) are requeued only
every 5 min, producing an 8-10 min dead zone in the E2E install where
the fast pack of HRs is Ready but the slow tail hasn't been retried yet.
30s collapses the gap.

Production defaults are unchanged. The override is opt-in per install.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-28 12:36:28 +05:00
Aleksei Sviridkin 3fea7cd487 fix(security): bump first-party Dockerfile builders to golang:1.26
The root go.mod now declares go 1.26.2 (raised in the first commit
of this PR). Every Dockerfile that copies the root go.mod and builds
a cozystack-owned binary has to use a matching base image, otherwise
the in-container go toolchain has to download go 1.26 on the fly
(via GOTOOLCHAIN=auto) at every build, which slows builds and breaks
in network-restricted CI sandboxes.

Bumps:
- packages/core/installer/images/cozystack-operator
- packages/system/backup-controller/images/backup-controller
- packages/system/backupstrategy-controller/images/backupstrategy-controller
- packages/system/cozystack-api/images/cozystack-api
- packages/system/cozystack-controller/images/cozystack-controller
- packages/system/flux-plunger/images/flux-plunger
- packages/system/kubeovn-plunger/images/kubeovn-plunger
- packages/system/lineage-controller-webhook/images/lineage-controller-webhook

All eight images go from golang:1.25-alpine to golang:1.26-alpine.
Also normalizes the cozystack-operator Dockerfile FROM stage keyword
from lowercase 'as' to uppercase 'AS' while touching it.

Vendored Dockerfiles (kubevirt-cloud-provider, redis-operator,
keycloak-operator, linstor-csi, multus-cni, objectstorage,
s3manager, kamaji) are intentionally untouched: they pull upstream
sources at build time and pin their go base image to whatever the
upstream commit was tested against.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-26 01:53:19 +03:00
cozystack-ci[bot] b52e2801b4 Prepare release v1.3.0
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-22 07:28:09 +00:00
cozystack-ci[bot] 12bf6b0e26 Prepare release v1.3.0-rc.1
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:30 +00:00
Andrei Kvapil 94223327fe Release v1.2.1 (#2306)
This PR prepares the release `v1.2.1`.
2026-04-02 18:56:08 +02:00
cozystack-bot c652e1e0f9 Prepare release v1.2.0
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-03-27 14:00:46 +00:00
Kirill Ilin 08676b9c05 fix(installer): add build revision marker to trigger platform reconciliation
When only leaf packages change but the platform chart remains unchanged,
the Flux ArtifactGenerator produces an ExternalArtifact with the same
content digest. This prevents the platform HelmRelease from reconciling,
so lookup() in repository.yaml never runs and cozystack-packages
OCIRepository does not pick up the new OCI digest.

Write a content-based hash of the working tree state to
core/platform/.build-revision before pushing the OCI artifact, ensuring
the platform chart content always changes on every build.

Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
2026-03-18 21:37:23 +05:00
cozystack-bot 3c971d0e1b Prepare release v1.1.0
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-03-06 12:03:56 +00:00
Andrei KvapilandClaude c83e41ea14 fix(installer): add keep annotation to Namespace and update migration script
Add helm.sh/resource-policy=keep annotation to the cozy-system Namespace
in the installer helm chart. This prevents Helm from deleting the
namespace when the HelmRelease is removed, which would otherwise destroy
all other HelmReleases within it.

Update the migration script to annotate the cozy-system namespace and
cozystack-version ConfigMap with helm.sh/resource-policy=keep before
generating the Package resource.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-02-28 11:46:09 +01:00
cozystack-bot 66a61bd63e Prepare release v1.0.0
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-26 15:17:04 +00:00
cozystack-bot 65c6936e95 Prepare release v1.0.0-rc.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-25 16:09:50 +00:00
cozystack-bot 7994976052 Prepare release v1.0.0-rc.1
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-21 20:58:57 +00:00
Aleksei SviridkinandClaude 0e8b6515af fix(installer): handle parsePlatformSourceURL error, restore variant validation
Check error from parsePlatformSourceURL instead of discarding it, preventing
silent fallback to OCIRepository on invalid URLs.

Move variant validation from deleted packagesource.yaml to
cozystack-operator.yaml template so invalid cozystackOperator.variant
values still fail at helm template/install time.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-19 17:42:31 +03:00
Aleksei SviridkinandClaude 655133b81c fix(installer): move PackageSource creation from Helm template to operator
Replace the Helm hook approach with programmatic PackageSource creation
in the operator startup sequence. Helm hooks are unsuitable for persistent
resources like PackageSource because before-hook-creation policy causes
cascade deletion of owned ArtifactGenerators during upgrades.

The operator now creates the platform PackageSource after installing CRDs
and the Flux source resource, using the same create-or-update pattern as
installPlatformSourceResource(). The sourceRef.kind is derived from the
platform source URL (OCIRepository for oci://, GitRepository for git).

Also fix stale comment in e2e test referencing deleted crds/ directory.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-19 17:38:23 +03:00
Aleksei SviridkinandClaude 668ddc552e refactor(installer): remove CRDs from Helm chart, rely on operator --install-crds
Remove the crds/ directory from the cozy-installer Helm chart. The operator
already installs embedded CRDs via server-side apply on every startup with
the --install-crds=true flag, making the Helm crds/ directory redundant.

Convert templates/packagesource.yaml to a Helm post-install/post-upgrade
hook so it is applied after the operator has started and installed CRDs.

Update codegen to write CRDs only to internal/crdinstall/manifests/ (single
source of truth) and update the Makefile to source build assets from there.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-19 17:23:37 +03:00
cozystack-bot bff5468b52 Prepare release v1.0.0-beta.6
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-17 22:36:40 +00:00
Aleksei SviridkinandClaude 92d261fc1e fix: address review findings in operator and tests
- Remove duplicate "Starting controller manager" log before install
  phases, keep only the one before mgr.Start()
- Rename misleading test "document without kind returns error" to
  "decoder rejects document without kind" to match actual behavior
- Document Helm uninstall CRD behavior in deployment template comment
- Use --health-probe-bind-address=0 consistently with metrics-bind
- Exclude all dotfiles in verify-crds diff, not just .gitattributes

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-18 00:49:57 +03:00
Aleksei SviridkinandClaude b3fe6a8c4a fix(crdinstall): hardcode CRD GVK, add timeout test, document dual install
- Use explicit apiextensions.k8s.io/v1 CRD GVK in waitForCRDsEstablished
  instead of fragile objects[0].GroupVersionKind()
- Add TestInstall_crdNotEstablished for context timeout path
- Add --recursive to diff in verify-crds Makefile target
- Document why both crds/ and --install-crds exist in deployment template

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-18 00:49:56 +03:00
Aleksei SviridkinandClaude 879b10b777 feat(installer): enable --install-crds in operator deployment
Add --install-crds=true to cozystack-operator container args so the
operator applies embedded CRD manifests on startup via server-side apply.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-18 00:49:55 +03:00
Aleksei SviridkinandClaude 55cd8fc0e1 refactor(installer): move CRDs to crds/ directory for proper Helm install ordering
Helm installs crds/ contents before processing templates, resolving the
chicken-and-egg problem where PackageSource CR validation fails because
its CRD hasn't been registered yet.

- Move definitions/ to crds/ in the installer chart
- Remove templates/crds.yaml (Helm auto-installs from crds/)
- Update codegen script to write CRDs to crds/
- Replace helm template with cat for static CRD manifest generation
- Remove pre-apply CRD workaround from e2e test

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-18 00:49:55 +03:00
cozystack-bot bae70596fc Prepare release v1.0.0-beta.5
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-16 16:01:30 +00:00
Aleksei SviridkinandClaude 7ca6e5ce9e feat(installer): add variant-aware templates for generic Kubernetes support
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-02-16 18:47:04 +03:00
cozystack-bot b6a840e873 Prepare release v1.0.0-beta.4
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-14 09:17:24 +00:00
Andrei KvapilandClaude ba7c729066 [installer] Merge operator templates into single variant-based template
Consolidate three separate operator deployment templates (talos, generic,
hosted) into a single template with variant selection via values.yaml
parameter. This resolves conflicts when running `make apply` from the
installer package, which previously rendered all three deployments.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-02-11 22:02:58 +01:00
cozystack-bot da359d558a Prepare release v1.0.0-beta.3
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-11 06:13:35 +00:00
cozystack-bot 8f3d686492 Prepare release v1.0.0-beta.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-02-02 12:34:58 +00:00
Aleksei SviridkinandClaude 77ce8227b4 feat(installer): split operator manifest into talos/generic/hosted variants
Per review feedback, create separate manifest files for different
deployment targets instead of using templated values:

- cozystack-operator.yaml (Talos): hardcoded localhost:7445 (KubePrism)
- cozystack-operator-generic.yaml: reads from cozystack-operator-config
  ConfigMap (user must create before applying)
- cozystack-operator-hosted.yaml: no env override (uses in-cluster SA)

This keeps installation flow clean - users apply the manifest matching
their deployment target without needing to modify original files.

Build system updated to generate all three variants.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-01-30 23:12:24 +03:00
Aleksei SviridkinandClaude f72860c827 feat(platform): add values-isp-full-generic.yaml for generic k8s bundle
Create dedicated values file for isp-full-generic bundle variant with
bundles.system.variant set to "isp-full-generic" instead of "isp-full".

This fixes the ArtifactGenerator generating identical digests for both
isp-full and isp-full-generic variants, which caused the wrong bundle
template to be selected.

Also adds isp-full-generic variant to installer PackageSource definition.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-01-30 23:12:22 +03:00
Aleksei SviridkinandClaude 083ce14609 feat(platform): add configurable Kubernetes API server settings for non-Talos deployments
Add networking.apiServer.host and networking.apiServer.port to platform values
to allow overriding the default Talos KubePrism settings (localhost:7445).

Also add networking.cilium.cgroup.autoMount for non-Talos clusters that need
Cilium to mount cgroups automatically.

Changes:
- packages/core/platform/values.yaml: Add apiServer and cilium.cgroup settings
- packages/core/platform/templates/bundles/system.yaml: Pass cilium values
- packages/core/installer/values.yaml: Add kubernetesServiceHost/Port
- packages/core/installer/templates/cozystack-operator.yaml: Template env vars

Closes: #1933

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-01-30 23:12:21 +03:00
cozystack-bot 4ffe453351 Prepare release v1.0.0-alpha.2
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-01-20 07:50:59 +00:00
Andrei Kvapil 74a3b1a8b3 [local-ccm] Add local-ccm package for cloud controller manager in local environments (#1831)
## What this PR does

This PR adds a new package `local-ccm` to the Cozystack platform.

The local-ccm package provides a Cloud Controller Manager for local
development and testing environments. It integrates the upstream Helm
chart from https://github.com/cozystack/local-ccm (version 0.2.0).

**Changes:**
- Created package structure in `packages/system/local-ccm/`
- Added Makefile with automated upstream chart synchronization from
GitHub
- Added PackageSource definition in
`packages/core/platform/sources/local-ccm.yaml`
- Configured dependency on networking components
- Set privileged mode as required for cloud controller operations

**Package details:**
- Namespace: `cozy-local-ccm`
- Release name: `local-ccm`
- Upstream: https://github.com/cozystack/local-ccm

### Release note

```release-note
[local-ccm] Add local-ccm package for cloud controller manager in local environments
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Local Cloud Controller Manager package providing automatic node
IP detection, internal and external IP support, initialization taint
removal, and continuous node reconciliation with configurable intervals.

* **Chores**
* Updated system component tolerations and scheduling policies to
improve compatibility with cloud infrastructure, including better
handling of node readiness, network initialization, and cloud provider
states.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-20 01:54:04 +01:00
Andrei Kvapil a7de06c62f Fix tolerations for uninitialized nodes
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-19 14:20:02 +01:00
Andrei KvapilandClaude 987a74ae5a refactor(telemetry): split telemetry between operator and controller
Split telemetry collection between cozystack-operator and cozystack-controller:

cozystack-operator now collects cluster-level metrics:
- cozy_cluster_info (cozystack_version, kubernetes_version)
- cozy_nodes_count (os, kernel)
- cozy_cluster_capacity (cpu, memory, nvidia.com/* resources)
- cozy_loadbalancers_count
- cozy_pvs_count (driver, size)
- cozy_package_info (name, variant)

cozystack-controller now collects application-level metrics:
- cozy_application_count (kind) - counts HelmReleases per ApplicationDefinition

Other changes:
- Add pkg/version for build-time version injection via ldflags
- Remove --cozystack-version flag (version now embedded at build time)
- Remove bundle/oidc configuration from telemetry (replaced by package_info)
- Remove cozy_workloads_count metric (replaced by cozy_application_count)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-19 13:52:15 +01:00
cozystack-bot 09290b5586 Prepare release v1.0.0-alpha.1
Signed-off-by: cozystack-bot <217169706+cozystack-bot@users.noreply.github.com>
2026-01-15 23:32:41 +00:00
Andrei KvapilandClaude 6dbfcbb93e feat(cozypkg): add cross-platform build targets with version injection
- Add pattern rule for building cozypkg binaries per platform
  (assets-cozypkg-<os>-<arch>) with checksums generation
- Add Version variable to cozypkg CLI injected via ldflags
- Split manifests into separate cozystack-crds.yaml and
  cozystack-operator.yaml files
- Update CI workflow to handle CRDs and operator as separate artifacts
- Update e2e tests and upload script for new manifest structure
- Suppress git describe stderr when no tags exist

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-15 17:18:01 +01:00
Andrei KvapilandClaude 3618abed62 refactor: move scripts to hack directory
Move common-envs.mk and package.mk from scripts/ to hack/ directory.
Update all Makefile includes to use new paths. Remove unused
issue-flux-certificates.sh script.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-15 16:06:56 +01:00
Andrei KvapilandClaude 99ac2fc710 refactor(installer): migrate installer to cozystack-operator
Remove legacy installer components (cozystack-assets-server,
installer.sh script, cozystack container image) in favor of
cozystack-operator based deployment.

Move migration scripts from scripts/migrations/ to
packages/core/platform/images/migrations/ for containerized execution.

Add grafana-dashboards image for centralized dashboard management.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-15 14:14:48 +01:00
Andrei Kvapil cf8ac03f45 Release v0.40.2 (#1858)
This PR prepares the release `v0.40.2`.
2026-01-13 17:25:18 +01:00
Andrei Kvapil dececc8587 Release v0.40.1 (#1855)
This PR prepares the release `v0.40.1`.

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-13 11:13:10 +01:00
Andrei KvapilandClaude 7b75903cee feat(operator): add valuesFrom injection to HelmReleases
Add automatic injection of cozystack-values secret reference into
HelmReleases created by Package reconciler. This enables charts to
access cluster and namespace configuration via .Values._cluster and
.Values._namespace.

Add annotation operator.cozystack.io/skip-cozystack-values to disable
injection for specific PackageSources (used for platform PackageSource
to avoid circular dependency).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-05 18:51:18 +01:00
Andrei Kvapil 7e86a04fab [cozystack-operator] Add deployment files (#1761)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
[cozystack-operator] Add deployment files
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Deployable CozyStack operator with configurable enablement, image, and
platform-source settings
* Operator-managed package source and variant-based platform
installation options

* **Chores**
* Release/packaging targets added to automate image and package
publishing and update manifests
* Configuration schema extended to include operator-related fields and
versioning controls

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-30 11:57:23 +01:00
Andrei Kvapil 935addebe9 [installer,dx] Rename cozypkg to cozyhr (#1763)
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


This PR renames cozypkg to cozyhr
https://github.com/cozystack/cozyhr

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
[installer,dx] Rename cozypkg to cozyhr
```
2025-12-30 11:56:59 +01:00