Commit Graph
3175 Commits
Author SHA1 Message Date
Aleksei Sviridkin 9fc99eed50 chore(etcd): regenerate values.schema.json and cozyrd manifest
Run 'make generate' under packages/extra/etcd to refresh the JSON schema
and the corresponding ResourceDefinition YAML after adding backup and
resources fields. Matches cozyvalues-gen v1.3.0 output used by the
pre-commit CI check.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-21 02:30:27 +03:00
Aleksei Sviridkin 6deca1b07d chore(etcd-operator): opt into CreateReplace upgrade policy for CRDs
v0.4.3 introduced new CRDs (EtcdBackup, EtcdBackupSchedule). Flux defaults to Skip on upgrade, so on existing clusters those CRDs never land without a manual kubectl apply.

Setting upgradeCRDs: CreateReplace on the etcd-operator component tells PackageReconciler to wire hr.Spec.Upgrade.Crds accordingly. etcd-operator's CRDs evolve additively between versions, so CreateReplace is safe here.

Depends on the PackageSource.Spec.Variants[].Components[].Install.UpgradeCRDs field — must ship with a cozystack-operator release that recognizes it. Without that, this field is ignored and the HelmRelease still defaults to Skip.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 18:59:21 +03:00
Aleksei Sviridkin ed77e9c6e3 fix(etcd): add helm unit tests, simplify backup guards, harden e2e
Fourth review round fixes.

Simplify backup validation by dropping placeholder-regex guards and switching default values of s3AccessKey, s3SecretKey, destinationPath, and endpointURL from angle-bracketed hints to empty strings. A single 'required' check on each field is now the whole validation for presence; the regex guard was brittle (silently stopped matching if the hint ever changed) and dropping it makes the values.yaml defaults match the behavior contract the README describes. s3:// prefix and '//' rejection still live in etcd-backup-schedule.yaml as template-time fail branches.

Add helm-unittest suites in packages/extra/etcd/tests/ covering both templates end-to-end: rendered output assertions for the happy path (bucket/key split, region passthrough, conditional rendering of optional fields) and failedTemplate assertions for every fail branch. A 'test' target is wired into the package Makefile so hack/helm-unit-tests.sh picks it up on 'make unit-tests'.

Add type: Opaque to backup-secret.yaml for consistency with the mongodb template.

E2E: wait for HelmRelease, Secret, and EtcdBackupSchedule deletion in setup() so the empty-backup case does not flake against lingering state from the previous test. Replace the fictitious minio-e2e hostname with no-such-endpoint.invalid and explicitly document that the e2e verifies schedule/CronJob materialization, not backup execution.

Restore via EtcdCluster.spec.bootstrap and the one-shot EtcdBackup CRD are still not exposed through this chart; both are explicit follow-ups called out in the README.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 17:01:58 +03:00
Aleksei Sviridkin 12fc9cd415 fix(etcd): address third review round
- Use name='etcd' in all three e2e cases. The chart pins the Helm
  release name to 'etcd' via check-release-name.yaml, so any other
  name causes template rendering to fail before the HelmRelease
  can reconcile. This is why the tenant chart hardcodes HR name
  'etcd' in packages/apps/tenant/templates/etcd.yaml.

- Consolidate all backup validation guards into
  etcd-backup-schedule.yaml so Helm's alphabetical render order no
  longer determines which failure a user sees first. Previously, a
  bad destinationPath would be masked by the earlier placeholder
  check in backup-secret.yaml.

- Use regex match '^<.*>$' for placeholder detection instead of
  exact string equality. The previous check silently stopped
  firing if anyone updated the default placeholder string in
  values.yaml; the regex form catches any angle-bracketed default.

- Reject destinationPath strings containing '//' after the s3://
  prefix. Previously 's3://bucket//subdir' silently produced a key
  beginning with '/', confusing S3 clients and retention policies.

- Document the backup contract in README.md: what enabling it
  produces, which fields are required, why placeholders are
  rejected, and that restore is a known follow-up gap.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 16:37:26 +03:00
Aleksei Sviridkin 561bc4dc23 fix(etcd): address second review round
- Mark backup.enabled as optional in the values typedef so
  generated JSON schema no longer lists it in 'required'.
  Previously, 'backup: {}' (relying on the enabled:false default)
  failed schema validation at CRD admission.

- Validate backup.destinationPath has s3:// prefix. Without this
  check, passing a path without the scheme silently used the
  first path segment as the bucket name.

- Add tail newline to values.schema.json to match repo convention
  (cozyvalues-gen output lacks it — worth fixing in that project
  separately).

- Add bats setup() that clears tenant-test etcd resources before
  each test so a failure mid-test does not leak into subsequent
  tests. Hardcoded chart-level resource names ('etcd',
  'etcd-s3-creds') make two Etcd apps in the same namespace
  collide; setup() enforces a single owner per test.

- Add dump_diagnostics() helper that dumps operator logs + the
  relevant CRD state on wait-for-ready timeouts, so CI failures
  surface actionable context instead of bare exit code 124.

- Add a third e2e case 'Create Etcd with empty backup block' that
  asserts backup:{} admits and produces no backup resources — a
  regression test for the required:['enabled'] bug fixed here.

Known scope: the upstream EtcdCluster.spec.bootstrap restore field
(also new in v0.4.3) is not exposed through this values.yaml yet.
Restoring a cluster from an existing snapshot is a follow-up PR.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 16:15:56 +03:00
Aleksei Sviridkin 0e03ed9ad4 fix(etcd): reject placeholder backup values and harden e2e
Review found three issues:

1. The guard in backup-secret.yaml only triggered on empty strings,
   not on the placeholder defaults ('<your-access-key>' etc). A user
   enabling backup.enabled=true without overriding the values would
   produce a Secret containing the literal placeholder, and the
   backup CronJob would fail at runtime with an opaque 403 far from
   the source. Add explicit template guards that fail rendering when
   any of the four required fields still hold their placeholder
   default.

2. The backup e2e test waited on .spec.schedule of the newly-created
   EtcdBackupSchedule, which is set at apply time — the wait was
   tautological and did not prove the etcd-operator reconciled
   anything. Replace it with a wait on the EtcdCluster's Ready
   condition plus a poll for the CronJob the operator generates
   from the schedule (selected via the
   etcd.aenix.io/etcdbackupschedule-name label).

3. The backup e2e test also did not wait for EtcdCluster readiness
   before tearing down. Add the same Ready wait as the baseline
   test, and verify the Secret contains the real access key value
   rather than just existing.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 16:08:43 +03:00
Aleksei Sviridkin ee8ceecc0e test(etcd): add e2e for etcd application and backup schedule
Adds two bats cases: a baseline Etcd managed-app deployment and a
deployment with backup.enabled=true that verifies the
EtcdBackupSchedule resource and the S3 credentials Secret are
materialized. The backup case uses fake S3 credentials — the
operator will not successfully push backups, but the schedule
resource itself is reconciled, which is what this test validates.

Also hardcodes the EtcdBackupSchedule and backup Secret names to
'etcd' / 'etcd-s3-creds' to match the existing hardcoded
EtcdCluster name in the chart (one etcd per tenant namespace).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 15:58:55 +03:00
Aleksei Sviridkin 51c8192e1e feat(etcd): add S3 backup schedule support
Expose the EtcdBackupSchedule CRD (etcd.aenix.io/v1alpha1) through
optional values so operators can declaratively configure periodic
S3 backups of the platform etcd cluster. Kamaji stores tenant
control-plane etcd data here, so losing this cluster means losing
every tenant cluster — a backup schedule is critical.

The backup is disabled by default (backup.enabled: false), keeping
existing installations unchanged. When enabled, a Secret is created
with AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY keys referenced by
the new EtcdBackupSchedule resource. Field naming mirrors the
existing postgres backup pattern (destinationPath, endpointURL,
s3AccessKey, s3SecretKey, schedule) for consistency.

Requires etcd-operator v0.4.3+ (shipped in the previous commit).

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 15:57:51 +03:00
Aleksei Sviridkin 8734c38da3 chore(etcd-operator): update vendored chart to v0.4.3
Upstream release notes: https://github.com/aenix-io/etcd-operator/releases/tag/v0.4.3

Notable changes:
- Add EtcdBackup CRD for declarative etcd snapshots (#307)
- Add EtcdBackupSchedule CRD for scheduled backups (#307)
- Expose metrics port on client service (#299)
- Replace deprecated gcr.io/kubebuilder/kube-rbac-proxy with
  quay.io/brancz/kube-rbac-proxy:v0.18.1 (#308)
- Update Kamaji example manifest

Note: Helm 3 does not install CRDs on upgrade. Existing clusters
require manual application of the new EtcdBackup and
EtcdBackupSchedule CRDs:

  kubectl apply --filename packages/system/etcd-operator/charts/etcd-operator/crds/

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-20 15:53:41 +03:00
Timofei Larkin 171c793cc5 feat(platform): add resourcePreset labels to workloadmonitor labels (#2416)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note

```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* WorkloadMonitor labels with the workloads.cozystack.io/ prefix are now
propagated onto created Workloads; created Workloads always include the
reserved workloads.cozystack.io/monitor label and source-object labels
take precedence on conflicts.
* Helm app charts now add workloads.cozystack.io/resource-preset
metadata to WorkloadMonitor manifests.

* **Tests**
* Added tests covering label extraction, propagation, conflict
resolution, and backward compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-20 16:12:57 +04:00
Andrei Kvapil e26894e71a [Backport release-1.3] fix(linstor): restrict linstor-gui to cozystack-cluster-admin group (#2419)
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.

Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.

Assisted-By: Claude <noreply@anthropic.com>

(cherry picked from commit 9b54e46723)

<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does
Backport of #2415 to release-1.3.

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note

```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated README to document that LINSTOR GUI access is restricted to
members of the `cozystack-cluster-admin` Keycloak group. Non-members
receive a 403 error.

* **Chores**
* Implemented group membership validation for LINSTOR GUI access
control.
  * Added validation tests for group membership enforcement.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-17 21:11:47 +02:00
Myasnikov Daniil 2461c239b0 fix(linstor): restrict linstor-gui to cozystack-cluster-admin group
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.

Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 9b54e46723)
2026-04-17 20:21:14 +05:00
Andrey Kolkov 2c141d3f38 feat(platform): add resourcePreset labels
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-04-17 15:04:25 +04:00
Timofei Larkin 4053fad9ed [apps] feat(bucket): add WorkloadMonitor and BucketClaim tracking to controller (#2391)
## What this PR does

Adds BucketClaim support to WorkloadMonitorReconciler, bringing S3
buckets in line with how Pods, PVCs, and Services are already tracked as
Workload resources.

- Add WorkloadMonitor CR to the bucket Helm chart with
`app.kubernetes.io/instance` label on BucketClaim for selector matching
- Add `reconcileBucketClaimForMonitor()` following the same pattern as
`reconcilePVCForMonitor()` — watches COSI `BucketClaim` objects and
creates corresponding `Workload` CRDs
- Query SeaweedFS bucket size metrics (logical + physical) from
VictoriaMetrics, with the monitoring endpoint resolved automatically
from the `namespace.cozystack.io/monitoring` namespace label
- COSI API types dependency (`container-object-storage-interface-api`)
for typed BucketClaim access

No configuration flags needed — the controller discovers the monitoring
stack for each tenant namespace automatically.

### Result

When a BucketClaim is matched by a WorkloadMonitor, the controller
creates a Workload with S3 storage metrics:

```yaml
apiVersion: cozystack.io/v1alpha1
kind: Workload
metadata:
  name: bucket-bucket-test-billing
  namespace: tenant-testing
  labels:
    workloads.cozystack.io/monitor: bucket-test-billing
  ownerReferences:
  - apiVersion: objectstorage.k8s.io/v1alpha1
    kind: BucketClaim
    name: bucket-test-billing
status:
  kind: bucket
  type: s3
  operational: true
  resources:
    s3-buckets: "1"
    s3-storage-bytes: "10485864"             # 10 MB logical
    s3-physical-storage-bytes: "20971728"    # 20 MB physical (replication factor 2)
```

- `s3-storage-bytes` — logical size (what the user stored), from
`SeaweedFS_s3_bucket_size_bytes`
- `s3-physical-storage-bytes` — physical size (with replicas), from
`SeaweedFS_s3_bucket_physical_size_bytes`
- When monitoring is not configured for the namespace, only `s3-buckets:
1` is tracked
- Sizes refresh every 60 seconds via `RequeueAfter`

Tested on a live dev cluster with SeaweedFS deployed in `tenant-root`
and a bucket created in `tenant-testing`.

### Release note

```release-note
[apps] Add WorkloadMonitor to bucket application. BucketClaims are now tracked as Workload resources with S3 storage size metrics resolved automatically from the tenant monitoring stack.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Monitor COSI bucket claims: creates/updates Workload records
reflecting bucket readiness and requeues periodically when buckets
exist.
* Prometheus-backed bucket storage metrics, with SeaweedFS-aware sizing
when available.
* Helm chart additions: WorkloadMonitor resource and instance label on
BucketClaim templates.

* **Tests**
* Unit tests covering bucket-monitor flows, metric querying, URL
resolution, and requeue behavior.

* **Chores**
  * Updated module dependencies to enable COSI integration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-17 12:37:57 +04:00
ZverGuyandClaude 3803e6e070 fix(controller): use dedicated HTTP client instead of http.DefaultClient
Replace http.DefaultClient with a package-level *http.Client with an
explicit 10-second timeout. Avoids sharing the process-wide default
transport with other libraries.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:48 +03:00
ZverGuyandClaude c72fadec3e fix(controller): scope Prometheus query to only matched bucket names
Pass bucket names from BucketClaim.Status.BucketName into the PromQL
query as a bucket=~"name1|name2" filter. This prevents O(N²) load
where N WorkloadMonitors each fetch all buckets globally.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 8183b4669a perf(controller): skip Prometheus calls when no BucketClaims matched
Wrap resolvePrometheusURL and queryAllBucketMetrics in a
len(bucketClaimList.Items) > 0 guard. Avoids unnecessary namespace
GET and HTTP request to Prometheus on every reconcile of non-bucket
WorkloadMonitors (postgres, redis, kubernetes, etc.).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 475d24b029 perf(controller): batch all bucket metrics into single Prometheus query
Replace 2×N per-bucket HTTP requests with a single query that fetches
all SeaweedFS bucket size metrics at once:

  {__name__=~"SeaweedFS_s3_bucket_(size|physical_size)_bytes"}

Results are keyed by bucket name in memory, then looked up per
BucketClaim. This reduces HTTP round-trips from 2N+1 to 2 (one
namespace lookup + one Prometheus query) regardless of bucket count.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude ed9808fad6 perf(controller): resolve Prometheus URL once per reconcile, not per BucketClaim
Move resolvePrometheusURL call before the BucketClaim loop and pass
the URL as parameter. Avoids redundant namespace lookups and reduces
reconcile time from 2×N+1 to 2×N HTTP calls (where N = BucketClaims).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 260ce84e5f fix(controller): distinguish empty buckets from monitoring unavailable
Change queryPrometheusMetric to return (int64, bool) so callers can
emit s3-storage-bytes=0 for empty buckets while omitting the field
entirely when monitoring is not configured.

Also:
- Add RBAC marker for core/namespaces GET (used by resolvePrometheusURL)
- Log namespace read errors instead of silently returning empty URL
- Add TestQueryPrometheusMetricZeroValue test

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude a6f74f3198 refactor(controller): remove s3-buckets count resource from bucket workloads
Bucket existence is already tracked via LifetimeHours. The s3-buckets
count resource produced a meaningless "s3-buckets-Hours" fallback type
in billing. Only storage size metrics (s3-storage-bytes,
s3-physical-storage-bytes) are now set on bucket Workloads.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 40620b2911 fix(controller): fix nil map panic and scientific notation parsing
- Initialize workload.Labels map inside CreateOrUpdate mutate function
  to prevent nil map panic when existing Workload has no labels
- Use strconv.ParseFloat instead of resource.ParseQuantity for
  Prometheus metric values, which may use scientific notation
  (e.g. "1.048576e+06") that ParseQuantity does not support

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 7f7f4b218d refactor(controller): resolve Prometheus URL from namespace labels
Replace static --prometheus-url flag with dynamic resolution from
namespace.cozystack.io/monitoring label. Each tenant namespace knows
which tenant hosts its monitoring stack, so the controller constructs
the vmselect URL automatically. This correctly handles multi-tenant
setups where different tenants may use different monitoring instances.

- Remove PrometheusURL field from WorkloadMonitorReconciler struct
- Remove --prometheus-url flag and prometheusUrl chart value
- Add resolvePrometheusURL() that reads namespace label
- queryPrometheusMetric() now accepts prometheusBaseURL as parameter
- Add tests for resolvePrometheusURL with and without label

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 0e6b8f7ba1 fix(controller): address review findings for BucketClaim reconciler
- Set replicas/minReplicas to 0 in bucket WorkloadMonitor (buckets
  have no pods, minReplicas=1 would mark monitor as not operational)
- Remove dead IsNotFound check on List (List returns empty, not 404)
- Trim trailing slash from PrometheusURL before path append
- Add strings import for TrimRight

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude bfdfe989e0 fix(controller): add HTTP status check and limit response body size
- Check resp.StatusCode before parsing Prometheus response
- Limit response body read to 1 MB via io.LimitReader
- Use strings.HasPrefix in test instead of fragile slice indexing
- Add TestQueryPrometheusMetricServerError for 500 responses

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 04f340ab84 feat(controller): add physical storage size metric for S3 buckets
Query SeaweedFS_s3_bucket_physical_size_bytes alongside the logical
size metric. Physical size includes all replicas and reflects actual
disk usage, while logical size reflects what the user stored.

Refactor queryBucketSizeBytes into generic queryPrometheusMetric
to reuse for both metrics.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude ddbc5f6f83 feat(chart): add prometheus-url flag to cozystack-controller deployment
Pass --prometheus-url to the controller container when configured in
values. This enables querying SeaweedFS bucket size metrics from a
Prometheus-compatible API for S3 bucket billing.

RBAC already covers BucketClaim access via existing wildcard rule
(apiGroups: ['*'], resources: ['*'], verbs: ["get", "list", "watch"]).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 0aab19f500 feat(controller): add SeaweedFS bucket size metrics via Prometheus
Query SeaweedFS_s3_bucket_size_bytes from a Prometheus-compatible API
to populate s3-storage-bytes resource on bucket Workloads. The
Prometheus URL is configurable via --prometheus-url flag. When set,
bucket WorkloadMonitors are requeued every 60s to keep sizes current.

When Prometheus is not configured, buckets still get tracked with
s3-buckets=1 for existence-based billing.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude 92286213ab feat(controller): add BucketClaim support to WorkloadMonitorReconciler
Add reconcileBucketClaimForMonitor() that watches COSI BucketClaim
objects and creates Workload CRDs with s3-buckets resource, following
the same pattern as PVC and Service reconcilers. This enables the
billing pipeline to discover and track S3 buckets per tenant.

Changes:
- Add COSI API types dependency (container-object-storage-interface-api)
- Register cosiv1alpha1 scheme in controller main
- Add BucketClaim watch in SetupWithManager
- Add BucketClaim list + reconcile in Reconcile loop
- Add RBAC annotation for objectstorage.k8s.io/bucketclaims
- Add unit tests for BucketClaim reconciliation

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:58:47 +03:00
ZverGuyandClaude ee4e71b139 feat(bucket): add WorkloadMonitor for billing integration
Add WorkloadMonitor CR to bucket Helm chart so that the billing
pipeline can discover and track S3 buckets. Add instance label
to BucketClaim metadata for WorkloadMonitor selector matching.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-17 10:53:43 +03:00
myasnikovdaniil 985cb9c889 fix(linstor): restrict linstor-gui to cozystack-cluster-admin group (#2415)
## What this PR does

Tightens access control on the `linstor-gui` Ingress so it is reachable
only by members of the `cozystack-cluster-admin` Keycloak group (the
same group that grants cluster-admin RBAC on the host cluster).

Before this change, the oauth2-proxy gatekeeper in front of
`linstor-gui` enforced only that the user could authenticate against the
`cozy` realm (`--email-domain=*`, no group restriction). Any realm user
— including tenant-scoped accounts — could reach the UI and the
underlying LINSTOR controller REST API, which the gatekeeper proxies
with a static mTLS client cert and no per-user RBAC.

Changes:

- `packages/system/linstor-gui/templates/gatekeeper.yaml` — add
`--allowed-group=cozystack-cluster-admin` and include `groups` in
`--scope` so the claim is available for validation. Extend the
top-of-file rationale block to document the gate.
- `packages/system/linstor-gui/tests/ingress_auth_test.yaml` — assert
both new args on the rendered Deployment.
- `packages/system/linstor-gui/README.md` — document the group
restriction under "Option 1 — Keycloak-protected Ingress".

No Keycloak-side changes are required: the `cozystack-cluster-admin`
KeycloakRealmGroup and the `groups` client scope are already provisioned
by `keycloak-configure`.

Port-forward access (Option 2) is unchanged; it remains gated by kubectl
RBAC on `cozy-linstor`.

### Release note

```release-note
fix(linstor): restrict `linstor-gui` Ingress access to members of the `cozystack-cluster-admin` Keycloak group.
```
2026-04-17 12:36:11 +05:00
Myasnikov Daniil 9b54e46723 fix(linstor): restrict linstor-gui to cozystack-cluster-admin group
Before this change oauth2-proxy fronting linstor-gui only enforced that
the user could authenticate against the `cozy` Keycloak realm
(`--email-domain=*`, no group restriction). Any realm user could reach
the UI and, through it, the LINSTOR controller REST API — which the
gatekeeper proxies with a static mTLS client cert and no per-user RBAC.

Add `--allowed-group=cozystack-cluster-admin` and include `groups` in
the OIDC scope so the claim is present at validation time. The
`cozystack-cluster-admin` KeycloakRealmGroup and the `groups` client
scope are already provisioned by keycloak-configure, so no cluster-wide
changes are needed.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-17 11:32:44 +05:00
Andrei Kvapil 91dc23efda docs(changelog): add v1.3.0-rc.1 changelog and update agent instructions (#2410)
## What this PR does

Adds the v1.3.0-rc.1 release changelog covering all changes since
v1.2.0, and updates the changelog agent instructions to include two new
side repositories.

**Changelog** (`docs/changelogs/v1.3.0-rc.1.md`):
- Feature Highlights: storage-aware scheduling, LINSTOR GUI, VM Default
Images, WorkloadsReady conditions, cross-namespace VM backup restore
- Covers main repo, website, talm, ansible-cozystack, and
external-apps-example changes
- Backported fixes marked with *(backported to v1.2.x)* annotation

**Agent instructions** (`docs/agents/changelog.md`):
- Adds `external-apps-example` and `ansible-cozystack` to the optional
repositories list

### Screenshots

N/A — documentation only.

### Release note

```release-note
docs(changelog): add changelog for v1.3.0-rc.1
```
2026-04-16 13:19:04 +02:00
Andrei KvapilandClaude 1e1bb3eb37 docs(agents): add external-apps-example and ansible-cozystack to changelog instructions
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-04-16 13:18:18 +02:00
Andrei KvapilandClaude e76b1ccc69 docs(changelog): add changelog for v1.3.0-rc.1
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-04-16 13:18:14 +02:00
Andrei Kvapil d9657bc4e9 Release v1.3.0-rc.1 (#2408)
This PR prepares the release `v1.3.0-rc.1`.
v1.3.0-rc.1
2026-04-16 12:30:49 +02:00
Timur Tukaev 53ba998777 Update README.md (#2409)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes for system components: dashboard, platform, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Scopes for managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Scopes for development and maintenance: api, hack, tests, ci, docs,
maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note

```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Updated the README's introductory description to refine the platform
positioning and improve clarity on its core capabilities.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-16 11:33:39 +02:00
Timur Tukaev 624f00f9c2 Update README.md
Signed-off-by: Timur Tukaev <90071493+tym83@users.noreply.github.com>
2026-04-16 14:30:51 +05:00
cozystack-ci[bot] 12bf6b0e26 Prepare release v1.3.0-rc.1
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com>
2026-04-16 08:50:30 +00:00
Andrei Kvapil 1a210a2907 feat(application): add WorkloadsReady condition and Events tab (#2356)
## What this PR does

Adds two features to improve application observability in the dashboard,
plus a bug fix:

### 1. WorkloadsReady condition on Application status
- Queries WorkloadMonitor resources to determine if all application pods
are running
- Exposes `WorkloadsReady` as a separate condition alongside `Ready`
- `Ready` continues to reflect HelmRelease state only — no override — to
preserve backward compatibility with existing tooling (kubectl wait,
GitOps health checks) and avoid false-negative Ready=False during normal
startup windows
- Handles three states: operational, not operational, unknown (pending
reconciliation)
- Fails open on WorkloadMonitor query errors (prefers availability)
- Integrates with Application Watch to emit MODIFIED events on
WorkloadMonitor changes
- Registers cozystack.io/v1alpha1 types in API server scheme with
informer cache

### 2. Events tab in dashboard
- Shows Kubernetes Events scoped to the application's namespace
- Uses status.namespace for Tenant applications (consistent with
Resource Quotas tab)
- Includes both lastTimestamp and eventTime columns for Kubernetes
version compatibility

### 3. Bug fix: WorkloadMonitor Operational status persistence
- Operational field was written to stale `monitor` variable instead of
`fresh` inside RetryOnConflict, so it was never persisted to the cluster

Closes #2359
Closes #2360

### Release note

```release-note
[dashboard] Added Events tab to application detail pages showing namespace-scoped Kubernetes Events
[application] Added WorkloadsReady condition exposing aggregated WorkloadMonitor status
[workloadmonitor] Fixed bug where Operational status was never persisted to the cluster
```
api/apps/v1alpha1/v1.3.0-rc.1
2026-04-16 10:18:55 +02:00
myasnikovdaniil 482d813d01 Add vm-default-images package (#2258)
<!-- Thank you for making a contribution! Here are some tips for you:
- Start the PR title with the [label] of Cozystack component:
- For system components: [platform], [system], [linstor], [cilium],
[kube-ovn], [dashboard], [cluster-api], etc.
- For managed apps: [apps], [tenant], [kubernetes], [postgres],
[virtual-machine] etc.
- For development and maintenance: [tests], [ci], [docs], [maintenance].
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same [label] as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
[vm-default-images] Added package that brings set of images that can be used clusterwide
[vm-disk] Updated source "image" for prettier dropdown selection
[vm-disk] Added new source for vm-disk called disk - to use as source vm-disk from same namespace.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Clone VM disks by specifying an existing vm-disk as the source.
* Global default image collection and chart to publish pre-provisioned
images.

* **UI**
* Forms provide selectable lists for default images and existing VM
disks.

* **Migration**
* Migration to rename existing image DataVolumes to the new
default-images naming and bumped migration version.

* **Documentation**
* VM disk docs and README updated to reflect image sourcing and cloning.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-16 08:05:23 +05:00
Aleksei Sviridkin 9cc5deeabe fix(dashboard): fall back to .firstTimestamp in Event Time column
core/v1 Events populate .lastTimestamp and .firstTimestamp but leave
.eventTime null; events.k8s.io/v1 Events do the opposite. The previous
column bound to .eventTime alone and rendered 'Invalid Date' for every
Helm-generated event.

Extend createTimestampColumn with an optional second jsonPath that is
encoded as a nested reqsJsonPath fallback in the template, and use
.eventTime → .firstTimestamp for Event Time and .lastTimestamp →
.eventTime for Last Seen so both APIs render correctly.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-16 01:10:10 +03:00
Aleksei Sviridkin b28bd598bb docs(ci): require screenshots for UI changes in PR template (#2407)
## What this PR does

Add a "Screenshots" section to the pull request template.
PRs that include UI changes are now required to attach screenshots
or screen recordings demonstrating the visual impact of the changes.

### Release note

```release-note
docs(ci): add mandatory screenshots requirement for UI-related pull requests
```
2026-04-16 00:36:10 +03:00
ZverGuyandClaude 2e6a685411 docs(ci): require screenshots for UI changes in PR template
Add a Screenshots section to the pull request template that requires
contributors to attach screenshots or screen recordings when their
changes affect the UI.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-04-16 00:34:52 +03:00
Timur Tukaev 545eecfdb2 Add Mattia Eleuteri (@mattia-eleuteri) as Maintainer (#2345)
## Maintainer Nomination: Mattia Eleuteri

Per the process in
[CONTRIBUTOR_LADDER.md](./CONTRIBUTOR_LADDER.md#maintainer), this PR
nominates **Mattia Eleuteri** (@mattia-eleuteri, Hidora) as a
Maintainer.

### Summary

Mattia has demonstrated sustained, high-quality contributions across
CSI/storage, networking, monitoring, and security. Key areas include:

- CSI fixes for migration, RWX NFS mounts, and multi-node volumes
- CiliumNetworkPolicy and VPC peering for multi-tenant environments
- Monitoring improvements (vmagent, infrastructure dashboards)
- Security scanning and hardening
- New platform packages (external-dns)

Full discussion and vote: #2343

### Process checklist

- [x] @mattia-eleuteri please comment confirming you agree to all
[Maintainer responsibilities](./CONTRIBUTOR_LADDER.md#maintainer)
- [x] Majority of current Maintainers (5 of 8) approve this PR

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Documentation**
* Added Mattia Eleuteri (Hidora) to the project maintainers list,
assigned to CSI, Storage, Networking & Security.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-15 20:52:46 +02:00
tym83andClaude Opus 4.6 8fc82b18c0 Add Mattia Eleuteri (@mattia-eleuteri) as Maintainer
Ref: #2343

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

-e
Signed-off-by: tym83 <6355522@gmail.com>
2026-04-15 23:46:43 +05:00
Timur Tukaev 56cb8783d7 Add Matthieu Robin (@matthieu-robin) as Maintainer (#2346)
## Maintainer Nomination: Matthieu Robin

Per the process in
[CONTRIBUTOR_LADDER.md](./CONTRIBUTOR_LADDER.md#maintainer), this PR
nominates **Matthieu Robin** (@matthieu-robin, Hidora) as a Maintainer.

### Summary

Matthieu has contributed consistently across managed applications,
platform quality, and community engagement. Key areas include:

- Full managed OpenSearch service (operator, packaging, validation)
- Workload monitoring with instance profile labels
- Operational hardening (etcd-defrag resource limits)
- Kubernetes benchmarking and platform validation
- CozySummit speaker and Program Committee member

Full discussion and vote: #2344

### Process checklist

- [x] @matthieu-robin please comment confirming you agree to all
[Maintainer responsibilities](./CONTRIBUTOR_LADDER.md#maintainer)
- [x] Majority of current Maintainers (5 of 8) approve this PR

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated the project maintainers documentation to reflect the current
team composition and responsibility assignments. The update documents
team members responsible for overseeing managed applications, platform
quality initiatives, and benchmarking efforts to ensure comprehensive
coverage of critical project areas.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-15 20:33:32 +02:00
Myasnikov DaniilandClaude Opus 4.6 e6ba9817be [platform] Make vm-default-images opt-in, not default in iaas bundle
The package imports ~320Gi of golden-image PVCs (ubuntu-noble, fedora,
debian, centos, etc.) as soon as it's installed. On small test and dev
clusters that's enough to consume the entire replicated storage pool,
after which no tenant PVCs — including the ones E2E itself provisions —
can be bound. It's also unreasonable to force that cost on every iaas
user: many deployments don't need prebuilt images at all, and the ones
that do often want to curate their own subset.

Switch the bundle entry from 'package.default' to 'package.optional.default',
matching the treatment already applied to gpu-operator directly below it.
Users who want the golden images can opt in via:

    bundles:
      enabledPackages:
        - cozystack.vm-default-images

The package, its Source, and migration 38 all stay in place — nothing
else changes for users who explicitly enable it. Users who previously
relied on the bundle auto-installing it will need to add the package
to enabledPackages on upgrade; this is called out in the release note.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-04-15 19:47:25 +05:00
Aleksei Sviridkin d369b64d71 fix(application): address review feedback for WorkloadMonitor integration
Tenant applications live in a child namespace computed from the Tenant
name, not in the HelmRelease's namespace. Scope the WorkloadMonitor
watch cluster-wide when kindName is Tenant and reverse-map WM events
back to the owning HelmRelease via computeTenantNamespace. The
condition-enrichment path now looks up monitors in the computed child
namespace too, so WorkloadsReady is populated for Tenants.

Buffer WorkloadMonitor events that arrive before the initial-events-end
bookmark and replay them once the bookmark is emitted, to preserve the
watch-list contract while not dropping workload-state transitions that
happen during the snapshot window.

Pass the fresh WorkloadMonitor object from the watch event into the
conversion path so WorkloadsReady reflects the state that triggered
the event, even when the cache client is lagging behind the watch
client.

Derive WorkloadsReady.LastTransitionTime from a stable source
(max of HelmRelease creation and condition timestamps, plus monitor
timestamps) instead of metav1.Now(), so repeated conversions of the
same underlying state produce identical timestamps.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-15 17:22:15 +03:00
Aleksei Sviridkin dc3387c635 fix(workload-monitor): use fresh spec for MinReplicas check on retry
Inside the RetryOnConflict block, derive the operational status from
fresh.Spec.MinReplicas instead of the stale monitor.Spec.MinReplicas
so that concurrent spec updates observed by the retry are respected.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-15 17:20:45 +03:00