A BucketClaim deleted with --cascade=orphan and recreated leaves its
bucket-bucket-<name> Workload without owners. The migration recognised a
legacy Workload only by the claim's UID among its owners, so that object
stayed next to the new one with a frozen size and the bucket was metered
twice. Treat an ownerless legacy Workload that carries the monitor's
label as the claim's: delete it with the same preconditions, but take no
sizes from it, since they may predate the claim.
The size carry-over kept whatever the Workload at the new name had
recorded. When that Workload was not yet owned by the claim, an ownerless
leftover, its sizes were not written for this bucket, yet they won over
the migrating legacy Workload's. Prefer the recorded sizes only when the
claim already owned the Workload.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
When the new Workload name was held by another owner the claim was
skipped altogether, so its legacy Workload stopped being refreshed and
billing kept reading the last size it had recorded. That happens briefly
with Bucket applications "x" and "bucket-x" until the first one has
migrated, and for good with a claim "x" outside the Bucket application
next to the application "x", which map to the same name.
Keep reconciling the bucket under its legacy name instead, without
migrating, until the new name is free. Skip it only when the legacy name
is held by another owner as well, which is the only option left for a
claim without the prefix, whose two names coincide; the warning event
then says that the bucket is not metered.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
A controller rolled back to the doubled prefix runs CreateOrUpdate on
bucket-bucket-x, which by then is the renamed Workload of the Bucket
application "bucket-x", and appends claim bucket-x as a second owner.
After the next upgrade claim bucket-x took that object for its own
legacy Workload: it carried the other bucket's size over and deleted the
object, and the other bucket's Workload came back without a size.
Treat a legacy Workload as the claim's own only when the claim is its
sole owner. When it has other owners, take neither its sizes nor the
object: drop the claim's owner reference and leave the rest.
Delete the legacy Workload with UID and resourceVersion preconditions,
so a Workload that took over the name between the lookup and the delete
survives; a conflict is not an error and the next reconcile retries.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
The adoption check only refused a Workload owned by another BucketClaim,
and the kind test in it had no case that could tell it apart: a bucket
Workload only ever has BucketClaim owners. Refuse any Workload that has
owners when the claim is not among them, which needs no kind at all, and
keep taking over one that has lost every owner reference, since nothing
else would ever release its name. Pin the ownerless case.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
On a live cluster the cache records the BucketClaim kind in a Workload's
owner reference, so a claim's own Workload passes through the "owned by
another claim" check on every reconcile, and only the own-UID match keeps
it from being skipped. The fake client leaves that kind empty, so no test
exercised it. Seed a Workload whose owner reference carries the kind and
the claim's UID, and expect it to be updated with no warning event.
Pin the physical size carry-over alongside the logical one, and word the
warning event so it holds when two claims map to one name for good.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
With Bucket applications "x" and "bucket-x" in one namespace, the legacy
name of the first, bucket-bucket-x, is the new name of the second. When
the second reconciled first, CreateOrUpdate took over the first one's
legacy Workload, which already carried bucket x's size, and appended a
second owner. The first claim then found its own UID among the owners,
treated the object as its legacy Workload and deleted it, and the
second bucket's Workload came back without a size.
Leave a Workload alone while it is owned by a different BucketClaim, log
it and emit a BucketWorkloadNameTaken warning on the monitor. In the
case above the other claim renames its Workload away on its own
reconcile, and the name is free on the next pass. The same refusal keeps
two claims that map to one name, such as "x" and "bucket-x", from
sharing one Workload.
Pin the edge cases the migration depends on: the reconcile order above
in both directions, a claim without the prefix keeping its own Workload,
and a recorded size taking precedence over the legacy one.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
A bucket's Workload was named "bucket-" + the BucketClaim name, and a
Bucket application names its claim after its release, which already
starts with "bucket-". A bucket an operator knows as cozy-backups
showed up as bucket-bucket-cozy-backups in every billing usage report.
Add the prefix only when the claim name does not carry it. Claims from
other charts, such as Harbor's <release>-registry, keep their names.
Existing Workloads under the doubled name are owned by their claim and
would otherwise live on next to the new one, metering the bucket twice.
When the replacement is reconciled, take the legacy Workload's recorded
sizes as the starting point, so the rename leaves no gap in the metered
size, and delete the legacy Workload. Only a Workload owned by the same
BucketClaim is touched.
Billing sees the renamed Workload as a new consumer: a usage report
spanning the upgrade lists the bucket once under each name.
Fixes#5014
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
A disk whose HTTP source answers 404 or does not resolve never leaves
ImportInProgress: CDI fails an import only on a fatal error and restarts
the importer pod otherwise, so the condition only ever said the disk was
still importing. The reason it will not finish was visible only in the
importer logs and on the DataVolume, whose Running condition carries the
error through the back-off between restarts.
Appending that message lets the user see why the disk is stuck from the
application status and the console, which already shows the
WorkloadsReady message for a disk.
Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The field comment still said DataVolumes do not count while the reader
is nil, and that the reader appears once the kind is served. Neither
holds: the reader is installed only after the informer has synced, and
until then a monitor keeps its stored verdict, which still decides
Operational.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A reconcile that runs before the DataVolume watch starts finds no
reader, keeps the monitor's stored verdict and is not requeued. The
watch's initial replay then delivers only the DataVolumes that still
exist, so a monitor whose DataVolume was deleted while the controller
was down kept reporting it not ready until the periodic resync, hours
later.
Installing the reader now queues every monitor that holds a DataVolume
verdict, through the controller's own queue, so each is recomputed
against the synced cache. Monitors without a verdict are left alone:
the replay already reaches every DataVolume they could select. If the
monitors cannot be listed, the next discovery poll retries.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
CDI copies the DataVolume labels onto the PVC it creates, so the vm-disk
monitor also publishes a Workload for that PVC, and its readiness was
bind state alone. Mid-import the PVC is already Bound: the Workload read
operational while the monitor, and the Application's WorkloadsReady,
said the disk was not populated.
A PVC whose controller is a DataVolume the monitor finds not ready now
reads not operational. While the DataVolumes cannot be read, its
Workload keeps the verdict it has, as the monitor does.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
While the DataVolume watch syncs, every WorkloadMonitor of every kind
is requeued every few seconds, and an informer that cannot sync keeps
that window open. Each pass wrote status unconditionally, so a cluster
with many monitors sent the API server a steady stream of writes that
changed nothing. Comparing against the stored status first leaves the
requeue as cache reads only.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A failed DataVolume List keeps the monitor's last verdict and retries,
and the only trace of it was a controller log line. A monitor with no
verdict yet then reads operational while its disks are unreadable, with
nothing a tenant can see saying why. A Warning event on the monitor
makes the failure visible where the bucket metrics failure already is.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A controller-runtime Kind source adds an event handler to the informer
when it starts and never removes it; cancelling it on a sync timeout
only ends its context. Building a new source on every failed sync
attempt therefore left one more handler on the DataVolume informer per
retry, each enqueueing the same events, until the controller restarted.
The source is now started once and later attempts only wait for the
informer to sync. Because the started source keeps delivering events
between attempts, the syncing flag stays set until the reader is
installed, so a reconcile queued in that gap is requeued as well.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Starting the DataVolume source replays every DataVolume to its
handler at once, and the handler queues the matching monitors, while
the reader is installed only after the source has synced. A monitor
reconciled in that window saw no reader, kept its last verdict and
returned with nothing left to queue it again, so a disk that failed
or finished while the controller was down could keep a wrong
WorkloadsReady until the next unrelated event.
Such a reconcile still publishes the replica counts and now asks to
be requeued after a few seconds, until the reader is in place or the
sync attempt gives up. It is a requeue rather than an error because
the window is expected: an error would count as a reconcile failure
and log a line for every monitor in the cluster while it lasts.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
WorkloadsReady=False covers different situations: a stopped VM has
no pod and its monitor reports it not operational on purpose, while a
disk whose DataVolume never populates is broken. A consumer that wants
to react to one and not the other had only the message text to go by.
The WorkloadMonitor status gains a Reason next to Message, set to
DataVolumeNotReady by the DataVolume check, and the Application API
carries it into the WorkloadsReady condition in place of the generic
reason whenever a monitor that is not operational names one.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The DataVolume List sits on the reconcile path of every
WorkloadMonitor, and returning its error before the status write froze
the replica counts and Operational of a monitor that selects no
DataVolume at all whenever CDI's API hiccupped.
The status is now written either way, with the last DataVolume verdict
kept whenever the DataVolumes could not be read, and the error still
returned so the request is retried. The same applies while no reader
is installed yet, so a stuck disk does not read as ready for the
window between a controller restart and the DataVolume informer
syncing.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
For a vm-disk, WorkloadsReady is the one condition that reports
whether CDI populated the disk, and its generic "One or more workloads
are not operational" gives an operator nothing to act on: a disk has
no workloads, and the stuck DataVolume and its phase are exactly what
they need to know.
The WorkloadMonitor status gains a message that the controller fills
with each selected DataVolume that is not ready and its phase, and the
Application API uses the messages of the monitors that are not
operational in WorkloadsReady instead of the generic text whenever one
of them names a cause.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A vm-disk application reported its workloads operational whatever
state its DataVolume was in, so an import or clone that never
finished left the disk looking healthy while it was unusable. Its
monitor has minReplicas 0 and only pods fed Operational.
The monitor now also lists the DataVolumes its selector matches and
reports not operational while any of them is in flight, Failed,
Unknown or without a phase. UploadReady counts as in flight: CDI
holds it from the moment the upload server is ready until the
transfer succeeds, so it covers an empty disk and a half-uploaded
one alike. Waiting for the first consumer counts as ready, since
that is where such a disk rests, and so does a phase this
controller does not know, so a CDI release that renames one cannot
turn working disks red.
This is deliberately not a Helm wait: a readiness gate holds a
helm-controller worker for its whole window, and a disk that never
populates would keep taking it back. The signal lands in the
application's WorkloadsReady condition instead, and Ready keeps
following the HelmRelease.
CDI ships with the IaaS bundle only and can be installed while this
controller is running. A watch on a kind whose CRD is absent keeps
the controller from starting, so the DataVolume watch is added from a
runnable that polls discovery until the kind is served. It starts on
the first reconcile, the earliest point at which a new watch is
started at once, rather than on the next tick, so a disk created
just after a restart is not read as populated for a poll interval.
The reader is installed only once that watch has synced within a
bound: a reconcile context has no deadline, and a List on an
informer that never syncs would park every WorkloadMonitor reconcile
for good.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Two small bug fixes, one per commit. Both were first proposed by
@officialasishkumar in #2334 and #2336. Those PRs went stale, and part
of each was fixed on main in the meantime, so this carries only what is
still broken.
ClickHouse Keeper hosts rendered without a cluster domain, as
`chk-<name>-keeper-cluster1-0-0.<namespace>.svc.` with nothing after
`svc.`. The zookeeper block in `clickhouse.yaml` redeclared
`$clusterDomain` from `.Values.clusterDomain`, which the chart never
defines, and that shadowed the value read from `_cluster.cluster-domain`
at the top of the template. Keeper is enabled by default, so every
ClickHouse release got these hosts. The keeper name test set
`clusterDomain` itself and hid the bug. It now sets only
`_cluster.cluster-domain`, like the platform does, and fails without the
fix.
The WorkloadMonitor controller panicked on a PVC with no
`storageClassName`, which is any PVC that takes the cluster default
class. The check used `||` where it needed `&&`. controller-runtime
recovers the panic, but the reconcile of that monitor fails on every
retry and never reaches its status update. A PVC with an empty
`storageClassName` also got a resource key with an empty class name
instead of `default`. A new table test covers unset, empty and explicit
classes.
The duplicate `template` key in the ClickHouse backup CronJob from #2334
and the `Operational` status fix from #2336 are already on main, so they
are not in this PR.
### Screenshots
Not a UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
fix(clickhouse, workloadmonitor): ClickHouse Keeper node hostnames now include the cluster domain, and the WorkloadMonitor controller no longer panics on a PVC without a storage class.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Workload resource labels now use the default storage class when a
PVC’s storage class is unset or empty, while continuing to reflect
explicitly configured storage classes.
* ClickHouse ZooKeeper hostnames now consistently use the configured
cluster domain, or `cozy.local` when no domain is set. This helps
hostnames align with the cluster’s domain configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
maps.Copy(workload.Labels, bc.Labels) can bring in a workloads.cozystack.io/bucket-class label directly set on the BucketClaim, which would survive unmodified when spec.bucketClassName is empty or invalid, leaving the Workload reporting a class that does not match the source BucketClaim.
Signed-off-by: harshitrwt <harshit.rawat1804@gmail.com>
The storage class check used || where it needed &&, so a PVC with no
storageClassName (one that takes the cluster default) was dereferenced
as a nil pointer. controller-runtime recovers the panic, but the
reconcile of that WorkloadMonitor fails on every retry and never
reaches its status update. A PVC with an empty storageClassName
produced a resource key with an empty class name instead of falling
back to "default".
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
This replaces the open Renovate PRs with one rollup. Go modules move to
their newest releases that keep the Kubernetes libraries and
controller-runtime where they are, the toolchain and builder images move
to Go 1.27.1, and the first-party code gets the Go 1.27 modernizers.
Tests for the code the bumps touch come first, in their own commits, and
pass on both the old and the new dependencies.
Before the bumps:
- The e2e sandbox image builds again: dl.min.io answers 410 for mc since
MinIO archived it, so mc now comes from its last GitHub release with a
pinned checksum.
- Renovate stops offering tags of the apimachinery fork. They mirror
upstream, so #2999 is a downgrade that drops our patch.
- `make unit-tests` now runs the tests of the nested Go modules
(kubevirt-csi-driver, token-proxy, kubeovn-webhook, api/apps/v1alpha1).
They never ran in CI before.
- New tests: JWT verification in the dashboard token proxy, the kube-ovn
plunger metrics, cert-manager object idempotency in the tenant gateway,
values-file layering in the operator, and the kubevirt CSI driver
wrapper.
- One real fix. The lineage webhook decoded admitted objects with
encoding/json, so an int64 above 2^53 came back rounded. jsonpatch v2.5
compares numbers exactly and would emit a replace that rewrites such a
field in the pod. The webhook now decodes with apimachinery's json.
Held back on purpose, with the reasons in the commit messages:
- Kubernetes libraries stay on v0.35. #3001 needs the apimachinery fork
rebased onto release-1.37 first.
- cel-go v0.29.2 and etcd v3.6.15, the lines k8s.io/apiserver v0.35 is
built and tested with.
- grpc v1.83.2. v1.84.0 is affected by GO-2026-6443 and no v1.84 patch
release carries the fix yet.
- kubevirt.io/api and the CDI API stay on what upstream csi-driver
builds against. From CDI v1.60 it no longer compiles.
- api/apps/v1alpha1 keeps go 1.26.4, so consumers of the published types
are not forced onto Go 1.27.
- go fix runs without embedlit (Go 1.27-only syntax, breaks backports to
release branches on Go 1.26) and omitzero (would make metadata, spec and
status required in the published OpenAPI schemas), and one errorsastype
rewrite that did not compile is left out.
govulncheck reports nothing reachable in any of the five modules,
standard library included.
Supersedes #4311, #4310, #4309, #3452, #3993, #3927, #3902, #3805,
#3451, #3415, #3414, #3413, #3302, #4050 and #3363. #3001 stays open.
Not checked locally: the images that build upstream projects (kube-ovn,
kamaji, cilium, linstor-csi and others) with the Go 1.27 builder. The
image build jobs of this PR cover them.
Closes#4487.
### Screenshots
No UI changes.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
chore(deps): move the toolchain and builder images to Go 1.27.1 and raise Go dependencies to their latest compatible releases, closing the published advisories; the lineage webhook no longer rewrites int64 fields above 2^53 in admitted objects.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a restore resource policy option for Velero strategies.
* Added a test target to run Go tests across tracked modules.
* **Bug Fixes**
* Updated the end-to-end test environment to download and verify the
MinIO client from its current release location.
* Added coverage for token validation, webhook patching, controller
behavior, and reconciliation.
* **Maintenance**
* Updated the Go toolchain, dependencies, and container image versions.
* Refreshed pinned workflow actions and base image digests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Run go fix over every first-party module: interface{} becomes any,
hand-written loops become slices, maps, min/max and range-over-int,
errors.As becomes errors.AsType, pointer helpers become new(expr) and
are inlined at their call sites, and so on. Every rewrite keeps
behaviour.
Three rewrites stay out. embedlit flattens embedded-field literals
such as ObjectMeta, syntax only Go 1.27 accepts, which would break
backports to release branches still on Go 1.26. omitzero drops
omitempty from struct fields, and openapi-gen reads omitempty as
optional, so metadata, spec and status would turn required in the
published schemas. And errorsastype rewrote an errors.As whose
target, apierrors.APIStatus, does not implement error, which
errors.AsType rejects at compile time.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
getWorkloadMetadata filled the workloads.cozystack.io/kubevirt-vmi-instance-profile
label from the annotation kubevirt.io/cluster-instanceprofile-name, a key KubeVirt
never stamps. KubeVirt records the cluster preference (which carries the guest OS
profile) under kubevirt.io/cluster-preference-name, so the profile label was never
emitted and a VM's instance profile never reached the billing metric. The sibling
instance-type label worked only because its key is the real one.
Read the correct annotation. The emitted label name is unchanged, so the downstream
metric pipeline is untouched.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
The controller discovers where to query SeaweedFS bucket size metrics
via the namespace.cozystack.io/monitoring label, which assumes the
monitoring stack that scrapes SeaweedFS runs in the same cluster. Some
deployments host SeaweedFS and its metrics in a separate cluster, so
add a --seaweedfs-metrics-endpoint flag to cozystack-controller (and a
matching cozystackController.seaweedfsMetricsEndpoint chart value): an
explicit base URL of a Prometheus-compatible query API, with scheme,
host, optional port and path prefix; /api/v1/query is appended. Basic
auth may be embedded as URL userinfo. The flag is validated and
normalized at startup, so a malformed URL fails the rollout rather
than the queries. Empty keeps label-based discovery unchanged.
Bucket sizes feed billing, so metrics failures no longer collapse to
"no size": previously any query failure yielded an empty result map
and the reconciler overwrote Workload resources with it, silently
wiping the recorded sizes. The reconciler now retains the last known
s3-storage-bytes and s3-physical-storage-bytes values, emits a
BucketMetricsUnavailable warning event on the WorkloadMonitor, and
returns the error from Reconcile so it lands in the reconcile error
metrics.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
Replace http.DefaultClient with a package-level *http.Client with an
explicit 10-second timeout. Avoids sharing the process-wide default
transport with other libraries.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Pass bucket names from BucketClaim.Status.BucketName into the PromQL
query as a bucket=~"name1|name2" filter. This prevents O(N²) load
where N WorkloadMonitors each fetch all buckets globally.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Wrap resolvePrometheusURL and queryAllBucketMetrics in a
len(bucketClaimList.Items) > 0 guard. Avoids unnecessary namespace
GET and HTTP request to Prometheus on every reconcile of non-bucket
WorkloadMonitors (postgres, redis, kubernetes, etc.).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Replace 2×N per-bucket HTTP requests with a single query that fetches
all SeaweedFS bucket size metrics at once:
{__name__=~"SeaweedFS_s3_bucket_(size|physical_size)_bytes"}
Results are keyed by bucket name in memory, then looked up per
BucketClaim. This reduces HTTP round-trips from 2N+1 to 2 (one
namespace lookup + one Prometheus query) regardless of bucket count.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Move resolvePrometheusURL call before the BucketClaim loop and pass
the URL as parameter. Avoids redundant namespace lookups and reduces
reconcile time from 2×N+1 to 2×N HTTP calls (where N = BucketClaims).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Change queryPrometheusMetric to return (int64, bool) so callers can
emit s3-storage-bytes=0 for empty buckets while omitting the field
entirely when monitoring is not configured.
Also:
- Add RBAC marker for core/namespaces GET (used by resolvePrometheusURL)
- Log namespace read errors instead of silently returning empty URL
- Add TestQueryPrometheusMetricZeroValue test
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Bucket existence is already tracked via LifetimeHours. The s3-buckets
count resource produced a meaningless "s3-buckets-Hours" fallback type
in billing. Only storage size metrics (s3-storage-bytes,
s3-physical-storage-bytes) are now set on bucket Workloads.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Initialize workload.Labels map inside CreateOrUpdate mutate function
to prevent nil map panic when existing Workload has no labels
- Use strconv.ParseFloat instead of resource.ParseQuantity for
Prometheus metric values, which may use scientific notation
(e.g. "1.048576e+06") that ParseQuantity does not support
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Replace static --prometheus-url flag with dynamic resolution from
namespace.cozystack.io/monitoring label. Each tenant namespace knows
which tenant hosts its monitoring stack, so the controller constructs
the vmselect URL automatically. This correctly handles multi-tenant
setups where different tenants may use different monitoring instances.
- Remove PrometheusURL field from WorkloadMonitorReconciler struct
- Remove --prometheus-url flag and prometheusUrl chart value
- Add resolvePrometheusURL() that reads namespace label
- queryPrometheusMetric() now accepts prometheusBaseURL as parameter
- Add tests for resolvePrometheusURL with and without label
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Set replicas/minReplicas to 0 in bucket WorkloadMonitor (buckets
have no pods, minReplicas=1 would mark monitor as not operational)
- Remove dead IsNotFound check on List (List returns empty, not 404)
- Trim trailing slash from PrometheusURL before path append
- Add strings import for TrimRight
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
- Check resp.StatusCode before parsing Prometheus response
- Limit response body read to 1 MB via io.LimitReader
- Use strings.HasPrefix in test instead of fragile slice indexing
- Add TestQueryPrometheusMetricServerError for 500 responses
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Query SeaweedFS_s3_bucket_physical_size_bytes alongside the logical
size metric. Physical size includes all replicas and reflects actual
disk usage, while logical size reflects what the user stored.
Refactor queryBucketSizeBytes into generic queryPrometheusMetric
to reuse for both metrics.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Query SeaweedFS_s3_bucket_size_bytes from a Prometheus-compatible API
to populate s3-storage-bytes resource on bucket Workloads. The
Prometheus URL is configurable via --prometheus-url flag. When set,
bucket WorkloadMonitors are requeued every 60s to keep sizes current.
When Prometheus is not configured, buckets still get tracked with
s3-buckets=1 for existence-based billing.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Add reconcileBucketClaimForMonitor() that watches COSI BucketClaim
objects and creates Workload CRDs with s3-buckets resource, following
the same pattern as PVC and Service reconcilers. This enables the
billing pipeline to discover and track S3 buckets per tenant.
Changes:
- Add COSI API types dependency (container-object-storage-interface-api)
- Register cosiv1alpha1 scheme in controller main
- Add BucketClaim watch in SetupWithManager
- Add BucketClaim list + reconcile in Reconcile loop
- Add RBAC annotation for objectstorage.k8s.io/bucketclaims
- Add unit tests for BucketClaim reconciliation
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Inside the RetryOnConflict block, derive the operational status from
fresh.Spec.MinReplicas instead of the stale monitor.Spec.MinReplicas
so that concurrent spec updates observed by the retry are respected.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add two features to improve application observability in the dashboard:
1. WorkloadsReady condition on Application status
- Query WorkloadMonitor resources to determine if all pods are running
- Expose WorkloadsReady as a separate condition alongside Ready
- Ready continues to reflect HelmRelease state only (no override) to
preserve backward compatibility with tooling and avoid false-negatives
during normal startup windows when pods are still coming up
- Handle three states: operational, not operational, unknown (pending)
- Fail-open on WorkloadMonitor query errors
- Integrate with Application Watch to emit MODIFIED events on
WorkloadMonitor changes (with initial-events-end safety)
- Register cozystack.io/v1alpha1 types in API server scheme
2. Events tab in dashboard
- Show Kubernetes Events scoped to application namespace
- Use status.namespace for Tenant applications
- Include both lastTimestamp and eventTime columns for K8s compat
3. Bug fix: WorkloadMonitor Operational status persistence
- Operational was written to stale 'monitor' instead of 'fresh'
inside RetryOnConflict, so it was never persisted to the cluster
Closes#2359Closes#2360
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Andrey Kolkov