mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 12:13:55 +09:00
fix(monitoring): select the vmalert Service the operator creates (#5094)
## What this PR does
`templates/vm/vmalert-scrape.yaml` selects `app.kubernetes.io/instance:
vmalert`. The chart names the VMAlert `vmalert-<first metricsStorages
name>` (`templates/vm/vmalert.yaml` renders only the first storage), and
the operator labels its Service with that name, so with the default
storages the Service `vmalert-vmalert-shortterm` carries
`app.kubernetes.io/instance=vmalert-shortterm`. The VMServiceScrape
matches no Service and the metrics of vmalert itself never reach the
storage.
Seen on three v1.6.5 clusters. The `vmalert` VMServiceScrape in
`tenant-root` selects no Service, `count({__name__=~"vmalert_.*"})` in
vmselect-shortterm returns nothing while
`count(up{job="vmalertmanager-alertmanager"} == 1)` returns 3, and the
`/metrics` endpoint of vmalert does serve those series. No other scrape
object covers it, and no rule under `packages/` uses `vmalert_*`
metrics, so rule evaluation and notifier errors of vmalert are not
visible anywhere in the storage.
The scrape dates from v0.0.1, when the VMAlert was called `vmalert`.
4471b4ba (#289) renamed it to `vmalert-<storage>` in August 2024 and the
selector stayed, so the metrics of vmalert have not been collected
since.
The selector now takes the instance from the first `metricsStorages`
entry, the one `vmalert.yaml` renders, and the scrape is not rendered
when there are no storages, like the VMAlert itself. Keeping `instance`
instead of dropping it keeps the selector pinned to the one VMAlert this
release owns.
A new suite `tests/vmalert_scrape_test.yaml` covers the selector with
two storages and the empty case. `helm unittest .` in
`packages/system/monitoring` passes 306 of 306 with helm-unittest 1.1.2
as CI pins it, and against the template on main both new cases fail. A
full `helm template` of the chart with the default storages renders the
scrape with `app.kubernetes.io/instance: vmalert-shortterm`. I have not
run the change on a live cluster.
The template is the same on release-1.6, and that backport is a separate
PR.
### Screenshots
Not a UI change.
### Downstream repositories
One chart template and a new unittest suite in
`packages/system/monitoring`. No values key, schema, default, README or
package change, so no trigger map entry applies.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
fix(monitoring): vmalert metrics are collected again. The vmalert VMServiceScrape selected app.kubernetes.io/instance=vmalert while the Service is labelled vmalert-<first metrics storage>, so it matched nothing.
```
This commit is contained in:
2 files changed
+41
-1
No files matched your search
@@ -1,3 +1,4 @@
|
||||
{{- with first .Values.metricsStorages }}
|
||||
---
|
||||
apiVersion: operator.victoriametrics.com/v1beta1
|
||||
kind: VMServiceScrape
|
||||
@@ -13,5 +14,6 @@ spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/component: monitoring
|
||||
app.kubernetes.io/instance: vmalert
|
||||
app.kubernetes.io/instance: vmalert-{{ .name }}
|
||||
app.kubernetes.io/name: vmalert
|
||||
{{- end }}
|
||||
@@ -0,0 +1,38 @@
|
||||
suite: vmalert metrics are scraped
|
||||
templates:
|
||||
- templates/vm/vmalert-scrape.yaml
|
||||
|
||||
release:
|
||||
name: monitoring
|
||||
namespace: tenant-root
|
||||
|
||||
set:
|
||||
_cluster:
|
||||
root-host: example.org
|
||||
_namespace:
|
||||
host: tenant-root.example.org
|
||||
ingress: tenant-root
|
||||
metricsStorages:
|
||||
- name: shortterm
|
||||
retentionPeriod: "3d"
|
||||
deduplicationInterval: "15s"
|
||||
- name: longterm
|
||||
retentionPeriod: "14d"
|
||||
deduplicationInterval: "5m"
|
||||
|
||||
tests:
|
||||
- it: selects the Service of the VMAlert the chart renders
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.selector.matchLabels
|
||||
value:
|
||||
app.kubernetes.io/component: monitoring
|
||||
app.kubernetes.io/instance: vmalert-shortterm
|
||||
app.kubernetes.io/name: vmalert
|
||||
|
||||
- it: renders nothing without metrics storages, like the VMAlert itself
|
||||
set:
|
||||
metricsStorages: []
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
Reference in new issue
Block a user