fix(monitoring): select the vmalert Service the operator creates (#5094)

## What this PR does

`templates/vm/vmalert-scrape.yaml` selects `app.kubernetes.io/instance:
vmalert`. The chart names the VMAlert `vmalert-<first metricsStorages
name>` (`templates/vm/vmalert.yaml` renders only the first storage), and
the operator labels its Service with that name, so with the default
storages the Service `vmalert-vmalert-shortterm` carries
`app.kubernetes.io/instance=vmalert-shortterm`. The VMServiceScrape
matches no Service and the metrics of vmalert itself never reach the
storage.

Seen on three v1.6.5 clusters. The `vmalert` VMServiceScrape in
`tenant-root` selects no Service, `count({__name__=~"vmalert_.*"})` in
vmselect-shortterm returns nothing while
`count(up{job="vmalertmanager-alertmanager"} == 1)` returns 3, and the
`/metrics` endpoint of vmalert does serve those series. No other scrape
object covers it, and no rule under `packages/` uses `vmalert_*`
metrics, so rule evaluation and notifier errors of vmalert are not
visible anywhere in the storage.

The scrape dates from v0.0.1, when the VMAlert was called `vmalert`.
4471b4ba (#289) renamed it to `vmalert-<storage>` in August 2024 and the
selector stayed, so the metrics of vmalert have not been collected
since.

The selector now takes the instance from the first `metricsStorages`
entry, the one `vmalert.yaml` renders, and the scrape is not rendered
when there are no storages, like the VMAlert itself. Keeping `instance`
instead of dropping it keeps the selector pinned to the one VMAlert this
release owns.

A new suite `tests/vmalert_scrape_test.yaml` covers the selector with
two storages and the empty case. `helm unittest .` in
`packages/system/monitoring` passes 306 of 306 with helm-unittest 1.1.2
as CI pins it, and against the template on main both new cases fail. A
full `helm template` of the chart with the default storages renders the
scrape with `app.kubernetes.io/instance: vmalert-shortterm`. I have not
run the change on a live cluster.

The template is the same on release-1.6, and that backport is a separate
PR.

### Screenshots

Not a UI change.

### Downstream repositories

One chart template and a new unittest suite in
`packages/system/monitoring`. No values key, schema, default, README or
package change, so no trigger map entry applies.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(monitoring): vmalert metrics are collected again. The vmalert VMServiceScrape selected app.kubernetes.io/instance=vmalert while the Service is labelled vmalert-<first metrics storage>, so it matched nothing.
```
This commit is contained in:
Aleksei Sviridkin authored and GitHub committed 2026-11-13 15:16:23 +03:00
commit 7ec746a360
2 files changed
+41 -1

No files matched your search

@@ -1,3 +1,4 @@
{{- with first .Values.metricsStorages }}
---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMServiceScrape
@@ -13,5 +14,6 @@ spec:
selector:
matchLabels:
app.kubernetes.io/component: monitoring
app.kubernetes.io/instance: vmalert
app.kubernetes.io/instance: vmalert-{{ .name }}
app.kubernetes.io/name: vmalert
{{- end }}
@@ -0,0 +1,38 @@
suite: vmalert metrics are scraped
templates:
- templates/vm/vmalert-scrape.yaml
release:
name: monitoring
namespace: tenant-root
set:
_cluster:
root-host: example.org
_namespace:
host: tenant-root.example.org
ingress: tenant-root
metricsStorages:
- name: shortterm
retentionPeriod: "3d"
deduplicationInterval: "15s"
- name: longterm
retentionPeriod: "14d"
deduplicationInterval: "5m"
tests:
- it: selects the Service of the VMAlert the chart renders
asserts:
- equal:
path: spec.selector.matchLabels
value:
app.kubernetes.io/component: monitoring
app.kubernetes.io/instance: vmalert-shortterm
app.kubernetes.io/name: vmalert
- it: renders nothing without metrics storages, like the VMAlert itself
set:
metricsStorages: []
asserts:
- hasDocuments:
count: 0