Aleksei Sviridkin 185c5eca72 fix(monitoring): route InfoInhibitor to the blackhole receiver (#4948)
## What this PR does

`InfoInhibitor` in
`packages/system/monitoring-agents/alerts/general.rules.yaml` is the
kube-prometheus rule that exists only to feed an Alertmanager inhibit
rule, and its own description says it should go to a null receiver. The
`update` target that first imported the rules rewrote its `severity:
none` to `major`, and the platform Alertmanager config has no inhibit
rule, so the alert goes to Alerta one level below critical. Alerta's
Telegram and Slack plugins forward it with the default
`disabledSeverity: []`, and once email delivery is configured, a
`notify_email="true"` label inherited from the info alert gets it
emailed as well.

It fires whenever an alert with `severity="info"` is active and no
warning or critical alert fires in the same namespace and cluster. #3799
moved the last first-party rule off `info`, but rules the platform does
not write still use it. CDI v1.66.1 generates
`CDIStorageProfilesIncomplete`, KubeVirt v1.9.0 generates
`KubeVirtVMGuestMemoryAvailableLow` and
`KubeVirtDeprecatedAPIRequested`, all at `info`, and operators add their
own. We hit it on a v1.6.4 cluster with one custom `severity: info` rule
that is always firing, and Alerta showed a permanent MAJOR
`InfoInhibitor`. The info alert itself cannot show up there, because
Alerta's default alarm model has no `info`.

This puts the label back to `none`, the value kube-prometheus ships. The
platform Alertmanager config already routes `severity="none"` to the
`blackhole` receiver ahead of every route that delivers, and the
Monitoring parameters already document that such alerts never reach
email. `hack/alert-severity-contract.bats` refuses `none` on purpose,
because an alert carrying it is never delivered, so InfoInhibitor is
listed as the one alert for which that is the point. The exemption
covers `none` and that name only.

The contract now also checks that InfoInhibitor carries `none`, that
`none` still fails on any other alert, and that the exemption does not
let InfoInhibitor through with a severity Alerta refuses. A new
helm-unittest suite in `packages/system/monitoring` pins the route
order, so that only the heartbeat route can come ahead of the blackhole,
with and without the email routes.

There is no inhibit rule here, and the expression still matches `info`
only. Wiring the inhibition up would hide every informational alert from
Alerta unless a warning fires in the same namespace, which changes
behaviour on every cluster and is a separate decision. Dropping
InfoInhibitor altogether would be equivalent for delivery, and I can
rework the PR that way if you prefer it.

v1.6.5 ships the same rule and the same routes, so a backport to
`release-1.6` would be appreciated. The commit cherry-picks onto it
without conflicts.

### Validation

- `bats hack/alert-severity-contract.bats` passes 12 of 12, and so does
the same file under `hack/cozytest.sh`. Without the label change the new
InfoInhibitor test fails on `[ "$delivered" -eq 0 ]`. With the exemption
removed the tree-wide contract fails on
`packages/system/monitoring-agents/alerts/general.rules.yaml:
InfoInhibitor -> 'none'`. With the exemption widened to any alert, or to
any severity, the matching fixture test fails.
- `helm unittest` in `packages/system/monitoring` passes 25 suites and
306 tests. Deleting the `severity="none"` route fails both new cases,
and an email route placed ahead of it fails the email case.
- `helm unittest` in `packages/system/monitoring-agents` passes 6 suites
and 19 tests, and the rendered `alerts-general.rules` VMRule carries
`severity: none` for InfoInhibitor.
- Rendered the Alertmanager config and ran `amtool config routes test`
from Alertmanager v0.31.0, the default of VM operator v0.68.4, with
InfoInhibitor's labels. `severity=major` routes to `alerta`, and to
`alerta,email` once email is configured and `notify_email=true` is
inherited. `severity=none` routes to `blackhole` in every case. `amtool
check-config` reports 0 inhibit rules.
- On `release-1.6` with this commit cherry-picked, the contract and both
helm-unittest suites pass.
- Not exercised on a live cluster.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

Walked the trigger map against the diff. It changes one alert label
under `packages/system/monitoring-agents/alerts/`, extends the test in
`hack/alert-severity-contract.bats` and adds a test suite under
`packages/system/monitoring/tests/`. No package is added or renamed, and
no values, schema, CRD, platform values, variant, make target, tool the
satellites call, node prerequisite or metric changes. The website
already says that `severity="none"` alerts are blackholed and never
reach email, and no page mentions InfoInhibitor, so nothing there goes
stale.

### Release note

```release-note
fix(monitoring): InfoInhibitor no longer reaches Alerta as a major alert. It carries severity none again, which the platform Alertmanager routes to the blackhole, as upstream kube-prometheus intends. Before, any active alert with severity info, from a custom rule or from one CDI or KubeVirt generates, raised a major InfoInhibitor in Alerta and in its Telegram and Slack notifications.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* `InfoInhibitor` alerts are now treated as non-notifying, preventing
them from generating alert notifications. Other alerts retain their
existing severity handling, and heartbeat alerts continue to follow
their configured route.
* The non-notifying route takes precedence over email delivery rules
that include `InfoInhibitor`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-13 15:56:58 +03:00
2025-10-08 09:43:34 +05:00
2024-02-08 12:04:32 +01:00

Cozystack Cozystack

GitHub Release Apache-2.0 License Commit activity OpenSSF Best Practices OpenSSF Scorecard CLOMonitor Support

Cozystack

Cozystack is a free and open-source platform and framework for building clouds.

Cozystack is a CNCF Sandbox project that was originally built and sponsored by Ænix.

With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.

Use Cozystack to build your own cloud or provide a cost-effective development environment.

Cozystack user interface

Use-Cases

Documentation

The documentation is located on the cozystack.io website.

Read the Getting Started section for a quick start.

If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.

Versioning

Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.

Contributions

Contributions are highly appreciated and very welcomed!

In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.

You are welcome to take on the fix yourself. Commits are used to generate the changelog, and their author will be referenced in it.

If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.

Community

You are welcome to join us in #cozystack, on either CNCF Slack (get an invite) or Kubernetes Slack (get an invite), in our Telegram group, and at our community meetings. Add them to your Google Calendar or iCal for convenience.

Project documents

Document What it covers
Governance How decisions are made and who makes them
Contributor Ladder How to grow from contributor to maintainer
Maintainers Who maintains the project
Code of Conduct What we expect of each other
Security Policy Reporting a vulnerability, and which releases get fixes
Adopters Who runs Cozystack in production
Roadmap Where the project is going

License

Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.

Commercial Support

A list of companies providing commercial support for this project can be found on official site.

Languages
Go 52%
Shell 34%
TypeScript 6.8%
Go Template 4.7%
Makefile 1.3%
Other 1.1%