Files
cozystack/packages/system/cozystack-basics/tests/monitoring-external-services_test.yaml
T
Alexey Artamonov f9afc73096 fix(monitoring): surface missing VictoriaLogs and gate dead-end vlinsert ExternalName (#3181)
Platform log ingestion could fail silently: fluent-bit shipped to
vlinsert-generic while the monitoring HelmRelease reported Ready, and a
vlinsert-generic ExternalName in cozy-monitoring dead-ended at a
tenant-root VictoriaLogs that did not exist when the root tenant had
monitoring disabled. Logs were dropped with no failing signal.

Three complementary changes:

- Readiness (do not stay silently Ready): the monitoring-system
  HelmRelease now uses a `poller` waitStrategy and a healthCheckExpr on
  its VLCluster (status.updateStatus), so a missing/unhealthy log store
  makes the release not-ready instead of flipping Ready as soon as the
  CR is applied. Mirrors the seaweedfs CNPG pattern.

- No empty-log-store hole: system/monitoring rejects an empty
  logsStorages at render time (helm fail). With no entry no VLCluster is
  created, the readiness gate has nothing to match, and fluent-bit would
  silently drop logs — so this is made a loud misconfiguration rather
  than a documented footgun.

- No silent fallback: the cozy-monitoring -> tenant-root
  vlinsert/vminsert ExternalName redirects in cozystack-basics are now
  created only when the root tenant hosts monitoring, gated on
  _cluster.monitoring-enabled — fed from a new platform value
  monitoring.rootEnabled (default true, preserving the historical
  always-create behaviour). monitoring.rootEnabled is the single
  render-time source of truth: changing it re-renders cozystack-basics
  (helm-controller watches the cozystack-values Secret it flows into). A
  live `lookup` of Tenant/root spec.monitoring is deliberately NOT used:
  cozystack-basics is an operator-generated HelmRelease with drift
  detection off, so it re-renders only on install/upgrade/values-change
  — never on a reconcile tick or when a lookup result changes — and a
  lookup would go stale in both directions. Operators must keep
  monitoring.rootEnabled in sync with Tenant/root spec.monitoring.

Covered by helm-unittest across all four charts: readiness gate fields,
empty-vs-populated logsStorages contract, ExternalName present/absent
and null-guard contract, and the platform wiring into the
cozystack-values Secret. The CEL health predicates are exercised live,
not in helm-unittest (which cannot evaluate CEL). Adds the missing
`test:` target to packages/extra/monitoring/Makefile so its suite runs
in CI.

Known limitation (follow-up): the legacy cozystack.monitoring
PackageSource that installs system/monitoring directly into
cozy-monitoring cannot carry the readiness gate, because the engine's
ComponentInstall/HelmRelease builder and the vendored helm-controller
API v1.4.3 lack waitStrategy/healthCheckExprs. That source is not
referenced by any active bundle; full coverage needs a helm-controller
bump plus engine support.

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-07-08 22:27:22 +03:00

82 lines
2.6 KiB
YAML

suite: monitoring ExternalName redirects in cozy-monitoring
templates:
- templates/monitoring-external-services.yaml
release:
name: cozystack-basics
namespace: cozy-system
# The vlinsert-generic / vminsert-shortterm / vminsert-longterm ExternalName
# Services redirect cozy-monitoring at the platform monitoring stack the root
# tenant hosts in tenant-root. They only resolve when the root tenant actually
# runs that stack (Tenant/root spec.monitoring=true). When it is off the
# redirects dead-end and fluent-bit silently drops logs (issue #3181), so they
# must NOT be rendered.
#
# The render is gated solely on _cluster.monitoring-enabled (fed from the
# platform monitoring.rootEnabled value; operators keep it in sync with
# Tenant/root spec.monitoring). The "true" default preserves the historical
# always-create behaviour for cozystack-values Secrets written before the key
# existed, and a null _cluster must not panic. The cases below pin the full
# contract: enabled, disabled, default, and the nil guard.
tests:
- it: renders all three ExternalName redirects when monitoring-enabled is "true"
set:
_cluster:
monitoring-enabled: "true"
asserts:
- hasDocuments:
count: 3
- equal:
path: metadata.name
value: vlinsert-generic
documentIndex: 0
- equal:
path: metadata.namespace
value: cozy-monitoring
documentIndex: 0
- equal:
path: metadata.name
value: vminsert-shortterm
documentIndex: 1
- equal:
path: metadata.name
value: vminsert-longterm
documentIndex: 2
- it: keeps the redirects (default) when monitoring-enabled is absent
asserts:
- hasDocuments:
count: 3
- it: omits all redirects when monitoring-enabled is "false"
set:
_cluster:
monitoring-enabled: "false"
asserts:
- hasDocuments:
count: 0
# Guard: a null _cluster (missing cozystack-values Secret / dry-run) must not
# panic the render; `index (.Values._cluster | default dict) ...` falls back
# to the "true" default and keeps the redirects.
- it: keeps the redirects (no panic) when _cluster is null
set:
_cluster: null
asserts:
- hasDocuments:
count: 3
- it: vlinsert-generic still points at tenant-root when rendered
set:
_cluster:
monitoring-enabled: "true"
documentIndex: 0
asserts:
- equal:
path: spec.type
value: ExternalName
- equal:
path: spec.externalName
value: vlinsert-generic.tenant-root.svc.cluster.local