mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
Alexey Artamonov
60b1b05434
Applications need one in-namespace OTLP endpoint that also enforces
sampling and redaction before spans reach the VictoriaTraces backend.
No OpenTelemetry operator ships with the platform, so the collector is
a plain Deployment, ConfigMap and Service ('otel-traces', gRPC 4317,
HTTP 4318) in the tenant namespace; app-to-collector traffic therefore
never leaves the tenant, and isolation comes from the existing tenant
network policies.
The collector renders only when it is enabled and a tracingStorages
backend is configured. Tracing is off by default, and an empty backend
list is a clean off-state rather than a render failure; an app exporter
that finds no Service simply retries. It writes to the first backend
only and rejects a list with several, instead of silently dropping
spans for the others.
redactAttributes is the PII and secret control point, so its keys are
rendered quoted and empty keys are dropped: an unquoted key with YAML
structural characters, or an empty one, makes the collector fail at
startup, and a key YAML re-types (numbers, booleans, null) silently
escapes redaction. replicas has a lower bound of 1 so a bad value fails
schema validation instead of the whole release at apply time; a zeroed
collector is expressed with enabled: false.
The collector gets a WorkloadMonitor and a dashboard-resources Role
entry under the same render gate, so it is part of tenant workload
accounting like every other workload in the chart.
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
274 lines
13 KiB
YAML
274 lines
13 KiB
YAML
## @section Common parameters
|
|
##
|
|
|
|
## @param {string} host - The hostname used to access Grafana externally (defaults to 'grafana' subdomain for the tenant host).
|
|
host: ""
|
|
|
|
##
|
|
## @section Metrics storage configuration
|
|
##
|
|
|
|
## @typedef {struct} Request - Minimum guaranteed resources.
|
|
## @field {quantity} [cpu] - CPU request.
|
|
## @field {quantity} [memory] - Memory request.
|
|
|
|
## @typedef {struct} Limit - Maximum allowed resources.
|
|
## @field {quantity} [cpu] - CPU limit.
|
|
## @field {quantity} [memory] - Memory limit.
|
|
|
|
## @typedef {struct} VMComponent - VictoriaMetrics component configuration.
|
|
## @field {Request} [minAllowed] - Minimum guaranteed resources.
|
|
## @field {Limit} [maxAllowed] - Maximum allowed resources.
|
|
|
|
## @typedef {struct} Resources - Combined resource configuration.
|
|
## @field {Request} [requests] - Resource requests.
|
|
## @field {Limit} [limits] - Resource limits.
|
|
|
|
## @typedef {struct} MetricsStorage - Configuration of metrics storage instance.
|
|
## @field {string} name - Name of the storage instance.
|
|
## @field {string} retentionPeriod - Retention period for metrics.
|
|
## @field {string} deduplicationInterval - Deduplication interval for metrics.
|
|
## @field {string} storage="10Gi" - Persistent volume size.
|
|
## @field {string} [storageClassName] - StorageClass used for the data.
|
|
## @field {VMComponent} [vminsert] - Configuration for vminsert.
|
|
## @field {VMComponent} [vmselect] - Configuration for vmselect.
|
|
## @field {VMComponent} [vmstorage] - Configuration for vmstorage.
|
|
|
|
## @param {[]MetricsStorage} metricsStorages - Configuration of metrics storage instances.
|
|
metricsStorages:
|
|
## Example:
|
|
## metricsStorages:
|
|
## - name: shortterm
|
|
## retentionPeriod: "3d"
|
|
## deduplicationInterval: "15s"
|
|
## storage: 10Gi
|
|
## storageClassName: ""
|
|
## vminsert:
|
|
## minAllowed:
|
|
## cpu: 200m
|
|
## memory: 512Mi
|
|
## maxAllowed:
|
|
## cpu: 1500m
|
|
## memory: 3Gi
|
|
## vmselect:
|
|
## minAllowed:
|
|
## cpu: 300m
|
|
## memory: 1Gi
|
|
## maxAllowed:
|
|
## cpu: 3500m
|
|
## memory: 6Gi
|
|
## vmstorage:
|
|
## minAllowed:
|
|
## cpu: 500m
|
|
## memory: 2Gi
|
|
## maxAllowed:
|
|
## cpu: 4000m
|
|
## memory: 8Gi
|
|
- name: shortterm
|
|
retentionPeriod: "3d"
|
|
deduplicationInterval: "15s"
|
|
storage: 10Gi
|
|
storageClassName: ""
|
|
- name: longterm
|
|
retentionPeriod: "14d"
|
|
deduplicationInterval: "5m"
|
|
storage: 10Gi
|
|
storageClassName: ""
|
|
|
|
##
|
|
## @section Logs storage configuration
|
|
##
|
|
|
|
## @typedef {struct} LogsStorage - Configuration of logs storage instance.
|
|
## @field {string} name - Name of the storage instance.
|
|
## @field {string} retentionPeriod="1" - Retention period for logs.
|
|
## @field {string} storage="10Gi" - Persistent volume size.
|
|
## @field {string} storageClassName="replicated" - StorageClass used to store the data.
|
|
|
|
## @param {[]LogsStorage} logsStorages - Configuration of logs storage instances.
|
|
logsStorages:
|
|
- name: generic
|
|
retentionPeriod: "1"
|
|
storage: 10Gi
|
|
storageClassName: replicated
|
|
|
|
##
|
|
## @section Traces storage configuration
|
|
##
|
|
|
|
## @enum {string} TracingMode - Deployment mode for a traces storage instance.
|
|
## @value cluster
|
|
## @value single
|
|
|
|
## @typedef {struct} TracingStorage - Configuration of traces storage instance.
|
|
## @field {string} name - Name of the storage instance: a lowercase RFC 1123 label of at most 42 characters. Treat as immutable once a backend exists: the VTCluster/VTSingle, its Services, PVCs and Grafana datasource are all named after it, so renaming an entry deploys a new, empty backend and removes the old one, the same way changing mode does (single mode loses the old PVC, cluster mode leaves the old vtstorage PVCs Bound until the whole Monitoring release is deleted).
|
|
## @field {TracingMode} mode="cluster" - Deployment mode: "cluster" renders a VTCluster; "single" renders a single-binary VTSingle for edge/dev/small clusters. Treat as immutable once a backend exists: mode selects which Kind is rendered under the same name, so changing it on a running backend re-identifies the object rather than reconfiguring it. Switching from "single" to "cluster" prunes the VTSingle and the operator garbage-collects its PVC (stored traces are lost); switching from "cluster" to "single" (or removing the entry) leaves the vtstorage PVCs Bound and billed until the whole Monitoring release is deleted. The API cannot guard this (transition rules are not available on an array-item field), so this note is the only guard: migrate deliberately, accepting the loss.
|
|
## @field {string} retentionPeriod="14d" - Retention period for traces.
|
|
## @field {quantity} storage="10Gi" - Persistent volume size.
|
|
## @field {string} storageClassName="replicated" - StorageClass used to store the data.
|
|
## @field {string} [retentionDiskUsageBytes] - Optional disk-usage cap as a VictoriaMetrics BytesString (e.g. "8GB" or "8GiB"), bounding disk use on top of the age-based retentionPeriod. Uses byte-suffix grammar (kb/mb/gb/tb, KiB/MiB/GiB/TiB), NOT the Kubernetes-quantity grammar of the sibling storage field, so "8Gi" is rejected by the operator. In cluster mode the cap is per vtstorage replica (replicaCount 2), so the effective total is twice the value.
|
|
|
|
## @param {[]TracingStorage} tracingStorages - Configuration of traces storage instances. Empty by default (tracing off): no backend and no trace collector are rendered. Adding an entry provisions a VictoriaTraces backend and, while tracingCollector.enabled is true, deploys the OTLP collector in front of it.
|
|
tracingStorages: []
|
|
|
|
##
|
|
## @section Traces collector configuration
|
|
##
|
|
|
|
## @typedef {struct} TracingCollector - Per-tenant OpenTelemetry Collector that fronts the traces backend. It is the app-facing OTLP ingest gateway and the point where sampling/redaction are enforced; tenant isolation itself comes from the tenant NetworkPolicies, not from this workload.
|
|
## @field {bool} enabled=true - Deploy the OTLP collector. When disabled, applications must send OTLP straight to the traces backend (no central sampling/redaction). The collector fronts a single backend: while it is enabled, tracingStorages must hold at most one entry, and adding a second one fails the whole Monitoring render until the extra entry is removed or the collector is disabled.
|
|
## @field {int} replicas=1 - Number of collector replicas. Head sampling is trace-id-deterministic, so replicas>1 are safe and remove the single-ingest-path SPOF.
|
|
## @minimum 1
|
|
## @field {int} samplingPercentage=10 - Head-sampling percentage applied at the collector before export.
|
|
## @minimum 0
|
|
## @maximum 100
|
|
## @field {[]string} [redactAttributes] - Span attribute keys to drop before export (the PII/secret control point, e.g. db.statement).
|
|
## @field {Resources} [resources] - Resource configuration for the collector.
|
|
|
|
## @param {TracingCollector} tracingCollector - OpenTelemetry Collector for trace ingestion.
|
|
tracingCollector:
|
|
enabled: true
|
|
replicas: 1
|
|
samplingPercentage: 10
|
|
redactAttributes: []
|
|
resources:
|
|
limits:
|
|
cpu: "1"
|
|
memory: 1Gi
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
|
|
##
|
|
## @section Alerta configuration
|
|
##
|
|
|
|
## @typedef {struct} TelegramAlerts - Telegram alert configuration.
|
|
## @field {string} token - Telegram bot token.
|
|
## @field {string} chatID - Telegram chat ID(s), separated by commas.
|
|
## @field {[]string} [disabledSeverity] - List of severities without alerts (e.g. ["informational","warning"]).
|
|
|
|
## @typedef {struct} SlackAlerts - Slack alert configuration.
|
|
## @field {string} url - Configuration uri for Slack alerts.
|
|
## @field {[]string} [disabledSeverity] - List of severities without alerts (e.g. ["informational","warning"]).
|
|
|
|
## @typedef {struct} Alerts - Alert routing configuration.
|
|
## @field {TelegramAlerts} [telegram] - Configuration for Telegram alerts.
|
|
## @field {SlackAlerts} [slack] - Configuration for Slack alerts.
|
|
|
|
## @typedef {struct} Alerta - Configuration for the Alerta service.
|
|
## @field {string} [storage] - Persistent volume size for the database.
|
|
## @field {string} [storageClassName] - StorageClass used for the database.
|
|
## @field {Resources} [resources] - Resource configuration.
|
|
## @field {Alerts} [alerts] - Alert routing configuration.
|
|
|
|
## @param {Alerta} alerta - Configuration for the Alerta service.
|
|
alerta:
|
|
storage: 10Gi
|
|
storageClassName: ""
|
|
resources:
|
|
limits:
|
|
cpu: "1"
|
|
memory: 1Gi
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
alerts:
|
|
telegram:
|
|
token: ""
|
|
chatID: ""
|
|
disabledSeverity: []
|
|
slack:
|
|
url: ""
|
|
disabledSeverity: []
|
|
##
|
|
## @section Grafana configuration
|
|
##
|
|
|
|
## @typedef {struct} GrafanaDB - Grafana database configuration.
|
|
## @field {string} [size] - Persistent volume size for the database.
|
|
|
|
## @typedef {struct} Grafana - Grafana service configuration.
|
|
## @field {GrafanaDB} [db] - Database configuration.
|
|
## @field {Resources} [resources] - Resource configuration.
|
|
|
|
## @param {Grafana} grafana - Configuration for Grafana.
|
|
grafana:
|
|
db:
|
|
size: 10Gi
|
|
resources:
|
|
limits:
|
|
cpu: "1"
|
|
memory: 1Gi
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
##
|
|
## @section Vmagent configuration
|
|
##
|
|
|
|
## @typedef {struct} VmagentRemoteWrite - Remote write configuration.
|
|
## @typedef {stringSlice} VmagentRemoteWriteURLs - List of remoteWrite endpoint URLs.
|
|
|
|
## @typedef {struct} VmagentExternalLabels - External labels for metrics.
|
|
## @field {string} [<key>] - Label value for the given key.
|
|
|
|
## @typedef {struct} Vmagent - VictoriaMetrics Agent configuration.
|
|
## @field {VmagentExternalLabels} [externalLabels] - External labels applied to all metrics.
|
|
## @field {VmagentRemoteWrite} [remoteWrite] - Remote write configuration.
|
|
|
|
## @param {Vmagent} vmagent - Configuration for VictoriaMetrics Agent.
|
|
vmagent:
|
|
externalLabels:
|
|
cluster: cozystack
|
|
remoteWrite:
|
|
urls:
|
|
- http://vminsert-shortterm:8480/insert/0/prometheus
|
|
- http://vminsert-longterm:8480/insert/0/prometheus
|
|
## inlineScrapeConfig: |
|
|
## - job_name: "custom"
|
|
## static_configs:
|
|
## - targets: ["my-service:9090"]
|
|
|
|
##
|
|
## @section OIDC authentication
|
|
##
|
|
|
|
## @enum {string} OIDCMode - Identity mode for the Grafana instance.
|
|
## @value None
|
|
## @value System
|
|
## @value CustomConfig
|
|
|
|
## @typedef {struct} OIDCSecretRef - Reference to a Secret carrying an `[auth.generic_oauth]` ini fragment.
|
|
## @field {string} [name] - Name of an existing Secret in the release namespace whose `auth.ini` key holds a ready-made `[auth.generic_oauth]` block. Mutually exclusive with `customConfig.config`.
|
|
|
|
## @typedef {struct} OIDCCustomConfigInline - Inline map of grafana.ini `[auth.generic_oauth]` keys.
|
|
## @field {string} [<key>] - Value for the generic_oauth key (e.g. `client_id`, `client_secret`, `auth_url`, `token_url`, `api_url`, `scopes`, `role_attribute_path`).
|
|
|
|
## @typedef {struct} OIDCCustomConfig - Tenant-supplied AuthN payload; consumed only when `mode: CustomConfig`. Exactly one of `config` or `secretRef.name` must be set.
|
|
## @field {OIDCCustomConfigInline} [config] - Inline map of `[auth.generic_oauth]` keys. Written verbatim into the Grafana CR `spec.config`. Mutually exclusive with `secretRef.name`.
|
|
## @field {OIDCSecretRef} [secretRef] - Reference to a Secret holding an `auth.ini` fragment; mounted under `/etc/grafana/oidc` and loaded via `GF_PATHS_CUSTOM_INI`.
|
|
|
|
## @enum {string} OIDCRole - Grafana org role bound to a listed user.
|
|
## @value Admin
|
|
## @value Editor
|
|
## @value Viewer
|
|
|
|
## @typedef {struct} OIDCUser - Grafana user reconciled into the release's org.
|
|
## @field {string} email - Email address matched against the `email` claim from the issuer. The chart pre-provisions a Grafana account with this email so the users-Job can set the org role before the operator's first login (`oauth_allow_insecure_email_lookup = true` glues the two together). Must match a conservative pattern: local part is `[A-Za-z0-9._%+-]+`, domain is `[A-Za-z0-9.-]+\.[A-Za-z]{2,}`. Whitespace, quoted literals, and unusual URL-reserved characters are rejected on schema level so the reconcile-Job's shell handling never has to defend against malformed input.
|
|
## @field {OIDCRole} role - Grafana org role to bind: `Admin`, `Editor`, or `Viewer`. Applied to the release's Grafana org (`Main Org.`); server-level `GrafanaAdmin` promotion is out of scope for Phase 1. Required — the users-Job POSTs `{loginOrEmail, role}` to `/api/orgs/1/users` and PATCHes `{role}` to converge across re-runs; omitting the field would render `role: "null"` into the body and fail with HTTP 400.
|
|
|
|
## @typedef {struct} OIDC - OIDC selector for the Grafana instance (Phase 1 — per-instance audience binding; no per-tenant Keycloak realm).
|
|
## @field {OIDCMode} mode - Identity mode. `None`: no OIDC, only the `grafana-admin-password` Secret works. `System`: trust the platform `cozy` realm via a per-instance confidential client and audience binding; zero-config default. `CustomConfig`: trust a tenant-supplied issuer directly (BYO); `cozy` is not in the path.
|
|
## @field {OIDCCustomConfig} [customConfig] - Tenant-supplied `[auth.generic_oauth]` payload; consumed only when `mode: CustomConfig`.
|
|
## @field {[]OIDCUser} [users] - Users granted access to this Grafana instance's org. Each entry is reconciled into Grafana org membership + role by a chart-owned post-install/post-upgrade Job. Works for both `System` and `CustomConfig` modes; ignored when `mode: None`. Users not listed here get nothing (no default org role from group mapping).
|
|
|
|
## @param {OIDC} oidc - OIDC authentication for the Grafana instance. See docs/oidc-grafana.md for the operator guide.
|
|
oidc:
|
|
mode: "None"
|
|
customConfig:
|
|
config: {}
|
|
secretRef:
|
|
name: ""
|
|
users: []
|