Files
cozystack/packages/apps/redis
Aleksei Sviridkin 23461e065d fix(redis): run one Sentinel when Redis runs a single pod (#5151)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->

## What this PR does

The chart pins `spec.sentinel.replicas: 3` whatever `replicas` says, so
an instance created with `replicas: 1` runs one Redis pod and three
Sentinel pods that have nothing to fail over to. This makes the Sentinel
count follow the data pods: one Sentinel when `replicas` is 1, three
otherwise.

The operator cannot run zero Sentinels (freshworks `Validate()` resets
`sentinel.replicas <= 0` to 3), and it needs at least one to locate the
master. With one Sentinel its quorum is `replicas/2+1` = 1, so the
single Sentinel works on its own.

The operator also gives the Sentinel Deployment a PodDisruptionBudget
with `minAvailable: 1` whenever Redis has two replicas or fewer
(`ensurePodDisruptionBudget` in
`operator/redisfailover/service/client.go`). On a single Sentinel that
budget refuses every eviction and would block node drains, so the chart
sets `sentinel.disablePodDisruptionBudget: true` in that case only. The
Sentinel `WorkloadMonitor` now expects the rendered count instead of a
fixed 3/2.

The operator only stops ensuring that PDB once the flag is set; it never
deletes the one it created while three Sentinels ran
(`EnsureSentinelDeployment` in freshworks v3.3.5). An existing instance
scaled to one replica would keep it, so
`templates/hooks/sentinel-pdb.yaml` adds a post-upgrade hook that
deletes `rfs-<release>`, with a Role scoped to that one name. It renders
on upgrade only, since a fresh one-replica install never gets that PDB.

From `replicas: 2` up nothing changes. Scaling an existing instance
between 1 and 2 replicas scales the Sentinel Deployment between 1 and 3
accordingly.

Not in scope: with `replicas: 1` the Redis data pod keeps its own
operator PDB (`minAvailable: 1`), which already blocks drains of its
node today; that is unchanged here.

Tests: `tests/sentinel_replicas_test.yaml` covers 1, 2, 3 and the
default, the PDB switch, and the `WorkloadMonitor`. Three of its cases
fail on `main` without the template change.
`tests/sentinel_pdb_hook_test.yaml` covers the hook. `helm unittest
packages/apps/redis`: 79 passed. Open PR #4990 also edits
`redisfailover.yaml` (Sentinel spreading); the two touch different keys.

Backport: `release-1.7` ships the same freshworks operator and CRD, so
`kind/backport` applies. `release-1.6` is not backportable as is: it
still runs spotahome `v1.3.0-rc1`, whose vendored CRD (chart 3.3.0) does
not declare `sentinel.disablePodDisruptionBudget`, and the operator's
CRDs are installed with Flux's default `crds: Create` policy, so they
are never updated on an existing cluster. The API server would prune the
field and the single Sentinel would keep a `minAvailable: 1` PDB that
blocks node drains. A 1.6 fix would need a CRD update path first.


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change (walked the
map: the diff changes a template, a test, and the wording of two
descriptions; no field, default, enum or name changes)
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(redis): an instance with `replicas: 1` now runs one Sentinel instead of three, without a Sentinel PodDisruptionBudget; on upgrade, the PDB the operator created earlier is deleted. Instances with two replicas or more are unchanged.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Redis deployments configured with one replica now use one Sentinel;
other replica settings continue to use three.
* Workload monitoring now reflects the configured Sentinel count. When
changing to one replica, upgrades remove the Sentinel
PodDisruptionBudget.
* **Documentation**
* Updated Redis configuration guidance to explain how Sentinel count
varies with the replica setting and clarify Sentinel resource defaults.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-09 12:17:32 +03:00
..
2024-08-12 14:47:11 +02:00
2025-09-24 12:32:37 +02:00

Managed Redis Service

Redis is a highly versatile and blazing-fast in-memory data store and cache that can significantly boost the performance of your applications. Managed Redis Service offers a hassle-free solution for deploying and managing Redis clusters, ensuring that your data is always available and responsive.

Deployment Details

Service utilizes the freshworks-oss Redis Operator (a maintained fork of the archived Spotahome operator) for efficient management and orchestration of Redis clusters.

storageClass is annotated as immutable in the chart schema — see storageClass immutability for the contract and which consumers enforce it.

Parameters

Common parameters

Name Description Type Value
replicas Number of Redis replicas. Three Sentinels watch them, or a single one when replicas is 1, since a lone pod has no replica to fail over to. int 2
resources Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in resourcesPreset is applied. object {}
resources.cpu CPU available to each replica. quantity ""
resources.memory Memory (RAM) available to each replica. quantity ""
resourcesPreset Default sizing preset used when resources is omitted. string t1.nano
size Persistent Volume Claim size available for application data. quantity 1Gi
storageClass StorageClass used to store the data. string ""
external Enable external access from outside the cluster. Can be enabled only when the instance is created, together with tls.enabled: true; turning it on for an existing instance, or changing tls.enabled while it is on, is refused at admission. bool false
version Redis major version to deploy. The Redis and Sentinel pods run the same image. string v8

TLS parameters

Name Description Type Value
tls TLS configuration. TLS is opt-in and is not inferred from external. object {}
tls.enabled Enable TLS for Redis and Sentinel connections. Disabled unless set. Enabling it moves Redis and Sentinel to a TLS-only listener, so existing plaintext clients must be migrated at the same time. Encryption is provided by the redis-operator fork that mounts the certificate Secret into both Redis and Sentinel pods at /tls. *bool null
tls.authClients Maps to the Redis tls-auth-clients directive. Defaults to no — the server certificate is presented but client certificates are not validated. yes requires a client certificate signed by this release's CA, which the platform does not issue. Changing it restarts the Redis and Sentinel pods, because the directive is read from the config file once at startup. string {}

Application-specific parameters

Name Description Type Value
authEnabled Enable password generation. bool true
minReplicasToWrite Redis min-replicas-to-write: the master refuses writes unless at least this many replicas are connected and within minReplicasMaxLag. Unset by default, which leaves the Redis default (disabled). With replicas: 2 and a value of 1 the master rejects writes while its only replica is down, trading availability for a bound on the writes lost if an isolated master is failed over. Replication stays asynchronous, so this bounds the loss, it does not remove it. Must be 0 or more and below replicas (a master with replicas pods has replicas - 1 replicas), otherwise every write is refused; the chart refuses such a value. *int null
minReplicasMaxLag Redis min-replicas-max-lag, in seconds: a replica whose last acknowledgement is older than this does not count towards minReplicasToWrite. Unset by default, which leaves the Redis default (10). Has no effect unless minReplicasToWrite is set. Must be at least 1 when minReplicasToWrite is set: Redis treats 0 as disabling the check, so the chart refuses it. *int null

Sentinel parameters

Name Description Type Value
sentinel Sentinel configuration. When sentinel.resources is empty and sentinel.resourcesPreset is unset, the Sentinel pods are sized like the Redis pods (resources and resourcesPreset). An empty sentinel.resources map counts as unset, as an empty top-level resources does. object {}
sentinel.resources Explicit CPU and memory configuration for each Sentinel pod. Keys left out are filled from the preset in sentinel.resourcesPreset. object {}
sentinel.resources.cpu CPU available to each replica. quantity ""
sentinel.resources.memory Memory (RAM) available to each replica. quantity ""
sentinel.resourcesPreset Sizing preset for the Sentinel pods. Defaults to resourcesPreset. string {}

Parameter examples and reference

resources and resourcesPreset

resources sets explicit CPU and memory configurations for each replica. Every resource it leaves unset is taken from the preset defined in resourcesPreset.

resources:
  cpu: 4000m
  memory: 4Gi

resourcesPreset sets named CPU and memory configurations for each replica. This setting is ignored if the corresponding resources value is set.

Presets follow a cloud-style <series>.<size> naming convention. Five series cover the full CPU-to-memory ratio range (t1 1:0.5, c1 1:1, s1 1:2, u1 1:4, m1 1:8) and each series ships eight sizes (nano through 4xlarge). The legacy flat names (nano, micro, small, medium, large, xlarge, 2xlarge) remain accepted as deprecated aliases and keep their original sizes, which do not follow one series: nano through small equal the t1 sizes of the same name, while medium equals c1.small rather than c1.medium.

See Resource management for the full size matrix and the legacy-to-instance-type mapping.

sentinel

sentinel.resources and sentinel.resourcesPreset size the Sentinel pods separately from the Redis pods. A Sentinel holds only its configuration and the failover state, so it needs a small fraction of what a Redis replica holding the dataset does. When the sentinel block sets neither, the Sentinel pods are sized like the Redis pods, which is how every instance was sized before these parameters existed.

resourcesPreset: u1.small
sentinel:
  resources:
    cpu: 100m
    memory: 64Mi

Every key sentinel.resources leaves unset is taken from sentinel.resourcesPreset, which defaults to resourcesPreset.

tls

tls.enabled turns on TLS for Redis and Sentinel connections. It is off unless set and is never inferred from external.

External access needs TLS and is chosen at creation. A Redis created with external: true must also set tls.enabled: true, and turning external on for an existing instance is refused at admission. TLS cannot be added to a running instance (see below), and an instance's tls.enabled does not prove what it runs, since the API accepts a change the operator then refuses. To expose an instance, create a new one with both set. While external access is on, tls.enabled cannot be changed either, so TLS cannot be dropped before the instance first comes up. Withdrawing external access works and lifts that freeze in the same edit, but external access cannot be turned back on. An instance that was already external without TLS before this rule keeps working and stays editable.

TLS is chosen when the instance is created. TLS replaces plaintext rather than running beside it: with TLS on, Redis and Sentinel are configured with port 0 and serve only on the TLS port. It cannot be switched on or off for a running instance, because a replica restarted onto the TLS listener cannot replicate from a master still on plaintext, and Sentinels already on TLS would fail over to a stale replica. The operator's RedisFailover schema rejects the change; on an existing instance that surfaces as a failed HelmRelease upgrade while the instance keeps serving what it served before. To add TLS, create a new instance with tls.enabled and move the clients and data to it. A Redis that was already switched keeps its rejected value until it is put back: the API accepts the change, the HelmRelease then fails on it, and every later edit to that instance waits behind the failure, so remove tls.enabled to unblock the release; on an instance with external access that edit is refused unless it also sets external: false, or is made by a system:masters user or a platform service account. This is also why TLS is not inferred from external: an instance reachable from outside the cluster is the one most likely to have clients that an upgrade must not disconnect.

Enabling TLS makes the chart issue a per-release cert-manager chain: a self-signed bootstrap Issuer, a CA certificate, a CA Issuer, and the server leaf certificate the operator mounts into the Redis and Sentinel pods. The CA belongs to this release alone; it is not a cluster-wide trust root, and nothing outside the release trusts it.

This means TLS requires cert-manager, which the platform installs in the variant-independent part of the system bundle, so isp-hosted has it too. On a cluster where the cert-manager controller has been removed but its CRDs remain, tls.enabled renders cert-manager.io/v1 resources that nothing issues, and no certificate is ever produced; with the CRDs gone too, the release fails on an unknown kind instead.

To verify the server, a client needs that CA certificate. It is published as redis-<name>.tenant-ca: an object holding ca.crt and nothing else, delivered to tenants through the core.cozystack.io/tenantsecrets API the base tenant roles already grant. Every object named below follows the Helm release rather than the Redis resource, so a Redis named cache gives redis-cache.tenant-ca, redis-cache.ca-tls, and so on.

The chart declares where it is lifted from by rendering a TenantProjection naming <release>.ca-tls, the Secret cert-manager writes the release CA to. Only ca.crt is ever published, and the CA-extraction controller refuses to publish anything that parses as private key material, so the CA private key sitting beside it in that Secret is never projected.

kubectl get tenantsecret redis-<name>.tenant-ca -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
REDISCLI_AUTH=$(kubectl get tenantsecret redis-<name>-auth -o jsonpath='{.data.password}' | base64 -d) \
  redis-cli --tls --cacert ca.crt -h <host> -p 6379

It is reached through tenantsecrets rather than by reading a Secret directly, and that is deliberate: tenantsecrets surfaces only objects the platform has vouched for, whereas a direct grant on a name would convey whatever happens to occupy that name. The operator publishes a CA-only Secret of its own at <release>.ca-cert and it is key-free today, but the name is the operator's to fill and RBAC cannot filter by key, so it is not granted to the tenant.

<host> has to be a name the certificate covers. In-cluster that is any of the rfr-, rfrm-, rfrs- and rfs- service names, and <release>-external-lb when external is on; all of them resolve normally.

From outside the cluster the covered names are <release>.<tenant-host> and <release>.redis.<tenant-host>, the second being the per-engine name meant for SNI routing through a TLS-passthrough listener on the tenant Gateway, which redis-cli reaches only when given --sni, since it sends no SNI by default. The chart publishes DNS for neither: the external Service is a plain LoadBalancer with no external-dns annotation, so nothing points either name at the LoadBalancer address. Connecting to the LoadBalancer IP instead fails for any client that verifies the hostname, because the only IP addresses in the certificate are the loopback ones the in-pod probes and the metrics sidecar use; redis-cli is not such a client, it checks the chain and not the name, so it connects to the IP and hides the mismatch. Until a name is published, an external client has to be pointed at it manually — a DNS record or a hosts entry mapping either name to the LoadBalancer address.

Neither the CA and its private key (<release>.ca-tls) nor the server leaf and its private key (<release>.tls) is readable by the tenant. The first would allow minting certificates that any client trusting this release accepts; the second would allow impersonating this release's Redis endpoints. <release>.tenant-ca exists precisely so the certificate can be handed over without either key going with it.

Certificate renewal reaches the pods through the operator. Redis and Sentinel read tls-cert-file and tls-key-file once at startup and never re-read them, so a renewed Secret changes nothing until the pods restart. On every reconcile the operator reads the TLS Secret and stamps a hash of its tls.crt and ca.crt into the pod template of both the Redis StatefulSet and the Sentinel Deployment, as the annotation redis-failover.freshworks.com/tls-secret-hash. When cert-manager renews the leaf, 30 days before the end of its one-year validity, the hash changes: the operator's own roller replaces the Redis pods one at a time, replicas before the master, and the Deployment controller rolls Sentinel. Nothing needs to touch the release for the renewed certificate to be served.

If the roll cannot happen, because the operator is down or its reconcile of this RedisFailover keeps failing, the deadline is the old certificate's expiry: past it, clients fail to verify, and so do the operator's own TLS connections to Sentinel and Redis, so its checks and healing stop as well. Restart the Redis and Sentinel pods by hand before then.

The CA certificate has its own clock. cert-manager issues it for five years and reissues it on the same key before that ends, so a ca.crt copied out of <release>.tenant-ca once keeps verifying new leaves only until the original CA certificate expires. Re-read the projection on a schedule shorter than that, or mount it and let the kubelet refresh it, rather than baking the copy into an image.

tls.authClients maps to the Redis tls-auth-clients directive and defaults to no, meaning the server presents its certificate and does not ask connecting clients for one. Setting it to optional or yes makes Redis request, and for yes require, a client certificate signed by the same per-release CA. Quote the value in a manifest: an unquoted yes or no is read as a boolean and the string field rejects it. Changing it on a running release restarts the Redis and Sentinel pods: Redis reads the directive from its config file only at startup, so the operator folds the effective value into the certificate hash that drives the rollout, and the change lands when it is made rather than at the next unrelated restart.

Note that the platform does not currently mint a client certificate for the tenant, and the CA private key needed to sign one is deliberately out of reach. Redis and Sentinel pods and the metrics sidecar authenticate with the leaf certificate the operator mounts for them, so they are unaffected, but an external client has no supported way to obtain a certificate this CA will accept. Use authClients: yes only when the client certificate is supplied out of band by whoever also controls the CA.