mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 22:13:18 +09:00
fix(redis): run one Sentinel when Redis runs a single pod (#5151)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The chart pins `spec.sentinel.replicas: 3` whatever `replicas` says, so an instance created with `replicas: 1` runs one Redis pod and three Sentinel pods that have nothing to fail over to. This makes the Sentinel count follow the data pods: one Sentinel when `replicas` is 1, three otherwise. The operator cannot run zero Sentinels (freshworks `Validate()` resets `sentinel.replicas <= 0` to 3), and it needs at least one to locate the master. With one Sentinel its quorum is `replicas/2+1` = 1, so the single Sentinel works on its own. The operator also gives the Sentinel Deployment a PodDisruptionBudget with `minAvailable: 1` whenever Redis has two replicas or fewer (`ensurePodDisruptionBudget` in `operator/redisfailover/service/client.go`). On a single Sentinel that budget refuses every eviction and would block node drains, so the chart sets `sentinel.disablePodDisruptionBudget: true` in that case only. The Sentinel `WorkloadMonitor` now expects the rendered count instead of a fixed 3/2. The operator only stops ensuring that PDB once the flag is set; it never deletes the one it created while three Sentinels ran (`EnsureSentinelDeployment` in freshworks v3.3.5). An existing instance scaled to one replica would keep it, so `templates/hooks/sentinel-pdb.yaml` adds a post-upgrade hook that deletes `rfs-<release>`, with a Role scoped to that one name. It renders on upgrade only, since a fresh one-replica install never gets that PDB. From `replicas: 2` up nothing changes. Scaling an existing instance between 1 and 2 replicas scales the Sentinel Deployment between 1 and 3 accordingly. Not in scope: with `replicas: 1` the Redis data pod keeps its own operator PDB (`minAvailable: 1`), which already blocks drains of its node today; that is unchanged here. Tests: `tests/sentinel_replicas_test.yaml` covers 1, 2, 3 and the default, the PDB switch, and the `WorkloadMonitor`. Three of its cases fail on `main` without the template change. `tests/sentinel_pdb_hook_test.yaml` covers the hook. `helm unittest packages/apps/redis`: 79 passed. Open PR #4990 also edits `redisfailover.yaml` (Sentinel spreading); the two touch different keys. Backport: `release-1.7` ships the same freshworks operator and CRD, so `kind/backport` applies. `release-1.6` is not backportable as is: it still runs spotahome `v1.3.0-rc1`, whose vendored CRD (chart 3.3.0) does not declare `sentinel.disablePodDisruptionBudget`, and the operator's CRDs are installed with Flux's default `crds: Create` policy, so they are never updated on an existing cluster. The API server would prune the field and the single Sentinel would keep a `minAvailable: 1` PDB that blocks node drains. A 1.6 fix would need a CRD update path first. ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change (walked the map: the diff changes a template, a test, and the wording of two descriptions; no field, default, enum or name changes) - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(redis): an instance with `replicas: 1` now runs one Sentinel instead of three, without a Sentinel PodDisruptionBudget; on upgrade, the PDB the operator created earlier is deleted. Instances with two replicas or more are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Redis deployments configured with one replica now use one Sentinel; other replica settings continue to use three. * Workload monitoring now reflects the configured Sentinel count. When changing to one replica, upgrades remove the Sentinel PodDisruptionBudget. * **Documentation** * Updated Redis configuration guidance to explain how Sentinel count varies with the replica setting and clarify Sentinel resource defaults. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
10 files changed
+443
-19
No files matched your search
Generated
+2
-2
@@ -17,7 +17,7 @@ type Config struct {
|
||||
}
|
||||
|
||||
type ConfigSpec struct {
|
||||
// Number of Redis replicas.
|
||||
// Number of Redis replicas. Three Sentinels watch them, or a single one when `replicas` is 1, since a lone pod has no replica to fail over to.
|
||||
// +kubebuilder:default:=2
|
||||
Replicas int `json:"replicas"`
|
||||
// Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in `resourcesPreset` is applied.
|
||||
@@ -51,7 +51,7 @@ type ConfigSpec struct {
|
||||
// Redis `min-replicas-max-lag`, in seconds: a replica whose last acknowledgement is older than this does not count towards `minReplicasToWrite`. Unset by default, which leaves the Redis default (10). Has no effect unless `minReplicasToWrite` is set. Must be at least 1 when `minReplicasToWrite` is set: Redis treats 0 as disabling the check, so the chart refuses it.
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
MinReplicasMaxLag *int `json:"minReplicasMaxLag,omitempty"`
|
||||
// Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the three Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.
|
||||
// Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.
|
||||
// +kubebuilder:default:={}
|
||||
Sentinel Sentinel `json:"sentinel,omitempty"`
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@ Service utilizes the freshworks-oss Redis Operator (a maintained fork of the arc
|
||||
|
||||
| Name | Description | Type | Value |
|
||||
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- | --------- |
|
||||
| `replicas` | Number of Redis replicas. | `int` | `2` |
|
||||
| `replicas` | Number of Redis replicas. Three Sentinels watch them, or a single one when `replicas` is 1, since a lone pod has no replica to fail over to. | `int` | `2` |
|
||||
| `resources` | Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in `resourcesPreset` is applied. | `object` | `{}` |
|
||||
| `resources.cpu` | CPU available to each replica. | `quantity` | `""` |
|
||||
| `resources.memory` | Memory (RAM) available to each replica. | `quantity` | `""` |
|
||||
@@ -48,13 +48,13 @@ Service utilizes the freshworks-oss Redis Operator (a maintained fork of the arc
|
||||
|
||||
### Sentinel parameters
|
||||
|
||||
| Name | Description | Type | Value |
|
||||
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----- |
|
||||
| `sentinel` | Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the three Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does. | `object` | `{}` |
|
||||
| `sentinel.resources` | Explicit CPU and memory configuration for each Sentinel pod. Keys left out are filled from the preset in `sentinel.resourcesPreset`. | `object` | `{}` |
|
||||
| `sentinel.resources.cpu` | CPU available to each replica. | `quantity` | `""` |
|
||||
| `sentinel.resources.memory` | Memory (RAM) available to each replica. | `quantity` | `""` |
|
||||
| `sentinel.resourcesPreset` | Sizing preset for the Sentinel pods. Defaults to `resourcesPreset`. | `string` | `{}` |
|
||||
| Name | Description | Type | Value |
|
||||
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----- |
|
||||
| `sentinel` | Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does. | `object` | `{}` |
|
||||
| `sentinel.resources` | Explicit CPU and memory configuration for each Sentinel pod. Keys left out are filled from the preset in `sentinel.resourcesPreset`. | `object` | `{}` |
|
||||
| `sentinel.resources.cpu` | CPU available to each replica. | `quantity` | `""` |
|
||||
| `sentinel.resources.memory` | Memory (RAM) available to each replica. | `quantity` | `""` |
|
||||
| `sentinel.resourcesPreset` | Sizing preset for the Sentinel pods. Defaults to `resourcesPreset`. | `string` | `{}` |
|
||||
|
||||
|
||||
## Parameter examples and reference
|
||||
@@ -79,7 +79,7 @@ See [Resource management](https://cozystack.io/docs/v1/guides/resource-managemen
|
||||
|
||||
### sentinel
|
||||
|
||||
`sentinel.resources` and `sentinel.resourcesPreset` size the three Sentinel pods separately from the Redis pods. A Sentinel holds only its configuration and the failover state, so it needs a small fraction of what a Redis replica holding the dataset does. When the `sentinel` block sets neither, the Sentinel pods are sized like the Redis pods, which is how every instance was sized before these parameters existed.
|
||||
`sentinel.resources` and `sentinel.resourcesPreset` size the Sentinel pods separately from the Redis pods. A Sentinel holds only its configuration and the failover state, so it needs a small fraction of what a Redis replica holding the dataset does. When the `sentinel` block sets neither, the Sentinel pods are sized like the Redis pods, which is how every instance was sized before these parameters existed.
|
||||
|
||||
```yaml
|
||||
resourcesPreset: u1.small
|
||||
|
||||
@@ -47,3 +47,7 @@ These presets are for basic testing and not meant to be used in production
|
||||
{{- printf "ERROR: Preset key '%s' invalid. Allowed values are %s" .type (join "," (keys $presets)) | fail -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "redis.sentinelReplicas" -}}
|
||||
{{- ternary 1 3 (eq (int .Values.replicas) 1) -}}
|
||||
{{- end -}}
|
||||
@@ -0,0 +1,122 @@
|
||||
{{- /*
|
||||
An instance scaled down to one Redis pod keeps the Sentinel PDB the operator
|
||||
created while it ran three Sentinels: setting disablePodDisruptionBudget only
|
||||
stops the operator from ensuring it, never deletes it, and minAvailable 1 on
|
||||
a single Sentinel refuses every eviction. A fresh install with one replica
|
||||
never gets that PDB, so this runs on upgrade only.
|
||||
*/}}
|
||||
{{- if and .Release.IsUpgrade (eq (include "redis.sentinelReplicas" . | int) 1) }}
|
||||
{{- $name := printf "%s-sentinel-pdb" .Release.Name }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
annotations:
|
||||
helm.sh/hook: post-upgrade
|
||||
helm.sh/hook-weight: "0"
|
||||
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
annotations:
|
||||
helm.sh/hook: post-upgrade
|
||||
helm.sh/hook-weight: "0"
|
||||
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
|
||||
rules:
|
||||
- apiGroups: ["policy"]
|
||||
resources: ["poddisruptionbudgets"]
|
||||
resourceNames: ["rfs-{{ .Release.Name }}"]
|
||||
verbs: ["get", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: {{ $name }}
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
annotations:
|
||||
helm.sh/hook: post-upgrade
|
||||
helm.sh/hook-weight: "0"
|
||||
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: {{ $name }}
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: {{ $name }}
|
||||
namespace: {{ .Release.Namespace }}
|
||||
---
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
{{- /* The API server copies a Job's name into its pods' job-name label,
|
||||
which takes 63 characters at most, while a release name may take 53.
|
||||
The objects above have no such limit and keep the full name. */}}
|
||||
name: {{ $name | trunc 63 | trimSuffix "-" }}
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
annotations:
|
||||
helm.sh/hook: post-upgrade
|
||||
helm.sh/hook-weight: "5"
|
||||
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
|
||||
spec:
|
||||
ttlSecondsAfterFinished: 3600
|
||||
# Bounds the kubectl call too: a --request-timeout would make kubectl skip
|
||||
# its in-cluster credentials (hack/check-hook-request-timeout.bats).
|
||||
activeDeadlineSeconds: 120
|
||||
backoffLimit: 2
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
policy.cozystack.io/allow-to-apiserver: "true"
|
||||
spec:
|
||||
serviceAccountName: {{ $name }}
|
||||
restartPolicy: Never
|
||||
terminationGracePeriodSeconds: 10
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
runAsGroup: 65534
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
containers:
|
||||
- name: delete-pdb
|
||||
image: '{{ include "cozy-lib.image" (list "clastix/kubectl:v1.32@sha256:b9ef7d8dbe65bcc81a46c09b8dc7543103055021c4f43287bf59e92a8f4fe05c" $) }}'
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
resources:
|
||||
requests:
|
||||
cpu: 10m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
env:
|
||||
- name: HOME
|
||||
value: /tmp
|
||||
command:
|
||||
- kubectl
|
||||
- --namespace={{ .Release.Namespace }}
|
||||
- delete
|
||||
- poddisruptionbudget
|
||||
- rfs-{{ .Release.Name }}
|
||||
- --ignore-not-found=true
|
||||
volumeMounts:
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
volumes:
|
||||
- name: tmp
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
@@ -9,6 +9,16 @@
|
||||
top-level resources falls back to resourcesPreset.
|
||||
*/}}
|
||||
{{- $sentinel := .Values.sentinel | default dict -}}
|
||||
{{- /*
|
||||
A lone Redis pod has no replica to promote, so three Sentinels would only
|
||||
watch it. One Sentinel still lets the operator find the master (its quorum is
|
||||
replicas/2+1, so 1). The operator gives the Sentinel Deployment a PDB with
|
||||
minAvailable 1 whenever Redis has two replicas or fewer, which on a single
|
||||
Sentinel would block every node drain, so that PDB is dropped with it; the
|
||||
operator does not delete a PDB it stops managing, hence
|
||||
hooks/sentinel-pdb.yaml.
|
||||
*/}}
|
||||
{{- $sentinelReplicas := include "redis.sentinelReplicas" . | int -}}
|
||||
{{- $sentinelPreset := $sentinel.resourcesPreset | default .Values.resourcesPreset -}}
|
||||
{{- $sentinelResources := .Values.resources -}}
|
||||
{{- if or $sentinel.resources $sentinel.resourcesPreset -}}
|
||||
@@ -65,7 +75,10 @@ spec:
|
||||
# pods they keep no RDB, only a sentinel.conf the init container rewrites
|
||||
# on every start, so a chart rollback to an older engine is safe here.
|
||||
image: {{ $redisImage | quote }}
|
||||
replicas: 3
|
||||
replicas: {{ $sentinelReplicas }}
|
||||
{{- if eq $sentinelReplicas 1 }}
|
||||
disablePodDisruptionBudget: true
|
||||
{{- end }}
|
||||
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list $sentinelPreset $sentinelResources $) | nindent 6 }}
|
||||
redis:
|
||||
image: {{ $redisImage | quote }}
|
||||
@@ -152,8 +165,8 @@ metadata:
|
||||
labels:
|
||||
workloads.cozystack.io/resource-preset: {{ $sentinelPreset | quote }}
|
||||
spec:
|
||||
minReplicas: 2
|
||||
replicas: 3
|
||||
minReplicas: {{ ternary 1 2 (eq $sentinelReplicas 1) }}
|
||||
replicas: {{ $sentinelReplicas }}
|
||||
kind: redis
|
||||
type: sentinel
|
||||
selector:
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
suite: stale Sentinel PDB removed on scale-down to one Redis pod
|
||||
|
||||
# The operator only stops ensuring the Sentinel PDB once
|
||||
# disablePodDisruptionBudget is set; it never deletes the one it created while
|
||||
# three Sentinels ran. Left in place, its minAvailable 1 refuses every eviction
|
||||
# of the single Sentinel and blocks node drains.
|
||||
|
||||
templates:
|
||||
- templates/hooks/sentinel-pdb.yaml
|
||||
|
||||
release:
|
||||
name: my-redis
|
||||
namespace: tenant-test
|
||||
|
||||
set:
|
||||
_cluster:
|
||||
cluster-domain: cluster.local
|
||||
_namespace:
|
||||
host: example.com
|
||||
|
||||
tests:
|
||||
- it: deletes the operator's Sentinel PDB on an upgrade to one replica
|
||||
release:
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Job
|
||||
asserts:
|
||||
- equal:
|
||||
path: metadata.annotations["helm.sh/hook"]
|
||||
value: post-upgrade
|
||||
- equal:
|
||||
path: spec.template.spec.containers[0].command
|
||||
value:
|
||||
- kubectl
|
||||
- --namespace=tenant-test
|
||||
- delete
|
||||
- poddisruptionbudget
|
||||
- rfs-my-redis
|
||||
- --ignore-not-found=true
|
||||
- notMatchRegex:
|
||||
path: spec.template.spec.containers[0].command[4]
|
||||
pattern: request-timeout
|
||||
|
||||
- it: lets the hook delete only that PDB
|
||||
release:
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Role
|
||||
asserts:
|
||||
- equal:
|
||||
path: rules
|
||||
value:
|
||||
- apiGroups: ["policy"]
|
||||
resources: ["poddisruptionbudgets"]
|
||||
resourceNames: ["rfs-my-redis"]
|
||||
verbs: ["get", "delete"]
|
||||
|
||||
- it: renders nothing on install, where no such PDB exists
|
||||
set:
|
||||
replicas: 1
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
- it: renders nothing while three Sentinels run
|
||||
release:
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 2
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
# The operator runs three Redis pods for a zero or negative count, so their
|
||||
# Sentinel PDB stays.
|
||||
- it: renders nothing for a zero replica count
|
||||
release:
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 0
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
- it: renders nothing for a negative replica count
|
||||
release:
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: -1
|
||||
asserts:
|
||||
- hasDocuments:
|
||||
count: 0
|
||||
|
||||
# The pods' job-name label takes 63 characters at most; a 53-character
|
||||
# release name plus -sentinel-pdb would pass it and Kubernetes would refuse
|
||||
# the Job, leaving the stale PDB in place.
|
||||
- it: keeps the Job name within 63 characters for the longest release name
|
||||
release:
|
||||
name: redis-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Job
|
||||
asserts:
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: ^[a-z0-9-]{1,63}$
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: '[a-z0-9]$'
|
||||
# The full release name stays in the Job name, so the hooks of two
|
||||
# releases in one namespace never replace each other's Job.
|
||||
- matchRegex:
|
||||
path: metadata.name
|
||||
pattern: ^redis-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-
|
||||
- equal:
|
||||
path: spec.template.spec.containers[0].command[4]
|
||||
value: rfs-redis-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
|
||||
- it: lets the hook delete the Sentinel PDB of the longest release name
|
||||
release:
|
||||
name: redis-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Role
|
||||
asserts:
|
||||
- equal:
|
||||
path: rules[0].resourceNames
|
||||
value:
|
||||
- rfs-redis-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
|
||||
- it: keeps the full Job name for a short release name
|
||||
release:
|
||||
upgrade: true
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: Job
|
||||
asserts:
|
||||
- equal:
|
||||
path: metadata.name
|
||||
value: my-redis-sentinel-pdb
|
||||
@@ -0,0 +1,131 @@
|
||||
suite: Sentinel count follows the Redis replicas
|
||||
|
||||
release:
|
||||
name: my-redis
|
||||
namespace: tenant-test
|
||||
|
||||
set:
|
||||
_cluster:
|
||||
cluster-domain: cluster.local
|
||||
_namespace:
|
||||
host: example.com
|
||||
|
||||
templates:
|
||||
- templates/redisfailover.yaml
|
||||
|
||||
tests:
|
||||
- it: "runs a single Sentinel for a single Redis pod"
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.replicas
|
||||
value: 1
|
||||
- equal:
|
||||
path: spec.redis.replicas
|
||||
value: 1
|
||||
|
||||
# The operator's Sentinel PDB is minAvailable 1 at two Redis replicas or
|
||||
# fewer; on one Sentinel it would refuse every eviction and block node drains.
|
||||
- it: "drops the Sentinel PDB when only one Sentinel runs"
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.disablePodDisruptionBudget
|
||||
value: true
|
||||
|
||||
- it: "keeps three Sentinels and their PDB from two Redis pods up"
|
||||
set:
|
||||
replicas: 2
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.replicas
|
||||
value: 3
|
||||
- notExists:
|
||||
path: spec.sentinel.disablePodDisruptionBudget
|
||||
|
||||
# The operator turns a zero or negative count into three Redis pods, so these
|
||||
# keep three Sentinels and their PDB rather than the single-pod case.
|
||||
- it: "keeps three Sentinels and their PDB for a zero replica count"
|
||||
set:
|
||||
replicas: 0
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.replicas
|
||||
value: 3
|
||||
- notExists:
|
||||
path: spec.sentinel.disablePodDisruptionBudget
|
||||
|
||||
- it: "keeps three Sentinels and their PDB for a negative replica count"
|
||||
set:
|
||||
replicas: -1
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.replicas
|
||||
value: 3
|
||||
- notExists:
|
||||
path: spec.sentinel.disablePodDisruptionBudget
|
||||
|
||||
- it: "keeps three Sentinels for three Redis pods"
|
||||
set:
|
||||
replicas: 3
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.replicas
|
||||
value: 3
|
||||
|
||||
- it: "keeps three Sentinels at the chart default"
|
||||
documentSelector:
|
||||
path: kind
|
||||
value: RedisFailover
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.sentinel.replicas
|
||||
value: 3
|
||||
|
||||
- it: "reports one expected Sentinel pod to the workload monitor"
|
||||
set:
|
||||
replicas: 1
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: my-redis-sentinel
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.replicas
|
||||
value: 1
|
||||
- equal:
|
||||
path: spec.minReplicas
|
||||
value: 1
|
||||
|
||||
- it: "reports three expected Sentinel pods to the workload monitor"
|
||||
set:
|
||||
replicas: 2
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: my-redis-sentinel
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.replicas
|
||||
value: 3
|
||||
- equal:
|
||||
path: spec.minReplicas
|
||||
value: 2
|
||||
@@ -3,7 +3,7 @@
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"replicas": {
|
||||
"description": "Number of Redis replicas.",
|
||||
"description": "Number of Redis replicas. Three Sentinels watch them, or a single one when `replicas` is 1, since a lone pod has no replica to fail over to.",
|
||||
"type": "integer",
|
||||
"default": 2
|
||||
},
|
||||
@@ -172,7 +172,7 @@
|
||||
"minimum": 0
|
||||
},
|
||||
"sentinel": {
|
||||
"description": "Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the three Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.",
|
||||
"description": "Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.",
|
||||
"type": "object",
|
||||
"default": {},
|
||||
"properties": {
|
||||
|
||||
@@ -55,7 +55,7 @@
|
||||
## @value xlarge
|
||||
## @value 2xlarge
|
||||
|
||||
## @param {int} replicas - Number of Redis replicas.
|
||||
## @param {int} replicas - Number of Redis replicas. Three Sentinels watch them, or a single one when `replicas` is 1, since a lone pod has no replica to fail over to.
|
||||
replicas: 2
|
||||
|
||||
## @param {Resources} [resources] - Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in `resourcesPreset` is applied.
|
||||
@@ -127,5 +127,5 @@ authEnabled: true
|
||||
## @field {Resources} [resources] - Explicit CPU and memory configuration for each Sentinel pod. Keys left out are filled from the preset in `sentinel.resourcesPreset`.
|
||||
## @field {ResourcesPreset} [resourcesPreset] - Sizing preset for the Sentinel pods. Defaults to `resourcesPreset`.
|
||||
|
||||
## @param {Sentinel} [sentinel] - Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the three Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.
|
||||
## @param {Sentinel} [sentinel] - Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.
|
||||
sentinel: {}
|
||||
+1
-1
@@ -8,7 +8,7 @@ spec:
|
||||
plural: redises
|
||||
singular: redis
|
||||
openAPISchema: |-
|
||||
{"title":"Chart Values","type":"object","properties":{"replicas":{"description":"Number of Redis replicas.","type":"integer","default":2},"resources":{"description":"Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in `resourcesPreset` is applied.","type":"object","default":{},"properties":{"cpu":{"description":"CPU available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"memory":{"description":"Memory (RAM) available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true}}},"resourcesPreset":{"description":"Default sizing preset used when `resources` is omitted.","type":"string","default":"t1.nano","enum":["t1.nano","t1.micro","t1.small","t1.medium","t1.large","t1.xlarge","t1.2xlarge","t1.4xlarge","c1.nano","c1.micro","c1.small","c1.medium","c1.large","c1.xlarge","c1.2xlarge","c1.4xlarge","s1.nano","s1.micro","s1.small","s1.medium","s1.large","s1.xlarge","s1.2xlarge","s1.4xlarge","u1.nano","u1.micro","u1.small","u1.medium","u1.large","u1.xlarge","u1.2xlarge","u1.4xlarge","m1.nano","m1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m1.2xlarge","m1.4xlarge","nano","micro","small","medium","large","xlarge","2xlarge"]},"size":{"description":"Persistent Volume Claim size available for application data.","default":"1Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"storageClass":{"description":"StorageClass used to store the data.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}},"external":{"description":"Enable external access from outside the cluster. Can be enabled only when the instance is created, together with `tls.enabled: true`; turning it on for an existing instance, or changing `tls.enabled` while it is on, is refused at admission.","type":"boolean","default":false},"version":{"description":"Redis major version to deploy. The Redis and Sentinel pods run the same image.","type":"string","default":"v8","enum":["v8","v7"]},"tls":{"description":"TLS configuration. TLS is opt-in and is not inferred from `external`.","type":"object","default":{},"properties":{"authClients":{"description":"Maps to the Redis `tls-auth-clients` directive. Defaults to `no` — the server certificate is presented but client certificates are not validated. `yes` requires a client certificate signed by this release's CA, which the platform does not issue. Changing it restarts the Redis and Sentinel pods, because the directive is read from the config file once at startup.","type":"string","enum":["no","optional","yes"]},"enabled":{"description":"Enable TLS for Redis and Sentinel connections. Disabled unless set. Enabling it moves Redis and Sentinel to a TLS-only listener, so existing plaintext clients must be migrated at the same time. Encryption is provided by the redis-operator fork that mounts the certificate Secret into both Redis and Sentinel pods at `/tls`.","type":"boolean"}}},"authEnabled":{"description":"Enable password generation.","type":"boolean","default":true},"minReplicasToWrite":{"description":"Redis `min-replicas-to-write`: the master refuses writes unless at least this many replicas are connected and within `minReplicasMaxLag`. Unset by default, which leaves the Redis default (disabled). With `replicas: 2` and a value of 1 the master rejects writes while its only replica is down, trading availability for a bound on the writes lost if an isolated master is failed over. Replication stays asynchronous, so this bounds the loss, it does not remove it. Must be 0 or more and below `replicas` (a master with `replicas` pods has `replicas - 1` replicas), otherwise every write is refused; the chart refuses such a value.","type":"integer","minimum":0},"minReplicasMaxLag":{"description":"Redis `min-replicas-max-lag`, in seconds: a replica whose last acknowledgement is older than this does not count towards `minReplicasToWrite`. Unset by default, which leaves the Redis default (10). Has no effect unless `minReplicasToWrite` is set. Must be at least 1 when `minReplicasToWrite` is set: Redis treats 0 as disabling the check, so the chart refuses it.","type":"integer","minimum":0},"sentinel":{"description":"Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the three Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map couLine truncated
|
||||
{"title":"Chart Values","type":"object","properties":{"replicas":{"description":"Number of Redis replicas. Three Sentinels watch them, or a single one when `replicas` is 1, since a lone pod has no replica to fail over to.","type":"integer","default":2},"resources":{"description":"Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in `resourcesPreset` is applied.","type":"object","default":{},"properties":{"cpu":{"description":"CPU available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"memory":{"description":"Memory (RAM) available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true}}},"resourcesPreset":{"description":"Default sizing preset used when `resources` is omitted.","type":"string","default":"t1.nano","enum":["t1.nano","t1.micro","t1.small","t1.medium","t1.large","t1.xlarge","t1.2xlarge","t1.4xlarge","c1.nano","c1.micro","c1.small","c1.medium","c1.large","c1.xlarge","c1.2xlarge","c1.4xlarge","s1.nano","s1.micro","s1.small","s1.medium","s1.large","s1.xlarge","s1.2xlarge","s1.4xlarge","u1.nano","u1.micro","u1.small","u1.medium","u1.large","u1.xlarge","u1.2xlarge","u1.4xlarge","m1.nano","m1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m1.2xlarge","m1.4xlarge","nano","micro","small","medium","large","xlarge","2xlarge"]},"size":{"description":"Persistent Volume Claim size available for application data.","default":"1Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"storageClass":{"description":"StorageClass used to store the data.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}},"external":{"description":"Enable external access from outside the cluster. Can be enabled only when the instance is created, together with `tls.enabled: true`; turning it on for an existing instance, or changing `tls.enabled` while it is on, is refused at admission.","type":"boolean","default":false},"version":{"description":"Redis major version to deploy. The Redis and Sentinel pods run the same image.","type":"string","default":"v8","enum":["v8","v7"]},"tls":{"description":"TLS configuration. TLS is opt-in and is not inferred from `external`.","type":"object","default":{},"properties":{"authClients":{"description":"Maps to the Redis `tls-auth-clients` directive. Defaults to `no` — the server certificate is presented but client certificates are not validated. `yes` requires a client certificate signed by this release's CA, which the platform does not issue. Changing it restarts the Redis and Sentinel pods, because the directive is read from the config file once at startup.","type":"string","enum":["no","optional","yes"]},"enabled":{"description":"Enable TLS for Redis and Sentinel connections. Disabled unless set. Enabling it moves Redis and Sentinel to a TLS-only listener, so existing plaintext clients must be migrated at the same time. Encryption is provided by the redis-operator fork that mounts the certificate Secret into both Redis and Sentinel pods at `/tls`.","type":"boolean"}}},"authEnabled":{"description":"Enable password generation.","type":"boolean","default":true},"minReplicasToWrite":{"description":"Redis `min-replicas-to-write`: the master refuses writes unless at least this many replicas are connected and within `minReplicasMaxLag`. Unset by default, which leaves the Redis default (disabled). With `replicas: 2` and a value of 1 the master rejects writes while its only replica is down, trading availability for a bound on the writes lost if an isolated master is failed over. Replication stays asynchronous, so this bounds the loss, it does not remove it. Must be 0 or more and below `replicas` (a master with `replicas` pods has `replicas - 1` replicas), otherwise every write is refused; the chart refuses such a value.","type":"integer","minimum":0},"minReplicasMaxLag":{"description":"Redis `min-replicas-max-lag`, in seconds: a replica whose last acknowledgement is older than this does not count towards `minReplicasToWrite`. Unset by default, which leaves the Redis default (10). Has no effect unless `minReplicasToWrite` is set. Must be at least 1 when `minReplicasToWrite` is set: Redis treats 0 as disabling the check, so the chart refuses it.","type":"integer","minimum":0},"sentinel":{"description":"Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the Sentinel Line truncated
|
||||
release:
|
||||
prefix: redis-
|
||||
labels:
|
||||
|
||||
Reference in new issue
Block a user