Lets an operator bound the writes lost when an isolated Redis master is
failed over: the master refuses writes while fewer replicas than
configured are connected and recent. Both values are unset by default, so
the rendered Redis config is unchanged.
Redis treats 0 in either directive as disabling the check, so the
chart refuses minReplicasMaxLag 0 next to a replica requirement
instead of silently turning the protection off.
Negative values are refused by the schema (minimum 0), since Redis
rejects them at CONFIG SET and the operator would stop healing the
Sentinels of that instance. A requirement of replicas or more is
refused at render: a master with replicas pods has replicas-1
replicas, so it would refuse every write for good.
Assisted-by: LLM
Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com>
The Sentinels were left on redis:6.2.6-alpine when the operator moved to
the freshworks-oss fork, so that its new compiled-in default could not
move them on upgrade. 6.2 is end of life, and every instance now runs a
Sentinel one or two major versions behind the Redis it monitors.
The rollback risk behind that pin, a newer engine rewriting dump.rdb in
a format the older one cannot load, does not apply to Sentinel. It
stores no RDB or AOF, only sentinel.conf on an emptyDir that the
config-copy init container rewrites from the operator's ConfigMap on
every start, so moving back to an older image starts from a clean
config. Taking the image from the same version map as the data pods also
means one image per node instead of two.
The e2e comment that relied on busybox's timeout exit code in the
Sentinel container is updated, since that container now carries
coreutils.
Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com>
Assisted-by: LLM
Every Sentinel pod gets the Redis pods' resources, so the three
Sentinels of an instance reserve three times a Redis replica's memory
while holding only their configuration and the failover state. On a
t1.nano instance that is 3 x 128Mi while the VPA recommender's uncapped
target for a Sentinel container is 16Mi, and the waste grows with every
larger preset, because the dataset that drives the Redis sizing never
reaches a Sentinel.
sentinel.resources and sentinel.resourcesPreset follow the convention of
the top-level pair, and sentinel.resourcesPreset falls back to
resourcesPreset. Explicit Redis resources are not carried over once the
sentinel block sets either key, since they were sized for the dataset.
An empty sentinel.resources map counts as unset, as an empty top-level
resources does.
An empty sentinel block keeps today's sizing, so existing instances
render unchanged and an upgrade does not roll their Sentinels. The
Sentinel WorkloadMonitor reports the preset the Sentinels actually use.
Signed-off-by: Yasar Rahman <yasar.rehman@gmail.com>
Assisted-by: LLM
A Redis release with external: true and TLS off publishes a plaintext
endpoint outside the cluster, and nothing stopped a tenant from asking
for one. The chart never infers TLS from external, because enabling it
moves Redis to a TLS-only listener, so external: true without an
explicit tls.enabled: true is exactly that case. A
ValidatingAdmissionPolicy on the Redis kind now refuses it with a
message naming the release and a remedy that works.
TLS is fixed when a Redis instance is created: the operator's
RedisFailover schema refuses a change, but the Redis API accepts it.
A spec's tls.enabled is therefore a request, not the instance's state,
and cannot prove on UPDATE that an instance has TLS; trusting it would
let a tenant flip it, expose the instance, and remove it again. So
external access can be enabled only at creation, where tls.enabled
does decide the instance, and any UPDATE turning it on is refused.
The RedisFailover schema does not exist before the first reconcile, so
a release admitted as external with TLS could have TLS dropped before
the instance is created. tls.enabled is therefore frozen while
external access stays on. Withdrawing external access in the same edit
lifts the freeze, which leaves a tenant a way to unblock an instance
whose TLS was flipped earlier, and the callers trusted by the tenant
host policy are exempt so an administrator can repair one in place.
Other edits to an instance that is already external pass, so releases
exposed in plaintext before the policy keep working.
Postgres is left out on purpose. CNPG pins ssl=on, and its default
pg_hba accepts plaintext from any client that asks, whatever the
chart's tls.enabled says; that field only decides whether the external
hostname goes into the certificate. A policy keyed on it would close
nothing.
The decisions are tested by compiling the shipped template with the
apiserver's own CEL compiler, since a structural chart test cannot
show what the policy admits.
BREAKING CHANGE: creating a Redis with external: true and without
tls.enabled: true, which applied before, is now refused, including a
GitOps re-apply of such a manifest into a fresh cluster. Turning
external on for an existing Redis is refused as well, so exposing an
instance means creating a new one with TLS, and tls.enabled can no
longer be changed on an instance with external access.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
freshworks-oss/redis-operator v3.3.5 has no TLS support. The operator
work that packages/apps/redis renders against was ported upstream as
freshworks-oss/redis-operator#85 and is not in a release yet, so carry it
here over the v3.3.5 tarball until one lands.
Three pieces, and TLS is silently off unless all three are present:
- images/redis-operator/patches/tls.diff is the source half. The
Dockerfile already applies patches/*.diff, and the shell expands that
to labels.diff then tls.diff; they touch disjoint files, so the order
does not matter, but it is alphabetically stable either way.
- patches/crd-tls.diff teaches the vendored CRD about spec.tls. Without
it the apiserver prunes spec.tls from every RedisFailover the redis
chart renders and the patched binary waits for a field it never sees.
- patches/rbac-tls.diff grants Create, Get and Update on Secrets, so the
operator can publish the CA-only Secret named by
spec.tls.caCertSecretName. Those are the only Secret calls in the
patched source: upstream v3.3.5 only ever reads one, and the patch
adds Create and Update on a Secret it addresses by name, so nothing
lists, watches, patches or deletes.
tls.diff is cut from that PR's commit 2ad2ed8d1 rebased onto v3.3.5, and
its header pins that SHA so the next refresh can be diffed against it.
Beyond the TLS listener it carries the renewal roll: on every reconcile the operator stamps a hash of the TLS
Secret's tls.crt and ca.crt into the pod template of the Redis
StatefulSet and the Sentinel Deployment, so a renewed certificate rolls
Redis through the operator's own roller and Sentinel through its
Deployment. The effective spec.tls.authClients value is folded into
the same hash, so changing it rolls the pods the same way instead of
landing at the next unrelated restart, and the operator presents its
client certificate whenever the Secret carries one, independent of that
field, so neither direction of the change locks it out of the instance
it reconciles. It also mounts the TLS volume 0440 when the pod declares an
fsGroup, rejects a caCertSecretName that names the TLS Secret itself,
and waits for the TLS material before it writes the ConfigMaps and the
workloads, so a first TLS install starts its pods once.
tls.enabled is fixed at creation. Redis replication and Sentinel cannot
straddle plaintext and TLS: a replica restarted onto the other protocol
cannot sync from the master, and Sentinels restarted onto it fail over
to such a replica while the master still takes writes. The rule that
refuses the change is a CEL transition rule on the RedisFailover spec;
it lives in the CRD, so it ships here through patches/crd-tls.diff
together with the schema block it references. The cut carries source
only, and the Dockerfile strips the tarball's manifests/, charts/,
example/ and .github/ before applying the patches, so the image builds
from source alone and the two upstream tests that read the CRD YAML skip
instead of failing against v3.3.5's stale copy; the CRD half is
patches/crd-tls.diff.
cert-manager Certificates get no grant. The patched operator manages a
Certificate only under spec.tls.certManager, and packages/apps/redis
pins the other mode: it always supplies spec.tls.certificateSecret, so
that code path is unreachable as the platform ships it. A comment at
the rule site records the decision and names the verbs the rule would
need, so a re-vendor review reads the absence as deliberate.
The chart patches go through the mechanism the packages that customise a
vendored chart already use — reloader, grafana-operator, ingress-nginx,
seaweedfs and kamaji all re-apply a patches/*.diff at the end of update:
— because `rm -rf charts` discards a hand-edit and the next `make update`
would take TLS back out without failing.
tls.diff also carries one change that is not TLS and takes effect with
TLS off: MakeSlaveOfWithPort connects to the replica on its own port
rather than on the master's. It changes a dial, not a pod template, so
nothing rolls when this image ships; the probes dial localhost only
under TLS, and the plaintext pod templates stay byte-identical to
v3.3.5. The patch header names it, so a reader of patches/ does not
take the file for TLS-only.
The grant is narrower than the ported operator's own chart asks for:
that one also grants patch on Secrets, and create, delete, get, list,
patch, update and watch on Certificates. None of that is reachable from
the source as shipped here.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
TLS was enabled automatically whenever external was true. In this
operator TLS is not additive — the rendered config sets port 0 and moves
the listener to tls-port — so that inference turned every already-running
externally-reachable Redis into a TLS-only server on its first reconcile
after the upgrade and cut every plaintext client with it. A platform
upgrade must not sever working connections.
Resolve tls.enabled to false when unset, whatever external says. Enabling
TLS becomes an explicit act at creation time, and existing releases keep
the behaviour they have today.
Creation time, not any time: a running instance cannot change mode. A
replica restarted onto the TLS listener cannot replicate from a master
still on plaintext, and Sentinels already on TLS would fail over to a
stale replica, so the operator's RedisFailover schema rejects a change of
tls.enabled and the HelmRelease upgrade fails while the instance keeps
serving what it served before. The README says so and points at creating
a new instance and migrating.
Pin the decision directly: external: true on its own now has to leave the
cert-manager chain unrendered, the operator without spec.tls, and the
tenant without TLS secrets. Every case that previously reached TLS through
external says enabled: true instead, and the null and empty-map forms
resolve to off alongside the unset one.
With nothing inferred there is no upgrade-time behaviour change left, so
the breaking-change marker and footer are gone from this branch.
Say how to get out of it as well. The cozystack Redis API accepts the
change, so the rejection lands on the HelmRelease, and every later edit
to that instance queues behind the failure until tls.enabled goes back
to what it was.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
TLS replaces plaintext in this operator rather than running beside it —
with TLS on, Redis and Sentinel are configured with port 0 and serve only
on the TLS port. Since tls.enabled falls back to external, an already
deployed instance with external: true switches to TLS-only on its first
reconcile after the upgrade and every plaintext client breaks. That was
documented as a feature and nowhere as a change of behaviour for existing
instances. State it as a breaking change and name tls.enabled: false as
the opt-out.
Also spell out which hostname the certificate actually covers. The SAN
list carries the four service names and the external tenant hostname, and
its only IP entries are the loopback addresses used by the in-pod probes
and the metrics sidecar, so connecting to the external LoadBalancer by IP
fails for clients that verify the hostname.
The TLS section marker sat above the version parameter, which put version
in the TLS table in the generated README instead of Common parameters.
Move the marker below it. Regenerating also reorders version ahead of tls
in keysOrder, matching how postgres orders the same two fields.
Pin the PackageSource CRD replacement policy the way the mongodb-operator
one is pinned. The RedisFailover CRD gained spec.tls with the fork, and
Helm never upgrades CRDs shipped in crds/, so losing CreateReplace would
leave the live CRD behind and the apiserver would strip spec.tls from
every RedisFailover the chart renders — TLS silently off, no error.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Lift the documented `no` / `optional` / `yes` values for tls.authClients
from prose into an `## @enum {string} AuthClients` typedef in
packages/apps/redis/values.yaml. cozyvalues-gen then propagates the
constraint into:
- values.schema.json — adds `"enum": ["no","optional","yes"]` so
invalid values fail chart-value validation up front.
- api/apps/v1alpha1/redis/types.go — `AuthClients` becomes its own
Go type with a `+kubebuilder:validation:Enum=...` annotation,
aligning the cozystack Redis CR with the operator-side enum on
the RedisFailover CRD.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Run the root `make generate` (hack/update-codegen.sh) to refresh
api/apps/v1alpha1/redis/zz_generated.deepcopy.go with the new TLS
struct introduced for operator-managed TLS support. The previous
commit only ran the per-package `make generate`, which updates the
chart schema and CRD ApplicationDefinition but not the deepcopy
methods generated by deepcopy-gen.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Expose tls.enabled in the Redis chart values, mirroring the pattern
already in use by mariadb, nats, opensearch, qdrant and rabbitmq. The
flag is tri-state: when unset, TLS follows the value of `external`; an
explicit boolean overrides the default.
When TLS is enabled the chart renders the standard cozystack 4-resource
cert-manager chain in the release namespace:
- Issuer (self-signed bootstrap)
- Certificate (CA, ECDSA, isCA, rotationPolicy: Never, 5y)
- Issuer (CA-backed)
- Certificate (leaf, secret <release>-tls, ECDSA, 1y, rotationPolicy: Always)
The leaf certificate covers the SAN list the redis-operator computes for
its own Certificate (operator/redisfailover/service/generator.go:
redisCertificateDNSNames): short, namespace, .svc and .svc.<clusterDomain>
forms for rfr- / rfrm- / rfrs- / rfs- services plus a wildcard for
per-pod headless DNS names of the redis statefulset. Localhost is
included for in-pod readiness probes; when `external` is true the
tenant LB hostname is added too.
The chart hands the resulting Secret to the operator via
spec.tls.certificateSecret.secretName so the operator never creates a
duplicate Certificate; it just mounts the Secret into both Redis and
Sentinel pods and wires the redis tls-* directives. tls.authClients
maps to the Redis tls-auth-clients directive and defaults to "no".
The dashboard-resources Role is extended with the new TLS secret names
only when TLS is on, so the cozystack dashboard can read them to
display connection details.
Helm unit tests in tests/tls_test.yaml cover the tri-state resolution
(four enable/external combinations), SAN list coverage for all four
service names and the wildcard, custom cluster-domain handling, the
external SAN injection path, authClients propagation, and dashboard
RBAC expansion.
The comment above the CA usages named the wrong bits: cert-manager maps
signing to the digital signature bit, and crl sign and ocsp signing are
separate usages this chart does not request.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Annotate storageClass as immutable across every stateful app whose
chart exposes one — clickhouse, foundationdb, harbor, http-cache,
kafka (Kafka + ZooKeeper), kubernetes (top-level only), mariadb,
mongodb, nats, openbao, opensearch, postgres, qdrant, rabbitmq,
redis, vm-disk. Without coverage across the catalogue the UI would
render storageClass read-only on some apps and editable on others —
a worse mental model than uniform behaviour.
The annotation lands at the schema layer (CEL XValidation rule
'self == oldSelf' on values.schema.json / types.go / embedded
openAPISchema in cozyrds/<app>.yaml). The cozystack aggregated
apiserver does not yet evaluate CEL on application-backed resources;
that is tracked at cozystack/cozystack#2657. The UI
(cozystack/cozystack-ui#6) consumes the marker today.
The PVC-pinning argument is identical for every chart: Kubernetes
fixes a PVC's storageClassName at creation time, so editing this
field on an existing resource never migrated data. The schema
annotation makes that contract explicit.
kubernetes.nodeGroups[name].storageClass is intentionally NOT
annotated: the field is optional and undefaulted, so a strict
'self == oldSelf' rule would block any future attempt to set it on
an existing node group. A richer ratcheting form is needed there —
see docs/storage-immutability.md.
A refactor in pkg/registry/apps/application/rest.go extracts the
inline schema-build path into buildSpecSchema so it can be
unit-tested. Three regression tests in rest_spec_schema_test.go pin
the contract that x-kubernetes-validations survives the
v1->internal->structural conversion (loaded from the actual MariaDB
ApplicationDefinition fixture on disk, not a hand-edited mock).
Each per-app README gains a one-line blockquote pointing to a new
docs/storage-immutability.md that explains the contract and which
consumers enforce it today.
Requires cozystack/cozyvalues-gen with @immutable support — bump the
pre-commit pin in .github/workflows/pre-commit.yml once the release
exists.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:
nano -> t1.nano micro -> t1.micro
small -> t1.small medium -> c1.small
large -> c1.medium xlarge -> c1.large
2xlarge -> c1.xlarge
Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.
Three hardcoded legacy literals in chart templates migrated:
- packages/apps/harbor/templates/harbor.yaml:121 (portal)
- packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
- packages/system/keycloak/templates/sts.yaml:115 (heap fallback)
packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.
Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>