Seven guides in docs/ were written next to the code and left in the
source tree, where no operator looks. The website carries them now, so
a second copy here only lets the two drift again: the Backup Classes
page had already fallen behind its original.
docs/release.md, docs/deprecating-applications.md and the two SeaweedFS
runbooks in docs/operations stay. A migration, a bats suite, a workflow
or AGENTS.md reads each of them by path, which is the test for keeping
a document in this repository.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
With the static seal the key sits in a Secret in the same cluster as the
data it unwraps, so the barrier is only as strong as access to Secrets
in that namespace, to etcd and to backups. The README now says so before
the reader picks a seal, and it says what the key identifier is for,
since OpenBAO picks the key for stored data by it.
Tenant roles grant nothing on Secrets and no pods/log, so a cluster
administrator creates the key Secret and reads the log line that
confirms a re-wrap.
The StatefulSet uses the OnDelete strategy, so neither a seal migration
nor the apiserver egress label reaches a running pod until it is
replaced. The README describes the migration from Shamir step by step,
following the OpenBAO seal migration guide, and says that pods created
before the label have to be replaced once.
The migration starts with a backup. The standalone file backend is not
transactional, so its volume is copied with the server stopped or
snapshotted atomically. The upgrade that sets allowMigration is the
point of no return for the chart, and the README says how a cluster
administrator backs out of it before any pod is replaced.
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
The upstream chart passes the server config through tpl, and its start
script replaces upper case placeholders such as HOSTNAME anywhere in the
copied file. A key identifier with template delimiters would run as a
template inside helm-controller, which runs as cluster-admin, and an
upper case one could come out different on every pod. Both key
identifiers now take lowercase alphanumerics, hyphens, dots and
underscores.
The Secret names become pod volume names of the form userconfig-<name>,
which Kubernetes validates as DNS labels. Both name fields now take a
DNS label of at most 52 characters, and the volume entries are quoted.
The apps API only defaults against the generated schema, so Helm
enforces these patterns when it installs or upgrades the release.
A previous Secret equal to the current one mounts the same volume twice,
which the API server rejects for the StatefulSet, and OpenBAO refuses to
start when equal key identifiers name different key material. Both
cases now fail at render. The deepcopy functions for the Seal type are
regenerated as well.
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
The OpenBAO app has no auto-unseal (#2787). Every restart of a server,
after a node drain, an image bump or an OOM kill, leaves it sealed until
an operator enters the Shamir key shares into each replica. In HA mode
the pods also hang at start (#2793). Cozystack's allow-to-apiserver
policy only admits egress from pods labelled
policy.cozystack.io/allow-to-apiserver, the app never set that label,
and service_registration "kubernetes" cannot reach the API server
without it.
A new seal value selects shamir, which stays the default, or OpenBAO's
seal "static". With static the chart renders the static seal stanza into
the standalone and the Raft config and mounts the key Secret named in
seal.secretName through the upstream chart's server.extraVolumes. The
chart never generates, stores or rotates the key, because a missed
lookup would replace it and leave the data unreadable.
seal.previousSecretName and seal.previousKeyId carry the n-1 key that
the static seal accepts during a rotation. The server pods always carry
the apiserver egress label.
Changing the seal of an initialized instance is a seal migration that
OpenBAO performs under bao operator unseal -migrate. The chart reads the
seal in use from the config ConfigMap the previous release rendered and
refuses a different seal.type unless seal.allowMigration acknowledges
the migration. Moving away from static needs the old stanza kept with
disabled = "true", which the chart does not render, so that direction is
refused regardless. The README documents the key Secret, the rotation
with the pod replacement that the OnDelete update strategy requires, and
the log line that confirms the re-wrap.
Signed-off-by: txmazing <txmazing@gmail.com>
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
After deleting an OpenBAO app CR, the StatefulSet data PVC (data-<release>-0)
was left orphaned in the tenant namespace, consuming storage permanently.
StatefulSet-templated PVCs are never garbage-collected on StatefulSet deletion
by default, and the chart set no persistentVolumeClaimRetentionPolicy.
Pass a retention policy through the HelmRelease values. The vendored upstream
chart renders it onto the StatefulSet from
server.persistentVolumeClaimRetentionPolicy (not server.dataStorage.*):
whenDeleted: Delete reclaims the data PVC on release deletion, while
whenScaled: Retain keeps a member's data PVC on HA scale-down.
Also add a 'test' Makefile target so the package's helm-unittest suite runs in
CI (hack/helm-unit-tests.sh only runs packages that define a 'test' target).
Signed-off-by: Алексей Артамонов <aleksei.artamonov@aenix.io>
The managed-service app deploys the system openbao chart, whose server
image follows that chart's appVersion (now v2.5.1). The app Chart.yaml
appVersion ships verbatim on the openbao app CR, so leaving it at 2.5.0
makes every instance misreport its running OpenBAO version.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add the storageclass dynamic-dropdown x-cozystack-options annotation to the
storageClass field of the data apps (openbao, clickhouse, mariadb, redis,
harbor, opensearch, rabbitmq, mongodb, postgres, nats, kafka, qdrant,
foundationdb, http-cache) at the values.yaml source, so the extension is
emitted by cozyvalues-gen instead of hand-edited into values.schema.json.
kafka carries two (Kafka and ZooKeeper structs); foundationdb is nested
under the Storage struct.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Mark fields that reference cluster resources with the x-cozystack-options
vendor keyword so the dashboard renders them as dynamic dropdowns:
GPU, instanceType, instanceProfile, network (vm-instance), GPU/
instanceType (kubernetes nodeGroups), image (vm-disk), storagePool
(bucket), storageClass (all apps) and disks[].name (vm-instance). Drop
the stale static instanceProfile enum.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Flux v2.8 / helm-controller v1.5.0 embeds Helm v4, which renamed --force
to --force-replace and rejects combining it with SSA's --force-conflicts.
Existing HelmReleases with spec.upgrade.force: true fail every upgrade with
"cannot use force conflicts and force replace together", looping
upgrade -> rollback -> upgrade and blocking dependants. Observed on PR #2619
CI for tenant-root/{ingress-nginx-system,seaweedfs-db}, tenant-test/info,
and others.
Per Flux docs, upgrade.force has been a no-op since v2.8 anyway, so removing
it is safe on green clusters and unblocks broken ones.
Also fix packages/apps/kafka/templates/workloadmonitor.yaml: it referenced
.Values.replicas (no such field; values.yaml has kafka.replicas and
zookeeper.replicas). Flux v2.7 silently dropped the null; v2.8 SSA forwards
it, and the WorkloadMonitor CRD rejects "spec.replicas: null" as not
type:integer, which blocks every kafka-test install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit 7e286cf8e1)
Annotate storageClass as immutable across every stateful app whose
chart exposes one — clickhouse, foundationdb, harbor, http-cache,
kafka (Kafka + ZooKeeper), kubernetes (top-level only), mariadb,
mongodb, nats, openbao, opensearch, postgres, qdrant, rabbitmq,
redis, vm-disk. Without coverage across the catalogue the UI would
render storageClass read-only on some apps and editable on others —
a worse mental model than uniform behaviour.
The annotation lands at the schema layer (CEL XValidation rule
'self == oldSelf' on values.schema.json / types.go / embedded
openAPISchema in cozyrds/<app>.yaml). The cozystack aggregated
apiserver does not yet evaluate CEL on application-backed resources;
that is tracked at cozystack/cozystack#2657. The UI
(cozystack/cozystack-ui#6) consumes the marker today.
The PVC-pinning argument is identical for every chart: Kubernetes
fixes a PVC's storageClassName at creation time, so editing this
field on an existing resource never migrated data. The schema
annotation makes that contract explicit.
kubernetes.nodeGroups[name].storageClass is intentionally NOT
annotated: the field is optional and undefaulted, so a strict
'self == oldSelf' rule would block any future attempt to set it on
an existing node group. A richer ratcheting form is needed there —
see docs/storage-immutability.md.
A refactor in pkg/registry/apps/application/rest.go extracts the
inline schema-build path into buildSpecSchema so it can be
unit-tested. Three regression tests in rest_spec_schema_test.go pin
the contract that x-kubernetes-validations survives the
v1->internal->structural conversion (loaded from the actual MariaDB
ApplicationDefinition fixture on disk, not a hand-edited mock).
Each per-app README gains a one-line blockquote pointing to a new
docs/storage-immutability.md that explains the contract and which
consumers enforce it today.
Requires cozystack/cozyvalues-gen with @immutable support — bump the
pre-commit pin in .github/workflows/pre-commit.yml once the release
exists.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:
nano -> t1.nano micro -> t1.micro
small -> t1.small medium -> c1.small
large -> c1.medium xlarge -> c1.large
2xlarge -> c1.xlarge
Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.
Three hardcoded legacy literals in chart templates migrated:
- packages/apps/harbor/templates/harbor.yaml:121 (portal)
- packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
- packages/system/keycloak/templates/sts.yaml:115 (heap fallback)
packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.
Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>