Files
cozystack/packages/system/linstor
Myasnikov Daniil ae431808ea fix(linstor): give linstor-controller a memory limit above its baked-in -Xmx8G
linstor-server bakes `-Xms256M -Xmx8G -XX:+CrashOnOutOfMemoryError` into
the generated bin/Controller (build.gradle, jvmControllerOpts), nothing in
this chart or in piraeus-entry.sh overrides it, and a JVM handed an
explicit -Xmx never reads memory.max. The container was deliberately left
with a 700Mi request and no limit at all, which was the safe choice while
nothing else supplied one.

It stops being safe alongside the system-namespace LimitRange: a container
declaring no memory limit is given the configured default, 4Gi, so the
process's own heap ceiling would sit at twice its cgroup ceiling and a
large enough DRBD resource count would have the kernel kill the storage
control plane where the JVM would otherwise have crashed itself loudly at
8G. Stating the limit here is also what keeps the container out of the
LimitRange's reach, since a container with its own limit is never
defaulted one.

10Gi is the 8G heap plus room for metaspace, code cache, thread stacks and
direct buffers. CPU stays unlimited. The helm-unittest assertion that
pinned "no limits" is replaced by one pinning the memory limit and the
absence of a CPU limit, so neither lowering it below the heap ceiling nor
dropping it again passes silently.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 05:36:47 +03:00
..
2024-02-08 12:04:32 +01:00
2024-02-08 12:04:32 +01:00

Talos satellite graceful shutdown

Cozystack's Talos configuration removes the systemd drbd-shutdown-guard. The optional satellite preStop hook releases unused Secondary DRBD resources so their backing devices, including ZFS zvols, can be released during normal shutdown.

Parameter Default Effect
talos.gracefulShutdown.enabled false Add the satellite preStop hook when both talos.enabled and drbd.enabled are also true.

The default satellite lifecycle is unchanged. To enable the hook, set these values for the linstor component of the cozystack.linstor package:

talos:
  gracefulShutdown:
    enabled: true

Keep the setting in the package's declarative configuration at spec.components.linstor.values.talos.gracefulShutdown.enabled. The LINSTOR Helm chart embeds hack/satellite-pre-stop.py in LinstorSatelliteConfiguration/cozystack-talos; Piraeus converts its podTemplate to a strategic merge patch for each selected satellite DaemonSet. Package reconciliation and upgrades therefore retain the hook while the value remains enabled. Editing a generated Pod or DaemonSet directly does not persist through reconciliation. The hook uses Python and drbdsetup already supplied by the Cozystack satellite image.

The hook runs on every graceful satellite container stop, including ordinary pod restarts. It skips all Primary resources and any resource with an open device, rechecks each candidate, then calls drbdsetup down without force. Unknown or malformed status, command failure and timeout stop further actions. It does not demote resources, call the Kubernetes API, or change access permissions.

Enable it only when satellite and node restarts are performed sequentially, with replication and quorum checked between nodes. Piraeus creates a separate DaemonSet for each node; this chart does not serialize their rollouts. A shared configuration or image update can therefore stop unused Secondary replicas on several nodes at once and pause I/O until they return. Changing or disabling the setting can trigger a pod rollout, and the old pod still runs its existing hook.

Drain workloads before a planned node shutdown. The hook has a 60-second work budget, with each status command limited to 4 seconds and each down command to 10 seconds, and requests a 90-second pod termination grace period. The kubelet's node-shutdown budget can terminate the hook earlier; the pod grace period does not override that budget. Forced reboot, failed kubelet shutdown, still-open devices and unfinished replication are not fixed by this hook.

Related upstream discussion: Piraeus orderly shutdown on Talos.

Run make test in this directory with Helm, helm-unittest and Python 3.11 or later. The tests cover rendering with each feature flag, resource selection and rechecks, bounded errors and timeouts, dry-run behavior, and logging in the shared pod PID namespace.