The system memory limits page read "resourceQuotas other than the GPU
budgets is configured", which does not parse.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The tenant chart now renders its ResourceQuota and the default
LimitRange only for keys other than the gpu.cozystack.io/* GPU budgets,
so a tenant with only GPU budgets has no LimitRange. The comments and
the system memory limits page that explain sidecar sizing and system
defaults by that LimitRange said any declared resourceQuotas brings it.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
An admitted Pod loses the distinction between explicit and defaulted
limits. Its disappearance cannot prove a lower default is safe for
its replacement. Keep that decision on the Namespace until an
administrator acknowledges the particular target, without waiving
current blockers or foreign policy ownership.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Preserve foreign same-name objects and keep create-time SSA non-force.
Only ignore failed pods controlled by terminal Jobs.
Make the failure-containment test inject Apply errors for real.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
List LimitRanges by the ownership label once per Package reconcile when
the feature is disabled, and remove managed objects even in namespaces
outside the current Package target set. Skip the per-namespace path while
disabled and keep cleanup failures non-fatal to namespace reconciliation.
Update the flag help, installer guidance, and operations documentation to
describe the global cleanup boundary.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The raise-and-grace machinery cannot work: LimitRanger writes the
raised limit onto the very container that justified the raise, so at
that container's next admission it carries a limit, the scan skips
containers that have one, and the raise has erased its own evidence.
The ceiling then drops on schedule and the workload is rejected
permanently. Any such machinery approximates admission logic ahead of
admission, and that is incomplete by construction: a
LimitRange default is one number for a whole namespace and cannot mean
"the configured limit, or this container's request, whichever is
larger". Only per-pod conditional defaulting expresses that, which is a
mutating webhook and not this.
The justification for the machinery had also expired. It was designed
when the default was 4Gi, where withholding was self-perpetuating for
VPA-managed namespaces. At 32Gi nothing in the tree approaches the
ceiling -- the largest static memory request is 2Gi -- and VPA sets
requests and limits together, so those containers carry their own limit
and were never blockers.
So a namespace holding a container that requests more than the
configured limit with no limit of its own is now left with no default at
all, and anything written there earlier is withdrawn. It fails in the
safe direction, which is the whole argument: a namespace with no
memory.max is a namespace behaving as it did in every release before
this, while a default below a request is a namespace whose workload
cannot be admitted. It needs no stored state and no timer, because
withholding is not a decision that has to survive between reconciles --
once the oversized request is gone the default is applied again on the
next pass.
Gone with it: the grace period, the clear-since annotation and its
clamp, the injected clock, containerMemoryDefault, raisedCeiling, and
fourteen tests. The scan stays, because deciding whether to apply or
withhold is what it was always for. The foreign-LimitRange ownership
guard, the disable path and the 32Gi default are untouched.
The residual sharp edge is stated rather than machinery: a workload
whose only evidence is a live Pod, built from a custom resource with no
template this scan can read, can still be missed if the scan runs while
that Pod is absent, and applying the default then rejects its
recreation. At 32Gi that is unreachable in this tree. It becomes
reachable if an operator lowers the knob below such a workload's
request, which the flag help, values.yaml and the operations page now
say plainly.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The coexistence guard treated a namespace as already-governed only when
another LimitRange carried a container memory default or max, and said
so in its comment. That is not the set of fields that collide with what
this reconciler writes. A container min above the default request
rejects the pod, and so does a maxLimitRequestRatio, since a 32Gi
default against a 32Mi default request is a ratio of 1024:1 and no cap
allows it. So the namespace-wide admission failure the guard exists to
prevent stayed reachable through fields the guard did not read, and
kube-system is where it is likeliest, this feature having deliberately
brought it in scope.
Enumerating the colliding fields does not fix it, because at the Pod
scope the collision is not decidable here. Whether a per-container
default of 32Gi breaches a pod-wide max depends on how many containers
the pod has, and the pods in question do not exist yet: any arithmetic
about them is a guess, and a wrong guess costs a namespace its entire
pod admission. So the rule is ownership rather than arithmetic. If
anything else in the namespace has an opinion about memory, at either
scope and in any field, the namespace is not this reconciler's to
default. Narrow still on the resource, because nothing here writes cpu
or storage and a LimitRange bounding those must keep coexisting. This
is the rule the tenant-namespace exclusion already rests on.
The pod scan no longer skips Failed pods, only Succeeded. A pod that
ran to completion is never recreated from its spec, but an eviction or
a node crash leaves a Failed carcass holding the same oversized request
that its controller will recreate, and that carcass is frequently the
only surviving evidence the namespace needs a raised ceiling, since the
shape the scan looks for cannot be admitted once the namespace has
settled. Dropping it lowered the ceiling under a workload that was
coming back, which is what the grace period exists to prevent.
A grace stamp dated ahead of the clock is clamped to now. Unclamped it
held the ceiling for the skew plus the grace period, silently, so an
hour of skew bought ninety minutes instead of thirty. Clamping never
shortens the wait, it only stops it being extended.
The two flag defaults are named constants the flag registration and its
test both read. The test previously duplicated the literal and still
asserted 4Gi after the default moved to 32Gi, so it misinformed a
reader about the shipped value and pinned nothing.
The disable claim is narrowed to what the code guarantees. "Removes any
previously created one" was universal, but the reconciler only runs for
namespaces an active Package still targets, so a namespace whose
Package was removed keeps its LimitRange. The flag help, values.yaml
and the operations page now say that, and name the managed-by label for
finding leftovers by hand.
The test asserting a Pod-scoped memory max should be ignored is gone,
since under the ownership rule a Pod-scoped memory bound withholds.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Two imprecisions in the page written alongside the grace period. It said
a lowered limit reaches previously-raised namespaces "within" the grace
period, which is backwards: the grace period is a floor, not a ceiling,
and since reconciles are event-driven the drop lands on the first one
after it elapses. And the withdraw path was described only as leaving
the namespace alone, without saying that it hands the namespace to the
administrator's policy in full, including any admission rejection that
policy implies -- which is theirs, and would have happened without
Cozystack, but is worth naming where an operator is being told the
operator stopped writing.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The grace period was documented as a property of a raised ceiling coming
back down, which is where it started, but it applies to any namespace
defaulting more than the configured limit -- and a namespace sitting at
a previously higher configured value is indistinguishable, on the
object, from one the operator raised. So lowering the knob reaches those
namespaces within the grace period rather than on the next reconcile,
which is worth stating before an operator concludes the setting did not
take. The page now also shows how to read the countdown off the objects.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
docs/operations/system-memory-limits.md merged ahead of the code it
describes, so main documents two installer knobs and a 4Gi default that
exist nowhere outside this branch. An operator following the shipped page
today sets a value that silently does nothing.
The page is corrected to the default this branch ships and gains the
three behaviours it had no account of at all: that a namespace holding a
request above the ceiling is raised rather than skipped, that a raised
ceiling only comes back down after the scan has been continuously empty
for a grace period and why that delay has to exist, and that a namespace
another LimitRange already governs is left alone. Each names the log line
to grep for, since none of the three is visible in the object.
The ceiling rationale is also stated more honestly: the scan reads
declared requests and never a container's real resident set, which is the
reason the default sits far above any plausible working set rather than
close to one.
Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Talos v1.14.0 dropped the global memory PSI clause from the default
OOM trigger, so the page no longer matches the version Cozystack
ships. The OOMConfig example now serves host nodes still on v1.13.x,
which can shed the clause without upgrading.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add docs/operations/system-memory-limits.md, recording why system
components carry memory limits. That reasoning is not recoverable from
the values themselves, and is exactly the kind of thing a well-meaning
later change removes as redundant.
The Talos userspace OOM handler (v1.12+, enabled by default) selects
its victim by ranking cgroups that lack memory.max. Three consequences
are each easy to get wrong on their own: only a limit grants immunity,
since a request merely moves BestEffort to Burstable and under
strictCgroupClassOrdering that means "killed second"; every container
in the pod needs one, init containers included, because kubelet sets
pod-level memory.max only when all of them have it; and PriorityClass,
system-node-critical included, has no bearing on it whatsoever,
because the handler walks cgroupfs and holds no Kubernetes client.
That last one is the trap, since setting a priority looks like a fix.
The page also covers the two installer knobs and the constraint that
makes them dangerous to lower (a defaulted limit below a container's
own request fails admission), how to confirm a pod cgroup really
carries memory.max rather than trusting the QoS class, what is
deliberately left uncovered, and how to relax the global-PSI backstop
clause of the trigger on a cluster that trips it persistently.
Every Talos claim is checked against the siderolabs/talos v1.13.6 tag,
which is the version packages/core/talos currently ships, and every
Cozystack claim against the tree.
Ordering: this documents behaviour introduced by the system-namespace
LimitRange and the node-DaemonSet resource changes, so it should not
merge ahead of them.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Aleksei Sviridkin