mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
Myasnikov Daniil
c9ba7a3cca
The Talos userspace OOM handler (v1.12+) ranks cgroups with
memory_max.hasValue() ? 0.0 :
{Besteffort: 1.0, Burstable: 0.5, ...}[class] * memory_current
and discards every cgroup scoring zero, so only a memory limit takes a
pod out of the victim set. Selection is decoupled from the trigger: a
tenant workload thrashing against its own limit drives node memory PSI,
and a limit-free system DaemonSet is killed for it. metallb-speaker and
linstor-satellite were the observed victims.
Set requests and limits on the node DaemonSets that had none. Requests
come from 14d peak working set measured on a 3-node dev cluster; limits
are deliberately loose, sized as a runaway backstop rather than a fitted
ceiling, because several of these scale with cluster size that a dev
cluster understates and because undersizing converts a rare
pressure-driven kill into a deterministic one. A limit costs nothing
until it is hit -- only the request reserves.
memory.max lands on the POD cgroup only when EVERY container in the pod
carries a limit, so sidecars and init containers are covered too; one
limit-free container returns the whole pod to the victim set.
Two DaemonSets are not chart-rendered and needed their own seam:
linstor-satellite via the LinstorSatelliteConfiguration podTemplate, and
virt-handler via KubeVirt customizeComponents.patches, since virt-operator
owns that DaemonSet. Note linstor-controller stays deliberately limit-free
-- it is a JVM that must not be OOM-killed mid-operation, and cluster_test
pins that.
Two things the multus manifest needed beyond the values change.
customize-deployment.patch is the source for that template, so the
limits are carried there rather than only in the generated file, which
`make update` would discard. And the reason kube-multus had no memory
limit was that this patch DELETED upstream's 50Mi rather than raising
it -- 50Mi being well under the 230Mi measured peak -- which is what
removed memory.max from the pod cgroup in the first place.
Not covered, because upstream hardcodes the containers with no resources
knob: the four frr-k8s cp-* init containers, the kube-ovn hostpath-init
and install-cni init containers, and cozy-proxy, whose vendored chart
exposes no resources value at all. hami and kilo are left alone as
optional components with no usage measurements behind them. All of these
still get a limit from the default LimitRange the operator maintains in
every system namespace.
Every limit is pinned by a helm-unittest assertion, and every assertion
was mutation-tested: changing the rendered value turns its suite red.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
58 lines
2.9 KiB
Makefile
58 lines
2.9 KiB
Makefile
CILIUM_TAG=$(shell awk '$$1 == "version:" {print $$2}' charts/cilium/Chart.yaml)
|
|
|
|
export NAME=cilium
|
|
export NAMESPACE=cozy-$(NAME)
|
|
|
|
include ../../../hack/common-envs.mk
|
|
include ../../../hack/package.mk
|
|
|
|
# hack/helm-unit-tests.sh discovers work by probing for this target, so a
|
|
# package without one is skipped in silence however many suites sit under
|
|
# tests/.
|
|
test:
|
|
helm unittest .
|
|
|
|
update:
|
|
rm -rf charts
|
|
helm repo add cilium https://helm.cilium.io/
|
|
helm repo update cilium
|
|
helm pull cilium/cilium --untar --untardir charts --version 1.19
|
|
$(SED_INPLACE) -e '/Used in iptables/d' -e '/SYS_MODULE/d' charts/cilium/values.yaml
|
|
# Drop the whole k8s<1.30 AppArmor annotations block from the cilium-agent
|
|
# DaemonSet. Cozystack manages these annotations through cilium.podAnnotations
|
|
# in values.yaml on every k8s version, so keeping the upstream block would
|
|
# produce duplicate mapping keys on k8s<1.30.
|
|
perl -i -0pe 's|\n \{\{- if not \.Values\.securityContext\.privileged \}\}\n \{\{- if semverCompare "<1\.30\.0".*?\n \{\{- end \}\}\n \{\{- end \}\}\n \{\{- end \}\}||s' \
|
|
charts/cilium/templates/cilium-agent/daemonset.yaml
|
|
@! grep -q 'container\.apparmor\.security\.beta\.kubernetes\.io' \
|
|
charts/cilium/templates/cilium-agent/daemonset.yaml || \
|
|
{ echo 'ERROR: perl patch did not remove the upstream AppArmor block from cilium-agent/daemonset.yaml (upstream template format may have changed)' >&2; exit 1; }
|
|
version=$$(awk '$$1 == "version:" {print $$2}' charts/cilium/Chart.yaml) && \
|
|
$(SED_INPLACE) "s/ARG VERSION=.*/ARG VERSION=v$${version}/" images/cilium/Dockerfile
|
|
# Verify the IPv6-passthrough BPF patch still applies to the new upstream tag,
|
|
# so the bump author learns about drift immediately instead of at image build.
|
|
version=$$(awk '$$1 == "version:" {print $$2}' charts/cilium/Chart.yaml) && \
|
|
tmpdir=$$(mktemp -d) && \
|
|
{ curl --fail --silent --show-error --location \
|
|
"https://raw.githubusercontent.com/cilium/cilium/v$${version}/bpf/bpf_host.c" \
|
|
-o "$$tmpdir/bpf_host.c" || { rm -rf "$$tmpdir"; exit 1; }; } && \
|
|
{ patch --dry-run --batch --forward --fuzz=0 -p1 -d "$$tmpdir" \
|
|
< images/cilium/patches/pass-ipv6-to-stack-when-ipv6-datapath-disabled.diff || \
|
|
{ echo "ERROR: images/cilium/patches/pass-ipv6-to-stack-when-ipv6-datapath-disabled.diff no longer applies to upstream bpf/bpf_host.c v$${version}; regenerate it for the new cilium version" >&2; \
|
|
rm -rf "$$tmpdir"; exit 1; }; } && \
|
|
rm -rf "$$tmpdir"
|
|
|
|
image:
|
|
docker buildx build images/cilium \
|
|
$(call image-tags,cilium,$(CILIUM_TAG)) \
|
|
$(call cache-args,cilium) \
|
|
--metadata-file images/cilium.json \
|
|
$(BUILDX_ARGS)
|
|
REPOSITORY="$(REGISTRY)/cilium" \
|
|
yq -i '.cilium.image.repository = strenv(REPOSITORY)' values.yaml
|
|
TAG=$(IMAGE_TAG) \
|
|
yq -i '.cilium.image.tag = strenv(TAG)' values.yaml
|
|
DIGEST=$$(yq e '."containerimage.digest"' images/cilium.json -o json -r) \
|
|
yq -i '.cilium.image.digest = strenv(DIGEST)' values.yaml
|
|
rm -f images/cilium.json
|