fix(backups): give the barman-cloud sidecar an explicit CPU limit

The sidecar was left without a CPU limit so that a throttled sidecar
would slow WAL archiving down rather than fail it. In a tenant with
resourceQuotas that does not hold: the tenant LimitRange gives every
container a 250m CPU limit by default, so the sidecar gets it and
compresses full backups and WAL segments on a quarter of a core. A
1.6 GB database took 30 minutes to back up.

Set an explicit one-core limit in both places that build the sidecar
configuration, the backup driver and cozy-lib, the same way the memory
request and limit already override the LimitRange defaults.

Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
This commit is contained in:
Mattia Eleuteri committed 2026-11-11 09:20:19 +01:00
1 parent c6cba19946
commit f1dd2d17fc
4 files changed
+21 -15

No files matched your search

@@ -2224,9 +2224,12 @@ func buildBarmanPlugin(objectStoreName, serverName string) cnpgtypes.PluginConfi
// high-water mark on a 38 MB database is 254 MiB, so 128Mi cannot hold it. A
// tenant that leaves resourceQuotas empty has no LimitRange, and there the
// sidecar had no requests and no limit at all. 256Mi holds the measured working
// set and 1Gi is four times the measurement. No CPU limit is set, matching the
// entityOperator precedent in packages/apps/kafka: a throttled sidecar stalls
// WAL archiving instead of failing it.
// set and 1Gi is four times the measurement. The CPU limit is explicit for the
// same reason: left unset, that LimitRange gives the sidecar its 250m default,
// so gzip compression of full backups and WAL segments runs on a quarter of a
// core (a 1.6 GB database took 30 minutes to back up). One core keeps the
// sidecar from starving the instance while removing that ceiling; it counts
// against the tenant's limits.cpu budget, one core per instance pod.
//
// Checksum: since botocore ~1.36 the default (when_supported) attaches a
// flexible checksum to every PutObject, which non-AWS S3-compatible backends
@@ -2246,6 +2249,7 @@ func barmanSidecarConfiguration() *cnpgtypes.InstanceSidecarConfiguration {
corev1.ResourceMemory: resource.MustParse("256Mi"),
},
Limits: corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse("1"),
corev1.ResourceMemory: resource.MustParse("1Gi"),
},
},
@@ -2152,8 +2152,11 @@ func TestApplyClusterPluginBackup_PatchesExistingCluster(t *testing.T) {
sc.Resources.Requests.Cpu().MilliValue() != 100 {
t.Errorf("ObjectStore leaves Barman subject to undersized tenant resource defaults: %+v", sc.Resources)
}
if _, ok := sc.Resources.Limits[corev1.ResourceCPU]; ok {
t.Errorf("ObjectStore caps sidecar CPU, which throttles WAL archiving instead of failing it: %+v", sc.Resources.Limits)
// Without an explicit CPU limit the tenant LimitRange caps the sidecar at
// its 250m default, and every backup and WAL upload compresses on a
// quarter of a core.
if sc.Resources.Limits.Cpu().MilliValue() != 1000 {
t.Errorf("ObjectStore leaves the sidecar CPU limit to the tenant LimitRange default: %+v", sc.Resources.Limits)
}
// The ObjectStore must be owner-referenced to the Cluster so Kubernetes GC
// removes it when the Cluster is deleted (no orphan in the platform flow,
@@ -13,7 +13,7 @@ set:
s3CredentialsSecret:
name: postgres-backup
tests:
- it: sets resources for native backup without inheriting the tenant 128Mi limit
- it: sets resources for native backup without inheriting the tenant 128Mi and 250m limits
set:
backup.enabled: true
asserts:
@@ -24,15 +24,11 @@ tests:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 1Gi
documentSelector:
path: kind
value: ObjectStore
- notExists:
path: spec.instanceSidecarConfiguration.resources.limits.cpu
documentSelector:
path: kind
value: ObjectStore
- equal:
path: spec.configuration.s3Credentials.accessKeyId.name
value: postgres-backup
@@ -13,10 +13,12 @@ LimitRange, which is cozy-keycloak and any tenant that leaves resourceQuotas
empty, gave the sidecar no requests and no limit at all, so nothing reserved
memory for it and nothing bounded it. The same request and limit settle both:
256Mi holds the measured working set and 1Gi is four times the measurement.
No CPU limit is set, matching the entityOperator precedent in
packages/apps/kafka: a throttled sidecar stalls WAL archiving instead of
failing it. Inside a tenant the limit counts against the ResourceQuota
limits.memory budget, 1Gi per instance pod.
The CPU limit is explicit for the same reason: left unset, that LimitRange
gives the sidecar its 250m default, so gzip compression of full backups and
WAL segments runs on a quarter of a core (a 1.6 GB database took 30 minutes
to back up). One core keeps the sidecar from starving the instance while
removing that ceiling. Inside a tenant both limits count against the
ResourceQuota budget, one core and 1Gi per instance pod.
Checksum: since botocore ~1.36 (early 2025) the default
RequestChecksumCalculation is when_supported, which attaches a flexible checksum
@@ -39,6 +41,7 @@ instanceSidecarConfiguration:
cpu: 100m
memory: 256Mi
limits:
cpu: "1"
memory: 1Gi
env:
- name: AWS_REQUEST_CHECKSUM_CALCULATION