mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 22:13:18 +09:00
Aleksei Sviridkin
3dc0d046b3
## What this PR does The chart renders no `shared_buffers`, and CloudNativePG does not set one either (its default parameters in `pkg/postgres/configuration.go` at v1.30.0, the operator version this repo ships, have no such key). PostgreSQL therefore starts with its built-in 128MB whatever the container gets. A full buffer pool is the whole memory limit on `t1.nano` and half of it on `t1.micro`, which is the chart's default preset. The pool is shared memory that the kernel cannot reclaim without swap. A small or idle database never fills it, but a dump, index scans over a large table or a replica catching up after a failover can, and then the instance is OOMKilled. Observed on a cluster running Cozystack 1.4.3, whose chart renders the same parameters, with a three-instance `t1.nano` database. A `pg_dumpall` run through `kubectl exec` inside the primary's container got that container OOMKilled. The dump client ran in the same container and shared its limit, so that first kill is weak evidence on its own. After the failover the old primaries could not rejoin. Their `pg_rewind` against the new 128Mi primary kept failing with `could not fetch file list: SSL SYSCALL error: EOF detected` until that primary's memory limit was raised to 512Mi, and then both rewinds completed. Below a 512Mi memory limit the chart defaults `shared_buffers` to a quarter of the limit with a 16MB floor, capped at the rendered memory request rounded down to whole MiB. CloudNativePG rejects a Cluster when `shared_buffers` exceeds that request, which can happen with memory overcommit. That is 32MB on `t1.nano` and 64MB on `t1.micro`. From 512Mi up a quarter is at least 128MB, so nothing is rendered and those clusters keep the PostgreSQL default exactly as before. An explicit `shared_buffers` in `postgresql.parameters` still wins at any size. The limit is read through `cozy-lib.resources.defaultingSanitize`, the helper that fills `spec.resources`, which is also how `packages/apps/mariadb/templates/_tuning.tpl` sizes the InnoDB buffer pool. Presets and partial `resources` overrides therefore resolve exactly as they do for the container. A quarter is the starting point the PostgreSQL documentation gives for a dedicated server with 1GB or more, and it advises a smaller share below 1GB, so this is a ceiling for small instances rather than tuning. Larger presets are left alone on purpose. Deriving a bigger pool for them is a tuning change that would restart every PostgreSQL instance on upgrade, and it does not belong in a fix. ### Upgrade impact `shared_buffers` needs a restart. On upgrade every cluster whose memory limit is below 512Mi and that does not set `shared_buffers` gets the new value, and that includes every cluster created with the default preset. The chart leaves `primaryUpdateStrategy` and `primaryUpdateMethod` at their CloudNativePG defaults, `unsupervised` and `restart`, so the operator restarts the replicas first and then the primary in place, and writes pause for the length of one PostgreSQL restart. A single-instance cluster is unavailable for that time. Clusters from 512Mi up render the same manifest as before and are not touched. ### Validation - `helm unittest` passes all 166 tests in 21 suites. The new suite covers the default preset, `t1.nano`, a `resources` override that sets only cpu, a 500Mi limit just under the threshold, the 16MB floor, `t1.small` and `t1.large` rendering no key, an explicit value on a small and on a large preset, and the other default parameters. Seven additional cases cover memory overcommit, rounding down a fractional MiB request, a request below the 16MB floor, explicit values and the unchanged 512Mi threshold. Four cases fail against the previous PR template. - Breaking the threshold comparison, the 16MB floor or the divisor, or dropping the new block, turns at least one case of the suite red each time. - `helm template` of the whole chart renders `t1.small` exactly as the base does, and the default preset differs only by `shared_buffers: "64MB"`. - `make generate` with cozyvalues-gen v1.7.0, the version CI pins, leaves `README.md`, `values.schema.json`, the Go types and the ApplicationDefinition unchanged, because `values.yaml` only gains a comment. - The new default was not exercised on a live cluster. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. It touches one chart template, a comment in `values.yaml` and a new test file. No package is added, renamed or removed, no field in `values.schema.json` is added, removed or renamed, and no default in `values.yaml` changes, so the Terraform provider has nothing to restate and the generated README is identical. Neither the website nor the provider mentions `shared_buffers`. No entry matches. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(postgres): default shared_buffers to a quarter of the memory limit with a 16MB floor, capped at the memory request, when the limit is below 512Mi and shared_buffers is not set. PostgreSQL's built-in 128MB was the whole memory limit on t1.nano and half of it on t1.micro, the default preset, so a dump, index scans over a large table or a replica resync could fill the buffer pool and get the instance OOMKilled. Affected clusters restart their instances once on upgrade, the primary last, with a short pause in writes. Clusters from 512Mi up and clusters that set shared_buffers explicitly are unchanged. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * PostgreSQL now automatically adjusts its default memory allocation for deployments with memory limits below 512 MiB, while retaining PostgreSQL’s default for larger limits. Explicitly configured values continue to take precedence. * **Documentation** * Clarified how the automatic memory allocation works, including its minimum and memory-request cap. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
How to test packages local
cd packages/core/installer
make image-cozystack REGISTRY=YOUR_CUSTOM_REGISTRY
make apply
kubectl delete po -l app=source-controller -n cozy-fluxcd