mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-08 21:14:56 +09:00
Andrey Kolkov
c503ae29d9
## What this PR does Fixes CNPG **barman-cloud plugin** backups to non-AWS, S3-compatible object stores (Ceph RADOS Gateway, and some MinIO / Cloudflare R2 builds). The plugin's sidecar uploads via **boto3**. Since botocore ~1.36 (early 2025) the default `RequestChecksumCalculation` is `when_supported`, so a flexible checksum (CRC32) plus the corresponding `x-amz-content-sha256` handling is attached to every `PutObject`. AWS S3 accepts it, but several S3-compatible backends reject the request: ``` InvalidArgument: x-amz-content-sha256 must be UNSIGNED-PAYLOAD, STREAMING-AWS4-HMAC-SHA256-PAYLOAD or a valid sha256 value. ``` so every backup / WAL-archive upload fails against them and the `ScheduledBackup` never stores anything. This sets `spec.instanceSidecarConfiguration.env` on the rendered `ObjectStore`s to carry `AWS_REQUEST_CHECKSUM_CALCULATION=when_required` (botocore reads this env var from the sidecar's environment). `when_required` computes a checksum only for operations that mandate one; a plain `PutObject` then carries none, which both AWS S3 and the affected backends accept — a safe default everywhere. No API/CRD change, no new value, no new dependency. Applied to every barman-cloud `ObjectStore` Cozystack creates: - **chart-rendered** — `packages/system/keycloak/templates/db.yaml` (Keycloak system DB) and `packages/apps/postgres/templates/db.yaml` (postgres app — backup **and** bootstrap-recovery `ObjectStore`s), via a shared `cozy-lib` helper (`cozy-lib.barman.checksumSidecarConfiguration`); - **Go-constructed** — the platform-managed `useSystemBucket=true` `ObjectStore` SSA-applied by `internal/backupcontroller/cnpgstrategy_controller.go` (`barmanSidecarConfiguration`), whose default system bucket is SeaweedFS. Regression coverage: a Go unit test on the SSA-applied `ObjectStore`, plus helm-unittest assertions on `spec.instanceSidecarConfiguration.env` for all three rendered `ObjectStore`s (mutation-checked — reverting the change fails them). Mirrors the same fix already applied to the etcd path in `cozystack/etcd-operator#342` (released in `v0.5.3`) for the same Ceph RGW backend. **Verification:** reproduced end-to-end against a real Ceph RGW endpoint. With default botocore, `PutObject` fails with the `InvalidArgument` above; with `AWS_REQUEST_CHECKSUM_CALCULATION=when_required` in the environment it succeeds. Deployed the plugin + a throwaway CNPG cluster whose `ObjectStore` carried this env: continuous WAL archiving reported `ContinuousArchiving=True` and an on-demand `Backup` completed, with the base backup and WAL segments present in the bucket. `helm template` renders the env on all three `ObjectStore`s; the `packages/apps/postgres` unit-test suite still passes. ### Screenshots N/A — no UI changes. ### Downstream repositories <!-- Walked the trigger map in docs/agents/contributing.md against the diff, file by file. The diff touches only two Helm templates (ObjectStore renders); it adds no package, no values.yaml / values.schema.json field, no API/CRD change, no variant/component/asset change, no ApplicationDefinition change, and no developer-tooling change. None of the trigger-map entries match. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(backups): CNPG barman-cloud backups now work against non-AWS S3-compatible object stores (Ceph RADOS Gateway, some MinIO/Cloudflare R2). The backup sidecar requests an S3 request checksum only when required, avoiding the "x-amz-content-sha256 must be UNSIGNED-PAYLOAD, ..." error that previously broke every backup and WAL-archive upload to those backends. No change for AWS S3. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved compatibility with S3-compatible, non-AWS storage backends for PostgreSQL and Keycloak backups and recovery. * Pin the barman-cloud sidecar to compute AWS request checksums only when required (`AWS_REQUEST_CHECKSUM_CALCULATION=when_required`) to prevent failures with strict S3 gateways. * **Tests** * Updated/added assertions in controller and Helm/YAML suites to verify the checksum pin is applied to generated and recovery ObjectStores (Postgres and Keycloak). <!-- end of auto-generated comment: release notes by coderabbit.ai -->