Commit Graph
4 Commits
Author SHA1 Message Date
Aleksei Sviridkin a2e10312de fix(platform): stop treating a failed ConfigMap read as a fresh cluster (#4792)
## What this PR does

A single failed API read at the start of the platform migration Job
could skip every pending migration and still end green.
`run-migrations.sh` checked whether the `cozystack-version` ConfigMap
exists with a silenced `kubectl get` and looked only at the exit code. A
timeout, a dropped connection or an apiserver restart during the upgrade
all counted as "no ConfigMap", so the script stamped the target version
and exited 0. The hook Job is rendered only when that ConfigMap already
exists, so in practice an error was the only way into this branch.

Now the script creates the ConfigMap only when it is really missing. The
check uses `--ignore-not-found --output name`: kubectl drops NotFound
and nothing else, so empty output means a genuine NotFound. Any other
error keeps a non-zero exit, `set -e` aborts the script, and the Job
retries the pod. The rest of the script already works this way: the read
of the stamped version aborts on error, and the stamp runs under
`pipefail`.

The tests cover both branches. A new case in `hack/run-migrations.bats`
makes the existence check fail with a connection error and expects a
non-zero exit, no migration run and the stamp left where it was. It
fails against the old check. The fake kubectl returns empty output for a
missing ConfigMap only when `--ignore-not-found` is passed, as real
kubectl does, so dropping the flag breaks the existing "no ConfigMap"
case.

No migration is added, so `targetVersion` stays. The script ships in the
migrations image, and the release build restamps that image's digest as
usual.

Several older migrations have the same problem in a smaller form. They
check for an object with `kubectl get ... >/dev/null 2>&1` and skip
their own work when the check fails, for example migrations 31, 32, 36,
48, 50 and 51. A transient error there skips one migration's work, not
all of them. Editing migrations that already shipped has its own risk,
so I'd track that separately.

Closes #4579

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(platform): a transient API error while the platform migration Job checks for the cozystack-version ConfigMap now fails the pod so the Job retries, instead of stamping the target version and silently skipping every pending migration.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Migration checks now distinguish a missing version record from an
error while checking for it. When the check fails, the pod stops instead
of treating the record as missing and proceeding with migrations.
* Added coverage for TLS-handshake timeouts during the check, confirming
that migrations do not run and the stamped version remains unchanged.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-10-10 20:37:53 +03:00
Aleksei Sviridkin cbbb69afc3 fix(platform): stop treating a failed ConfigMap read as a fresh cluster
run-migrations.sh decided whether cozystack-version exists by the exit
status of a silenced kubectl get, so a timeout or a dropped connection
counted as "no ConfigMap": the runner stamped the target version and
exited 0, and every pending migration was skipped for good. The hook Job
only renders when the ConfigMap already exists, so in practice an error
was the only way to reach that branch.

--ignore-not-found suppresses NotFound alone; any other error keeps its
non-zero status and now fails the pod, so the Job retries the read.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-07 12:06:18 +03:00
Myasnikov Daniil cc942fed68 test(tests): run unit suites with Bats and retain POSIX coverage
The shell translator rewrites test bodies and loses Bats reporting.
The unit lane needs complete TAP results and JUnit reports without
changing how live-cluster suites run.

Sourced production helpers still need a dash pass, and unit test bodies
need nounset without exporting shell options into subprocesses.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-10-07 11:04:00 +03:00
Aleksei Sviridkin 3a07885b70 fix(platform): resume a retried migration Job from the stamped version
The migration hook Job gets CURRENT_VERSION once, at render time, and a
failed pod is retried with that same environment. Each migration that
finished has stamped the cozystack-version ConfigMap by then, yet the
retry started from the rendered version and ran those migrations again.
Migrations are written to run once, so a re-run is not safe in general.

The runner now starts from the version stamped in the ConfigMap. A
failed read aborts the pod, since falling back would bring the re-run
back and the Job retries anyway. The rendered value is kept only when
the ConfigMap holds no integer version, which is also the case where
the chart could not derive a real one from it. An all-digit stamp past
the shell's integer range aborts too: its comparison fails inside an
if, out of reach of set -e, and the Job would go green having run
nothing, while the chart renders that stamp as 0.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-02 21:54:46 +03:00