mirror of
https://github.com/cozystack/cozystack.git
synced 2026-10-24 20:16:50 +09:00
main
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
42a27567ce |
fix(fluxcd): keep Helm uninstall working during flux-plunger recovery
flux-plunger recovers a "has no deployed releases" HelmRelease by briefly suspending it, deleting the stale storage Secret and unsuspending. helm-controller v1.5.x skips the uninstall and drops its finalizer when it handles a delete on a suspended HelmRelease, and it wakes on the same delete event flux-plunger does, so a delete inside that window orphaned the release's DataVolume, PVCs, RBAC and storage Secret. Lifting the suspension on the delete event cannot win that race reliably. Hold the release with a flux-plunger finalizer while it is suspended. The finalizer is applied together with the suspension in one server-side apply. On a delete flux-plunger lifts its suspension and keeps the finalizer, helm-controller reconciles the unsuspended generation and runs the uninstall, and flux-plunger removes the finalizer once helm-controller's own is gone and that generation is observed. This holds whichever controller handles the delete first. A completed recovery lifts the suspension and the finalizer together, and a terminating release never begins recovery. Decide ownership of a suspension by which field manager owns spec.suspend in metadata.managedFields rather than by an annotation. The apiserver transfers the field away the moment an operator, cozyhr resume or a backup controller writes a different value to it, so a later suspension is never misread as flux-plunger's, and one set through the shared flux-client-side-apply manager (platform migrations, the tenant delete hook, a manual kubectl apply) is never claimed. A release deleted under another manager's suspension is left to that manager's decision. Acquire applies without forcing first and, on a conflict, re-fetches and forces only if spec.suspend is still false: a backup controller owns the field at false through a dynamic-client Update, which a non-forced apply can never take, while true is an active suspension to leave alone. Every apply is preconditioned on the resourceVersion and uid of the read that decided on it. Without the resourceVersion, the unforced apply of true over a foreign true is accepted as shared ownership, a window the optimistic-locked merge patch this replaces had closed, and lifting the suspension and the finalizer from a release read as live would delete it without an uninstall if helm-controller had meanwhile handled a delete. Without the uid, an apply to a release that has gone would create an empty one. The apply carries only spec.suspend and the finalizer, so it cannot leak the required spec.interval. Read the release, every re-check and the storage Secret list from the apiserver rather than the informer cache. A lagging cache would let the processed-version guard delete the revision below the one just reclaimed, and the ClusterRole grants secrets get and list but not the watch an informer needs. Emit a Warning when flux-plunger holds a suspension it failed to clear, and Normal events when it leaves an external suspension alone or defers a recovery. An external suspension on a release flux-plunger has recovered before is a Warning instead: it is most likely one an earlier flux-plunger left under the shared manager, which nothing lifts unless an operator does. A release deleted under another manager's suspension is a Normal event when that was a migration's choice, and a Warning when the release carries flux-plunger's processed-version annotation, since then the skipped uninstall is an accident. Keep a single writer with leader election. A second process able to act under the same field manager could clear a suspension the first is still recovering, and no rollout strategy prevents one: the next pod starts as soon as the old one is terminating, and an eviction or a node drain replaces a pod the same way. The old pod keeps its Lease, unreleased, through its stop and drain, and the Lease outlasts both, so the next pod acts only after the old one is done; the price is that a new pod waits for the Lease to run out. The strategy sets maxSurge 0 with maxUnavailable 1 and stays RollingUpdate because existing installs carry the apiserver-defaulted rollingUpdate block and helm-controller applies server-side, where type Recreate over it is rejected on upgrade. Give the finalizer a way out when flux-plunger itself goes. Once the manager has stopped, within a bounded time, and with it every reconcile, flux-plunger drains its finalizers under a deadline that, with the stop, fits inside the pod's termination grace period. A stop that overran its bound is drained too and still reported, and only a pod that led drains. The drain writes its events straight to the apiserver, since the manager's recorder stops with the manager. A recovery in progress is aborted, the suspension and the finalizer lifted together for the next flux-plunger to start again, and a release being deleted has the suspension lifted and keeps the finalizer until helm-controller has run the uninstall. A release helm-controller does not handle in time keeps it. The waiting lives in the controller, where the envtest suites hold it against a real apiserver's generations and finalizers, rather than in a shell script against a model of them. A pre-delete hook on the chart is the backstop for what the drain could not finish, so uninstalling the component never leaves a release that tenant teardown waits on forever. It stops flux-plunger, waits out its grace period, then lifts any remaining suspension and finalizer without waiting again, naming each release it lets go of before helm-controller uninstalled it. It lifts a suspension only where flux-plunger alone holds it, with a JSON patch preconditioned on the resourceVersion and uid it read: kubectl apply --server-side on a release last written by a client-side apply moves that apply's fields to the applying manager and strips every one the body leaves out. The Job's deadline sits under the 300s helm-controller gives a pre-delete hook by default. A Deployment that cannot be stopped or a listing that never succeeds ends the hook before any finalizer is touched, and a finalizer it cannot remove ends it after the sweep; either way it exits non-zero: a failed pre-delete hook leaves the release untouched and helm-controller retries the uninstall, hook included, from a state it can still converge from, where exiting 0 would remove the component and strand the finalizer. A flux-plunger pod still present after its grace period, typically one stuck Terminating on a NotReady node, is only reported, since failing on it would fail every retry until the node returns; because it may still be setting finalizers, they are listed again after the sweep and any found fails the hook. The Job runs once, since helm-controller's retry is the one that starts afresh, and is kept after success so that its report stays readable; helm.sh/hook-output-log-policy would make helm-controller call a nil output writer (fluxcd/helm-controller#1562). A delete still held after ten minutes raises a Warning, and the held state is re-checked on a timer instead of only on helm-controller's next status write. The chart README documents the finalizer and how to clear it. Run the envtest suites on every pull request through a new make target that fetches a kube-apiserver and etcd with a pinned setup-envtest and takes the HelmRelease CRD from the in-tree Flux manifests: the unit tests drive a fake client, and everything the finalizer rests on (apply preconditions, the finalizer set-merge, real managedFields) only holds against a real apiserver. CI caches those binaries, and the target uses them without asking the release index, so a pull request does not depend on their sources being reachable; the control plane gets a longer start timeout because it starts beside the other test goals. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
|
cc942fed68 |
test(tests): run unit suites with Bats and retain POSIX coverage
The shell translator rewrites test bodies and loses Bats reporting. The unit lane needs complete TAP results and JUnit reports without changing how live-cluster suites run. Sourced production helpers still need a dash pass, and unit test bodies need nounset without exporting shell options into subprocesses. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
|
44507d2cd0 |
docs(agents): add downstream repository checklist to the PR template
Cozystack is upstream for repositories that nothing keeps in sync with it, and no CI job compares the two sides, so a change here breaks them in silence. Only two narrow paths are automated, and both fire on a release rather than on a PR. It has already happened. The website docs generator works from an app list hardcoded in its own Makefile, and packages/apps/opensearch was never added to it, so an application we ship has no reference page at all. The Terraform provider still offers a Kubernetes version this repo removed. The plugin repo still sends operators to hack/ scripts that were deleted, one of them rewritten as a Go command. Add a Downstream Repositories trigger map to the contributing guide, naming per repository which change here forces a change over there and which file to touch, and a checklist in the PR template that links to it. The template addresses AI agents directly: they author a large share of PRs and will otherwise tick a box without opening the follow-up it claims. The coupling is wider than the obvious ones, and mostly invisible. talm restates the node contract, overlapping ours but already diverged, so it must be diffed rather than overwritten. cozyhr reads an annotation the operator writes and silently merges the wrong values if it is renamed. cozy-proxy is vendored in from its own repo, yet consumes labels produced here and quietly stops handling external VM services if they move. The telemetry server queries metric names emitted here, and a rename just empties its dashboards. None of these fail loudly. Guard the map with a unit test, because a map about silent drift must not drift silently itself. It pins every in-repo path the map cites to a file that exists, pins the map and the checklist to the same repository list, and pins the workflow carve-out below to the map's location, to the job that reads the flag, and to the output that carries it: cutting any one of those three leaves the other two looking wired up. The header states plainly what the suite cannot check, so a green tick is not mistaken for a correct map: whether a coupling is real, and whether it still holds over there, stays a human's job. Editing the map alone is a docs-only PR, which skips the unit tests and would exempt the map from its own guard. Rather than widen the docs-only rule, which also gates the Talos image build, the plan step exports a separate flag that only the unit-test job reads: a docs PR now runs the tests and still builds nothing. It detects the map by its old path too, since a rename is reported as the new path alone. Document that gh pr create --body and --body-file replace the body wholesale, which would otherwise drop the checklist from every CLI-authored PR. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |