Backport of #4557 to release-1.6. The upstream project moved GitHub organisations from intreecom to treetscom, and no image has been published under the old location since. The chart on this branch still points there, so the component cannot pull at all: the Deployment sits in ImagePullBackOff while the HelmRelease reports healthy, because Helm only waited for the objects to be created.
0.1.0 also carries the fix that makes balancer targeting follow the service's externalTrafficPolicy, which a cluster relying on `Local` needs in order to keep sending traffic only to nodes that run the backing pods.
(cherry picked from commit b314514950)
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
dl.min.io now answers 410 Gone for every community mc build, since
MinIO archived the project, so the sandbox image no longer builds.
The final mc release is still published as a GitHub release asset.
Pin it and verify the per-arch checksum, because the old install
trusted whatever the URL served.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit e66f134564)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Promotes `v1.6.4-rc.1` to stable `v1.6.4` without rebuilding containers.
Their digests remain bit-for-bit what passed rc e2e; the tag-rewritten
packages tree is re-serialized as a temporary OCI candidate and pinned
here by digest. On merge, pull-requests-release.yaml verifies that
candidate against this tree before creating the write-once `v1.6.4` tag,
retagging the candidate and container digests to `v1.6.4` (+`:latest`
when this is the newest stable), publishing the stable cozy-installer
chart, and publishing the release. Do NOT squash-merge (decision B): the
stable tag must attach to a real merge commit.
⚠️ **RC e2e gate bypassed** — this promotion did not verify green full
e2e evidence for `v1.6.4-rc.1`.
ℹ️ E2E already ran against the immutable rc, so this promote PR does not
run E2E by default. A maintainer may add the `full-e2e` label to run the
full suite again.
✅ Includes `docs/changelogs/v1.6.4.md`, which finalize uses verbatim as
the published release body.
✅ Website docs PR opened/refreshed on `cozystack/website` (branch
`update-docs-v1.6.4`), generated from the `release-1.6.4` staging
branch. **Do NOT merge that PR until `v1.6.4` is published** — merging
it early flips the site's latest-version pointer to an unpublished
version and 404s its docs/API links (see the PR body).
This reopens a fork pull request from a branch in this repository so its
CI can run. The commit is the one from #4481 by @yankawai, unchanged. On
release-1.6 the pull-request CI pushes the images it builds, and a run
from a fork has no registry credentials, so #4481 could never get past
the build jobs to e2e. The description below is the author's.
## What this PR does
Wait for both ZooKeeper PVCs to disappear after Kafka deletion. Negative
assertions can stop on an API read error before their timeout; deletion
watches wait for the resource to be removed. The existing Bound and
KafkaTopic checks are unchanged.
Carries the test-only change from c92828ce in #4339, which was not
included in #4402. This targets `release-1.6`; the Kafka suite on `main`
uses KRaft without ZooKeeper.
Validated with Chainsaw 0.2.15: delayed deletion, an already absent PVC,
an interrupted watch, and the read-error fixture pass. A retained PVC
and forbidden access fail. The full Kafka E2E run remains for CI.
### Screenshots
Not applicable.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
### Release note
```release-note
NONE
```
This reopens a fork backport from a branch in this repository so its CI
can run. The commit is the one from #4339 by @yankawai, unchanged. On
release-1.6 the pull-request CI pushes the images it builds, and a run
from a fork has no registry credentials, so #4339 stopped at the build
jobs and never reached e2e. The description below is the author's.
## What this PR does
Manual backport of #4254 to `release-1.6`.
The automated backport, #4322, stopped with conflict markers in four
files as its only commit, so it is red on DCO and unmergeable as it
stands.
Every conflict has the same cause: the `kubevirt.disabledFeatureGates`
work landed on main after 1.6 was cut, and the cherry-pick carried it
into hunks this line does not have. Four differences follow:
- platform values gains a `kubevirt` block holding `migrations` alone,
and the iaas bundle gains only the migrations hop;
- the wiring test takes the four migrations cases and the case for a
null `kubevirt` block, without the assertions that belong to
disabledFeatureGates cases this line does not have;
- `update_idempotency_test.sh` gains `run_update_logged`, a four-line
helper that arrived on main with that work and the new cases need;
- the kubevirt Makefile header counts six sed patches rather than five.
Verified on this branch: 9 kubevirt chart tests, 31 platform wiring
tests (102 across the platform suite), and `update_idempotency_test`
PASS under GNU make and sed. Removing the new awk guard from the
Makefile turns that test red naming the migrations block, so the ported
guard is doing work. On this line `(.Values.kubevirt).migrations` is the
only reader of `.Values.kubevirt`; rewriting it as
`.Values.kubevirt.migrations` turns the null-block case red with a nil
pointer.
Feature summary, unchanged from #4254: cluster-wide migration settings
could only be applied by patching the KubeVirt CR, because the kubevirt
chart did not render `migrations` and the platform bundle did not
forward it. `kubevirt.migrations` now passes through the generated
Package to `spec.configuration.migrations`. A hand patch of that Package
is undone on the next platform render, so this hop is the only supported
setter.
```yaml
kubevirt:
migrations:
bandwidthPerMigration: 625M
parallelMigrationsPerCluster: 2
parallelOutboundMigrationsPerNode: 1
```
### Screenshots
Not a UI change.
### Downstream repositories
Walked the trigger map against the diff: platform values, the iaas
bundle and the kubevirt chart. The v1.6 reference page of the platform
values had no `kubevirt` section, so the row goes there as a follow-up
that should land with this backport.
- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: cozystack/website#705
### Release note
```release-note
feat(kubevirt): expose `kubevirt.migrations` in the platform values and forward it to `spec.configuration.migrations` on the KubeVirt CR, so live-migration bandwidth and parallelism can be set without patching a Package that the platform re-renders.
```
# Description
Backport of #4253 to `release-1.6`, together with #3460 which it depends
on. On 1.6 kube-ovn chart and image still come from
cozystack/kubeovn-chart, so draft Dockerfile had nothing to build it and
kubeovn build failed (no `image` target). #3460 moves build in-tree on
same kube-ovn 1.15.10 this branch runs, and brings nightly-mirror fix
for kube-ovn split registry host that becomes needed once kube-ovn is
built by CI. Bot draft also dropped second commit of #4253
(`fix-vmim-target-pod-retry.diff`), it is back now.
I built image locally, all three patches apply to kube-ovn v1.15.10 and
`TestVMIMigrationSchedulingRetry` passes.
This reopens a fork backport from a branch in this repository so its CI
can run. The commit is the one from #4338 by @yankawai, unchanged. On
release-1.6 the pull-request CI pushes the images it builds, and a run
from a fork has no registry credentials, so #4338 stopped at the build
jobs and never reached e2e. The description below is the author's.
## What this PR does
Manual backport of #4292 to `release-1.6`.
The automated backport, #4320, stopped on a conflict in
`packages/system/linstor/values.yaml` and pushed the markers as its only
commit, so it is red on DCO and would ship conflict markers in a values
file if merged.
The conflict is `drbd.enabled`, which does not exist on this line: the
DRBD-less-substrate switch landed on main after 1.6 was cut, and the
cherry-pick pulled its values block in with the hunk it needed.
Everything else in the change applies unchanged, so three differences
follow from that one cause:
- the preStop hook is gated on `talos.gracefulShutdown.enabled` alone
rather than on it and `drbd.enabled`;
- the values block adds `talos.gracefulShutdown` only;
- the case asserting the hook is omitted on a Talos substrate without
DRBD goes with the flag it sets, and the README line that named both
flags names `talos.enabled`.
Verified on this branch: 11 Helm tests and the 20 behavioural Python
tests pass. The file set and line counts match #4292 apart from those
three points.
Feature summary, unchanged from #4292: Cozystack's Talos configuration
removes the systemd DRBD shutdown guard without a replacement.
`talos.gracefulShutdown.enabled`, disabled by default, releases unused
Secondary DRBD resources before the satellite stops so their backing
devices, including ZFS zvols, can be released during normal shutdown.
The hook skips Primary resources and open devices, rechecks each
candidate, and stops on ambiguous status, command failure or timeout. It
never forces, demotes, calls the Kubernetes API or asks for extra
permissions. It is opt-in because preStop also runs on ordinary
satellite restarts, and Piraeus has one DaemonSet per node with no
cluster-wide rollout barrier.
### Screenshots
Not a UI change.
### Downstream repositories
Walked the trigger map against the diff: this touches one system chart,
its hook script and its tests. No downstream repository restates the
linstor chart values.
- [x] No downstream repository is affected by this change
### Release note
```release-note
fix(linstor): add `talos.gracefulShutdown.enabled`, off by default, so a satellite releases unused Secondary DRBD resources before it stops and their backing devices, ZFS zvols included, can be released during a normal Talos shutdown. Enable it only where satellite and node restarts are serialized: the hook also runs on ordinary pod restarts, and Piraeus has one DaemonSet per node.
```
The CAPK build suite came over from main asserting that an image build
under OCI_EXPORT_DIR writes an OCI archive with --push=0. That export
path lives in hack/common-envs.mk on main and was never brought to this
branch, where every build pushes, so the suite failed here on every
run.
What the suite is for still holds on this branch: the build stamps a
digest-pinned ref into the .tag file that the image-ref tooling finds,
it leaves the shipped components untouched, and a failed build or a
missing digest keeps the previous pin. Those checks stay; the OCI
parameter and the three archive checks go.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
CAPK v0.1.10 pins kubevirt.io/api v1.5.0, whose
VirtualMachineInstanceMigrationSpec carries only vmiName; upstream #392
sets priority against v1.9.0. The typed migration does not compile against
the pinned API, which is why the patch builds it unstructured.
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
(cherry picked from commit bc2ba106e1)
hack/bats-no-exit-trap.bats is an exact ratchet, and the new suite installs
three traps inside explicit subshells without declaring them, so
make unit-tests fails on the merge commit. The header now declares the
count and says why the three are not debt.
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
(cherry picked from commit 0e4d6a305c)
A migration that KubeVirt never processes must not block host eviction indefinitely. Keep its submission deadline separate from active migration and fallback drain windows.
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
(cherry picked from commit c85f35070c)
Backport of #4254 to release-1.6.
Cluster-wide migration settings could only be applied by patching the KubeVirt
CR: the kubevirt chart did not render `migrations`, and the platform bundle did
not forward it. The optional `kubevirt.migrations` platform value now passes
through the generated Package to `spec.configuration.migrations`, and a hand
patch of that Package is undone on the next platform render, so this hop is the
only supported setter.
Four differences from main, all from release-1.6 predating the
disabledFeatureGates work:
- platform values gains a `kubevirt` block holding `migrations` alone, and the
iaas bundle gains only the migrations hop;
- the wiring test takes the four migrations cases and the case for a null
`kubevirt` block, without the assertions that belong to disabledFeatureGates
cases absent here;
- update_idempotency_test.sh gains `run_update_logged`, which arrived on main
alongside that work and the new cases need;
- the kubevirt Makefile header counts six sed patches rather than five.
Verified on this branch: 9 kubevirt chart tests, 31 platform wiring tests (102
across the platform suite), and update_idempotency_test PASS under GNU make and
sed. Removing the new awk guard from the Makefile turns that test red naming the
migrations block, so the ported guard is not decorative. On this line
(.Values.kubevirt).migrations is the only reader of .Values.kubevirt, and
rewriting it as .Values.kubevirt.migrations turns the null-block case red with a
nil pointer.
The automated backport in #4322 stopped with conflict markers in four files as
its only commit, which is why this exists.
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
Backport of #4292 to release-1.6.
Cozystack's Talos configuration removes the systemd DRBD shutdown guard
without a replacement. talos.gracefulShutdown.enabled, disabled by default,
releases unused Secondary DRBD resources before the satellite stops so their
backing devices, including ZFS zvols, can be released during normal shutdown.
The hook skips Primary resources and open devices, rechecks each candidate,
and stops on ambiguous status, command failure or timeout. It uses no force,
demotion, Kubernetes API calls or extra permissions.
Three differences from main, all from the same cause: release-1.6 has no
drbd.enabled flag, since the DRBD-less-substrate switch landed after 1.6 was
cut. The hook is therefore gated on talos.gracefulShutdown.enabled alone, the
values block adds only that key, and the case asserting the hook is omitted on
a substrate without DRBD is left out along with the flag it sets. The README
line naming both flags names talos.enabled instead. Nothing else differs.
The automated backport in #4320 stopped at the same values.yaml hunk and
pushed the conflict markers as its only commit, which is why this exists.
Verified on this branch: 11 Helm tests and the 20 behavioural Python tests
pass.
Assisted-by: LLM
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The nightly host rewrite is a literal "<src-registry>/" substring
replace, so it only reaches a host sitting contiguously in front of the
repository. kubeovn keeps its host in global.registry.address with the
bare repository in a sibling key: no trailing slash, no match. The
image was mirrored to the public registry and the tree kept pointing at
the private build registry.
The gap was dormant while kubeovn's image came from a separate
repository, because the baked ref carried the public host and the
ownership filter skipped it. Building the image in-tree stamps the
build-registry host into that key and turns it live, so fix it here.
Add an end-of-line-anchored expression that fires only when the source
registry is the complete value; a contiguous ref continues past the
host, so the two expressions cannot both fire on one reference.
keycloak-operator splits at a different boundary and stays unfixed --
neither of its keys holds the source registry as a whole. Documented,
and dormant while no CI path rebuilds it.
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
(cherry picked from commit 6cb579338b)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
kubeovn was the last package whose chart and image were produced by a
separate repository: packages/system/kubeovn pulled a tarball from the
latest cozystack/kubeovn-chart tag at update time, and that repository
built and pushed the image. The wrapper existed only to cache the
upstream chart; with the current caching that reason is gone, and the
split cost a second repository, a second release step, and a package
whose provenance a reader could not follow without leaving the tree.
Move the wrapper in, shaped like packages/system/cilium: update pulls
the upstream kube-ovn chart directly and applies patches/mtu.diff, and
image builds images/kubeovn and pins the digest into values.yaml. The
MTU patch already carried monorepo-relative paths, so it applies here
unchanged.
Also carries cozystack/kubeovn-chart#5, which never got a tag: pin
OVN_IPSEC_KEY_DIR under /var/lib. The kube-ovn default sits in
/etc/origin, which a Talos node cannot create, so enabling IPsec left
the node without ESP security associations.
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
Co-authored-by: Kirill Ilin <stitch14@yandex.ru>
Co-authored-by: Aleksei Sviridkin <f@lex.la>
Assisted-By: Claude
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
(cherry picked from commit 7ebce2e942)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
# Description
Backport of #4333 to `release-1.6`, together with #4231 which it depends
on. 1.6 has no in-tree opensearch-operator build, so bot draft added
only the Dockerfile and patch from #4333 would never reach an image.
#4231 brings the build (`image` target, values wiring, root `Makefile`
entry) plus its own single-replica fix. In root `Makefile` I kept only
opensearch-operator line, redis-operator is not built in-tree on 1.6.
I built image locally, both patches apply to operator v2.8.0 and stall
watchdog specs from #4333 pass.
Backport of #3742 to `release-1.6`.
`TestWaitForCRDsEstablished_timeout` fails `Unit & controller tests` on
this branch at random, last time on #4402: `error should mention stuck
CRD name, got: context cancelled while waiting for CRDs to be
established: context deadline exceeded`. `WaitForCRDsEstablished` here
still has two cancellation exits with different messages, and when the
deadline and a ticker tick are ready together the select picks one of
them at random. I ran the timeout test as 16 concurrent processes to get
the timing jitter CI has: 3 failures out of 160 runs on the current
branch, 0 out of 320 with this change.
Cherry-picked clean with `-x`.
### Testing
- stress run above, plus `make unit-tests` and `make test-controllers`
green, POSIX sh sweep clean.
```release-note
fix(manifestutil): a CRD wait cut short by a cancelled context now reports one error naming the CRD it was waiting on, where before the name was present or missing depending on which of two exits the scheduler happened to pick
```
WaitForCRDsEstablished reported a cancelled context two different ways.
The check at the top of the poll loop returned a message with no CRD
name; the select at the bottom named the CRD the poll had stopped on.
Which one ran was left to the scheduler: when the deadline and a tick
come ready in the same instant, a select picks between the ready cases
at random, so one cancellation reached the operator either way.
Build the message in one place and return it from both exits, so the two
are indistinguishable by construction. The name is carried across
iterations, since a cancellation noticed at the top of the loop has no
poll of its own to take one from, and starts as the first CRD of the set
for a wait cancelled before any poll observed anything.
Keeping the check at the top is what makes carrying the name worthwhile.
A client refuses Get on a dead context at the first name in the list
whichever CRD is actually lagging, so a loop that re-entered and polled
anyway would overwrite the name its last poll settled on. The check
closes that window and no more than that: a context dying part-way
through a poll still records the first Get to fail, because this branch
cannot tell a refused request from a CRD status.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
(cherry picked from commit 5747747087)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Backport of #3938 and #4280 to `release-1.6`, together, because #3938
alone breaks every Kafka deletion (#4276) and #4280 is what fixes that.
Kafka suite on this branch fails the delete step on 2 of the last 3 E2E
runs on the container lane (#4328, #4377), both times with
`UninstallFailed ... KafkaTopic/tenant-test/kafka-test-test-results
termination timeout: context deadline exceeded`. Helm removes the topic
operator together with the release, a KafkaTopic that still carries the
`strimzi.io/topic-operator` finalizer then has nobody to release it, so
uninstall waits out its timeout and leaves the topic behind, which is
what #3793 hits on reinstall. When the operator wins the race the same
delete takes 3s. The pre-delete hook from #3938 deletes the release
topics while the operator is still running, #4280 makes it actually
reach the apiserver and reaps its credentials.
All ten commits cherry-picked clean with `-x`. Kafka on this branch is
still ZooKeeper-based, the hook does not depend on that: it selects
topics by `strimzi.io/cluster=<release>`, the label the 1.6 chart puts
on them.
### Testing
- `make unit-tests` green (kafka helm unit tests and
`hack/kafka-pre-delete-hook.bats` included), POSIX sh sweep clean.
- E2E on this PR runs the kafka suite, which now asserts both release
topics are gone after deletion.
```release-note
fix(kafka): delete release-owned KafkaTopics before removing the Strimzi topic operator
```
getWorkloadMetadata filled the workloads.cozystack.io/kubevirt-vmi-instance-profile
label from the annotation kubevirt.io/cluster-instanceprofile-name, a key KubeVirt
never stamps. KubeVirt records the cluster preference (which carries the guest OS
profile) under kubevirt.io/cluster-preference-name, so the profile label was never
emitted and a VM's instance profile never reached the billing metric. The sibling
instance-type label worked only because its key is the real one.
Read the correct annotation. The emitted label name is unchanged, so the downstream
metric pipeline is untouched.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 1f9bad0643)
The policies are right and nothing here changes them. What is missing is
the reason, and this file has now been changed in both directions inside
a week by people reasoning from the annotations alone, so the reason is
the part worth keeping.
The Job carries no hook-succeeded because the script exits 0 on the
degraded path as well as the clean one, and its log is the only record
that a finalizer outlived the wait; ttlSecondsAfterFinished reaps it on a
timescale a human can act within. The ServiceAccount, Role and
RoleBinding have no such clock, and before-hook-creation fires on a next
activation that after the last uninstall never comes, so without a
terminal policy they sit in the tenant namespace for the life of the
cluster still granting delete over every KafkaTopic in it.
Reaping them cannot strand a running pod, which is the objection the
symmetrical shape was written against: helm applies hook-succeeded only
once the hook is terminal, and activeDeadlineSeconds ends the Job inside
the uninstall hook timeout. hook-failed would decide nothing, because a
failing hook already reaps the earlier successful ones under
hook-succeeded.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 3a43bcf391)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
A non-zero --request-timeout makes kubectl skip its in-cluster credentials
altogether. client-go consults them only while the assembled config still
equals the default, and the flag lands in that config, so the branch that
would load the ServiceAccount token never runs. kubectl falls back to its
built-in default cluster, localhost:8080, is refused, and the script reads
the refusal as a failed delete and exits 1. A non-zero pre-delete hook
returns from Uninstall.Run before any resource is removed, so no Kafka
release could finish deleting and helm-controller retried forever.
Measured on the pinned image rather than argued from the docs: with the
token and ca.crt mounted and KUBERNETES_SERVICE_HOST set, the same delete
reaches the cluster without the flag, goes to localhost:8080 with
--request-timeout=20s, goes to localhost:8080 with 1m, and reaches the
cluster again with 0.
The three in-tree hooks that work do not pass it. apps/kubernetes names it
in a comment only, apps/harbor and apps/mariadb never mention it, and
apps/tenant records that its DELETE calls "carry kubectl's default
--request-timeout of 0". The watch stays bounded by --timeout, and the Job
by activeDeadlineSeconds, which is that same shape.
The suite now rejects the flag. It needs the "=" because the explanation
in the script names the flag too, and matching the bare name would fail on
the comment rather than on a use.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit e63b8595d3)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The Job carries no hook-succeeded, so it outlives the hook run and
ttlSecondsAfterFinished reclaims it, which is what leaves a readable
record when the finalizer outlives the wait. Its ServiceAccount, Role and
RoleBinding still carried hook-succeeded and hook-failed, so Helm reaped
them at the end of the run either way and left the Job behind them.
A Job whose ServiceAccount is gone still starts a pod. kubelet mounts the
projected volume without a token, client-go finds no in-cluster config,
kubectl falls back to localhost:8080, and the script takes its catch-all
branch and exits 1. A non-zero pre-delete hook returns from Uninstall.Run
before any resource goes, so the release never finishes deleting and
helm-controller retries it: the e2e run on the merge commit shows six
such pods in five minutes, in two namespaces, every one of them dead the
same way. The hook meant to prevent one stuck Kafka release was instead
stopping all of them.
Give all four objects the same before-hook-creation policy, so the
credentials live exactly as long as the consumer and the next run
replaces them. Both other in-tree hooks that reach the apiserver,
apps/kubernetes and apps/harbor, are symmetric this way already.
The suite now pins the policy on each of the three, because nothing
compared them: the asymmetry rendered cleanly, passed 37 helm-unittest
cases and 5 bats cases, and only a cluster showed it.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 64d5689434)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Both suites matched "strimzi.io/cluster=kafka-test" as a bare substring,
and that is a prefix of the sibling release kafka-test2 and of
kafka-test-topics. Rendering the hook with either selector left both
suites green, so neither pinned the scoping they exist to pin. Anchor on
the leading "-l " and on the trailing delimiter; both variants now red in
both suites, and the six earlier hook mutations still red exactly where
they did.
Drop the notMatchRegex on a leading slash. It cannot fail on its own: the
anchored image pattern above rejects a leading slash first, and an empty
_cluster.images-registry renders no slash to begin with. Its rationale
moves into that pattern's comment.
Record why activeDeadlineSeconds and the 180s wait have to move together.
A deadline that fires mid-wait kills the Job, and with backoffLimit: 0 the
hook then returns non-zero, which is the wedged uninstall the deliberate
exit 0 on the timeout branch exists to avoid.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 637098f222)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The suite used bats' run helper. make bats-unit-tests does not invoke the bats
binary: it runs hack/cozytest.sh over every hack/*.bats, and that runner
implements @test and little else, so the file died at 'run: command not found'
and took make unit-tests with it. Under bats it was 5/5, which is where the 5/5
in the previous commit came from. Save $? by hand instead, the shape
app-cleanup-hooks_test.bats already uses; verified under both runners.
The fake kubectl no longer defaults DELETE_RC to 0. An unset value made every
failure-path assertion vacuous; now it exits 64 and says so. Cover both
RESTMapper wordings for an absent CRD rather than just the first.
Widen the image pattern to v[0-9]+\.[0-9]+(\.[0-9]+)? like harbor and mariadb:
pinning v1\.32 reintroduced the breakage the digest was made pattern-based to
avoid, one Renovate bump later. Assert the _cluster.images-registry override
and the no-leading-slash default, neither of which was covered: the hook is the
one image in this chart whose pull failure surfaces at uninstall.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 415aa82409)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The hook's exit code is the whole contract: a non-zero one returns from
Uninstall.Run before any release resource goes, keeping the topic operator
alive; a zero one lets Helm remove it. delete_test.yaml matches manifest text,
so it stayed green when the timeout branch was put back to exit 1, when the
CRD-absent branch was, and when the selector was widened past this release.
Add a bats suite in the shape hack/app-cleanup-hooks_test.bats already uses for
qdrant and harbor: render the script, run it against a fake kubectl, pin the
exit code of each branch. Each of the three mutations above now turns it red.
Drop hook-succeeded from the delete policy and reclaim the Job with
ttlSecondsAfterFinished instead. The script exits 0 on the degraded path too,
where topics still hold the finalizer, so deleting on success removed the only
record of that seconds after it was written. Assert both in delete_test.yaml.
Pin the hook image by shape rather than by literal digest: Renovate rewrites
digests under packages/*/templates/hooks/ and nothing rewrites tests/, so the
literal turns the next bump red for no defect.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 3326e0fb9e)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The post-delete absence assertions could pass vacuously if the two
release topics were never created. Asserting both topics first makes
the E2E prove that the pre-delete cleanup actually removed them.
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>
(cherry picked from commit 9d2fddd4e6)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
cloudflareProxy passed the Cloudflare ranges to the controller under
set_real_ip_from, which is an nginx directive and not an ingress-nginx
ConfigMap key, so the controller dropped it without a warning. With
use-forwarded-headers on, the realip module then trusted every source
from the default proxy-real-ip-cidr of 0.0.0.0/0, and any client that
reached the load balancer directly could set CF-Connecting-IP to an
address of its choice. That address is what whitelist-source-range
checks, so the tenant whitelist, and any per-Ingress one, could be
walked past with one header.
proxy-real-ip-cidr is the key the controller reads for the trusted
ranges.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit 3800f63513)
The cluster controller runs nine component reconcilers in sequence and none
of them logs unconditionally when it is entered: the TLS one's only line
above its work sits on the no-security early return, which the chart in this
repository never renders, and the rolling restart one opens at V(1), off at
the verbosity the operator ships with. So a call that stops returning leaves
nothing behind at all. The operator log ends on whatever ran before it, the
object is silently no longer reconciled, and from the outside that is
indistinguishable from an operator with nothing to do. Two e2e runs ended
exactly that way, the operator going quiet at the moment the chart rewrote
the CR and never coming back: https://github.com/cozystack/cozystack/issues/4332.
Each component is now wrapped so a call still in flight after a minute
reports its own name and how long it has been running, and keeps reporting on
that interval while it is stuck. A single line dated at the start of a stall
does not answer the question a reader has five minutes in, which is whether
the call is still there. Nothing is written while a component behaves, so a
healthy cluster costs no extra lines.
The finalizer path is left alone: every DeleteResources at this version
returns immediately, so a watchdog there could only spawn goroutines for
calls that cannot block.
A diagnostic rather than a fix: drop it once the stall itself is understood.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit 6a34c9f037)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The operator deletes the bootstrap pod as soon as initialization completes and
never looks at the voting configuration first. With one cluster_manager replica
there are exactly two eligible nodes, OpenSearch keeps the configuration at an
odd size of one, and the voter it keeps is the bootstrap node that formed the
cluster. Deleting that pod takes away the only vote there was, and the survivor
can never hold an election again, because changing the configuration itself
needs a quorum. The cluster is unusable about a minute after it installs, and a
node with no cluster-manager still answers the readiness probe, so nothing says
so until something restarts the pod.
Build the operator in-tree, the way kamaji and redis-operator already are, and
carry a patch that excludes the bootstrap node from the voting configuration and
waits for that to commit before deleting the pod. Reaching the cluster is a
precondition rather than a nicety: every failure keeps the pod and requeues,
because a bootstrap pod that outlives its purpose costs a pod, while a cluster
that cannot elect does not recover without a human.
Deleting a pod is a request rather than a wait, so clearing the exclusion
afterwards can find the node still in the cluster and fail. That path requeues
and is retried once the pod is really gone, which keeps a stale entry out of the
capped exclusion list; clearing stays at the API default wait_for_removal=true
so it cannot re-admit a node some other operation is still draining.
Upstream carries the same defect, tracked as
opensearch-project/opensearch-k8s-operator#1448.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
(cherry picked from commit b222cb6580)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Backport of #4020 to `release-1.6`, without its product changes.
On this branch kubernetes-previous and kubernetes-latest failed on
tenant node-join in 8 of the last 11 E2E runs: tenant workers sit one
virtualization level too deep on the shared runners and get their CSR
signed after the window closes. #4020 fixed that on main by running the
merge-gating lane on Talos containers. This brings the same lane here,
but 1.6 is a patch line and nothing it ships should change, so the four
product changes #4020 carried are replaced by e2e-only steps. Each one
is gated on the container lane (`COZY_LINSTOR_DRBD_ENABLED=false`,
`COZY_E2E_STORAGE_CLASS=local`), QEMU path stays as it was.
- linstor `drbd.enabled`: post-install prep applies its own
`LinstorSatelliteConfiguration` `e2e-no-drbd` that deletes the
drbd-logger sidecar, then reads the live DaemonSets back. The name has
to sort after the chart's `cozystack-plunger` because piraeus merges
configurations by name.
- linstor `--strict-topology`: the last install test patches
csi-provisioner args onto the live `LinstorCluster` and waits for the
rollout, tenant suites and vminstance re-check the flag before they
import.
- kubevirt-cdi importer resources: same test patches
`spec.config.podResourceRequirements` on the `cdi` CR and waits for
CDIConfig to report 4Gi.
- vm-disk immediate binding: vminstance suite annotates the DataVolume
and PVC with `cdi.kubevirt.io/storage.bind.immediate.requested` on
`local`.
Only file outside `hack/` and workflows is
`packages/core/testing/Makefile`, the e2e sandbox driver, same as on
main. OIDC suites are folded into kubernetes-latest like on main.
`e2e-tag.yaml` (rc validation) stays on QEMU, also like main. Backup
preflight, unit-test parallelism and the node-join soft-red from #3932
are not included.
`run-kubernetes.sh` here is 1163 lines against 5782 at the base of
#4020, so the lane parts were ported by hand instead of backporting ~20
diagnostics PRs first. Adapted commits say what was dropped.
### Testing
- `make unit-tests` green, POSIX sh sweep clean.
- Real check is E2E on this PR: install log must show the `e2e-no-drbd`
apply and the strict-topology and CDI patches, and kubernetes-latest,
kubernetes-previous and vminstance must be green.
One assumption: nothing upgrades linstor or kubevirt-cdi after the last
install test, otherwise the live patches get re-rendered away. If that
happens the tenant suites fail on the strict-topology check by name.
```release-note
NONE
```
Backport of #3891 to `release-1.6`.
`hack/e2e-prepare-cluster.bats` on this branch writes the Talos secrets
bundle, machine configs, kubeconfig, boot image and `srv*/` dirs to the
repo root when the sandbox is brought up outside CI, and nothing in
`.gitignore` matches them, so `git add -A` after a local e2e run picks
up `secrets.yaml` with the cluster private keys.
`hack/run-kubernetes-schedulable_test.bats` from #4435 also leaves
`tenantkubeconfig-test-latest-version` in the root on every `make
unit-tests`.
Cherry-picked clean with `-x`, `.gitignore` only.
```release-note
NONE
```
The container lane moves out of "In-flight direction (not yet the merged
standard)" and into the body, because it is now what both merge-gating
lanes run. The node-join section is rewritten the other way round: it
described a deadline that failed the suite without blocking the lane,
and there is no longer any such carve-out to describe.
What the section gains is the part that was missing while the lane was a
spike -- what moves off the per-PR path with the substrate, and where it
still gets covered. DRBD and the replicated StorageClass were already
named; Immediate binding, tenant StorageClass propagation and the
Cozystack Talos node image were not, and all three now say nightly and
release-candidate rather than reading as lost. The host CPU and memory
visibility gap is stated too, since it is what a reader meets first on a
red run and it looks like a product bug.
Adapted for release-1.6: this branch never had the node-join soft-red
carve-out, so there is no section to rewrite for it, and nightly.yaml
does not run from here, so the QEMU coverage the section points to is
e2e-tag.yaml alone. The section also describes the e2e-side settings
that stand in for chart values this branch does not have -- the
no-drbd satellite configuration, CSI strict topology, the CDI worker
memory ceiling and immediate binding for VM disks -- and it lands after
them for that reason.
(cherry picked from commit 12157a2f93)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-by: LLM
The vminstance suite waits for each VMDisk's DataVolume to be Ready
before anything consumes it. On `replicated` that works because the
class binds Immediate; on the container lane's `local` class, which is
WaitForFirstConsumer, CDI starts no importer for a disk nothing has
consumed yet, so the DataVolume sits in PendingPopulation and the suite
times out with no error anywhere. main makes the vm-disk chart request
immediate binding for every source; the chart on this branch requests it
only for uploads, and the lane must not change what it ships.
So on the container lane, right after each VMDisk is applied, the suite
waits up to 180s for CDI to create the DataVolume and its claim and
annotates both with cdi.kubevirt.io/storage.bind.immediate.requested,
the same request the chart already writes for uploads. Both, because
the DataVolume decides what phase CDI reports and the claim is what the
populator reads to start the import without a consumer. The QEMU lane
annotates nothing.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-by: LLM
IvanHunters