Commit Graph
21 Commits
Author SHA1 Message Date
Aleksei Sviridkin 520af57905 chore: drop private cluster, host and personal names from the tree
Private infrastructure names, a real hostname, a commented-out
internal mirror block, an internal product name, and a reviewer's
name in a comment do not belong in a public repository; drop them
and replace what needs a stand-in with neutral placeholders.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-10-07 04:38:06 +03:00
Aleksei Sviridkin 458827ccdd test(e2e): load drbd_transport_tcp explicitly on Talos nodes
Since v1.14.0 Talos builds the kernel with an empty modprobe path
(siderolabs/pkgs#1565), so request_module() no longer loads anything.
DRBD asks for its transport that way on the first new-peer, and
without the module drbdsetup new-peer fails with "Failed to create
transport (drbd_transport_xxx module missing?)" and every resource on
the node stays in Connecting. The module ships in the drbd extension
on v1.13 and v1.14 alike, so listing it next to drbd is safe on both.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-30 04:23:39 +03:00
IvanHunters cd72d81bb4 Merge branch 'main' into feat/kubernetes-nodes-phase2b
Reconcile the phase-2b split with main's configurable worker-health
feature (MHC condition timeouts, maxNodeProvisionTime, podCpuLimit/
podCpuRequest), which landed on both charts after this branch forked:

- kubernetes (parent): pool rendering stays removed; main's parent
  worker-pool additions do not apply post-split.
- kubernetes-nodes (pool): absorb the feature; keep the directional
  parent-version skew guard and the immutable cluster/storageClass rules.
  KubevirtMachineTemplate content hashes are unchanged (render snapshot).
- migration 54: carry podCpuLimit/podCpuRequest (KMT-hash inputs) and the
  cluster-wide readyUnknownTimeout/readyFalseTimeout/maxNodeProvisionTime
  into adopted pools, so adoption neither rolls nor resets a tuned pool.
  Covered by a new adopt-values bats case.
- drop the parent chart's obsolete MHC-timeout suites; the
  kubernetes-nodes twins cover the feature.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-08-19 13:14:11 +03:00
Aleksei Sviridkin b1b8cbecc2 test(e2e): price the layers above the sandbox on the node-join window
Every CPU reading on this failure path has a subject inside the sandbox:
a cgroup, a tenant worker's threads, a sandbox node's own /proc/stat.
Each of them can say a guest was given its ticks while getting no work
done, and none can say what the layers above spent. That is where this
suite's own tax is paid: the sandbox nodes' guest time is charged to the
runner kernel as user time, and whatever the runner VM loses to the
machine hosting it is charged to nobody the sandbox can see.

Two readings close that. /proc/stat is not namespaced, so reading it from
the sandbox container reads the runner VM's kernel -- the only place
steal on that layer appears, which is what separates a slow suite from a
machine sharing a core with somebody else. The sandbox VMs are QEMU
processes in this container's own PID namespace, so their threads are a
local read too, and they say which node spent the kernel's user time and
whether the three spent it evenly.

Both are taken at the same two points as the KVM counters and for the
same reason: the values are cumulative, so a rate needs a pair, and the
interval worth measuring is the window the guest is failing to boot in.
The thread reading runs the probe the worker capture already uses rather
than a copy of it, since a second copy would drift exactly at the
failure paths.

Each carries the layer it describes in its own legend, because three
kernels in this report publish a /proc/stat and two captures read QEMU
threads, one layer apart in each direction -- and a reader who takes one
for the other compares a hypervisor against its own guest.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-08-18 07:25:49 +03:00
IvanHunters 64815cf12b Merge remote-tracking branch 'origin/main' into pr-3315-lgtm 2026-08-12 11:01:04 +03:00
Aleksei Sviridkin 9ace63b441 fix(e2e): stop the sandbox cluster CIDR overlapping the pod range
The sandbox Talos config and the platform install step both used
10.244.0.0/16, one as the Kubernetes cluster CIDR and the other as the
kube-ovn POD_CIDR, so two allocators drew from one range.

kube-ovn owns pod IPAM and allocates flat from POD_CIDR. Cilium is
chained behind it and allocates no pod addresses, but it still carves
its own per-node router IP and Ingress IP out of Node.spec.podCIDR,
which kube-controller-manager slices from the cluster CIDR. Sooner or
later kube-ovn hands a pod an address cilium already holds. When that
address is the Ingress IP the agent refuses the sandbox with "IP
ipv4:<addr> is already in use", and cilium never relents on it, so the
pod only proceeds if something reschedules it onto a different address.
In the failure this comes from, the same address was refused seven
times across six pod UIDs before the suite ran out of budget. When it
is the router IP nothing complains at all and only host-to-pod traffic
breaks.

Move the cluster CIDR to 100.65.0.0/16, which no other range in the
tree uses. That deliberately stops the sandbox matching the layout the
install documentation prescribes, so it is not a value to put back.
Pod addresses are unaffected: they still come from kube-ovn out of
10.244.0.0/16, and cilium does not require a chained endpoint's address
to sit inside its own allocation CIDR.

Excluding the addresses on the kube-ovn side would not work, because
cilium picks its offsets by random scan over the node's range.

The new guard checks the invariant rather than the literals, so either
value may move as long as they stay disjoint. It covers the service
and join ranges too, since landing on one of those instead is the next
way to get this wrong.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <3811295@gmail.com>
2026-08-11 02:31:27 +03:00
IvanHunters 847c83a2f6 fix(e2e): boot CI QEMU guests with cache=unsafe to keep etcd unstalled
The ephemeral CI VM disks ran with QEMU's default cache=writeback, so
every guest fsync propagated to the host-backed raw files. During the
VM-heavy suites the management-cluster etcd fsync on each Raft commit
serialized behind the concurrent image-unpack and CDI import I/O, the
apiserver returned context-deadline-exceeded, and operators lost their
leader-election leases together, failing a different suite each run.
The VMs are recreated per run, so the traded-away durability is unused.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
2026-07-28 20:45:51 +03:00
Andrei Kvapil ad35afa534 fix(e2e): disable guest fsync on ephemeral CI VM disks
The e2e QEMU guests boot with three raw virtio -drive lines that carry
no cache= option, so QEMU defaults to cache=writeback and honors every
guest fsync. The management-cluster Talos etcd fsyncs on each Raft
commit, while containerd unpacks dozens of images and CDI imports
multi-GB VM disks onto the same host-backed raw files during the
VM-heavy suites. The fsync storm serializes etcd's writes into
multi-second stalls, the mgmt apiserver times out lease PUTs, and many
operators (capk, etcd-operator, fdb-operator, cilium-operator, kamaji)
lose leadership at once — failing whichever chainsaw suite is
mid-flight, a different one each run.

Add cache=unsafe to each guest drive so guest flushes become no-ops
against the host page cache, removing the fsync-gated etcd stalls. This
is safe only because these CI VMs are ephemeral and destroyed every
run; never use it for real data.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-07-28 17:48:40 +02:00
Aleksei Sviridkin c7bd3bcf2b test(e2e): widen etcd convergence budget to 5m in Talos bootstrap
The "Bootstrap Talos cluster" step times out the etcd-members health gate
at 180s. A 3-node etcd converges via Talos's serialized learner promotion
(one learner admitted at a time), so on a loaded ephemeral runner this
deterministic path can approach ~3m wall clock and photo-finish against the
180s ceiling — failing a cluster that is in fact healthy. The wait is
already event-driven; only the ceiling was too tight. Widen it to 300s.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-24 22:10:11 +03:00
Aleksei Sviridkin c3ebdb310e docs(tests): correct the e2e kube-version pin rationale for talosctl 1.13.5
talosctl 1.13.5 now defaults to Kubernetes 1.36.2, so the comment's premise
— that talosctl's bundled default predates the KCM VAP nil-pointer fix — no
longer holds (1.36.2 already carries that fix). The reason e2e pins v1.33.12
is now that 1.36 is outside cozystack's supported range; reword to lead with
that and keep the KCM VAP fix as the secondary property v1.33.12 carries.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-06-23 05:41:03 +03:00
Myasnikov DaniilandClaude Opus 4.8 96b9f5eac6 test(e2e): pin Kubernetes to v1.33.12 to avoid KCM VAP type-checker panic
The e2e cluster bootstrap let `talosctl gen config` pick its bundled
default Kubernetes version (v1.33.1 for Talos v1.13.0). That version
predates the fix for the kube-controller-manager ValidatingAdmissionPolicy
status type-checker nil-pointer panic on `additionalProperties: true`
schemas (kubernetes/kubernetes#135155, backported to release-1.33 in
#136958, first released in v1.33.10). cozystack-api's aggregated Tenant
schema publishes `additionalProperties: true`, so on the older default
KCM crash-loops cluster-wide during install and fails E2E.

Pin the e2e management cluster to v1.33.12 (latest 1.33 patch) so it runs
a Kubernetes that contains the fix. This complements the schema-side fix
in #2867 and is independent of it.

Refs: #2863

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-06-10 20:22:07 +03:00
Andrei KvapilandClaude 4dc2367fe4 test(tests): exclude loop devices from host LVM scanning
Add an /etc/lvm/lvm.conf overwrite to the e2e Talos machine config so the
host LVM ignores /dev/loop devices (alongside drbd/dm/zd). This prevents the
host from scanning and activating volume groups inside loop-mounted disk
images used during e2e.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
2026-06-03 14:00:16 +03:00
Aleksei Sviridkin 8cae6f79d1 fix(tests): increase Bootstrap Talos cluster timeout to 60s
On fast ephemeral runners a freshly booted Talos node has not yet
converged NTP when the previous 10s outer timeout fired, so the
embedded 'until talosctl bootstrap; do sleep 1; done' retry loop only
got one 5-6s attempt before the loop was killed. Every RPC returned
'FailedPrecondition: time is not in sync yet' and E2E failed in the
Prepare environment step.

Bump the outer timeout to 60s and the inter-attempt sleep to 2s. This
aligns with the surrounding bootstrap gates (etcd members check at
180s, drain RPC errors at 60s, node registration at 60s) and gives
NTP plenty of time to converge on ephemeral runners.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-04-21 00:05:56 +03:00
Andrei Kvapil 505b693c35 [ci] Run e2e tests on shared runners
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2026-01-20 22:13:16 +01:00
nbykov0 c19cddf08e [core] rm talos lldp extension
Signed-off-by: nbykov0 <166552198+nbykov0@users.noreply.github.com>
2025-11-04 12:21:23 +03:00
Timofei Larkin 868148709c [talos] Add lldpd extension to image
This patch adds the lldpd extension to Cozystack's Talos build.
Additionally it changes the profile generation scripts to use
Siderolabs' recommended way to get appropriate extension image versions
to include with Talos.

Release note:
```release-note
[talos] Add LLDP support in default Talos build.
```

Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2025-08-26 11:10:23 +03:00
Andrei Kvapil 168a24ffdf [tests] Add more space for e2e tests
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-08-11 13:04:00 +03:00
Timofei Larkin 771fbc817f [ci] Stop using personal domain for CI
Migrate away from using a private domain for build infra.

Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2025-08-07 17:52:48 +03:00
Andrei Kvapil 52714f5cce [tests] Introduce self destructing environments
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
2025-07-02 03:42:14 +03:00
Timofei Larkin 4648c7b4c1 [ci] Use Nexus as a pull-through cache for CI
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2025-06-26 16:45:45 +03:00
Timofei Larkin 746641e523 Split testing job into several
This patch separates the Test job of the PR workflow into several
smaller jobs: 1) create a testing sandbox and deploy Talos, 2) install
Cozystack and configure it, 3) install managed applications and run e2e
tests. This lets developers shorten the feedback loop if tests are
merely acting flaky and aren't really broken. It's not the right way,
but it's 80/20.

Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
2025-06-17 18:47:09 +03:00