mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-02 18:16:53 +09:00
test(kubernetes): name the Proxmox fixture's node pool <cluster>-<pool> (#5000)
## What this PR does The Proxmox suite (`hack/e2e-chainsaw/kubernetes-proxmox/`) has now run for the first time on a real self-hosted stand: a dedicated Cozystack v1.7.0-rc.1 management cluster on Proxmox VE, with the runner CageForge offered to host in cozystack/community#98. The suite failed before it reached the hypervisor, because the fixture and the chart disagree on what a node pool may be called. `kubernetes-nodes` names its release `kubernetes-nodes-<CR name>` and, sincefd14126be, refuses to render one that does not start with `kubernetes-nodes-<spec.cluster>-` (`packages/apps/kubernetes-nodes/templates/_helpers.tpl`). The fixture, added later in7500a9fba, names its `KubernetesNodes` `md0`. On the stand that gave: ``` Helm install failed for release tenant-e2eproxmox/kubernetes-nodes-md0 ...: kubernetes-nodes: release name "kubernetes-nodes-md0" must start with "kubernetes-nodes-k8s-e2e-" — name the KubernetesNodes CR <cluster>-<pool> (cluster="k8s-e2e") ``` No `Machine` was ever created. The `tenant-and-cluster` step passed its `ProxmoxCluster` assert in 20 seconds and then waited out the 20-minute `Machine` assert. - The fixture names the pool `k8s-e2e-md0`. Nothing else in the suite, `run-proxmox.sh` or the workflow refers to the pool by name. - `hack/proxmox-e2e-fixture_test.bats` gets a case that reads `metadata.name` and `spec.cluster` of the fixture's `KubernetesNodes` and fails unless the name is `<cluster>-<pool>`. It is POSIX sh like the rest of the file, so `hack/cozytest.sh` can still run it. ### Testing - `bats hack/proxmox-e2e-fixture_test.bats` (bats 1.14.0, the CI commit): 7/7. The new case was checked to fail against the old fixture: `FAIL: KubernetesNodes md0 must be named k8s-e2e-<pool>`. - `hack/check-commit-trailers.sh upstream/main..HEAD`: OK. - Lane, before the fix (fork `main` @aed7fa896): https://github.com/themoriarti/cozystack/actions/runs/39402535098. Red on the `Machine` assert, as above; the teardown removed the tenant and left no VM on the hypervisor. - Lane, with this commit: https://github.com/themoriarti/cozystack/actions/runs/39442983145. The pool release `kubernetes-nodes-k8s-e2e-md0` installs, the `Machine` is created, capmox clones the worker from the template and boots it, and the worker joins the tenant and goes `Ready` (Talos v1.13.6). The run is still red, one step later and for a known reason: the tenant's CCM crash-loops on `pccm:v0.15.0` with `panic: version string "0.15.0" doesn't match expected regular expression: "^v(\d+\.\d+\.\d+)"`, so the node keeps `node.cloudprovider.kubernetes.io/uninitialized` and no providerID, and the `Machine` assert reports `Provisioned`, expected `Running`. That is #4917. The teardown removed the tenant, the worker VM and its cloud-init ISO. Two earlier runs on the same commit, 39406900563 and 39409765857, stopped at stand configuration rather than at this repository: capmox's token could not see an ISO storage, and the stand's `publishing.host` did not resolve from the fixture's public resolvers. Both are fixed on the stand. They are listed below because the lane does not say a stand needs them. ### Found on the way, not changed here - **What a stand must provide**, beyond the `PROXMOX_*` variables: - an `iso`-content storage on which capmox's token holds `Datastore.AllocateTemplate` (to upload the cloud-init ISO) and `Datastore.Allocate` (to delete it with the VM). go-proxmox uploads to the first `iso` storage the token can see; without delete rights the `Machine` never finishes deleting. - a `publishing.host` whose `<cluster>.<tenant>.<host>` resolves through the fixture's own resolvers (`8.8.8.8`, `1.1.1.1`) to a root ingress with SSL passthrough: on the Proxmox substrate that name is the tenant's `controlPlaneEndpoint`. - **`COZY_PVE_ADDRESSES/GATEWAY/PREFIX/BRIDGE/POOL`** are not passed by the workflow. Unset, they default to `vmbr50` and `10.0.0.180-185`. This stand sets them in the runner's environment. - **The fixture's `templateTags`** (`capmox-template, cozystack, talos, talos-v1.13.6`) are not the set that `hack/proxmox-talos-template.sh` writes (`cozystack, schematic-…, talos, talos-v…`). capmox matches tags as a set, so a template built by the script is not selected. This stand tags its template to match the fixture. - **Teardown reports success once the CRs are deleted.** It does not wait for the VMs to be gone; on this stand a worker VM outlived its `Machine` for hours while capmox retried a forbidden delete. The CCM fix itself is #4917. Once a release carries it, this lane should get past the `Machine` assert. The lane stays manual. It has no `push` or `pull_request` trigger, and its nightly schedule is a no-op unless `PROXMOX_NIGHTLY` is set, which it is not on this stand. The runner is registered to a fork whose only collaborator is its owner. Running it from cozystack/cozystack needs a runner group restricted to this workflow and a protected environment, which is a follow-up for community#98, not this PR. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change The diff touches one fixture under `hack/e2e-chainsaw/` and one bats file. It moves or renames nothing under `hack/`, changes no make target and no node prerequisite. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Updated the Kubernetes node pool fixture to use the required cluster-prefixed naming convention, improving compatibility with Kubernetes node validation. - **Tests** - Added validation to confirm that Kubernetes node fixtures include the required cluster reference and use the expected cluster-based name format. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
2 files changed
+19
-1
No files matched your search
@@ -65,7 +65,10 @@ spec:
|
||||
apiVersion: apps.cozystack.io/v1alpha1
|
||||
kind: KubernetesNodes
|
||||
metadata:
|
||||
name: md0
|
||||
# <cluster>-<pool>: kubernetes-nodes refuses to render a release whose name
|
||||
# does not start with kubernetes-nodes-<spec.cluster>-, so a bare "md0" never
|
||||
# creates a Machine and the suite only times out waiting for one.
|
||||
name: k8s-e2e-md0
|
||||
namespace: tenant-e2eproxmox
|
||||
spec:
|
||||
cluster: k8s-e2e
|
||||
|
||||
@@ -16,6 +16,21 @@ RUNNER="$REPO_ROOT/hack/e2e-chainsaw/_lib/run-proxmox.sh"
|
||||
if grep -qE '\$\{[A-Za-z_]+:[-=?+]' "$FIXTURE"; then echo "FAIL: the fixture must not carry a shell-style default envsubst leaves in place"; false; fi
|
||||
}
|
||||
|
||||
# kubernetes-nodes names its release kubernetes-nodes-<CR name> and refuses one
|
||||
# that does not start with kubernetes-nodes-<spec.cluster>- (_helpers.tpl), so a
|
||||
# pool named anything but <cluster>-<pool> fails at helm install and the suite
|
||||
# only learns it from a Machine that never appears.
|
||||
@test "the fixture's KubernetesNodes is named <cluster>-<pool>, as kubernetes-nodes requires" {
|
||||
doc="$(sed -n '/^kind: KubernetesNodes$/,/^---$/p' "$FIXTURE")"
|
||||
name="$(printf '%s\n' "$doc" | sed -n 's/^ name: *//p' | head -n 1 | tr -d '"')"
|
||||
cluster="$(printf '%s\n' "$doc" | sed -n 's/^ cluster: *//p' | head -n 1 | tr -d '"')"
|
||||
if [ -z "$name" ] || [ -z "$cluster" ]; then echo "FAIL: no KubernetesNodes name or spec.cluster in the fixture"; false; fi
|
||||
case "$name" in
|
||||
"$cluster"-?*) ;;
|
||||
*) echo "FAIL: KubernetesNodes $name must be named $cluster-<pool>"; false ;;
|
||||
esac
|
||||
}
|
||||
|
||||
@test "render-tenant expands every variable with the runner's defaults" {
|
||||
out="$(COZY_PVE_SSH=x COZY_PROXMOX_STORAGE=teststore "$RUNNER" render-tenant)"
|
||||
if printf '%s' "$out" | grep -q '\${'; then echo "FAIL: the rendered tenant must not keep an unexpanded variable"; false; fi
|
||||
|
||||
Reference in new issue
Block a user