[Backport release-1.4] fix(platform): migrate ephemeralStorage to diskSize via pre-upgrade hook (#2712)

Add pre-upgrade migration 41 that walks all
kuberneteses.apps.cozystack.io Application CRs and renames
nodeGroups[*].ephemeralStorage to diskSize, preserving the user's value.

Without this migration, clusters upgraded after PR #2454 either fail to
reconcile (hard fail blocks all Flux operations) or silently lose the
user's disk size setting (default 20Gi replaces whatever was
configured).

The migration is idempotent and best-effort: a failed patch is logged
and leaves the version stamp at 41 for retry on next upgrade.

Update the chart-side guard error message to direct operators to the
migration job logs when the field appears post-upgrade (regression
detector). Bump migrations.targetVersion 41 -> 42.

Assisted-By: Claude <noreply@anthropic.com>

(cherry picked from commit ed1bb53d2c)

<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note

```

packages/apps/kubernetes/README.md

@@ -85,7 +85,7 @@ See the reference for components utilized in this service:
 
 ## Breaking Changes
 
-- **`ephemeralStorage` renamed to `diskSize`**: The `nodeGroups[name].ephemeralStorage` field has been renamed to `nodeGroups[name].diskSize` to better reflect its purpose (persistent disk for kubelet and containerd data). There is no backward-compatibility fallback; users MUST update their configurations to use `diskSize` instead of `ephemeralStorage`. If `ephemeralStorage` is still present in values, Helm template rendering will fail with an error directing you to use `diskSize`. When upgrading the CRD directly (bypassing Helm), the unrecognized field is silently dropped and kubelet storage reverts to the default 20Gi. Existing VMs will be automatically rolling-updated via CAPI when the new values are applied. State persists across same-VM reboots (virt-launcher restart, guest reboot, node failure); VM replacement by CAPI (e.g. nodeGroup field change, MachineHealthCheck remediation) provisions a fresh PVC.
+- **`ephemeralStorage` renamed to `diskSize`** (v1.4): The `nodeGroups[name].ephemeralStorage` field has been renamed to `nodeGroups[name].diskSize` to better reflect its purpose (persistent disk for kubelet and containerd data). Existing clusters are migrated transparently by platform migration 41 during the pre-upgrade hook — no manual action is required. Newly written values should use `diskSize`. Existing VMs will be automatically rolling-updated via CAPI when the new values are applied. State persists across same-VM reboots (virt-launcher restart, guest reboot, node failure); VM replacement by CAPI (e.g. nodeGroup field change, MachineHealthCheck remediation) provisions a fresh PVC.
 
 ## Parameters
 

packages/apps/kubernetes/templates/cluster.yaml

@@ -39,7 +39,7 @@ spec:
             cdi.kubevirt.io/storage.usePopulator: "false"
         spec:
           {{- if hasKey .group "ephemeralStorage" }}
-          {{- fail (printf "nodeGroup %q: ephemeralStorage is no longer supported. Rename it to diskSize. See README.md for migration instructions." .groupName) }}
+          {{- fail (printf "nodeGroup %q: ephemeralStorage is no longer supported and should have been automatically migrated to diskSize by platform migration 41. If you see this error after upgrading, the migration did not run — check the cozystack-migration-hook Job logs in cozy-system." .groupName) }}
           {{- end }}
           source:
             blank: {}

packages/apps/kubernetes/tests/cluster_test.yaml

@@ -351,7 +351,7 @@ tests:
       nodeGroups.md0.ephemeralStorage: 50Gi
     asserts:
       - failedTemplate:
-          errorMessage: 'nodeGroup "md0": ephemeralStorage is no longer supported. Rename it to diskSize. See README.md for migration instructions.'
+          errorMessage: 'nodeGroup "md0": ephemeralStorage is no longer supported and should have been automatically migrated to diskSize by platform migration 41. If you see this error after upgrading, the migration did not run — check the cozystack-migration-hook Job logs in cozy-system.'
 
   - it: fails when ephemeralStorage is set to empty string
     release:
@@ -361,7 +361,7 @@ tests:
       nodeGroups.md0.ephemeralStorage: ""
     asserts:
       - failedTemplate:
-          errorMessage: 'nodeGroup "md0": ephemeralStorage is no longer supported. Rename it to diskSize. See README.md for migration instructions.'
+          errorMessage: 'nodeGroup "md0": ephemeralStorage is no longer supported and should have been automatically migrated to diskSize by platform migration 41. If you see this error after upgrading, the migration did not run — check the cozystack-migration-hook Job logs in cozy-system.'
 
   - it: fails when both ephemeralStorage and diskSize are set
     release:
@@ -372,7 +372,7 @@ tests:
       nodeGroups.md0.diskSize: 50Gi
     asserts:
       - failedTemplate:
-          errorMessage: 'nodeGroup "md0": ephemeralStorage is no longer supported. Rename it to diskSize. See README.md for migration instructions.'
+          errorMessage: 'nodeGroup "md0": ephemeralStorage is no longer supported and should have been automatically migrated to diskSize by platform migration 41. If you see this error after upgrading, the migration did not run — check the cozystack-migration-hook Job logs in cozy-system.'
 
   ###############################################
   # KubeadmConfigTemplate — files block         #

packages/core/platform/images/migrations/migrations/41

@@ -0,0 +1,113 @@
+#!/bin/bash
+# Migration 41 --> 42
+# Rename nodeGroups[*].ephemeralStorage to nodeGroups[*].diskSize in every
+# kuberneteses.apps.cozystack.io Application CR.
+#
+# PR #2454 renamed the field in the kubernetes app chart with no fallback.
+# Without this migration, upgraded clusters either fail to reconcile (hard fail
+# behavior) or silently lose their disk size (silent-drop behavior).
+#
+# Idempotent: re-running on already-migrated values is a no-op
+# (has("ephemeralStorage") is false on the post-migration shape).
+# Best-effort: a failed patch is logged and leaves the version stamp at 41 so
+# the migration retries on the next platform upgrade.
+
+set -euo pipefail
+
+DRY_RUN="${MIGRATION_DRY_RUN:-0}"
+FAILURES=0
+PATCHED=0
+SKIPPED=0
+
+# Emit ephemeralStorage: null rather than dropping the key, because JSON merge
+# patch only deletes a field when an explicit null is present in the patch
+# document. A patch that simply omits a key leaves the existing key intact.
+JQ_TRANSFORM='
+  walk(
+    if type == "object" and has("nodeGroups") and (.nodeGroups | type) == "object" then
+      .nodeGroups |= with_entries(
+        .value |= (
+          if type == "object" and has("ephemeralStorage") then
+            (if has("diskSize") and (.diskSize // "") != "" then .
+             else .diskSize = .ephemeralStorage end)
+            | .ephemeralStorage = null
+          else . end
+        )
+      )
+    else . end
+  )
+'
+
+patch_object() {
+  local kind="$1" ns="$2" name="$3" selector="$4"
+  local obj current new patch
+
+  obj=$(kubectl get "$kind" --namespace "$ns" "$name" --output json 2>/dev/null || true)
+  if [ -z "$obj" ]; then SKIPPED=$((SKIPPED + 1)); return 0; fi
+
+  current=$(printf '%s' "$obj" | jq -c "$selector // null")
+  if [ "$current" = "null" ]; then SKIPPED=$((SKIPPED + 1)); return 0; fi
+
+  new=$(printf '%s' "$current" | jq -c "$JQ_TRANSFORM")
+  if [ "$current" = "$new" ]; then SKIPPED=$((SKIPPED + 1)); return 0; fi
+
+  patch=$(jq --null-input --argjson val "$new" --arg path "${selector#.}" '
+    def to_path: split(".");
+    {} | setpath($path | to_path; $val)
+  ')
+
+  if [ "$DRY_RUN" = "1" ]; then
+    echo "[dry-run] would patch $kind $ns/$name ($selector)"
+    diff <(printf '%s' "$current" | jq --sort-keys .) \
+         <(printf '%s' "$new"     | jq --sort-keys .) || true
+    PATCHED=$((PATCHED + 1))
+    return 0
+  fi
+
+  local kubectl_err
+  if kubectl_err=$(printf '%s' "$patch" | kubectl patch "$kind" --namespace "$ns" "$name" \
+       --type merge --patch-file /dev/stdin 2>&1 >/dev/null); then
+    echo "Patched $kind $ns/$name ($selector)"
+    PATCHED=$((PATCHED + 1))
+  else
+    echo "WARN: failed to patch $kind $ns/$name ($selector): ${kubectl_err:-unknown error}" >&2
+    FAILURES=$((FAILURES + 1))
+  fi
+}
+
+list_objects() {
+  local kind="$1" out
+  if ! out=$(kubectl get "$kind" --all-namespaces \
+       --output 'jsonpath={range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\n"}{end}' \
+       2>&1); then
+    case "$out" in
+      *"the server doesn't have a resource type"*|*"no matches for kind"*|*"NotFound"*) return 0 ;;
+    esac
+    echo "ERROR: kubectl get $kind failed: $out" >&2
+    return 1
+  fi
+  printf '%s' "$out"
+}
+
+echo "==> Migrating kuberneteses.apps.cozystack.io"
+items=$(list_objects kuberneteses.apps.cozystack.io) || exit 1
+while IFS=$'\t' read -r ns name; do
+  [ -z "$ns" ] && continue
+  patch_object kuberneteses.apps.cozystack.io "$ns" "$name" .spec
+done <<<"$items"
+
+echo "==> Summary: patched=$PATCHED skipped=$SKIPPED failures=$FAILURES" | tee /dev/stderr
+
+if [ "$DRY_RUN" = "1" ]; then
+  echo "Dry-run complete; version stamp not advanced"
+  exit 0
+fi
+
+if [ "$FAILURES" -gt 0 ]; then
+  echo "WARN: $FAILURES patch(es) failed; leaving CURRENT_VERSION at 41 for retry on next upgrade."
+  exit 0
+fi
+
+kubectl create configmap --namespace cozy-system cozystack-version \
+  --from-literal version=42 --dry-run=client --output yaml \
+  | kubectl apply --filename -

packages/core/platform/values.yaml

@@ -6,7 +6,7 @@ sourceRef:
 migrations:
   enabled: false
   image: ghcr.io/cozystack/cozystack/platform-migrations:v1.4.0-rc.2@sha256:17390197a9b11d76a8dd1b24c6d1512adb7d3226b28b5bcba8551b14b1ec4be0
-  targetVersion: 40
+  targetVersion: 42
 # Bundle deployment configuration
 bundles:
   system:

packages/core/testing/images/e2e-sandbox/Dockerfile

@@ -19,8 +19,8 @@ RUN curl -sSL "https://github.com/mikefarah/yq/releases/download/v4.44.3/yq_${TA
  && chmod +x /usr/local/bin/yq
 RUN curl -sSL "https://fluxcd.io/install.sh" | bash
 RUN curl -sSL "https://github.com/cozystack/cozyhr/raw/refs/heads/main/hack/install.sh" | sh -s -- -v "${COZYHR_VERSION}"
-RUN curl https://dl.min.io/client/mc/release/${TARGETOS}-${TARGETARCH}/mc --create-dirs -o  /usr/local/bin/mc \
-&& chmod +x /usr/local/bin/mc
+RUN curl -fsSL "https://dl.min.io/client/mc/release/${TARGETOS}-${TARGETARCH}/mc" --create-dirs -o /usr/local/bin/mc \
+ && chmod +x /usr/local/bin/mc
 RUN <<'EOF'
 cat <<'EOT' >> /etc/bash.bashrc
 . /etc/bash_completion