fix(kubernetes): break bootstrap-token Job out of Helm hook to unblock fresh installs

The bootstrap-token-tenant Job was wired as a Helm post-install/post-upgrade
hook. post-* hooks fire only after Helm's wait phase resolves, and the tenant
kubernetes release wait blocks on MachineDeployment Ready, which blocks on
nodes registering, which needs the kubelet CSR path — for which the very
bootstrap-token this Job produces has to already exist in the tenant
kube-system. Fresh installs and any HR reconcile that finds MD in a
non-Ready state deadlock; HR.upgradeFailures climbs while post-upgrade
hooks never get their turn to run.

pre-install/pre-upgrade would not help either: they fire before non-hook
templates apply, so <release>-admin-kubeconfig (materialised by Kamaji
after KamajiControlPlane is applied) is not yet available.

Move the ServiceAccount and Job to the main install phase, matching what
talos-reconcile already does in this chart. Helm applies them together
with the KamajiControlPlane and talos-secrets; the Job Pod stays pending
until Kamaji publishes the admin kubeconfig Secret (backoffLimit covers
the retry window) and then completes before MachineDeployment can
converge. Helm's wait treats the Job as another resource to reach
terminal state and does not gate it on MD.

Extend the existing template test to pin the absence of helm.sh/hook
on both the ServiceAccount and the Job, so this cannot regress silently.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>

packages/apps/kubernetes/templates/talos/bootstrap-token-tenant-job.yaml

@@ -8,10 +8,21 @@
   kube-system, which the management cluster can only reach via the
   Kamaji-issued admin kubeconfig stored as <release>-admin-kubeconfig.
 
-  The Job runs as a Helm post-install/post-upgrade hook so it executes
-  after the KamajiControlPlane has reconciled and the admin kubeconfig
-  Secret exists. It is idempotent (kubectl apply) and re-runs harmlessly
-  on every upgrade.
+  Why not a Helm post-install/post-upgrade hook (the earlier shape):
+  post-* hooks execute only after Helm's wait phase resolves. On the
+  tenant `kubernetes` release the wait blocks on MachineDeployment Ready,
+  which blocks on nodes registering, which blocks on kubelet CSR
+  authentication, which needs the bootstrap-token this Job produces —
+  a hard deadlock on every fresh install and on every HR reconcile that
+  finds MD in a non-Ready state. pre-* hooks are not usable either: they
+  fire before non-hook templates apply, so <release>-admin-kubeconfig
+  (materialised by Kamaji after KamajiControlPlane is applied) is not
+  yet available. The workable shape is a regular resource in the chart:
+  it lands together with the KamajiControlPlane and the talos-secrets,
+  its Pod stays pending until Kamaji publishes the admin kubeconfig
+  Secret (backoffLimit covers the retry window), then runs and completes
+  before MachineDeployment can converge. Helm's wait treats the Job as
+  another resource to reach terminal state and does not gate it on MD.
 */}}
 ---
 apiVersion: v1
@@ -21,10 +32,6 @@ metadata:
   namespace: {{ .Release.Namespace }}
   labels:
     {{- include "kubernetes.labels" . | nindent 4 }}
-  annotations:
-    "helm.sh/hook": post-install,post-upgrade
-    "helm.sh/hook-weight": "0"
-    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
 ---
 apiVersion: batch/v1
 kind: Job
@@ -33,10 +40,6 @@ metadata:
   namespace: {{ .Release.Namespace }}
   labels:
     {{- include "kubernetes.labels" . | nindent 4 }}
-  annotations:
-    "helm.sh/hook": post-install,post-upgrade
-    "helm.sh/hook-weight": "10"
-    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
 spec:
   # backoffLimit + restartPolicy: OnFailure gives the Job up to 11 attempts at
   # the tenant apiserver with exponential backoff (capped at 6 min between

packages/apps/kubernetes/tests/talos_templates_test.yaml

@@ -362,6 +362,21 @@ tests:
           path: metadata.name
           value: test-k8s-bootstrap-token-tenant
         documentIndex: 1
+      # Main-phase, not a hook: NONE of the bootstrap-token resources may
+      # carry helm.sh/hook. The Job must run at t0 alongside the tenant
+      # KamajiControlPlane so that the bootstrap-token-<id> Secret lands
+      # in the tenant kube-system before MachineDeployment starts
+      # scaling. post-install/post-upgrade hooks would gate this on
+      # Helm's wait phase, which itself waits on MachineDeployment
+      # Ready, which waits on nodes to register, which waits on the
+      # very token this Job produces — a deadlock. Pin the absence on
+      # both documents (0 SA, 1 Job).
+      - notExists:
+          path: metadata.annotations["helm.sh/hook"]
+        documentIndex: 0
+      - notExists:
+          path: metadata.annotations["helm.sh/hook"]
+        documentIndex: 1
 
   - it: talos-pki emits two cert-manager Certificates and two Issuers
     templates: