fix(linstor-scheduler): re-vendor to upstream 0.3.1 with native admission webhook

Re-vendor the linstor-scheduler chart from upstream 0.3.1 (appVersion
v0.3.6), which carries the admission webhook binary that stopped
stripping unknown Pod fields — the image VolumeSource and native-sidecar
restartPolicy, which broke KubeVirt containerDisk VMs and native sidecars
cluster-wide — and now reloads its TLS certificate on rotation instead of
serving a stale one until restarted.

The admission webhook ships in the upstream chart, so the previous
downstream copy is dropped: `make update` pulls it directly and applies
only cozystack-scheduler-deployment.patch (pod component label and a
distro-suffix-stripped image tag), keeping the vendoring deterministic.
Because the webhook now reloads its own certificate, the external
cert-reload workaround is dropped.

The upstream admission Deployment keeps the same name but changes its
immutable pod selector, so an in-place upgrade of an existing install
would fail. A migration deletes the old admission Deployment before the
upgrade so the new one is recreated cleanly, and bumps the migration
target version.

Add a helm-unittest suite covering the admission resources, PDB selector
isolation, least-privilege RBAC, the scheduler image tag, and backward
compatibility.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
This commit is contained in:
Aleksei Sviridkin committed 2026-07-16 16:57:41 +03:00
1 parent 249f7b5a5e
commit f4aef73053
23 files changed
+753 -171

No files matched your search

+28
View File
@@ -0,0 +1,28 @@
#!/bin/sh
# Migration 52 --> 53
# Delete the old linstor-scheduler admission webhook Deployment so the upgrade
# can recreate it.
#
# Cozystack used to ship the linstor-scheduler admission webhook as loose
# downstream templates whose Deployment selected pods by the base scheduler
# selectorLabels plus app.kubernetes.io/component: admission. Upstream chart
# 0.3.0 now ships the admission webhook natively; its Deployment keeps the same
# name (linstor-scheduler-admission) but selects on a distinct
# app.kubernetes.io/name: linstor-scheduler-admission. A Deployment's
# spec.selector is immutable, so an in-place upgrade of an existing install
# would be rejected. Deleting the old Deployment before the component reconciles
# lets the new one be created cleanly (its ReplicaSet and pods follow); the
# webhook's failurePolicy is Ignore, so the brief gap does not block Pod
# creation. Safe when absent: fresh installs and clusters without the linstor
# stack simply skip it via --ignore-not-found.
set -euo pipefail
# shellcheck source=lib/cozystack-version.sh
. "$(dirname "$0")/lib/cozystack-version.sh"
kubectl delete deployment linstor-scheduler-admission --namespace cozy-linstor --ignore-not-found
# Stamp version via the shared helper so the cozystack-version ConfigMap keeps
# the platform.cozystack.io/no-delete label.
stamp_cozystack_version 53
+1 -1
View File
@@ -14,7 +14,7 @@ migrations:
# refuses to advance past a migration file missing from the image (exit 1), so
# a stale pin fails loudly rather than silently skipping the etcd adoption.
image: ghcr.io/cozystack/cozystack/platform-migrations:v1.5.0@sha256:8bf61f17e99eb4a3365394e3d97ac52a1d68cc84aa104894ecb09de25721dc1d
targetVersion: 52
targetVersion: 53
# Bundle deployment configuration
bundles:
system:
+6 -3
View File
@@ -10,6 +10,9 @@ update:
rm -rf charts
helm repo add piraeus-charts https://piraeus.io/helm-charts/
helm repo update piraeus-charts
helm pull piraeus-charts/linstor-scheduler --untar --untardir charts
patch --no-backup-if-mismatch -p4 < patches/disable-ca-key-rotation.patch
patch --no-backup-if-mismatch -p4 < patches/add-scheduler-component-label.patch
helm pull piraeus-charts/linstor-scheduler --untar --untardir charts --version 0.3.1
# Drop the vendored upstream helm-unittest suite: it asserts upstream
# defaults and fails under the cozystack umbrella values. The cozystack-side
# tests/ are the source of truth (same policy as packages/system/coredns).
rm -rf charts/linstor-scheduler/tests
patch --no-backup-if-mismatch -p4 < patches/cozystack-scheduler-deployment.patch
@@ -1,5 +1,5 @@
apiVersion: v2
appVersion: v0.3.2
appVersion: v0.3.6
deprecated: true
description: 'Deploys a new kubernetes scheduler, configured to take advantage of
storage system information. If a Pod is using a LINSTOR volume, the scheduler will
@@ -16,4 +16,4 @@ name: linstor-scheduler
sources:
- https://github.com/piraeusdatastore/linstor-scheduler-extender
type: application
version: 0.2.3
version: 0.3.1
@@ -69,4 +69,29 @@ The following options are available:
| `serviceAccount.create` | Create the service account resource | `true` |
| `serviceAccount.name` | Sets the name of the service account. If left empty, will use the release name as default | `""` |
| `podDisruptionBudget.enabled` | Enable creation of a pod disruption budget to protect the availability of the scheduler | `true` |
| `autoscaling.enabled` | Enable creation of a horizontal pod autoscaler to ensure availability in case of high usage` | `"false` |
| `autoscaling.enabled` | Enable creation of a horizontal pod autoscaler to ensure availability in case of high usage | `false` |
| `admission.enabled` | Deploy the admission webhook that auto-sets `schedulerName` on Pods using LINSTOR-backed PVCs. | `false` |
| `admission.replicaCount` | Number of webhook replicas to deploy. | `2` |
| `admission.driver` | CSI driver name treated as LINSTOR-backed. Pods whose PVCs resolve to it get the mutation. | `linstor.csi.linbit.com` |
| `admission.failurePolicy` | Webhook `failurePolicy`. `Ignore` keeps Pod creation working if the webhook is unavailable. | `Ignore` |
| `admission.timeoutSeconds` | Webhook call timeout. Kept under the Kubernetes default of 10s to limit Pod-creation latency. | `5` |
| `admission.namespaceSelector` | Namespace selector for the webhook. Empty selects all namespaces (including `kube-system`). | `{}` |
| `admission.resources` | Resources to request and limit on the webhook container. | `{}` |
| `admission.podSecurityContext` | Pod-level security context for the webhook pod. | `{runAsNonRoot: true, runAsUser: 1000, runAsGroup: 1000, fsGroup: 1000}` |
| `admission.securityContext` | Container-level security context for the webhook container. | `{allowPrivilegeEscalation: false, capabilities: {drop: [ALL]}, readOnlyRootFilesystem: true}` |
| `admission.createTLS` | How to provision the webhook TLS: `cert-manager`, `helm`, or `""` (bring your own). See note below. | `cert-manager` |
| `admission.tls.secretName` | Name of the TLS secret. Empty defaults to `<fullname>-admission-tls`. | `""` |
| `admission.tls.caBundle` | CA bundle (base64 PEM) injected into the webhook. Only used when `createTLS` is `""`. | `""` |
| `admission.tls.certManager.rotationPolicy` | `privateKey.rotationPolicy` of the cert-manager CA. `Never` keeps the CA key (and caBundle) stable. | `Never` |
| `admission.affinity` | Affinity for the webhook pods. Empty applies a soft anti-affinity spreading replicas across nodes. | `{}` |
| `admission.reloadOnCertChange` | Add Stakater Reloader annotations so the pods restart when the TLS secret changes (see note below). | `false` |
## Admission webhook (optional)
As noted above, the LINSTOR Scheduler itself is no longer maintained and `volumeBindingMode: WaitForFirstConsumer` is the preferred approach for new deployments. For clusters that still run this scheduler, the optional admission webhook (`admission.enabled`, disabled by default) removes the need to set `schedulerName` on every Pod: it watches Pod creation and, when a Pod uses a PersistentVolumeClaim backed by the LINSTOR CSI driver, sets `schedulerName` to this scheduler automatically. It changes nothing when disabled.
> **Note:** `admission.createTLS: helm` regenerates the CA and certificate on every `helm upgrade`; it suits one-shot installs, not GitOps or continuously-reconciled setups (Flux/Argo) where the certificate would churn on each reconcile. Use `cert-manager` or bring your own secret (`""`) there.
> **Note:** the default `failurePolicy: Ignore` fails open, so an unreachable webhook never blocks Pod creation (affected Pods just fall back to the default scheduler). Setting `failurePolicy: Fail` makes the webhook mandatory for every Pod `CREATE` in the selected namespaces; with `createTLS: cert-manager` this also requires the cert-manager CA injector to be running so the `caBundle` is populated, otherwise all matching Pod creation is blocked until injection completes. Because the empty default `namespaceSelector` also intercepts the webhook's own namespace and `kube-system`, running with `failurePolicy: Fail` can deadlock a restart of the webhook (its own new Pods cannot be admitted while all replicas are down) — exclude the release namespace and `kube-system` via `admission.namespaceSelector` if you switch to `Fail`.
> **Note:** the webhook binary loads its serving certificate once at startup and does not hot-reload it. When the certificate is renewed — cert-manager rotates the leaf roughly yearly, or you rotate an external secret — the running Pods keep serving the old certificate until they restart, after which the webhook silently stops mutating. Set `admission.reloadOnCertChange: true` (requires the Stakater Reloader controller) to roll the Deployment automatically when the secret changes, or restart the webhook Pods manually after each renewal. In `createTLS: helm` mode a `helm upgrade` regenerates the certificate and already rolls the Pods.
@@ -3,10 +3,28 @@ Scheduler {{ include "linstor-scheduler.fullname" . }} deployed!
Used LINSTOR URL: {{ include "linstor-scheduler.linstorEndpoint" .}}
Please run `helm test {{ .Release.Name }}` to ensure it's properly working.
{{ if .Values.admission.enabled -}}
The optional admission webhook is enabled: Pods using LINSTOR-backed PersistentVolumeClaims
are automatically scheduled by {{ include "linstor-scheduler.fullname" . }}, so you
do not need to set schedulerName by hand. To schedule a Pod that does not use a
LINSTOR volume, set its schedulerName explicitly:
---
spec:
schedulerName: {{ include "linstor-scheduler.fullname" . }}
---
{{- if eq .Values.admission.createTLS "" }}
{{ if not .Values.admission.tls.caBundle }}
NOTE: admission.createTLS is "" and admission.tls.caBundle is empty. Unless an
external controller injects the webhook's caBundle, the API server cannot reach
the webhook and Pods will be scheduled WITHOUT the mutation (failurePolicy: {{ .Values.admission.failurePolicy }}).
{{- end }}
{{- end }}
{{- else -}}
Specify the scheduler on your pods to start smart scheduling based on your Persistent Volumes:
---
spec:
schedulerName: {{ include "linstor-scheduler.fullname" . }}
---
{{- end }}
@@ -61,6 +61,65 @@ Create the name of the service account to use
{{- end }}
{{- end }}
{{/*
Fully qualified name for the admission webhook resources. The base fullname is
truncated before the "-admission" suffix (and further sub-suffixes such as
"-selfsigned") are appended, so the longest derived name still fits the 63-char
Kubernetes name limit for long release names.
*/}}
{{- define "linstor-scheduler.admissionFullname" -}}
{{- printf "%s-admission" (include "linstor-scheduler.fullname" . | trunc 42 | trimSuffix "-") }}
{{- end }}
{{/*
Selector labels for the admission webhook. The name carries an "-admission"
suffix so the webhook pods are NOT matched by the base scheduler's selectors
(Deployment, PodDisruptionBudget), whose selector fields are immutable and so
cannot be narrowed after the fact. This keeps the base PDB and HPA scoped to the
scheduler pods only. The base name is truncated BEFORE the suffix is appended so
a long nameOverride can never eat the "-admission" suffix and collapse the name
back onto the base selector.
*/}}
{{- define "linstor-scheduler.admissionSelectorLabels" -}}
app.kubernetes.io/name: {{ printf "%s-admission" (include "linstor-scheduler.name" . | trunc 53 | trimSuffix "-") }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Common labels for the admission webhook resources.
*/}}
{{- define "linstor-scheduler.admissionLabels" -}}
helm.sh/chart: {{ include "linstor-scheduler.chart" . }}
{{ include "linstor-scheduler.admissionSelectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/component: admission
{{- end }}
{{/*
Name of the secret holding the admission webhook's TLS certificate.
Defaults to "<admissionFullname>-tls" unless admission.tls.secretName is set,
so users managing the secret externally can point the Deployment at their own.
*/}}
{{- define "linstor-scheduler.admissionTLSSecretName" -}}
{{- default (printf "%s-tls" (include "linstor-scheduler.admissionFullname" .)) .Values.admission.tls.secretName }}
{{- end }}
{{/*
Name of the service account used by the admission webhook. When the chart
creates service accounts it gets a dedicated one, so the webhook pod does not
inherit the scheduler's system:kube-scheduler / system:volume-scheduler bindings.
*/}}
{{- define "linstor-scheduler.admissionServiceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- include "linstor-scheduler.admissionFullname" . }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
{{/*
Get the kubernetes version we should assume for creating scheduler configs.
Strips distribution suffixes like +k3s1, +rke2r1 from version string.
@@ -0,0 +1,58 @@
{{- if and .Values.admission.enabled (eq .Values.admission.createTLS "cert-manager") }}
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "linstor-scheduler.admissionFullname" . }}-selfsigned
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
secretName: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
duration: 43800h # 5 years
commonName: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
privateKey:
rotationPolicy: {{ .Values.admission.tls.certManager.rotationPolicy }}
issuerRef:
name: {{ include "linstor-scheduler.admissionFullname" . }}-selfsigned
isCA: true
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "linstor-scheduler.admissionFullname" . }}-ca-issuer
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
ca:
secretName: {{ include "linstor-scheduler.admissionFullname" . }}-root-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "linstor-scheduler.admissionFullname" . }}-cert
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
secretName: {{ include "linstor-scheduler.admissionTLSSecretName" . }}
duration: 8760h # 1 year
renewBefore: 24h
issuerRef:
name: {{ include "linstor-scheduler.admissionFullname" . }}-ca-issuer
commonName: {{ include "linstor-scheduler.admissionFullname" . }}
dnsNames:
- {{ include "linstor-scheduler.admissionFullname" . }}
- {{ include "linstor-scheduler.admissionFullname" . }}.{{ .Release.Namespace }}.svc
{{- end }}
@@ -3,36 +3,47 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission
name: {{ include "linstor-scheduler.admissionFullname" . }}
{{- if .Values.admission.reloadOnCertChange }}
annotations:
# The webhook binary loads its serving certificate once at startup and does
# not hot-reload it, so the pods must restart when the TLS secret changes
# (e.g. cert-manager renewal). These annotations let Stakater Reloader roll
# the Deployment automatically when the secret is updated.
secret.reloader.stakater.com/reload: {{ include "linstor-scheduler.admissionTLSSecretName" . | quote }}
{{- end }}
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
app.kubernetes.io/component: admission
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
replicas: {{ .Values.admission.replicaCount }}
selector:
matchLabels:
{{- include "linstor-scheduler.selectorLabels" . | nindent 6 }}
app.kubernetes.io/component: admission
{{- include "linstor-scheduler.admissionSelectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
{{- if or .Values.podAnnotations (eq .Values.admission.createTLS "helm") }}
annotations:
{{- if eq .Values.admission.createTLS "helm" }}
# In "helm" TLS mode the CA and certificate are regenerated on every
# "helm upgrade". Rolling the pods each revision makes them pick up the
# new certificate that matches the freshly generated webhook caBundle.
checksum/tls-revision: {{ .Release.Revision | quote }}
{{- end }}
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
labels:
{{- include "linstor-scheduler.selectorLabels" . | nindent 8 }}
{{- include "linstor-scheduler.admissionSelectorLabels" . | nindent 8 }}
app.kubernetes.io/component: admission
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "linstor-scheduler.serviceAccountName" . }}
serviceAccountName: {{ include "linstor-scheduler.admissionServiceAccountName" . }}
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
{{- toYaml .Values.admission.podSecurityContext | nindent 8 }}
containers:
- name: linstor-scheduler-admission
image: {{ .Values.extender.image.repository }}:{{ .Values.extender.image.tag | default .Chart.AppVersion }}
@@ -40,38 +51,21 @@ spec:
command: ["/linstor-scheduler-admission"]
args:
- -scheduler={{ include "linstor-scheduler.fullname" . }}
- -driver={{ .Values.admission.driver }}
- -tls-cert-file=/etc/webhook/certs/tls.crt
- -tls-key-file=/etc/webhook/certs/tls.key
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
{{- toYaml .Values.admission.securityContext | nindent 12 }}
ports:
- containerPort: 8080
name: https
protocol: TCP
env:
- name: LS_CONTROLLERS
value: {{ include "linstor-scheduler.linstorEndpoint" . }}
{{- if include "linstor-scheduler.linstorClientSecretName" . }}
- name: LS_USER_CERTIFICATE
valueFrom:
secretKeyRef:
name: {{ include "linstor-scheduler.linstorClientSecretName" . }}
key: tls.crt
- name: LS_USER_KEY
valueFrom:
secretKeyRef:
name: {{ include "linstor-scheduler.linstorClientSecretName" . }}
key: tls.key
- name: LS_ROOT_CA
valueFrom:
secretKeyRef:
name: {{ include "linstor-scheduler.linstorClientSecretName" . }}
key: ca.crt
{{- end }}
readinessProbe:
tcpSocket:
port: https
livenessProbe:
tcpSocket:
port: https
volumeMounts:
- name: webhook-certs
mountPath: /etc/webhook/certs
@@ -81,16 +75,25 @@ spec:
volumes:
- name: webhook-certs
secret:
secretName: {{ include "linstor-scheduler.fullname" . }}-admission-tls
defaultMode: 0400
secretName: {{ include "linstor-scheduler.admissionTLSSecretName" . }}
defaultMode: 0440
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if .Values.admission.affinity }}
{{- toYaml .Values.admission.affinity | nindent 8 }}
{{- else }}
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchLabels:
{{- include "linstor-scheduler.admissionSelectorLabels" . | nindent 20 }}
topologyKey: kubernetes.io/hostname
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
@@ -0,0 +1,19 @@
{{- if and .Values.admission.enabled .Values.podDisruptionBudget.enabled }}
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "linstor-scheduler.admissionFullname" . }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
selector:
matchLabels:
{{- include "linstor-scheduler.admissionSelectorLabels" . | nindent 6 }}
{{- if .Values.podDisruptionBudget.minAvailable }}
minAvailable: {{ .Values.podDisruptionBudget.minAvailable }}
{{- end }}
{{- if .Values.podDisruptionBudget.maxUnavailable }}
maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable }}
{{- end }}
{{- end }}
@@ -3,10 +3,9 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission
name: {{ include "linstor-scheduler.admissionFullname" . }}
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
app.kubernetes.io/component: admission
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
spec:
type: ClusterIP
ports:
@@ -15,6 +14,5 @@ spec:
protocol: TCP
name: https
selector:
{{- include "linstor-scheduler.selectorLabels" . | nindent 4 }}
app.kubernetes.io/component: admission
{{- include "linstor-scheduler.admissionSelectorLabels" . | nindent 4 }}
{{- end }}
@@ -0,0 +1,13 @@
{{- if and .Values.admission.enabled .Values.serviceAccount.create }}
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "linstor-scheduler.admissionServiceAccountName" . }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
@@ -0,0 +1,78 @@
{{- if .Values.admission.enabled }}
{{/*
Validate the TLS provisioning method and guard against a silently-broken
bring-your-own configuration (no cert-manager, no generated cert, and no CA to
trust) that would leave the webhook unusable.
*/}}
{{- if not (has .Values.admission.createTLS (list "cert-manager" "helm" "")) }}
{{- fail (printf "admission.createTLS must be one of \"cert-manager\", \"helm\" or \"\", got %q" .Values.admission.createTLS) }}
{{- end }}
{{- if and (eq .Values.admission.createTLS "") (not .Values.admission.tls.secretName) (not .Values.admission.tls.caBundle) }}
{{- fail "admission.createTLS is \"\" (bring your own TLS): set admission.tls.secretName to your certificate secret and admission.tls.caBundle to its CA (or use an external ca-injector), or switch admission.createTLS to \"cert-manager\" or \"helm\"" }}
{{- end }}
{{/*
For the "helm" TLS method the CA is generated exactly once here and shared
between the TLS Secret (ca.crt) and the webhook caBundle below, so both are
byte-identical. genCA is non-deterministic per call, so it must not be invoked
from a second template file.
*/}}
{{- $ca := dict }}
{{- $cert := dict }}
{{- if eq .Values.admission.createTLS "helm" }}
{{- $commonName := include "linstor-scheduler.admissionFullname" . }}
{{- $altNames := list $commonName (printf "%s.%s.svc" $commonName .Release.Namespace) }}
{{- $ca = genCA (printf "%s-ca" (include "linstor-scheduler.admissionFullname" .)) 3650 }}
{{- $cert = genSignedCert $commonName (list) $altNames 3650 $ca }}
---
apiVersion: v1
kind: Secret
type: kubernetes.io/tls
metadata:
name: {{ include "linstor-scheduler.admissionTLSSecretName" . }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
stringData:
tls.crt: {{ $cert.Cert | quote }}
tls.key: {{ $cert.Key | quote }}
ca.crt: {{ $ca.Cert | quote }}
{{- end }}
---
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: {{ include "linstor-scheduler.admissionFullname" . }}
{{- if eq .Values.admission.createTLS "cert-manager" }}
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "linstor-scheduler.admissionFullname" . }}-cert
{{- end }}
labels:
{{- include "linstor-scheduler.admissionLabels" . | nindent 4 }}
webhooks:
- name: linstor-scheduler-admission.linbit.com
admissionReviewVersions: ["v1"]
sideEffects: None
failurePolicy: {{ .Values.admission.failurePolicy }}
timeoutSeconds: {{ .Values.admission.timeoutSeconds }}
clientConfig:
{{- if eq .Values.admission.createTLS "helm" }}
caBundle: {{ $ca.Cert | b64enc }}
{{- else if .Values.admission.tls.caBundle }}
caBundle: {{ .Values.admission.tls.caBundle }}
{{- end }}
service:
name: {{ include "linstor-scheduler.admissionFullname" . }}
namespace: {{ .Release.Namespace }}
path: /mutate
port: 443
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
scope: "*"
{{- with .Values.admission.namespaceSelector }}
namespaceSelector:
{{- toYaml . | nindent 6 }}
{{- end }}
{{- end }}
@@ -1,58 +0,0 @@
{{- if .Values.admission.enabled }}
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission-selfsigned
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission-root-ca
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
spec:
secretName: {{ include "linstor-scheduler.fullname" . }}-admission-root-ca
duration: 43800h # 5 years
commonName: {{ include "linstor-scheduler.fullname" . }}-admission-root-ca
issuerRef:
name: {{ include "linstor-scheduler.fullname" . }}-admission-selfsigned
isCA: true
privateKey:
rotationPolicy: Never
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission-ca-issuer
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
spec:
ca:
secretName: {{ include "linstor-scheduler.fullname" . }}-admission-root-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission-cert
namespace: {{ .Release.Namespace }}
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
spec:
secretName: {{ include "linstor-scheduler.fullname" . }}-admission-tls
duration: 8760h # 1 year
renewBefore: 24h
issuerRef:
name: {{ include "linstor-scheduler.fullname" . }}-admission-ca-issuer
commonName: {{ include "linstor-scheduler.fullname" . }}-admission
dnsNames:
- {{ include "linstor-scheduler.fullname" . }}-admission
- {{ include "linstor-scheduler.fullname" . }}-admission.{{ .Release.Namespace }}.svc
{{- end }}
@@ -1,32 +0,0 @@
{{- if .Values.admission.enabled }}
---
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "linstor-scheduler.fullname" . }}-admission-cert
labels:
{{- include "linstor-scheduler.labels" . | nindent 4 }}
webhooks:
- name: linstor-scheduler-admission.linbit.com
admissionReviewVersions: ["v1", "v1beta1"]
sideEffects: None
failurePolicy: {{ .Values.admission.failurePolicy }}
clientConfig:
service:
name: {{ include "linstor-scheduler.fullname" . }}-admission
namespace: {{ .Release.Namespace }}
path: /mutate
port: 443
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
scope: "*"
{{- with .Values.admission.namespaceSelector }}
namespaceSelector:
{{- toYaml . | nindent 6 }}
{{- end }}
{{- end }}
@@ -29,10 +29,10 @@ rules:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission
name: {{ include "linstor-scheduler.admissionFullname" . }}
rules:
- apiGroups: [""]
resources: ["pods", "persistentvolumeclaims", "persistentvolumes"]
resources: ["persistentvolumeclaims", "persistentvolumes"]
verbs: ["get"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
@@ -41,14 +41,14 @@ rules:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "linstor-scheduler.fullname" . }}-admission
name: {{ include "linstor-scheduler.admissionFullname" . }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "linstor-scheduler.fullname" . }}-admission
name: {{ include "linstor-scheduler.admissionFullname" . }}
subjects:
- kind: ServiceAccount
name: {{ include "linstor-scheduler.serviceAccountName" . }}
name: {{ include "linstor-scheduler.admissionServiceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end }}
---
@@ -87,10 +87,21 @@ tolerations: []
affinity: {}
# Optional admission webhook that automatically sets the schedulerName on Pods
# using LINSTOR-backed PersistentVolumeClaims, so they are scheduled by this
# scheduler without having to set schedulerName by hand.
admission:
enabled: false
replicaCount: 2
# CSI driver name the webhook treats as LINSTOR-backed. Pods whose PVCs resolve
# to this provisioner get the schedulerName mutation. Defaults to the LINSTOR
# CSI driver; override only for a non-standard driver name.
driver: linstor.csi.linbit.com
failurePolicy: Ignore
# Webhook call timeout in seconds. Kept below the Kubernetes default of 10s so
# an unreachable webhook adds less latency to Pod creation before failurePolicy
# takes effect.
timeoutSeconds: 5
namespaceSelector: {}
# matchExpressions:
# - key: kubernetes.io/metadata.name
@@ -104,3 +115,51 @@ admission:
# requests:
# cpu: 10m
# memory: 64Mi
# Pod-level security context for the webhook pod.
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
# Container-level security context for the webhook container.
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
# Affinity for the webhook pods. When empty, a soft (preferred) pod
# anti-affinity spreads the replicas across nodes by hostname; set this to
# override it entirely (e.g. a required anti-affinity or a nodeAffinity).
affinity: {}
# The webhook binary loads its TLS certificate once at startup and does NOT
# hot-reload it. When the certificate is renewed (cert-manager rotates the
# leaf roughly yearly, or you rotate an external secret) the running pods keep
# serving the old certificate until they restart, after which the webhook
# silently stops mutating (failurePolicy: Ignore). Set this to true to add
# Stakater Reloader annotations so the pods roll automatically when the TLS
# secret changes (requires the Reloader controller in the cluster); otherwise
# restart the webhook pods manually after each renewal.
reloadOnCertChange: false
# How to provision the webhook's TLS certificate:
# * "cert-manager" uses a self-signed cert-manager.io Issuer (default).
# * "helm" makes Helm itself generate the CA and server certificate.
# NOTE: the "helm" method regenerates the CA and certificate on every
# "helm upgrade". It suits one-shot installs, not GitOps or any
# continuously-reconciled setup (e.g. Flux/Argo), where the certificate
# would churn on each reconcile. Use "cert-manager" or "" there.
# * "" creates nothing: bring your own secret (see tls.secretName / tls.caBundle).
createTLS: cert-manager
tls:
# Name of the secret holding the webhook's TLS certificate.
# Empty defaults to "<fullname>-admission-tls". Set it to point the
# Deployment at a secret you manage yourself.
secretName: ""
# CA bundle injected into the webhook (base64-encoded PEM). Only used when
# createTLS is "" and you inject your own CA. Leave empty if an external
# controller (e.g. cert-manager's ca-injector) patches the caBundle.
caBundle: ""
certManager:
# rotationPolicy for the CA certificate's private key. "Never" keeps the
# CA key stable so the webhook caBundle does not churn on renewal.
rotationPolicy: Never
@@ -1,9 +0,0 @@
diff --git a/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml b/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml
--- a/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml
+++ b/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml
@@ -21,4 +21,5 @@
labels:
{{- include "linstor-scheduler.selectorLabels" . | nindent 8 }}
+ app.kubernetes.io/component: scheduler
spec:
{{- with .Values.imagePullSecrets }}
@@ -0,0 +1,35 @@
# Cozystack downstream customizations to the linstor-scheduler Deployment.
# Both edits touch the same file and are deliberately kept in one patch:
# splitting them into two patches on deployment.yaml would create fragile
# line-offset coupling between the hunks.
#
# (a) Add the pod label 'app.kubernetes.io/component: scheduler' so the
# wrapper chart's templates/extender-service.yaml can select the
# scheduler pods by that label.
# (b) Derive the kube-scheduler image tag from the stripped kubeVersion
# helper instead of raw .Capabilities.KubeVersion.Version, so a
# distribution suffix (e.g. v1.31.0+k3s1) does not resolve to a
# non-existent registry.k8s.io/kube-scheduler image tag.
#
# Chart source: https://piraeus.io/helm-charts (linstor-scheduler 0.3.1)
diff --git a/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml b/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml
index 49898b8..289021f 100644
--- a/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml
+++ b/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/deployment.yaml
@@ -20,6 +20,7 @@ spec:
{{- end }}
labels:
{{- include "linstor-scheduler.selectorLabels" . | nindent 8 }}
+ app.kubernetes.io/component: scheduler
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
@@ -30,7 +31,7 @@ spec:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: kube-scheduler
- image: "{{ .Values.scheduler.image.repository }}:{{ .Values.scheduler.image.tag | default .Capabilities.KubeVersion.Version }}"
+ image: "{{ .Values.scheduler.image.repository }}:{{ .Values.scheduler.image.tag | default (include "linstor-scheduler.kubeVersion" .) }}"
securityContext:
{{- toYaml .Values.scheduler.securityContext | nindent 12 }}
command:
@@ -1,13 +0,0 @@
diff --git a/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/certmanager.yaml b/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/certmanager.yaml
index 3942555b..760a9d5d 100644
--- a/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/certmanager.yaml
+++ b/packages/system/linstor-scheduler/charts/linstor-scheduler/templates/certmanager.yaml
@@ -24,6 +24,8 @@ spec:
issuerRef:
name: {{ include "linstor-scheduler.fullname" . }}-admission-selfsigned
isCA: true
+ privateKey:
+ rotationPolicy: Never
---
apiVersion: cert-manager.io/v1
kind: Issuer
@@ -0,0 +1,258 @@
suite: admission webhook (cozystack)
templates:
- charts/linstor-scheduler/templates/admission-deployment.yaml
- charts/linstor-scheduler/templates/admission-service.yaml
- charts/linstor-scheduler/templates/admission-webhook.yaml
- charts/linstor-scheduler/templates/admission-certmanager.yaml
- charts/linstor-scheduler/templates/admission-serviceaccount.yaml
- charts/linstor-scheduler/templates/admission-poddisruptionbudget.yaml
- charts/linstor-scheduler/templates/poddisruptionbudget.yaml
- charts/linstor-scheduler/templates/rbac.yaml
release:
name: linstor-scheduler
tests:
# Pin 1: with admission on (package default), the admission stack renders,
# including the cert-manager issuer/cert chain.
- it: renders the admission Deployment
template: charts/linstor-scheduler/templates/admission-deployment.yaml
asserts:
- isKind:
of: Deployment
- equal:
path: metadata.name
value: linstor-scheduler-admission
- it: renders the admission Service
template: charts/linstor-scheduler/templates/admission-service.yaml
asserts:
- isKind:
of: Service
- equal:
path: metadata.name
value: linstor-scheduler-admission
- it: renders the MutatingWebhookConfiguration
template: charts/linstor-scheduler/templates/admission-webhook.yaml
asserts:
- isKind:
of: MutatingWebhookConfiguration
- it: cert-manager mode emits the Issuer/Certificate chain with a stable CA key
template: charts/linstor-scheduler/templates/admission-certmanager.yaml
asserts:
- hasDocuments:
count: 4
- matchRegex:
path: metadata.name
pattern: linstor-scheduler-admission
- documentIndex: 1
isKind:
of: Certificate
- documentIndex: 1
equal:
path: spec.privateKey.rotationPolicy
value: Never
# Pin 4: dedicated (least-privilege) admission ServiceAccount.
- it: renders a separate admission ServiceAccount
template: charts/linstor-scheduler/templates/admission-serviceaccount.yaml
asserts:
- isKind:
of: ServiceAccount
- equal:
path: metadata.name
value: linstor-scheduler-admission
# Pin 3: admission image is pinned to the extender at the chart appVersion.
- it: admission container uses the linstor-scheduler-extender image at v0.3.6
template: charts/linstor-scheduler/templates/admission-deployment.yaml
asserts:
- matchRegex:
path: spec.template.spec.containers[0].image
pattern: /linstor-scheduler-extender:v0\.3\.6$
# Pin 2: node-drain safety. The base PDB must select ONLY the scheduler
# (app.kubernetes.io/name: linstor-scheduler), and admission pods must carry a
# DISTINCT name so the base PDB cannot cover them. If either regresses, the
# single scheduler pod becomes evictable while admission replicas keep quorum.
- it: base PDB selects the scheduler, not admission
template: charts/linstor-scheduler/templates/poddisruptionbudget.yaml
asserts:
- isKind:
of: PodDisruptionBudget
- equal:
path: spec.selector.matchLabels["app.kubernetes.io/name"]
value: linstor-scheduler
- it: admission pods carry a distinct name so the base PDB cannot select them
template: charts/linstor-scheduler/templates/admission-deployment.yaml
asserts:
- equal:
path: spec.selector.matchLabels["app.kubernetes.io/name"]
value: linstor-scheduler-admission
- equal:
path: spec.template.metadata.labels["app.kubernetes.io/name"]
value: linstor-scheduler-admission
- it: admission has its own PDB targeting admission pods
template: charts/linstor-scheduler/templates/admission-poddisruptionbudget.yaml
asserts:
- isKind:
of: PodDisruptionBudget
- equal:
path: spec.selector.matchLabels["app.kubernetes.io/name"]
value: linstor-scheduler-admission
# Pin 5: with admission disabled, nothing admission renders, base PDB stays.
- it: renders no admission resources when disabled
set:
linstor-scheduler.admission.enabled: false
asserts:
- hasDocuments:
count: 0
template: charts/linstor-scheduler/templates/admission-deployment.yaml
- hasDocuments:
count: 0
template: charts/linstor-scheduler/templates/admission-service.yaml
- hasDocuments:
count: 0
template: charts/linstor-scheduler/templates/admission-webhook.yaml
- hasDocuments:
count: 0
template: charts/linstor-scheduler/templates/admission-certmanager.yaml
- hasDocuments:
count: 0
template: charts/linstor-scheduler/templates/admission-serviceaccount.yaml
- hasDocuments:
count: 0
template: charts/linstor-scheduler/templates/admission-poddisruptionbudget.yaml
- it: base PDB still renders when admission is disabled
set:
linstor-scheduler.admission.enabled: false
template: charts/linstor-scheduler/templates/poddisruptionbudget.yaml
asserts:
- isKind:
of: PodDisruptionBudget
# The whole point of the re-vendoring: the webhook matches LINSTOR-backed PVCs
# by CSI provisioner, which must equal the cozystack LINSTOR driver name.
- it: admission container is configured for the LINSTOR CSI driver
template: charts/linstor-scheduler/templates/admission-deployment.yaml
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: -driver=linstor.csi.linbit.com
# The v0.3.6 extender reloads its serving certificate on rotation, so the
# package leaves reloadOnCertChange at the upstream default and no Reloader
# annotation is emitted. Pin the absence: re-enabling the knob would buy only
# a redundant pod restart on every cert-manager renewal.
- it: emits no Reloader annotation, the webhook reloads its own certificate
template: charts/linstor-scheduler/templates/admission-deployment.yaml
asserts:
- notExists:
path: metadata.annotations["secret.reloader.stakater.com/reload"]
# createTLS is a closed enum; an unknown value must fail the render, not
# silently produce a half-configured webhook.
- it: rejects an unknown createTLS value
set:
linstor-scheduler.admission.enabled: true
linstor-scheduler.admission.createTLS: bogus
template: charts/linstor-scheduler/templates/admission-webhook.yaml
asserts:
- failedTemplate:
errorPattern: createTLS must be one of
# Bring-your-own TLS with nothing supplied would leave the webhook unusable;
# the guard must fail the render rather than ship a broken MutatingWebhook.
- it: rejects bring-your-own TLS with no secret or caBundle provided
set:
linstor-scheduler.admission.enabled: true
linstor-scheduler.admission.createTLS: ""
template: charts/linstor-scheduler/templates/admission-webhook.yaml
asserts:
- failedTemplate:
errorPattern: bring your own TLS
# helm mode self-signs: it must emit the TLS Secret and inject the matching
# CA inline into the webhook's clientConfig (no cert-manager involved).
- it: helm mode emits a TLS Secret and an inline caBundle
set:
linstor-scheduler.admission.enabled: true
linstor-scheduler.admission.createTLS: helm
template: charts/linstor-scheduler/templates/admission-webhook.yaml
asserts:
- hasDocuments:
count: 2
- documentIndex: 0
isKind:
of: Secret
- documentIndex: 0
equal:
path: type
value: kubernetes.io/tls
- documentIndex: 1
isKind:
of: MutatingWebhookConfiguration
- documentIndex: 1
isNotEmpty:
path: webhooks[0].clientConfig.caBundle
# Bring-your-own TLS: no Secret is created, the supplied caBundle is wired into
# the webhook, and the Deployment mounts the caller-managed secret.
- it: bring-your-own TLS wires the supplied secret and caBundle
set:
linstor-scheduler.admission.enabled: true
linstor-scheduler.admission.createTLS: ""
linstor-scheduler.admission.tls.secretName: my-admission-tls
linstor-scheduler.admission.tls.caBundle: TESTCABUNDLE
asserts:
- template: charts/linstor-scheduler/templates/admission-webhook.yaml
hasDocuments:
count: 1
- template: charts/linstor-scheduler/templates/admission-webhook.yaml
isKind:
of: MutatingWebhookConfiguration
- template: charts/linstor-scheduler/templates/admission-webhook.yaml
equal:
path: webhooks[0].clientConfig.caBundle
value: TESTCABUNDLE
- template: charts/linstor-scheduler/templates/admission-deployment.yaml
equal:
path: spec.template.spec.volumes[0].secret.secretName
value: my-admission-tls
# Least-privilege guard. The webhook only reads PVC/PV/StorageClass via the
# Kubernetes API (the Pod arrives in the AdmissionReview), so its ClusterRole
# must not grant pods access. Pin it so a future re-vendor cannot silently
# re-widen the permissions.
- it: admission ClusterRole stays least-privilege (no pods access)
template: charts/linstor-scheduler/templates/rbac.yaml
documentIndex: 1
asserts:
- isKind:
of: ClusterRole
- equal:
path: metadata.name
value: linstor-scheduler-admission
- notContains:
path: rules[0].resources
content: pods
- equal:
path: rules[0].resources
value:
- persistentvolumeclaims
- persistentvolumes
- equal:
path: rules[0].verbs
value:
- get
- equal:
path: rules[1].resources
value:
- storageclasses
@@ -33,3 +33,30 @@ tests:
- equal:
path: spec.selector["app.kubernetes.io/component"]
value: scheduler
# Cozystack downstream patch strips distribution suffixes (e.g. +k3s1) from the
# kube-scheduler image tag via the kubeVersion helper, so the tag resolves to a
# real registry.k8s.io image on k3s/rke2/Talos. Pin it so a future revendor
# cannot silently drop the fix.
- it: base kube-scheduler image tag carries no distribution suffix
set:
linstor-scheduler.scheduler.image.compatibleKubernetesRelease: v1.31.0+k3s1
template: charts/linstor-scheduler/templates/deployment.yaml
asserts:
- matchRegex:
path: spec.template.spec.containers[0].image
pattern: :v1\.31\.0$
- notMatchRegex:
path: spec.template.spec.containers[0].image
pattern: \+
# Node-drain safety: once admission pods are isolated from the base PDB, a
# single scheduler replica under the base minAvailable:1 PDB would make its
# node undrainable (disruptionsAllowed 0). Two replicas keep it drainable and
# give real HA via leader election.
- it: base scheduler runs two replicas so its node stays drainable
template: charts/linstor-scheduler/templates/deployment.yaml
asserts:
- equal:
path: spec.replicas
value: 2
@@ -1,7 +1,20 @@
linstor-scheduler:
fullnameOverride: linstor-scheduler
# Run two scheduler replicas (active/standby via the chart's leader election).
# A single replica under the base minAvailable:1 PodDisruptionBudget yields
# disruptionsAllowed 0, which makes the scheduler's node undrainable (blocks
# kubectl drain and Talos rolling upgrades); two replicas keep the node
# drainable and give leader-elected failover when a replica dies. The base
# scheduler sets no anti-affinity, so both replicas can still land on one
# node: this buys drain safety, not node-failure HA.
replicaCount: 2
linstor:
endpoint: "https://linstor-controller.cozy-linstor.svc:3371"
clientSecret: "linstor-client-tls"
admission:
enabled: true
# reloadOnCertChange stays at the upstream default (false). The vendored
# chart's values.yaml still documents the knob as required, but that text
# predates the binary the chart now ships: the v0.3.6 extender serves its
# certificate via tls.Config.GetCertificate and reloads it on rotation, so
# the Stakater Reloader annotation would only buy a redundant pod restart.