mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 20:23:39 +09:00
fix(vpa): shorten the admission webhook timeout to 5s (#5242)
## What this PR does The VPA admission controller registers its Pod CREATE webhook in every namespace with `failurePolicy: Ignore` and a 30s timeout. When its pod is down and the Service IP blackholes (a cold start after a full cluster shutdown, before the CNI is back), each Pod creation waits the full 30s on top of the other webhooks and fails with `context deadline exceeded`. That includes the CNI's own DaemonSet pods, so the cluster cannot recover without deleting the webhook configuration by hand. Seen on Cozystack (Talos, 3 nodes) after a full shutdown and power-on: the Cilium DaemonSet stayed at 1/3 and Kube-OVN's `ovn-central` could not reach quorum until `vpa-webhook-config` was deleted. Once it was deleted, everything recovered, and VPA re-registered its webhook on start. A mutation takes milliseconds; 5s keeps slow responses working and keeps the total wait of the sequential Pod CREATE webhooks well inside the request deadline. Uses the admission controller's documented `--webhook-timeout-seconds` flag through the chart's `extraArgs`. Refs #3125 (this covers the VPA part of it; `lineage` and `linstor-scheduler-admission` are separate). Supersedes #4903, rebased onto current `main` (no conflicts); `helm unittest` of the chart passes. ### Screenshots No UI change. ### Downstream repositories Walked the trigger map: the diff is one value in `packages/system/vertical-pod-autoscaler/values.yaml` (a system package, no app schema, no platform value, no tooling). - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: ### Release note ```release-note [vpa] Shorten the VPA admission webhook timeout to 5s so pod creation does not stall cluster-wide while the admission controller is unreachable. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Configuration** * The vertical pod autoscaler admission controller now uses a 5-second webhook timeout. * Configuration comments clarify the timeout and its behavior during Pod creation. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
1 file changed
+11
@@ -49,6 +49,17 @@ vertical-pod-autoscaler:
|
||||
app.kubernetes.io/name: vertical-pod-autoscaler
|
||||
app.kubernetes.io/instance: "{{ .Release.Name }}"
|
||||
app.kubernetes.io/component: admission-controller
|
||||
extraArgs:
|
||||
# The admission controller registers its mutating webhook for Pod CREATE in every
|
||||
# namespace with failurePolicy Ignore and, by default, a 30s timeout. Ignore only
|
||||
# applies after the timeout: when its pod is down and the Service IP blackholes (e.g. a
|
||||
# cold start after a full cluster shutdown, before the CNI is back), the API server
|
||||
# still waits the full 30s. The Pod CREATE webhooks are called one after another within
|
||||
# one request, so their waits add up; past the request's deadline the creation fails
|
||||
# with "context deadline exceeded", including for the CNI's own DaemonSet pods (#3125).
|
||||
# A mutation takes milliseconds; 5s keeps slow responses working and keeps the total
|
||||
# wait well inside the deadline.
|
||||
webhook-timeout-seconds: 5
|
||||
resources:
|
||||
limits:
|
||||
memory: 256Mi
|
||||
|
||||
Reference in new issue
Block a user