Files
Aleksei Sviridkin 2869f1554d fix(dashboard): unblock token-proxy startup when JWKS is briefly unreachable (#2745)
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does

Closes #2741.

token-proxy panicked on every cold start where the in-cluster JWKS
endpoint at `kubernetes.default.svc` was briefly unreachable — exactly
the window where CoreDNS, the CNI, or the apiserver endpoint is still
warming up. The pod entered CrashLoopBackOff, the dashboard HelmRelease
never installed, and every release that depended on dashboard stalled
behind it.

Two startup paths touched the network synchronously:
`jwk.Cache.Register` with the default `WaitReady(true)` and an explicit
follow-up `Refresh`, both wrapped in `panic`. `WaitReady(true)` had no
built-in timeout, so a permanently unreachable JWKS would have turned
the panic into an unbounded startup wait rather than fixing anything.

This PR:

- Moves JWK cache construction out of `init()` into a testable function.
- Passes `WithWaitReady(false)` on `Register` and drops the redundant
`Refresh`. Cache setup now returns immediately; the background worker
keeps retrying.
- Lowers `MinInterval` from 5 minutes to 5 seconds. A single bad
response from the apiserver no longer holds the pod NotReady for several
minutes.
- Wires `httprc.WithErrorSink` so background fetch failures show up in
the container log instead of being invisible behind a 503.
- Adds `/healthz/live` (process up) and `/healthz/ready` (JWK cache
populated) plus a `startupProbe` so a genuinely misconfigured JWKS
eventually surfaces as `RestartCount`/`CrashLoopBackOff` instead of a
silently NotReady pod.
- Disambiguates the cold-start window from a real bad token at the
sign-in handler via an `errJWKSNotReady` sentinel.

The auth-proxy (`oidc-enabled=true`) branch keeps its existing `/ping`
probes unchanged.

### Testing

- Go test coverage: WaitReady(false) contract, background-worker
recovery from transient HTTP 5xx (no explicit Refresh — the real
production path), lazy population, catastrophic-misconfig surface
(missing/malformed CA still errors hard), errJWKSNotReady
disambiguation, SA token injection, and the /healthz/ready 503→200
transition.
- helm-unittest pins the gatekeeper template structure for both
`oidc-enabled` branches so a future refactor that collapses the if/else
differently fails the suite.
- `make test` now runs both `helm unittest` and `go test`, picked up
automatically by `hack/helm-unit-tests.sh`.

### Screenshots

N/A — no UI changes.

### Release note

```release-note
fix(dashboard): token-proxy no longer crash-loops when the in-cluster JWKS endpoint is briefly unreachable during a cold install; readiness now reflects JWK cache state and a startupProbe surfaces permanent JWKS misconfig as CrashLoopBackOff instead of a silently NotReady pod.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added dedicated health check endpoints (`/healthz/live` and
`/healthz/ready`) for better container orchestration integration
* Enhanced health probes with separate startup, liveness, and readiness
checks

* **Tests**
  * Added comprehensive test suite for token-proxy functionality

* **Chores**
  * Updated build system with improved test targets
  * Refined Docker image build configuration for better reliability
  * Updated runtime user ID for cluster parity

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2745?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-28 14:18:26 +03:00
..