From a9c8f206e86469f81060083db5a53ea9299d3494 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Wed, 20 May 2026 21:52:13 +0300 Subject: [PATCH] chore(metallb): bump to v0.16.1 and rotate host-network port denylist MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Upgrade vendored MetalLB chart from 0.15.2 to 0.16.1 and rebuild the cozystack-built controller / speaker images at the new version (pinned to linux/amd64 in the Makefile to match the prior baseline and the e2e Talos sandbox arch — without the pin, a contributor building on an arm64 host like Apple Silicon ships single-arch arm64 images that crashloop on amd64 nodes with "exec format error"). Adopt the upstream-default BGP backend: FRR-K8s replaces the deprecated FRR mode. Rotate the cilium-networkpolicy clusterwide host-network port denylist to match the new metallb endpoints. The Makefile now vendors the frr-k8s subchart instead of dropping it, and pins the chart version so future make update invocations are deterministic. Upstream highlights between 0.15.2 and 0.16.1: - 0.16.0: FRR-K8s is the new default BGP backend; legacy FRR is deprecated. - 0.16.0: new probe contract — metallb controller and speaker serve /healthz and /readyz on a dedicated host-local health port 17472 (distinct from the metrics port). The frr-k8s controller uses its own host-local probe port 7572. The cozystack-built image must move in lockstep with the chart so probes resolve against the new endpoints. - 0.16.0: kube-rbac-proxy is replaced by native TLS; metrics endpoints are HTTPS-only, served on the formerly-rbac-proxy port 9120. Default TLS minimum is now 1.3 (was 1.2); operators with TLS 1.2 scrapers override via metallb.tls.minVersion: VersionTLS12. - 0.16.0: new tls.* and networkpolicies.* value blocks (opt-in defaults). - 0.16.1: chart-only patch on top of 0.16.0 — the liveness/readiness probe port is now a configurable value (default unchanged at 17472) and the controller probes drop the hardcoded host: 127.0.0.1; pods, ServiceMonitor and PodMonitor gain prometheus.io/scheme: https for the HTTPS metrics endpoint; the frr-k8s subchart gains an opt-in ServiceMonitor toggle (default off); one CRD numeric field widens from int32 to int64. - 0.15.3: ConfigurationState CRD exposes reconcile errors per component; Helm chart gained NetworkPolicy support and tightened pod SecurityContext. Adaptations: - packages/system/metallb/Makefile: pin helm pull --version 0.16.1; remove the rm -rf charts/metallb/charts/frr-k8s line so the new default BGP backend ships with the chart; add a test: target running helm unittest; hardcode --platform=linux/amd64 on the buildx invocations so the image arch does not depend on the contributor's host. - packages/system/metallb/charts/metallb/: regenerated via make update. - packages/system/metallb/values.yaml: cozystack-built controller and speaker image tags moved to v0.16.1 with fresh linux/amd64 digests pushed to ghcr.io/cozystack/cozystack. The chart's new probe contract requires the v0.16.x binary; leaving the digests on v0.15.2 would CrashLoopBackOff every pod on main between merge and the next release-prep commit. A header comment documents the backend default switch, the metrics-port move (7472 HTTP → 9120 HTTPS), the new TLS 1.3 default, the new host-network ports (9140, 9141, 19443), and the partial nature of the FRR-mode escape hatch — flipping metallb.frrk8s.enabled=false plus metallb.speaker.frr.enabled=true switches the speaker but does not suppress the idle frr-k8s subchart, because Helm evaluates a dep's `condition:` against the declaring chart's own values context and a grandparent override from the umbrella cannot reach that resolution. - packages/system/metallb/tests/metallb_test.yaml: new helm-unittest suite pinning post-bump invariants — speaker is the single-container daemonset, BGP_TYPE is frr-k8s, frr-k8s daemonset and CRDs are installed, NetworkPolicies stay off by default, controller/speaker image tags exactly match the chart appVersion (the test that would have caught the chart / binary skew up-front), and the documented FRR-mode escape hatch is pinned in both halves (speaker switches to four-container FRR layout, frr-k8s subchart still renders idle — the latter test is the contract that a future cleanup must update together with the values.yaml note). - packages/system/cilium-networkpolicy/templates/networkpolicy.yaml: drop the stale FRR-mode metrics port 7473; add every metallb v0.16 host-network listener that this policy file is responsible for — 7946 (speaker memberlist gossip, TCP/UDP; cluster peers still reach it via the same policy's ingress fromEntities[cluster,host] clause), 9120 (speaker metrics), 9121 (legacy FRR metrics, kept defensively so the documented escape hatch stays safe), 9140 (frr-k8s controller metrics), 9141 (frr-k8s FRR metrics), 19443 (frr-k8s statuscleaner webhook). Port 9443 stays in the deny list with a tightened comment: the host-network bind belongs to the etcd kube-rbac-proxy DaemonSet in cozy-monitoring; the metallb controller webhook collides on the same port number but on pod-IP, out of scope for a world-deny. - packages/system/cilium-networkpolicy/tests/networkpolicy_test.yaml: new helm-unittest suite that enumerates every metallb v0.16 host-network port in the deny list (including 7946 memberlist and the escape-hatch 9121), pins port 9443 to its etcd kube-rbac-proxy owner with the pod-IP collision note, and asserts the legacy 7473 entry is gone. Assisted-By: Claude Signed-off-by: Aleksei Sviridkin --- packages/system/cilium-networkpolicy/Makefile | 3 + .../templates/networkpolicy.yaml | 19 +- .../tests/networkpolicy_test.yaml | 87 +++ packages/system/metallb/Makefile | 7 +- .../system/metallb/charts/metallb/Chart.lock | 8 +- .../system/metallb/charts/metallb/Chart.yaml | 8 +- .../system/metallb/charts/metallb/README.md | 41 +- .../charts/metallb/charts/crds/Chart.yaml | 4 +- .../metallb/charts/crds/templates/crds.yaml | 280 +++++++++- .../charts/metallb/charts/frr-k8s/.helmignore | 23 + .../charts/metallb/charts/frr-k8s/Chart.lock | 6 + .../charts/metallb/charts/frr-k8s/Chart.yaml | 16 + .../charts/metallb/charts/frr-k8s/README.md | 98 ++++ .../charts/frr-k8s/charts/crds/.helmignore | 23 + .../charts/frr-k8s/charts/crds/Chart.yaml | 10 + .../charts/frr-k8s/charts/crds/README.md | 14 + .../frrk8s.metallb.io_bgpsessionstates.yaml | 77 +++ .../frrk8s.metallb.io_frrconfigurations.yaml | 501 ++++++++++++++++++ ...rrk8s.metallb.io_frrk8sconfigurations.yaml | 65 +++ .../frrk8s.metallb.io_frrnodestates.yaml | 65 +++ .../charts/frr-k8s/templates/NOTES.txt | 4 + .../charts/frr-k8s/templates/_helpers.tpl | 63 +++ .../charts/frr-k8s/templates/controller.yaml | 462 ++++++++++++++++ .../charts/frr-k8s/templates/rbac.yaml | 113 ++++ .../frr-k8s/templates/service-accounts.yaml | 16 + .../frr-k8s/templates/service-monitor.yaml | 128 +++++ .../frr-k8s/templates/status-cleaner.yaml | 129 +++++ .../charts/frr-k8s/templates/webhooks.yaml | 43 ++ .../metallb/charts/frr-k8s/values.schema.json | 394 ++++++++++++++ .../charts/metallb/charts/frr-k8s/values.yaml | 215 ++++++++ .../charts/metallb/policy/controller.rego | 2 + .../charts/metallb/policy/speaker.rego | 4 + .../charts/metallb/templates/NOTES.txt | 7 + .../charts/metallb/templates/_helpers.tpl | 17 +- .../charts/metallb/templates/controller.yaml | 68 +-- .../metallb/templates/exclude-l2-config.yaml | 3 +- .../metallb/templates/networkpolicy.yaml | 42 ++ .../charts/metallb/templates/podmonitor.yaml | 2 + .../charts/metallb/templates/rbac.yaml | 16 +- .../metallb/templates/servicemonitor.yaml | 30 +- .../charts/metallb/templates/speaker.yaml | 194 ++++--- .../metallb/charts/metallb/values.schema.json | 37 +- .../system/metallb/charts/metallb/values.yaml | 107 ++-- .../system/metallb/tests/metallb_test.yaml | 185 +++++++ packages/system/metallb/values.yaml | 57 +- 45 files changed, 3424 insertions(+), 269 deletions(-) create mode 100644 packages/system/cilium-networkpolicy/tests/networkpolicy_test.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/.helmignore create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.lock create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/README.md create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/.helmignore create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/Chart.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/README.md create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_bgpsessionstates.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrconfigurations.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrk8sconfigurations.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrnodestates.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/NOTES.txt create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/_helpers.tpl create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/controller.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/rbac.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-accounts.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-monitor.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/status-cleaner.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/templates/webhooks.yaml create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/values.schema.json create mode 100644 packages/system/metallb/charts/metallb/charts/frr-k8s/values.yaml create mode 100644 packages/system/metallb/charts/metallb/templates/networkpolicy.yaml create mode 100644 packages/system/metallb/tests/metallb_test.yaml diff --git a/packages/system/cilium-networkpolicy/Makefile b/packages/system/cilium-networkpolicy/Makefile index a24a54b51..bc2cbfb8c 100644 --- a/packages/system/cilium-networkpolicy/Makefile +++ b/packages/system/cilium-networkpolicy/Makefile @@ -3,3 +3,6 @@ export NAMESPACE=cozy-$(NAME) include ../../../hack/common-envs.mk include ../../../hack/package.mk + +test: + helm unittest . diff --git a/packages/system/cilium-networkpolicy/templates/networkpolicy.yaml b/packages/system/cilium-networkpolicy/templates/networkpolicy.yaml index 3c0cf1af2..ef12d5d88 100644 --- a/packages/system/cilium-networkpolicy/templates/networkpolicy.yaml +++ b/packages/system/cilium-networkpolicy/templates/networkpolicy.yaml @@ -12,12 +12,27 @@ spec: - port: "2379" # etcd - port: "2380" # etcd - port: "3367" # linstor - - port: "7473" # frr-metrics (metallb) + # metallb speaker memberlist gossip is bound on both TCP and UDP on + # host-network. Cilium PortProtocol matches all transport protocols + # (TCP+UDP+SCTP) when `protocol:` is "ANY", omitted, or empty — so the + # other entries in this list already cover their (single-protocol) + # listeners. The explicit `protocol: ANY` on 7946 is documentation of + # intent for the one multi-protocol listener in the set; it is not a + # behavior change vs. omitting the field. Cluster peers still reach + # 7946 via the ingress fromEntities[cluster,host] clause below + # regardless of the deny. + - port: "7946" + protocol: ANY - port: "8123" # cozy assets server - - port: "9443" # kube-rbac-proxy + - port: "9120" # metallb speaker metrics (HTTPS, host-network) + - port: "9121" # metallb legacy FRR metrics (HTTPS, host-network; only bound when speaker.frr.enabled=true escape hatch is engaged) + - port: "9140" # metallb frr-k8s controller metrics (HTTPS, host-network) + - port: "9141" # metallb frr-k8s FRR metrics (HTTPS, host-network) + - port: "9443" # etcd kube-rbac-proxy host-network bind (cozy-monitoring DaemonSet); metallb controller also binds 9443 on pod-IP for its webhook — pod-IP is out of scope for this world-deny but the port number collides - port: "10250" # kubelet - port: "10257" # kube-controller-manager - port: "10259" # kube-scheduler + - port: "19443" # metallb frr-k8s statuscleaner webhook (HTTPS, host-network) ingress: - fromEntities: - world diff --git a/packages/system/cilium-networkpolicy/tests/networkpolicy_test.yaml b/packages/system/cilium-networkpolicy/tests/networkpolicy_test.yaml new file mode 100644 index 000000000..b16f6fa70 --- /dev/null +++ b/packages/system/cilium-networkpolicy/tests/networkpolicy_test.yaml @@ -0,0 +1,87 @@ +suite: cilium-networkpolicy port-deny invariants +release: + name: cilium-networkpolicy + namespace: cozy-cilium-networkpolicy +templates: + - templates/networkpolicy.yaml +tests: + - it: denies world ingress to every metallb v0.16 host-network listener + # These ports are opened on host-network by metallb v0.16: + # 7946 — speaker memberlist gossip (TCP/UDP). Cluster peers still + # reach it via the ingress fromEntities[cluster,host] clause + # on the same policy; the deny only blocks `world`. + # 9120 — speaker metrics (HTTPS). + # 9121 — legacy FRR metrics (HTTPS, only bound when the escape hatch + # speaker.frr.enabled=true is engaged; kept in the denylist + # defensively so the escape hatch stays safe). + # 9140 — frr-k8s controller metrics (HTTPS, DaemonSet). + # 9141 — frr-k8s FRR metrics (HTTPS, DaemonSet). + # 19443 — frr-k8s statuscleaner webhook (HTTPS, Deployment with + # hostNetwork: true and master+control-plane tolerations). + # The deny list MUST cover all six; missing any one leaks the + # corresponding endpoint to world on any node that schedules the pod. + # Port 9443 is bound on host-network by the cozy-monitoring etcd + # kube-rbac-proxy DaemonSet — asserted separately below; metallb + # controller webhook collides on the same port number but on pod-IP, + # out of scope for this world-deny. + asserts: + # 7946 carries memberlist gossip on both TCP and UDP. Cilium's + # PortProtocol matches all transport protocols when "ANY", omitted, + # or empty, so this entry would be a functional no-op without + # `protocol: ANY` too — the assertion pins the explicit-intent + # marker as the contract for the one multi-protocol listener, so a + # future edit that drops `protocol: ANY` (and silently relies on + # the implicit match) fails this test. + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "7946" + protocol: ANY + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "9120" + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "9121" + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "9140" + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "9141" + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "19443" + + - it: denies world ingress to the host-network 9443 listener (etcd kube-rbac-proxy) + # The host-network bind on port 9443 belongs to the cozy-monitoring + # kube-rbac-proxy DaemonSet + # (packages/system/monitoring-agents/templates/etcd-proxy-scrape.yaml, + # --secure-listen-address=$(NODE_IP):9443). Port 9443 is ALSO bound by + # the metallb controller Deployment as its webhook server, but on pod-IP + # (non-host-network), which is out of scope for this world-deny. Pin the + # entry here so a future PR that drops the cozy-monitoring stack does + # not silently remove this deny — the port-number collision with the + # metallb controller webhook does not give that webhook any protection + # under this policy regardless. + asserts: + - contains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "9443" + + - it: drops the stale frr-mode metrics port now that frrk8s is the default + # Port 7473 was metallb's legacy FRR-mode metrics endpoint. v0.16 disables + # FRR mode by default and migrates BGP to frr-k8s; the port is no longer + # bound by anything cozystack ships, so listing it in the deny rule is + # misleading. + asserts: + - notContains: + path: spec.ingressDeny[0].toPorts[0].ports + content: + port: "7473" diff --git a/packages/system/metallb/Makefile b/packages/system/metallb/Makefile index 5262e558c..55db17165 100644 --- a/packages/system/metallb/Makefile +++ b/packages/system/metallb/Makefile @@ -8,13 +8,16 @@ update: rm -rf charts helm repo add metallb https://metallb.github.io/metallb helm repo update metallb - helm pull metallb/metallb --untar --untardir charts - rm -rf charts/metallb/charts/frr-k8s + helm pull metallb/metallb --version 0.16.1 --untar --untardir charts + +test: + helm unittest . image-controller image-speaker: $(eval TARGET := $(subst image-,,$@)) $(eval VERSION := $(shell yq '.appVersion' charts/metallb/Chart.yaml)) docker buildx build images/metallb \ + --platform=linux/amd64 \ --target $(TARGET) \ --build-arg VERSION=$(VERSION) \ --tag $(REGISTRY)/metallb-$(TARGET):$(VERSION) \ diff --git a/packages/system/metallb/charts/metallb/Chart.lock b/packages/system/metallb/charts/metallb/Chart.lock index 22f2f5b56..7b863d009 100644 --- a/packages/system/metallb/charts/metallb/Chart.lock +++ b/packages/system/metallb/charts/metallb/Chart.lock @@ -1,9 +1,9 @@ dependencies: - name: crds repository: "" - version: 0.15.2 + version: 0.16.1 - name: frr-k8s repository: https://metallb.github.io/frr-k8s - version: 0.0.20 -digest: sha256:fcba6d1c8e25efca7a64d2ea30079bfcf041c277605d5881d65d5b3d09a41483 -generated: "2025-06-04T12:37:51.709832109+02:00" + version: 0.0.25 +digest: sha256:91f6f08e93bcfc87c9388e5b063b814754151fced237f489cc84feaf5cf13584 +generated: "2026-05-27T22:48:56.242027047+02:00" diff --git a/packages/system/metallb/charts/metallb/Chart.yaml b/packages/system/metallb/charts/metallb/Chart.yaml index 175370dcb..0efda70d4 100644 --- a/packages/system/metallb/charts/metallb/Chart.yaml +++ b/packages/system/metallb/charts/metallb/Chart.yaml @@ -1,14 +1,14 @@ apiVersion: v2 -appVersion: v0.15.2 +appVersion: v0.16.1 dependencies: - condition: crds.enabled name: crds repository: "" - version: 0.15.2 + version: 0.16.1 - condition: frrk8s.enabled name: frr-k8s repository: https://metallb.github.io/frr-k8s - version: 0.0.20 + version: 0.0.25 description: A network load-balancer implementation for Kubernetes using standard routing protocols home: https://metallb.universe.tf @@ -18,4 +18,4 @@ name: metallb sources: - https://github.com/metallb/metallb type: application -version: 0.15.2 +version: 0.16.1 diff --git a/packages/system/metallb/charts/metallb/README.md b/packages/system/metallb/charts/metallb/README.md index c4f12883d..f08870eb4 100644 --- a/packages/system/metallb/charts/metallb/README.md +++ b/packages/system/metallb/charts/metallb/README.md @@ -17,7 +17,7 @@ Kubernetes: `>= 1.19.0-0` | Repository | Name | Version | |------------|------|---------| | | crds | 0.0.0 | -| https://metallb.github.io/frr-k8s | frr-k8s | 0.0.20 | +| https://metallb.github.io/frr-k8s | frr-k8s | 0.0.25 | ## Values @@ -34,6 +34,7 @@ Kubernetes: `>= 1.19.0-0` | controller.livenessProbe.failureThreshold | int | `3` | | | controller.livenessProbe.initialDelaySeconds | int | `10` | | | controller.livenessProbe.periodSeconds | int | `10` | | +| controller.livenessProbe.port | int | `17472` | | | controller.livenessProbe.successThreshold | int | `1` | | | controller.livenessProbe.timeoutSeconds | int | `1` | | | controller.logLevel | string | `"info"` | Controller log level. Must be one of: `all`, `debug`, `info`, `warn`, `error` or `none` | @@ -44,6 +45,7 @@ Kubernetes: `>= 1.19.0-0` | controller.readinessProbe.failureThreshold | int | `3` | | | controller.readinessProbe.initialDelaySeconds | int | `10` | | | controller.readinessProbe.periodSeconds | int | `10` | | +| controller.readinessProbe.port | int | `17472` | | | controller.readinessProbe.successThreshold | int | `1` | | | controller.readinessProbe.timeoutSeconds | int | `1` | | | controller.resources | object | `{}` | | @@ -55,20 +57,22 @@ Kubernetes: `>= 1.19.0-0` | controller.serviceAccount.create | bool | `true` | | | controller.serviceAccount.name | string | `""` | | | controller.strategy.type | string | `"RollingUpdate"` | | -| controller.tlsCipherSuites | string | `""` | | -| controller.tlsMinVersion | string | `"VersionTLS12"` | | | controller.tolerations | list | `[]` | | +| controller.webhookMode | string | `"enabled"` | | | crds.enabled | bool | `true` | | | crds.validationFailurePolicy | string | `"Fail"` | | -| frrk8s.enabled | bool | `false` | | -| frrk8s.external | bool | `false` | | -| frrk8s.namespace | string | `""` | | +| frr-k8s.prometheus.serviceMonitor.enabled | bool | `false` | Enable Prometheus ServiceMonitor for frr-k8s metrics. | +| frrk8s.enabled | bool | `true` | If set, enables frrk8s as a backend. This is mutually exclusive to frr mode. | +| frrk8s.external | bool | `false` | If true, uses an external frr-k8s installation instead of the bundled subchart. | +| frrk8s.namespace | string | `""` | Namespace where external frr-k8s is installed (only used when external=true). | | fullnameOverride | string | `""` | | | imagePullSecrets | list | `[]` | | | loadBalancerClass | string | `""` | | | nameOverride | string | `""` | | -| prometheus.controllerMetricsTLSSecret | string | `""` | | -| prometheus.metricsPort | int | `7472` | | +| networkpolicies.apiPort | int | `6443` | | +| networkpolicies.defaultDeny | bool | `false` | | +| networkpolicies.enabled | bool | `false` | | +| prometheus.metricsPort | int | `9120` | | | prometheus.namespace | string | `""` | | | prometheus.podMonitor.additionalLabels | object | `{}` | | | prometheus.podMonitor.annotations | object | `{}` | | @@ -99,9 +103,6 @@ Kubernetes: `>= 1.19.0-0` | prometheus.prometheusRule.staleConfig.enabled | bool | `true` | | | prometheus.prometheusRule.staleConfig.labels.severity | string | `"warning"` | | | prometheus.rbacPrometheus | bool | `true` | | -| prometheus.rbacProxy.pullPolicy | string | `nil` | | -| prometheus.rbacProxy.repository | string | `"gcr.io/kubebuilder/kube-rbac-proxy"` | | -| prometheus.rbacProxy.tag | string | `"v0.12.0"` | | | prometheus.scrapeAnnotations | bool | `false` | | | prometheus.serviceAccount | string | `""` | | | prometheus.serviceMonitor.controller.additionalLabels | object | `{}` | | @@ -115,28 +116,32 @@ Kubernetes: `>= 1.19.0-0` | prometheus.serviceMonitor.speaker.additionalLabels | object | `{}` | | | prometheus.serviceMonitor.speaker.annotations | object | `{}` | | | prometheus.serviceMonitor.speaker.tlsConfig.insecureSkipVerify | bool | `true` | | -| prometheus.speakerMetricsTLSSecret | string | `""` | | | rbac.create | bool | `true` | | | speaker.affinity | object | `{}` | | +| speaker.bgpDebounceTimeout | string | `nil` | BGP debounce timeout for FRR configuration reloads, in milliseconds. Only applies when BGP type is frr. Default (when unset) is 3000 ms. This feature is experimental | | speaker.enabled | bool | `true` | | | speaker.excludeInterfaces.enabled | bool | `true` | | | speaker.extraContainers | list | `[]` | | -| speaker.frr.enabled | bool | `true` | | +| speaker.frr.enabled | bool | `false` | | | speaker.frr.image.pullPolicy | string | `nil` | | | speaker.frr.image.repository | string | `"quay.io/frrouting/frr"` | | -| speaker.frr.image.tag | string | `"9.1.0"` | | -| speaker.frr.metricsPort | int | `7473` | | +| speaker.frr.image.tag | string | `"10.5.3"` | | +| speaker.frr.metricsPort | int | `9121` | | | speaker.frr.resources | object | `{}` | | | speaker.frrMetrics.resources | object | `{}` | | | speaker.ignoreExcludeLB | bool | `false` | | | speaker.image.pullPolicy | string | `nil` | | | speaker.image.repository | string | `"quay.io/metallb/speaker"` | | | speaker.image.tag | string | `nil` | | +| speaker.initContainers.cpFrrFiles.resources | object | `{}` | | +| speaker.initContainers.cpMetrics.resources | object | `{}` | | +| speaker.initContainers.cpReloader.resources | object | `{}` | | | speaker.labels | object | `{}` | | | speaker.livenessProbe.enabled | bool | `true` | | | speaker.livenessProbe.failureThreshold | int | `3` | | | speaker.livenessProbe.initialDelaySeconds | int | `10` | | | speaker.livenessProbe.periodSeconds | int | `10` | | +| speaker.livenessProbe.port | int | `17472` | | | speaker.livenessProbe.successThreshold | int | `1` | | | speaker.livenessProbe.timeoutSeconds | int | `1` | | | speaker.logLevel | string | `"info"` | Speaker log level. Must be one of: `all`, `debug`, `info`, `warn`, `error` or `none` | @@ -151,6 +156,7 @@ Kubernetes: `>= 1.19.0-0` | speaker.readinessProbe.failureThreshold | int | `3` | | | speaker.readinessProbe.initialDelaySeconds | int | `10` | | | speaker.readinessProbe.periodSeconds | int | `10` | | +| speaker.readinessProbe.port | int | `17472` | | | speaker.readinessProbe.successThreshold | int | `1` | | | speaker.readinessProbe.timeoutSeconds | int | `1` | | | speaker.reloader.resources | object | `{}` | | @@ -166,6 +172,11 @@ Kubernetes: `>= 1.19.0-0` | speaker.tolerateMaster | bool | `true` | | | speaker.tolerations | list | `[]` | | | speaker.updateStrategy.type | string | `"RollingUpdate"` | | +| tls.cipherSuites | string | `""` | Comma-separated list of TLS cipher suites. If empty, uses Go defaults. Only applies to TLS 1.2. | +| tls.controllerMetricsTLSSecret | string | `""` | The name of the secret to be mounted in the controller pod to provide TLS certificates for metrics endpoints. If not present, a self-signed certificate is auto-generated. | +| tls.curvePreferences | string | `""` | Comma-separated list of numeric CurveID values (e.g. 29,4588). See https://pkg.go.dev/crypto/tls#CurveID. If empty, uses Go defaults. | +| tls.minVersion | string | `""` | Minimum TLS version (VersionTLS12 or VersionTLS13). Defaults to VersionTLS13. | +| tls.speakerMetricsTLSSecret | string | `""` | The name of the secret to be mounted in the speaker pod to provide TLS certificates for metrics endpoints. If not present, a self-signed certificate is auto-generated. | ---------------------------------------------- Autogenerated from chart metadata using [helm-docs v1.10.0](https://github.com/norwoodj/helm-docs/releases/v1.10.0) diff --git a/packages/system/metallb/charts/metallb/charts/crds/Chart.yaml b/packages/system/metallb/charts/metallb/charts/crds/Chart.yaml index e88450b13..40bc274bf 100644 --- a/packages/system/metallb/charts/metallb/charts/crds/Chart.yaml +++ b/packages/system/metallb/charts/metallb/charts/crds/Chart.yaml @@ -1,5 +1,5 @@ apiVersion: v2 -appVersion: v0.15.2 +appVersion: v0.16.1 description: MetalLB CRDs home: https://metallb.universe.tf icon: https://metallb.universe.tf/images/logo/metallb-white.png @@ -7,4 +7,4 @@ name: crds sources: - https://github.com/metallb/metallb type: application -version: 0.15.2 +version: 0.16.1 diff --git a/packages/system/metallb/charts/metallb/charts/crds/templates/crds.yaml b/packages/system/metallb/charts/metallb/charts/crds/templates/crds.yaml index 3821ae76c..07347dccc 100644 --- a/packages/system/metallb/charts/metallb/charts/crds/templates/crds.yaml +++ b/packages/system/metallb/charts/metallb/charts/crds/templates/crds.yaml @@ -2,7 +2,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: bfdprofiles.metallb.io spec: group: metallb.io @@ -123,7 +123,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: bgpadvertisements.metallb.io spec: group: metallb.io @@ -315,7 +315,64 @@ spec: items: type: string type: array + serviceSelectors: + description: |- + ServiceSelectors limits the set of services that will be advertised via this advertisement. + If empty, all services from the selected pools are advertised. + This field is mutually exclusive with aggregationLength and aggregationLengthV6 - + services can only be selected when using the default /32 (IPv4) or /128 (IPv6) aggregation. + items: + description: |- + A label selector is a label query over a set of resources. The result of matchLabels and + matchExpressions are ANDed. An empty label selector matches all objects. A null + label selector matches no objects. + properties: + matchExpressions: + description: matchExpressions is a list of label selector requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + type: array type: object + x-kubernetes-validations: + - message: serviceSelectors and aggregationLength are mutually exclusive, can only be used together with default aggregationLength (32) and aggregationLengthV6 (128) + rule: '!has(self.serviceSelectors) || self.serviceSelectors.size() == 0 || ((!has(self.aggregationLength) || self.aggregationLength == 32) && (!has(self.aggregationLengthV6) || self.aggregationLengthV6 == 128))' status: description: BGPAdvertisementStatus defines the observed state of BGPAdvertisement. type: object @@ -329,7 +386,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: bgppeers.metallb.io spec: conversion: @@ -404,7 +461,7 @@ spec: type: string myASN: description: AS number to use for the local end of the session. - format: int32 + format: int64 maximum: 4294967295 minimum: 0 type: integer @@ -443,7 +500,7 @@ spec: type: string peerASN: description: AS number to expect from the remote end of the session. - format: int32 + format: int64 maximum: 4294967295 minimum: 0 type: integer @@ -547,14 +604,14 @@ spec: - external type: string ebgpMultiHop: - description: To set if the BGPPeer is multi-hops away. Needed for FRR mode only. + description: To set if the BGPPeer is multi-hops away. Needed for FRR-based modes (FRR-K8s, FRR) only. type: boolean enableGracefulRestart: description: |- EnableGracefulRestart allows BGP peer to continue to forward data packets along known routes while the routing protocol information is being restored. This field is immutable because it requires restart of the BGP - session. Supported for FRR mode only. + session. Supported for FRR-based modes (FRR-K8s, FRR) only. type: boolean x-kubernetes-validations: - message: EnableGracefulRestart cannot be changed after creation @@ -573,9 +630,20 @@ spec: keepaliveTime: description: Requested BGP keepalive time, per RFC4271. type: string + localASN: + description: |- + LocalASN allows advertising a different AS number to the peer using BGP's + local-as feature. When set, MetalLB will advertise this ASN to the peer + via "neighbor local-as no-prepend replace-as", overriding + the router-level MyASN for this specific session. + Not supported in native BGP mode. + format: int64 + maximum: 4294967295 + minimum: 1 + type: integer myASN: description: AS number to use for the local end of the session. - format: int32 + format: int64 maximum: 4294967295 minimum: 0 type: integer @@ -653,7 +721,7 @@ spec: description: |- AS number to expect from the remote end of the session. ASN and DynamicASN are mutually exclusive and one of them must be specified. - format: int32 + format: int64 maximum: 4294967295 minimum: 0 type: integer @@ -693,7 +761,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: communities.metallb.io spec: group: metallb.io @@ -758,7 +826,139 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 + name: configurationstates.metallb.io +spec: + group: metallb.io + names: + kind: ConfigurationState + listKind: ConfigurationStateList + plural: configurationstates + singular: configurationstate + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.result + name: Result + type: string + - jsonPath: .status.errorSummary + name: ErrorSummary + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1beta1 + schema: + openAPIV3Schema: + description: |- + ConfigurationState is a status-only CRD that reports configuration validation results from MetalLB components. + Labels: + - metallb.io/component-type: "controller" or "speaker" + - metallb.io/node-name: node name (only for speaker) + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: ConfigurationStateStatus defines the observed state of ConfigurationState. + properties: + conditions: + description: Conditions contains the status conditions from the reconcilers running in this component. + items: + description: Condition contains details for one aspect of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + errorSummary: + description: |- + ErrorSummary contains the aggregated error messages from reconciliation failures. + This field is empty when Result is "Valid". + type: string + result: + description: Result indicates the configuration validation result. + enum: + - Valid + - Invalid + - Unknown + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.19.0 name: ipaddresspools.metallb.io spec: group: metallb.io @@ -990,7 +1190,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: l2advertisements.metallb.io spec: group: metallb.io @@ -1156,6 +1356,58 @@ spec: type: object x-kubernetes-map-type: atomic type: array + serviceSelectors: + description: |- + ServiceSelectors limits the set of services that will be advertised via this advertisement. + If empty, all services from the selected pools are advertised. + items: + description: |- + A label selector is a label query over a set of resources. The result of matchLabels and + matchExpressions are ANDed. An empty label selector matches all objects. A null + label selector matches no objects. + properties: + matchExpressions: + description: matchExpressions is a list of label selector requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + type: array type: object status: description: L2AdvertisementStatus defines the observed state of L2Advertisement. @@ -1170,7 +1422,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: servicebgpstatuses.metallb.io spec: group: metallb.io @@ -1255,7 +1507,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.17.2 + controller-gen.kubebuilder.io/version: v0.19.0 name: servicel2statuses.metallb.io spec: group: metallb.io diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/.helmignore b/packages/system/metallb/charts/metallb/charts/frr-k8s/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.lock b/packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.lock new file mode 100644 index 000000000..a908b0f20 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.lock @@ -0,0 +1,6 @@ +dependencies: +- name: crds + repository: "" + version: 0.0.25 +digest: sha256:fadaadfc20fb2ce98510dd77b848ebd867a5d2e95a253120fedd5af31e766dd8 +generated: "2026-05-07T19:06:46.617697725+02:00" diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.yaml new file mode 100644 index 000000000..99a1970e7 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/Chart.yaml @@ -0,0 +1,16 @@ +apiVersion: v2 +appVersion: v0.0.25 +dependencies: +- condition: crds.enabled + name: crds + repository: "" + version: 0.0.25 +description: A cloud native wrapper of FRR +home: https://metallb.universe.tf +icon: https://metallb.universe.tf/images/logo/metallb-white.png +kubeVersion: '>= 1.19.0-0' +name: frr-k8s +sources: +- https://github.com/metallb/frr-k8s +type: application +version: 0.0.25 diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/README.md b/packages/system/metallb/charts/metallb/charts/frr-k8s/README.md new file mode 100644 index 000000000..4a9399132 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/README.md @@ -0,0 +1,98 @@ +# frr-k8s + +![Version: 0.0.25](https://img.shields.io/badge/Version-0.0.25-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.0.25](https://img.shields.io/badge/AppVersion-v0.0.25-informational?style=flat-square) + +A cloud native wrapper of FRR + +**Homepage:** + +## Source Code + +* + +## Requirements + +Kubernetes: `>= 1.19.0-0` + +| Repository | Name | Version | +|------------|------|---------| +| | crds | 0.0.25 | + +## Values + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| crds.enabled | bool | `true` | Enable installation of CRDs. | +| crds.validationFailurePolicy | string | `"Fail"` | Validation failure policy for CRDs. Can be Fail or Ignore. | +| frrk8s.affinity | object | `{}` | Affinity for pod assignment. | +| frrk8s.alwaysBlock | string | `""` | A comma separated list of cidrs to always block for incoming routes. | +| frrk8s.bgpDebounceTimeout | integer | `nil` | BGP debounce timeout for FRR configuration reloads, in milliseconds. Default (when unset) is 3000 ms.This feature is experimental | +| frrk8s.disableCertRotation | bool | `false` | Specifies whether the cert rotator works as part of the webhook. | +| frrk8s.frr.acceptIncomingBGPConnections | bool | `false` | Allow FRR to accept incoming BGP connections. | +| frrk8s.frr.image.pullPolicy | string | `nil` | The FRR image pull policy. | +| frrk8s.frr.image.repository | string | `"quay.io/frrouting/frr"` | The FRR image repository. | +| frrk8s.frr.image.tag | string | `"10.4.3"` | The FRR image tag. | +| frrk8s.frr.metricsBindAddress | string | `"127.0.0.1"` | Bind address for FRR metrics. | +| frrk8s.frr.metricsPort | int | `7573` | Port for FRR metrics. | +| frrk8s.frr.resources | object | `{}` | Resource limits and requests for the FRR container. | +| frrk8s.frr.secureMetricsPort | int | `9141` | Secure metrics port for FRR. | +| frrk8s.frrMetrics.resources | object | `{}` | Resource limits and requests for the FRR metrics container. | +| frrk8s.frrStatus.pollInterval | string | `"2m"` | Polling interval for FRR status updates. | +| frrk8s.frrStatus.resources | object | `{}` | Resource limits and requests for the FRR status container. | +| frrk8s.image.pullPolicy | string | `nil` | The frr-k8s image pull policy. | +| frrk8s.image.repository | string | `"quay.io/metallb/frr-k8s"` | The frr-k8s image repository. | +| frrk8s.image.tag | string | `nil` | The frr-k8s image tag. If not set, defaults to the chart appVersion. | +| frrk8s.labels | object | `{"app":"frr-k8s"}` | Additional labels to add to the pod. | +| frrk8s.livenessProbe.enabled | bool | `true` | Enable liveness probe. | +| frrk8s.livenessProbe.failureThreshold | int | `3` | Number of failures before the probe is considered failed. | +| frrk8s.livenessProbe.initialDelaySeconds | int | `10` | Number of seconds after the container has started before liveness probes are initiated. | +| frrk8s.livenessProbe.periodSeconds | int | `10` | How often (in seconds) to perform the probe. | +| frrk8s.livenessProbe.successThreshold | int | `1` | Minimum consecutive successes for the probe to be considered successful. | +| frrk8s.livenessProbe.timeoutSeconds | int | `1` | Number of seconds after which the probe times out. | +| frrk8s.logLevel | string | `"info"` | Controller log level that is passed as a CLI flag. Must be one of: `all`, `debug`, `info`, `warn`, `error` or `none` | +| frrk8s.nodeSelector | object | `{}` | Node selector for pod assignment. | +| frrk8s.podAnnotations | object | `{}` | Additional annotations to add to the pod. | +| frrk8s.priorityClassName | string | `""` | Priority class name for the pod. | +| frrk8s.readinessProbe.enabled | bool | `true` | Enable readiness probe. | +| frrk8s.readinessProbe.failureThreshold | int | `3` | Number of failures before the probe is considered failed. | +| frrk8s.readinessProbe.initialDelaySeconds | int | `10` | Number of seconds after the container has started before readiness probes are initiated. | +| frrk8s.readinessProbe.periodSeconds | int | `10` | How often (in seconds) to perform the probe. | +| frrk8s.readinessProbe.successThreshold | int | `1` | Minimum consecutive successes for the probe to be considered successful. | +| frrk8s.readinessProbe.timeoutSeconds | int | `1` | Number of seconds after which the probe times out. | +| frrk8s.reloader.resources | object | `{}` | Resource limits and requests for the reloader container. | +| frrk8s.resources | object | `{}` | Resource limits and requests for the frr-k8s controller container. | +| frrk8s.restartOnRotatorSecretRefresh | bool | `false` | Specifies whether the pod restarts when the rotator refreshes the cert secret. Useful for webhook stability during redeployments. | +| frrk8s.runtimeClassName | string | `""` | Runtime class name for the pod. | +| frrk8s.serviceAccount.annotations | object | `{}` | Additional annotations to add to the ServiceAccount. | +| frrk8s.serviceAccount.create | bool | `true` | Specifies whether a ServiceAccount should be created. | +| frrk8s.serviceAccount.name | string | `""` | The name of the ServiceAccount to use. If not set and create is true, a name is generated using the fullname template. | +| frrk8s.startupProbe.enabled | bool | `true` | Enable startup probe. | +| frrk8s.startupProbe.failureThreshold | int | `30` | Number of failures before the probe is considered failed. | +| frrk8s.startupProbe.periodSeconds | int | `5` | How often (in seconds) to perform the probe. | +| frrk8s.tolerateMaster | bool | `true` | Tolerate master nodes for pod scheduling. | +| frrk8s.tolerations | list | `[]` | Tolerations for pod assignment. | +| frrk8s.updateStrategy.type | string | `"RollingUpdate"` | Specify the FRR-K8s daemonset update strategy. | +| frrk8s.webhookPort | int | `19443` | Port for the webhook server. | +| fullnameOverride | string | `""` | String to override the default fully qualified app name. | +| nameOverride | string | `""` | String to override the default chart name. | +| prometheus.namespace | string | `""` | The namespace where Prometheus is deployed. Required when ".Values.prometheus.rbacPrometheus == true" and "prometheus.serviceMonitor.enabled=true". | +| prometheus.rbacPrometheus | bool | `false` | Give Prometheus permission to scrape metallb's namespace. | +| prometheus.scrapeAnnotations | bool | `false` | Add Prometheus metric auto-collection annotations to pods. | +| prometheus.secureMetricsPort | int | `9140` | Port frr-k8s will listen on for secure metrics. | +| prometheus.serviceAccount | string | `""` | The service account used by Prometheus. Required when ".Values.prometheus.rbacPrometheus == true" and "prometheus.serviceMonitor.enabled=true" | +| prometheus.serviceMonitor.additionalLabels | object | `{}` | Additional labels to add to the ServiceMonitor. | +| prometheus.serviceMonitor.annotations | object | `{}` | Optional additional annotations for the controller serviceMonitor. | +| prometheus.serviceMonitor.enabled | bool | `false` | Enable support for Prometheus Operator. | +| prometheus.serviceMonitor.interval | string | `nil` | Scrape interval. If not set, the Prometheus default scrape interval is used. | +| prometheus.serviceMonitor.jobLabel | string | `"app.kubernetes.io/name"` | Job label for scrape target. | +| prometheus.serviceMonitor.metricRelabelings | list | `[]` | Metric relabel configs to apply to samples before ingestion. | +| prometheus.serviceMonitor.relabelings | list | `[]` | Relabel configs to apply to samples before ingestion. | +| prometheus.serviceMonitor.tlsConfig.insecureSkipVerify | bool | `true` | Disables SSL certificate verification | +| rbac.create | bool | `true` | Specifies whether to install and use RBAC rules. | +| tls.cipherSuites | string | `""` | Comma-separated list of TLS cipher suites. If empty, uses Go defaults. Only applies to TLS 1.2. | +| tls.curvePreferences | string | `""` | Comma-separated list of numeric CurveID values (e.g. 29,4588). See https://pkg.go.dev/crypto/tls#CurveID. If empty, uses Go defaults. | +| tls.metricsTLSSecret | string | `""` | The name of the secret to be mounted in the pods to provide TLS certificates for metrics endpoints. If not present, a self-signed certificate is auto-generated. | +| tls.minVersion | string | `""` | Minimum TLS version (VersionTLS12 or VersionTLS13). Defaults to VersionTLS13. | + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.10.0](https://github.com/norwoodj/helm-docs/releases/v1.10.0) diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/.helmignore b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/Chart.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/Chart.yaml new file mode 100644 index 000000000..e8a46336f --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/Chart.yaml @@ -0,0 +1,10 @@ +apiVersion: v2 +appVersion: v0.0.25 +description: FRR K8s CRDs +home: https://metallb.universe.tf +icon: https://metallb.universe.tf/images/logo/metallb-white.png +name: crds +sources: +- https://github.com/metallb/frr-k8s +type: application +version: 0.0.25 diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/README.md b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/README.md new file mode 100644 index 000000000..65e636c6d --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/README.md @@ -0,0 +1,14 @@ +# crds + +![Version: 0.0.0](https://img.shields.io/badge/Version-0.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.0.0](https://img.shields.io/badge/AppVersion-v0.0.0-informational?style=flat-square) + +FRR-K8s CRDs + +**Homepage:** + +## Source Code + +* + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.10.0](https://github.com/norwoodj/helm-docs/releases/v1.10.0) diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_bgpsessionstates.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_bgpsessionstates.yaml new file mode 100644 index 000000000..485ddf1f8 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_bgpsessionstates.yaml @@ -0,0 +1,77 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.3 + name: bgpsessionstates.frrk8s.metallb.io +spec: + group: frrk8s.metallb.io + names: + kind: BGPSessionState + listKind: BGPSessionStateList + plural: bgpsessionstates + singular: bgpsessionstate + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.node + name: Node + type: string + - jsonPath: .status.peer + name: Peer + type: string + - jsonPath: .status.vrf + name: VRF + type: string + - jsonPath: .status.bgpStatus + name: BGP + type: string + - jsonPath: .status.bfdStatus + name: BFD + type: string + name: v1beta1 + schema: + openAPIV3Schema: + description: BGPSessionState exposes the status of a BGP Session from the + FRR instance running on the node. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: BGPSessionStateSpec defines the desired state of BGPSessionState. + type: object + status: + description: BGPSessionStateStatus defines the observed state of BGPSessionState. + properties: + bfdStatus: + type: string + bgpStatus: + type: string + node: + type: string + peer: + type: string + vrf: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrconfigurations.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrconfigurations.yaml new file mode 100644 index 000000000..f74fa4648 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrconfigurations.yaml @@ -0,0 +1,501 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.3 + name: frrconfigurations.frrk8s.metallb.io +spec: + group: frrk8s.metallb.io + names: + kind: FRRConfiguration + listKind: FRRConfigurationList + plural: frrconfigurations + singular: frrconfiguration + scope: Namespaced + versions: + - name: v1beta1 + schema: + openAPIV3Schema: + description: FRRConfiguration is a piece of FRR configuration. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: FRRConfigurationSpec defines the desired state of FRRConfiguration. + properties: + bgp: + description: BGP is the configuration related to the BGP protocol. + properties: + bfdProfiles: + description: BFDProfiles is the list of bfd profiles to be used + when configuring the neighbors. + items: + description: |- + BFDProfile is the configuration related to the BFD protocol associated + to a BGP session. + properties: + detectMultiplier: + description: |- + Configures the detection multiplier to determine + packet loss. The remote transmission interval will be multiplied + by this value to determine the connection loss detection timer. + format: int32 + maximum: 255 + minimum: 2 + type: integer + echoInterval: + description: |- + Configures the minimal echo receive transmission + interval that this system is capable of handling in milliseconds. + Defaults to 50ms + format: int32 + maximum: 60000 + minimum: 10 + type: integer + echoMode: + description: |- + Enables or disables the echo transmission mode. + This mode is disabled by default, and not supported on multi + hops setups. + type: boolean + minimumTtl: + description: |- + For multi hop sessions only: configure the minimum + expected TTL for an incoming BFD control packet. + format: int32 + maximum: 254 + minimum: 1 + type: integer + name: + description: |- + The name of the BFD Profile to be referenced in other parts + of the configuration. + type: string + passiveMode: + description: |- + Mark session as passive: a passive session will not + attempt to start the connection and will wait for control packets + from peer before it begins replying. + type: boolean + receiveInterval: + description: |- + The minimum interval that this system is capable of + receiving control packets in milliseconds. + Defaults to 300ms. + format: int32 + maximum: 60000 + minimum: 10 + type: integer + transmitInterval: + description: |- + The minimum transmission interval (less jitter) + that this system wants to use to send BFD control packets in + milliseconds. Defaults to 300ms + format: int32 + maximum: 60000 + minimum: 10 + type: integer + required: + - name + type: object + type: array + routers: + description: Routers is the list of routers we want FRR to configure + (one per VRF). + items: + description: Router represent a neighbor router we want FRR + to connect to. + properties: + asn: + description: ASN is the AS number to use for the local end + of the session. + format: int64 + maximum: 4294967295 + minimum: 0 + type: integer + id: + description: ID is the BGP router ID + type: string + imports: + description: Imports is the list of imported VRFs we want + for this router / vrf. + items: + description: Import represents the possible imported VRFs + to a given router. + properties: + vrf: + description: Vrf is the vrf we want to import from + type: string + type: object + type: array + neighbors: + description: Neighbors is the list of neighbors we want + to establish BGP sessions with. + items: + description: Neighbor represents a BGP Neighbor we want + FRR to connect to. + properties: + address: + description: Address is the IP address to establish + the session with. + type: string + asn: + description: |- + ASN is the AS number to use for the local end of the session. + ASN and DynamicASN are mutually exclusive and one of them must be specified. + format: int64 + maximum: 4294967295 + minimum: 0 + type: integer + bfdProfile: + description: |- + BFDProfile is the name of the BFD Profile to be used for the BFD session associated + to the BGP session. If not set, the BFD session won't be set up. + type: string + connectTime: + description: Requested BGP connect time, controls + how long BGP waits between connection attempts to + a neighbor. + type: string + x-kubernetes-validations: + - message: connect time should be between 1 seconds + to 65535 + rule: duration(self).getSeconds() >= 1 && duration(self).getSeconds() + <= 65535 + - message: connect time should contain a whole number + of seconds + rule: duration(self).getMilliseconds() % 1000 == + 0 + disableMP: + default: false + description: |- + DisableMP is no longer used and has no effect. + Use DualStackAddressFamily instead to enable the neighbor for both IPv4 and IPv6 address families. + + Deprecated: This field is ignored. Use DualStackAddressFamily instead. + type: boolean + dualStackAddressFamily: + default: false + description: |- + To set if we want to enable the neighbor not only for the ipfamily related to its session, + but also the other one. This allows to advertise/receive IPv4 prefixes over IPv6 sessions and vice versa. + type: boolean + dynamicASN: + description: |- + DynamicASN detects the AS number to use for the local end of the session + without explicitly setting it via the ASN field. Limited to: + internal - if the neighbor's ASN is different than the router's the connection is denied. + external - if the neighbor's ASN is the same as the router's the connection is denied. + ASN and DynamicASN are mutually exclusive and one of them must be specified. + enum: + - internal + - external + type: string + ebgpMultiHop: + description: EBGPMultiHop indicates if the BGPPeer + is multi-hops away. + type: boolean + enableGracefulRestart: + description: |- + EnableGracefulRestart allows BGP peer to continue to forward data packets along + known routes while the routing protocol information is being restored. If + the session is already established, the configuration will have effect + after reconnecting to the peer + type: boolean + holdTime: + description: |- + HoldTime is the requested BGP hold time, per RFC4271. + Defaults to 180s. + type: string + interface: + description: |- + Interface is the node interface over which the unnumbered BGP peering will + be established. No API validation takes place as that string value + represents an interface name on the host and if user provides an invalid + value, only the actual BGP session will not be established. + Address and Interface are mutually exclusive and one of them must be specified. + Note: when enabling unnumbered, the neighbor will be enabled for both + IPv4 and IPv6 address families. + type: string + keepaliveTime: + description: |- + KeepaliveTime is the requested BGP keepalive time, per RFC4271. + Defaults to 60s. + type: string + localASN: + description: |- + LocalASN allows advertising a different AS number to the peer using BGP's + local-as feature. When set, FRR will advertise this ASN to the peer + via "neighbor local-as no-prepend replace-as", overriding + the router-level ASN for this specific session. + Note: this field is only applicable to eBGP sessions (where the peer ASN differs + from the router ASN). Setting it on an iBGP session is rejected. + format: int64 + maximum: 4294967295 + minimum: 1 + type: integer + password: + description: |- + Password to be used for establishing the BGP session. + Password and PasswordSecret are mutually exclusive. + type: string + passwordSecret: + description: |- + PasswordSecret is name of the authentication secret for the neighbor. + the secret must be of type "kubernetes.io/basic-auth", and created in the + same namespace as the frr-k8s daemon. The password is stored in the + secret as the key "password". + Password and PasswordSecret are mutually exclusive. + properties: + name: + description: name is unique within a namespace + to reference a secret resource. + type: string + namespace: + description: namespace defines the space within + which the secret name must be unique. + type: string + type: object + x-kubernetes-map-type: atomic + port: + description: |- + Port is the port to dial when establishing the session. + Defaults to 179. + maximum: 16384 + minimum: 0 + type: integer + sourceaddress: + description: |- + SourceAddress is the IPv4 or IPv6 source address to use for the BGP + session to this neighbour, may be specified as either an IP address + directly or as an interface name + type: string + toAdvertise: + description: |- + ToAdvertise represents the list of prefixes to advertise to the given neighbor + and the associated properties. + properties: + allowed: + description: |- + Allowed is is the list of prefixes allowed to be propagated to + this neighbor. They must match the prefixes defined in the router. + properties: + mode: + default: filtered + description: |- + Mode is the mode to use when handling the prefixes. + When set to "filtered", only the prefixes in the given list will be allowed. + When set to "all", all the prefixes configured on the router will be allowed. + enum: + - all + - filtered + type: string + prefixes: + items: + type: string + type: array + type: object + withCommunity: + description: |- + PrefixesWithCommunity is a list of prefixes that are associated to a + bgp community when being advertised. The prefixes associated to a given local pref + must be in the prefixes allowed to be advertised. + items: + description: CommunityPrefixes is a list of + prefixes associated to a community. + properties: + community: + description: Community is the community + associated to the prefixes. + type: string + prefixes: + description: Prefixes is the list of prefixes + associated to the community. + format: cidr + items: + type: string + minItems: 1 + type: array + type: object + type: array + withLocalPref: + description: |- + PrefixesWithLocalPref is a list of prefixes that are associated to a local + preference when being advertised. The prefixes associated to a given local pref + must be in the prefixes allowed to be advertised. + items: + description: LocalPrefPrefixes is a list of + prefixes associated to a local preference. + properties: + localPref: + description: LocalPref is the local preference + associated to the prefixes. + format: int32 + type: integer + prefixes: + description: Prefixes is the list of prefixes + associated to the local preference. + format: cidr + items: + type: string + minItems: 1 + type: array + type: object + type: array + type: object + toReceive: + description: ToReceive represents the list of prefixes + to receive from the given neighbor. + properties: + allowed: + description: |- + Allowed is the list of prefixes allowed to be received from + this neighbor. + properties: + mode: + default: filtered + description: |- + Mode is the mode to use when handling the prefixes. + When set to "filtered", only the prefixes in the given list will be allowed. + When set to "all", all the prefixes configured on the router will be allowed. + enum: + - all + - filtered + type: string + prefixes: + items: + description: PrefixSelector is a filter + of prefixes to receive. + properties: + ge: + description: |- + The prefix length modifier. This selector accepts any matching prefix with length + greater or equal the given value. + format: int32 + maximum: 128 + minimum: 1 + type: integer + le: + description: |- + The prefix length modifier. This selector accepts any matching prefix with length + less or equal the given value. + format: int32 + maximum: 128 + minimum: 1 + type: integer + prefix: + format: cidr + type: string + type: object + type: array + type: object + type: object + type: object + type: array + prefixes: + description: Prefixes is the list of prefixes we want to + advertise from this router instance. + items: + type: string + type: array + vrf: + description: VRF is the host vrf used to establish sessions + from this router. + type: string + required: + - asn + type: object + type: array + type: object + nodeSelector: + description: |- + NodeSelector limits the nodes that will attempt to apply this config. + When specified, the configuration will be considered only on nodes + whose labels match the specified selectors. + When it is not specified all nodes will attempt to apply this config. + properties: + matchExpressions: + description: matchExpressions is a list of label selector requirements. + The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + raw: + description: |- + Raw is a snippet of raw frr configuration that gets appended to the + one rendered translating the type safe API. + properties: + priority: + description: |- + Priority is the order with this configuration is appended to the + bottom of the rendered configuration. A higher value means the + raw config is appended later in the configuration file. + type: integer + rawConfig: + description: |- + Config is a raw FRR configuration to be appended to the configuration + rendered via the k8s api. + type: string + type: object + type: object + status: + description: FRRConfigurationStatus defines the observed state of FRRConfiguration. + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrk8sconfigurations.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrk8sconfigurations.yaml new file mode 100644 index 000000000..340a75dba --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrk8sconfigurations.yaml @@ -0,0 +1,65 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.3 + name: frrk8sconfigurations.frrk8s.metallb.io +spec: + group: frrk8s.metallb.io + names: + kind: FRRK8sConfiguration + listKind: FRRK8sConfigurationList + plural: frrk8sconfigurations + singular: frrk8sconfiguration + scope: Namespaced + versions: + - name: v1beta1 + schema: + openAPIV3Schema: + description: |- + FRRK8sConfiguration holds the FRR Operator configuration with global + settings for the K8s and FRR. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: FRRK8sConfigurationSpec defines the desired state of FRRK8sConfiguration. + properties: + logLevel: + description: |- + LogLevel sets the logging verbosity for the FRR-K8s components at runtime. + When configured, this value overrides the defaults established by the --log-level CLI flag. + Valid values are: all, debug, info, warn, error, none. + enum: + - all + - debug + - info + - warn + - error + - none + type: string + type: object + status: + description: FRRK8sConfigurationStatus defines the observed state of FRRK8sConfiguration. + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrnodestates.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrnodestates.yaml new file mode 100644 index 000000000..037ff6078 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrnodestates.yaml @@ -0,0 +1,65 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.3 + name: frrnodestates.frrk8s.metallb.io +spec: + group: frrk8s.metallb.io + names: + kind: FRRNodeState + listKind: FRRNodeStateList + plural: frrnodestates + singular: frrnodestate + scope: Cluster + versions: + - name: v1beta1 + schema: + openAPIV3Schema: + description: FRRNodeState exposes the status of the FRR instance running on + each node. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: FRRNodeStateSpec defines the desired state of FRRNodeState. + type: object + status: + description: FRRNodeStateStatus defines the observed state of FRRNodeState. + properties: + lastConversionResult: + description: LastConversionResult is the status of the last translation + between the `FRRConfiguration`s resources and FRR's configuration, + contains "success" or an error. + type: string + lastReloadResult: + description: LastReloadResult represents the status of the last configuration + update operation by FRR, contains "success" or an error. + type: string + runningConfig: + description: RunningConfig represents the current FRR running config, + which is the configuration the FRR instance is currently running + with. + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/NOTES.txt b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/NOTES.txt new file mode 100644 index 000000000..5b5b84a17 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/NOTES.txt @@ -0,0 +1,4 @@ +FRR-k8s is now running in the cluster. + +Now you can configure it via its CRs. Please refer to the frr-k8s official docs +on how to use the CRs. diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/_helpers.tpl b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/_helpers.tpl new file mode 100644 index 000000000..4e35f6fc6 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/_helpers.tpl @@ -0,0 +1,63 @@ +{{/* vim: set filetype=mustache: */}} +{{/* +Expand the name of the chart. +*/}} +{{- define "frrk8s.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "frrk8s.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "frrk8s.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "frrk8s.labels" -}} +helm.sh/chart: {{ include "frrk8s.chart" . }} +{{ include "frrk8s.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "frrk8s.selectorLabels" -}} +app.kubernetes.io/name: {{ include "frrk8s.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the frrk8s service account to use +*/}} +{{- define "frrk8s.serviceAccountName" -}} +{{- if .Values.frrk8s.serviceAccount.create }} +{{- default (printf "%s-controller" (include "frrk8s.fullname" .)) .Values.frrk8s.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.frrk8s.serviceAccount.name }} +{{- end }} +{{- end }} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/controller.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/controller.yaml new file mode 100644 index 000000000..51656a2ec --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/controller.yaml @@ -0,0 +1,462 @@ +# FRR expects to have these files owned by frr:frr on startup. +# Having them in a ConfigMap allows us to modify behaviors: for example enabling more daemons on startup. +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "frrk8s.fullname" . }}-frr-startup + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "frrk8s.labels" . | nindent 4 }} + app.kubernetes.io/component: frr-k8s +data: + daemons: | + # This file tells the frr package which daemons to start. + # + # Sample configurations for these daemons can be found in + # /usr/share/doc/frr/examples/. + # + # ATTENTION: + # + # When activating a daemon for the first time, a config file, even if it is + # empty, has to be present *and* be owned by the user and group "frr", else + # the daemon will not be started by /etc/init.d/frr. The permissions should + # be u=rw,g=r,o=. + # When using "vtysh" such a config file is also needed. It should be owned by + # group "frrvty" and set to ug=rw,o= though. Check /etc/pam.d/frr, too. + # + # The watchfrr and zebra daemons are always started. + # + bgpd=yes + ospfd=no + ospf6d=no + ripd=no + ripngd=no + isisd=no + pimd=no + ldpd=no + nhrpd=no + eigrpd=no + babeld=no + sharpd=no + pbrd=no + bfdd=yes + fabricd=no + vrrpd=no + + # + # If this option is set the /etc/init.d/frr script automatically loads + # the config via "vtysh -b" when the servers are started. + # Check /etc/pam.d/frr if you intend to use "vtysh"! + # + vtysh_enable=yes + zebra_options=" -A 127.0.0.1 -K 120 -s 90000000 --limit-fds 100000" + bgpd_options=" -A 127.0.0.1 {{ if not .Values.frrk8s.frr.acceptIncomingBGPConnections }} -p 0 {{- end }} --limit-fds 100000" + ospfd_options=" -A 127.0.0.1" + ospf6d_options=" -A ::1" + ripd_options=" -A 127.0.0.1" + ripngd_options=" -A ::1" + isisd_options=" -A 127.0.0.1" + pimd_options=" -A 127.0.0.1" + ldpd_options=" -A 127.0.0.1" + nhrpd_options=" -A 127.0.0.1" + eigrpd_options=" -A 127.0.0.1" + babeld_options=" -A 127.0.0.1" + sharpd_options=" -A 127.0.0.1" + pbrd_options=" -A 127.0.0.1" + staticd_options="-A 127.0.0.1 --limit-fds 100000" + bfdd_options=" -A 127.0.0.1 --limit-fds 100000" + fabricd_options="-A 127.0.0.1" + vrrpd_options=" -A 127.0.0.1" + + # configuration profile + # + #frr_profile="traditional" + #frr_profile="datacenter" + + # + # This is the maximum number of FD's that will be available. + # Upon startup this is read by the control files and ulimit + # is called. Uncomment and use a reasonable value for your + # setup if you are expecting a large number of peers in + # say BGP. + #MAX_FDS=1024 + + # The list of daemons to watch is automatically generated by the init script. + #watchfrr_options="" + + # for debugging purposes, you can specify a "wrap" command to start instead + # of starting the daemon directly, e.g. to use valgrind on ospfd: + # ospfd_wrap="/usr/bin/valgrind" + # or you can use "all_wrap" for all daemons, e.g. to use perf record: + # all_wrap="/usr/bin/perf record --call-graph -" + # the normal daemon command is added to this at the end. + vtysh.conf: |+ + service integrated-vtysh-config + frr.conf: |+ + ! This file gets overriden the first time the speaker renders a config. + ! So anything configured here is only temporary. + frr version 8.0 + frr defaults traditional + hostname Router + line vty + log stdout informational +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: {{ template "frrk8s.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "frrk8s.labels" . | nindent 4 }} + app.kubernetes.io/component: frr-k8s + {{- range $key, $value := .Values.frrk8s.labels }} + {{ $key }}: {{ $value | quote }} + {{- end }} +spec: + {{- if .Values.frrk8s.updateStrategy }} + updateStrategy: {{- toYaml .Values.frrk8s.updateStrategy | nindent 4 }} + {{- end }} + selector: + matchLabels: + {{- include "frrk8s.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: frr-k8s + template: + metadata: + labels: + {{- include "frrk8s.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: frr-k8s + {{- range $key, $value := .Values.frrk8s.labels }} + {{ $key }}: {{ $value | quote }} + {{- end }} + spec: + {{- if .Values.frrk8s.runtimeClassName }} + runtimeClassName: {{ .Values.frrk8s.runtimeClassName }} + {{- end }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + serviceAccountName: {{ template "frrk8s.serviceAccountName" . }} + terminationGracePeriodSeconds: 0 + hostNetwork: true + volumes: + - name: frr-sockets + emptyDir: {} + - name: frr-startup + configMap: + name: {{ template "frrk8s.fullname" . }}-frr-startup + - name: frr-conf + emptyDir: {} + - name: reloader + emptyDir: {} + - name: metrics + emptyDir: {} + - name: frr-status + emptyDir: {} + - name: frr-lib + emptyDir: {} + - name: frr-tmp + emptyDir: {} + {{- if .Values.tls.metricsTLSSecret }} + - name: metrics-certs + secret: + secretName: {{ .Values.tls.metricsTLSSecret }} + {{- end }} + initContainers: + # Copies the initial config files with the right permissions to the shared volume. + - name: cp-frr-files + image: {{ .Values.frrk8s.frr.image.repository }}:{{ .Values.frrk8s.frr.image.tag | default .Chart.AppVersion }} + securityContext: + runAsUser: 100 + runAsGroup: 101 + command: ["/bin/sh", "-c", "cp -rLf /tmp/frr/* /etc/frr/"] + volumeMounts: + - name: frr-startup + mountPath: /tmp/frr + - name: frr-conf + mountPath: /etc/frr + # Copies the reloader to the shared volume between the speaker and reloader. + - name: cp-reloader + image: {{ .Values.frrk8s.image.repository }}:{{ .Values.frrk8s.image.tag | default .Chart.AppVersion }} + command: ["/bin/sh", "-c", "cp -f /frr-reloader.sh /etc/frr_reloader/"] + volumeMounts: + - name: reloader + mountPath: /etc/frr_reloader + # Copies the metrics exporter + - name: cp-metrics + image: {{ .Values.frrk8s.image.repository }}:{{ .Values.frrk8s.image.tag | default .Chart.AppVersion }} + command: ["/bin/sh", "-c", "cp -f /frr-metrics /etc/frr_metrics/"] + volumeMounts: + - name: metrics + mountPath: /etc/frr_metrics + - name: cp-frr-status + image: {{ .Values.frrk8s.image.repository }}:{{ .Values.frrk8s.image.tag | default .Chart.AppVersion }} + command: ["/bin/sh", "-c", "cp -f /frr-status /etc/frr_status/"] + volumeMounts: + - name: frr-status + mountPath: /etc/frr_status + shareProcessNamespace: true + containers: + - name: controller + image: {{ .Values.frrk8s.image.repository }}:{{ .Values.frrk8s.image.tag | default .Chart.AppVersion }} + {{- if .Values.frrk8s.image.pullPolicy }} + imagePullPolicy: {{ .Values.frrk8s.image.pullPolicy }} + {{- end }} + command: + - /frr-k8s + args: + - "--node-name=$(NODE_NAME)" + - "--namespace=$(NAMESPACE)" + - "--metrics-bind-address=0.0.0.0:{{ .Values.prometheus.secureMetricsPort }}" + {{- with .Values.frrk8s.logLevel }} + - --log-level={{ . }} + {{- end }} + {{- if .Values.frrk8s.alwaysBlock }} + - --always-block={{ .Values.frrk8s.alwaysBlock }} + {{- end }} + {{- if .Values.frrk8s.bgpDebounceTimeout }} + - --bgp-debounce-timeout={{ .Values.frrk8s.bgpDebounceTimeout }} + {{- end }} + {{- if .Values.tls.cipherSuites }} + - --tls-cipher-suites={{ .Values.tls.cipherSuites }} + {{- end }} + {{- if .Values.tls.curvePreferences }} + - --tls-curve-preferences={{ .Values.tls.curvePreferences }} + {{- end }} + {{- if .Values.tls.minVersion }} + - --tls-min-version={{ .Values.tls.minVersion }} + {{- end }} + {{- if .Values.tls.metricsTLSSecret }} + - --metrics-cert-dir=/etc/metrics + {{- end }} + env: + - name: FRR_CONFIG_FILE + value: /etc/frr_reloader/frr.conf + - name: FRR_RELOADER_PID_FILE + value: /etc/frr_reloader/reloader.pid + - name: NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + ports: + - containerPort: {{ .Values.prometheus.secureMetricsPort }} + name: metricshttps + {{- if .Values.frrk8s.livenessProbe.enabled }} + livenessProbe: + httpGet: + path: /healthz + port: 7572 + host: 127.0.0.1 + initialDelaySeconds: {{ .Values.frrk8s.livenessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.frrk8s.livenessProbe.periodSeconds }} + timeoutSeconds: {{ .Values.frrk8s.livenessProbe.timeoutSeconds }} + successThreshold: {{ .Values.frrk8s.livenessProbe.successThreshold }} + failureThreshold: {{ .Values.frrk8s.livenessProbe.failureThreshold }} + {{- end }} + {{- if .Values.frrk8s.readinessProbe.enabled }} + readinessProbe: + httpGet: + path: /readyz + port: 7572 + host: 127.0.0.1 + initialDelaySeconds: {{ .Values.frrk8s.readinessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.frrk8s.readinessProbe.periodSeconds }} + timeoutSeconds: {{ .Values.frrk8s.readinessProbe.timeoutSeconds }} + successThreshold: {{ .Values.frrk8s.readinessProbe.successThreshold }} + failureThreshold: {{ .Values.frrk8s.readinessProbe.failureThreshold }} + {{- end }} + {{- with .Values.frrk8s.resources }} + resources: + {{- toYaml . | nindent 10 }} + {{- end }} + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + drop: + - ALL + add: + - NET_RAW + volumeMounts: + - name: reloader + mountPath: /etc/frr_reloader + {{- if .Values.tls.metricsTLSSecret }} + - name: metrics-certs + mountPath: /etc/metrics + readOnly: true + {{- end }} + - name: frr + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + add: + - NET_ADMIN + - NET_RAW + - SYS_ADMIN + - NET_BIND_SERVICE + image: {{ .Values.frrk8s.frr.image.repository }}:{{ .Values.frrk8s.frr.image.tag | default .Chart.AppVersion }} + {{- if .Values.frrk8s.frr.image.pullPolicy }} + imagePullPolicy: {{ .Values.frrk8s.frr.image.pullPolicy }} + {{- end }} + env: + - name: TINI_SUBREAPER + value: "true" + volumeMounts: + - name: frr-sockets + mountPath: /var/run/frr + - name: frr-conf + mountPath: /etc/frr + - name: frr-lib + mountPath: /var/lib/frr + - name: frr-tmp + mountPath: /var/tmp/frr + command: + - /bin/sh + - -c + - /sbin/tini -- /usr/lib/frr/docker-start + {{- with .Values.frrk8s.frr.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- if .Values.frrk8s.livenessProbe.enabled }} + livenessProbe: + httpGet: + path: /livez + port: {{ .Values.frrk8s.frr.secureMetricsPort }} + scheme: HTTPS + periodSeconds: {{ .Values.frrk8s.livenessProbe.periodSeconds }} + failureThreshold: {{ .Values.frrk8s.livenessProbe.failureThreshold }} + {{- end }} + {{- if .Values.frrk8s.startupProbe.enabled }} + startupProbe: + httpGet: + path: /livez + port: {{ .Values.frrk8s.frr.secureMetricsPort }} + scheme: HTTPS + failureThreshold: {{ .Values.frrk8s.startupProbe.failureThreshold }} + periodSeconds: {{ .Values.frrk8s.startupProbe.periodSeconds }} + {{- end }} + - name: reloader + image: {{ .Values.frrk8s.frr.image.repository }}:{{ .Values.frrk8s.frr.image.tag | default .Chart.AppVersion }} + {{- if .Values.frrk8s.frr.image.pullPolicy }} + imagePullPolicy: {{ .Values.frrk8s.frr.image.pullPolicy }} + {{- end }} + command: ["/etc/frr_reloader/frr-reloader.sh"] + volumeMounts: + - name: frr-sockets + mountPath: /var/run/frr + - name: frr-conf + mountPath: /etc/frr + - name: reloader + mountPath: /etc/frr_reloader + {{- with .Values.frrk8s.reloader.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + - name: frr-metrics + image: {{ .Values.frrk8s.frr.image.repository }}:{{ .Values.frrk8s.frr.image.tag | default .Chart.AppVersion }} + command: ["/etc/frr_metrics/frr-metrics"] + args: + - --metrics-port={{ .Values.frrk8s.frr.secureMetricsPort }} + - --metrics-bind-address=0.0.0.0 + {{- if .Values.tls.metricsTLSSecret }} + - --tls-cert-file=/etc/metrics/tls.crt + - --tls-private-key-file=/etc/metrics/tls.key + {{- end }} + {{- if .Values.tls.cipherSuites }} + - --tls-cipher-suites={{ .Values.tls.cipherSuites }} + {{- end }} + {{- if .Values.tls.curvePreferences }} + - --tls-curve-preferences={{ .Values.tls.curvePreferences }} + {{- end }} + {{- if .Values.tls.minVersion }} + - --tls-min-version={{ .Values.tls.minVersion }} + {{- end }} + ports: + - containerPort: {{ .Values.frrk8s.frr.secureMetricsPort }} + name: frrmetricshttps + volumeMounts: + - name: frr-sockets + mountPath: /var/run/frr + - name: frr-conf + mountPath: /etc/frr + - name: metrics + mountPath: /etc/frr_metrics + {{- if .Values.tls.metricsTLSSecret }} + - name: metrics-certs + mountPath: /etc/metrics + readOnly: true + {{- end }} + {{- with .Values.frrk8s.frrMetrics.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + - name: frr-status + args: + - --node-name=$(NODE_NAME) + - --namespace=$(NAMESPACE) + - --pod-name=$(POD_NAME) + {{- with .Values.frrk8s.logLevel }} + - --log-level={{ . }} + {{- end }} + {{- if .Values.frrk8s.frrStatus.pollInterval }} + - --poll-interval={{ .Values.frrk8s.frrStatus.pollInterval }} + {{- end }} + command: + - /etc/frr_status/frr-status + env: + - name: NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + image: {{ .Values.frrk8s.frr.image.repository }}:{{ .Values.frrk8s.frr.image.tag | default .Chart.AppVersion }} + volumeMounts: + - mountPath: /var/run/frr + name: frr-sockets + - mountPath: /etc/frr + name: frr-conf + - mountPath: /etc/frr_status + name: frr-status + {{- with .Values.frrk8s.frrStatus.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} + nodeSelector: + "kubernetes.io/os": linux + {{- with .Values.frrk8s.nodeSelector }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.frrk8s.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- if or .Values.frrk8s.tolerateMaster .Values.frrk8s.tolerations }} + tolerations: + {{- if .Values.frrk8s.tolerateMaster }} + - key: node-role.kubernetes.io/master + effect: NoSchedule + operator: Exists + - key: node-role.kubernetes.io/control-plane + effect: NoSchedule + operator: Exists + {{- end }} + {{- with .Values.frrk8s.tolerations }} + {{- toYaml . | nindent 6 }} + {{- end }} + {{- end }} + {{- with .Values.frrk8s.priorityClassName }} + priorityClassName: {{ . | quote }} + {{- end }} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/rbac.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/rbac.yaml new file mode 100644 index 000000000..7da9ea445 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/rbac.yaml @@ -0,0 +1,113 @@ +{{- if .Values.rbac.create -}} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ include "frrk8s.fullname" . }}-controller + labels: {{- include "frrk8s.labels" . | nindent 4 }} +rules: +- apiGroups: ["frrk8s.metallb.io"] + resources: ["frrk8sconfigurations"] + verbs: ["get", "list", "watch"] +- apiGroups: ["frrk8s.metallb.io"] + resources: ["frrconfigurations"] + verbs: ["get", "list", "watch"] +- apiGroups: ["frrk8s.metallb.io"] + resources: ["frrnodestates"] + verbs: ["get", "list", "watch", "create", "delete", "patch", "update"] +- apiGroups: ["frrk8s.metallb.io"] + resources: ["frrnodestates/status"] + verbs: ["get", "patch", "update"] +- apiGroups: ["frrk8s.metallb.io"] + resources: ["bgpsessionstates"] + verbs: ["get", "list", "watch", "create", "delete", "patch", "update"] +- apiGroups: ["frrk8s.metallb.io"] + resources: ["bgpsessionstates/status"] + verbs: ["get", "patch", "update"] +- apiGroups: [""] + resources: ["nodes"] + verbs: ["get", "list", "watch"] +- apiGroups: ["authentication.k8s.io"] + resources: ["tokenreviews"] + verbs: ["create"] +- apiGroups: ["authorization.k8s.io"] + resources: ["subjectaccessreviews"] + verbs: ["create"] +- apiGroups: ["admissionregistration.k8s.io"] + resources: ["validatingwebhookconfigurations"] + verbs: ["get", "list", "watch"] +- apiGroups: ["admissionregistration.k8s.io"] + resourceNames: ["frr-k8s-validating-webhook-configuration"] + resources: ["validatingwebhookconfigurations"] + verbs: ["update"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "frrk8s.fullname" . }}-controller + labels: {{- include "frrk8s.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ include "frrk8s.fullname" . }}-controller +subjects: +- kind: ServiceAccount + name: {{ include "frrk8s.serviceAccountName" . }} + namespace: {{ .Release.Namespace | quote }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "frrk8s.fullname" . }}-controller + namespace: {{ .Release.Namespace | quote }} + labels: {{- include "frrk8s.labels" . | nindent 4 }} +rules: +- apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "list", "watch","update"] +- apiGroups: [""] + resources: ["pods"] + verbs: ["get","list","watch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "frrk8s.fullname" . }}-controller + namespace: {{ .Release.Namespace | quote }} + labels: {{- include "frrk8s.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "frrk8s.fullname" . }}-controller +subjects: +- kind: ServiceAccount + name: {{ include "frrk8s.serviceAccountName" . }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "frrk8s.fullname" . }}-nodestate-cleaner + namespace: {{ .Release.Namespace | quote }} + labels: {{- include "frrk8s.labels" . | nindent 4 }} +rules: +- apiGroups: [""] + resources: ["pods"] + verbs: ["get", "list", "watch"] +- apiGroups: ["frrk8s.metallb.io"] + resources: ["frrnodestates"] + verbs: ["get", "list", "watch", "delete"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "frrk8s.fullname" . }}-nodestate-cleaner + namespace: {{ .Release.Namespace | quote }} + labels: {{- include "frrk8s.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "frrk8s.fullname" . }}-nodestate-cleaner +subjects: +- kind: ServiceAccount + name: {{ include "frrk8s.serviceAccountName" . }} +{{ end -}} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-accounts.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-accounts.yaml new file mode 100644 index 000000000..3c64c5cb2 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-accounts.yaml @@ -0,0 +1,16 @@ +{{- if .Values.frrk8s.serviceAccount.create }} +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ template "frrk8s.serviceAccountName" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "frrk8s.labels" . | nindent 4 }} + app.kubernetes.io/component: controller + {{- with .Values.frrk8s.serviceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} + diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-monitor.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-monitor.yaml new file mode 100644 index 000000000..c6f91304e --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/service-monitor.yaml @@ -0,0 +1,128 @@ +{{- if .Values.prometheus.serviceMonitor.enabled }} +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: {{ template "frrk8s.fullname" . }}-frr-k8s-monitor + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "frrk8s.labels" . | nindent 4 }} + app.kubernetes.io/component: frr-k8s + {{- if .Values.prometheus.serviceMonitor.additionalLabels }} +{{ toYaml .Values.prometheus.serviceMonitor.additionalLabels | indent 4 }} + {{- end }} + {{- if .Values.prometheus.serviceMonitor.annotations }} + annotations: +{{ toYaml .Values.prometheus.serviceMonitor.annotations | indent 4 }} + {{- end }} +spec: + endpoints: + - port: "metricshttps" + honorLabels: true + {{- if .Values.prometheus.serviceMonitor.metricRelabelings }} + metricRelabelings: + {{- toYaml .Values.prometheus.serviceMonitor.metricRelabelings | nindent 8 }} + {{- end -}} + {{- if .Values.prometheus.serviceMonitor.relabelings }} + relabelings: + {{- toYaml .Values.prometheus.serviceMonitor.relabelings | nindent 8 }} + {{- end }} + {{- if .Values.prometheus.serviceMonitor.interval }} + interval: {{ .Values.prometheus.serviceMonitor.interval }} + {{- end -}} +{{ if .Values.prometheus.secureMetricsPort }} + bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" + scheme: "https" +{{- if .Values.prometheus.serviceMonitor.tlsConfig }} + tlsConfig: +{{ toYaml .Values.prometheus.serviceMonitor.tlsConfig | indent 8 }} +{{- end }} +{{ end }} +{{ if .Values.frrk8s.frr.secureMetricsPort }} + - port: "frrmetricshttps" + honorLabels: true + {{- if .Values.prometheus.serviceMonitor.metricRelabelings }} + metricRelabelings: + {{- toYaml .Values.prometheus.serviceMonitor.metricRelabelings | nindent 8 }} + {{- end -}} + {{- if .Values.prometheus.serviceMonitor.relabelings }} + relabelings: + {{- toYaml .Values.prometheus.serviceMonitor.relabelings | nindent 8 }} + {{- end }} + {{- if .Values.prometheus.serviceMonitor.interval }} + interval: {{ .Values.prometheus.serviceMonitor.interval }} + {{- end }} + bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" + scheme: "https" +{{- if .Values.prometheus.serviceMonitor.tlsConfig }} + tlsConfig: +{{ toYaml .Values.prometheus.serviceMonitor.tlsConfig | indent 8 }} +{{- end }} +{{- end }} + jobLabel: {{ .Values.prometheus.serviceMonitor.jobLabel | quote }} + namespaceSelector: + matchNames: + - {{ .Release.Namespace }} + selector: + matchLabels: + name: {{ template "frrk8s.fullname" . }}-frr-k8s-monitor-service +--- +apiVersion: v1 +kind: Service +metadata: + annotations: + prometheus.io/scrape: "true" + {{- if .Values.prometheus.serviceMonitor.annotations }} +{{ toYaml .Values.prometheus.serviceMonitor.annotations | indent 4 }} + {{- end }} + labels: + name: {{ template "frrk8s.fullname" . }}-frr-k8s-monitor-service + name: {{ template "frrk8s.fullname" . }}-frr-k8s-monitor-service + namespace: {{ .Release.Namespace | quote }} +spec: + selector: + {{- include "frrk8s.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: frr-k8s + clusterIP: None + ports: + - name: "metricshttps" + port: {{ .Values.prometheus.secureMetricsPort }} + targetPort: {{ .Values.prometheus.secureMetricsPort }} + - name: frrmetricshttps + port: {{ .Values.frrk8s.frr.secureMetricsPort }} + targetPort: {{ .Values.frrk8s.frr.secureMetricsPort }} + sessionAffinity: None + type: ClusterIP +--- +{{- if .Values.prometheus.rbacPrometheus }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ template "frrk8s.fullname" . }}-prometheus + namespace: {{ .Release.Namespace | quote }} +rules: + - apiGroups: + - "" + resources: + - pods + - services + - endpoints + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ template "frrk8s.fullname" . }}-prometheus + namespace: {{ .Release.Namespace | quote }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ template "frrk8s.fullname" . }}-prometheus +subjects: + - kind: ServiceAccount + name: {{ required ".Values.prometheus.serviceAccount must be defined when .Values.prometheus.serviceMonitor.enabled == true" .Values.prometheus.serviceAccount }} + namespace: {{ required ".Values.prometheus.namespace must be defined when .Values.prometheus.serviceMonitor.enabled == true" .Values.prometheus.namespace }} +{{- end }} +{{- end }} diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/status-cleaner.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/status-cleaner.yaml new file mode 100644 index 000000000..4dc0c33df --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/status-cleaner.yaml @@ -0,0 +1,129 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "frrk8s.fullname" . }}-statuscleaner + namespace: {{ .Release.Namespace | quote }} + labels: + {{- include "frrk8s.labels" . | nindent 4 }} + app.kubernetes.io/component: statuscleaner + {{- range $key, $value := .Values.frrk8s.labels }} + {{ $key }}: {{ $value | quote }} + {{- end }} +spec: + selector: + matchLabels: + app.kubernetes.io/component: statuscleaner + template: + metadata: + annotations: + kubectl.kubernetes.io/default-container: frr-k8s-statuscleaner + labels: + app.kubernetes.io/component: statuscleaner + spec: + {{- if .Values.frrk8s.runtimeClassName }} + runtimeClassName: {{ .Values.frrk8s.runtimeClassName }} + {{- end }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - command: + - /statuscleaner + args: + - --webhook-port={{ .Values.frrk8s.webhookPort }} + {{- with .Values.frrk8s.logLevel }} + - --log-level={{ . }} + {{- end }} + {{- if .Values.frrk8s.disableCertRotation }} + - "--disable-cert-rotation=true" + {{- end }} + {{- if .Values.frrk8s.restartOnRotatorSecretRefresh }} + - "--restart-on-rotator-secret-refresh=true" + {{- end }} + - "--namespace=$(NAMESPACE)" + {{- if .Values.tls.cipherSuites }} + - --tls-cipher-suites={{ .Values.tls.cipherSuites }} + {{- end }} + {{- if .Values.tls.curvePreferences }} + - --tls-curve-preferences={{ .Values.tls.curvePreferences }} + {{- end }} + {{- if .Values.tls.minVersion }} + - --tls-min-version={{ .Values.tls.minVersion }} + {{- end }} + env: + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: {{ .Values.frrk8s.image.repository }}:{{ .Values.frrk8s.image.tag | default .Chart.AppVersion }} + {{- if .Values.frrk8s.image.pullPolicy }} + imagePullPolicy: {{ .Values.frrk8s.image.pullPolicy }} + {{- end }} + name: frr-k8s-statuscleaner + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + readOnlyRootFilesystem: true + ports: + - containerPort: {{ .Values.frrk8s.webhookPort }} + name: webhook + {{- if .Values.frrk8s.livenessProbe.enabled }} + livenessProbe: + httpGet: + path: /healthz + port: webhook + scheme: HTTPS + initialDelaySeconds: {{ .Values.frrk8s.livenessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.frrk8s.livenessProbe.periodSeconds }} + failureThreshold: {{ .Values.frrk8s.livenessProbe.failureThreshold }} + {{- end }} + {{- if .Values.frrk8s.readinessProbe.enabled }} + readinessProbe: + httpGet: + path: /healthz + port: webhook + scheme: HTTPS + initialDelaySeconds: {{ .Values.frrk8s.readinessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.frrk8s.readinessProbe.periodSeconds }} + failureThreshold: {{ .Values.frrk8s.readinessProbe.failureThreshold }} + {{- end }} + {{- with .Values.frrk8s.resources }} + resources: + {{- toYaml . | nindent 10 }} + {{- end }} + volumeMounts: + - name: cert + mountPath: /tmp/k8s-webhook-server/serving-certs + readOnly: true + {{- with .Values.frrk8s.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- if or .Values.frrk8s.tolerateMaster .Values.frrk8s.tolerations }} + tolerations: + {{- if .Values.frrk8s.tolerateMaster }} + - key: node-role.kubernetes.io/master + effect: NoSchedule + operator: Exists + - key: node-role.kubernetes.io/control-plane + effect: NoSchedule + operator: Exists + {{- end }} + {{- with .Values.frrk8s.tolerations }} + {{- toYaml . | nindent 6 }} + {{- end }} + {{- end }} + {{- with .Values.frrk8s.priorityClassName }} + priorityClassName: {{ . | quote }} + {{- end }} + volumes: + - name: cert + secret: + defaultMode: 420 + secretName: frr-k8s-webhook-server-cert + serviceAccountName: {{ template "frrk8s.serviceAccountName" . }} + terminationGracePeriodSeconds: 10 + hostNetwork: true diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/webhooks.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/webhooks.yaml new file mode 100644 index 000000000..212f05345 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/templates/webhooks.yaml @@ -0,0 +1,43 @@ +apiVersion: v1 +kind: Secret +metadata: + name: frr-k8s-webhook-server-cert + namespace: {{ .Release.Namespace | quote }} +--- +apiVersion: v1 +kind: Service +metadata: + name: frr-k8s-webhook-service + namespace: {{ .Release.Namespace | quote }} +spec: + ports: + - port: 443 + targetPort: webhook + selector: + app.kubernetes.io/component: statuscleaner +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +metadata: + name: frr-k8s-validating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: frr-k8s-webhook-service + namespace: {{ .Release.Namespace }} + path: /validate-frrk8s-metallb-io-v1beta1-frrconfiguration + failurePolicy: {{ .Values.crds.validationFailurePolicy }} + name: frrconfigurationsvalidationwebhook.metallb.io + rules: + - apiGroups: + - frrk8s.metallb.io + apiVersions: + - v1beta1 + operations: + - CREATE + - UPDATE + resources: + - frrconfigurations + sideEffects: None diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/values.schema.json b/packages/system/metallb/charts/metallb/charts/frr-k8s/values.schema.json new file mode 100644 index 000000000..3678da6f6 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/values.schema.json @@ -0,0 +1,394 @@ +{ + "$schema": "https://json-schema.org/draft-07/schema#", + "title": "Values", + "type": "object", + "definitions": { + "prometheusAlert": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + }, + "labels": { + "type": "object", + "additionalProperties": { + "type": "string" + } + } + }, + "required": [ + "enabled" + ] + }, + "probe": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + }, + "failureThreshold": { + "type": "integer" + }, + "initialDelaySeconds": { + "type": "integer" + }, + "periodSeconds": { + "type": "integer" + }, + "successThreshold": { + "type": "integer" + }, + "timeoutSeconds": { + "type": "integer" + } + }, + "required": [ + "failureThreshold", + "initialDelaySeconds", + "periodSeconds", + "successThreshold", + "timeoutSeconds" + ] + }, + "component": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + }, + "logLevel": { + "type": "string", + "enum": [ + "all", + "debug", + "info", + "warn", + "error", + "none" + ] + }, + "image": { + "type": "object", + "properties": { + "repository": { + "type": "string" + }, + "tag": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ] + }, + "pullPolicy": { + "anyOf": [ + { + "type": "null" + }, + { + "type": "string", + "enum": [ + "Always", + "IfNotPresent", + "Never" + ] + } + ] + } + } + }, + "serviceAccount": { + "type": "object", + "properties": { + "create": { + "type": "boolean" + }, + "name": { + "type": "string" + }, + "annotations": { + "type": "object" + } + } + }, + "resources": { + "type": "object" + }, + "nodeSelector": { + "type": "object" + }, + "tolerations": { + "type": "array", + "items": { + "type": "object" + } + }, + "priorityClassName": { + "type": "string" + }, + "runtimeClassName": { + "type": "string" + }, + "affinity": { + "type": "object" + }, + "podAnnotations": { + "type": "object" + }, + "livenessProbe": { + "$ref": "#/definitions/probe" + }, + "readinessProbe": { + "$ref": "#/definitions/probe" + } + }, + "required": [ + "image", + "serviceAccount" + ] + } + }, + "properties": { + "imagePullSecrets": { + "description": "Secrets used for pulling images", + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string" + } + }, + "required": [ + "name" + ], + "additionalProperties": false + } + }, + "nameOverride": { + "description": "Override chart name", + "type": "string" + }, + "fullNameOverride": { + "description": "Override fully qualified app name", + "type": "string" + }, + "tls": { + "description": "TLS configuration for all components", + "type": "object", + "properties": { + "cipherSuites": { + "description": "Comma-separated list of TLS cipher suites", + "type": "string" + }, + "curvePreferences": { + "description": "Comma-separated list of numeric CurveID values", + "type": "string" + }, + "minVersion": { + "description": "Minimum TLS version (VersionTLS12 or VersionTLS13)", + "type": "string" + }, + "metricsTLSSecret": { + "description": "Secret name for metrics TLS certificates. If empty, self-signed certs are auto-generated.", + "type": "string" + } + } + }, + "rbac": { + "description": "RBAC configuration", + "type": "object", + "properties": { + "create": { + "description": "Enable RBAC", + "type": "boolean" + } + } + }, + "prometheus": { + "description": "Prometheus monitoring config", + "type": "object", + "properties": { + "scrapeAnnotations": { + "type": "boolean" + }, + "secureMetricsPort": { + "type": "integer" + }, + "rbacPrometheus": { + "type": "boolean" + }, + "serviceAccount": { + "type": "string" + }, + "namespace": { + "type": "string" + }, + "serviceMonitor": { + "description": "Prometheus Operator ServiceMonitors", + "type": "object", + "properties": { + "enabled": { + "type": "boolean" + }, + "jobLabel": { + "type": "string" + }, + "interval": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ] + }, + "metricRelabelings": { + "type": "array", + "items": { + "type": "object" + } + }, + "relabelings": { + "type": "array", + "items": { + "type": "object" + } + } + } + } + }, + "frrk8s": { + "allOf": [ + { + "$ref": "#/definitions/component" + }, + { + "description": "FRR-K8s controller", + "type": "object", + "properties": { + "tolerateMaster": { + "type": "boolean" + }, + "updateStrategy": { + "type": "object", + "properties": { + "type": { + "type": "string" + } + }, + "required": [ + "type" + ] + }, + "runtimeClassName": { + "type": "string" + }, + "secretName": { + "type": "string" + }, + "frr": { + "description": "The FRR properties in the controller", + "type": "object", + "properties": { + "image": { + "$ref": "#/definitions/component/properties/image" + }, + "metricsPort": { + "type": "integer" + }, + "secureMetricsPort": { + "type": "integer" + }, + "resources:": { + "type": "object" + } + }, + "required": [ + "enabled" + ] + }, + "command": { + "type": "string" + }, + "reloader": { + "type": "object", + "properties": { + "resources": { + "type": "object" + } + } + }, + "frrMetrics": { + "type": "object", + "properties": { + "resources": { + "type": "object" + } + } + } + }, + "required": [ + "tolerateMaster" + ] + } + ] + }, + "crds": { + "description": "CRD configuration", + "type": "object", + "properties": { + "enabled": { + "description": "Enable CRDs", + "type": "boolean" + }, + "validationFailurePolicy": { + "description": "Failure policy to use with validating webhooks", + "type": "string", + "enum": [ + "Ignore", + "Fail" + ] + } + } + } + }, + "frrk8s": { + "allOf": [ + { + "$ref": "#/definitions/component" + }, + { + "description": "FRRk8s Controller", + "type": "object", + "properties": { + "strategy": { + "type": "object", + "properties": { + "type": { + "type": "string" + } + }, + "required": [ + "type" + ] + }, + "command": { + "type": "string" + }, + "webhookMode": { + "type": "string" + } + } + } + ] + } + }, + "required": [ + "frrk8s" + ] +} \ No newline at end of file diff --git a/packages/system/metallb/charts/metallb/charts/frr-k8s/values.yaml b/packages/system/metallb/charts/metallb/charts/frr-k8s/values.yaml new file mode 100644 index 000000000..2dfa32818 --- /dev/null +++ b/packages/system/metallb/charts/metallb/charts/frr-k8s/values.yaml @@ -0,0 +1,215 @@ +# Default values for frr-k8s. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +# -- String to override the default chart name. +nameOverride: "" +# -- String to override the default fully qualified app name. +fullnameOverride: "" + +tls: + # -- Comma-separated list of TLS cipher suites. If empty, uses Go defaults. Only applies to TLS 1.2. + cipherSuites: "" + # -- Comma-separated list of numeric CurveID values (e.g. 29,4588). See https://pkg.go.dev/crypto/tls#CurveID. If empty, uses Go defaults. + curvePreferences: "" + # -- Minimum TLS version (VersionTLS12 or VersionTLS13). Defaults to VersionTLS13. + minVersion: "" + # -- The name of the secret to be mounted in the pods to provide TLS certificates for metrics endpoints. If not present, a self-signed certificate is auto-generated. + metricsTLSSecret: "" + +rbac: + # -- Specifies whether to install and use RBAC rules. + create: true + +prometheus: + # scrape annotations specifies whether to add Prometheus metric + # auto-collection annotations to pods. See + # https://github.com/prometheus/prometheus/blob/release-2.1/documentation/examples/prometheus-kubernetes.yml + # for a corresponding Prometheus configuration. Alternatively, you + # may want to use the Prometheus Operator + # (https://github.com/coreos/prometheus-operator) for more powerful + # monitoring configuration. If you use the Prometheus operator, this + # can be left at false. + # -- Add Prometheus metric auto-collection annotations to pods. + scrapeAnnotations: false + + # -- Port frr-k8s will listen on for secure metrics. + secureMetricsPort: 9140 + + # -- Give Prometheus permission to scrape metallb's namespace. + rbacPrometheus: false + + # -- The service account used by Prometheus. Required when ".Values.prometheus.rbacPrometheus == true" and "prometheus.serviceMonitor.enabled=true" + serviceAccount: "" + + # -- The namespace where Prometheus is deployed. Required when ".Values.prometheus.rbacPrometheus == true" and "prometheus.serviceMonitor.enabled=true". + namespace: "" + + # Prometheus Operator ServiceMonitors. + serviceMonitor: + # -- Enable support for Prometheus Operator. + enabled: false + + # -- Additional labels to add to the ServiceMonitor. + additionalLabels: {} + # -- Optional additional annotations for the controller serviceMonitor. + annotations: {} + ## Optional TLS configuration for the controller serviceMonitor, in case secure metrics are enabled. + tlsConfig: + # -- Disables SSL certificate verification + insecureSkipVerify: true + + # -- Job label for scrape target. + jobLabel: "app.kubernetes.io/name" + + # -- Scrape interval. If not set, the Prometheus default scrape interval is used. + interval: + + # -- Metric relabel configs to apply to samples before ingestion. + metricRelabelings: [] + # - action: keep + # regex: 'kube_(daemonset|deployment|pod|namespace|node|statefulset).+' + # sourceLabels: [__name__] + + # -- Relabel configs to apply to samples before ingestion. + relabelings: [] + # - sourceLabels: [__meta_kubernetes_pod_node_name] + # separator: ; + # regex: ^(.*)$ + # target_label: nodename + # replacement: $1 + # action: replace + +# controller contains configuration specific to the FRRK8s controller +# daemonset. +frrk8s: + # -- Controller log level that is passed as a CLI flag. Must be one of: `all`, `debug`, `info`, `warn`, `error` or `none` + logLevel: info + # -- Tolerate master nodes for pod scheduling. + tolerateMaster: true + image: + # -- The frr-k8s image repository. + repository: quay.io/metallb/frr-k8s + # -- The frr-k8s image tag. If not set, defaults to the chart appVersion. + tag: + # -- The frr-k8s image pull policy. + pullPolicy: + ## @param controller.updateStrategy.type FRR-K8s controller daemonset strategy type + ## ref: https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/ + ## + updateStrategy: + ## StrategyType + ## Can be set to RollingUpdate or OnDelete + # -- Specify the FRR-K8s daemonset update strategy. + type: RollingUpdate + serviceAccount: + # -- Specifies whether a ServiceAccount should be created. + create: true + # -- The name of the ServiceAccount to use. If not set and create is true, a name is generated using the fullname template. + name: "" + # -- Additional annotations to add to the ServiceAccount. + annotations: {} + ## Defines a secret name for the controller to generate a memberlist encryption secret + ## By default secretName: {{ "metallb.fullname" }}-memberlist + ## + # secretName: + # -- Resource limits and requests for the frr-k8s controller container. + resources: {} + # limits: + # cpu: 100m + # memory: 100Mi + # -- Node selector for pod assignment. + nodeSelector: {} + # -- Tolerations for pod assignment. + tolerations: [] + # -- Priority class name for the pod. + priorityClassName: "" + # -- Affinity for pod assignment. + affinity: {} + ## Selects which runtime class will be used by the pod. + # -- Runtime class name for the pod. + runtimeClassName: "" + # -- Additional annotations to add to the pod. + podAnnotations: {} + # -- Additional labels to add to the pod. + labels: + app: frr-k8s + # -- Port for the webhook server. + webhookPort: 19443 + livenessProbe: + # -- Enable liveness probe. + enabled: true + # -- Number of failures before the probe is considered failed. + failureThreshold: 3 + # -- Number of seconds after the container has started before liveness probes are initiated. + initialDelaySeconds: 10 + # -- How often (in seconds) to perform the probe. + periodSeconds: 10 + # -- Minimum consecutive successes for the probe to be considered successful. + successThreshold: 1 + # -- Number of seconds after which the probe times out. + timeoutSeconds: 1 + readinessProbe: + # -- Enable readiness probe. + enabled: true + # -- Number of failures before the probe is considered failed. + failureThreshold: 3 + # -- Number of seconds after the container has started before readiness probes are initiated. + initialDelaySeconds: 10 + # -- How often (in seconds) to perform the probe. + periodSeconds: 10 + # -- Minimum consecutive successes for the probe to be considered successful. + successThreshold: 1 + # -- Number of seconds after which the probe times out. + timeoutSeconds: 1 + startupProbe: + # -- Enable startup probe. + enabled: true + # -- Number of failures before the probe is considered failed. + failureThreshold: 30 + # -- How often (in seconds) to perform the probe. + periodSeconds: 5 + # -- A comma separated list of cidrs to always block for incoming routes. + alwaysBlock: "" + # -- (integer) BGP debounce timeout for FRR configuration reloads, in milliseconds. Default (when unset) is 3000 ms.This feature is experimental + bgpDebounceTimeout: null + # -- Specifies whether the cert rotator works as part of the webhook. + disableCertRotation: false + ## Note: Enabling this proved useful for the webhook's stability when it is redeployed multiple times in succession. + # -- Specifies whether the pod restarts when the rotator refreshes the cert secret. Useful for webhook stability during redeployments. + restartOnRotatorSecretRefresh: false + # frr contains configuration specific to the FRR container, + frr: + image: + # -- The FRR image repository. + repository: quay.io/frrouting/frr + # -- The FRR image tag. + tag: 10.4.3 + # -- The FRR image pull policy. + pullPolicy: + # -- Bind address for FRR metrics. + metricsBindAddress: 127.0.0.1 + # -- Port for FRR metrics. + metricsPort: 7573 + # -- Resource limits and requests for the FRR container. + resources: {} + # -- Secure metrics port for FRR. + secureMetricsPort: 9141 + # -- Allow FRR to accept incoming BGP connections. + acceptIncomingBGPConnections: false + reloader: + # -- Resource limits and requests for the reloader container. + resources: {} + frrMetrics: + # -- Resource limits and requests for the FRR metrics container. + resources: {} + frrStatus: + # -- Polling interval for FRR status updates. + pollInterval: "2m" + # -- Resource limits and requests for the FRR status container. + resources: {} +crds: + # -- Enable installation of CRDs. + enabled: true + # -- Validation failure policy for CRDs. Can be Fail or Ignore. + validationFailurePolicy: Fail diff --git a/packages/system/metallb/charts/metallb/policy/controller.rego b/packages/system/metallb/charts/metallb/policy/controller.rego index b7a0ea1a0..5ded9de26 100644 --- a/packages/system/metallb/charts/metallb/policy/controller.rego +++ b/packages/system/metallb/charts/metallb/policy/controller.rego @@ -3,6 +3,7 @@ package main # validate serviceAccountName deny[msg] { input.kind == "Deployment" + endswith(input.metadata.name, "-controller") serviceAccountName := input.spec.template.spec.serviceAccountName not serviceAccountName == "release-name-metallb-controller" msg = sprintf("controller serviceAccountName '%s' does not match expected value", [serviceAccountName]) @@ -11,6 +12,7 @@ deny[msg] { # validate node selector includes builtin when custom ones are provided deny[msg] { input.kind == "Deployment" + endswith(input.metadata.name, "-controller") not input.spec.template.spec.nodeSelector["kubernetes.io/os"] == "linux" msg = "controller nodeSelector does not include '\"kubernetes.io/os\": linux'" } diff --git a/packages/system/metallb/charts/metallb/policy/speaker.rego b/packages/system/metallb/charts/metallb/policy/speaker.rego index 146a0373c..68d675a16 100644 --- a/packages/system/metallb/charts/metallb/policy/speaker.rego +++ b/packages/system/metallb/charts/metallb/policy/speaker.rego @@ -3,6 +3,7 @@ package main # validate serviceAccountName deny[msg] { input.kind == "DaemonSet" + endswith(input.metadata.name, "-speaker") serviceAccountName := input.spec.template.spec.serviceAccountName not serviceAccountName == "release-name-metallb-speaker" msg = sprintf("speaker serviceAccountName '%s' does not match expected value", [serviceAccountName]) @@ -11,6 +12,7 @@ deny[msg] { # validate METALLB_ML_SECRET_KEY (memberlist) deny[msg] { input.kind == "DaemonSet" + endswith(input.metadata.name, "-speaker") not input.spec.template.spec.containers[0].env[5].name == "METALLB_ML_SECRET_KEY_PATH" msg = "speaker env does not contain METALLB_ML_SECRET_KEY_PATH at env[5]" } @@ -18,6 +20,7 @@ deny[msg] { # validate node selector includes builtin when custom ones are provided deny[msg] { input.kind == "DaemonSet" + endswith(input.metadata.name, "-speaker") not input.spec.template.spec.nodeSelector["kubernetes.io/os"] == "linux" msg = "controller nodeSelector does not include '\"kubernetes.io/os\": linux'" } @@ -25,6 +28,7 @@ deny[msg] { # validate tolerations include the builtins when custom ones are provided deny[msg] { input.kind == "DaemonSet" + endswith(input.metadata.name, "-speaker") not input.spec.template.spec.tolerations[0] == { "key": "node-role.kubernetes.io/master", "effect": "NoSchedule", "operator": "Exists" } msg = "controller tolerations does not include node-role.kubernetes.io/master:NoSchedule" } diff --git a/packages/system/metallb/charts/metallb/templates/NOTES.txt b/packages/system/metallb/charts/metallb/templates/NOTES.txt index 23d1d5bc1..e32cc49d2 100644 --- a/packages/system/metallb/charts/metallb/templates/NOTES.txt +++ b/packages/system/metallb/charts/metallb/templates/NOTES.txt @@ -2,3 +2,10 @@ MetalLB is now running in the cluster. Now you can configure it via its CRs. Please refer to the metallb official docs on how to use the CRs. +{{- if .Values.speaker.frr.enabled }} + +WARNING: The FRR mode (speaker.frr.enabled) is DEPRECATED and will be removed +in a future release. Please migrate to the frr-k8s mode (frrk8s.enabled: true), +which is now the default BGP backend and provides the same features with +additional flexibility. See https://metallb.io/concepts/bgp/#frr-k8s-mode +{{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/_helpers.tpl b/packages/system/metallb/charts/metallb/templates/_helpers.tpl index cd89381d7..a8b20121a 100644 --- a/packages/system/metallb/charts/metallb/templates/_helpers.tpl +++ b/packages/system/metallb/charts/metallb/templates/_helpers.tpl @@ -81,34 +81,19 @@ Create the name of the settings Secret to use. {{- end -}} {{- define "metrics.exposedportname" -}} -{{- if .Values.prometheus.secureMetricsPort -}} "metricshttps" -{{- else -}} -"metrics" -{{- end -}} {{- end -}} {{- define "metrics.exposedfrrportname" -}} -{{- if .Values.speaker.frr.secureMetricsPort -}} "frrmetricshttps" -{{- else -}} -"frrmetrics" -{{- end }} {{- end }} {{- define "metrics.exposedport" -}} -{{- if .Values.prometheus.secureMetricsPort -}} -{{ .Values.prometheus.secureMetricsPort }} -{{- else -}} {{ .Values.prometheus.metricsPort }} -{{- end -}} {{- end }} {{- define "metrics.exposedfrrport" -}} -{{- if .Values.speaker.frr.secureMetricsPort -}} -{{ .Values.speaker.frr.secureMetricsPort }} -{{- else -}} {{ .Values.speaker.frr.metricsPort }} {{- end }} -{{- end }} + diff --git a/packages/system/metallb/charts/metallb/templates/controller.yaml b/packages/system/metallb/charts/metallb/templates/controller.yaml index 8fd9c477e..cae0ab34f 100644 --- a/packages/system/metallb/charts/metallb/templates/controller.yaml +++ b/packages/system/metallb/charts/metallb/templates/controller.yaml @@ -24,6 +24,7 @@ spec: annotations: {{- if .Values.prometheus.scrapeAnnotations }} prometheus.io/scrape: "true" + prometheus.io/scheme: "https" prometheus.io/port: "{{ .Values.prometheus.metricsPort }}" {{- end }} {{- with .Values.controller.podAnnotations }} @@ -71,13 +72,23 @@ spec: {{- if .Values.controller.webhookMode }} - --webhook-mode={{ .Values.controller.webhookMode }} {{- end }} - {{- if .Values.controller.tlsMinVersion }} - - --tls-min-version={{ .Values.controller.tlsMinVersion }} + {{- if .Values.tls.cipherSuites }} + - --tls-cipher-suites={{ .Values.tls.cipherSuites }} {{- end }} - {{- if .Values.controller.tlsCipherSuites }} - - --tls-cipher-suites={{ .Values.controller.tlsCipherSuites }} + {{- if .Values.tls.curvePreferences }} + - --tls-curve-preferences={{ .Values.tls.curvePreferences }} + {{- end }} + {{- if .Values.tls.minVersion }} + - --tls-min-version={{ .Values.tls.minVersion }} + {{- end }} + {{- if .Values.tls.controllerMetricsTLSSecret }} + - --metrics-cert-dir=/etc/metrics {{- end }} env: + - name: METALLB_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name {{- if and .Values.speaker.enabled .Values.speaker.memberlist.enabled }} - name: METALLB_ML_SECRET_NAME value: {{ include "metallb.secretName" . }} @@ -93,20 +104,27 @@ spec: value: frr-k8s {{- end }} ports: - - name: monitoring + - name: metricshttps containerPort: {{ .Values.prometheus.metricsPort }} + {{- if and .Values.controller.webhookMode (ne .Values.controller.webhookMode "disabled") }} - containerPort: 9443 name: webhook-server protocol: TCP + {{- end }} volumeMounts: - mountPath: /tmp/k8s-webhook-server/serving-certs name: cert readOnly: true + {{- if .Values.tls.controllerMetricsTLSSecret }} + - name: metrics-certs + mountPath: /etc/metrics + readOnly: true + {{- end }} {{- if .Values.controller.livenessProbe.enabled }} livenessProbe: httpGet: - path: /metrics - port: monitoring + path: /healthz + port: {{ .Values.controller.livenessProbe.port }} initialDelaySeconds: {{ .Values.controller.livenessProbe.initialDelaySeconds }} periodSeconds: {{ .Values.controller.livenessProbe.periodSeconds }} timeoutSeconds: {{ .Values.controller.livenessProbe.timeoutSeconds }} @@ -116,8 +134,8 @@ spec: {{- if .Values.controller.readinessProbe.enabled }} readinessProbe: httpGet: - path: /metrics - port: monitoring + path: /readyz + port: {{ .Values.controller.readinessProbe.port }} initialDelaySeconds: {{ .Values.controller.readinessProbe.initialDelaySeconds }} periodSeconds: {{ .Values.controller.readinessProbe.periodSeconds }} timeoutSeconds: {{ .Values.controller.readinessProbe.timeoutSeconds }} @@ -134,34 +152,6 @@ spec: capabilities: drop: - ALL - {{- if .Values.prometheus.secureMetricsPort }} - - name: kube-rbac-proxy - image: {{ .Values.prometheus.rbacProxy.repository }}:{{ .Values.prometheus.rbacProxy.tag }} - imagePullPolicy: {{ .Values.prometheus.rbacProxy.pullPolicy }} - args: - - --logtostderr - - --secure-listen-address=:{{ .Values.prometheus.secureMetricsPort }} - - --upstream=http://127.0.0.1:{{ .Values.prometheus.metricsPort }}/ - - --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 - {{- if .Values.prometheus.controllerMetricsTLSSecret }} - - --tls-private-key-file=/etc/metrics/tls.key - - --tls-cert-file=/etc/metrics/tls.crt - {{- end }} - ports: - - containerPort: {{ .Values.prometheus.secureMetricsPort }} - name: metricshttps - resources: - requests: - cpu: 10m - memory: 20Mi - terminationMessagePolicy: FallbackToLogsOnError - {{- if .Values.prometheus.controllerMetricsTLSSecret }} - volumeMounts: - - name: metrics-certs - mountPath: /etc/metrics - readOnly: true - {{- end }} - {{ end }} {{- if .Values.controller.extraContainers }} {{- toYaml .Values.controller.extraContainers | nindent 6 }} {{- end }} @@ -186,9 +176,9 @@ spec: secret: defaultMode: 420 secretName: metallb-webhook-cert - {{- if .Values.prometheus.controllerMetricsTLSSecret }} + {{- if .Values.tls.controllerMetricsTLSSecret }} - name: metrics-certs secret: - secretName: {{ .Values.prometheus.controllerMetricsTLSSecret }} + secretName: {{ .Values.tls.controllerMetricsTLSSecret }} {{- end }} {{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/exclude-l2-config.yaml b/packages/system/metallb/charts/metallb/templates/exclude-l2-config.yaml index 932c2d62a..be10e15a9 100644 --- a/packages/system/metallb/charts/metallb/templates/exclude-l2-config.yaml +++ b/packages/system/metallb/charts/metallb/templates/exclude-l2-config.yaml @@ -22,4 +22,5 @@ data: - ^kube-ipvs.* - ^cni.* - ^nodelocaldns.* -{{- end }} \ No newline at end of file + - ^lxc.* +{{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/networkpolicy.yaml b/packages/system/metallb/charts/metallb/templates/networkpolicy.yaml new file mode 100644 index 000000000..40e47babd --- /dev/null +++ b/packages/system/metallb/charts/metallb/templates/networkpolicy.yaml @@ -0,0 +1,42 @@ +{{- if .Values.networkpolicies.enabled }} +{{- if .Values.networkpolicies.defaultDeny }} +--- +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: default-deny + namespace: {{ .Release.Namespace | quote }} +spec: + podSelector: {} + policyTypes: + - Ingress + - Egress +{{- end }} +--- +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: controller + namespace: {{ .Release.Namespace | quote }} +spec: + podSelector: + matchLabels: + {{- include "metallb.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: controller + egress: + - ports: + - protocol: TCP + port: {{ .Values.networkpolicies.apiPort }} + ingress: + - ports: + - protocol: TCP + port: metricshttps + {{- if and .Values.controller.webhookMode (ne .Values.controller.webhookMode "disabled") }} + - protocol: TCP + port: webhook-server + {{- end }} + policyTypes: + - Egress + - Ingress +# speaker pods do not need network policy because they are hostnetworked +{{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/podmonitor.yaml b/packages/system/metallb/charts/metallb/templates/podmonitor.yaml index 42de88189..206f85683 100644 --- a/packages/system/metallb/charts/metallb/templates/podmonitor.yaml +++ b/packages/system/metallb/charts/metallb/templates/podmonitor.yaml @@ -25,6 +25,7 @@ spec: podMetricsEndpoints: - port: monitoring path: /metrics + scheme: https {{- if .Values.prometheus.podMonitor.interval }} interval: {{ .Values.prometheus.podMonitor.interval }} {{- end }} @@ -64,6 +65,7 @@ spec: podMetricsEndpoints: - port: monitoring path: /metrics + scheme: https {{- if .Values.prometheus.podMonitor.interval }} interval: {{ .Values.prometheus.podMonitor.interval }} {{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/rbac.yaml b/packages/system/metallb/charts/metallb/templates/rbac.yaml index 11d2ae047..baa07bfba 100644 --- a/packages/system/metallb/charts/metallb/templates/rbac.yaml +++ b/packages/system/metallb/charts/metallb/templates/rbac.yaml @@ -28,19 +28,23 @@ rules: - apiGroups: ["apiextensions.k8s.io"] resources: ["customresourcedefinitions"] resourceNames: ["bfdprofiles.metallb.io","bgpadvertisements.metallb.io", - "bgppeers.metallb.io","ipaddresspools.metallb.io","l2advertisements.metallb.io","communities.metallb.io"] + "bgppeers.metallb.io","ipaddresspools.metallb.io","l2advertisements.metallb.io","communities.metallb.io","configurationstates.metallb.io"] verbs: ["create", "delete", "get", "list", "patch", "update", "watch"] - apiGroups: ["apiextensions.k8s.io"] resources: ["customresourcedefinitions"] verbs: ["list", "watch"] -{{- if .Values.prometheus.secureMetricsPort }} +- apiGroups: ["metallb.io"] + resources: ["configurationstates"] + verbs: ["create", "delete", "get", "list", "patch", "update", "watch"] +- apiGroups: ["metallb.io"] + resources: ["configurationstates/status"] + verbs: ["get", "patch", "update"] - apiGroups: ["authentication.k8s.io"] resources: ["tokenreviews"] verbs: ["create"] - apiGroups: ["authorization.k8s.io"] resources: ["subjectaccessreviews"] verbs: ["create"] -{{- end }} {{- if .Values.speaker.enabled }} --- apiVersion: rbac.authorization.k8s.io/v1 @@ -60,16 +64,14 @@ rules: resources: ["events"] verbs: ["create", "patch"] - apiGroups: ["metallb.io"] - resources: ["servicel2statuses","servicel2statuses/status"] + resources: ["servicel2statuses","servicel2statuses/status","configurationstates","configurationstates/status"] verbs: ["*"] -{{- if .Values.prometheus.secureMetricsPort }} - apiGroups: ["authentication.k8s.io"] resources: ["tokenreviews"] verbs: ["create"] - apiGroups: ["authorization.k8s.io"] resources: ["subjectaccessreviews"] verbs: ["create"] -{{- end }} {{- if or .Values.frrk8s.enabled .Values.frrk8s.external }} - apiGroups: ["frrk8s.metallb.io"] resources: ["frrconfigurations"] @@ -202,6 +204,8 @@ roleRef: kind: Role name: {{ include "metallb.fullname" . }}-pod-lister subjects: +- kind: ServiceAccount + name: {{ template "metallb.controller.serviceAccountName" . }} - kind: ServiceAccount name: {{ include "metallb.speaker.serviceAccountName" . }} {{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/servicemonitor.yaml b/packages/system/metallb/charts/metallb/templates/servicemonitor.yaml index 2a92e48a8..277abd367 100644 --- a/packages/system/metallb/charts/metallb/templates/servicemonitor.yaml +++ b/packages/system/metallb/charts/metallb/templates/servicemonitor.yaml @@ -23,6 +23,8 @@ spec: endpoints: - port: {{ template "metrics.exposedportname" . }} honorLabels: true + bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" + scheme: "https" {{- if .Values.prometheus.serviceMonitor.metricRelabelings }} metricRelabelings: {{- toYaml .Values.prometheus.serviceMonitor.metricRelabelings | nindent 8 }} @@ -33,28 +35,22 @@ spec: {{- end }} {{- if .Values.prometheus.serviceMonitor.interval }} interval: {{ .Values.prometheus.serviceMonitor.interval }} - {{- end -}} -{{ if .Values.prometheus.secureMetricsPort }} - bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" - scheme: "https" + {{- end }} {{- if .Values.prometheus.serviceMonitor.speaker.tlsConfig }} tlsConfig: -{{ toYaml .Values.prometheus.serviceMonitor.speaker.tlsConfig | indent 8 }} +{{ toYaml .Values.prometheus.serviceMonitor.speaker.tlsConfig | indent 8 }} {{- end }} -{{ end }} {{- if .Values.speaker.frr.enabled }} - port: {{ template "metrics.exposedfrrportname" . }} honorLabels: true -{{ if .Values.speaker.frr.secureMetricsPort }} + bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" + scheme: "https" {{- if .Values.prometheus.serviceMonitor.interval }} interval: {{ .Values.prometheus.serviceMonitor.interval }} {{- end }} - bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" - scheme: "https" {{- if .Values.prometheus.serviceMonitor.speaker.tlsConfig }} tlsConfig: -{{ toYaml .Values.prometheus.serviceMonitor.speaker.tlsConfig | indent 8 }} -{{- end }} +{{ toYaml .Values.prometheus.serviceMonitor.speaker.tlsConfig | indent 8 }} {{- end }} {{- end }} jobLabel: {{ .Values.prometheus.serviceMonitor.jobLabel | quote }} @@ -70,6 +66,7 @@ kind: Service metadata: annotations: prometheus.io/scrape: "true" + prometheus.io/scheme: "https" {{- if .Values.prometheus.serviceMonitor.speaker.annotations }} {{ toYaml .Values.prometheus.serviceMonitor.speaker.annotations | indent 4 }} {{- end }} @@ -113,6 +110,9 @@ metadata: spec: endpoints: - port: {{ template "metrics.exposedportname" . }} + honorLabels: true + bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" + scheme: "https" {{- if .Values.prometheus.serviceMonitor.metricRelabelings }} metricRelabelings: {{- toYaml .Values.prometheus.serviceMonitor.metricRelabelings | nindent 8 }} @@ -124,14 +124,9 @@ spec: {{- if .Values.prometheus.serviceMonitor.interval }} interval: {{ .Values.prometheus.serviceMonitor.interval }} {{- end }} - honorLabels: true -{{- if .Values.prometheus.secureMetricsPort }} - bearerTokenFile: "/var/run/secrets/kubernetes.io/serviceaccount/token" - scheme: "https" {{- if .Values.prometheus.serviceMonitor.controller.tlsConfig }} tlsConfig: -{{ toYaml .Values.prometheus.serviceMonitor.controller.tlsConfig | indent 8 }} -{{- end }} +{{ toYaml .Values.prometheus.serviceMonitor.controller.tlsConfig | indent 8 }} {{- end }} jobLabel: {{ .Values.prometheus.serviceMonitor.jobLabel | quote }} namespaceSelector: @@ -146,6 +141,7 @@ kind: Service metadata: annotations: prometheus.io/scrape: "true" + prometheus.io/scheme: "https" {{- if .Values.prometheus.serviceMonitor.controller.annotations }} {{ toYaml .Values.prometheus.serviceMonitor.controller.annotations | indent 4 }} {{- end }} diff --git a/packages/system/metallb/charts/metallb/templates/speaker.yaml b/packages/system/metallb/charts/metallb/templates/speaker.yaml index 8c3dbf28b..f3da9f9ac 100644 --- a/packages/system/metallb/charts/metallb/templates/speaker.yaml +++ b/packages/system/metallb/charts/metallb/templates/speaker.yaml @@ -115,7 +115,7 @@ data: frr defaults traditional hostname Router line vty - log file /etc/frr/frr.log informational + log stdout informational {{- end }} --- {{- if .Values.speaker.enabled }} @@ -145,6 +145,7 @@ spec: {{- if .Values.prometheus.scrapeAnnotations }} prometheus.io/scrape: "true" {{- if not .Values.speaker.frr.enabled }} + prometheus.io/scheme: "https" prometheus.io/port: "{{ .Values.prometheus.metricsPort }}" {{- end }} {{- end }} @@ -174,10 +175,10 @@ spec: {{- toYaml .Values.speaker.securityContext | nindent 8 }} {{- end }} volumes: - {{- if .Values.prometheus.speakerMetricsTLSSecret }} + {{- if .Values.tls.speakerMetricsTLSSecret }} - name: metrics-certs secret: - secretName: {{ .Values.prometheus.speakerMetricsTLSSecret }} + secretName: {{ .Values.tls.speakerMetricsTLSSecret }} {{- end }} {{- if .Values.speaker.memberlist.enabled }} - name: memberlist @@ -191,6 +192,10 @@ spec: defaultMode: 256 name: metallb-excludel2 {{- end }} + - name: memberlist-config + configMap: + name: metallb-memberlist-config + optional: true {{- if .Values.speaker.frr.enabled }} - name: frr-sockets emptyDir: {} @@ -203,6 +208,12 @@ spec: emptyDir: {} - name: metrics emptyDir: {} + - name: frr-tmp + emptyDir: {} + - name: frr-lib + emptyDir: {} + - name: frr-log + emptyDir: {} initContainers: # Copies the initial config files with the right permissions to the shared volume. - name: cp-frr-files @@ -216,6 +227,10 @@ spec: mountPath: /tmp/frr - name: frr-conf mountPath: /etc/frr + {{- with .Values.speaker.initContainers.cpFrrFiles.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} # Copies the reloader to the shared volume between the speaker and reloader. - name: cp-reloader image: {{ .Values.speaker.image.repository }}:{{ .Values.speaker.image.tag | default .Chart.AppVersion }} @@ -223,6 +238,10 @@ spec: volumeMounts: - name: reloader mountPath: /etc/frr_reloader + {{- with .Values.speaker.initContainers.cpReloader.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} # Copies the metrics exporter - name: cp-metrics image: {{ .Values.speaker.image.repository }}:{{ .Values.speaker.image.tag | default .Chart.AppVersion }} @@ -230,6 +249,10 @@ spec: volumeMounts: - name: metrics mountPath: /etc/frr_metrics + {{- with .Values.speaker.initContainers.cpMetrics.resources }} + resources: + {{- toYaml . | nindent 12 }} + {{- end }} shareProcessNamespace: true {{- end }} containers: @@ -256,12 +279,24 @@ spec: {{- if .Values.speaker.ignoreExcludeLB}} - --ignore-exclude-lb {{- end }} - {{- if .Values.prometheus.secureMetricsPort }} - - --host=localhost + {{- if .Values.speaker.bgpDebounceTimeout }} + - --bgp-debounce-timeout={{ .Values.speaker.bgpDebounceTimeout }} {{- end }} {{- if .Values.frrk8s.external }} - --frrk8s-namespace={{ required "namespace is required when frrk8s is external" .Values.frrk8s.namespace }} {{- end }} + {{- if .Values.tls.cipherSuites }} + - --tls-cipher-suites={{ .Values.tls.cipherSuites }} + {{- end }} + {{- if .Values.tls.curvePreferences }} + - --tls-curve-preferences={{ .Values.tls.curvePreferences }} + {{- end }} + {{- if .Values.tls.minVersion }} + - --tls-min-version={{ .Values.tls.minVersion }} + {{- end }} + {{- if .Values.tls.speakerMetricsTLSSecret }} + - --metrics-cert-dir=/etc/metrics + {{- end }} env: - name: METALLB_NODE_NAME valueFrom: @@ -305,7 +340,7 @@ spec: fieldRef: fieldPath: metadata.name ports: - - name: monitoring + - name: metricshttps containerPort: {{ .Values.prometheus.metricsPort }} {{- if .Values.speaker.memberlist.enabled }} - name: memberlist-tcp @@ -318,11 +353,9 @@ spec: {{- if .Values.speaker.livenessProbe.enabled }} livenessProbe: httpGet: - {{- if .Values.prometheus.secureMetricsPort }} - host: localhost - {{- end }} - path: /metrics - port: monitoring + path: /healthz + port: {{ .Values.speaker.livenessProbe.port }} + host: 127.0.0.1 initialDelaySeconds: {{ .Values.speaker.livenessProbe.initialDelaySeconds }} periodSeconds: {{ .Values.speaker.livenessProbe.periodSeconds }} timeoutSeconds: {{ .Values.speaker.livenessProbe.timeoutSeconds }} @@ -332,11 +365,9 @@ spec: {{- if .Values.speaker.readinessProbe.enabled }} readinessProbe: httpGet: - {{- if .Values.prometheus.secureMetricsPort }} - host: localhost - {{- end }} - path: /metrics - port: monitoring + path: /readyz + port: {{ .Values.speaker.readinessProbe.port }} + host: 127.0.0.1 initialDelaySeconds: {{ .Values.speaker.readinessProbe.initialDelaySeconds }} periodSeconds: {{ .Values.speaker.readinessProbe.periodSeconds }} timeoutSeconds: {{ .Values.speaker.readinessProbe.timeoutSeconds }} @@ -355,8 +386,11 @@ spec: - ALL add: - NET_RAW - {{- if or .Values.speaker.frr.enabled .Values.speaker.memberlist.enabled .Values.speaker.excludeInterfaces.enabled }} + {{- if or .Values.speaker.frr.enabled .Values.speaker.memberlist.enabled .Values.speaker.excludeInterfaces.enabled .Values.tls.speakerMetricsTLSSecret }} volumeMounts: + - name: memberlist-config + mountPath: /etc/metallb/memberlist + readOnly: true {{- if .Values.speaker.memberlist.enabled }} - name: memberlist mountPath: {{ .Values.speaker.memberlist.mlSecretKeyPath }} @@ -369,10 +403,17 @@ spec: - name: metallb-excludel2 mountPath: /etc/metallb {{- end }} + {{- if .Values.tls.speakerMetricsTLSSecret }} + - name: metrics-certs + mountPath: /etc/metrics + readOnly: true + {{- end }} {{- end }} {{- if .Values.speaker.frr.enabled }} - name: frr securityContext: + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false capabilities: add: - NET_ADMIN @@ -391,20 +432,14 @@ spec: mountPath: /var/run/frr - name: frr-conf mountPath: /etc/frr - # The command is FRR's default entrypoint & waiting for the log file to appear and tailing it. - # If the log file isn't created in 60 seconds the tail fails and the container is restarted. - # This workaround is needed to have the frr logs as part of kubectl logs -c frr < speaker_pod_name >. + - name: frr-tmp + mountPath: /var/tmp/frr + - name: frr-lib + mountPath: /var/lib/frr command: - /bin/sh - -c - - | - /sbin/tini -- /usr/lib/frr/docker-start & - attempts=0 - until [[ -f /etc/frr/frr.log || $attempts -eq 60 ]]; do - sleep 1 - attempts=$(( $attempts + 1 )) - done - tail -f /etc/frr/frr.log + - /sbin/tini -- /usr/lib/frr/docker-start {{- with .Values.speaker.frr.resources }} resources: {{- toYaml . | nindent 12 }} @@ -412,11 +447,9 @@ spec: {{- if .Values.speaker.livenessProbe.enabled }} livenessProbe: httpGet: - {{- if .Values.prometheus.secureMetricsPort }} - host: localhost - {{- end }} - path: livez + path: /livez port: {{ .Values.speaker.frr.metricsPort }} + scheme: HTTPS initialDelaySeconds: {{ .Values.speaker.livenessProbe.initialDelaySeconds }} periodSeconds: {{ .Values.speaker.livenessProbe.periodSeconds }} timeoutSeconds: {{ .Values.speaker.livenessProbe.timeoutSeconds }} @@ -426,11 +459,9 @@ spec: {{- if .Values.speaker.startupProbe.enabled }} startupProbe: httpGet: - {{- if .Values.prometheus.secureMetricsPort }} - host: localhost - {{- end }} path: /livez port: {{ .Values.speaker.frr.metricsPort }} + scheme: HTTPS failureThreshold: {{ .Values.speaker.startupProbe.failureThreshold }} periodSeconds: {{ .Values.speaker.startupProbe.periodSeconds }} {{- end }} @@ -439,6 +470,9 @@ spec: {{- if .Values.speaker.frr.image.pullPolicy }} imagePullPolicy: {{ .Values.speaker.frr.image.pullPolicy }} {{- end }} + securityContext: + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false command: ["/etc/frr_reloader/frr-reloader.sh"] volumeMounts: - name: frr-sockets @@ -447,24 +481,40 @@ spec: mountPath: /etc/frr - name: reloader mountPath: /etc/frr_reloader + - name: frr-log + mountPath: /var/log/frr {{- with .Values.speaker.reloader.resources }} resources: {{- toYaml . | nindent 12 }} {{- end }} - name: frr-metrics image: {{ .Values.speaker.frr.image.repository }}:{{ .Values.speaker.frr.image.tag | default .Chart.AppVersion }} + securityContext: + readOnlyRootFilesystem: true + allowPrivilegeEscalation: false command: ["/etc/frr_metrics/frr-metrics"] args: - --metrics-port={{ .Values.speaker.frr.metricsPort }} - {{- if .Values.prometheus.secureMetricsPort }} - - --host=localhost + - --metrics-bind-address=0.0.0.0 + {{- if .Values.tls.speakerMetricsTLSSecret }} + - --tls-cert-file=/etc/metrics/tls.crt + - --tls-private-key-file=/etc/metrics/tls.key + {{- end }} + {{- if .Values.tls.cipherSuites }} + - --tls-cipher-suites={{ .Values.tls.cipherSuites }} + {{- end }} + {{- if .Values.tls.curvePreferences }} + - --tls-curve-preferences={{ .Values.tls.curvePreferences }} + {{- end }} + {{- if .Values.tls.minVersion }} + - --tls-min-version={{ .Values.tls.minVersion }} {{- end }} env: - name: VTYSH_HISTFILE value: /dev/null ports: - containerPort: {{ .Values.speaker.frr.metricsPort }} - name: monitoring + name: frrmetricshttps volumeMounts: - name: frr-sockets mountPath: /var/run/frr @@ -472,74 +522,16 @@ spec: mountPath: /etc/frr - name: metrics mountPath: /etc/frr_metrics + {{- if .Values.tls.speakerMetricsTLSSecret }} + - name: metrics-certs + mountPath: /etc/metrics + readOnly: true + {{- end }} {{- with .Values.speaker.frrMetrics.resources }} resources: {{- toYaml . | nindent 12 }} {{- end }} {{- end }} - {{- if .Values.prometheus.secureMetricsPort }} - - name: kube-rbac-proxy - image: {{ .Values.prometheus.rbacProxy.repository }}:{{ .Values.prometheus.rbacProxy.tag }} - imagePullPolicy: {{ .Values.prometheus.rbacProxy.pullPolicy }} - args: - - --logtostderr - - --secure-listen-address=:{{ .Values.prometheus.secureMetricsPort }} - - --upstream=http://localhost:{{ .Values.prometheus.metricsPort }}/ - - --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 - {{- if .Values.prometheus.speakerMetricsTLSSecret }} - - --tls-private-key-file=/etc/metrics/tls.key - - --tls-cert-file=/etc/metrics/tls.crt - {{- end }} - ports: - - containerPort: {{ .Values.prometheus.secureMetricsPort }} - name: metricshttps - resources: - requests: - cpu: 10m - memory: 20Mi - terminationMessagePolicy: FallbackToLogsOnError - {{- if .Values.prometheus.speakerMetricsTLSSecret }} - volumeMounts: - - name: metrics-certs - mountPath: /etc/metrics - readOnly: true - {{- end }} - {{ end }} - {{- if .Values.speaker.frr.enabled }} - {{- if .Values.speaker.frr.secureMetricsPort }} - - name: kube-rbac-proxy-frr - image: {{ .Values.prometheus.rbacProxy.repository }}:{{ .Values.prometheus.rbacProxy.tag | default .Chart.AppVersion }} - imagePullPolicy: {{ .Values.prometheus.rbacProxy.pullPolicy }} - args: - - --logtostderr - - --secure-listen-address=:{{ .Values.speaker.frr.secureMetricsPort }} - - --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 - - --upstream=http://localhost:{{ .Values.speaker.frr.metricsPort }}/ - {{- if .Values.prometheus.speakerMetricsTLSSecret }} - - --tls-private-key-file=/etc/metrics/tls.key - - --tls-cert-file=/etc/metrics/tls.crt - {{- end }} - ports: - - containerPort: {{ .Values.speaker.frr.secureMetricsPort }} - name: frrmetricshttps - env: - - name: METALLB_HOST - valueFrom: - fieldRef: - fieldPath: status.hostIP - resources: - requests: - cpu: 10m - memory: 20Mi - terminationMessagePolicy: FallbackToLogsOnError - {{- if .Values.prometheus.speakerMetricsTLSSecret }} - volumeMounts: - - name: metrics-certs - mountPath: /etc/metrics - readOnly: true - {{- end }} - {{ end }} - {{- end }} {{- if .Values.speaker.extraContainers }} {{- toYaml .Values.speaker.extraContainers | nindent 6 }} {{- end }} diff --git a/packages/system/metallb/charts/metallb/values.schema.json b/packages/system/metallb/charts/metallb/values.schema.json index bc0dd8402..54ee991dd 100644 --- a/packages/system/metallb/charts/metallb/values.schema.json +++ b/packages/system/metallb/charts/metallb/values.schema.json @@ -182,14 +182,6 @@ "rbacPrometheus": { "type": "boolean" }, "serviceAccount": { "type": "string" }, "namespace": { "type": "string" }, - "rbacProxy": { - "description": "kube-rbac-proxy configuration", - "type": "object", - "properties": { - "repository": { "type": "string" }, - "tag": { "type": "string" } - } - }, "podMonitor": { "description": "Prometheus Operator PodMonitors", "type": "object", @@ -367,6 +359,12 @@ "ignoreExcludeLB": { "type": "boolean" }, + "bgpDebounceTimeout": { + "anyOf": [ + { "type": "integer", "minimum": 0 }, + { "type": "null" } + ] + }, "updateStrategy": { "type": "object", "properties": { @@ -414,6 +412,29 @@ "resources": { "type": "object" } } }, + "initContainers": { + "type": "object", + "properties": { + "cpFrrFiles": { + "type": "object", + "properties": { + "resources": { "type": "object" } + } + }, + "cpReloader": { + "type": "object", + "properties": { + "resources": { "type": "object" } + } + }, + "cpMetrics": { + "type": "object", + "properties": { + "resources": { "type": "object" } + } + } + } + }, "extraContainers": { "type": "array", "items": { diff --git a/packages/system/metallb/charts/metallb/values.yaml b/packages/system/metallb/charts/metallb/values.yaml index fc6e0b233..391b28720 100644 --- a/packages/system/metallb/charts/metallb/values.yaml +++ b/packages/system/metallb/charts/metallb/values.yaml @@ -14,6 +14,18 @@ rbac: # create specifies whether to install and use RBAC rules. create: true +tls: + # -- Comma-separated list of TLS cipher suites. If empty, uses Go defaults. Only applies to TLS 1.2. + cipherSuites: "" + # -- Comma-separated list of numeric CurveID values (e.g. 29,4588). See https://pkg.go.dev/crypto/tls#CurveID. If empty, uses Go defaults. + curvePreferences: "" + # -- Minimum TLS version (VersionTLS12 or VersionTLS13). Defaults to VersionTLS13. + minVersion: "" + # -- The name of the secret to be mounted in the controller pod to provide TLS certificates for metrics endpoints. If not present, a self-signed certificate is auto-generated. + controllerMetricsTLSSecret: "" + # -- The name of the secret to be mounted in the speaker pod to provide TLS certificates for metrics endpoints. If not present, a self-signed certificate is auto-generated. + speakerMetricsTLSSecret: "" + prometheus: # scrape annotations specifies whether to add Prometheus metric # auto-collection annotations to pods. See @@ -25,22 +37,9 @@ prometheus: # can be left at false. scrapeAnnotations: false - # port both controller and speaker will listen on for metrics - metricsPort: 7472 - - # if set, enables rbac proxy on the controller and speaker to expose - # the metrics via tls. - # secureMetricsPort: 9120 - - # the name of the secret to be mounted in the speaker pod - # to expose the metrics securely. If not present, a self signed - # certificate to be used. - speakerMetricsTLSSecret: "" - - # the name of the secret to be mounted in the controller pod - # to expose the metrics securely. If not present, a self signed - # certificate to be used. - controllerMetricsTLSSecret: "" + # port both controller and speaker will listen on for metrics (always HTTPS). + # Matches the port kube-rbac-proxy previously served on. + metricsPort: 9120 # prometheus doesn't have the permission to scrape all namespaces so we give it permission to scrape metallb's one rbacPrometheus: true @@ -53,12 +52,6 @@ prometheus: # required when " .Values.prometheus.rbacPrometheus == true " and " .Values.prometheus.podMonitor.enabled=true or prometheus.serviceMonitor.enabled=true " namespace: "" - # the image to be used for the kuberbacproxy container - rbacProxy: - repository: gcr.io/kubebuilder/kube-rbac-proxy - tag: v0.12.0 - pullPolicy: - # Prometheus Operator PodMonitors podMonitor: # enable support for Prometheus Operator @@ -199,7 +192,7 @@ controller: # -- Controller log level. Must be one of: `all`, `debug`, `info`, `warn`, `error` or `none` logLevel: info # command: /controller - # webhookMode: enabled + webhookMode: enabled image: repository: quay.io/metallb/controller tag: @@ -240,6 +233,7 @@ controller: labels: {} livenessProbe: enabled: true + port: 17472 failureThreshold: 3 initialDelaySeconds: 10 periodSeconds: 10 @@ -247,13 +241,12 @@ controller: timeoutSeconds: 1 readinessProbe: enabled: true + port: 17472 failureThreshold: 3 initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 - tlsMinVersion: "VersionTLS12" - tlsCipherSuites: "" extraContainers: [] @@ -275,6 +268,8 @@ speaker: enabled: true # ignore the exclude-from-external-loadbalancer label ignoreExcludeLB: false + # -- BGP debounce timeout for FRR configuration reloads, in milliseconds. Only applies when BGP type is frr. Default (when unset) is 3000 ms. This feature is experimental + bgpDebounceTimeout: null image: repository: quay.io/metallb/speaker @@ -314,6 +309,7 @@ speaker: labels: {} livenessProbe: enabled: true + port: 17472 failureThreshold: 3 initialDelaySeconds: 10 periodSeconds: 10 @@ -321,6 +317,7 @@ speaker: timeoutSeconds: 1 readinessProbe: enabled: true + port: 17472 failureThreshold: 3 initialDelaySeconds: 10 periodSeconds: 10 @@ -332,26 +329,32 @@ speaker: periodSeconds: 5 # frr contains configuration specific to the MetalLB FRR container, # for speaker running alongside FRR. + # DEPRECATED: The FRR mode is deprecated and will be removed in a future + # release. Use the frr-k8s mode (frrk8s.enabled) instead, which is now + # the default BGP backend. frr: - enabled: true + enabled: false image: repository: quay.io/frrouting/frr - tag: 9.1.0 + tag: 10.5.3 pullPolicy: - metricsPort: 7473 + metricsPort: 9121 resources: {} - # if set, enables a rbac proxy sidecar container on the speaker to - # expose the frr metrics via tls. - # secureMetricsPort: 9121 - - reloader: resources: {} frrMetrics: resources: {} + initContainers: + cpFrrFiles: + resources: {} + cpReloader: + resources: {} + cpMetrics: + resources: {} + extraContainers: [] crds: @@ -363,8 +366,40 @@ crds: # This allows configuring additional frr parameters in combination to those # applied by MetalLB. frrk8s: - # if set, enables frrk8s as a backend. This is mutually exclusive to frr - # mode. - enabled: false + # -- If set, enables frrk8s as a backend. This is mutually exclusive to frr mode. + enabled: true + # -- If true, uses an external frr-k8s installation instead of the bundled subchart. external: false + # -- Namespace where external frr-k8s is installed (only used when external=true). namespace: "" + +# Values passed to the frr-k8s subchart (note the hyphen in "frr-k8s"). +# For all available options, see: +# https://github.com/metallb/frr-k8s/blob/main/charts/frr-k8s/values.yaml +frr-k8s: + prometheus: + serviceMonitor: + # -- Enable Prometheus ServiceMonitor for frr-k8s metrics. + enabled: false + # The FRR-K8s BGP/BFD metrics are exposed with the "frrk8s_" prefix + # (e.g. frrk8s_bgp_session_up, frrk8s_bfd_session_up). + # To rename them to the legacy "metallb_" prefix for backward compatibility + # with existing dashboards or alerts, enable and configure metric relabelings: + # metricRelabelings: + # - sourceLabels: [__name__] + # regex: "frrk8s_bgp_(.*)" + # targetLabel: "__name__" + # replacement: "metallb_bgp_$1" + # - sourceLabels: [__name__] + # regex: "frrk8s_bfd_(.*)" + # targetLabel: "__name__" + # replacement: "metallb_bfd_$1" + +# networkpolicies +networkpolicies: + # if set, networkpolicies for metallb components will be installed in the metallb namespace + enabled: false + # if set, a default deny network policy will be installed in the metallb namespace + defaultDeny: false + # to override internal k8s api targetPort + apiPort: 6443 diff --git a/packages/system/metallb/tests/metallb_test.yaml b/packages/system/metallb/tests/metallb_test.yaml new file mode 100644 index 000000000..2f11c0ffd --- /dev/null +++ b/packages/system/metallb/tests/metallb_test.yaml @@ -0,0 +1,185 @@ +suite: cozy-metallb chart rendering invariants +release: + name: metallb + namespace: cozy-metallb +set: + # The upstream metallb chart leaves `frr-k8s.prometheus:` as a YAML null + # with only commented examples beneath it. Helm merges that null on top of + # the frr-k8s subchart's own `prometheus:` map and `helm template` happens + # to short-circuit the missing `serviceMonitor.enabled` lookup, but + # helm-unittest's value-merge dereferences the nil and panics. Pin the + # subchart's prometheus block here so all templates render cleanly inside + # the suite — this does not change the production rendering path. + metallb.frr-k8s.prometheus.serviceMonitor.enabled: false + metallb.frr-k8s.prometheus.podMonitor.enabled: false +tests: + - it: renders the speaker DaemonSet with a single speaker container + template: charts/metallb/templates/speaker.yaml + documentSelector: + path: kind + value: DaemonSet + asserts: + - equal: + path: metadata.name + value: metallb-speaker + - lengthEqual: + path: spec.template.spec.containers + count: 1 + - equal: + path: spec.template.spec.containers[0].name + value: speaker + + - it: drives the speaker into frr-k8s BGP mode + template: charts/metallb/templates/speaker.yaml + documentSelector: + path: kind + value: DaemonSet + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: METALLB_BGP_TYPE + value: frr-k8s + + - it: ships the frr-k8s DaemonSet alongside the speaker + template: charts/metallb/charts/frr-k8s/templates/controller.yaml + documentSelector: + path: kind + value: DaemonSet + asserts: + - equal: + path: metadata.name + value: metallb-frr-k8s + - lengthEqual: + path: spec.template.spec.containers + count: 5 + - equal: + path: spec.template.spec.containers[0].name + value: controller + + - it: keeps the controller Deployment in the release namespace + template: charts/metallb/templates/controller.yaml + documentSelector: + path: kind + value: Deployment + asserts: + - equal: + path: metadata.name + value: metallb-controller + - equal: + path: metadata.namespace + value: cozy-metallb + + - it: installs the configuration-state CRD added in 0.15.3 + template: charts/metallb/charts/crds/templates/crds.yaml + documentSelector: + path: metadata.name + value: configurationstates.metallb.io + asserts: + - isKind: + of: CustomResourceDefinition + + - it: installs the frr-k8s FRRConfiguration CRD + template: charts/metallb/charts/frr-k8s/charts/crds/templates/frrk8s.metallb.io_frrconfigurations.yaml + asserts: + - isKind: + of: CustomResourceDefinition + - equal: + path: metadata.name + value: frrconfigurations.frrk8s.metallb.io + + - it: leaves NetworkPolicies off by default (opt-in via networkpolicies.enabled) + template: charts/metallb/templates/networkpolicy.yaml + asserts: + - hasDocuments: + count: 0 + + # The cozystack outer values.yaml pins .metallb.controller.image.tag and + # .metallb.speaker.image.tag to a ghcr.io/cozystack/cozystack build that + # MUST match the appVersion of the vendored chart. v0.16.0 introduced a + # new probe contract (/healthz on port 17472) that earlier binaries don't + # serve — a bump that lets these two drift produces CrashLoopBackOff pods + # on the default branch between merge and the next release-prep commit. + # The two assertions below pin the rendered image tag to the exact string + # captured at chart appVersion v0.16.1; any in-place edit to values.yaml + # (intentional or otherwise) without a matching test fixture refresh fails + # CI before the change lands. Treat a failure here as "you edited + # values.yaml; if the edit was intentional, run `make image` to push fresh + # digests and update this fixture in the same commit". A future appVersion + # bump in Chart.yaml requires updating both halves: `make image` writes + # the new digest to values.yaml, then the fixture below is regenerated to + # match — the test catches a forgotten regeneration as part of the bump. + - it: pins controller image tag to the chart appVersion + template: charts/metallb/templates/controller.yaml + documentSelector: + path: kind + value: Deployment + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: ghcr.io/cozystack/cozystack/metallb-controller:v0.16.1@sha256:0d8d567780c0f1d487e15a5df2377c40f9c608a55f1a24581052c7c69eb4ccfd + + - it: pins speaker image tag to the chart appVersion + template: charts/metallb/templates/speaker.yaml + documentSelector: + path: kind + value: DaemonSet + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: ghcr.io/cozystack/cozystack/metallb-speaker:v0.16.1@sha256:aa16cee1c88923b67dbfcf69fe3e4dcb9a4ded95125a7f078752b4be240b5690 + + # The values.yaml header comment documents that operators can revert the + # speaker to legacy FRR mode by flipping these two keys. Pin both halves + # of the documented behavior so an upstream chart restructure that renames + # frr.enabled / frrk8s.enabled fails the suite before silently breaking + # an operator's downgrade path. + # + # IMPORTANT: this escape hatch is partial. It switches the speaker BGP + # backend but does NOT suppress the frr-k8s subchart — the dep condition + # in the vendored metallb chart does not propagate from the cozy-metallb + # umbrella, so the metallb-frr-k8s DaemonSet remains rendered (idle). + # The companion test below pins that limitation so a future fix changes + # the documented behavior and this expected-but-unfortunate render at + # the same time. + - it: switches speaker into legacy FRR mode under the escape hatch + template: charts/metallb/templates/speaker.yaml + documentSelector: + path: kind + value: DaemonSet + set: + metallb.frrk8s.enabled: false + metallb.speaker.frr.enabled: true + asserts: + - lengthEqual: + path: spec.template.spec.containers + count: 4 + - equal: + path: spec.template.spec.containers[0].name + value: speaker + - equal: + path: spec.template.spec.containers[1].name + value: frr + - contains: + path: spec.template.spec.containers[0].env + content: + name: METALLB_BGP_TYPE + value: frr + + - it: leaves the idle frr-k8s subchart in place under the escape hatch + template: charts/metallb/charts/frr-k8s/templates/controller.yaml + documentSelector: + path: kind + value: DaemonSet + set: + metallb.frrk8s.enabled: false + metallb.speaker.frr.enabled: true + asserts: + # The subchart still renders metallb-frr-k8s. If/when the umbrella + # chart grows a way to fully suppress the dep (e.g. a top-level + # frrk8s.enabled wired into Chart.yaml.dependencies), flip this + # assertion to hasDocuments count: 0 and update the values.yaml + # header comment together. + - equal: + path: metadata.name + value: metallb-frr-k8s diff --git a/packages/system/metallb/values.yaml b/packages/system/metallb/values.yaml index dd5f76e65..988f7b78f 100644 --- a/packages/system/metallb/values.yaml +++ b/packages/system/metallb/values.yaml @@ -1,11 +1,64 @@ +# Notes for operators on bumps that crossed v0.15.x → v0.16.x: +# +# - BGP backend: upstream defaults flipped from FRR mode to FRR-K8s. This chart +# ships frr-k8s as a sibling DaemonSet (cozy-metallb/metallb-frr-k8s). Speaker +# talks to it via in-cluster API; existing BGPPeer / BGPAdvertisement objects +# keep working unchanged. To revert speaker to legacy FRR mode (deprecated +# upstream), set BOTH of these here: +# metallb.frrk8s.enabled: false +# metallb.speaker.frr.enabled: true +# This switches the speaker DaemonSet to the four-container layout (speaker, +# frr, reloader, frr-metrics) and METALLB_BGP_TYPE=frr. The frr-k8s subchart +# stays installed but idle — Helm evaluates a `condition:` on a dep against +# the declaring chart's own values context (here: the vendored metallb +# chart), so a grandparent override from the cozy-metallb umbrella cannot +# suppress that dep via the condition. The metallb-frr-k8s DaemonSet and +# metallb-frr-k8s-statuscleaner Deployment are still rendered with zero +# replicas worth of effect on the chosen BGP path. The frr-k8s metrics ports +# (9140/9141) remain host-network and the cilium-networkpolicy denylist +# covers them; under this escape hatch the additional legacy FRR metrics +# port 9121 on the speaker pod becomes active and the same denylist covers +# it. If you need a fully-clean revert, deinstall the chart and reinstall +# from a prior cozystack release that still ran v0.15.x. +# - Metrics endpoints: kube-rbac-proxy is gone. Speaker and controller now serve +# /metrics directly on HTTPS at port 9120 (was HTTP on 7472). Any external +# scrape config that targeted 7472 must move to 9120 and add a TLS skip / CA. +# - TLS minimum version: upstream removed controller.tlsMinVersion in favor of +# a global tls.minVersion. The new default (empty string) resolves to TLS +# 1.3 per the upstream chart README; the previous chart pinned TLS 1.2. +# Prometheus scrapers without TLS 1.3 support fail until you override: +# metallb.tls.minVersion: VersionTLS12 +# - Host-network ports added by the frr-k8s DaemonSet: 9140 (controller metrics), +# 9141 (FRR metrics), plus 19443 from the frr-k8s statuscleaner Deployment. +# The cozystack cilium-networkpolicy denies world ingress to all of these. metallb: crds: enabled: true controller: image: repository: ghcr.io/cozystack/cozystack/metallb-controller - tag: v0.15.2@sha256:623ce74b5802bff6e29f29478ccab29ce4162a64148be006c69e16cc3207e289 + tag: v0.16.1@sha256:0d8d567780c0f1d487e15a5df2377c40f9c608a55f1a24581052c7c69eb4ccfd speaker: image: repository: ghcr.io/cozystack/cozystack/metallb-speaker - tag: v0.15.2@sha256:f264058afd9228452a260ab9c9dd1859404745627a2a38c2ba4671e27f3b3bb2 + tag: v0.16.1@sha256:aa16cee1c88923b67dbfcf69fe3e4dcb9a4ded95125a7f078752b4be240b5690 + # The vendored metallb chart's values.yaml leaves `frr-k8s.prometheus:` as + # a YAML block containing only commented examples — it parses to null. + # Helm-controller's deep-merge then writes that null over the frr-k8s + # subchart's own `prometheus:` map (which carries the schema-driving + # defaults for serviceMonitor / podMonitor). The next render of + # `frr-k8s/templates/service-monitor.yaml` dereferences nil at + # `.Values.prometheus.serviceMonitor.enabled` and aborts the install + # with "nil pointer evaluating interface {}.serviceMonitor". + # + # Pin the merge target here so the subchart always sees a real (false) + # prometheus block. This matches the corresponding test-fixture + # workaround in tests/metallb_test.yaml and unblocks the bump; the + # underlying upstream bug is filed at metallb/frr-k8s (the schema + # placement issue tracked alongside it). + frr-k8s: + prometheus: + serviceMonitor: + enabled: false + podMonitor: + enabled: false