mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 14:02:56 +09:00
feat(platform): add opt-in in-cluster east-west encryption toggle (#2984)
## What this PR does Adds `networking.encryption.enabled` to the platform chart — a single operator-facing opt-in, off by default, that turns on in-cluster (east-west) transport encryption for the active datapath. When enabled, pod-to-pod traffic across nodes is encrypted at the network layer regardless of whether the workload implements TLS: defense-in-depth for the whole mesh without per-application changes. The default `isp-full` bundle selects the `kubeovn-cilium` datapath (and `isp-full-generic` selects `kubeovn-cilium-generic`), where KubeOVN owns the datapath and the overlay is a Geneve tunnel. The toggle therefore maps to KubeOVN's overlay IPsec (`kube-ovn` `ENABLE_OVN_IPSEC`). KubeOVN's own controller provisions the IPsec CA and per-node certificates through the Kubernetes CSR API, so no cert-manager wiring is needed. This complements, and does not replace, application-level / end-to-end TLS: in-cluster encryption never reaches an external client and gives a tenant no verifiable endpoint certificate. The values comments document the per-datapath scope — including that Kilo's WireGuard mesh (`meshGranularity: location`) covers only cross-location traffic and leaves the intra-location gap this toggle targets, and that the Cilium-native WireGuard/IPsec path on the `cilium` / `cilium-generic` variants is not reached because the system bundle does not select those variants — plus the MTU/throughput overhead. Scope: only the default KubeOVN datapath is wired through the platform toggle, which is what the system bundle actually selects. Wiring the Cilium-native path would require the bundle to expose cilium-only variant selection; that is out of scope here and called out in the values comments. Validation: `helm unittest` on the platform chart passes (66 tests, 7 new) covering on / off / absent / null and verifying the IPsec flag deep-merges without dropping the existing `kube-ovn` overrides, for both `isp-full` and `isp-full-generic`. Closes #2977 Part of #2811 ### Screenshots N/A — no UI changes. ### Release note ```release-note feat(platform): add opt-in `networking.encryption.enabled` (off by default) that enables in-cluster east-west transport encryption for the default KubeOVN datapath via overlay IPsec; complements, and does not replace, application-level TLS. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced opt-in in-cluster network encryption using KubeOVN IPsec for east-west traffic protection. * Added `networking.encryption` configuration toggle (disabled by default) with full backward compatibility. * **Tests** * Added comprehensive test coverage for network encryption configurations across system variants and edge cases. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
3 files changed
+190
No files matched your search
@@ -13,6 +13,15 @@
|
||||
{{- if .Values.networking.kubeovn.MASTER_NODES -}}
|
||||
{{- $_ := set $kubeovnDict "MASTER_NODES" .Values.networking.kubeovn.MASTER_NODES -}}
|
||||
{{- end -}}
|
||||
{{- /* Opt-in east-west encryption (networking.encryption.enabled): on the
|
||||
default kubeovn-cilium datapath the overlay is KubeOVN's Geneve
|
||||
tunnel, so encrypting it means turning on KubeOVN IPsec. KubeOVN's
|
||||
own controller provisions the IPsec CA and per-node certs through
|
||||
the Kubernetes CSR API — no cert-manager wiring needed here. Off by
|
||||
default; complements (does not replace) application-level TLS. */ -}}
|
||||
{{- if dig "encryption" "enabled" false .Values.networking -}}
|
||||
{{- $_ := set $kubeovnDict "func" (dict "ENABLE_OVN_IPSEC" true) -}}
|
||||
{{- end -}}
|
||||
{{- $kubeovnValues := dict "kube-ovn" $kubeovnDict -}}
|
||||
{{- $_ := set $networkingComponents "kubeovn" (dict "values" $kubeovnValues) -}}
|
||||
{{- /* For Talos (isp-full): use KubePrism endpoint and disable cgroup autoMount */ -}}
|
||||
@@ -83,6 +92,12 @@
|
||||
{{- end -}}
|
||||
{{- /* For generic k8s (k3s), control-plane label has value "true" */ -}}
|
||||
{{- $_ := set $kubeovnDict "MASTER_NODES_LABEL" "node-role.kubernetes.io/control-plane=true" -}}
|
||||
{{- /* Opt-in east-west encryption (networking.encryption.enabled): same
|
||||
KubeOVN Geneve-overlay IPsec path as isp-full. See the isp-full
|
||||
branch above for the mechanism notes. */ -}}
|
||||
{{- if dig "encryption" "enabled" false .Values.networking -}}
|
||||
{{- $_ := set $kubeovnDict "func" (dict "ENABLE_OVN_IPSEC" true) -}}
|
||||
{{- end -}}
|
||||
{{- $kubeovnValues := dict "kube-ovn" $kubeovnDict -}}
|
||||
{{- $_ := set $networkingComponents "kubeovn" (dict "values" $kubeovnValues) -}}
|
||||
{{- /* Cilium configuration - for generic k8s, always enable cgroup autoMount */ -}}
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
suite: networking.encryption bundle wiring
|
||||
templates:
|
||||
- templates/bundles/system.yaml
|
||||
release:
|
||||
name: cozystack
|
||||
namespace: cozy-system
|
||||
tests:
|
||||
- it: leaves KubeOVN overlay IPsec unset on the default isp-full datapath when encryption is off
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full
|
||||
networking:
|
||||
encryption:
|
||||
enabled: false
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.variant
|
||||
value: kubeovn-cilium
|
||||
- notExists:
|
||||
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
|
||||
|
||||
- it: leaves KubeOVN overlay IPsec unset on isp-full when networking.encryption is absent (backward compatible, no nil-pointer)
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.variant
|
||||
value: kubeovn-cilium
|
||||
- notExists:
|
||||
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
|
||||
|
||||
- it: enables KubeOVN Geneve-overlay IPsec on the default isp-full datapath when encryption is on
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full
|
||||
networking:
|
||||
encryption:
|
||||
enabled: true
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.variant
|
||||
value: kubeovn-cilium
|
||||
- equal:
|
||||
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
|
||||
value: true
|
||||
|
||||
- it: enables KubeOVN overlay IPsec on the generic kubeovn-cilium-generic datapath when encryption is on
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full-generic
|
||||
networking:
|
||||
encryption:
|
||||
enabled: true
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.variant
|
||||
value: kubeovn-cilium-generic
|
||||
- equal:
|
||||
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
|
||||
value: true
|
||||
|
||||
- it: leaves the generic datapath IPsec unset when encryption is off
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full-generic
|
||||
networking:
|
||||
encryption:
|
||||
enabled: false
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.variant
|
||||
value: kubeovn-cilium-generic
|
||||
- notExists:
|
||||
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
|
||||
|
||||
- it: keeps the existing kube-ovn ipv4 component overrides alongside the IPsec toggle (deep merge, not replace)
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full
|
||||
networking:
|
||||
encryption:
|
||||
enabled: true
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.components.kubeovn.values.kube-ovn.ipv4.POD_CIDR
|
||||
value: 10.244.0.0/16
|
||||
- equal:
|
||||
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
|
||||
value: true
|
||||
|
||||
- it: renders without a nil-pointer error when networking is set to null and encryption is therefore unreachable
|
||||
set:
|
||||
bundles:
|
||||
system:
|
||||
enabled: true
|
||||
variant: isp-full
|
||||
networking: null
|
||||
documentSelector:
|
||||
path: metadata.name
|
||||
value: cozystack.networking
|
||||
asserts:
|
||||
- equal:
|
||||
path: spec.variant
|
||||
value: kubeovn-cilium
|
||||
- notExists:
|
||||
path: spec.components
|
||||
@@ -71,6 +71,44 @@ networking:
|
||||
# Set this to comma-separated list of master node IPs to override.
|
||||
kubeovn:
|
||||
MASTER_NODES: ""
|
||||
# In-cluster (east-west) transport encryption. Opt-in, OFF by default.
|
||||
#
|
||||
# When enabled, pod-to-pod traffic that crosses nodes is encrypted at the
|
||||
# network layer by the ACTIVE datapath, regardless of whether the workload
|
||||
# speaks TLS — defense-in-depth for the whole mesh without per-application
|
||||
# changes. This COMPLEMENTS, and does NOT replace, application-level /
|
||||
# end-to-end TLS: it never reaches an external client and gives a tenant no
|
||||
# verifiable endpoint certificate (see the per-app certificate series). Use
|
||||
# it alongside, not instead of, TLS on exposed endpoints.
|
||||
#
|
||||
# The mechanism is datapath-specific (selected by bundles.system.variant):
|
||||
#
|
||||
# * kubeovn-cilium (isp-full, the default) and kubeovn-cilium-generic
|
||||
# (isp-full-generic): KubeOVN owns the datapath; its Geneve overlay is
|
||||
# encrypted with IPsec (kube-ovn ENABLE_OVN_IPSEC). KubeOVN's own
|
||||
# controller provisions the IPsec CA and per-node certificates through
|
||||
# the Kubernetes CSR API — no cert-manager wiring is required. This is
|
||||
# the path this toggle drives today.
|
||||
# * cilium / cilium-generic (Cilium owns the datapath): Cilium-native
|
||||
# WireGuard/IPsec is the right mechanism there, but the platform system
|
||||
# bundle does not select those variants, so this toggle does not reach
|
||||
# them yet.
|
||||
# * cilium-kilo: Kilo already establishes a WireGuard mesh, but only at
|
||||
# meshGranularity=location — it encrypts CROSS-LOCATION traffic and
|
||||
# leaves intra-location pod-to-pod in cleartext by design. This toggle
|
||||
# targets exactly that intra-location gap and composes with Kilo rather
|
||||
# than conflicting with it.
|
||||
#
|
||||
# CAVEATS: encrypting the overlay adds per-packet ESP overhead and lowers
|
||||
# the effective MTU, so east-west throughput drops and latency rises; the
|
||||
# cost is highest for high-bandwidth flows. IPsec here protects the
|
||||
# inter-node overlay tunnels — same-node pod-to-pod traffic never leaves the
|
||||
# host and is not tunneled or encrypted. Flipping this on an existing
|
||||
# cluster triggers a rollout of the CNI components. On variants without a
|
||||
# cozystack-managed encryptable datapath (e.g. isp-hosted / noop) the toggle
|
||||
# has no effect.
|
||||
encryption:
|
||||
enabled: false
|
||||
# Service publishing and ingress configuration
|
||||
publishing:
|
||||
host: "example.org"
|
||||
|
||||
Reference in new issue
Block a user