feat(platform): add opt-in in-cluster east-west encryption toggle (#2984)

## What this PR does

Adds `networking.encryption.enabled` to the platform chart — a single
operator-facing opt-in, off by default, that turns on in-cluster
(east-west) transport encryption for the active datapath. When enabled,
pod-to-pod traffic across nodes is encrypted at the network layer
regardless of whether the workload implements TLS: defense-in-depth for
the whole mesh without per-application changes.

The default `isp-full` bundle selects the `kubeovn-cilium` datapath (and
`isp-full-generic` selects `kubeovn-cilium-generic`), where KubeOVN owns
the datapath and the overlay is a Geneve tunnel. The toggle therefore
maps to KubeOVN's overlay IPsec (`kube-ovn` `ENABLE_OVN_IPSEC`).
KubeOVN's own controller provisions the IPsec CA and per-node
certificates through the Kubernetes CSR API, so no cert-manager wiring
is needed.

This complements, and does not replace, application-level / end-to-end
TLS: in-cluster encryption never reaches an external client and gives a
tenant no verifiable endpoint certificate. The values comments document
the per-datapath scope — including that Kilo's WireGuard mesh
(`meshGranularity: location`) covers only cross-location traffic and
leaves the intra-location gap this toggle targets, and that the
Cilium-native WireGuard/IPsec path on the `cilium` / `cilium-generic`
variants is not reached because the system bundle does not select those
variants — plus the MTU/throughput overhead.

Scope: only the default KubeOVN datapath is wired through the platform
toggle, which is what the system bundle actually selects. Wiring the
Cilium-native path would require the bundle to expose cilium-only
variant selection; that is out of scope here and called out in the
values comments.

Validation: `helm unittest` on the platform chart passes (66 tests, 7
new) covering on / off / absent / null and verifying the IPsec flag
deep-merges without dropping the existing `kube-ovn` overrides, for both
`isp-full` and `isp-full-generic`.

Closes #2977

Part of #2811

### Screenshots

N/A — no UI changes.

### Release note

```release-note
feat(platform): add opt-in `networking.encryption.enabled` (off by default) that enables in-cluster east-west transport encryption for the default KubeOVN datapath via overlay IPsec; complements, and does not replace, application-level TLS.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Introduced opt-in in-cluster network encryption using KubeOVN IPsec
for east-west traffic protection.
* Added `networking.encryption` configuration toggle (disabled by
default) with full backward compatibility.

* **Tests**
* Added comprehensive test coverage for network encryption
configurations across system variants and edge cases.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Aleksei Sviridkin authored and GitHub committed 2026-07-27 22:47:38 +03:00
commit 8bffa34b54
3 files changed
+190

No files matched your search

@@ -13,6 +13,15 @@
{{- if .Values.networking.kubeovn.MASTER_NODES -}}
{{- $_ := set $kubeovnDict "MASTER_NODES" .Values.networking.kubeovn.MASTER_NODES -}}
{{- end -}}
{{- /* Opt-in east-west encryption (networking.encryption.enabled): on the
default kubeovn-cilium datapath the overlay is KubeOVN's Geneve
tunnel, so encrypting it means turning on KubeOVN IPsec. KubeOVN's
own controller provisions the IPsec CA and per-node certs through
the Kubernetes CSR API — no cert-manager wiring needed here. Off by
default; complements (does not replace) application-level TLS. */ -}}
{{- if dig "encryption" "enabled" false .Values.networking -}}
{{- $_ := set $kubeovnDict "func" (dict "ENABLE_OVN_IPSEC" true) -}}
{{- end -}}
{{- $kubeovnValues := dict "kube-ovn" $kubeovnDict -}}
{{- $_ := set $networkingComponents "kubeovn" (dict "values" $kubeovnValues) -}}
{{- /* For Talos (isp-full): use KubePrism endpoint and disable cgroup autoMount */ -}}
@@ -83,6 +92,12 @@
{{- end -}}
{{- /* For generic k8s (k3s), control-plane label has value "true" */ -}}
{{- $_ := set $kubeovnDict "MASTER_NODES_LABEL" "node-role.kubernetes.io/control-plane=true" -}}
{{- /* Opt-in east-west encryption (networking.encryption.enabled): same
KubeOVN Geneve-overlay IPsec path as isp-full. See the isp-full
branch above for the mechanism notes. */ -}}
{{- if dig "encryption" "enabled" false .Values.networking -}}
{{- $_ := set $kubeovnDict "func" (dict "ENABLE_OVN_IPSEC" true) -}}
{{- end -}}
{{- $kubeovnValues := dict "kube-ovn" $kubeovnDict -}}
{{- $_ := set $networkingComponents "kubeovn" (dict "values" $kubeovnValues) -}}
{{- /* Cilium configuration - for generic k8s, always enable cgroup autoMount */ -}}
@@ -0,0 +1,137 @@
suite: networking.encryption bundle wiring
templates:
- templates/bundles/system.yaml
release:
name: cozystack
namespace: cozy-system
tests:
- it: leaves KubeOVN overlay IPsec unset on the default isp-full datapath when encryption is off
set:
bundles:
system:
enabled: true
variant: isp-full
networking:
encryption:
enabled: false
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.variant
value: kubeovn-cilium
- notExists:
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
- it: leaves KubeOVN overlay IPsec unset on isp-full when networking.encryption is absent (backward compatible, no nil-pointer)
set:
bundles:
system:
enabled: true
variant: isp-full
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.variant
value: kubeovn-cilium
- notExists:
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
- it: enables KubeOVN Geneve-overlay IPsec on the default isp-full datapath when encryption is on
set:
bundles:
system:
enabled: true
variant: isp-full
networking:
encryption:
enabled: true
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.variant
value: kubeovn-cilium
- equal:
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
value: true
- it: enables KubeOVN overlay IPsec on the generic kubeovn-cilium-generic datapath when encryption is on
set:
bundles:
system:
enabled: true
variant: isp-full-generic
networking:
encryption:
enabled: true
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.variant
value: kubeovn-cilium-generic
- equal:
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
value: true
- it: leaves the generic datapath IPsec unset when encryption is off
set:
bundles:
system:
enabled: true
variant: isp-full-generic
networking:
encryption:
enabled: false
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.variant
value: kubeovn-cilium-generic
- notExists:
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
- it: keeps the existing kube-ovn ipv4 component overrides alongside the IPsec toggle (deep merge, not replace)
set:
bundles:
system:
enabled: true
variant: isp-full
networking:
encryption:
enabled: true
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.components.kubeovn.values.kube-ovn.ipv4.POD_CIDR
value: 10.244.0.0/16
- equal:
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
value: true
- it: renders without a nil-pointer error when networking is set to null and encryption is therefore unreachable
set:
bundles:
system:
enabled: true
variant: isp-full
networking: null
documentSelector:
path: metadata.name
value: cozystack.networking
asserts:
- equal:
path: spec.variant
value: kubeovn-cilium
- notExists:
path: spec.components
+38
View File
@@ -71,6 +71,44 @@ networking:
# Set this to comma-separated list of master node IPs to override.
kubeovn:
MASTER_NODES: ""
# In-cluster (east-west) transport encryption. Opt-in, OFF by default.
#
# When enabled, pod-to-pod traffic that crosses nodes is encrypted at the
# network layer by the ACTIVE datapath, regardless of whether the workload
# speaks TLS — defense-in-depth for the whole mesh without per-application
# changes. This COMPLEMENTS, and does NOT replace, application-level /
# end-to-end TLS: it never reaches an external client and gives a tenant no
# verifiable endpoint certificate (see the per-app certificate series). Use
# it alongside, not instead of, TLS on exposed endpoints.
#
# The mechanism is datapath-specific (selected by bundles.system.variant):
#
# * kubeovn-cilium (isp-full, the default) and kubeovn-cilium-generic
# (isp-full-generic): KubeOVN owns the datapath; its Geneve overlay is
# encrypted with IPsec (kube-ovn ENABLE_OVN_IPSEC). KubeOVN's own
# controller provisions the IPsec CA and per-node certificates through
# the Kubernetes CSR API — no cert-manager wiring is required. This is
# the path this toggle drives today.
# * cilium / cilium-generic (Cilium owns the datapath): Cilium-native
# WireGuard/IPsec is the right mechanism there, but the platform system
# bundle does not select those variants, so this toggle does not reach
# them yet.
# * cilium-kilo: Kilo already establishes a WireGuard mesh, but only at
# meshGranularity=location — it encrypts CROSS-LOCATION traffic and
# leaves intra-location pod-to-pod in cleartext by design. This toggle
# targets exactly that intra-location gap and composes with Kilo rather
# than conflicting with it.
#
# CAVEATS: encrypting the overlay adds per-packet ESP overhead and lowers
# the effective MTU, so east-west throughput drops and latency rises; the
# cost is highest for high-bandwidth flows. IPsec here protects the
# inter-node overlay tunnels — same-node pod-to-pod traffic never leaves the
# host and is not tunneled or encrypted. Flipping this on an existing
# cluster triggers a rollout of the CNI components. On variants without a
# cozystack-managed encryptable datapath (e.g. isp-hosted / noop) the toggle
# has no effect.
encryption:
enabled: false
# Service publishing and ingress configuration
publishing:
host: "example.org"