From 39918b7a729ccd26210db5865a66663ea1525dd2 Mon Sep 17 00:00:00 2001 From: Timofei Larkin Date: Fri, 18 Sep 2026 16:07:19 +0300 Subject: [PATCH] feat(platform): package the address-controller and its MetalLB driver Two optional system packages bring in IP addresses as a first-class resource (cozystack/community#35): cozystack.address-controller, the class-agnostic core (IPAddressClass / IPAddress / IPAddressClaim, local.sdn.cozystack.io), and cozystack.metallb-iad, the driver implementing the contract for addresses announced by MetalLB. Both follow the umbrella shape: the upstream chart is vendored under charts/ from cozystack/address-controller and cozystack/metallb-iad, and what belongs to Cozystack rather than to a generic Kubernetes controller lives in the wrapper. Neither project publishes a chart artifact, so each `update` target clones at a pinned commit, and the wrapper pins the image built from the same controller code by digest; the charts' own floating :main defaults would otherwise leave the controllers free to move underneath a cluster. The wrapper carries the tenant grant. The upstream chart is Cozystack- agnostic and grants only the controller, so nothing there lets a tenant use the API it serves; cozy:addresses, cozy:addresses:view and cozy:addresses:admin aggregate into the platform's base tenant roles, which resolve by label, so the grant is present exactly while the package is. Only IPAddressClaim is granted: the other two kinds are cluster-scoped and cozy:tenant* reaches a tenant through a per-namespace RoleBinding, so a rule for either could never be consulted. The wrapper also turns on the driver's admission policy for tenant namespaces. MetalLB honours a pin or pool selector from any writer of a Service, through its current annotations, their metallb.universe.tf predecessors and spec.loadBalancerIP alike, and a tenant cluster's cloud provider copies a guest Service's annotations and loadBalancerIP onto the Service it creates in the tenant namespace. Without the policy, a tenant could steer MetalLB at an address reserved for someone else. Tenant namespaces are the ones carrying namespace.cozystack.io/host, which tenant-root and the tenant chart set and nothing in cozy-* does. An admission policy binds every principal regardless of RBAC, so the allowlist names Cozystack's OIDC administrators, cozystack-cluster-admin, beside the chart's own two admin groups. The driver points at cozy-metallb instead of the upstream default metallb-system. Its PackageSource declares dependsOn on the core, whose CRDs it needs, and on cozystack.metallb. The core installs with upgradeCRDs: CreateReplace since its CRDs are v1alpha1 and still evolving. Neither package is in the postgres E2E install closure, so both join the selective-install fixture's disable list. Assisted-by: LLM Signed-off-by: Timofei Larkin --- .../platform/sources/address-controller.yaml | 28 ++ .../core/platform/sources/metallb-iad.yaml | 27 ++ .../platform/templates/bundles/system.yaml | 6 + .../selective-install-postgres-values.yaml | 2 + packages/system/address-controller/Chart.yaml | 3 + packages/system/address-controller/Makefile | 23 ++ .../charts/address-controller/.helmignore | 9 + .../charts/address-controller/Chart.yaml | 15 ++ ...ocal.sdn.cozystack.io_ipaddressclaims.yaml | 182 +++++++++++++ ...cal.sdn.cozystack.io_ipaddressclasses.yaml | 97 +++++++ .../local.sdn.cozystack.io_ipaddresses.yaml | 249 ++++++++++++++++++ .../address-controller/templates/_helpers.tpl | 15 ++ .../templates/admission-policy.yaml | 55 ++++ .../templates/controller.yaml | 79 ++++++ .../address-controller/templates/metrics.yaml | 121 +++++++++ .../address-controller/templates/role.yaml | 89 +++++++ .../charts/address-controller/values.yaml | 61 +++++ .../templates/tenant-clusterroles.yaml | 44 ++++ .../tests/tenant-clusterroles_test.yaml | 57 ++++ .../system/address-controller/values.yaml | 6 + packages/system/metallb-iad/Chart.yaml | 3 + packages/system/metallb-iad/Makefile | 16 ++ .../charts/metallb-iad/.helmignore | 9 + .../metallb-iad/charts/metallb-iad/Chart.yaml | 15 ++ .../charts/metallb-iad/templates/_helpers.tpl | 20 ++ .../templates/admission-policy.yaml | 85 ++++++ .../metallb-iad/templates/controller.yaml | 86 ++++++ .../charts/metallb-iad/templates/role.yaml | 74 ++++++ .../charts/metallb-iad/values.yaml | 59 +++++ packages/system/metallb-iad/values.yaml | 23 ++ 30 files changed, 1558 insertions(+) create mode 100644 packages/core/platform/sources/address-controller.yaml create mode 100644 packages/core/platform/sources/metallb-iad.yaml create mode 100644 packages/system/address-controller/Chart.yaml create mode 100644 packages/system/address-controller/Makefile create mode 100644 packages/system/address-controller/charts/address-controller/.helmignore create mode 100644 packages/system/address-controller/charts/address-controller/Chart.yaml create mode 100644 packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclaims.yaml create mode 100644 packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclasses.yaml create mode 100644 packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddresses.yaml create mode 100644 packages/system/address-controller/charts/address-controller/templates/_helpers.tpl create mode 100644 packages/system/address-controller/charts/address-controller/templates/admission-policy.yaml create mode 100644 packages/system/address-controller/charts/address-controller/templates/controller.yaml create mode 100644 packages/system/address-controller/charts/address-controller/templates/metrics.yaml create mode 100644 packages/system/address-controller/charts/address-controller/templates/role.yaml create mode 100644 packages/system/address-controller/charts/address-controller/values.yaml create mode 100644 packages/system/address-controller/templates/tenant-clusterroles.yaml create mode 100644 packages/system/address-controller/tests/tenant-clusterroles_test.yaml create mode 100644 packages/system/address-controller/values.yaml create mode 100644 packages/system/metallb-iad/Chart.yaml create mode 100644 packages/system/metallb-iad/Makefile create mode 100644 packages/system/metallb-iad/charts/metallb-iad/.helmignore create mode 100644 packages/system/metallb-iad/charts/metallb-iad/Chart.yaml create mode 100644 packages/system/metallb-iad/charts/metallb-iad/templates/_helpers.tpl create mode 100644 packages/system/metallb-iad/charts/metallb-iad/templates/admission-policy.yaml create mode 100644 packages/system/metallb-iad/charts/metallb-iad/templates/controller.yaml create mode 100644 packages/system/metallb-iad/charts/metallb-iad/templates/role.yaml create mode 100644 packages/system/metallb-iad/charts/metallb-iad/values.yaml create mode 100644 packages/system/metallb-iad/values.yaml diff --git a/packages/core/platform/sources/address-controller.yaml b/packages/core/platform/sources/address-controller.yaml new file mode 100644 index 000000000..cafcec9b5 --- /dev/null +++ b/packages/core/platform/sources/address-controller.yaml @@ -0,0 +1,28 @@ +--- +apiVersion: cozystack.io/v1alpha1 +kind: PackageSource +metadata: + name: cozystack.address-controller +spec: + sourceRef: + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + path: / + variants: + # IP addresses as a first-class resource (IPAddressClass / IPAddress / + # IPAddressClaim, local.sdn.cozystack.io) — the PVC/PV-binding analog. The + # core owns claim–address binding and reclaim; allocation belongs to + # per-class drivers such as cozystack.metallb-iad. + - name: default + dependsOn: + - cozystack.networking + components: + - name: address-controller + path: system/address-controller + install: + namespace: cozy-address-controller + releaseName: address-controller + # The vendored chart ships its CRDs in crds/; they are v1alpha1 and + # still evolving, so upgrades must be allowed to replace them. + upgradeCRDs: CreateReplace diff --git a/packages/core/platform/sources/metallb-iad.yaml b/packages/core/platform/sources/metallb-iad.yaml new file mode 100644 index 000000000..388af652d --- /dev/null +++ b/packages/core/platform/sources/metallb-iad.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: cozystack.io/v1alpha1 +kind: PackageSource +metadata: + name: cozystack.metallb-iad +spec: + sourceRef: + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + path: / + variants: + # The MetalLB IP Allocation Driver — the per-class driver implementing the + # cozystack.address-controller contract for addresses announced by MetalLB. + # It holds reservations as placeholder Services in its own namespace + # (cozy-metallb-iad, which tenants must have no write access to) and needs + # both the core CRDs/controller and MetalLB itself in place. + - name: default + dependsOn: + - cozystack.address-controller + - cozystack.metallb + components: + - name: metallb-iad + path: system/metallb-iad + install: + namespace: cozy-metallb-iad + releaseName: metallb-iad diff --git a/packages/core/platform/templates/bundles/system.yaml b/packages/core/platform/templates/bundles/system.yaml index 906c8d95f..6dea77ef8 100644 --- a/packages/core/platform/templates/bundles/system.yaml +++ b/packages/core/platform/templates/bundles/system.yaml @@ -387,5 +387,11 @@ variants and opt-in on the slim ones. */}} {{- end }} {{include "cozystack.platform.package.optional.default" (list "cozystack.hetzner-robotlb" $) }} {{include "cozystack.platform.package.optional.default" (list "cozystack.talos-log-collector" $) }} +{{- /* IP addresses as a first-class resource: the class-agnostic core plus the + MetalLB driver. Enable both together — the driver's PackageSource + dependsOn the core (and cozystack.metallb, which the isp-hosted variant + never emits), so enabling only the driver leaves it waiting forever. */}} +{{include "cozystack.platform.package.optional.default" (list "cozystack.address-controller" $) }} +{{include "cozystack.platform.package.optional.default" (list "cozystack.metallb-iad" $) }} {{- end }} diff --git a/packages/core/platform/tests/fixtures/selective-install-postgres-values.yaml b/packages/core/platform/tests/fixtures/selective-install-postgres-values.yaml index 6758b1b42..8f743da23 100644 --- a/packages/core/platform/tests/fixtures/selective-install-postgres-values.yaml +++ b/packages/core/platform/tests/fixtures/selective-install-postgres-values.yaml @@ -2,6 +2,7 @@ # Regenerate after changing the PackageSource inventory or install baseline. bundles: disabledPackages: + - cozystack.address-controller - cozystack.bootbox - cozystack.bootbox-application - cozystack.capi-provider-infra-proxmox @@ -37,6 +38,7 @@ bundles: - cozystack.local-storage - cozystack.mariadb-application - cozystack.mariadb-operator + - cozystack.metallb-iad - cozystack.mongodb-application - cozystack.mongodb-operator - cozystack.monitoring diff --git a/packages/system/address-controller/Chart.yaml b/packages/system/address-controller/Chart.yaml new file mode 100644 index 000000000..10ecb85b2 --- /dev/null +++ b/packages/system/address-controller/Chart.yaml @@ -0,0 +1,3 @@ +apiVersion: v2 +name: cozy-address-controller +version: 0.0.0 # Placeholder, the actual version will be automatically set during the build process diff --git a/packages/system/address-controller/Makefile b/packages/system/address-controller/Makefile new file mode 100644 index 000000000..7b0907acc --- /dev/null +++ b/packages/system/address-controller/Makefile @@ -0,0 +1,23 @@ +export NAME=address-controller +export NAMESPACE=cozy-$(NAME) + +include ../../../hack/package.mk + +# Upstream publishes no chart artifact, so the chart is taken from the +# repository at a pinned commit. Bump together with the image in values.yaml, +# which must carry the same controller code: upstream's release job tags each +# main build as main-, and the digest is in that job's log or from +# `crane digest`. +ADDRESS_CONTROLLER_REPO ?= https://github.com/cozystack/address-controller.git +ADDRESS_CONTROLLER_REF ?= 0ed4cc53de935ba108c012e43008fc456b806f79 + +test: + helm unittest . + +update: + rm -rf charts tmp-repo + git clone --filter=tree:0 --no-checkout $(ADDRESS_CONTROLLER_REPO) tmp-repo + git -C tmp-repo checkout --quiet $(ADDRESS_CONTROLLER_REF) + mkdir -p charts + cp -r tmp-repo/chart/address-controller charts/ + rm -rf tmp-repo diff --git a/packages/system/address-controller/charts/address-controller/.helmignore b/packages/system/address-controller/charts/address-controller/.helmignore new file mode 100644 index 000000000..b4d0f7f17 --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/.helmignore @@ -0,0 +1,9 @@ +# Patterns to ignore when building packages. +.DS_Store +.git/ +.gitignore +*.swp +*.bak +*.tmp +*.orig +*~ diff --git a/packages/system/address-controller/charts/address-controller/Chart.yaml b/packages/system/address-controller/charts/address-controller/Chart.yaml new file mode 100644 index 000000000..820993b98 --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/Chart.yaml @@ -0,0 +1,15 @@ +apiVersion: v2 +name: address-controller +description: Core controllers for IP addresses as a first-class resource — IPAddressClass, IPAddress, IPAddressClaim (local.sdn.cozystack.io) +type: application +version: 0.2.0 +appVersion: "0.2.0" +# CEL validation rules on the CRDs (immutability, union exactness) need 1.29+. +kubeVersion: ">=1.29.0-0" +keywords: + - ipam + - networking + - cozystack +home: https://github.com/cozystack/address-controller +sources: + - https://github.com/cozystack/address-controller diff --git a/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclaims.yaml b/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclaims.yaml new file mode 100644 index 000000000..2723b8aba --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclaims.yaml @@ -0,0 +1,182 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.1 + name: ipaddressclaims.local.sdn.cozystack.io +spec: + group: local.sdn.cozystack.io + names: + kind: IPAddressClaim + listKind: IPAddressClaimList + plural: ipaddressclaims + singular: ipaddressclaim + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.phase + name: Phase + type: string + - jsonPath: .status.className + name: Class + type: string + - jsonPath: .status.addresses[*].address + name: Addresses + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + IPAddressClaim is a namespaced request for an address — the whole + tenant-facing API, the PersistentVolumeClaim of the address model. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: IPAddressClaimSpec defines the desired state of IPAddressClaim. + properties: + addressName: + description: |- + addressName pins the claim to a specific Available IPAddress instead + of letting the controller match or the driver provision one — the + analogue of PersistentVolumeClaim.spec.volumeName. Only meaningful + for single-family claims. + type: string + className: + description: |- + className names the IPAddressClass to draw from. Empty means the + default class (the one annotated + ipaddressclass.local.sdn.cozystack.io/is-default-class: "true"). + type: string + family: + default: IPv4 + description: family selects the IP families requested. + enum: + - IPv4 + - IPv6 + - Dual + type: string + type: object + status: + description: IPAddressClaimStatus defines the observed state of IPAddressClaim. + properties: + addresses: + description: |- + addresses lists the bound addresses. A list deliberately: a Dual + claim binds a v4 and a v6 IPAddress and must report both. For a + single-family claim the list has one entry. + items: + description: BoundAddress reports one IPAddress bound to the claim. + properties: + address: + description: address is the IP itself — what the tenant reads + and puts in DNS. + type: string + name: + description: name of the bound IPAddress object. + type: string + required: + - address + - name + type: object + type: array + className: + description: |- + className is the class the claim resolved to. For claims created + without a className it records which default class was picked; the + resolution is sticky. + type: string + conditions: + description: |- + conditions represent the latest available observations of the + claim's state. + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + phase: + description: phase is the current lifecycle phase of the claim. + enum: + - Pending + - Bound + - Lost + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclasses.yaml b/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclasses.yaml new file mode 100644 index 000000000..792d220ae --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddressclasses.yaml @@ -0,0 +1,97 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.1 + name: ipaddressclasses.local.sdn.cozystack.io +spec: + group: local.sdn.cozystack.io + names: + kind: IPAddressClass + listKind: IPAddressClassList + plural: ipaddressclasses + singular: ipaddressclass + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .spec.provisioner + name: Provisioner + type: string + - jsonPath: .spec.reclaimPolicy + name: ReclaimPolicy + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + IPAddressClass describes an address source: which pool, which driver. + It is the StorageClass of the address model. Marking a class with the + annotation ipaddressclass.local.sdn.cozystack.io/is-default-class: "true" + makes it the class for claims that do not name one. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: IPAddressClassSpec defines the desired state of IPAddressClass. + properties: + parameters: + description: |- + parameters is an opaque, driver-specific configuration blob (for + example the address ranges to carve from). The core controller never + interprets it. + type: object + x-kubernetes-preserve-unknown-fields: true + provisioner: + description: |- + provisioner names the per-class driver that fulfils claims of this + class, exactly as StorageClass.provisioner names a CSI driver. The + core controller never provisions addresses itself; it stamps this + value onto pending claims so the named driver can pick them up. + minLength: 1 + type: string + x-kubernetes-validations: + - message: provisioner is immutable + rule: self == oldSelf + reclaimPolicy: + default: Retain + description: |- + reclaimPolicy is copied to IPAddress objects provisioned for this + class and governs what happens to them when their claim is deleted. + enum: + - Retain + - Delete + type: string + releaseDelay: + description: |- + releaseDelay, under reclaimPolicy Retain, returns a Released address + of this class to Available once it has been Released this long: the + core controller clears its claimRef. Unset keeps a Released address + until an admin clears it. An address annotated + ipaddress.local.sdn.cozystack.io/held: "true" is exempt. + type: string + required: + - provisioner + type: object + type: object + served: true + storage: true + subresources: {} diff --git a/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddresses.yaml b/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddresses.yaml new file mode 100644 index 000000000..c75e8440f --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/crds/local.sdn.cozystack.io_ipaddresses.yaml @@ -0,0 +1,249 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.1 + name: ipaddresses.local.sdn.cozystack.io +spec: + group: local.sdn.cozystack.io + names: + kind: IPAddress + listKind: IPAddressList + plural: ipaddresses + singular: ipaddress + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .spec.address + name: Address + type: string + - jsonPath: .spec.className + name: Class + type: string + - jsonPath: .status.phase + name: Phase + type: string + - jsonPath: .spec.claimRef.namespace + name: ClaimNamespace + type: string + - jsonPath: .spec.claimRef.name + name: Claim + type: string + - jsonPath: .status.associatedTo.name + name: AttachedTo + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + IPAddress is a concrete address the cluster owns — the inventory object, + the PersistentVolume of the address model. Created by a per-class driver + (or an admin, for static pre-provisioning), never by tenants. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: IPAddressSpec defines the desired state of IPAddress. + properties: + address: + description: |- + address is the IP address itself, in canonical textual form + (e.g. "203.0.113.7" or "2001:db8::7"). + minLength: 1 + type: string + x-kubernetes-validations: + - message: address is immutable + rule: self == oldSelf + claimRef: + description: |- + claimRef binds this address to a claim. Set either by the driver at + provisioning time (pre-bound) or by the core controller when it + matches an Available address to a claim. An address with no claimRef + is Available. After a Retain reclaim the field survives the claim + (phase Released) until an admin clears it. + properties: + name: + description: name of the claim. + minLength: 1 + type: string + namespace: + description: namespace of the claim. + minLength: 1 + type: string + uid: + description: |- + uid of the claim. A driver pre-binding an address may leave it empty; + the core controller completes it when it accepts the binding. + type: string + required: + - name + - namespace + type: object + className: + description: className names the IPAddressClass this address belongs + to. + minLength: 1 + type: string + reclaimPolicy: + default: Retain + description: |- + reclaimPolicy governs what happens to this object when its claim is + deleted; copied from the class at provisioning time. + enum: + - Retain + - Delete + type: string + source: + description: source records where the address came from. + properties: + fromClass: + description: fromClass marks the address as allocated from the + class's range. + type: object + providerRef: + description: |- + providerRef marks the address as adopted from a provider-side + reservation. + properties: + id: + description: |- + id is the provider-side stable handle of the reservation, for example + an AWS allocation id ("eipalloc-0a1b..."). + minLength: 1 + type: string + required: + - id + type: object + type: object + x-kubernetes-validations: + - message: exactly one of fromClass or providerRef must be set + rule: '(has(self.fromClass) ? 1 : 0) + (has(self.providerRef) ? + 1 : 0) == 1' + required: + - address + - className + - source + type: object + status: + description: IPAddressStatus defines the observed state of IPAddress. + properties: + associatedTo: + description: |- + associatedTo names the workload the address is currently announced + for. Nil means reserved but inert. Maintained by the per-class + driver, never by the core controller. + properties: + kind: + description: kind of the associated object, e.g. "Service". + minLength: 1 + type: string + name: + description: name of the associated object. + minLength: 1 + type: string + namespace: + description: namespace of the associated object. + minLength: 1 + type: string + required: + - kind + - name + - namespace + type: object + conditions: + description: |- + conditions represent the latest available observations of the + address's state. + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + phase: + description: phase is the current lifecycle phase of the address. + enum: + - Pending + - Available + - Bound + - Released + - Conflict + - Lost + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/packages/system/address-controller/charts/address-controller/templates/_helpers.tpl b/packages/system/address-controller/charts/address-controller/templates/_helpers.tpl new file mode 100644 index 000000000..6ae6b0e1b --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/templates/_helpers.tpl @@ -0,0 +1,15 @@ +{{/* Common metadata labels (not used as selectors — selectors stay stable). */}} +{{- define "address-controller.labels" -}} +app.kubernetes.io/name: address-controller +app.kubernetes.io/part-of: address-controller +app.kubernetes.io/managed-by: {{ .Release.Service }} +helm.sh/chart: {{ printf "address-controller-%s" .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end -}} + +{{/* imagePullSecrets block, rendered only when set. Call with the root context. */}} +{{- define "address-controller.imagePullSecrets" -}} +{{- with .Values.imagePullSecrets }} +imagePullSecrets: + {{- toYaml . | nindent 2 }} +{{- end }} +{{- end -}} diff --git a/packages/system/address-controller/charts/address-controller/templates/admission-policy.yaml b/packages/system/address-controller/charts/address-controller/templates/admission-policy.yaml new file mode 100644 index 000000000..44b1744b9 --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/templates/admission-policy.yaml @@ -0,0 +1,55 @@ +{{- /* Optional admission policy (proposal, Security: "spec.addressName cannot + be checked in CEL"): only allowlisted principals may set, change or + remove IPAddressClaim.spec.addressName, which binds one specific + Available address and so lets its writer pick an address out of a + class. Writes that leave the field as it was are always admitted. */ -}} +{{- if .Values.admissionPolicy.enabled }} +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicy +metadata: + name: address-controller-claim-address-name + labels: + {{- include "address-controller.labels" . | nindent 4 }} +spec: + failurePolicy: Fail + matchConstraints: + resourceRules: + - apiGroups: ["local.sdn.cozystack.io"] + apiVersions: ["v1alpha1"] + operations: ["CREATE", "UPDATE"] + resources: ["ipaddressclaims"] + matchConditions: + - name: not-allowlisted + expression: >- + !(request.userInfo.username in {{ toJson .Values.admissionPolicy.allowedUsernames }}) && + !(has(request.userInfo.groups) && + request.userInfo.groups.exists(g, g in {{ toJson .Values.admissionPolicy.allowedGroups }})) + variables: + - name: oldAddressName + expression: >- + request.operation == "UPDATE" && has(oldObject.spec.addressName) + ? oldObject.spec.addressName : "" + - name: newAddressName + expression: >- + has(object.spec.addressName) ? object.spec.addressName : "" + validations: + - expression: variables.newAddressName == variables.oldAddressName + message: >- + spec.addressName is reserved to the platform; create the claim without it + and the class assigns an address + reason: Forbidden +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicyBinding +metadata: + name: address-controller-claim-address-name + labels: + {{- include "address-controller.labels" . | nindent 4 }} +spec: + policyName: address-controller-claim-address-name + validationActions: {{ toJson .Values.admissionPolicy.validationActions }} + matchResources: + namespaceSelector: + {{- toYaml .Values.admissionPolicy.namespaceSelector | nindent 6 }} +{{- end }} diff --git a/packages/system/address-controller/charts/address-controller/templates/controller.yaml b/packages/system/address-controller/charts/address-controller/templates/controller.yaml new file mode 100644 index 000000000..3322c59f3 --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/templates/controller.yaml @@ -0,0 +1,79 @@ +{{- /* The ClusterRole lives in templates/role.yaml, GENERATED by controller-gen + from the +kubebuilder:rbac markers (make manifests). Everything + deployment-shaped is hand-written here. */ -}} +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: address-controller + namespace: {{ .Release.Namespace }} + labels: + {{- include "address-controller.labels" . | nindent 4 }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: address-controller + labels: + {{- include "address-controller.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: address-controller +subjects: + - kind: ServiceAccount + name: address-controller + namespace: {{ .Release.Namespace }} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: address-controller + namespace: {{ .Release.Namespace }} + labels: + app: address-controller + {{- include "address-controller.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.controller.replicas }} + selector: + matchLabels: + app: address-controller + template: + metadata: + labels: + app: address-controller + spec: + serviceAccountName: address-controller + {{- with .Values.controller.priorityClassName }} + priorityClassName: {{ . }} + {{- end }} + {{- include "address-controller.imagePullSecrets" . | nindent 6 }} + securityContext: + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault + containers: + - name: controller + image: {{ .Values.image }} + imagePullPolicy: {{ .Values.imagePullPolicy }} + args: + - --leader-elect={{ .Values.controller.leaderElect }} + - --metrics-bind-address={{ if .Values.metrics.enabled }}:8443{{ else }}{{ .Values.controller.metricsBindAddress }}{{ end }} + - --health-probe-bind-address=:8081 + {{- if .Values.metrics.enabled }} + ports: + - name: metrics + containerPort: 8443 + {{- end }} + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + drop: ["ALL"] + resources: + {{- toYaml .Values.controller.resources | nindent 12 }} + livenessProbe: + httpGet: { path: /healthz, port: 8081 } + initialDelaySeconds: 10 + readinessProbe: + httpGet: { path: /readyz, port: 8081 } diff --git a/packages/system/address-controller/charts/address-controller/templates/metrics.yaml b/packages/system/address-controller/charts/address-controller/templates/metrics.yaml new file mode 100644 index 000000000..225483c0b --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/templates/metrics.yaml @@ -0,0 +1,121 @@ +{{- if .Values.metrics.enabled }} +--- +apiVersion: v1 +kind: Service +metadata: + name: address-controller-metrics + namespace: {{ .Release.Namespace }} + labels: + app: address-controller-metrics + {{- include "address-controller.labels" . | nindent 4 }} +spec: + selector: + app: address-controller + ports: + - name: metrics + port: 8443 + targetPort: metrics +--- +# Lets the controller authenticate and authorize scrape requests. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: address-controller-metrics-auth + labels: + {{- include "address-controller.labels" . | nindent 4 }} +rules: + - apiGroups: ["authentication.k8s.io"] + resources: ["tokenreviews"] + verbs: ["create"] + - apiGroups: ["authorization.k8s.io"] + resources: ["subjectaccessreviews"] + verbs: ["create"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: address-controller-metrics-auth + labels: + {{- include "address-controller.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: address-controller-metrics-auth +subjects: + - kind: ServiceAccount + name: address-controller + namespace: {{ .Release.Namespace }} +--- +# Bind a scraper's identity to this role to let it read the metrics. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: address-controller-metrics-reader + labels: + {{- include "address-controller.labels" . | nindent 4 }} +rules: + - nonResourceURLs: ["/metrics"] + verbs: ["get"] +{{- if and .Values.metrics.serviceMonitor.enabled (.Capabilities.APIVersions.Has "monitoring.coreos.com/v1/ServiceMonitor") }} +--- +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: address-controller + namespace: {{ .Release.Namespace }} + labels: + {{- include "address-controller.labels" . | nindent 4 }} + {{- with .Values.metrics.serviceMonitor.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + selector: + matchLabels: + app: address-controller-metrics + endpoints: + - port: metrics + scheme: https + bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token + tlsConfig: + # The controller serves a self-signed certificate. + insecureSkipVerify: true +{{- end }} +{{- if and .Values.metrics.prometheusRule.enabled (.Capabilities.APIVersions.Has "monitoring.coreos.com/v1/PrometheusRule") }} +--- +apiVersion: monitoring.coreos.com/v1 +kind: PrometheusRule +metadata: + name: address-controller + namespace: {{ .Release.Namespace }} + labels: + {{- include "address-controller.labels" . | nindent 4 }} +spec: + groups: + - name: address-controller + rules: + - alert: IPAddressConflict + expr: sum by (class) (address_controller_ipaddresses{phase="Conflict"}) > 0 + for: 5m + labels: + severity: warning + annotations: + summary: "{{ "{{ $value }}" }} IPAddress(es) of class {{ "{{ $labels.class }}" }} in Conflict" + description: A Service holds an address that its binding does not authorize. kubectl get ipaddresses names the offender. + - alert: IPAddressLost + expr: sum by (class) (address_controller_ipaddresses{phase="Lost"}) > 0 + for: 5m + labels: + severity: warning + annotations: + summary: "{{ "{{ $value }}" }} IPAddress(es) of class {{ "{{ $labels.class }}" }} Lost" + description: The backend no longer holds a reserved address. Its claim reports Lost until the address is restored or deleted. + - alert: IPAddressClaimPending + expr: sum by (namespace, class) (address_controller_ipaddressclaims{phase="Pending"}) > 0 + for: 15m + labels: + severity: warning + annotations: + summary: "{{ "{{ $value }}" }} IPAddressClaim(s) in {{ "{{ $labels.namespace }}" }} Pending for 15m" + description: The claim's class may be exhausted, missing or refused by its driver. kubectl describe on the claim shows why. +{{- end }} +{{- end }} diff --git a/packages/system/address-controller/charts/address-controller/templates/role.yaml b/packages/system/address-controller/charts/address-controller/templates/role.yaml new file mode 100644 index 000000000..324169087 --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/templates/role.yaml @@ -0,0 +1,89 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: address-controller +rules: +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - "" + resources: + - services + verbs: + - get + - list + - patch + - watch +- apiGroups: + - "" + resources: + - services/finalizers + verbs: + - update +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddressclaims + verbs: + - create + - get + - list + - patch + - update + - watch +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddressclaims/finalizers + - ipaddressclasses/finalizers + - ipaddresses/finalizers + verbs: + - update +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddressclaims/status + - ipaddresses/status + verbs: + - get + - patch + - update +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddressclasses + verbs: + - get + - list + - patch + - update + - watch +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddresses + verbs: + - delete + - get + - list + - patch + - update + - watch diff --git a/packages/system/address-controller/charts/address-controller/values.yaml b/packages/system/address-controller/charts/address-controller/values.yaml new file mode 100644 index 000000000..58c2e1726 --- /dev/null +++ b/packages/system/address-controller/charts/address-controller/values.yaml @@ -0,0 +1,61 @@ +# Container image for the core controller. +image: ghcr.io/cozystack/address-controller:main +imagePullPolicy: IfNotPresent +imagePullSecrets: [] + +controller: + replicas: 1 + # Leader election matters once replicas > 1, but is on by default so scaling + # up is safe without a config change. + leaderElect: true + # ":8443" (HTTPS) or ":8080" (HTTP) to expose controller-runtime metrics; + # "0" disables the endpoint. + metricsBindAddress: "0" + priorityClassName: "" + resources: + requests: + cpu: 10m + memory: 64Mi + limits: + memory: 128Mi + +metrics: + # Serve controller-runtime and inventory metrics (address_controller_*) + # over HTTPS on 8443, behind the Service address-controller-metrics. + # Requests are authenticated and authorized against the API server, so a + # scraper needs GET on the /metrics non-resource URL: bind its identity + # to the address-controller-metrics-reader ClusterRole. Overrides + # controller.metricsBindAddress. + enabled: false + serviceMonitor: + # A monitoring.coreos.com ServiceMonitor for the Service; rendered only + # where that API exists. The scraping Prometheus's ServiceAccount still + # needs a ClusterRoleBinding to address-controller-metrics-reader, which + # the chart cannot create without knowing that ServiceAccount. + enabled: false + # Extra labels on the ServiceMonitor, e.g. to match a Prometheus + # serviceMonitorSelector. + labels: {} + prometheusRule: + # Alerts on Conflict and Lost addresses and on long-Pending claims; + # rendered only where monitoring.coreos.com exists. + enabled: false + +# Optional ValidatingAdmissionPolicy reserving IPAddressClaim.spec.addressName, +# which binds one specific Available address of a class, to the principals +# below. Writes that leave the field as it was are always admitted. In a +# brokered deployment, allowlist the machinery that moves addresses between +# namespaces. +admissionPolicy: + enabled: false + # [Deny], or e.g. [Warn, Audit] to observe what would be denied first. + validationActions: [Deny] + # Namespaces the policy applies to; scope it to tenant namespaces. + namespaceSelector: {} + # Admission policies bind system:masters too, so cluster admins must be + # listed; kubeadm since 1.29 issues admin credentials in + # kubeadm:cluster-admins instead. + allowedUsernames: [] + allowedGroups: + - system:masters + - kubeadm:cluster-admins diff --git a/packages/system/address-controller/templates/tenant-clusterroles.yaml b/packages/system/address-controller/templates/tenant-clusterroles.yaml new file mode 100644 index 000000000..69d2bc8a3 --- /dev/null +++ b/packages/system/address-controller/templates/tenant-clusterroles.yaml @@ -0,0 +1,44 @@ +{{- /* Only IPAddressClaim is granted. IPAddress and IPAddressClass are + cluster-scoped and cozy:tenant* reaches a tenant through a per-namespace + RoleBinding, so a rule for either would never be consulted; a tenant + reads the address it was given from its own claim's status.addresses[]. */ -}} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cozy:addresses + labels: + rbac.cozystack.io/aggregate-to-tenant: "true" +rules: +- apiGroups: ["local.sdn.cozystack.io"] + resources: ["ipaddressclaims"] + verbs: ['*'] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cozy:addresses:view + labels: + rbac.cozystack.io/aggregate-to-tenant-view: "true" +rules: +- apiGroups: ["local.sdn.cozystack.io"] + resources: ["ipaddressclaims"] + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cozy:addresses:admin + labels: + rbac.cozystack.io/aggregate-to-tenant-admin: "true" +rules: +- apiGroups: ["local.sdn.cozystack.io"] + resources: ["ipaddressclaims"] + verbs: + - create + - update + - patch + - delete diff --git a/packages/system/address-controller/tests/tenant-clusterroles_test.yaml b/packages/system/address-controller/tests/tenant-clusterroles_test.yaml new file mode 100644 index 000000000..4884393d9 --- /dev/null +++ b/packages/system/address-controller/tests/tenant-clusterroles_test.yaml @@ -0,0 +1,57 @@ +suite: address-controller tenant ClusterRoles aggregate into the cozystack base roles + +templates: + - templates/tenant-clusterroles.yaml + +release: + name: address-controller + namespace: cozy-address-controller + +# The aggregation label is the whole wiring: cozy:tenant, cozy:tenant:view and +# cozy:tenant:admin carry an aggregationRule selecting these, so a typo in a +# label silently grants a tenant nothing and nothing else fails. +tests: + - it: the tenant ServiceAccount role manages claims and aggregates to cozy:tenant + documentSelector: + path: metadata.name + value: cozy:addresses + asserts: + - equal: + path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant"] + value: "true" + - contains: + path: rules + content: + apiGroups: ["local.sdn.cozystack.io"] + resources: ["ipaddressclaims"] + verbs: ['*'] + + - it: the view role reads claims and aggregates to cozy:tenant:view + documentSelector: + path: metadata.name + value: cozy:addresses:view + asserts: + - equal: + path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-view"] + value: "true" + - contains: + path: rules + content: + apiGroups: ["local.sdn.cozystack.io"] + resources: ["ipaddressclaims"] + verbs: ["get", "list", "watch"] + + - it: the admin role writes claims and aggregates to cozy:tenant:admin + documentSelector: + path: metadata.name + value: cozy:addresses:admin + asserts: + - equal: + path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-admin"] + value: "true" + - contains: + path: rules + content: + apiGroups: ["local.sdn.cozystack.io"] + resources: ["ipaddressclaims"] + verbs: ["create", "update", "patch", "delete"] diff --git a/packages/system/address-controller/values.yaml b/packages/system/address-controller/values.yaml new file mode 100644 index 000000000..c1027c5ad --- /dev/null +++ b/packages/system/address-controller/values.yaml @@ -0,0 +1,6 @@ +address-controller: + # The chart's own default is the floating `:main` tag, which would let the + # running controller change with no change in this tree. Pinned here rather + # than in charts/, which `make update` regenerates. Bump in lockstep with + # ADDRESS_CONTROLLER_REF in the Makefile. + image: ghcr.io/cozystack/address-controller:main-90e65ca@sha256:74e4c7cf63f730ebe61527fdcd2b06e1bca2c383a66522dd4734aab9e66a9331 diff --git a/packages/system/metallb-iad/Chart.yaml b/packages/system/metallb-iad/Chart.yaml new file mode 100644 index 000000000..f3bb1e9e7 --- /dev/null +++ b/packages/system/metallb-iad/Chart.yaml @@ -0,0 +1,3 @@ +apiVersion: v2 +name: cozy-metallb-iad +version: 0.0.0 # Placeholder, the actual version will be automatically set during the build process diff --git a/packages/system/metallb-iad/Makefile b/packages/system/metallb-iad/Makefile new file mode 100644 index 000000000..3b4aee657 --- /dev/null +++ b/packages/system/metallb-iad/Makefile @@ -0,0 +1,16 @@ +export NAME=metallb-iad +export NAMESPACE=cozy-$(NAME) + +include ../../../hack/package.mk + +# Same pinning story as packages/system/address-controller. +METALLB_IAD_REPO ?= https://github.com/cozystack/metallb-iad.git +METALLB_IAD_REF ?= 2f983b67ed6ab86d745922a6bff9d89f18fc681d + +update: + rm -rf charts tmp-repo + git clone --filter=tree:0 --no-checkout $(METALLB_IAD_REPO) tmp-repo + git -C tmp-repo checkout --quiet $(METALLB_IAD_REF) + mkdir -p charts + cp -r tmp-repo/chart/metallb-iad charts/ + rm -rf tmp-repo diff --git a/packages/system/metallb-iad/charts/metallb-iad/.helmignore b/packages/system/metallb-iad/charts/metallb-iad/.helmignore new file mode 100644 index 000000000..b4d0f7f17 --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/.helmignore @@ -0,0 +1,9 @@ +# Patterns to ignore when building packages. +.DS_Store +.git/ +.gitignore +*.swp +*.bak +*.tmp +*.orig +*~ diff --git a/packages/system/metallb-iad/charts/metallb-iad/Chart.yaml b/packages/system/metallb-iad/charts/metallb-iad/Chart.yaml new file mode 100644 index 000000000..7021025da --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/Chart.yaml @@ -0,0 +1,15 @@ +apiVersion: v2 +name: metallb-iad +description: MetalLB IP Allocation Driver — the reference per-class driver for the address-controller core (local.sdn.cozystack.io) +type: application +version: 0.2.0 +appVersion: "0.2.0" +kubeVersion: ">=1.29.0-0" +keywords: + - ipam + - metallb + - networking + - cozystack +home: https://github.com/cozystack/metallb-iad +sources: + - https://github.com/cozystack/metallb-iad diff --git a/packages/system/metallb-iad/charts/metallb-iad/templates/_helpers.tpl b/packages/system/metallb-iad/charts/metallb-iad/templates/_helpers.tpl new file mode 100644 index 000000000..79eb62a99 --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/templates/_helpers.tpl @@ -0,0 +1,20 @@ +{{/* Common metadata labels (not used as selectors — selectors stay stable). */}} +{{- define "metallb-iad.labels" -}} +app.kubernetes.io/name: metallb-iad +app.kubernetes.io/part-of: metallb-iad +app.kubernetes.io/managed-by: {{ .Release.Service }} +helm.sh/chart: {{ printf "metallb-iad-%s" .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end -}} + +{{/* The namespace reservation placeholders live in (defaults to the release namespace). */}} +{{- define "metallb-iad.placeholderNamespace" -}} +{{- default .Release.Namespace .Values.placeholderNamespace -}} +{{- end -}} + +{{/* imagePullSecrets block, rendered only when set. Call with the root context. */}} +{{- define "metallb-iad.imagePullSecrets" -}} +{{- with .Values.imagePullSecrets }} +imagePullSecrets: + {{- toYaml . | nindent 2 }} +{{- end }} +{{- end -}} diff --git a/packages/system/metallb-iad/charts/metallb-iad/templates/admission-policy.yaml b/packages/system/metallb-iad/charts/metallb-iad/templates/admission-policy.yaml new file mode 100644 index 000000000..280e395fd --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/templates/admission-policy.yaml @@ -0,0 +1,85 @@ +{{- /* Optional admission layer (design doc §8): only the driver and allowlisted + principals may set, change or remove the Service fields that steer + MetalLB at a specific address or pool, or the driver's own markers. + Writes that leave those fields as they were are always admitted. */ -}} +{{- if .Values.admissionPolicy.enabled }} +{{- $users := append .Values.admissionPolicy.allowedUsernames (printf "system:serviceaccount:%s:metallb-iad" .Release.Namespace) }} +{{- $groups := .Values.admissionPolicy.allowedGroups }} +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicy +metadata: + name: metallb-iad-service-pins + labels: + {{- include "metallb-iad.labels" . | nindent 4 }} +spec: + failurePolicy: Fail + matchConstraints: + resourceRules: + - apiGroups: [""] + apiVersions: ["v1"] + operations: ["CREATE", "UPDATE"] + resources: ["services"] + matchConditions: + - name: not-allowlisted + expression: >- + !(request.userInfo.username in {{ toJson $users }}) && + !(has(request.userInfo.groups) && + request.userInfo.groups.exists(g, g in {{ toJson $groups }})) + variables: + - name: protectedKeys + expression: >- + [ + "metallb.io/loadBalancerIPs", + "metallb.io/address-pool", + "metallb.io/allow-shared-ip", + "metallb.universe.tf/loadBalancerIPs", + "metallb.universe.tf/address-pool", + "metallb.universe.tf/allow-shared-ip", + "metallb.drivers.local.sdn.cozystack.io/pinned" + ] + - name: oldAnnotations + expression: >- + request.operation == "UPDATE" && has(oldObject.metadata.annotations) + ? oldObject.metadata.annotations : {} + - name: newAnnotations + expression: >- + has(object.metadata.annotations) ? object.metadata.annotations : {} + - name: changedKeys + expression: >- + variables.protectedKeys.filter(k, + (k in variables.newAnnotations) != (k in variables.oldAnnotations) || + (k in variables.newAnnotations && variables.newAnnotations[k] != variables.oldAnnotations[k])) + - name: oldLoadBalancerIP + expression: >- + request.operation == "UPDATE" && has(oldObject.spec.loadBalancerIP) + ? oldObject.spec.loadBalancerIP : "" + - name: newLoadBalancerIP + expression: >- + has(object.spec.loadBalancerIP) ? object.spec.loadBalancerIP : "" + validations: + - expression: size(variables.changedKeys) == 0 + messageExpression: >- + "Service annotations " + variables.changedKeys.join(", ") + + " are reserved to the address driver; reserve an IPAddressClaim and name it" + + " with local.sdn.cozystack.io/ip-address-claim instead" + reason: Forbidden + - expression: variables.newLoadBalancerIP == variables.oldLoadBalancerIP + message: >- + spec.loadBalancerIP is reserved to the address driver; reserve an + IPAddressClaim and name it with local.sdn.cozystack.io/ip-address-claim instead + reason: Forbidden +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingAdmissionPolicyBinding +metadata: + name: metallb-iad-service-pins + labels: + {{- include "metallb-iad.labels" . | nindent 4 }} +spec: + policyName: metallb-iad-service-pins + validationActions: {{ toJson .Values.admissionPolicy.validationActions }} + matchResources: + namespaceSelector: + {{- toYaml .Values.admissionPolicy.namespaceSelector | nindent 6 }} +{{- end }} diff --git a/packages/system/metallb-iad/charts/metallb-iad/templates/controller.yaml b/packages/system/metallb-iad/charts/metallb-iad/templates/controller.yaml new file mode 100644 index 000000000..8b99c6f68 --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/templates/controller.yaml @@ -0,0 +1,86 @@ +{{- /* The ClusterRole lives in templates/role.yaml, GENERATED by controller-gen + from the +kubebuilder:rbac markers (make manifests). Everything + deployment-shaped is hand-written here. */ -}} +{{- if ne (include "metallb-iad.placeholderNamespace" .) .Release.Namespace }} +--- +apiVersion: v1 +kind: Namespace +metadata: + name: {{ include "metallb-iad.placeholderNamespace" . }} + labels: + {{- include "metallb-iad.labels" . | nindent 4 }} +{{- end }} +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: metallb-iad + namespace: {{ .Release.Namespace }} + labels: + {{- include "metallb-iad.labels" . | nindent 4 }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: metallb-iad + labels: + {{- include "metallb-iad.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: metallb-iad +subjects: + - kind: ServiceAccount + name: metallb-iad + namespace: {{ .Release.Namespace }} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: metallb-iad + namespace: {{ .Release.Namespace }} + labels: + app: metallb-iad + {{- include "metallb-iad.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.controller.replicas }} + selector: + matchLabels: + app: metallb-iad + template: + metadata: + labels: + app: metallb-iad + spec: + serviceAccountName: metallb-iad + {{- with .Values.controller.priorityClassName }} + priorityClassName: {{ . }} + {{- end }} + {{- include "metallb-iad.imagePullSecrets" . | nindent 6 }} + securityContext: + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault + containers: + - name: driver + image: {{ .Values.image }} + imagePullPolicy: {{ .Values.imagePullPolicy }} + args: + - --metallb-namespace={{ .Values.metallbNamespace }} + - --placeholder-namespace={{ include "metallb-iad.placeholderNamespace" . }} + - --lb-class={{ .Values.loadBalancerClass }} + - --leader-elect={{ .Values.controller.leaderElect }} + - --metrics-bind-address={{ .Values.controller.metricsBindAddress }} + - --health-probe-bind-address=:8081 + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + drop: ["ALL"] + resources: + {{- toYaml .Values.controller.resources | nindent 12 }} + livenessProbe: + httpGet: { path: /healthz, port: 8081 } + initialDelaySeconds: 10 + readinessProbe: + httpGet: { path: /readyz, port: 8081 } diff --git a/packages/system/metallb-iad/charts/metallb-iad/templates/role.yaml b/packages/system/metallb-iad/charts/metallb-iad/templates/role.yaml new file mode 100644 index 000000000..b0bf1d8ac --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/templates/role.yaml @@ -0,0 +1,74 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: metallb-iad +rules: +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - "" + resources: + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddressclaims + - ipaddressclasses + verbs: + - get + - list + - watch +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddresses + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - local.sdn.cozystack.io + resources: + - ipaddresses/status + verbs: + - get + - patch + - update +- apiGroups: + - metallb.io + resources: + - ipaddresspools + verbs: + - get + - list + - watch diff --git a/packages/system/metallb-iad/charts/metallb-iad/values.yaml b/packages/system/metallb-iad/charts/metallb-iad/values.yaml new file mode 100644 index 000000000..b9fd39280 --- /dev/null +++ b/packages/system/metallb-iad/charts/metallb-iad/values.yaml @@ -0,0 +1,59 @@ +# Container image for the driver. +image: ghcr.io/cozystack/metallb-iad:main +imagePullPolicy: IfNotPresent +imagePullSecrets: [] + +# Namespace MetalLB is installed in. The IPAddressPools that classes name +# (parameters.pool) are looked up here. +metallbNamespace: metallb-system + +# The --lb-class of the MetalLB instance this driver fronts. Empty for a +# MetalLB that serves Services without a loadBalancerClass, the cluster +# default. Placeholders carry it as spec.loadBalancerClass. +loadBalancerClass: "" + +# Namespace the reservation placeholder Services live in. Empty means the +# release namespace. Tenants must have no write access to it: a Service +# carrying the placeholder label is exempt from conflict detection only +# inside this namespace, so whoever can create Services there can hold +# addresses. Setting it to anything other than the release namespace renders +# a Namespace object as part of the chart. +placeholderNamespace: "" + +controller: + replicas: 1 + # Leader election matters once replicas > 1, but is on by default so scaling + # up is safe without a config change. + leaderElect: true + # ":8443" (HTTPS) or ":8080" (HTTP) to expose controller-runtime metrics; + # "0" disables the endpoint. + metricsBindAddress: "0" + priorityClassName: "" + resources: + requests: + cpu: 10m + memory: 64Mi + limits: + memory: 128Mi + +# Optional ValidatingAdmissionPolicy reserving the Service fields that steer +# MetalLB at a specific address or pool — the metallb.io/ (and legacy +# metallb.universe.tf/) loadBalancerIPs, address-pool and allow-shared-ip +# annotations, spec.loadBalancerIP — and the driver's own pinned marker. +# Only the driver's ServiceAccount and the principals below may set, change +# or remove them; writes that leave them as they were are always admitted. +admissionPolicy: + enabled: false + # [Deny], or e.g. [Warn, Audit] to observe what would be denied first. + validationActions: [Deny] + # Namespaces the policy applies to. Scope it to tenant namespaces when + # platform components elsewhere set these annotations themselves. + namespaceSelector: {} + # Usernames and groups exempt in addition to the driver's ServiceAccount. + # Admission policies bind system:masters too, so cluster admins must be + # listed; kubeadm since 1.29 issues admin credentials in + # kubeadm:cluster-admins instead. + allowedUsernames: [] + allowedGroups: + - system:masters + - kubeadm:cluster-admins diff --git a/packages/system/metallb-iad/values.yaml b/packages/system/metallb-iad/values.yaml new file mode 100644 index 000000000..0b12f6ecc --- /dev/null +++ b/packages/system/metallb-iad/values.yaml @@ -0,0 +1,23 @@ +metallb-iad: + # Cozystack runs MetalLB in cozy-metallb, not the chart's upstream default of + # metallb-system; classes name their IPAddressPool in this namespace. + metallbNamespace: cozy-metallb + # Same digest pin as packages/system/address-controller; bump in lockstep + # with METALLB_IAD_REF in the Makefile. + image: ghcr.io/cozystack/metallb-iad:main-1faf937@sha256:a2a8b05a887f9f28927d75f8b57e264be40e8770d0177798b4f06a0f69c3e793 + # Reserve MetalLB's pool, pin and sharing inputs on tenant Services to the + # driver. A tenant that could set them could steer MetalLB at an address + # reserved for someone else. Every tenant namespace, tenant-root included, + # and nothing else carries namespace.cozystack.io/host, so platform + # components that set these fields in cozy-* namespaces are unaffected. + admissionPolicy: + enabled: true + namespaceSelector: + matchExpressions: + - key: namespace.cozystack.io/host + operator: Exists + # A values list replaces the chart's defaults rather than extending them. + allowedGroups: + - system:masters + - kubeadm:cluster-admins + - cozystack-cluster-admin