Files
scooby87 bef42229d4 fix(harbor): clean up orphaned jobservice PVC and ACME solver resources on delete (#3061) (#3077)
## What this PR does

After deleting a `Harbor` app CR, two things were left orphaned in the
tenant
namespace:

- the **jobservice / trivy PVCs** (`<release>-jobservice`, and
`<release>-trivy`
when Trivy is enabled), because
`packages/apps/harbor/templates/harbor.yaml`
sets `persistence.resourcePolicy: "keep"`, which annotates the upstream
Harbor
subchart's PVCs with `helm.sh/resource-policy: keep` so Helm/Flux
deliberately
  never deletes them;
- cert-manager's **ACME http01-solver** resources
(`cm-acme-http-solver-*`), via
the same stuck-challenge mechanism as the Bucket issue (#3060) when
Harbor
  issues a non-wildcard cert.

This adds a single `post-delete` cleanup hook that:

- **deletes the upstream Harbor subchart's kept PVCs** — selecting by
`app.kubernetes.io/instance=<release>-system` (the subchart runs as the
  `<release>-system` HelmRelease, so its PVCs carry that label, **not**
`<release>`). A hook is required precisely because `resourcePolicy:
keep`
blocks normal reclaim. This intentionally covers both the jobservice and
trivy
  PVCs: both hold only regenerable data (jobservice file logs; trivy's
  vulnerability-DB cache), and registry images live in S3.
- **deletes this release's `Certificate`** (deterministic name
`<release>-ingress-tls`); cert-manager then cascade-deletes the `Order`,
`Challenge` and solver `Pod`/`Service`/`Ingress` objects it owns via
owner
  references.
- **host-scoped belt-and-braces sweep** for any ACME http01 solver
objects
cert-manager may have orphaned (e.g. the Certificate was already gone,
so
deleting it did not cascade). The sweep is scoped **strictly to this
release's
Harbor host** — Challenges filtered by `.spec.dnsName`, solver Ingresses
by
  `.spec.rules[0].host` — **not** a namespace-wide
`acme.cert-manager.io/http01-solver=true` label sweep, because a
Cozystack
tenant namespace can host many apps and a blanket label sweep would
delete
  sibling apps' in-flight challenge solvers.

The Job is hardened: non-root (uid 65534), read-only root fs, all caps
dropped,
`seccompProfile: RuntimeDefault`, with CPU/memory requests and limits
set. RBAC
is scoped to exactly the resources it deletes (`persistentvolumeclaims`,
`certificates`, `orders`, `challenges`, solver `ingresses`) and verified
against
the vendored cert-manager CRDs. Failures are logged but never block
teardown.

## Cleanup scope — what is and isn't removed

**Removed:** the upstream Harbor subchart's kept PVCs (jobservice +
trivy) and
this release's cert-manager ACME artifacts (Certificate, and host-scoped
Order/Challenge/solver-Ingress).

**Not touched (by design):**

- **CNPG database PVCs** (`<release>-db-N`) — owned by the CNPG
`Cluster` and
  governed by CNPG's own reclaim policy; they carry CNPG labels, not
`app.kubernetes.io/instance=<release>-system`. Re-installing Harbor with
a
different DB size into the same namespace can therefore leak the old DB
PVCs;
  this is left to CNPG's lifecycle on purpose (the data is the user's).
- **Redis PVCs** — managed by the redis-operator with their own labels,
likewise
  untouched.
- **Solver `Pod`/`Service`** — not deleted directly; they are removed by
cert-manager's owner-ref cascade when the Certificate/Challenge is
deleted, so
  the hook grants no RBAC for them.

> [!NOTE]
> The PVC sweep is eventually-consistent: while the inner
`<release>-system`
> HelmRelease is still being uninstalled by Flux, a PVC may briefly stay
in
> `Terminating` until its consumer Pods (jobservice/trivy) exit. The
Job's
> deletion request always succeeds; reclaim completes once the pods are
gone.

> [!WARNING]
> **This overrides the intentional `persistence.resourcePolicy: "keep"`
setting
> and force-deletes the jobservice/trivy PVCs on app deletion.** Those
PVCs hold
> only jobservice file logs and the trivy vuln-DB cache (registry images
live in
> S3), so data-loss risk is low — but if retaining them across
re-installs is
> desired, this is a conscious product decision to revisit. The ACME
cleanup
> affects only cert-manager's transient artifacts.

Fixes #3061.

> Follow-up tracked in #3088: digest-pin the cleanup `kubectl` image and
route it
> through `cozy-lib.images-registry` (pre-existing debt shared with the
MariaDB
> cleanup hook), out of scope for this PR.

### Release note

```release-note
fix(harbor): clean up the orphaned jobservice/trivy PVCs and cert-manager ACME http01-solver resources left behind after a Harbor app is deleted. NOTE: this force-deletes the jobservice/trivy PVCs, overriding persistence.resourcePolicy=keep (jobservice file logs and trivy vuln-DB cache only; registry images are in S3). CNPG database PVCs are not touched.
```
2026-06-30 15:55:52 +03:00
..

Managed Harbor Container Registry

Harbor is an open-source trusted cloud-native registry project that stores, signs, and scans content.

Prerequisites

The Cozystack Harbor app stores its registry data exclusively in S3-compatible object storage: the chart pins the registry backend to S3 and exposes no filesystem option. That bucket is provisioned through COSI (objectstorage.k8s.io) from a SeaweedFS deployment, so before deploying Harbor the tenant must have SeaweedFS available — enabled on the same tenant or inherited from a parent tenant (the resolved class is propagated down the tenant tree, surfaced as the namespace.cozystack.io/seaweedfs namespace annotation).

Enable it by setting seaweedfs: true on the tenant (or a parent tenant):

seaweedfs: true

Without object storage in the tenant chain, Harbor cannot provision its registry bucket: the <release>-registry BucketClaim/BucketAccess never produces the <release>-registry-bucket credentials secret, so the Harbor HelmRelease stays unreconciled, waiting on BucketInfo.

storageClass is annotated as immutable in the chart schema — see docs/storage-immutability.md for the contract and which consumers enforce it.

Parameters

Common parameters

Name Description Type Value
host Hostname for external access to Harbor (defaults to 'harbor' subdomain for the tenant host). string ""
storageClass StorageClass used to store the data. string ""

Component configuration

Name Description Type Value
core Core API server configuration. object {}
core.resources Explicit CPU and memory configuration. When omitted, the preset defined in resourcesPreset is applied. object {}
core.resources.cpu Number of CPU cores allocated. quantity ""
core.resources.memory Amount of memory allocated. quantity ""
core.resourcesPreset Default sizing preset used when resources is omitted. string t1.small
registry Container image registry configuration. object {}
registry.resources Explicit CPU and memory configuration. When omitted, the preset defined in resourcesPreset is applied. object {}
registry.resources.cpu Number of CPU cores allocated. quantity ""
registry.resources.memory Amount of memory allocated. quantity ""
registry.resourcesPreset Default sizing preset used when resources is omitted. string t1.small
jobservice Background job service configuration. object {}
jobservice.resources Explicit CPU and memory configuration. When omitted, the preset defined in resourcesPreset is applied. object {}
jobservice.resources.cpu Number of CPU cores allocated. quantity ""
jobservice.resources.memory Amount of memory allocated. quantity ""
jobservice.resourcesPreset Default sizing preset used when resources is omitted. string t1.nano
trivy Trivy vulnerability scanner configuration. object {}
trivy.enabled Enable or disable the vulnerability scanner. bool true
trivy.size Persistent Volume size for vulnerability database cache. quantity 5Gi
trivy.resources Explicit CPU and memory configuration. When omitted, the preset defined in resourcesPreset is applied. object {}
trivy.resources.cpu Number of CPU cores allocated. quantity ""
trivy.resources.memory Amount of memory allocated. quantity ""
trivy.resourcesPreset Default sizing preset used when resources is omitted. string t1.nano
database PostgreSQL database configuration. object {}
database.size Persistent Volume size for database storage. quantity 5Gi
database.replicas Number of database instances. int 2
redis Redis cache configuration. object {}
redis.size Persistent Volume size for cache storage. quantity 1Gi
redis.replicas Number of Redis replicas. int 2