0 Fork 0

mirror of https://github.com/cozystack/cozystack.git synced 2026-10-29 16:17:20 +09:00

Files

T

Aleksei Sviridkin 94c41d100f fix(hack): fail when a chart runs no Helm unit test suites

helm-unittest is fail-closed on a suite it cannot parse and on one
declaring `tests: []`, and fail-open on suites that are not there
at all: it prints "Test Suites: 0 passed, 0 total" and exits 0.
Measured on plugin 1.1.1 against a throwaway chart, all three
forms.

hack/helm-unit-tests.sh judges each package by that exit code
alone, so a chart whose suites were deleted, moved, or renamed past
the tests/*_test.yaml glob reports success having asserted nothing.
Moving packages/system/velero/tests aside reproduces it: the script
prints "All Helm unit tests passed" and exits 0.

Check the captured output for the zero-suite line and fail the
package when it appears. Neither `--strict` nor an explicit
non-matching `-f` glob does this on its own; both exit 0.

Matching that line names the cause where it applies, but the set of
ways to run nothing is not closed: a recipe that never invokes
helm-unittest, and a `test:` rule make finds nothing to do for,
both print no marker at all, so no further match reaches them. So
also require the line a real run always emits, a `Test Suites:` count
of at least one. Every way of asserting nothing exits 0, and asking
what the run asserted is the question that has an answer. All 77
packages the script visits emit that line today. The tradeoff is
that a `test` target which legitimately runs no helm-unittest would
need renaming or an opt-out, which is the bargain the zero-suite
check already struck. That is a contract contributors have to know
about, so docs/agents/overview.md states it where it describes the
sweep.

Because the check matches helm-unittest's own summary wording, a
plugin upgrade that reworded it would fail every package at once
while blaming each package's Makefile. The error text says so, so
the reader of a tree-wide red has the real cause in front of them
rather than 77 copies of the wrong one.

The discovery gate has the same shape from the other side.
`make -C dir -n test` succeeds against a file-backed target, so a
path named `test` beside a package Makefile whose rule is not phony
would make both the gate and the run exit 0 without the recipe
firing. Of the packages the script visits that define a `test` rule,
18 declare it phony, and no such path exists today. Match what make
reports rather than looking for the path, because a package that
declares the target phony runs its recipe whatever sits next to it,
and refusing that would be a false failure. The
quoting around the target name differs between make 3.x and 4.x, so
the match accepts either opening quote; the trailing quote alone
would also match a sub-make reporting another target whose name
ends in "test".

Running the suite now captures output rather than streaming it,
because a pipeline's exit status in POSIX sh reports the last
command rather than make. The capture file is allocated once and
removed by the EXIT trap that already owns the failure list, and the
run is pinned to LC_ALL=C: all three checks match English wording,
one on make's and two on helm-unittest's, so a localized make would
disarm the first of them silently, which is the failure mode this
guard exists to remove. The locale reaches every package's recipe,
not only make.

Extend hack/helm-unit-tests.bats with eight tests. Three pin the new
refusals; two pin what they must not refuse, a phony target with a
colliding path and a sub-make reporting an unrelated target up to
date; one pins that a chart which does run suites still passes; two
pin behaviour that was already there, namely that a failing package
is still reported by name and a package with no `test` rule is
skipped rather than failed. That last one carries a second package
that does run a suite, because with only the target-less package
present the script takes its early exit above the failure summary
and the assertion holds whatever the script did. The fixtures stub
make output instead of invoking helm, so they need no plugin
installed.

Both tests that assert a package is named in the failure summary
match the summary's "  - " prefix rather than the bare path. The
script announces "Running tests in $dir" before it runs anything, so
the bare path is present in the output whatever the summary does;
matching it asserted nothing, and the summary could stop naming
packages entirely with the suite still green.

The failing-package fixture reports a passing suite count alongside
its non-zero exit, so that the exit status is the only thing that
can decide the outcome. A fixture that fails without printing the
marker is refused by the positive-evidence check first, which leaves
the exit-status branch unpinned. That shape occurs in the tree: a
`test` target that runs helm-unittest and then a second step, a
render-parity script or a vendored-version guard or go tests, emits
the marker and still fails.

The two tests already in that file cover a package reached under
packages/tests, and their fixture printed a marker and nothing else.
That is the shape the positive-evidence check refuses, so the
fixture now also emits the line a real run emits; without it both
would fail for a reason unrelated to what they assert.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>

2026-08-12 18:45:22 +03:00

8.6 KiB

Raw Permalink Blame History

Cozystack Project Overview

This document provides detailed information about Cozystack project structure and conventions for AI agents.

About Cozystack

Cozystack is an open-source Kubernetes-based platform and framework for building cloud infrastructure. It provides:

Managed Services: Databases, VMs, Kubernetes clusters, object storage, and more
Multi-tenancy: Full isolation and self-service for tenants
GitOps-driven: FluxCD-based continuous delivery
Modular Architecture: Extensible with custom packages and services
Developer Experience: Simplified local development with cozyhr tool

The platform exposes infrastructure services via the Kubernetes API with ready-made configs, built-in monitoring, and alerts.

Code Layout

.
├── packages/               # Main directory for cozystack packages
│   ├── core/              # Core platform logic charts (installer, platform)
│   ├── system/            # System charts (CSI, CNI, operators, etc.)
│   ├── apps/              # User-facing charts shown in dashboard catalog
│   └── extra/             # Tenant-specific modules, singleton charts which are used as dependencies
├── dashboards/            # Grafana dashboards for monitoring
├── hack/                  # Helper scripts for local development
│   └── e2e-chainsaw/     # End-to-end application tests (Kyverno Chainsaw)
├── scripts/               # Scripts used by cozystack container
│   └── migrations/       # Version migration scripts
├── docs/                  # Documentation
│   ├── agents/           # AI agent instructions
│   └── changelogs/       # Release changelogs
├── cmd/                   # Go command entry points
│   ├── cozystack-api/
│   ├── cozystack-controller/
│   └── cozystack-assets-server/
├── internal/              # Internal Go packages
│   ├── controller/       # Controller implementations
│   └── lineagecontrollerwebhook/
├── pkg/                   # Public Go packages
│   ├── apis/
│   ├── apiserver/
│   └── registry/
└── api/                   # Kubernetes API definitions (CRDs)
    └── v1alpha1/

Package Structure

Every package is a Helm chart following the umbrella chart pattern:

packages/<category>/<package-name>/
├── Chart.yaml                           # Chart definition and parameter docs
├── Makefile                             # Development workflow targets
├── charts/                              # Vendored upstream charts
├── images/                              # Dockerfiles and image build context
├── patches/                             # Optional upstream chart patches
├── templates/                           # Additional manifests
├── templates/dashboard-resourcemap.yaml # Dashboard resource mapping
├── values.yaml                          # Override values for upstream
└── values.schema.json                   # JSON schema for validation and UI

Build and Development Commands

Root targets (run from the repo root):

make build          # Build all Docker images (needs: docker, skopeo, jq, gh, helm, yq, GNU tar/sed/awk)
make unit-tests     # Run all unit tests (Helm, BATS, Go, etc.)
make generate       # Code generation (hack/update-codegen.sh) — CRDs, DeepCopy, clients, RBAC
make manifests      # Generate CRD manifests and operator YAML variants
make cozypkg        # Build the cozypkg CLI
make test           # Full E2E tests (requires a cluster)
make prepare-env    # Prepare the E2E test environment (apply + prepare-cluster)

Per-package targets (run inside packages/{apps,system,extra,core}/<name>/):

make image          # Build the package image
make show           # Render the Helm template
make apply          # Apply the Helm release to a cluster (needs NAME, NAMESPACE)
make diff           # Diff the template against the cluster
make test           # Package-specific tests
make generate       # Regenerate values.schema.json + README.md from values.yaml

Build environment variables:

REGISTRY — Docker registry (default ghcr.io/cozystack/cozystack).
PUSH=1 / LOAD=0 — control buildx push/load behaviour.
LOAD=1 PUSH=0 — load images locally instead of pushing.

Values schema generation

Package values.yaml files carry annotations (@param, @typedef, @field, @enum, @section) that cozyvalues-gen turns into values.schema.json and README.md. Run make generate in the package after editing annotations. A pre-commit hook runs make generate across packages/apps/* and packages/extra/* on every commit to keep these in sync — see contributing.md for the regenerate-before-commit rule.

Testing

Helm unit tests: make helm-unit-tests (runs hack/helm-unit-tests.sh over every package that defines a test target). A test target is expected to run helm-unittest: the sweep fails a package whose run reports no suite, because exiting 0 having asserted nothing is indistinguishable from passing. A target that drives something else belongs under another name, as packages/core/testing already does with its sandbox flow. make unit-tests runs the full unit suite — Helm, BATS, Go, and the preset/readiness checks.
E2E tests: Kyverno Chainsaw suites in hack/e2e-chainsaw/ (one directory per app), run with chainsaw test. Cluster bootstrap (hack/e2e-install-cozystack.bats) and the OpenAPI checks (hack/e2e-test-openapi.bats) remain BATS. Conventions for writing and stabilising them — and the CI that runs them — live in e2e-testing.md.
Go tests: standard go test, with Ginkgo/Gomega for controllers.

Conventions

Helm Charts

Follow umbrella chart pattern for system components
Include upstream charts in charts/ directory (vendored, not referenced)
Override configuration in root values.yaml
Use values.schema.json for input validation and dashboard UI rendering

Go Code

Follow standard Go conventions and idioms
Use controller-runtime patterns for Kubernetes controllers
Prefer kubebuilder for API definitions and controllers
Add proper error handling and structured logging

Git Commits

Conventional Commits (type(scope): description) with --signoff, kept atomic and focused. Full format, scopes, AI-attribution trailer, and PR workflow are in contributing.md — the source of truth; do not duplicate them here.

PackageSource CRD upgrade policy

Each component in a PackageSource may set install.upgradeCRDs to control how CRDs from the chart's crds/ directory are handled on HelmRelease upgrades. Allowed values: Skip (default — helm-controller does not touch CRDs on upgrade), Create (create new CRDs only), CreateReplace (create new and overwrite existing).

Set upgradeCRDs: CreateReplace for operators whose upstream regularly adds new CRDs between versions (etcd-operator, cnpg, kubevirt, kamaji). Without it, new CRDs from a chart bump do not land on existing clusters — only fresh installs get them.

Do not set CreateReplace blindly: it overwrites every CRD in crds/ and can cause silent data loss if upstream drops a field from a CRD that has live objects. Only enable it for operators whose schema evolution is additive-only. When in doubt, leave it unset and apply new CRDs manually.

Documentation

Documentation is organized as follows:

docs/ - General documentation
docs/agents/ - Instructions for AI agents
docs/changelogs/ - Release changelogs
Main website: https://github.com/cozystack/website

Domain references for specific tasks

Before working on these areas, read the relevant doc first:

Backup subsystem design: api/backups/v1alpha1/DESIGN.md
A new app package: copy the structure of an existing one (e.g. packages/apps/postgres/)

Things Agents Should Not Do

Never Edit These

Do not modify files in /vendor/ (Go dependencies)
Do not edit generated files: zz_generated.*.go
Do not change go.mod/go.sum manually (use go get)
Do not edit upstream charts in packages/*/charts/ directly (use patches)
Do not modify image digests in values.yaml (generated by build)

Version Control

Do not commit built artifacts from _out
Do not commit test artifacts or temporary files

Git Operations

Do not force push to main/master
Do not update git config
Do not perform destructive operations without explicit request

Core Components

Do not modify packages/core/platform/ without understanding migration impact

8.6 KiB Raw Permalink Blame History