mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 22:13:18 +09:00
Aleksei Sviridkin
c5f352829c
shellcheck ran nowhere: not from the Makefile, not from a workflow, not from pre-commit. Seven files under hack/ nevertheless carried `# shellcheck disable=` directives, twelve of the twenty-eight with a rationale beside them, so the tree was written as though the check existed. hack/shellcheck.sh lints every shell script this repository owns and compares the result with hack/shellcheck-baseline.txt. Enumeration is a union of shebang and `.sh` suffix, and neither half is redundant. The suffix alone misses fifty-seven files with no extension, fifty-three of them the platform migrations run-migrations.sh runs on a cluster upgrade and which no linter had seen before this; the shebang alone misses thirteen held by the suffix, the sourced libraries that open with a `# shellcheck shell=` directive and four with no marker at all. Narrowing to either one loses files, and what the run then reports never names the enumeration as the cause. Measured, and it differs by half: a script that leaves scope carrying baseline entries goes STALE, one carrying none -- sixty-six of the hundred and ninety -- leaves in silence, and dropping the suffix branch also removes the sourced libraries from the command line, so their sourcers gain SC1091 as NEW. The reader is shown a fixed finding to regenerate away, or a new finding to go fix, and in neither case a narrower gate. The baseline records the findings that predate the gate as `path check count` triples. Among the scripts in scope no file is skipped and nothing is excluded on the command line, though in-file `# shellcheck disable=` directives are still honoured, so a finding suppressed that way is absent from the list rather than recorded in it; two classes sit outside the scan and say so in the baseline header, the vendored chart scripts that make update regenerates and the .bats files, whose bats shebang the enumeration does not match. shellcheck parses those perfectly well, with a dialect and checks of its own for them; they are not baselined yet, and the .bats arm of the filter is a guard for one that acquires a shell shebang rather than an exclusion that does anything today. Dropping line and column numbers keeps an unrelated edit from reddening the ratchet, while the count still catches a new finding in a script already listed. An entry whose findings are gone fails the run too, so fixing a script forces the list to shrink rather than leaving a standing licence to reintroduce what was removed. The regenerate advice is withheld whenever the same run also reports a NEW finding. Printed together, the last line read wins, and regenerating a mixed run records the new findings rather than reporting them -- the cure offered for one half launders the other. Three ways the gate could have passed without checking anything are closed. A missing binary fails rather than skips. The version is pinned on both sides and the script refuses to run when they disagree, because findings differ between shellcheck releases and an unpinned binary reports against a baseline not written for it. And shellcheck's own exit status is kept instead of being piped through xargs, which collapses it: a status above 1 means an input was never analysed, and that now fails rather than reading as a clean file. A path holding a space or a tab is refused instead of linted. A baseline entry is three space-separated fields, so such a path cannot be recorded: the summary splits it and the entry names a file that does not exist. Refusing is also what licenses the word splitting the enumeration relies on, which until now was a comment asserting the tree had no such name rather than a check that it does not. hack/shellcheck.bats drives the gate through a mutation that introduces a violation and its removal, both directions of the ratchet, each of the four refusals above, and the enumeration each half of the union catches, the shrink half in the case where a count drops without reaching zero as well as the one where an entry vanishes, and the whitespace refusal on a tab as well as on a space. Every red path asserts the exit status before it asserts the message: `! cmd` alone proves nothing, because `set -e` never fires on an inverted command, and a gate that printed each ERROR line while exiting 0 satisfied all thirteen output greps when that was measured. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
170 lines
10 KiB
Markdown
170 lines
10 KiB
Markdown
# Cozystack Project Overview
|
|
|
|
This document provides detailed information about Cozystack project structure and conventions for AI agents.
|
|
|
|
## About Cozystack
|
|
|
|
Cozystack is an open-source Kubernetes-based platform and framework for building cloud infrastructure. It provides:
|
|
|
|
- **Managed Services**: Databases, VMs, Kubernetes clusters, object storage, and more
|
|
- **Multi-tenancy**: Full isolation and self-service for tenants
|
|
- **GitOps-driven**: FluxCD-based continuous delivery
|
|
- **Modular Architecture**: Extensible with custom packages and services
|
|
- **Developer Experience**: Simplified local development with cozyhr tool
|
|
|
|
The platform exposes infrastructure services via the Kubernetes API with ready-made configs, built-in monitoring, and alerts.
|
|
|
|
## Code Layout
|
|
|
|
```
|
|
.
|
|
├── packages/ # Main directory for cozystack packages
|
|
│ ├── core/ # Core platform logic charts (installer, platform)
|
|
│ ├── system/ # System charts (CSI, CNI, operators, etc.)
|
|
│ ├── apps/ # User-facing charts shown in dashboard catalog
|
|
│ └── extra/ # Tenant-specific modules, singleton charts which are used as dependencies
|
|
├── dashboards/ # Grafana dashboards for monitoring
|
|
├── hack/ # Helper scripts for local development
|
|
│ └── e2e-chainsaw/ # End-to-end application tests (Kyverno Chainsaw)
|
|
├── scripts/ # Scripts used by cozystack container
|
|
│ └── migrations/ # Version migration scripts
|
|
├── docs/ # Documentation
|
|
│ ├── agents/ # AI agent instructions
|
|
│ └── changelogs/ # Release changelogs
|
|
├── cmd/ # Go command entry points
|
|
│ ├── cozystack-api/
|
|
│ ├── cozystack-controller/
|
|
│ └── cozystack-assets-server/
|
|
├── internal/ # Internal Go packages
|
|
│ ├── controller/ # Controller implementations
|
|
│ └── lineagecontrollerwebhook/
|
|
├── pkg/ # Public Go packages
|
|
│ ├── apis/
|
|
│ ├── apiserver/
|
|
│ └── registry/
|
|
└── api/ # Kubernetes API definitions (CRDs)
|
|
└── v1alpha1/
|
|
```
|
|
|
|
## Package Structure
|
|
|
|
Every package is a Helm chart following the umbrella chart pattern:
|
|
|
|
```
|
|
packages/<category>/<package-name>/
|
|
├── Chart.yaml # Chart definition and parameter docs
|
|
├── Makefile # Development workflow targets
|
|
├── charts/ # Vendored upstream charts
|
|
├── images/ # Dockerfiles and image build context
|
|
├── patches/ # Optional upstream chart patches
|
|
├── templates/ # Additional manifests
|
|
├── templates/dashboard-resourcemap.yaml # Dashboard resource mapping
|
|
├── values.yaml # Override values for upstream
|
|
└── values.schema.json # JSON schema for validation and UI
|
|
```
|
|
|
|
## Build and Development Commands
|
|
|
|
Root targets (run from the repo root):
|
|
|
|
```bash
|
|
make build # Build all Docker images (needs: docker, skopeo, jq, gh, helm, yq, GNU tar/sed/awk)
|
|
make unit-tests # Run all unit tests (Helm, BATS, Go, etc.; needs shellcheck 0.11.0)
|
|
make generate # Code generation (hack/update-codegen.sh) — CRDs, DeepCopy, clients, RBAC
|
|
make manifests # Generate CRD manifests and operator YAML variants
|
|
make cozypkg # Build the cozypkg CLI
|
|
make test # Full E2E tests (requires a cluster)
|
|
make prepare-env # Prepare the E2E test environment (apply + prepare-cluster)
|
|
```
|
|
|
|
Per-package targets (run inside `packages/{apps,system,extra,core}/<name>/`):
|
|
|
|
```bash
|
|
make image # Build the package image
|
|
make show # Render the Helm template
|
|
make apply # Apply the Helm release to a cluster (needs NAME, NAMESPACE)
|
|
make diff # Diff the template against the cluster
|
|
make test # Package-specific tests
|
|
make generate # Regenerate values.schema.json + README.md from values.yaml
|
|
```
|
|
|
|
Build environment variables:
|
|
|
|
- `REGISTRY` — Docker registry (default `ghcr.io/cozystack/cozystack`).
|
|
- `PUSH=1` / `LOAD=0` — control buildx push/load behaviour.
|
|
- `LOAD=1 PUSH=0` — load images locally instead of pushing.
|
|
|
|
### Values schema generation
|
|
|
|
Package `values.yaml` files carry annotations (`@param`, `@typedef`, `@field`, `@enum`, `@section`) that `cozyvalues-gen` turns into `values.schema.json` and `README.md`. Run `make generate` in the package after editing annotations. A pre-commit hook runs `make generate` across `packages/apps/*` and `packages/extra/*` on every commit to keep these in sync — see [`contributing.md`](./contributing.md) for the regenerate-before-commit rule.
|
|
|
|
## Testing
|
|
|
|
- **Helm unit tests:** `make helm-unit-tests` (runs `hack/helm-unit-tests.sh` over every package that defines a `test` target). `make unit-tests` runs the full unit suite — Helm, BATS, Go, shellcheck, and the preset/readiness checks.
|
|
- **E2E tests:** Kyverno Chainsaw suites in `hack/e2e-chainsaw/` (one directory per app), run with `chainsaw test`. Cluster bootstrap (`hack/e2e-install-cozystack.bats`) and the OpenAPI checks (`hack/e2e-test-openapi.bats`) remain BATS. Conventions for writing and stabilising them — and the CI that runs them — live in [`e2e-testing.md`](./e2e-testing.md).
|
|
- **Go tests:** standard `go test`, with Ginkgo/Gomega for controllers.
|
|
- **Shell linting:** `make shellcheck` (runs `hack/shellcheck.sh`, part of `make unit-tests`). It covers every shell script this repository owns, enumerated by shebang **and** by `.sh` suffix, and compares the result against `hack/shellcheck-baseline.txt`. Neither half of that union is redundant: a `*.sh` glob alone would miss 57 files with no extension, 53 of them the platform migrations `run-migrations.sh` runs on a cluster upgrade, and a shebang alone would miss 13 held by the suffix, nine opening with a `# shellcheck shell=` directive and four with no marker at all. The baseline records the findings that predate the gate as `path check count` triples; among the scripts in scope no file is skipped and nothing is excluded on the command line, though in-file `# shellcheck disable=` directives are still honoured, so a finding suppressed that way is absent from the baseline rather than recorded in it. Two classes sit outside the scan: the vendored chart scripts under `packages/*/*/charts/`, which `make update` regenerates, and the `.bats` files, whose `#!/usr/bin/env bats` shebang the enumeration does not match. shellcheck can lint those with `--shell=bats` and has bats-specific checks; they are simply not baselined yet. A finding it does not record fails the run, and so does an entry whose findings have since been fixed — that second half is what makes the list shrink instead of ossify. After fixing findings, run `make shellcheck-baseline` and commit the smaller file. One case is deliberately not caught: a change that both removes and introduces a finding of the same check in the same script leaves the count equal and passes, because pinning identity any tighter than a count means a baseline that reddens whenever an edit shifts a line. The shellcheck version is pinned in both `hack/shellcheck.sh` and the unit-test job, because findings differ across releases.
|
|
|
|
## Conventions
|
|
|
|
### Helm Charts
|
|
- Follow **umbrella chart** pattern for system components
|
|
- Include upstream charts in `charts/` directory (vendored, not referenced)
|
|
- Override configuration in root `values.yaml`
|
|
- Use `values.schema.json` for input validation and dashboard UI rendering
|
|
|
|
### Go Code
|
|
- Follow standard **Go conventions** and idioms
|
|
- Use **controller-runtime** patterns for Kubernetes controllers
|
|
- Prefer **kubebuilder** for API definitions and controllers
|
|
- Add proper error handling and structured logging
|
|
|
|
### Git Commits
|
|
|
|
Conventional Commits (`type(scope): description`) with `--signoff`, kept atomic and focused. Full format, scopes, AI-attribution trailer, and PR workflow are in [`contributing.md`](./contributing.md) — the source of truth; do not duplicate them here.
|
|
|
|
### PackageSource CRD upgrade policy
|
|
|
|
Each component in a `PackageSource` may set `install.upgradeCRDs` to control how CRDs from the chart's `crds/` directory are handled on `HelmRelease` upgrades. Allowed values: `Skip` (default — helm-controller does not touch CRDs on upgrade), `Create` (create new CRDs only), `CreateReplace` (create new and overwrite existing).
|
|
|
|
Set `upgradeCRDs: CreateReplace` for operators whose upstream regularly adds new CRDs between versions (etcd-operator, cnpg, kubevirt, kamaji). Without it, new CRDs from a chart bump do not land on existing clusters — only fresh installs get them.
|
|
|
|
Do **not** set `CreateReplace` blindly: it overwrites every CRD in `crds/` and can cause silent data loss if upstream drops a field from a CRD that has live objects. Only enable it for operators whose schema evolution is additive-only. When in doubt, leave it unset and apply new CRDs manually.
|
|
|
|
### Documentation
|
|
|
|
Documentation is organized as follows:
|
|
- `docs/` - General documentation
|
|
- `docs/agents/` - Instructions for AI agents
|
|
- `docs/changelogs/` - Release changelogs
|
|
- Main website: https://github.com/cozystack/website
|
|
|
|
### Domain references for specific tasks
|
|
|
|
Before working on these areas, read the relevant doc first:
|
|
|
|
- **Backup subsystem design:** `api/backups/v1alpha1/DESIGN.md`
|
|
- **A new app package:** copy the structure of an existing one (e.g. `packages/apps/postgres/`)
|
|
|
|
## Things Agents Should Not Do
|
|
|
|
### Never Edit These
|
|
- Do not modify files in `/vendor/` (Go dependencies)
|
|
- Do not edit generated files: `zz_generated.*.go`
|
|
- Do not change `go.mod`/`go.sum` manually (use `go get`)
|
|
- Do not edit upstream charts in `packages/*/charts/` directly (use patches)
|
|
- Do not modify image digests in `values.yaml` (generated by build)
|
|
|
|
### Version Control
|
|
- Do not commit built artifacts from `_out`
|
|
- Do not commit test artifacts or temporary files
|
|
|
|
### Git Operations
|
|
- Do not force push to main/master
|
|
- Do not update git config
|
|
- Do not perform destructive operations without explicit request
|
|
|
|
### Core Components
|
|
- Do not modify `packages/core/platform/` without understanding migration impact
|
|
|