## What this PR does
Reverts the `publishing.exposure=loadBalancer` mode for ingress-nginx
added in #2468.
The mode rendered the ingress-nginx Service as `type: LoadBalancer` and
a `CiliumLoadBalancerIPPool` over `publishing.externalIPs`, as a
migration path off `Service.spec.externalIPs` (deprecated upstream in
Kubernetes v1.36 per KEP-5707). In default Cozystack the implementation
produces two failure modes:
- `publishing.externalIPs` is the operator-supplied list of node public
IPs (addresses the cluster nodes already hold and listen on, see [docs
section "4b. Node Public IP
Setup"](https://cozystack.io/docs/v1.3/install/cozystack/platform/#4b-node-public-ip-setup)),
not a virtual-VIP pool. Treating it as one and DNATing those IPs via
Cilium LB-IPAM collides with the existing ingress-nginx DNAT on the same
IPs (`Service.spec.externalIPs` path), so traffic is forwarded
non-deterministically.
- Cilium in default Cozystack runs with `ipam.mode: kubernetes` and no
L2/BGP announcer, so a pool over addresses outside the node range is
allocated but never advertised — the Service stays unreachable.
The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side `IPPool` baked into the tenant chart. That follow-up belongs
in its own PR.
### Removed
- `publishing.exposure` value (and the inline comment) from
`core/platform/values.yaml`.
- `expose-mode` plumbing line in `core/platform/templates/apps.yaml`.
- `CiliumLoadBalancerIPPool` template at
`apps/tenant/templates/cilium-lb-pool.yaml`.
- `type: LoadBalancer + externalTrafficPolicy: Local` branch in
`extra/ingress/templates/nginx-ingress.yaml`. The Service now renders as
`ClusterIP + spec.externalIPs` when externalIPs are set, falling through
to `type: LoadBalancer` when they are not — pre-#2468 shape.
- Exposure mode section from `extra/ingress/README.md`.
- `exposure_test.yaml` fixtures in both charts.
### Compatibility
Operators who explicitly set `publishing.exposure=loadBalancer` will see
their ingress Service revert to `ClusterIP + spec.externalIPs` on
upgrade. Anyone on the default value (`externalIPs`) is unaffected.
### Release note
```release-note
revert(ingress): remove the publishing.exposure=loadBalancer mode added in #2468. The default externalIPs path is unchanged; operators who explicitly opted into loadBalancer will have their ingress Service revert to ClusterIP + spec.externalIPs on upgrade. Migration off Service.spec.externalIPs (deprecated in KEP-5707) will be reintroduced via a mechanism-agnostic API knob in a follow-up.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Refactor**
* Removed conditional generation of the Cilium load‑balancer IP pool and
simplified service exposure to rely on explicit external IP values.
* Streamlined configuration for service exposure keys.
* **Documentation**
* Removed the exposure-mode guidance from ingress documentation.
* **Tests**
* Deleted the exposure-mode-related test suites covering various
external IP and loadBalancer scenarios.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2603)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Marks the `generate-changelog` job in `.github/workflows/tags.yaml` with
`continue-on-error: true`, and changes the `update-website-docs` gate to
`always() && needs.prepare-release.result == 'success' &&
needs.prepare-release.outputs.skip != 'true'`.
Result: a failure inside `generate-changelog` (AI step timing out, PR
creation step erroring, etc.) no longer blocks `update-website-docs`.
`prepare-release` remains the hard gate — if it fails or signals
`skip=true`, downstream jobs still don't run.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
This pull request contains internal infrastructure changes only and has
no impact on end-users or the product.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2670?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Mark the generate-changelog job with continue-on-error and switch the
update-website-docs gate to always() so a failed or skipped changelog
generation no longer blocks the website docs update. prepare-release
remains the hard gate.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix regression of broken auth by token to the cozy-dashboard
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed a regression in JWK cache initialization where custom HTTP
client configurations for cluster authentication were not being properly
applied, ensuring secure cluster communication and service-account token
handling work as intended.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2661)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Adds `docs/changelogs/v1.2.4.md` so the `Update Release Notes` workflow
can publish the existing `v1.2.4` draft release.
`v1.2.4` is a hotfix patch backporting only the duplicate-`template:`
key removal from #2279 — the breaking service-naming changes in that PR
were intentionally omitted as not appropriate for a patch (see #2652 for
the minimal-backport details).
The `Generate Changelog` step on the tag-push workflow failed twice in a
row (yesterday's cron run and today's manual dispatch) due to `402 You
have no quota` on the Copilot API, so this changelog is hand-written
following `docs/changelogs/patch-template.md` and matches the v1.2.3
style.
## Test plan
- [x] file follows `patch-template.md` structure
- [x] author attribution verified against `gh pr view` for #2279 and
#2652
- [ ] CI passes
- [ ] after merge, `Update Release Notes` picks up the file and
publishes draft release `v1.2.4`
Hand-written patch changelog for v1.2.4 (Copilot quota exhausted on the
automatic Generate Changelog step). Single fix: minimal backport of the
MariaDB backup CronJob duplicate-key removal from #2279 via #2652.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## Summary
Replaces the old openapi-ui with the new cozystack-ui build from the
cozystack-ui repository.
## Key Changes
### New UI
- **Image**: `999669/cozystack-ui:latest@sha256:ea4e832c2...`
- **Architecture**: `linux/amd64`
- **Repository**: https://github.com/cozystack/cozystack-ui
(feat/ui-overhaul branch)
### Removed Components
- Removed BFF (Backend-For-Frontend) container - new UI communicates
directly with Kubernetes API
- Removed old openapi-ui and openapi-ui-k8s-bff build targets from
Makefile
- Removed old UI Dockerfiles and patch files
### RBAC Updates
- Added read permissions for `cozystack.io/applicationdefinitions` to
fix 403 errors
- New UI requires access to applicationdefinitions for marketplace and
catalog functionality
### Bug Fixes
- Fixed VNC WebSocket connections to use dynamic URLs instead of
hardcoded localhost:8001
- VNC now works properly in-cluster deployments
## Testing
Tested on dev3.infra.aenix.org cluster:
- ✅ Dashboard loads correctly
- ✅ Application definitions accessible
- ✅ VNC connections working
- ✅ Backup pages functional
- ✅ All TypeScript errors resolved
## Migration Notes
The new UI is a complete rewrite with:
- Modern React 19 + TypeScript stack
- Improved backup management pages
- Better type safety and error handling
- Direct Kubernetes API communication (no BFF needed)
… with others
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
align Altinity backup strategy field name for similarity with others
```
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
implement backup controller for foundationdb backups
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added FoundationDB backup & restore support via a new FoundationDB
backup strategy and controller; controller now handles FoundationDB
backup/restore jobs.
* **Examples**
* Added a complete FoundationDB backup/restore demo with helper scripts,
step-by-step step scripts, run-all, and cleanup tooling.
* **Documentation**
* New admin and tenant guides plus a README explaining the example
workflow and verification steps.
* **Deprecations**
* In-chart FoundationDB backup configuration marked DEPRECATED—migrate
to BackupClass + FoundationDB strategy.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2606)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
implement backup strategy for MariaDB
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added native MariaDB backup/restore strategy using the operator-native
BackupClass/Plan/RestoreJob flow with example manifests for end-to-end
workflows.
* **Documentation**
* Added MariaDB how‑to and examples; marked legacy restic/chart-side
backup parameters as DEPRECATED and documented migration guidance.
* **Tests**
* Added unit tests covering MariaDB backup/restore reconciliation,
templating, storage handling, idempotency, URIs, and deadlines.
* **Chores**
* Extended controller RBAC to allow observing MariaDB operator
resources.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2605)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Adds infrastructure for building a patched kilo image from upstream
`main` branch source, following the same pattern as the kamaji package.
The patch introduces a new `cross` mesh granularity mode — WireGuard
tunnels are created only between nodes in different locations, while
intra-location traffic stays on the CNI overlay.
Changes:
- `images/kilo/Dockerfile` — multi-stage build: fetches upstream
tarball, applies patches, builds `kg`/`kgctl` binaries with CNI plugins
and iptables-wrapper
- `images/kilo/patches/feat-add-cross-mesh-granularity.diff` — adds
`CrossGranularity` to the kilo binary
- `Makefile` — new `image:` target for building the patched image,
updated `update:` target to pin upstream commit SHA
### Screenshots
N/A — no UI changes.
### Release note
```release-note
feat(kilo): add patched image build with cross mesh granularity support
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added "cross" mesh granularity mode for finer topology grouping,
adjusted routing/encryption behavior
* Added e2e tests covering cross-mode scenarios for improved reliability
* **Chores**
* Build automation now tracks latest main commit and derives image
digests from build metadata
* Introduced multi-stage container build and runtime packaging for kilo,
including required networking tooling
* **Configuration**
* Updated default image reference to a new repository and digest
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2604)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Two related quality-of-life additions to the Keycloak system component:
1. **`keycloak` chart now accepts `extraEnv`.** Operators can append
extra
`EnvVar` entries to the Keycloak StatefulSet container without forking
the
chart. Useful when a custom theme needs runtime configuration via
`kcContext.properties`, or when an extension expects environment
variables.
2. **`keycloak-configure` chart now exposes a top-level `userProfile`
value**
that is mirrored verbatim into
`ClusterKeycloakRealm.spec.userProfileConfig`.
This lets operators declare custom user-profile attributes (required
fields, validators, permissions, ...) declaratively, alongside the
existing
`login` and `smtp` blocks.
Both values default to empty, so existing deployments are unaffected.
### Release note
```release-note
feat(keycloak): support extraEnv values and userProfile customization
```
## What this PR does
Closes#1959.
Continuation of @matthieu-robin's instance-type resource presets work,
addressing the migration gaps and resource regressions surfaced during
the review of #1959.
Resource presets now follow a cloud-style `<series>.<size>` naming
convention. Five series cover the full CPU-to-memory ratio range, each
with eight sizes (`nano` through `4xlarge`):
| Series | Ratio | Use case |
| ------ | ----- | -------- |
| `t1` | 1:0.5 | Tiny / burstable, low memory |
| `c1` | 1:1 | Compute-balanced |
| `s1` | 1:2 | Standard — proxies, caches |
| `u1` | 1:4 | Universal — databases, messaging |
| `m1` | 1:8 | Memory — search, analytics |
40 instance-type presets in total. The seven legacy flat names (`nano`,
`micro`, `small`, `medium`, `large`, `xlarge`, `2xlarge`) remain
accepted as deprecated aliases.
### Upgrade path
The combination of three mechanisms makes the rename a no-op for
existing deployments:
1. **`cozy-lib` legacy aliases** — preserve the exact pre-rename CPU and
memory for every legacy name. `helm template ... --set
resourcesPreset=small` continues to render `1 CPU / 512Mi`, identical to
before. No HelmRelease that already lives on the cluster breaks.
2. **Migration 39** — a pre-upgrade hook walks every
`HelmRelease.spec.values` and every app CR under
`apps.cozystack.io/v1alpha1`, rewriting legacy values to their 1:1
instance-type equivalents (`nano → t1.nano`, `medium → c1.small`, etc).
Idempotent, best-effort, dry-runnable via `MIGRATION_DRY_RUN=1`,
distinguishes "no items" from "kubectl errored" so a broken cluster
cannot silently stamp the version.
3. **`cozystack-api` deprecation warning** — every `Create` and `Update`
on an app CR with a legacy value emits a `klog` warning naming the JSON
path and the 1:1 replacement, so operators discover any remaining legacy
data after the platform upgrade.
The legacy-to-instance-type mapping is 1:1 by CPU and memory:
| Legacy | CPU | Memory | Instance-type |
| -------- | ---- | ------ | ------------- |
| `nano` | 250m | 128Mi | `t1.nano` |
| `micro` | 500m | 256Mi | `t1.micro` |
| `small` | 1 | 512Mi | `t1.small` |
| `medium` | 1 | 1Gi | `c1.small` |
| `large` | 2 | 2Gi | `c1.medium` |
| `xlarge` | 4 | 4Gi | `c1.large` |
| `2xlarge`| 8 | 8Gi | `c1.xlarge` |
Mirrored in four places (cozy-lib template, Go package, migration jq,
docs) with cross-reference comments so drift is visible.
### Why not just #1959
The implementation in #1959 had three blockers that motivated this
rewrite:
- **`cozy-lib` lost the legacy names entirely**, so any existing
HelmRelease with `resourcesPreset: small` failed render with `Preset key
'small' invalid`. Aliases close this hole.
- **Migration only touched HelmReleases**, leaving app CRs unmigrated;
the next `kubectl edit` would fail enum validation. The new migration
walks both.
- **Defaults silently inflated memory**, e.g. Redis `nano → m1.nano`
jumped from 128Mi to 2Gi (×16). With two replicas plus sentinel the e2e
VM ran out of memory, producing the Redis 90s timeout that blocked the
original PR. New defaults preserve resources 1:1.
### Deprecation timeline
Legacy aliases are scheduled for removal in a future cozystack release.
The combination of migration + operator warnings is expected to converge
production clusters before then. Removal will land as a separate PR with
its own migration cleanup pass.
### Test plan
- `make unit-tests` — Go tests for the preset walker + warning helper,
`helm unittest` legacy↔instance-type parity suites for
`postgres`/`clickhouse`/`opensearch`, RD schema invariance check via
`hack/check-rd-presets.sh`.
- `make generate` is idempotent.
- `helm template ./packages/apps/redis` with the new default renders
250m / 128Mi (matches pre-rename Redis exactly — closes the e2e
blocker).
- `helm template ./packages/apps/postgres --set resourcesPreset=small`
renders 1 / 512Mi (legacy alias works).
- Migration dry-run is exercised via `MIGRATION_DRY_RUN=1`.
### Documentation
See
[`docs/operations/resource-presets.md`](docs/operations/resource-presets.md)
for the full size matrix and migration table.
### Release note
```release-note
feat(cozy-lib): introduce `<series>.<size>` instance-type resource presets (t1/c1/s1/u1/m1 × 8 sizes). Legacy flat names (nano…2xlarge) remain accepted as deprecated 1:1 aliases; migration 39 rewrites stored values to the new form. CPU and memory are preserved exactly across the rename.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded resource preset options to instance-style names (e.g., t1.*,
c1.*, m1.*); legacy flat names remain supported as deprecated aliases.
* Deprecation warnings now surface when legacy presets are detected.
* **Documentation**
* Added a comprehensive resource-presets guide and updated chart/app
docs to the new naming convention.
* **Tests**
* Added compatibility tests ensuring legacy ↔ instance-type presets
render identical resources.
* **Chores**
* Migration and CI checks to detect/upgrade legacy preset values.
[](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2607)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Replace deployment-specific examples in the new extraEnv and
userProfile docs with generic placeholders so the chart comments do
not lean on any particular downstream use case.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Operators can now declare custom user-profile attributes (required
fields, validators, permissions, ...) via a new top-level userProfile
value. The value is mirrored verbatim into the
ClusterKeycloakRealm.spec.userProfileConfig managed by the EDP
keycloak-operator.
The default remains empty, so realms that don't need custom user
profile configuration are unaffected.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Custom themes (Keycloakify, FTL) often need runtime values from the
deployment environment via kcContext.properties. Adding an extraEnv
list to the chart lets operators inject those values per cluster
without forking the chart.
The new field appends verbatim to the container env list and accepts
the full corev1.EnvVar schema, including valueFrom for secret or
configMap-backed values.
Assisted-By: Claude AI
Signed-off-by: Kirill Ilin <stitch14@yandex.ru>
Address review feedback from coderabbitai across:
- packages/apps/kubernetes/README.md
- packages/apps/mariadb/README.md
- packages/apps/nats/README.md
- packages/apps/qdrant/README.md
- packages/apps/redis/README.md
- packages/apps/tcp-balancer/README.md
- packages/apps/vpn/README.md
Each of these READMEs left an inline 7-row preset table listing only
the legacy flat names after the parameters block above already
defaulted to an instance-type value (e.g. `t1.nano`). cozyvalues-gen
only regenerates the `## Parameters` block, so the hand-written
section drifted.
Replace the inline tables with the same short paragraph already used
by postgres and kafka READMEs: name the five series, note that the
seven legacy names remain accepted as deprecated aliases, and link to
the canonical reference at docs/operations/resource-presets.md so the
full size matrix and legacy mapping live in exactly one place.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from coderabbitai on
pkg/apis/apps/presets/legacy_test.go:151.
Replace the index-by-index loop over `got` with a single t.Fatalf
that prints both slices via %#v. The previous loop iterated only
across `got`, so extra entries in `want` would slip past unreported
when `got` was shorter than `want`; t.Fatalf with full slice
representations names both sides regardless of length.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on
pkg/registry/apps/application/rest.go:1777.
When app.Spec.Raw cannot be unmarshalled to map[string]any the
deprecation walker now logs a klog.V(2) line naming the kind, name,
namespace and the underlying error before returning. Upstream
validation still owns rejecting invalid specs (this hook is purely
informational), so the log stays at debug verbosity rather than
surfacing as a warning that would fire on transient inputs the
caller is about to reject anyway.
Helper signature changed to (kind, namespace, name string, raw []byte)
so the malformed-input path is reachable from existing unit tests
without forcing a fake Application wrapper.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Address review feedback from gemini-code-assist on hack/check-rd-presets.sh:26
and coderabbitai on hack/check-rd-presets.sh:37.
- Switch the jq enum extractor from a description-substring heuristic
(test("preset"; "i")) to a JSON path-suffix selector that matches
fields literally named resourcesPreset. Eliminates false positives
from unrelated fields whose descriptions happen to mention "preset".
- Use grep -F for literal enum membership so the "." in entries like
t1.nano cannot accidentally match arbitrary characters (e.g. a typo
t1xnano would have satisfied t1.nano under basic regex).
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Document the five instance-type series (t1/c1/s1/u1/m1) introduced
alongside the legacy preset deprecation, the per-size CPU and memory
matrix, and the 1:1 legacy-to-instance-type mapping table that powers
cozy-lib aliases, migration 39, and the operator-side deprecation
warning.
Single source of truth referenced from:
- packages/library/cozy-lib/templates/_resourcepresets.tpl
- packages/core/platform/images/migrations/migrations/39
- pkg/apis/apps/presets/legacy.go
Includes an explicit callout that legacy `medium` (1 CPU / 1Gi) and
instance-type `*.medium` (2 CPU) are not the same — same word,
different resources.
Replace inline preset tables in packages/apps/postgres/README.md and
packages/apps/kafka/README.md with a short paragraph that names the
five series and points at the canonical doc, since the per-app
inline tables listed only the legacy seven and would drift further
as the canonical matrix evolves.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Pin the 1:1 equivalence between legacy flat preset names and their
instance-type counterparts in helm-unittest for three charts that
exercise different preset shapes:
- postgres — top-level only
- clickhouse — top + nested clickhouseKeeper preset
- opensearch — top + nested dashboards preset
Each case asserts that nano <-> t1.nano, small <-> t1.small,
medium <-> c1.small, large <-> c1.medium, and micro <-> t1.micro
render the same CPU and memory.
packages/apps/{clickhouse,opensearch}/Makefile gain a `test:` target
that calls helm unittest so hack/helm-unit-tests.sh actually runs the
new suites in CI; without it the suites would land as dead weight.
hack/check-rd-presets.sh enforces two invariants in `make unit-tests`:
- every packages/system/*-rd/cozyrds/*.yaml openAPISchema carries
the canonical 47-preset enum (40 instance-type + 7 legacy);
- no chart template carries a hardcoded legacy preset literal in
a defaultingSanitize (list "<legacy>" ...) form.
Both turn an accidental drop of a per-chart Makefile step or an
accidental legacy hardcode into a CI failure rather than silent
production-time validation rejection.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Add a new pre-upgrade hook migration that walks every HelmRelease
spec.values and every app CR spec under apps.cozystack.io/v1alpha1, and
rewrites legacy flat resourcesPreset values (nano/micro/small/medium/
large/xlarge/2xlarge) to their instance-type equivalents (t1/c1).
Mapping is 1:1 by CPU and memory — the migration never bumps resource
usage. The same table lives in cozy-lib (legacy aliases) and in
pkg/apis/apps/presets/legacy.go.
Properties:
- Idempotent: re-running on a converted cluster is a no-op because
non-legacy values fall through the jq map_legacy else arm.
- Best-effort: a single failed patch logs a warning and continues;
cozy-lib legacy aliases keep charts rendering with identical CPU
and memory even if the migration is incomplete. kubectl patch
stderr is captured and surfaced in the WARN line so operators see
why a patch was rejected.
- Distinguishes "no items" from "kubectl errored" (RBAC stripped,
network down, CRD missing) so a broken cluster cannot look
identical to a clean no-op run and stamp the version
unconditionally.
- Dry-run: MIGRATION_DRY_RUN=1 prints the would-be patches and
leaves the version stamp at 39.
- Final summary echoed to both stdout and stderr.
- Version stamp only advances to 40 on a fully successful real run.
Bump migrations.targetVersion 39 -> 40 so the hook executes on the
next platform upgrade.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
cozystack-api now scans each app CR Create and Update for the legacy
flat resourcesPreset names (nano, micro, small, medium, large, xlarge,
2xlarge) and emits a klog warning per occurrence with the dotted JSON
path inside the spec and the suggested 1:1 instance-type replacement.
Example log line:
Postgres/foo in tenant-x uses deprecated resourcesPreset "small" at
spec.resourcesPreset; migrate to "t1.small" (1:1 equivalent CPU and
memory)
Warnings are non-blocking. Legacy values still render correctly through
the cozy-lib legacy alias table; the warning helps operators discover
remaining legacy data after the platform upgrade.
The mapping table lives in pkg/apis/apps/presets/legacy.go and is the
single Go-side source of truth. The walker handles top-level, nested,
in-array, non-string and malformed-spec inputs. The wrapper warns are
emitted via a pure helper deprecationMessagesFor so the message format
is unit-testable without intercepting klog.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rewrite resourcesPreset values in every apps/* and extras/* chart from
the legacy flat names to the new instance-type form, using the 1:1
mapping:
nano -> t1.nano micro -> t1.micro
small -> t1.small medium -> c1.small
large -> c1.medium xlarge -> c1.large
2xlarge -> c1.xlarge
Each affected values.yaml carries the canonical 47-value enum (40
instance-type names plus 7 legacy aliases) so the schema accepts both
forms during the deprecation window. cozyvalues-gen regenerates the
matching api/apps/v1alpha1/*/types.go kubebuilder enum and defaults,
the chart values.schema.json, the README parameter table, and the
ApplicationDefinition openAPISchema in packages/system/*-rd/cozyrds.
Three hardcoded legacy literals in chart templates migrated:
- packages/apps/harbor/templates/harbor.yaml:121 (portal)
- packages/apps/opensearch/templates/opensearch.yaml:43 (dashboards)
- packages/system/keycloak/templates/sts.yaml:115 (heap fallback)
packages/apps/foundationdb/Makefile gains the missing hack/update-crd.sh
call. Without it, every regen left foundationdb-rd cozyrds stale and
new FoundationDB app CRs through cozystack-api were rejected at
validation for any non-legacy resourcesPreset value.
Every default-rendered chart produces identical CPU and memory to the
value applied before the rename, courtesy of the legacy alias table in
cozy-lib and the 1:1 default mapping.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Restructure the resource preset table into a flat <series>.<size> naming
convention inspired by KubeVirt and cloud-provider instance types. Five
series cover the full CPU-to-memory ratio range:
t1 (Tiny, 1:0.5) — burstable, low memory
c1 (Compute, 1:1) — CPU-balanced, CPU-bound workloads
s1 (Standard, 1:2) — proxies, caches, lightweight services
u1 (Universal, 1:4) — databases and messaging
m1 (Memory, 1:8) — search and analytics
Each series provides eight sizes (nano, micro, small, medium, large,
xlarge, 2xlarge, 4xlarge), yielding 40 presets in total. Ephemeral
storage remains 2Gi for every preset.
The seven legacy flat names (nano, micro, small, medium, large, xlarge,
2xlarge) are retained as backward-compatibility aliases returning their
original CPU and memory values. Existing HelmReleases and app CRs
continue to render identical resources without a migration step. The
legacy block is deprecated and will be removed in a future release; the
1:1 conversion table is:
nano -> t1.nano (250m / 128Mi)
micro -> t1.micro (500m / 256Mi)
small -> t1.small (1 / 512Mi)
medium -> c1.small (1 / 1Gi)
large -> c1.medium (2 / 2Gi)
xlarge -> c1.large (4 / 4Gi)
2xlarge -> c1.xlarge (8 / 8Gi)
Also fix the preset helper docstring to pass the preset string straight
to unsanitizedPreset, replacing the prior list applied-to-tuple form
that silently invoked the dot value as a function.
Co-authored-by: Matthieu ROBIN <matthieu@hidora.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Point values.yaml to ghcr.io/cozystack/cozystack/kilo with the
freshly built image digest, replacing the upstream ghcr.io/squat/kilo
reference.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Address review feedback from gemini-code-assist on Dockerfile:36:
Remove custom Fastly CDN repo pinned to v3.20 while base image is
alpine:3.23 — use default repos from the base image instead.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Altinity-based ClickHouse backup/restore flow: cluster-scoped
strategy, injected sidecar when enabled, optional referencing of
existing S3 credential Secrets with per-key mapping, s3PathOverride, and
API auth secret.
* **Documentation**
* Complete ClickHouse demo scripts, scenario guides, and updated backup
configuration docs clarifying legacy vs Altinity behavior.
* **Tests**
* New unit and E2E coverage for Altinity flows, ownership, namespace
invariants, and parameter round-trips.
* **Chores**
* RBAC extended for job lifecycle and added image build/publish step.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add Dockerfile for building kilo from upstream main branch source with
patch support, CNI plugins, and iptables-wrapper. Update Makefile with
`image:` target for building the patched image and an updated `update:`
target that pins the upstream commit SHA.
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Reverts the ingress loadBalancer exposure mode added in #2468.
The mode rendered the ingress-nginx Service as type: LoadBalancer and a
CiliumLoadBalancerIPPool over publishing.externalIPs, as a migration path
off Service.spec.externalIPs (deprecated upstream in Kubernetes v1.36 per
KEP-5707). In default Cozystack the implementation produces two failure
modes:
- publishing.externalIPs is the operator-supplied list of node public IPs
(addresses the cluster nodes already hold and listen on), not a
virtual-VIP pool. Treating it as one and DNATing those IPs via Cilium
LB-IPAM collides with the existing ingress-nginx DNAT on the same IPs
(Service.spec.externalIPs path), so traffic is forwarded
non-deterministically.
- Cilium in default Cozystack runs with ipam.mode: kubernetes and no
L2/BGP announcer, so a pool over addresses outside the node range is
allocated but never advertised — the Service stays unreachable.
The KEP-5707 deprecation is real, but the right migration is a
mechanism-agnostic tenant API knob translated by a Cozystack-side
controller into whichever backend the cluster admin has configured
(MetalLB pool, Cilium LB-IPAM, robotlb, externalIPs pin) — not a
chart-side IPPool baked into the tenant chart. That follow-up belongs in
its own PR.
Removed:
- publishing.exposure value (and the inline comment) from
core/platform/values.yaml
- expose-mode plumbing line in core/platform/templates/apps.yaml
- CiliumLoadBalancerIPPool template at
apps/tenant/templates/cilium-lb-pool.yaml
- type: LoadBalancer + externalTrafficPolicy: Local branch in
extra/ingress/templates/nginx-ingress.yaml (Service renders back as
ClusterIP + spec.externalIPs when externalIPs are set, falling through
to type: LoadBalancer when they are not — pre-#2468 shape)
- Exposure mode section from extra/ingress/README.md
- exposure_test.yaml fixtures in both charts
Operators who explicitly set publishing.exposure=loadBalancer will see
their ingress Service revert to ClusterIP + spec.externalIPs on upgrade.
Anyone on the default value (externalIPs) is unaffected.
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
## What this PR does
Bumps the vendored cozy-ouroboros chart to v0.7.2 and lands the
ouroboros e2e bats hardening that finally drives the addon test green on
the cozystack tenant runner.
The v0.7.2 release of lexfrei/ouroboros adds a `--log-level` flag,
plants Debug logs at every informer event handler
(`AddFunc`/`UpdateFunc`/`DeleteFunc` with kind+namespace/name) and at
reconcile entry/exit, wires `SetWatchErrorHandler` on
Ingress/Gateway/HTTPRoute informers, and surfaces cache-sync elapsed at
Info. The cozystack-side default is `info` (upstream chart default); a
tenant operator who needs debug verbosity can opt in via
`addons.ouroboros.valuesOverride.ouroboros.controller.logLevel: debug`.
## Root cause of the test flake
The ouroboros bats was failing on `kubectl --namespace kube-system get
configmap coredns-custom --output
"jsonpath={.data['ouroboros.override']}"`. The jsonpath bracket-notation
`[ 'key' ]` for a ConfigMap data key that contains a literal dot
(`ouroboros.override`) is a silent-empty kubectl trap: the parser treats
the bracket as an array index, the single-quoted string is not a valid
numeric index, and rather than erroring the result is `""`. The original
comment in the bats source claimed the bracket form "tolerates dotted
keys" — "tolerates" turned out to mean "does not crash, returns empty
without error", not "extracts the value".
Reproduced standalone on kind v1.35.0 against a ConfigMap whose only
data key is literally `ouroboros.override`:
```
$ kubectl -n default get configmap test --output 'jsonpath={.data}'
{"ouroboros.override":"rewrite name foo bar."}
$ kubectl -n default get configmap test --output "jsonpath={.data['ouroboros.override']}"
# ← empty string, exit 0
$ kubectl -n default get configmap test --output 'jsonpath={.data["ouroboros.override"]}'
error: error parsing jsonpath {.data["ouroboros.override"]}, invalid array index "ouroboros.override"
```
The fix is to dump the whole `.data` map (`jsonpath={range
.items[*]}…{.data}`) and let `grep -q "rewrite name <host>"` find the
entry. This is the same shape that `dump_tenant_state` had been using
successfully all along — the failure dumps consistently showed the
up-to-date ConfigMap while the bats poll was reporting empty.
This is a known kubectl jsonpath limitation, not a kamaji /
kube-apiserver issue. Two earlier diagnostic runs (replicas=1,
replicas=2 with apiserver logs piped into `dump_host_state`) were
chasing the wrong hypothesis; the kubectl test reproduces in any
cluster.
## Changes
- `packages/system/ouroboros/`: vendored chart bumped 0.7.0 → 0.7.2;
controller image digest + chart manifest digest updated in lockstep;
helm-unittest assertions updated for new image pin and chart-default
`--resync=30s` / `--log-level=info`.
- `packages/apps/kubernetes/templates/helmreleases/ouroboros.yaml`:
`cozystack.defaultOuroborosValues` pins `controller.clusterDomain:
cluster.local` (auto-detect path returns `cozy.local` on cozystack
tenants and the proxy `/readyz` NXDOMAINs forever).
- `packages/apps/kubernetes/tests/ouroboros_test.yaml`: new fixtures pin
the clusterDomain default and the tenant escape-hatch override paths
(clusterDomain override, logLevel opt-in-to-debug).
- `hack/e2e-apps/ouroboros.bats`: poll switched off jsonpath
bracket-notation (silent-empty trap) onto a whole-`.data`-map dump,
`dump_host_state` helper added with Kamaji apiserver +
konnectivity-server logs, `dump_tenant_state` now calls
`dump_host_state` first, and the rewrite-poll deadline raised 180s →
900s.
- `.github/workflows/pull-requests.yaml`: E2E job timeout 120m → 180m to
cover the long tail (full-matrix e2e on this branch ran 2h05m).
## Companion docs
cozystack/website#534 (draft) updates
`content/en/docs/next/networking/hairpin-proxy-protocol.md` to reflect
the new `clusterDomain` pin and drops stale 0.7.0 / digest hardcodes
from the air-gapped operators section.
## Release note
```release-note
NONE
```
The v0.7.2 ouroboros release adds --log-level / OUROBOROS_CONTROLLER_LOG_LEVEL,
plants Debug logs at every informer event handler (AddFunc/UpdateFunc/
DeleteFunc with kind+namespace/name via describeObj) and at reconcile
entry/exit, wires SetWatchErrorHandler on Ingress / Gateway / HTTPRoute
informers, and surfaces cache-sync elapsed at Info. The diagnostic
surface targets the kamaji-tenant-via-konnectivity flake where bats
sees a fresh Ingress create succeed against the tenant apiserver but
the controller's coredns-custom rewrite lands roughly 7-10 minutes
later — without per-event visibility the failure mode is
indistinguishable between 'watch never fires', 'reconcile silently
errors', and 'API list returns empty until reflector reconnect'.
Bumps the chart-manifest digest, the controller-image digest, and the
cozystack-side image-pin assertion in lockstep. The bats deadline for
the coredns-custom rewrite check rises from 180s to 900s — even on
the broken-watch path the reflector eventually reconnects within
that budget, so the test stops failing on the symptom while we keep
the diagnostic on to learn what the symptom actually is.
cozystack tenants force controller.logLevel: debug via the
defaultOuroborosValues helper; operators that want quieter logs can
override through addons.ouroboros.valuesOverride. Two helm-unittest
fixtures pin both the default and the override path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The 2-hour ceiling on the E2E job was clipping otherwise-working
runs on the oracle-vm-24cpu-96gb-x86-64 runners: image pulls,
multi-stage Talos bootstrap, full helm-release fan-out plus the
post-install bats matrix routinely consume 100-180 minutes of
wall-clock, and runs that would have eventually gone green were
getting cancelled by GitHub at the timeout.
Recent successful E2E runs on the same runner pool sit between
60 and 184 minutes; lift the cap to 180m to cover the long tail
without inviting truly-stuck jobs to linger forever.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The CI run after pinning controller.clusterDomain showed the test was finally getting past the helm-controller wait — the proxy /readyz unblocked, the HR reaches Ready, and the controller pod becomes Ready — but the next assertion (the rewrite snippet appearing in kube-system/coredns-custom within 180s of creating the probe Ingress) is still failing on every retry. Without a dump on this assertion the failure surfaces only as `❌ Test failed (exit 1)` with the polled `snippet=` empty lines preceding it; the controller logs that would explain why the reconciler skipped the Ingress are nowhere visible.
Refactor the existing inline diagnostic block into a `dump_tenant_state` helper that is invoked on every assertion's failure path. Wrap the rewrite-snippet check, the proxy ClusterIP non-empty check, and the dnscheck pod-phase check in `if ! ...; then dump_tenant_state; exit 1; fi` so each failure dumps the same set of host/tenant pods, events, controller and proxy logs, ingress-nginx Service+endpoints, and tenant CoreDNS Corefile that the original HR-wait failure path already covered. Also append a `kubectl get ingress -n default -o yaml` dump of the probe Ingress so a controller log line like "skipped Ingress without TLS host" is correlatable against the actual rendered manifest at the time of failure.
The follow-up fix for whatever the next CI run surfaces will land on this same branch.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The cozystack tenant ouroboros wrapper deliberately left
`controller.clusterDomain` empty so the upstream chart's per-pod
`/etc/resolv.conf` auto-detect could pick up whatever the tenant CoreDNS
actually serves. In a cozystack tenant that auto-detect resolves to the
host platform's `cozy.local` rather than the tenant's `cluster.local`:
the cozystack tenant kubelet injects the host's cluster-domain into pod
search lists, but Kamaji's TenantControlPlane is configured with
`networkProfile.clusterDomain: cluster.local` and the tenant CoreDNS
Corefile serves only that domain. The mismatch makes the proxy compose
its `BackendHost` as
`ingress-nginx-controller.cozy-ingress-nginx.svc.cozy.local`, which
NXDOMAINs against tenant CoreDNS — `/readyz` returns 503 forever, the
proxy Deployment never reaches Ready, helm install times out at 10
minutes, and the e2e job hits the 2h cancel without `ouroboros.bats`
ever progressing past the first kubectl wait.
Pin `controller.clusterDomain: cluster.local` in `defaultOuroborosValues`
so the chart bakes the resolved BackendHost against the domain tenant
CoreDNS actually serves. This is the contract Kamaji TCP defines for
every cozystack tenant; operators on a non-default tenant CoreDNS can
still override via
`addons.ouroboros.valuesOverride.ouroboros.controller.clusterDomain`.
The underlying tenant kubelet injecting the wrong cluster-domain into
pod resolv.conf is a separate bootstrap concern that should be addressed
on its own track; pinning the chart input here keeps ouroboros
operational on the default cozystack tenant topology in the meantime.
Update the corresponding helm-unittest to assert the new contract.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Every tenant-side kubectl call in hack/e2e-apps/ouroboros.bats was failing inside the e2e sandbox container with `dial tcp: lookup kubernetes-${cluster}.${ns}.svc on 169.254.169.254:53: no such host`. The bats test pulled the `super-admin.svc` variant of the tenant admin-kubeconfig, which points at the in-cluster Service DNS name `<release>.<ns>.svc:6443` — a name only host-cluster CoreDNS can resolve. The sandbox container's resolver is the Docker/QEMU host link-local 169.254.169.254, not the host-cluster CoreDNS, so every `KUBECONFIG=$kubeconfig kubectl ...` invocation in the test was returning NXDOMAIN.
The check that gated whether the test even reached its assertions (the `kubectl wait helmrelease kubernetes-${cluster}-ouroboros`) ran against the host kube-context, which masked the broken kubeconfig path until the helmrelease itself failed to reconcile and the diagnostic block fired — at which point every dump command returned only NXDOMAIN errors instead of actionable tenant-cluster state.
Switch to `super-admin.conf` (the operator-facing kubeconfig variant), rewrite `clusters[0].cluster.server` to `https://localhost:${pf_port}`, and run a `kubectl port-forward service/kubernetes-${cluster} ${pf_port}:6443` in the background — the same pattern hack/e2e-apps/run-kubernetes.sh already uses for tenant-cluster reachability. Replace the inline trap RM with a `cleanup_kubeconfig` helper that also kills the port-forward process so retries don't leak listeners.
This is a test infrastructure fix; the underlying ouroboros HelmRelease install timeout itself is a separate root cause that the next CI run, with usable diagnostics, will surface.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The smoke test at hack/e2e-apps/ouroboros.bats waits for the kubernetes-${cluster}-ouroboros HelmRelease to reach Ready and exits with `wait: timed out` on failure. cozyreport collects host-cluster state only — a tenant-side proxy or controller failure surfaces as opaque "0 of 2 pods ready" with no actionable signal, so every retry is fully blind.
Extract the tenant admin-kubeconfig before the wait (gated on the parent kubernetes HR being Ready, which is the precondition for the secret existing) and, on wait failure, dump the host HelmRelease describe, cozy-ouroboros pods, pod descriptions, recent events, controller and proxy logs (200 lines each), the upstream cozy-ingress-nginx Service and endpoints used as the proxy backend, and the tenant kube-system Corefile plus the coredns-custom ConfigMap data.
Drops the ouroboros-HR wait timeout from 25m to 20m to leave room for the diagnostic block within the bats test budget.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Andrei Kvapil
