Aleksei Sviridkin bc6d5e67b7 fix(e2e): keep failure snapshots complete and gate tenant patch on aggregated-API auth (#3444)
## What this PR does

Two unrelated defects made the sandbox install job's failure handling
unreliable. This fixes both.

The first hit every red run. When a test fails, the diagnostic snapshot
runs `crust-gather collect`, and its host-log leg spawns a privileged
debug pod per node. The sandbox runs baseline PodSecurity, which rejects
each of those pods with a 403, so crust-gather keeps retrying until it
burns its whole collection budget and exits 1. The snapshot verdict then
printed `FAILED` on every failed run even though the namespaced object
state was already captured, so the line that says whether the uploaded
snapshot is trustworthy was always wrong. The host-log leg collects
nothing usable here anyway, so this passes `--disable-additional-logs`
at all three collect sites: the host snapshot from the BATS runner, the
host snapshot from the Chainsaw error handler, and the nested tenant
snapshot. Object collection now finishes and the verdict again separates
a real collection failure from a complete snapshot.

The second is a race in the install test. The "Configure Tenant" step
issues its first `kubectl patch` of the root tenant as soon as an
earlier test sees the aggregated APIService go `Available`. That
condition only tracks endpoint availability, not the serving pod's auth
state. A freshly rolled cozystack-api pod can start answering before it
has loaded the requestheader client-CA, and until it does it drops the
front-proxy identity and answers as `system:anonymous`, so the patch is
rejected with a 403 on the test's very first command. This adds a short
authenticated GET against the same resource before the patch, so the
step proceeds only once the aggregated API both serves and authenticates
the request. If the aggregation layer is actually wedged past two
minutes the gate fails loud instead of hiding a real regression.

Both were traced from a recent sandbox failure (run 30004030331, during
PR #3441's CI).

Out of scope: the broader convergence flakiness of the install step is
not touched here. That covers the long reconcile windows for the storage
and platform releases and the app-level races in the Chainsaw suite. The
durable product-side fix for the second defect, where cozystack-api
should not report ready before the requestheader client-CA is loaded, is
a separate follow-up.

This partially addresses #3368, its crust-gather diagnostics item, and
does not close it.

### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

No UI changes.

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

The diff touches only the e2e test harness (`hack/cozytest.sh`,
`hack/e2e-chainsaw/.chainsaw.yaml`,
`hack/e2e-chainsaw/_lib/run-kubernetes.sh`,
`hack/e2e-install-cozystack.bats`). The trigger map's `hack/` couplings
are `hack/e2e-prepare-cluster.bats` (node prerequisites for
ansible-cozystack and talm) and `hack/package.mk`,
`hack/common-envs.mk`, `hack/update-crd.sh` plus the package layout
(ccp, external-apps-example, cozyhr). None of those are moved, renamed,
or edited here, and nothing else in the map matches.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(e2e): keep failure snapshots complete when PodSecurity blocks crust-gather's host-log leg, and wait for the aggregated API to authenticate before patching the root tenant so the install step no longer flakes on a system:anonymous 403
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved failure diagnostics during automated Kubernetes and tenant
tests by avoiding unsupported privileged log collection steps.
- Ensured failure snapshots complete more reliably instead of timing out
or returning incomplete results.
- Added a readiness check before configuring the root tenant, preventing
intermittent authorization failures during installation tests.

- **Tests**
- Increased reliability of installation and end-to-end test workflows
under baseline security policies.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-08 04:24:48 +03:00
2025-10-08 09:43:34 +05:00
2025-04-01 18:48:14 +02:00
2024-02-08 12:04:32 +01:00

Cozystack Cozystack

Open Source Apache-2.0 License Support Active GitHub Release GitHub Commit OpenSSF Best Practices OpenSSF Scorecard

Cozystack

Cozystack is a free platform and framework for building clouds.

Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.

With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.

Use Cozystack to build your own cloud or provide a cost-effective development environment.

Cozystack user interface

Use-Cases

Documentation

The documentation is located on the cozystack.io website.

Read the Getting Started section for a quick start.

If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.

Versioning

Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.

Contributions

Contributions are highly appreciated and very welcomed!

In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.

You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.

If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.

Community

You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.

License

Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.

Commercial Support

A list of companies providing commercial support for this project can be found on official site.

Languages
Go 47.2%
Shell 37.6%
TypeScript 7%
Go Template 5.4%
Makefile 1.4%
Other 1.2%