mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
597d966d7d688b7fbaec27632b4e87f61ba6ca1f
451
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Aleksei Sviridkin
|
1bc7effadb |
fix(ci): let the fork e2e lane check out the pull request again
actions/checkout v4.4.0 refuses to check out a fork's pull request ref from a workflow_run workflow unless the step opts in. The Renovate roll-up that moved e2e-fork.yaml to v4.4.0 made the publish job fail at checkout for every fork PR, so the required "E2E Tests" status now goes red on all of them regardless of the change. The lane was built to take fork code this way: publish runs no fork-authored script and e2e holds no push token. Opting in restores the behaviour it had on v4.3.1, and a unit test fails if either pull-request checkout loses the opt-in. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
IvanHunters
|
95e68bad86 |
ci(workflows): use a literal match in the flux version guards
The guard ran `grep -qx "flux: v${FLUX_VERSION}"`, where the dots in the
version are regex metacharacters, so a client printing `flux: v2X8X6`
would satisfy it. Use `grep -Fqx` for a fixed-string match in the build
toolchain guards this change added.
Also correct the step comments that still read "install if absent
(idempotent)": the step now installs whenever the pinned version is not
the one present, not only when flux is missing.
Assisted-by: LLM
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
|
||
|
IvanHunters
|
8c0f91ff00 |
ci(workflows): pin flux version in remaining install-latest steps
The nightly toolchain steps still fetched flux through the installer's floating "latest". Pin them to FLUX_VERSION 2.8.6 with the same version-gated guard so nightly's flux push artifact step uses a fixed, reproducible CLI rather than whatever latest resolves to. Also switch e2e-fork's already-pinned step from a presence check to the same version gate, so a runner with a pre-cached older flux is corrected rather than left as is, and fix that step's comment to point at the flux distro cozystack actually ships (internal/fluxinstall/manifests/fluxcd.yaml, v2.8.0). Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
edbc4f1b3e |
ci(workflows): pin flux version in build toolchain steps
The "Set up build toolchain" steps installed flux with a bare `curl https://fluxcd.io/install.sh | sudo bash`, letting the installer resolve its floating "latest". latest had advanced to v2.9.5, and a matrix job on PR #4429 failed when that release's checksums transiently 4xx'd during install. Pin FLUX_VERSION to 2.8.6 and guard on the installed client version, the same shape the "Set up promotion toolchain" steps already use, so the flux CLI is fixed and reproducible across CI instead of following latest. This does not remove the checksum host's transient errors; it removes the uncontrolled version bump. Related to the yq floating-latest pin in #3640. Assisted-by: LLM Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Aleksei Sviridkin
|
2c2c0301f7 |
chore(deps): roll up pending Renovate updates and move to Go 1.27.1 (#4447)
## What this PR does This replaces the open Renovate PRs with one rollup. Go modules move to their newest releases that keep the Kubernetes libraries and controller-runtime where they are, the toolchain and builder images move to Go 1.27.1, and the first-party code gets the Go 1.27 modernizers. Tests for the code the bumps touch come first, in their own commits, and pass on both the old and the new dependencies. Before the bumps: - The e2e sandbox image builds again: dl.min.io answers 410 for mc since MinIO archived it, so mc now comes from its last GitHub release with a pinned checksum. - Renovate stops offering tags of the apimachinery fork. They mirror upstream, so #2999 is a downgrade that drops our patch. - `make unit-tests` now runs the tests of the nested Go modules (kubevirt-csi-driver, token-proxy, kubeovn-webhook, api/apps/v1alpha1). They never ran in CI before. - New tests: JWT verification in the dashboard token proxy, the kube-ovn plunger metrics, cert-manager object idempotency in the tenant gateway, values-file layering in the operator, and the kubevirt CSI driver wrapper. - One real fix. The lineage webhook decoded admitted objects with encoding/json, so an int64 above 2^53 came back rounded. jsonpatch v2.5 compares numbers exactly and would emit a replace that rewrites such a field in the pod. The webhook now decodes with apimachinery's json. Held back on purpose, with the reasons in the commit messages: - Kubernetes libraries stay on v0.35. #3001 needs the apimachinery fork rebased onto release-1.37 first. - cel-go v0.29.2 and etcd v3.6.15, the lines k8s.io/apiserver v0.35 is built and tested with. - grpc v1.83.2. v1.84.0 is affected by GO-2026-6443 and no v1.84 patch release carries the fix yet. - kubevirt.io/api and the CDI API stay on what upstream csi-driver builds against. From CDI v1.60 it no longer compiles. - api/apps/v1alpha1 keeps go 1.26.4, so consumers of the published types are not forced onto Go 1.27. - go fix runs without embedlit (Go 1.27-only syntax, breaks backports to release branches on Go 1.26) and omitzero (would make metadata, spec and status required in the published OpenAPI schemas), and one errorsastype rewrite that did not compile is left out. govulncheck reports nothing reachable in any of the five modules, standard library included. Supersedes #4311, #4310, #4309, #3452, #3993, #3927, #3902, #3805, #3451, #3415, #3414, #3413, #3302, #4050 and #3363. #3001 stays open. Not checked locally: the images that build upstream projects (kube-ovn, kamaji, cilium, linstor-csi and others) with the Go 1.27 builder. The image build jobs of this PR cover them. Closes #4487. ### Screenshots No UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note chore(deps): move the toolchain and builder images to Go 1.27.1 and raise Go dependencies to their latest compatible releases, closing the published advisories; the lineage webhook no longer rewrites int64 fields above 2^53 in admitted objects. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a restore resource policy option for Velero strategies. * Added a test target to run Go tests across tracked modules. * **Bug Fixes** * Updated the end-to-end test environment to download and verify the MinIO client from its current release location. * Added coverage for token validation, webhook patching, controller behavior, and reconciliation. * **Maintenance** * Updated the Go toolchain, dependencies, and container image versions. * Refreshed pinned workflow actions and base image digests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
9c5c5f9d4e |
fix(ci): overlay the right line's refs for a PR on a feature branch
The base-branch overlay pulled cozystack-packages:<base_ref>. A stacked PR's base is a feature branch such as fix/foo: the slash is not valid in an OCI tag and no artifact is published for it, so the pull failed, the step only warned, and every package the PR did not rebuild ran its committed release ref. With a committed ref older than its chart, that broke E2E (Grafana plugin 404). main and release-X.Y still read their own artifact. Any other base reads the artifact of whichever of main and the release lines it has the fewest commits on top of. Always picking main would put main's images under a release line's charts for a stack rooted on that line, the cross-generation mix that fails install. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
renovate[bot]
|
a7dfbe6f8c |
chore(deps): update container image & action digests
Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
509bd9362f |
chore(renovate): stop proposing tags of the apimachinery fork
The replace for k8s.io/apimachinery points at a pseudo-version of the cozystack/apimachinery fork that carries the ConvertToVersion patch. The fork's tags mirror upstream, so Renovate offers one as an update that would silently drop the patch and downgrade the module. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
tym83
|
e24ffa3bce |
docs(governance): nominate @themoriarti as Reviewer for the Proxmox integration
Marian has driven the Proxmox integration since the first Proxmox bundle and is now landing the Cluster API infrastructure provider, CCM and CSI. Giving him code-owner approval on those packages lets Proxmox changes be reviewed by the person who knows them best, without granting approval rights over the rest of the tree. Assisted-by: LLM Signed-off-by: tym83 <6355522@gmail.com> |
||
|
Timofei Larkin
|
06b41da030 |
feat(apps): declare the Kubernetes and Kafka name caps in their charts
Both caps are facts about what each chart renders, and nothing outside the chart can derive them. Every Kubernetes worker pool is a separate KubernetesNodes release named kubernetes-nodes-<name>-<pool>, so a cluster name has to leave room for the default md0 pool inside the 53-character Helm release name limit: 32 characters. A Kafka KRaft controller pod is named kafka-<name>-c-<hash>-<id> with an 8-character hash, and that hostname has to fit the 63-character DNS-1123 label limit: 44 characters. Each chart now states its cap with a @name annotation, which cozyvalues-gen v1.7.0 publishes as x-cozystack-name in values.schema.json and hack/update-crd.sh carries into the ApplicationDefinition. The pre-commit job moves to that release, the first that understands the annotation. Assisted-by: LLM Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Myasnikov Daniil
|
975ea6acb7 |
ci(workflows): gate Talos builds on their inputs
Classify the rename-blind changed-path list against direct Talos sources and shared build inputs. This lets same-repository pull requests skip the heavy build while removals, renames, and build configuration changes still select it. Fork builds remain unconditional. Preserve the matchbox digest produced by a successful Talos build when finalize overlays base-branch image references. When the job is skipped, bootbox continues to receive the current base reference. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrei Kvapil
|
774f788016 |
ci(agents): check commit trailers on every pull request (#4103)
## What this PR does `docs/agents/contributing.md` makes a few properties of a commit message blocking at review: the attribution trailer takes the value `LLM` and nothing else and appears once, no message carries a session trailer or a link to a transcript, no line is a generation byline naming the tool. Nothing checked any of it, so the only reader was a human scrolling `git log`. That reader is expensive and misses things. One open PR carried eight commits naming a model in the trailer, another four, all found by hand after the work was done, each one costing a review round and a rebase. `hack/check-commit-trailers.sh` decides all of it from the text, and the `Commit trailers` job in `.github/workflows/pre-commit.yml` runs it over the commits between the merge base and the head. Over all 6062 commits on `main` it rejects 1300, every one a true positive and no prose false-fires: 1289 attribution trailers naming a model, 24 `Generated with <tool>` bylines, 15 commits carrying two trailers, two `claude.ai/code/session_...` links. ### Where it lives Next to the other per-PR text lint in `pre-commit.yml`. The unit-test job in `pull-requests.yaml` was the other candidate and is wrong for this: it is gated on the diff touching code, so a docs-only PR skips it, and a trailer is wrong or right whatever files the commit changed. The range is `merge-base..head`, recomputed every run, so a rebase or a force-push is read as it now is. It never reaches the history on `main`, roughly a quarter of which carries the old form. ### Decisions worth arguing with It fails rather than warns. A warning in a job nobody watches guards nothing. It is not a required check, and that part is deliberate. Branch protection on `main` requires `pre-commit` and `E2E Tests`, and `Commit trailers` is a new context, so a PR can still merge with it red. 77 of the 200 open PRs carry at least one commit this rejects, so requiring it today turns those red at their next push. Flipping it is a branch-protection change once the open set has drained, and somebody else's call. Fork PRs are checked like any other. The job runs on `pull_request` with a read-only token and reads commit messages as data. A merge commit with no trailer of its own passes. Nothing is required of a commit, only what is written is judged, so a merge commit that does carry a bad trailer fails like the rest. Backport branches are exempt. The backport action cherry-picks merged commits verbatim, so those messages are history and not the author's to rewrite. The exemption matches the generated name `backport-<pr>-to-<branch>`, which is the only signal a `pull_request` event carries, since the label sits on the original PR. So it is a convention, not something enforced. The checker is read from the base branch instead of the checked-out PR. A `pull_request` event takes the workflow definition from the merge of base and head, so the job shows up on every open PR the moment it lands, including the ones whose head predates the script, where reading it from the head is exit 127 and a red check saying nothing about trailers. It also keeps the checker out of reach of the PR being measured, though the job around it is not. Two shapes are left to the reviewer. A model named in `Co-authored-by:` needs a list of model names to tell it from a person, and such a list either misses next month's model or fires on somebody's colleague; most of the 771 such lines on `main` name a model, so it would be deciding a lot of PRs. A `Generated-by:` trailer has never appeared here at all. The byline rule matches a capital `G` for a related reason: case-insensitively, one line in 25 on `main` is an ordinary sentence that wrapped onto "generated with cozyvalues-gen". ### Known costs `hack/check-commit-trailers.sh` matches `hack/[^/]+\.sh$` in `select-e2e.sh`'s `full_suite_pattern`, so this PR and any later one touching the script escalates to the full e2e suite. That pattern's own comment scopes it to the e2e harness, which this script is not, and `inert_config_pattern` already carries `hack/[^/]+\.bats$`. Escalation is fail-safe so it is cost rather than breakage, and reclassifying it wants its own review. The step drops the explicit `git fetch origin "$BASE_REF"` that `api-review-gate.yaml` keeps before its own `merge-base`. `fetch-depth: 0` already fetches `+refs/heads/*:refs/remotes/origin/*`, and if that were ever wrong the failure is a loud `merge-base` error under `set -euo pipefail`, not a silent pass. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff. The two entries that could fire are `cozystack/ccp` and `cozystack/community`, both keyed on a change to the commit convention itself. The convention text is untouched here, only its enforcement is new, so both copies stay accurate. `ccp`'s `package-bump` skill already writes `Assisted-by: LLM`, which this check accepts. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note ci(agents): a pull request now fails when one of its commits carries an attribution trailer whose value is not `LLM`, a second attribution trailer, a session trailer or transcript link, or a `Generated with <tool>` byline. The check reads only the commits the pull request adds; backport branches are exempt. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Pull requests now include automated checks for commit attribution trailers, session links, and generation bylines. * Backport branches are excluded from this validation. * **Documentation** * Added guidance describing the commit-message checks, accepted attribution formats, exceptions, and how to run the validation locally. * **Tests** * Added coverage for valid and invalid commit-message formats, range handling, merge commits, and exempted content. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
d473454529 |
chore(labels): add triage/needs-proposal for requests that go through cozystack/community (#4042)
## What this PR does Adds the `triage/needs-proposal` label to `.github/labels.yml` for feature requests that need a design proposal in [cozystack/community](https://github.com/cozystack/community) before any work starts here. Until now that routing rule lived only in chat, so such requests kept landing in this repository and were re-filed by hand with nothing on the issue saying why. AGENTS.md now spells out the handling next to the rest of the triage guidance: the request is re-filed in cozystack/community, closed here with the label and a link to the proposal, and work in this repository starts from a new issue once the proposal is accepted. ### Screenshots Not applicable, no UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: The diff touches `.github/labels.yml` and `AGENTS.md` only; neither file appears in the downstream trigger map in `docs/agents/contributing.md`. ### Release note ```release-note chore(labels): add triage/needs-proposal for feature requests that need a design proposal in cozystack/community before work starts here ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Chores** - Added a new triage label for issues that require a design proposal before implementation. - Documented the process for submitting, reviewing, and accepting feature proposals before work begins. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
d949099f5e |
fix(ci): map the recurring app scopes to their area labels
Scopes that PR titles use for apps under packages/apps were missing from the labeler's scopeToArea table. A PR whose title carries only such a scope falls through to the area/uncategorized fallback, and the area label has to be set by hand afterwards. Each scope added here clears the bar the table's own comment sets for a new entry, counted over the whole PR history in both title forms the labeler parses, and has an unambiguous home among the labels that already exist. A scope that clears the bar but has no such home is left out rather than pushed into the nearest label: AGENTS.md asks for a new area/* in that case, which is a separate decision. The homes come from the label descriptions. area/virtualization names cdi, and vm-disk renders a DataVolume while vm-instance renders a VirtualMachine. area/kubernetes is the tenant Kubernetes app, and kubernetes-nodes is the worker half split out of it. area/database is described as managed databases and already lists kafka, so rabbitmq, nats, qdrant and foundationdb sit there without stretching it. vpc renders Vpc and Subnet, and area/networking already names kube-ovn. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrei Kvapil
|
54621bc046 |
feat(migration): tenant self-service VM import from VMware vSphere
Hidora's customers are leaving VMware now, and a tenant has to be able to import their own machines without a platform administrator. The chart approach could not express that: a tenant can neither create the Secret holding vCenter credentials nor name the VDDK image, so both had to come from somewhere they cannot reach. Two CRDs in a new `forklift.cozystack.io` group, split by lifecycle. A VMImportSource is a long-lived connection -- type, endpoint, credentials, readiness -- and a VMImportTask is the one-shot operation that names machines and produces VMDisks and VMInstances. The outputs carry no owner reference back to the task, so deleting a finished import leaves the machines alone. Konveyor Forklift does the transfer underneath; the controller renders its objects, mirrors its verdicts, and hands each finished volume into a VMDisk without copying it. What a live vCenter migration forced out, and no amount of source reading would have: - A guest copied verbatim keeps the drivers it had under VMware and no virtio, so on a virtio disk it does not fail the import -- it succeeds and then does not boot. Imported disks go on SATA. - VDDK connects straight to the ESXi host at the address vCenter advertises, which is routinely unreachable, and on the first cluster collided with the Service CIDR. `spec.hosts` redirects per host. - Firmware must be carried across or a UEFI guest imports as BIOS and never boots (stacked on #3002). - Deleting a task mid-transfer orphaned its DataVolume; a finalizer now removes the engine's volumes and leaves the outputs. The console gains a Migration section: connections are editable, and the VM field is a picker built from a machine list the controller publishes, since the aggregated API cannot reach the inventory and should not hold a source's credentials to try. Two general mechanisms fall out of that -- an option source may take an argument, and one named in a CRD annotation may reference a sibling field. Design: cozystack/community#62. Docs: cozystack/website#673. Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
myasnikovdaniil
|
7d7eb1c23e |
chore(codeowners): rebuild ownership by area, five owners minimum (#4129)
## What this PR does
CODEOWNERS applies only the last matching pattern and it replaces every
broader rule including `*`, it does not merge them. Half of what that
costs is already fixed: `bd7b76691` added @kvaps and @lllamnyp to the
rules that were missing them, and `hack/codeowners-invariant.bats` keeps
them there. That invariant reads "every rule with owners is a superset
of the `*` owners", and the other half of the problem satisfies it
without a complaint:
```
/dashboards/ @myasnikovdaniil @IvanHunters @kvaps @lllamnyp <- widened by
|
||
|
Myasnikov Daniil
|
62f2234aea |
fix(e2e): stop a failed teardown posting a failed test verdict
The teardown ends both merge-gating lanes under `if: always()`, and this branch made it `exit "$cleanup_rc"`. So a `docker compose down` hiccup, a busy zpool or a failed rmdir after a fully passing suite posts the required "E2E Tests" status as a failure, and nothing downstream can tell that apart from a test that actually broke. The runner is discarded when the job ends, so there is no next run for the strictness to protect; a reused host is a developer machine, and an annotation serves it. Loud, not fatal. Two comments described mechanisms that are gone. The breakpoint step still explained reading a soft-red output that this branch removed, so a maintainer would go looking for it; the node-join deadline now fails its suite like any other, which is what makes `failure()` sufficient. The build-talos header still described building the nocloud disk and e2e needing it, when the job builds neither and the container lane downloads no disk -- the profile is compiled by nightly and by the tag build's `make assets` instead, and what a PR still compiles is installer.yaml, which shares every input with it bar the platform and output format. And the platform-package override now states its consequence for both packages it replaces, not only for CDI: the merge-gating lane exercises neither the shipped CDI default nor `drbd.enabled: true`. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-By: LLM |
||
|
Myasnikov Daniil
|
16a49740fe |
chore(codeowners): rebuild ownership by area, five owners minimum
CODEOWNERS applies only the last matching pattern and it replaces every broader rule including the catch-all, it does not merge them. Half of what that costs is already fixed: |
||
|
Aleksei Sviridkin
|
d45663f1af |
ci(agents): check commit trailers on every pull request
The attribution and session-link rules in docs/agents/contributing.md are decided by the text of a commit message alone, and until now the only reader was a human scrolling git log. That reader is expensive and misses things: one open pull request carried eight commits naming a model in the attribution trailer, another four, all found by hand after the work was otherwise done. Every one of them cost a review round to report and a rebase to fix. The check reads what is written and requires nothing, so a commit with no trailer passes and a merge commit that carries none passes with it. A trailer that exists and is wrong fails, naming the commit and the accepted form. It runs over the commits between the merge base and the head, so it never reaches the history on main, roughly a quarter of which predates the rule. A backport branch is exempt: its commits are cherry-picked verbatim and their messages are not the author's to rewrite. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
ad00f35f2d |
test(e2e): correct two comments that describe the opposite of the code
pull-requests.yaml said the QEMU substrate "is not wired into any workflow now" and that prepare-cluster and hack/e2e-prepare-cluster.bats stay in the tree for a nightly lane that will carry the coverage. Both halves are wrong in the direction that costs the most: nightly.yaml and e2e-tag.yaml already run `make prepare-env`, which calls prepare-cluster, so anyone trusting that comment deletes live code and takes nightly and every release-candidate run with it. docs/agents/e2e-testing.md states it correctly, so the tree disagreed with itself. hack/e2e-platform-packages.sh said the lane replaces "only LINSTOR with an otherwise-identical Package". It replaces two, and the kubevirt-cdi one is not identical: it raises the importer memory limit from CDI's 600M to 4Gi and the request from 60M to 256Mi. The consequence belongs next to the override -- the merge-gating lane does not exercise the shipped CDI default, and whether a tenant worker disk imports at 600M is answered by the QEMU lanes, which take no override. Also drops the dead half of `resolve_assets`. It resolved nocloud-amd64.raw.xz and exported disk_id; the container lane boots no disk and nothing in this workflow reads that output any more, so all the lookup could still do was fail a release-labelled PR over an asset it does not use. The job stays -- e2e's release arm gates on its result -- and now proves what is still worth proving, that the draft release exists. e2e-tag.yaml keeps resolving the asset, because it consumes it. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> Assisted-By: LLM |
||
|
Myasnikov Daniil
|
186547d835 |
test(e2e): run the merge-gating lanes on Talos containers
The srv1-srv3 nodes of both lanes that gate a merge -- the same-repo one in pull-requests.yaml and the fork one in e2e-fork.yaml -- run as Talos containers instead of QEMU guests, so a tenant worker sits at L2 rather than L3. Measured in CI on commits where both substrates ran the same tree: whole-job wall clock down 24-49 minutes, essentially all of it in Chainsaw, while the two tenant-Kubernetes suites go from 1143-2040s to 466-809s and stop depending on the runner's vgif flag. The node-join soft-red gate goes with the substrate, which is what it was for. It existed because nested virtualisation on the shared runners degraded far enough that a worker registering in minutes elsewhere could miss any deadline this test could afford; #3513 established that the discriminator was the host's kvm_amd vgif flag, and removing a nesting level removes what it acted on. So hack/e2e-node-join-soft-red.sh, the SOFT-RED-node-join.txt marker and the soft_red job outputs are gone from all four workflows, and a missed deadline is an ordinary red again. The ::warning annotation survives, because the reason a run went red should be legible without opening the log, and so does the 124-versus-everything -else test, because "the workers were slow" and "the wait never ran" send a reader to different places. Done by swapping the substrate inside the existing e2e jobs rather than by promoting a separate container job. Those jobs carry more than a substrate -- the GitHub App token, the report-overrun guard, the images list, the SSH breakpoint -- and promoting a job that never had them would have dropped four features without saying so. Three properties of container mode drive the design, and each fails quietly rather than loudly. machine.kernel.modules is a silent no-op -- kernel_module_spec.go returns early on ModeContainer with no error and no event -- so the host loads openvswitch and zfs before the nodes start and the workflow step asserts both rather than trusting them; a missing module surfaces an hour later looking like a CNI or a storage regression. Docker gives a privileged container its own tmpfs /dev seeded once at creation, so /dev/kvm works but ZFS zvols created later never appear and linstor-csi fails ControllerPublishVolume; the compose file binds the host devtmpfs, which is also why talosctl cluster create docker can never carry LINSTOR. And there is no ARP VIP, so 192.168.123.11 stands in for the QEMU lane's .10. The QEMU machinery stays in the tree and keeps running: nightly.yaml and e2e-tag.yaml still call `make prepare-env`, so DRBD, the replicated StorageClass, its Immediate binding mode, the tenant StorageClass propagation that only applies to remotely-accessible classes, and the Cozystack Talos node image with its extensions all keep nightly and release-candidate coverage. None of them is exercised per pull request any more. That is the trade. Nothing on the PR path builds or downloads a nocloud disk now, so build-talos drops the talos-nocloud step and the talos-image artifact. build-talos itself stays: finalize consumes its digest fragment. One property this does not fix, recorded beside the runner class because it will be read off a red run eventually: a container node reports the HOST's CPU and memory as capacity, and hack/e2e-container-up.sh counters that with kubelet systemReserved. That corrects the scheduler and not what a pod reads from /proc, so a workload sizing itself from /proc/cpuinfo or /proc/meminfo is configured for the whole runner. Also carried here because the same files carry them: the ghcr.io pull-through mirror removal (see #4007 -- whichever lands first makes the other a no-op), the authoritative HelmRelease readiness gate, the backup-credential preflight, the prepull guard against an image-less container fragment, and folding the two standalone OIDC suites into the latest-version tenant cluster. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
e49633208c |
ci(tests): parallelize unit test targets
The BATS half of `make unit-tests` ran as one shell loop over ~60 files, which is a nine-minute serial tail no amount of runner CPU could touch. Each file becomes its own Make prerequisite instead, so `-j4` schedules them, and the workflow asks for `make unit-tests test-controllers -j4 --output-sync=target` in a single step: one make process owns all four runner CPUs, the Helm and Go targets share the same bounded pool rather than waiting for each other, and output stays grouped by target instead of four logs interleaving. The existence check moves into its own prerequisite rather than disappearing. It guards against the wildcard matching nothing, which would otherwise make the whole target succeed by having no work. COZYTEST_TRACE rides per target. It arrived with the serial loop it now replaces, and a target that dropped it would go back to streaming full xtrace for every suite -- so the parallelism guard requires the prefix rather than tolerating it. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
51471bb5f6 |
chore(labels): add triage/needs-proposal for requests that go through cozystack/community
Feature requests that need a design decision are routed through cozystack/community first, as an issue or a design-proposal PR, and work in this repository starts only once that proposal is accepted. That rule lived only in chat, so such requests kept being filed here and were moved by hand, with nothing on the issue recording why. A dedicated triage state makes the decision visible on the issue itself: the request is re-filed in cozystack/community and closed here carrying the label and a link to the proposal. Re-filing rather than transferring keeps the label on the closed issue, since a transfer only preserves labels the target repository defines. While the issue is still open, the daily triage sweep re-applies triage/needs-triage to any open issue without a triage/* label, so the routing has to be recorded by replacing the label rather than removing it. AGENTS.md documents the rule next to the rest of the triage guidance. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
61026b732c |
docs(agents): require the neutral Assisted-by: LLM trailer
The attribution trailer exists for disclosure, not to advertise a model. A model name in the trailer tells a reviewer nothing they can act on, and it goes stale with every tool change, so the accepted form is a single `Assisted-by: LLM` line however many tools took part. A link to an assistant session is worse still: it rots, and it carries nothing about the change itself, so no session trailer and no transcript URL belong in a commit, a PR description, or a comment. The commit body explains why a change is made; the diff already shows what changed. The hygiene rules for messages, trailers and in-code comments are written down as review blockers for the same reason: a NOT LGTM on one of them is then a stated rule, not a reviewer's taste. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Andrei Kvapil
|
440c3b58f3 |
test(valkey): add e2e test and wire build, labeler and docs
Add the Valkey e2e bats test (mirrors redis), build the compat image in the top-level build target, map the valkey scope to area/database in the PR labeler, and list valkey among charts with an immutable storageClass. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Myasnikov Daniil
|
a0c297a08a |
docs(ci): mark the legacy backport label arms as transitional
The dual acceptance exists so the rename can land in any order relative to disabling dosubot's PR labelling, not because two spellings are wanted long term. Naming the deletion sites keeps that from becoming permanent by default. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
624a424d29 |
fix(ci): keep the backport concurrency guard working for namespaced labels
The guard from #3569 appends -label to the concurrency group for any labeled event that is not a backport label, so a backport label never cancels a run already in flight. It enumerated the two legacy names only, so once the labels are namespaced a kind/backport event takes the -label branch and the guard stops covering the case it exists for. Both spellings are listed for the same reason the trigger lists both: the org-level dosubot still applies the legacy names. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
IvanHunters
|
0fa6cf72a5 |
chore(ci): complete the backport label rename across all consumers
The initial rename missed several literal consumers of the old label names beyond labels.yml and backport.yaml: - stale.yaml exempt-pr-labels still listed backport,backport-previous. actions/stale matches label names exactly and does not resolve label-sync aliases, so after the rename a kind/backport PR would lose its stale exemption and get auto-closed after inactivity. This is the one behavioural regression; the rest are documentation drift. - PULL_REQUEST_TEMPLATE.md told contributors to apply the backport label. - release.md label-target table and the gh pr list --search commands referenced the old names (the searches would silently return nothing). - contributing.md claimed a backport label is auto-applied on the [Backport] prefix; pr-labeler only adds area/release, so the line was already stale and is corrected while renaming. Historical narrative in release.md and the backport scope-to-area doc rows (a Conventional-Commit scope, not the label) are left as-is. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
7b71053a0b |
chore(ci): namespace the backport labels under kind/
The labels.yml header declares that names follow the Kubernetes scheme, but backport and backport-previous were top-level, violating it. Rename them to kind/backport and kind/backport-previous, grouped under the kind/ section, and carry the old names as aliases so EndBug/label-sync renames in place without dropping history on already-tagged issues and PRs. Update the label literals in backport.yaml in lockstep (the if filter and the label-detection script). pr-labeler.yaml's 'backport' entry is a Conventional-Commit scope-to-area mapping, not a label reference, so it is left unchanged. The top-level release label is a functional trigger keyed by name in several workflows; renaming it is a wider cross-workflow change out of scope here, so it is left as-is intentionally. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Andrei Kvapil
|
6663c44dd5 |
ci(labeler): map the kubeovn-webhook scope to area/networking
The package has its own scope in commit history but no entry, so its PRs land on the area/uncategorized fallback. It is a kube-ovn admission component, which is the same area as the kube-ovn scope beside it. Assisted-By: Claude Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
ad62a8d62b |
test(e2e): stop blocking merges on the tenant node-join deadline
The tenant node-join wait no longer discriminates. Nested virtualisation on the shared runners degrades far enough that a guest which registers in minutes on real hardware can miss any deadline this test can afford, so a red on that one wait says as much about the machine the run landed on as about the product, and it teaches a reader to re-run the job. The suite still fails on it. That is what keeps Chainsaw's catch running: the previous-instance container logs, the host snapshot, the data-plane capture and the event dump are all keyed on the test failing, and the report stays honest. What changes is one layer up. The wait writes a marker beside its diagnostics, and a new lane-side gate reads that marker to decide whether the job blocks. It opens only when every failed test in the run carries one, and blocks on anything it cannot attribute: a report it could not read, a failed test the report does not name, a failure with no marker. Only the lanes that block a merge ask it. The two pull-request lanes do; nightly and the tag lane run the same suites and stay hard. A soft pass on the tag lane would ship a release candidate whose tenant clusters brought up no worker nodes, and nightly is the only place an e2e failure reaches anyone who did not open a pull request, since no lane in this tree notifies anywhere. A softened red finishes the job green, so the job's result stops answering whether the suite passed, and both kinds of consumer are handed the decision instead of inferring it. A step gated on the job failing would silently stop running: the SSH breakpoint is that case, and it still opens for a pull request marked for debugging. A job that reports the result would silently start reporting a pass: the required commit status now says the deadline was missed and that the merge is not blocked on it, rather than that the suite passed. The deadline goes to 29m. That is the test's own budget and nothing more: its clock starts after the machinedeployment wait, the LoadBalancer assignment and the healthz probe, while the chart's autoscaler and health-check timers have been running since the Machine was created, so no ordering against them is claimed. The chart's 20m and 30m give a scale to pick against, and a budget over the first leaves room for a remediation cycle without promising that one fits. The operation ceiling both suites give the script rises with the deadline, and the diagnostics phase budget rises inside it, because the collectors gated ahead of the serial console cost more between them than the old budget covered. Both budget guards read their terms from the sources that set them. The e2e job cap rises with them, and its guard holds what a guard can hold: the pair's operation and catch ceilings, and one figure across every lane that runs the suite. It also records that the suites' full summed ceilings have never fit any cap in that file, so the margin above the bound is taken from a measured run instead. The prose that restates the cap elsewhere moves with it, and the sweep that holds the deadline's restatements now reads the figure spelled in words as well as in minutes. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
61f7939ab7 |
ci(workflows): make E2E breakpoint opt-in via labels only
Drop the author-association branch from the "Breakpoint on E2E failure" step condition, so a red end-to-end run only opens an SSH breakpoint when the pull request carries the debug or release label. The dropped branch held an ephemeral CI runner open for up to twenty minutes on every red maintainer-authored pull request, whether or not anyone intended to attach. Trim the step's explanatory comment to match: drop the paragraph about the removed authorship path, including its stale claim that only the debug label affects runner selection, since this workflow has no conditional runs-on to begin with. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
1592fd9e7c |
fix(ci): resolve fork PRs by open-PR head SHA, not commit association (#3922)
Fork PR e2e lane never ran. `resolve` looks up PR for head sha with
`listPullRequestsAssociatedWithCommit` on base repo, and this endpoint
returns nothing for fork PR head, because fork commit reaches base repo
only as `refs/pull/<n>/head` and association index doesn't walk
`refs/pull/*`. So every fork PR failed closed on `expected exactly one
open PR for <sha>, found 0`, got red required `E2E Tests` that its
author cant clear by pushing, and `publish`, `e2e`, `report` skipped.
Endpoint works fine for same-repo heads, that is why nobody saw it -
same-repo PRs get `E2E Tests` from `pull-requests.yaml` and never enter
this workflow at all. Of 935 runs of `E2E (fork)` 930 skipped as
designed, and all 5 that reached `resolve` stopped right there (three on
`found 0`, one on genuinely failed fork build). One of those reds was
later overwritten by mirrored same-repo run, so symptom got hidden
second time.
Checked on live api: on base repo that endpoint returns 0 for head sha
of all 5 currently open fork PRs and 1 for same-repo PRs, on fork repo
it returns 1 for same commit. Listing open PRs of base repo and matching
head sha returns 1.
### Fix
```js
const openPrs = await github.paginate(
github.rest.pulls.list,
{ owner, repo, state: 'open', per_page: 100 },
);
const open = openPrs.filter(p => p.head && p.head.sha === HEAD_SHA);
```
Same trust properties as call it replaces, this is plain base-repo api
data that fork cannot forge, and asking `state: 'open'` at source keeps
old rule that closed or merged PR never counts. Exactly-one guard and
head sha match are not changed, so stacked lower PR whose head is only
contained in upper branch still passes, and two PRs really sitting at
same head still fail closed. Pagination becomes load bearing here, repo
has 180 open PRs and fork PR that surfaced this sits on page two.
Most of diff is the comment block, rewritten so association call doesn't
come back later as an optimization.
Simulated new logic on live api, failing head resolves to exactly one
open PR and `labels` are present in list payload, so `full-e2e` override
still reads. actionlint, zizmor, `node --check` on extracted script and
pre-commit pass.
### What is not verified
`publish`, `e2e` and `report` never executed in this workflow, so first
fork PR going through the gate will exercise them for the first time. I
would watch digest push from OCI archives and `HEAD^2 == head_sha`
assert. Also there is no test harness for inline `resolve` script and
this PR doesn't add one.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved fork pull request detection by matching the exact triggering
commit.
* Prevented stacked or unrelated pull requests containing the same
commit from being incorrectly selected.
* Ensured the workflow proceeds only when exactly one matching open pull
request is found.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Andrei Kvapil
|
7cca3d75e7 |
feat(kubernetes)!: remove the FluxCD tenant addon
The optional `addons.fluxcd` toggle of the Kubernetes app deployed a Flux Operator + FluxInstance into a tenant cluster. It was served by two vendored charts, packages/system/fluxcd-operator and packages/system/ fluxcd, which were the last AGPL-3.0 content in this Apache-2.0 repository and a governance blocker for CNCF Incubation. The management cluster does not use these charts (it runs classic Flux via internal/fluxinstall; migration 21 already removed the operator from it), and the addon defaulted to disabled. Rather than maintain an AGPL payload for an unused feature, drop it entirely: - delete both system packages and their platform source entries - remove the addons.fluxcd HelmReleases, values and generated schema from the Kubernetes app - drop the addons.fluxcd block still submitted by the e2e harness - drop the removed fluxcd package entries from CODEOWNERS BREAKING CHANGE: the addons.fluxcd option of the Kubernetes app is removed. Tenant clusters that had it enabled keep their running Flux release; see the companion migration for the upgrade path. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Myasnikov Daniil
|
7ac2ffe504 |
fix(ci): resolve fork PRs by open-PR head SHA, not commit association
The fork-PR e2e lane never ran. `resolve` looked the PR up with
`listPullRequestsAssociatedWithCommit` against the base repo, and that
endpoint returns nothing for a fork PR head: the commit reaches
cozystack/cozystack only as `refs/pull/<n>/head`, and the association
index does not walk `refs/pull/*`. So every fork PR fail-closed on
"expected exactly one open PR for <sha>, found 0", stamped a red
required "E2E Tests" its author could not clear by pushing, and skipped
publish/e2e/report.
It resolves same-repo heads correctly, which is why the hole stayed
invisible — same-repo PRs get "E2E Tests" from pull-requests.yaml and
never enter this workflow. Of 935 runs, 930 skipped as designed and all
5 that reached `resolve` stopped there; three on `found 0`, and one of
those reds was later overwritten by a mirrored same-repo run, which hid
it again.
Enumerate the base repo's open pull requests instead and match on the
head SHA. Same trust properties as the call it replaces — plain
base-repo API data the fork cannot forge — and `state: 'open'` at the
source keeps the rule that a closed or merged PR never counts. The
exactly-one guard and the head_sha match are unchanged, so a stacked
lower PR still passes and two PRs genuinely at the same head still fail
closed. Pagination stops being a corner case and becomes load-bearing:
the repo has 180 open PRs, and the fork PR that surfaced this sits on
page two.
Verified against the live API for the failing case: PR #1982 at
|
||
|
Aleksei Sviridkin
|
467e431765 |
chore(ci): add area/cilium label and map the cilium PR scope to it
Cilium has enough open issues of its own (version pins and bumps, the Gateway API implementation, network policy, IPAM next to kube-ovn, LB-IPAM) that area/networking no longer narrows a search. Declare area/cilium in labels.yml, route the cilium scope of PR titles to it instead of area/networking, and update the scope table in the contributing guide. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
myasnikovdaniil
|
9f5550abdd |
fix(release): publish rewritten packages during RC promotion (#3795)
## What this PR does Fixes #3477. - Publishes a new packages OCI candidate after rewriting RC image tags to stable tags. - Pins the candidate digest in the stable installer without rebuilding container images. - Verifies the candidate against the prospective merge and original RC artifact before creating stable names. - Refuses promotion when the target maintenance branch lacks the candidate-aware pipeline. - Protects candidates referenced by open release PRs from retention cleanup. - Leaves already published v1.6.0 and v1.6.1 unchanged. ### Screenshots Not applicable. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Testing - `bats --no-tempdir-cleanup hack/promote-packages-artifact_test.bats hack/verify-promoted-packages_test.bats hack/promote-packages-contract.bats hack/promotion-retention-contract.bats` - `actionlint .github/workflows/promote-rc.yaml .github/workflows/pull-requests.yaml .github/workflows/pull-requests-release.yaml .github/workflows/retention.yaml` - `sh -n hack/promote-packages-artifact.sh hack/verify-promoted-packages.sh` - `git diff --check origin/main...HEAD` ### Release note ```release-note fix(release): publish the rewritten stable packages artifact when promoting a release candidate ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Release candidates now use temporary, immutable package artifacts verified before stable publication. - Stable promotions validate package contents, installer references, container digests, and release metadata before tagging. - Temporary promotion artifacts are retained while needed and cleaned up after expiration. - **Bug Fixes** - Improved handling of release labels, fork pull requests, and promotion pull requests. - Added end-to-end verification before release promotion. - Prevented invalid, ambiguous, or untrusted package references from being promoted. - **Documentation** - Updated release and promotion guidance for candidate verification, digest pinning, and transactional promotion. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
db2ddfab2b |
fix(ci): give the nightly e2e runner headroom over its guest vCPUs
The e2e sandbox boots three QEMU management nodes at `-smp 8`, so 24 guest vCPUs are committed before any tenant VM starts. The nightly lane ran that sandbox on a 24-vCPU runner, a 1:1 commit that leaves the host nothing for QEMU's emulation and IO threads, containerd, the kernel and the runner agent. The tag, fork and pull-request lanes already run the same sandbox on the 32-vCPU class, so move nightly there too. This aligns headroom and claims no performance effect. Measured boot data shows the load a workflow puts on the host, not the runner class, dominates guest starvation, and the two are confounded in that data; no red run is attributed to the class here. The case is the arithmetic: a 1:1 commit leaves no headroom, and two lanes running the identical sandbox on different classes is drift nothing was watching. Add hack/sandbox-runner-headroom.bats to keep the two sides paired. It derives the guest vCPU and memory demand from the sandbox bats file, and the runner class of every job that drives the sandbox make targets, then requires the runner to be strictly larger on both axes. Neither side is a literal list, so raising `-smp`, adding a management node or adding a lane on a smaller runner turns it red instead of drifting apart silently. Input the derivation cannot read in full is refused rather than guessed: a guess produces a demand smaller than the real one, which passes a runner that cannot hold the sandbox, while a refusal names the line. That covers a launch written outside any `@test` block, a block left open at end of file, a `for` opener the strict pattern misses, a non-literal loop list, and a `while` loop sharing the block. The derivation runs against the live tree, which passes whether or not those refusals still work, so each one is pinned by a fixture feeding a synthetic file to the same program. Loosening the parser turns a fixture red rather than quietly widening what the check accepts. Record the rule alongside the other environment invariants in docs/agents/e2e-testing.md, so a new lane picks its runner class from the documentation rather than from a failing unit suite. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov Daniil
|
ffae003d6b |
fix(release): stop a failed org package listing from exiting green
The nightly sweep's own listing shared one `|| true` with the grep that filters it, and grep legitimately exits 1 when no cozystack/* package matches. A failed /orgs/.../packages read therefore arrived as an empty list, took the "No cozystack/* container packages found" branch and exited 0 -- below the nightly sweep but ABOVE the deferred guard, so a protection set that could not be resolved reported success having swept nothing. That bypasses the guard rather than varying it: replaying the step against a stub gh, a failed /pulls combined with a legitimately empty org listing exited 0 after printing its own ::error::. List and filter as two statements so one `|| true` stops meaning two things, handle the listing failure like the other two (::error::, blocked flag, emptied list), and consult the flag before the early exit. That exit is non-zero, matching the other two listing failures: no sweep is left to carry the run down to the guard, so this is the last place the flag can be honoured, and an unresolved protection set is exactly what fail-closed means. The legitimate empty-org case still prints the same line and exits 0. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
dee398f3b3 |
fix(release): defer a retention listing failure to the promotion guard
Both candidate-protection sources open with a `gh api --paginate` listing and neither handled a failure. Under `set -euo pipefail` a transient /pulls or /branches blip therefore exited the step before the unrelated nightly GHCR sweep had run -- the exact coupling the deferred guard at the bottom exists to remove, and which hack/promotion-retention-contract.bats already asserts positionally. Route both listings into that guard instead: raise the blocked flag and empty the list, so the promotion sweep is skipped and the job still fails while the nightly sweep finishes. Emptying is not cosmetic -- a half-paginated list left in place is a smaller protected set, and a smaller protected set is the one shape that can authorise a delete. Replaying the step's real shell against a stub gh: with /pulls failing it exited 5 and never reached the nightly line; it now exits 1 with the nightly sweep done and the promotion sweep skipped. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
04fa8471fe |
docs(release): say what the gates actually guarantee
Three claims that were stronger than the code, plus one legibility fix. The PR gate's comment said a `packages/` edit "on the PR or its base" fails the status before merge. The base half is not true: GitHub fires no `pull_request` event when the base branch advances, it only recomputes refs/pull/N/merge, so a change landing on the base after this went green is never re-verified and the PR merges on the stale result. Checking the merge tree does make base drift visible WHEN the job runs, which is worth keeping — but finalize is what actually catches the drift case, after the merge and before any stable name. Say both, in the workflow and in the reviewer checklist, so nobody reads the green as covering more than it does. "Transactional at every consumer-visible name" is a judgement call dressed as an absolute, inherited from the "this dispatch performs NO registry mutation" it replaced. The dispatch now does write to the registry: the `promotion-*` candidate is a real, publicly listable tag. What it creates is no *stable* name — no vX.Y.Z, no :latest, no stable installer. Stated that way the sentence stops needing the reader to already know which writes were meant. The retention paragraph listed two of the conditions the workflow enforces. It now lists all of them, including the ones the workflow gained here: same-repository, bot-authored, `release-X.Y.Z` branch, the `release` label, exactly one valid digest pin, and protection from a base-branch tip as well as an open PR. Also: `flux pull artifact` reads its credentials from ~/.docker/config.json, which is why finalize now runs `docker login` — but its runner class is not one of the docker-using build ones and the setup step installs everything except docker. Check for it there, where the message can name what is missing, instead of failing with "command not found" halfway down the release path. And the verifier's exclusion list is a restatement of flux's `excludeOCI`, not a derivation — the CLI does not expose it — so the coupling and the pinned version are now written down next to the list. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
820e72d572 |
fix(release): protect a candidate until its retag, not until PR merge
Retention protected a packages candidate only while an OPEN bot release PR pinned its digest. The digest stops being eligible for deletion only when finalize's retag adds a stable tag to the manifest, because that is what makes the `all(tags match promotion-*)` predicate stop matching — and finalize runs after the merge that closes the PR. The window between them is where a half-finished release sits. Finalize creates the write-once stable git tag, then dies before the retag: a lost runner, a 5xx publishing the draft, a non-fast-forward on the maintenance branch. What is left pins the candidate from a merge commit and a tag nobody can move, with no open PR and only a temporary tag on the manifest. Thirty days later retention deletes it, after which re-running finalize 404s at the candidate check and re-dispatching promote is refused because the stable tag exists. The release can then only be finished by hand — the exact class of manual cleanup this design exists to remove. Protect the tips of the branches a promote PR can merge into as well. The two sources are read differently on purpose. An open promote PR's head is supposed to carry exactly one candidate pin, so anything else is an anomaly and blocks. A branch tip is an additional source: every digest pin found is protected and a branch without one contributes nothing, because a maintenance line cut before this pipeline existed cannot pin a candidate and must not wedge retention forever. Blocking now stops the promotion sweep alone. It used to `exit 1` inside the resolution loop, which also killed the unrelated nightly prune further down the same step — so a WIP commit adding a trailing comment to a pin on a release branch would silently stop GHCR nightly pruning. The job still fails; it fails after the nightly sweep has run. Verified by running the step's script against a stubbed `gh`: a digest pinned by main's tip and referenced by no open PR is now excluded while a genuinely abandoned one is still listed; a manifest that also carries a stable tag stays excluded by the existing predicate; and a malformed pin on a PR head leaves the nightly sweep running, skips the promotion sweep and exits 1. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
9fa8eeeda2 |
ci(release): pin the promotion toolchain and cap the candidate gate
Three review findings on the new promotion steps.
`yq` was fetched from `releases/latest/download` in all three steps this
pipeline installs it in — the promote dispatch, the promote PR gate and
finalize. That makes an unpinned moving dependency part of what decides
whether a stable release may be created, and part of what rewrites the
installer pin: two promotions of the same rc can be served different
binaries. Pin it the way FLUX_VERSION already is, version-gated so a
matching preinstalled yq is reused, on the version the e2e sandbox image
already tracks.
A version authenticates nothing about the bytes it names, though, and
these are the workflows that mint write-once stable names. yq is
therefore also pinned BY CONTENT: its SHA-256 is committed beside its
version and verified before install.
Scoped to yq deliberately. It is the one downloaded binary with write
authority over what a release contains — it stamps the candidate digest
into installer values at promote time and platformVersion into the
stable chart at finalize. skopeo, helm and oras only move bytes that are
already digest-addressed, so a substituted one is caught downstream; a
substituted yq writes the wrong pin and every later check verifies that
pin faithfully.
The Flux installer is left as it is, on purpose. It fetches
flux_${FLUX_VERSION}_checksums.txt from the pinned release and verifies
the binary against it, so the binary is not the exposure — the ~6 KB
bootstrap script from fluxcd.io is, and it is mutable by design and
fetched identically in ten other workflows here. Pinning its bytes in
these three files would break on the next regeneration of a script the
project does not control while leaving the same fetch unpinned
everywhere else. The comment names the residual risk instead.
No precedent existed either way: nothing under .github/workflows/ or
hack/ verified a downloaded tool before this, and every version-pinned
download in the tree (oras 1.2.0, cozyvalues-gen 1.6.0) is unverified.
This is a new convention rather than an application of one, confined to
the release path and justified by write authority, and the comment says
so where the next reader will be standing. The three unpinned fetches
outside this pipeline (nightly.yaml, e2e-fork.yaml) are left alone.
The checksum is trust-on-first-use, which is the honest description: it
fixes the bytes to what was checked when the version was chosen and
makes any later substitution loud. Provenance is the release's own
published checksums list cross-checked against the downloaded asset;
both agree on
fa52a4e758c63d38299163fbdd1edfb4c4963247918bf9c1c5d31d84789eded4, and
that binary self-reports v4.53.3.
Verified locally against the real asset: the version gate skips a
matching preinstalled yq and re-installs a mismatched one, the checksum
line reports OK for the genuine binary and exits 1 for a tampered copy,
which aborts the step under the runner's `bash -e`.
`verify-release-candidate` had no `timeout-minutes`, so it inherited the
6-hour default while `e2e-report` waits on it through `needs` — a
stalled `flux pull artifact` would hold the required `E2E Tests` status
pending for the whole of it. 30 minutes, matching `checks` and
`finalize`.
The contract pins that all three install steps carry both versions and
the checksum, run the checksum check, name no `latest/download`, and
agree on one yq version and one checksum. So the unpinned one-line idiom
cannot come back into a release step by copy-paste from the workflows
that still use it, and bumping the version in one file without its
checksum cannot pass. Both mutations are caught.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
|
||
|
Myasnikov Daniil
|
2d734fb229 |
fix(release): preflight the rc tree's own pin before publishing
hack/promote-packages-artifact.sh pushes the candidate BEFORE rewriting installer values, so the artifact carries the rc tree's platformSourceUrl/platformSourceRef verbatim — and hack/verify-promoted-packages.sh reads exactly those two fields back out of it as the candidate's rc baseline, requiring a trusted repository and an immutable `digest=sha256:<64 hex>` pin. The publisher checked neither. A tag-form ref or an untrusted URL in the rc tree published a candidate and pushed two commits regardless; the refusal arrived later, in the promote PR gate or in finalize, once the staging branch and the draft release already existed. Check both at dispatch time instead, while nothing has been written. A copied regex would drift from the guard it is bringing forward and quietly restore the late failure, so the trusted repository and the digest pattern move into hack/lib/promoted-packages.sh and both halves source it. That makes the library part of what the verifier needs at run time, so promote's base-branch precondition gains it: the list exists because the base's own workflows run after merge, and a base carrying the verifier without its library fails at the same two late points. Tests: both preflights are covered, each asserting the mock registry client was never invoked — "exited non-zero" alone does not say nothing was written. Both fail with the preflight removed. The publisher fixtures now pin the trusted repository the way a real rc tree does, and still send the candidate elsewhere via REGISTRY so the rewrite assertions keep their teeth. Also covers a malformed PROMOTION_ID. It is not attacker-influenced, but it is the run-uniqueness suffix of the candidate tag and retention.yaml deletes a candidate only when every tag on the manifest matches `promotion-v…-run-[0-9]+-[0-9]+`; a value outside that shape publishes an artifact no cleanup pass can ever collect. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
5d56681f8c |
fix(ci): put every label guard in lockstep, forks included
Promotion needs the `release` label to start work, so this PR added it to the set of labels whose `labeled` event is not discarded. That set is stated in four places and was updated in three. `gate-concurrency-contract.bats` caught the shape of the problem and then became the first casualty of it. It asserted that the concurrency key excludes exactly ONE label from the `-label` group, by counting occurrences of `github.event.label.name != '`. A second publishing label made the count 2, turning the suite red — and with it `checks`, then `finalize`, then `e2e`, then the required `E2E Tests` status. Raising the constant to 2 would re-hardcode the same brittleness one label later and throw away the invariant the test exists for: the set of labels excluded from the `-label` suffix is exactly the set `plan` runs for. Both sets are now extracted from the workflow and compared, so the assertion holds at any number of labels. `resolve_assets` was the guard that got missed. It sits on `e2e`'s release arm, so a label event `plan` runs for and it skips for leaves `e2e` with both arms false — `finalize` is force-skipped for release PRs — while `e2e-report` still runs. On a promote PR already carrying `full-e2e`, re-applying `release` therefore superseded the green run and posted `E2E Tests` = failure, because `full-e2e` bypasses the "e2e not repeated" branch and `E2E_RESULT` is `skipped`. `unlabeled` is not a trigger, so removing the label could not clear it; only a push could. `release` must also not publish on the fork lane. e2e-fork.yaml's `resolve` decides "this run built nothing, so it is not a verdict" by asking whether `Plan build` skipped. Admitting `release` at `plan` made that false: a fork run reaches `plan`, has `build`/`finalize` gated off by its own label, succeeds having produced no pr-patch, and the fork lane fails closed on the missing artifact — red on a suite that was never going to run, on a PR whose only fault is a mislabel. `plan` and `resolve_assets` now admit `release` for same-repo PRs only, and the concurrency key carries the complementary term, so NOT(A OR B OR (C AND NOT F)) stays exactly (NOT A) AND (NOT B) AND (NOT C OR F). For a same-repo PR F is false and every one of them reduces to what it was, so none of this changes non-fork behaviour. The same shape still reaches the fork lane from a PUSH to a fork PR somebody mislabelled `release` — `build` is gated on the label, not on the event, so this predates promotion using the label. Left alone; the comment in e2e-fork.yaml now records it instead of asserting an invariant this PR would have made false. The contract enumerated `plan` and `e2e-report` by name, which is why it did not catch `resolve_assets`: a named-job loop cannot notice a job nobody thought to name. It now derives the job list from the file and requires every job that decides from a label name to decide from the same set, and pins the fork term on both halves of the complement. The file's existing properties are kept — extraction runs over executable-only lines, so a commented-out key cannot satisfy the contract, and every comparison asserts a non-empty input first, so an empty extraction fails closed under both bats and cozytest.sh. Verified by mutation, each caught: excluding a label `plan` skips for, admitting one the key does not exclude, reverting `resolve_assets`, deleting `e2e-report`'s guard, narrowing `verify-release-candidate`'s, renaming the label in one guard only, commenting out the group key, and dropping the fork term from either side alone. Deleting `plan`'s step-level fork guard is caught too, by an assertion that had to stop matching on the bare identifier once `plan`'s own `if:` named it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
f21d277eba |
fix(release): retain live promotion candidates
Protect package candidates referenced by open release PRs while pruning abandoned run-specific promotion artifacts. Document the stable-candidate lifecycle and maintenance-branch rollout requirement. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: GPT-5 <noreply@openai.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
c2e145709b |
fix(release): publish stable packages candidate
Serialize the tag-rewritten packages tree during RC promotion and pin its immutable digest in the installer. Verify the candidate against the prospective merge and the original RC artifact before creating stable names. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: GPT-5 <noreply@openai.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Aleksei Sviridkin
|
ea78d09dd0 |
feat(ci): auto-apply triage labels to issues
Issues created via the API or gh bypass the issue templates and arrive without any triage/* label; 252 of the 396 open issues carry none. Label issues on arrival (opened/reopened) and run a daily sweep as a backstop: issues already prioritised, assigned, or marked as epics get triage/accepted, the rest triage/needs-triage. A signal has to be on stale.yaml's exempt-issue-labels, because triage/accepted is itself exempt and a signal that is not would grant a permanent stale reprieve as a side effect of being labeled. That is necessary and not sufficient: lifecycle/frozen is exempt and still not a signal, because a lifecycle/* label says what the stale bot may do with an issue and nothing about whether anybody reviewed it. The sweep paces its writes and caps how many it makes per run, so a pass over a large backlog stays inside GitHub's secondary rate limits instead of failing partway through. Each run writes a job summary with the counts, the issue numbers per label, and both remainders it can leave behind, because a backlog pass logs a line per issue and nobody reads 252 of those to find out what happened. The manual dispatch defaults to dry-run, so the one entry point a human can reach writes nothing until somebody unchecks the box. The schedule is the one event that writes without being asked. Deleting a triage label does not undo a decision: an issue left with none is relabeled by the next sweep, so AGENTS.md's labeling section says to change a decision by replacing the label instead. No PR lane can exercise this workflow, so the first real run is a sweep over every open issue. A contract test pins the executable lines that bound that: the pull-request skip, the already-triaged skip, the pacing and the cap, the dry-run default, the pinned action, the token scopes, that no signal comes from the lifecycle/* namespace, and that the summary is written with the await that makes it flush. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
cf6bf2e379 |
fix(ci): stop labeled events from cancelling and re-running backports (#3569)
## What this PR does A run joins its concurrency group before any job-level `if` is evaluated, so `prepare`'s guard in `backport.yaml` is too late to help. By the time it works out that a label event is irrelevant, that run has already cancelled whatever was in flight for the same PR. Labels set with the default `GITHUB_TOKEN` start no run, which is why the in-repo labelers never showed this. Third-party GitHub Apps do start runs, and they arrive in bursts. The delivery survived, which is why this went unnoticed for so long. The old guard admitted a run on the PR's cumulative label set, so the last label of a burst, the one nothing cancels, requalified and redid the work. In the bursts on record the kill lands within seconds of the merge run starting, well before the cherry-pick, so the cost was a restart rather than a lost backport, recorded as nothing more than a cancelled run sitting next to a green one. Two changes to the concurrency key, and both are needed. A label event that requests no backport now goes into a group of its own, so it can neither cancel nor displace the run doing the work. A label event that does request one queues behind that run instead of killing it, because `backport` and `backport-previous` are separate requests and both have to finish. Splitting alone would let `backport` cancel the merge-triggered run. Queuing alone would let an unrelated label evict a pending request, since the group holds a single pending run and a newer one replaces it. `prepare`'s condition is scoped in the same commit. It read the PR's cumulative label set, so once a merged PR carried `backport`, any later unrelated label re-entered the job to redo a backport already delivered. That run used to get cancelled by the next label in the burst. After the change above it runs to completion, so the scoping has to land here rather than as a follow-up. The coupling runs the other way too, and it is the sharper half. That redundant run is exactly what redelivered the backport the burst had just killed, so narrowing the guard on its own would convert a noisy delivery into a missing one. Neither change is safe to land without the other. Both jobs also gain a `timeout-minutes` ceiling, which is a consequence of the queuing rather than housekeeping. A cancelling key disposed of a stuck run by killing it; queuing makes the next genuine request wait behind it instead, on the six-hour job default, which turns one wedged job into a six-hour hole in the release line. That ceiling reaches a stuck job and nothing else: a run can also wedge at the run level with every job already finished, and `timeout-minutes` has no run-level equivalent, so nothing in the workflow reaches that state. `docs/release.md` now tells an operator to look for a run still in flight and clear it before re-applying the label, because the retry queues behind it rather than replacing it. The ceiling bounds execution, not the wait for a runner, and thirty minutes is generous rather than calculated. Execution on run 29844315422 was six seconds for `prepare` and two and sixteen for the matrix legs, against 9m24s, 5m28s and 5m45s of runner queue for the same three jobs. That queue is a third unbounded case: GitHub's limits page says "job execution time" for the six-hour job cap while spelling out that the 35-day run cap "includes execution duration, and time spent on waiting and approval", and [orgs/community#50926](https://github.com/orgs/community/discussions/50926) shows a job sitting six hours on "Waiting for a runner to pick up this job" under `timeout-minutes: 5`. So the value is picked to leave room for a cherry-pick far larger than any on record while replacing the six-hour default, and nothing here is sized against the queue, because nothing here can be. This removes a mitigation, and the trade is worth naming on both sides. Under an unconditional cancel a run wedged at the run level cleared itself, because the next event killed whatever was sitting in the group. That is the same behaviour that killed live backports, so the mitigation is inseparable from the bug being fixed: keeping it means keeping label events that cancel a backport in flight. What it buys is a change of failure mode rather than one fewer failure. A backport lost silently, discovered weeks later as a fix missing from a release line, becomes a delay the operator can see in the run list, with the recovery written into `docs/release.md`. There is no lossless alternative here: `queue: max` removes the eviction but only by dropping the conditional cancel entirely, and the group key cannot tell a manual retry from a second genuine request. The run cited above, 31232645133, is still live and retained on purpose: it is the only observed instance of the run-level wedge, so clearing it would remove the evidence for the paragraph that describes it. Its exposure is one pull request, #3262, which carries no backport label, so nothing is queued behind it. The old condition also carried a disjunct that could never decide anything. A `labeled` payload already lists the new label in `pull_request.labels`, so `contains` was true whenever the explicit `github.event.label.name` check was, and the name check only ever agreed with a `contains` that had already matched. The restructure drops it by construction rather than by deletion: the name check is now the whole of what a `labeled` event may match on. ### Why this no longer touches `pull-requests.yaml` `main` fixed it there by splitting the group key, then refined that so a `full-e2e` label stays in the main group instead of publishing a second `E2E Tests` status on one SHA. That is the better mechanism, it neither queues nor delays, and `gate-concurrency-contract.bats` now pins `cancel-in-progress: true` in that file, so what this PR originally did would turn it red. Nothing was left to add there, so the hunk is dropped and its pin with it. The split-key idea is applied in `backport.yaml` instead, where it did not exist. One claim from the earlier description is withdrawn. It said the leftover risk was a duplicate backport PR, opened when the original had merged and its branch had been deleted before the queued run pushed. `main` has since gained a guard that lists closed PRs on the `backport-<n>-to-<target>` branch and skips the target when one of them merged. Branch deletion does not defeat it: `GET /branches/backport-3510-to-release-1.6` is 404 while the same head filter on `/pulls?state=closed` still returns the merged PR. So a redundant run only cost runner minutes, and the scoping above removes the run. ### Screenshots Not applicable, no UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. The only workflow it cites is `tags.yaml`, for the release-prep behaviour `cozystack/ccp` documents. Every `hack/` trigger it lists either names a specific file another repository mirrors or gates on, or covers moving and renaming under `hack/`. This PR appends tests to a contract file the map does not name, and moves nothing. No package, chart, values file, CRD, image reference, namespace, variant or node prerequisite in the diff. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(ci): stop `labeled` pull request events from cancelling the automatic backport run already in flight. A burst of labels from a third-party app could previously kill the backport the merge had just started, leaving the work to be redone by whichever label arrived last. ``` |
||
|
Aleksei Sviridkin
|
e0d6c6be92 |
fix(ci): stop labeled events from cancelling and re-running backports
A run joins its concurrency group before any job-level `if` is evaluated, so `prepare`'s guard cannot stop a label event from disturbing the run already backporting a PR. Labels set with the default GITHUB_TOKEN start no run, but labels set by third-party GitHub Apps do, and they arrive in bursts, so each one cancelled the backport the merge had just started. The delivery survived anyway, which is why this went unnoticed: the old guard admitted a run on the PR's cumulative labels, so the last label of a burst requalified and redid the work. In the bursts on record the kill lands within seconds, well before the cherry-pick, so the cost was a restart rather than a lost backport. Move a label event that requests no backport into a group of its own, and let one that does request a backport queue rather than cancel. Both halves are needed. `backport` and `backport-previous` are separate requests that both have to complete, so they share a group and must not cancel each other, while every other label does no work here and only has to stay out of the way. Scope `prepare` to the labels each trigger is entitled to read at the same time. Reading the cumulative label set on a `labeled` event let any later unrelated label on a merged PR still carrying `backport` re-enter the job to redo a backport already delivered. That redo is also what redelivered a killed backport, so narrowing the guard alone would turn a noisy delivery into a missing one: neither change is safe to land without the other. Bound both jobs with `timeout-minutes` for the same reason, since a queued request now waits on the run ahead of it rather than replacing it. The ceiling bounds execution and not the wait for a runner, which no job-level key reaches, so 30 is generous rather than calculated: every execution measured on this workflow has been seconds, against minutes of queue that nothing here can bound. Document the recovery path the narrowing removes: with no accidental re-run left, re-applying whichever backport label the PR already carries is how an operator retries a merge-triggered run that died. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |