admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-11-09 22:13:18 +09:00
Code Issues Packages Projects Releases Wiki Activity

chore(release): retire the backport bot, its labels and backport-audit

Browse Source 
The batch replaces the label-driven backport workflow and the audit
that gated a cut against it. The kind/backport labels, the stale
exemption for them, their contract tests and the audit's make target
go with them. labels.yml gains a backport/ category instead, with
backport/include and backport/skip, the overrides the batch reads off
a merged pull request.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This commit is contained in:
Myasnikov Daniil committed 2026-11-06 12:33:37 +00:00
1 parent 4a750a1e23
commit eaa0169ff3
10 files changed
+35 -4041
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/labels.yml
+15 -12
View File
@@ -22,6 +22,7 @@
# do-not-merge/ PR merge blockers
# security/ security-finding severity and status (Cozystack-specific)
# size/ PR size (auto-applied)
# backport/ maintainer overrides of the backport batch rules (Cozystack-specific)
#
# `aliases:` lets EndBug/label-sync rename existing labels without losing
# references on already-tagged issues and PRs.
@@ -78,18 +79,6 @@
color: 'e11d21'
description: Indicates the change introduces a breaking API or behaviour change
# kind/backport-previous has no upstream Kubernetes equivalent (k8s has no
# two-release-line model); it stays under kind/ per the file convention.
- name: kind/backport
color: 'fbca04'
description: Categorizes issue or PR as requiring a backport to the current release line
aliases: ['backport']
- name: kind/backport-previous
color: 'fbd876'
description: Categorizes issue or PR as requiring a backport to the previous release line
aliases: ['backport-previous']
# ──────────────────────────────────────────────
# priority/ — urgency
# ──────────────────────────────────────────────
@@ -411,3 +400,17 @@
- name: security/fixed
color: '0e8a16'
description: Fix released
# ──────────────────────────────────────────────
# backport/ — maintainer overrides of the backport batch rules
# ──────────────────────────────────────────────
# Read by cmd/backport-batch from the merged pull request on main;
# docs/release/backports.md describes both.
- name: backport/include
color: 'c2e0c6'
description: The backport batch takes this PR though rules 2-4 keep it out; it must still pick and pass CI
- name: backport/skip
color: 'f9d0c4'
description: The backport batch keeps this PR out although the rules take it
.github/workflows/backport.yaml
-305
View File
@@ -1,305 +0,0 @@
name: Automatic Backport
on:
pull_request_target:
types: [closed, labeled] # fires when PR is closed (merged) or labeled
# Two mechanisms here, answering two different failures.
#
# A run joins its concurrency group before any job-level `if` is evaluated, so
# `prepare`'s guard below cannot stop a label event from disturbing the run
# already backporting this PR; it only decides what the label run does after the
# damage. Labels set with the default GITHUB_TOKEN start no run, but labels set
# by third-party GitHub Apps do, and they arrive in bursts. Under a plain
# `cancel-in-progress: true` each one cancelled the backport the merge had just
# started, and nothing reports that beyond a cancelled run sitting next to a
# green one.
#
# The delivery survived anyway, because the old guard admitted a run on the PR's
# cumulative labels: the last label of a burst requalified and redid the work.
# `prepare`'s narrowing below removes that accidental safety net, so the two
# changes land together — narrowing alone would turn a noisy delivery into an
# absent one.
#
# So the group key moves a label event that is NOT a backport request aside, and
# `cancel-in-progress` stops one that IS from cancelling rather than queuing.
# Both are load-bearing. `backport` and `backport-previous` are separate requests
# that both have to complete, so they share the main group and must not cancel
# each other; every other label does no work here and only has to stay out of
# the way.
#
# That is also why this file does not simply copy pull-requests.yaml, which
# splits the key and leaves cancellation unconditional: a discarded label run
# there publishes nothing, so isolating it is the whole fix.
#
# Queuing protects a pending run from the run ahead of it and never from the one
# behind: the group holds a single pending run, and a newer one evicts it. With
# the split in place that costs nothing, and for the same reason the fan-out
# below is harmless — the only events left in this group are backport requests,
# the evictor is by definition the newer of the two, and `prepare`'s `Check which
# labels are present` step reads the cumulative label set. So the survivor covers
# a superset of what the evicted run would have delivered, for any sequence that
# only adds labels. Removing one in between is the exception and needs no cover:
# the evictor then carries less because the request was withdrawn.
#
# That is exactly what the split buys. Without it the evictor could be any label
# at all, and a run that does not qualify delivers nothing in place of the
# request it displaced.
#
# `queue: max` raises the single pending slot to a hundred and is the obvious
# thing to reach for instead of the split. Taking it means giving up the
# conditional cancel, because `queue: max` and `cancel-in-progress: true` are a
# validation error together and this key still evaluates to true on a merge.
# The trade is available and was not taken: eviction is what keeps a burst to one
# surviving run rather than a hundred queued ones, and the feature only shipped
# 2026-05-07, on a path that delivers fixes to release lines.
concurrency:
group: backport-${{ github.workflow }}-${{ github.event.pull_request.number }}${{ (github.event.action == 'labeled' && github.event.label.name != 'backport' && github.event.label.name != 'backport-previous' && github.event.label.name != 'kind/backport' && github.event.label.name != 'kind/backport-previous') && '-label' || '' }}
# Stated positively, matching the group key above. `!= 'labeled'` is equivalent
# only while `types:` holds exactly these two, and that form fails open: adding
# `unlabeled` or `reopened` there would hand those events cancellation in the
# main group, which is the behaviour this file exists to remove.
#
# The positive form removes the cancelling, not the occupying. A third type
# still lands in the main group, still takes its single pending slot, and its
# `prepare` still skips — so it can evict a genuine backport request and
# deliver nothing in its place. `types:` is load-bearing for that reason and is
# pinned in hack/release-freeze-contract.bats; widening it needs the group key
# widened with it.
cancel-in-progress: ${{ github.event.action == 'closed' }}
# Top-level token defaults to read-only. A job-level block replaces this one
# rather than adding to it, so each job restates every scope it needs.
permissions:
contents: read
jobs:
# Determine which backports are needed
prepare:
# This guard admits each action on the labels it is entitled to. `closed`
# carries no label of its own, so it reads the set the PR ended up with.
# `labeled` reads only the label just added: matching the cumulative set
# here means every later unrelated label on a merged PR that still carries
# `backport` re-enters this job to redo a backport already delivered.
#
# Scope, so this is not read as covering the whole job: the `Check which
# labels are present` step below still reads the cumulative set, so a
# `backport-previous` event on a PR also carrying `backport` fans out to
# both matrix legs. That is deliberate, and each state a re-entered leg can
# find is already accounted for: delivered is skipped by the merged-backport
# guard, still open is skipped by backport-action on the 422, and closed
# unmerged gets a fresh backport PR, which is the retry that guard
# deliberately preserves rather than an accident. What the narrowing here
# decides is whether the job RUNS, not which targets it picks.
if: |
github.event.pull_request.merged == true &&
github.event.pull_request.base.ref == 'main' &&
(
(github.event.action == 'closed' && (contains(github.event.pull_request.labels.*.name, 'backport') || contains(github.event.pull_request.labels.*.name, 'backport-previous') || contains(github.event.pull_request.labels.*.name, 'kind/backport') || contains(github.event.pull_request.labels.*.name, 'kind/backport-previous'))) ||
(github.event.action == 'labeled' && (github.event.label.name == 'backport' || github.event.label.name == 'backport-previous' || github.event.label.name == 'kind/backport' || github.event.label.name == 'kind/backport-previous'))
)
# GitHub API calls only — no docker, no repo toolchain: GitHub-hosted.
runs-on: ubuntu-latest
# Queuing gives a stuck run a way to hurt the next one that a cancelling key
# did not: the label event that used to kill it now waits behind it rather
# than clearing it.
#
# This ceiling bounds a stuck JOB, which is the case it can reach. A run can
# also wedge at the RUN level with every job already finished, and no
# job-level key expires that one; it holds the group until GitHub retires the
# run. Rare but real: run 31232645133 has been `in_progress` since
# 2026-08-08 with `prepare` and `backport` both skipped. Nothing here
# reaches that state, so docs/release.md tells an operator to look for a run
# still in flight and clear it before re-applying the label.
#
# What it bounds is EXECUTION, not the wait for a runner. GitHub's limits
# page says "job execution time" for the 6-hour job cap while spelling out
# that the 35-day run cap "includes execution duration, and time spent on
# waiting and approval" — the contrast is deliberate — and
# orgs/community#50926 shows a job sitting six hours on "Waiting for a runner
# to pick up this job" under `timeout-minutes: 5`. So queue time is a third
# unbounded case alongside the run-level wedge, not something this covers.
#
# 30 is therefore generous rather than calculated. Execution on run
# 29844315422 was 6s for `prepare` and 2s and 16s for the two matrix legs,
# against 9m24s, 5m28s and 5m45s of queue for the same three jobs. Every
# execution measured on this workflow has been seconds, and no maximum is
# quoted on purpose: an absolute maximum over a growing set is stale as soon
# as a slower run lands. The value leaves room for a cherry-pick far larger
# than anything seen while replacing the 6-hour default, and nothing here is
# sized against the queue because nothing here can be.
timeout-minutes: 30
permissions:
contents: read
outputs:
backport_current: ${{ steps.labels.outputs.backport }}
backport_previous: ${{ steps.labels.outputs.backport_previous }}
current_branch: ${{ steps.branches.outputs.current_branch }}
previous_branch: ${{ steps.branches.outputs.previous_branch }}
steps:
- name: Check which labels are present
id: labels
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const pr = context.payload.pull_request;
const labels = pr.labels.map(l => l.name);
// TRANSITIONAL: both spellings are accepted so the rename can land
// in any order relative to the org-level dosubot, which still
// applies the legacy names and is today the dominant applier on
// real backports. Once dosubot's PR labelling is switched off and
// the in-repo labeler is the only applier, the legacy arms here,
// in the `if:` gate above and in the concurrency group can all be
// deleted in one pass.
const isBackport = labels.includes('kind/backport') || labels.includes('backport');
const isBackportPrevious = labels.includes('kind/backport-previous') || labels.includes('backport-previous');
core.setOutput('backport', isBackport ? 'true' : 'false');
core.setOutput('backport_previous', isBackportPrevious ? 'true' : 'false');
console.log(`backport label: ${isBackport}, backport-previous label: ${isBackportPrevious}`);
- name: Determine target branches
id: branches
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
// Enumerate the real release-X.Y branches rather than deriving them
// from the latest published release.
//
// Since cutting an rc freezes the line (cut-prerelease.yaml),
// release-X.Y is created when the first vX.Y.0-rc.N is cut — before
// vX.Y.0 is ever published. getLatestRelease returns the newest
// published STABLE, so throughout the freeze window it still named
// the previous line: a `backport` on a fix for the release being
// stabilised would have landed on release-X.(Y-1) and missed the
// release it was written for. That window is exactly when backports
// matter most, since the frozen branch is the only way in.
//
// Enumerating branches also drops the contiguous-minor assumption:
// `previous` used to be computed as min-1, which named a
// non-existent branch whenever a minor was skipped.
const branches = await github.paginate(github.rest.repos.listBranches, {
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 100
});
const lines = branches
.map(b => /^release-(\d+)\.(\d+)$/.exec(b.name))
.filter(Boolean)
.map(m => ({ name: m[0], maj: Number(m[1]), min: Number(m[2]) }))
.sort((a, b) => (b.maj - a.maj) || (b.min - a.min));
if (lines.length === 0) {
core.setFailed('No release-X.Y branches found; cannot determine backport target.');
return;
}
const currentBranch = lines[0].name;
const previousBranch = lines.length > 1 ? lines[1].name : '';
core.setOutput('current_branch', currentBranch);
core.setOutput('previous_branch', previousBranch);
console.log(`Release lines found: ${lines.map(l => l.name).join(', ')}`);
console.log(`Current branch: ${currentBranch}, Previous branch: ${previousBranch || 'N/A'}`);
// Both targets come from the enumeration above, so they exist by
// construction — no getBranch probe needed. What CAN still fail is
// asking for a previous line when only one exists.
if (!previousBranch && '${{ steps.labels.outputs.backport_previous }}' === 'true') {
core.setFailed('backport-previous requested, but only one release line exists.');
return;
}
backport:
needs: prepare
if: |
github.event.pull_request.merged == true &&
github.event.pull_request.base.ref == 'main' &&
(needs.prepare.outputs.backport_current == 'true' || needs.prepare.outputs.backport_previous == 'true')
# git cherry-picks via backport-action — no docker, no repo toolchain: GitHub-hosted.
runs-on: ubuntu-latest
# See `prepare` above: a queued request waits on this one, so it needs a
# ceiling rather than the 6-hour default. Bounds this job's execution only;
# the wait for a runner is outside it either way.
timeout-minutes: 30
permissions:
contents: write
pull-requests: write
strategy:
matrix:
backport_type: [current, previous]
steps:
# 1. Determine target branch based on matrix
- name: Set target branch
id: target
if: |
(matrix.backport_type == 'current' && needs.prepare.outputs.backport_current == 'true') ||
(matrix.backport_type == 'previous' && needs.prepare.outputs.backport_previous == 'true')
run: |
if [ "${{ matrix.backport_type }}" == "current" ]; then
echo "branch=${{ needs.prepare.outputs.current_branch }}" >> $GITHUB_OUTPUT
echo "Target branch: ${{ needs.prepare.outputs.current_branch }}"
else
echo "branch=${{ needs.prepare.outputs.previous_branch }}" >> $GITHUB_OUTPUT
echo "Target branch: ${{ needs.prepare.outputs.previous_branch }}"
fi
# 2. Skip if this target already has a merged backport.
#
# The action's backport branch name is deterministic,
# backport-<pull_number>-to-<target_branch>. A merged PR with that head
# means an earlier run already delivered this change to the target — a
# second label event (e.g. backport-previous added after backport
# already merged) would re-enter this job and cherry-pick commits the
# target branch already has. Git reports that as an empty pick (exit 1)
# rather than a conflict, which draft_commit_conflicts below misreads
# as a real conflict and reports as a failed backport on the original
# PR. Skipping here avoids the false report; it does not touch the
# empty-pick behavior itself, which lives in the action.
- name: Check for existing merged backport
id: guard
if: steps.target.outcome == 'success'
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const targetBranch = '${{ steps.target.outputs.branch }}';
const head = `backport-${context.payload.pull_request.number}-to-${targetBranch}`;
const { data: prs } = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'closed',
head: `${context.repo.owner}:${head}`,
});
const alreadyMerged = prs.some(p => p.merged_at !== null);
core.setOutput('already_merged', alreadyMerged ? 'true' : 'false');
if (alreadyMerged) {
console.log(`Skipping backport to ${targetBranch}: ${head} already merged.`);
}
# 3. Checkout (required by backport‑action)
- name: Checkout repository
if: steps.target.outcome == 'success' && steps.guard.outputs.already_merged != 'true'
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
# 4. Create the back‑port pull request
- name: Create back‑port PR
id: backport
if: steps.target.outcome == 'success' && steps.guard.outputs.already_merged != 'true'
uses: korthout/backport-action@0193454f0c5947491d348f33a275c119f30eb736 # v3.2.1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
label_pattern: '' # don't read labels for targets
target_branches: ${{ steps.target.outputs.branch }}
merge_commits: skip
# conflict_resolution lives inside the experimental JSON input, not as a
# top-level key. Passed top-level it is silently ignored and the action
# falls back to the experimental default ("fail"), which on a conflict
# creates no PR and only comments. Nesting it here makes a conflicting
# backport open a draft PR with the conflict committed for manual fixup.
experimental: |
{
"conflict_resolution": "draft_commit_conflicts"
}
.github/workflows/stale.yaml
+1 -2
View File
@@ -39,8 +39,7 @@ jobs:
security/triage-needed,security/in-progress,
triage/accepted,kind/regression
exempt-pr-labels: >-
lifecycle/frozen,do-not-merge/hold,
kind/backport,kind/backport-previous
lifecycle/frozen,do-not-merge/hold
days-before-stale: 60
days-before-close: 14
operations-per-run: 100
.gitignore
+1 -1
View File
@@ -11,7 +11,7 @@ _repos
# root-level binary waiting to happen — each ~80 MB, and nothing else here
# matched them, leaving `git add .` one keystroke from committing one.
/api-gate
/backport-audit
/backport-batch
/backup-controller
/backupstrategy-controller
/check-readiness
Makefile
+6 -6
View File
@@ -1,4 +1,4 @@
.PHONY: manifests assets prepare-env prepare-env-container unit-tests helm-unit-tests bats-unit-tests bats-unit-files-check bats-posix-compat-tests print-bats-unit-files print-bats-posix-compat-files print-bats-jobs rd-presets-check migrations-target-check test test-controllers test-controllers-envtest test-backport-audit preflight
.PHONY: manifests assets prepare-env prepare-env-container unit-tests helm-unit-tests bats-unit-tests bats-unit-files-check bats-posix-compat-tests print-bats-unit-files print-bats-posix-compat-files print-bats-jobs rd-presets-check migrations-target-check test test-controllers test-controllers-envtest test-backport-batch preflight
include hack/common-envs.mk
@@ -119,7 +119,7 @@ test:
make -C packages/core/testing apply
make -C packages/core/testing e2e
unit-tests: helm-unit-tests bats-unit-tests bats-posix-compat-tests go-unit-tests go-module-tests rd-presets-check test-check-readiness test-backport-audit migrations-target-check
unit-tests: helm-unit-tests bats-unit-tests bats-posix-compat-tests go-unit-tests go-module-tests rd-presets-check test-check-readiness test-backport-batch migrations-target-check
helm-unit-tests:
hack/helm-unit-tests.sh
@@ -194,9 +194,9 @@ test-controllers-envtest: $(SETUP_ENVTEST)
test-check-readiness:
go test ./test/check-readiness/ -count=1
# ./cmd/... is excluded from go-unit-tests, so the audit needs its own target.
test-backport-audit:
go test ./cmd/backport-audit/ -count=1
# ./cmd/... is excluded from go-unit-tests, so the batch tool needs its own target.
test-backport-batch:
go test ./cmd/backport-batch/... -count=1
# Unit discovery is one level deep. Live-cluster files need an invocation in
# packages/core/testing; hack/bats-runner-coverage.bats checks that boundary.
@@ -294,7 +294,7 @@ bats-posix-compat-tests:
# own too; hack/unit-lane-no-cluster.bats fails when one is missing.
NO_CLUSTER_TARGETS := unit-tests test-controllers helm-unit-tests bats-unit-tests \
bats-posix-compat-tests go-unit-tests go-module-tests rd-presets-check \
test-check-readiness test-backport-audit migrations-target-check
test-check-readiness test-backport-batch migrations-target-check
$(NO_CLUSTER_TARGETS): export KUBECONFIG := /dev/null
$(NO_CLUSTER_TARGETS): export KUBERNETES_SERVICE_HOST :=
cmd/backport-audit/README.md
-226
View File
@@ -1,226 +0,0 @@
# backport-audit
Answers one question, before a release or an rc is cut: did everything labelled for backport actually land on the release branch? It prints the URLs of the ones that did not, and exits non-zero, so it can gate the cut.
The manual alternative is `gh pr list --search "is:merged label:kind/backport"`, which lists every PR ever labelled and says nothing about whether the change reached the branch. Answering that by hand means reading the branch history per PR, which is where backports quietly go missing.
```console
$ go run ./cmd/backport-audit release-1.5
=== release-1.5 === 43 candidate PRs: MISSING=2 pending=3 dropped=5 backported=33
MISSING -- labelled for this line, no trace of it here (2):
https://github.com/cozystack/cozystack/pull/3449
#3449 ci(release): move release validation to rc time — e2e lane, changelog button, promote gate, docs at promote
label=kind/backport-previous author=myasnikovdaniil merged=2026-07-29 -- no backport PR, nothing on branch
```
The URL is always the **original PR on main** — the thing you decide about. The evidence line names the backport PR when one exists. A labelled PR already on the branch is counted in the header and never listed, so silence under the outstanding headings means done. Two more sections, described in [Backports the labels do not account for](#backports-the-labels-do-not-account-for), start from the backport PRs on the branch instead of from labels: `DUPLICATE`, which is outstanding, and `UNLABELLED`, which is informational.
## Usage
```bash
go run ./cmd/backport-audit # the two newest release lines
go run ./cmd/backport-audit release-1.5 # one line
go run ./cmd/backport-audit release-1.6 release-1.5 --json
```
Requires `git` and an authenticated `gh` on PATH. The audit does not modify GitHub or working-tree files. Unless `--no-fetch` is set, it updates local Git refs and objects while fetching.
| Flag | Meaning |
|------|---------|
| `--remote NAME` | Git remote holding the release branches (default `origin`) |
| `--limit N` | Max merged PRs scanned per label spelling (default 1000, which is also the ceiling — see [Exit code](#exit-code)) |
| `--no-fetch` | Skip `git fetch`; both main and release refs must be current so shared history stays out of backport evidence |
| `--json` | Machine-readable output |
| `--no-color` | Disable color (auto-disabled on a non-TTY and under `--json`) |
Arguments and flags may be given in any order.
## Exit code
`0` when nothing is outstanding, `1` when something is, `2` when the audit could not be completed. Outstanding means a `MISSING`, `partial`, `unverified`, `pending` or `dropped` verdict, or any `DUPLICATE` entry; `UNLABELLED` entries never move the exit code. That is the point of the tool, so it holds in `--json` mode too:
```bash
go build ./cmd/backport-audit
./backport-audit release-1.5
case $? in
0) echo "clean" ;;
1) echo "do not cut yet" ;;
*) echo "the audit itself failed; no answer yet" ;;
esac
```
Use the built binary (`go build` as above, or `go install ./cmd/backport-audit`) wherever the exit code matters. `go run` does not preserve it: any non-zero exit of the program comes back as `go run`'s own `1`, after an `exit status N` line on stderr, so under `go run` an audit that could not be completed looks exactly like one that found outstanding work. `go run` is fine for reading the report.
Exit `2` covers the cases where an answer cannot be trusted rather than merely being bad news, and a saturated listing is one of them: `gh` truncates at `--limit` in silence, and a truncated list does not make the audit partial, it makes it wrong — the PRs past the cut are reported nowhere and the exit code says clean. Each of the three listings is checked against its cap and fails instead, naming the cap to raise. The label listings go through GitHub search, which never returns more than 1000 results however far it is paginated, so they are checked against the lower of `--limit` and 1000: a `--limit` above 1000 cannot make a listing cut at 1000 pass for complete. A git read that a verdict depends on is the other case: when one fails, a merge commit missing from the local repository for instance, the audit stops with `2` instead of judging the PR without it.
## Statuses
| Status | Meaning | Action |
|--------|---------|--------|
| `in-branch` | The PR merged before the branch was cut, so its change is already there | none, no backport was ever needed |
| `backported` | Every commit of the PR is on the branch, or, for a PR of a single commit, a backport PR merged | none |
| `confirmed` | Would be `partial` or `unverified`, but a maintainer attested the backport complete on the merged backport PR; the report names who and links the comment | none |
| `partial` | Some of the PR's commits are on the branch and some are not, listed with SHA and subject — a merged bot conflict draft, which stops at the first commit that did not apply, or a backport squashed by hand | backport the missing commits, or [attest the backport complete](#attesting-a-backport-complete) |
| `unverified` | A backport PR merged, but none of the PR's several commits is on the branch under its own SHA, `-x` reference or subject — typically a backport squashed or reworded by hand | check it, then attest it complete or backport what is missing |
| `pending` | A backport PR exists and is still open, including the drafts the bot opens with the conflict committed | merge it, or finish the draft |
| `dropped` | A backport PR was closed unmerged, reported with whatever reason someone left on it | none if the reason still holds |
| `MISSING` | No backport PR ever existed and nothing on the branch matches | cherry-pick it by hand, or drop it deliberately |
`dropped` is usually healthy — a maintainer deciding a fix does not apply to that line, e.g. because the feature it repairs never shipped there. The recorded reason is printed so the next release does not re-open the same investigation. A `dropped` entry reading `no reason recorded` is the one that needs a human.
## Backports the labels do not account for
The verdicts start from labels, so on their own they cannot see a backport of a PR nobody labelled. Two further sections start from the other side: every PR on the release branch, and the originals it names by the bot's head branch or in a `Backport of` phrase (see [How landing is established](#how-landing-is-established)). The header carries their counts after a `|`, e.g. `backported=42 | DUPLICATE=3 unlabelled=13`.
`UNLABELLED` lists each original that backport PRs on the branch claim although it is not a candidate for the branch, with every backport PR claiming it and that PR's state. Each entry also says what those PRs amount to: `backport PR merged` once one of them merged, `claimed here, not merged` while the only live one is still open, and `claimed here, closed unmerged` when every one was closed. A merged claim is only a merged PR, since an unlabelled original is not checked commit by commit; it reads `backport PR merged, confirmed by @login` when the PR carries the [attestation marker](#attesting-a-backport-complete). Most are hand backports of PRs that never carried a label, merged on purpose. An original that does carry a backport request, one that resolved to a different line at merge time, is listed with its labels. The section is informational and never moves the exit code: the gate answers whether everything labelled landed, and an unlabelled backport can only add to a branch, never leave a labelled change off it.
`DUPLICATE` lists each original claimed by more than one live backport PR, labelled or not:
| Kind | Meaning | Action |
|------|---------|--------|
| `open-twice` | Two or more backport PRs are open for it — typically the bot's conflict draft next to a hand backport, or a fork PR next to its reopening from a branch in this repository | merge one, close the rest |
| `open-after-merge` | A backport already merged and another one is still open | close the leftover, or merge it if it is the second half of a split backport |
Both kinds make the audit exit `1`. Neither reaches a verdict, which judges the commits on the branch and names at most one open backport PR, and in both a human has to decide what happens to the open PR before the cut. A closed PR next to an open one is the normal shape of a redone backport and is not flagged, and neither are two merged ones, which are history the branch already carries.
```console
DUPLICATE -- one original, more than one live backport PR (3):
https://github.com/cozystack/cozystack/pull/3936
#3936 fix(rabbitmq): right-size the default resources preset to s1.nano
label=kind/backport -- a backport PR still open after another one merged
backport #4372 OPEN (draft) https://github.com/cozystack/cozystack/pull/4372
backport #4393 MERGED https://github.com/cozystack/cozystack/pull/4393
UNLABELLED -- backport PRs here for originals not labelled for this line (13, informational):
https://github.com/cozystack/cozystack/pull/4280
#4280 fix(kafka): keep the pre-delete hook's credentials as long as its Job
backport PR merged
backport #4421 CLOSED https://github.com/cozystack/cozystack/pull/4421
backport #4456 MERGED https://github.com/cozystack/cozystack/pull/4456
```
The titles of unlabelled originals are not in any listing the audit already makes, so they come from one GraphQL request covering the whole run. If it fails, the report goes out without them and says so on stderr; the URLs are derived locally and the verdicts and exit code do not depend on it.
## How a PR is matched to a release line
A `kind/backport` label does not name a branch. The audit selects merged PRs that currently carry a backport label, then assigns each to the release line selected at its merge time: `kind/backport` means the current line, and `kind/backport-previous` means the previous line. A label added after merge still makes the PR a candidate. [`backport.yaml`](../../.github/workflows/backport.yaml) also runs when a label is added and selects its target at that time, so a late label can make the audit and the bot disagree; see [Limits](#limits).
What "the current line" means changed once, so the audit carries both rules and picks by the PR's merge date:
| PR merged | rule `backport.yaml` was running | a line exists from |
|-----------|----------------------------------|--------------------|
| before 2026-08-03 | `getLatestRelease` | its first non-prerelease release publishing |
| on or after 2026-08-03 | the two newest existing `release-X.Y` branches | its `release-X.Y` branch being pushed |
The cutover is a single moment because both halves landed in one push: 53a7fc4dc made cutting an rc freeze the line into `release-X.Y`, and 6ff5b98d5 pointed the bot at the branch list. Before that a release branch was created at promote time, so the newest branch and the newest published stable named the same line and the two rules could not disagree. After it they disagree for the length of every freeze window — `release-X.Y` is created when the first `vX.Y.0-rc.N` is cut, potentially weeks before `vX.Y.0` publishes, and everything labelled in between belongs to the line being stabilised. That window is exactly when a backport has to reach the branch, because the frozen branch is the only way into the release.
The audit estimates branch opening from the earliest `vX.Y.0-rc.N` tagged commit dated on or after the cutover. It ignores earlier tags individually; a line with no remaining tag falls back to its first stable release date. [`cut-prerelease.yaml`](../../.github/workflows/cut-prerelease.yaml) creates the branch at the tagged commit, but the commit timestamp is not the branch-push timestamp. PRs merged in that interval can be assigned to the new line before the bot could target it. The rc release's `publishedAt` is later still, after the build finishes.
Lines are then ranked by version and never by when they opened, matching the numeric descending sort the workflow applies to its branch list. The two orders come apart as soon as a freeze overlaps the previous line's stabilisation, and version order is also what keeps `release-1.10` above `release-1.9`.
The labels themselves were namespaced under `kind/` in 7b71053a0, which renamed them in place via label-sync aliases, so every historical PR reads back under the new name. The bare `backport` and `backport-previous` are still queried alongside them, because the org-level dosubot goes on applying the old spelling to new PRs — the same transitional allowance [`backport.yaml`](../../.github/workflows/backport.yaml) makes, and it retires at the same time. A PR found under either spelling is reported under the `kind/` one.
## How landing is established
Evidence is weighed per commit, because a commit is what a backport can lose:
1. **Reachability.** The PR's merge commit is reachable from the release branch, i.e. it merged before the branch was cut (or `main` was later merged in). Nothing was ever needed: `in-branch`.
2. **A linked backport PR.** Found by the bot's head branch `backport-<N>-to-release-X.Y`, or by a `Backport of #N` reference in the body, which is what a hand-written backport carries. It never outranks the commits (item 3): a merged backport PR with one of the PR's commits missing is `partial`, because the bot's conflict drafts stop at the first commit that does not apply and drop the rest, so a merged draft looks finished at PR level. `MERGED` settles it alone only for a PR of a single commit — including on a `--no-fetch` run whose local refs predate the backport's merge. For a PR of several commits, none of them found and a merged backport PR is `unverified`. `OPEN` is `pending`; `CLOSED` is `dropped`.
A backport carrying several changes links to every original its `Backport of` phrase names: a list (`Backport of #3938 and #4280`, `Backport of #1, #2, and #3`), and the `Backport of #4253 to release-1.6, together with #3460` form used for a dependency pulled along. Only that phrase is read, so the issue a backport fixes or a CI run it cites further on is never taken for an original. A list counts in full only when it visibly ends — at the end of the line or the sentence, or where the `to release-X.Y` clause starts. One that runs on into anything else (`Backport of #10, #20 is not included`) may be saying something about its later items, so only its first reference counts. A reference qualified with this repository (`cozystack/cozystack#N`, any case) counts like a bare `#N`, and one qualified with any other repository is skipped, because reading `other/repo#20` as local #20 would let an unrelated backport vouch for whichever local PR carries that number.
3. **The branch's own history, commit by commit.** Every commit the PR contributed has to be on the branch; merge commits and commits that change nothing are skipped, since the backport bot drops both. A commit counts when it is reachable, when an `-x` cherry-pick reference names it, or when a commit with its subject is there. Subject and `-x` evidence comes only from commits on the release branch that are not reachable from `<remote>/main`; shared history still counts for direct reachability. Subjects are matched one to one: two commits with the same subject, `fix tests` say, need two branch commits of that name, and a branch commit whose `-x` reference names any other commit than the PR's merge commit is evidence only through that reference. Nothing weaker is taken: not an `-x` reference to the PR's merge commit, which survives a cherry-pick later amended to drop a commit, and not matching lines, which an unrelated line of the same text satisfies. A wrong `backported` is the one answer a release gate must not give, while a false alarm costs a look and can be settled by an attestation. All commits is `backported`, some is `partial`. This is also what catches a hand backport nobody linked.
### Attesting a backport complete
Some complete backports cannot be shown commit by commit, most often a fork contributor's backport squashed into one commit that carries only the first commit's subject. A maintainer who has checked such a backport can say so on the merged backport PR, with a comment that consists of this single line and nothing else:
```text
backport-audit: complete
```
That turns the candidate's `partial` or `unverified` into `confirmed`, which does not fail the gate; the report names the author, links the comment, and keeps listing the commits nothing on the branch names.
The marker is an explicit attestation by whoever wrote it, never something the audit infers. It counts only when it is the whole comment: exactly `backport-audit: complete`, in that case, starting at the beginning of its line, with nothing but blank lines before it and nothing but whitespace after it. Any other text makes the comment something other than an attestation, so a sentence such as "do not post backport-audit: complete yet", a quote or a code block never counts, and neither does the marker with an explanation next to it; post the explanation as a separate comment. It counts only on a backport PR that merged, since an open or closed one put nothing on the branch to vouch for. And it counts only from someone with a hand in the repository: the comment's GitHub author association has to be `OWNER`, `MEMBER` or `COLLABORATOR`, and comments from automation are ignored whatever their association — logins ending in `[bot]`, and the review and dependency bots gh reports without that suffix, such as `coderabbitai`, `dependabot`, `renovate` and `copilot`. A review bot quoting the marker, or a contributor vouching for their own backport, confirms nothing.
## Machine-readable output
```bash
# URLs of what never landed, ready to paste into a tracking issue
go run ./cmd/backport-audit --json release-1.5 | jq -r '.[].candidates[] | select(.status=="MISSING") | .url'
# every open backport PR the audit links, labelled or not, to merge or close, across lines
go run ./cmd/backport-audit --json release-1.6 release-1.5 \
| jq -r '[.[] | (.candidates[], .unlabelled[]) | .backport_prs[] | select(.state=="OPEN") | .url] | unique[]'
# dropped items with the reason someone recorded
go run ./cmd/backport-audit --json release-1.4 \
| jq -r '.[].candidates[] | select(.status=="dropped") | "#\(.number)\t\(.reason)"'
# commits a partial backport still lacks, one per line
go run ./cmd/backport-audit --json release-1.6 \
| jq -r '.[].candidates[] | select(.status=="partial") | "#\(.number)\t\(.missing_commits[] | "\(.oid[0:9]) \(.subject)")"'
# open backport PRs competing for one original, to settle before the cut
go run ./cmd/backport-audit --json release-1.6 \
| jq -r '.[].duplicates[] | "#\(.number) \(.kind): \([.backport_prs[] | select(.state=="OPEN") | .url] | join(" "))"'
```
Output is an object keyed by branch, each holding three arrays: `candidates`, `unlabelled` and `duplicates`, each always present and possibly empty. `partial`, `unverified` and `confirmed` candidates carry `missing_commits`, each an `oid` and a `subject`, and a `confirmed` one also carries `confirmation`: `by`, `url` and `backport_pr`. An `unlabelled` record carries the same `confirmation` when its merged backport PR has the marker. A `candidates` record:
```json
{
"number": 3963,
"title": "fix(clickhouse): scheme the backup S3_ENDPOINT on the system-bucket flow",
"url": "https://github.com/cozystack/cozystack/pull/3963",
"author": "androndo",
"mergedAt": "2026-08-28T11:05:15Z",
"label": "kind/backport",
"backport_prs": [{"number": 3987, "state": "OPEN", "url": "..."}],
"status": "pending",
"evidence": "backport PR #3987 open"
}
```
An `unlabelled` record, where `labels` lists the backport requests the original carries for other lines and is empty when it carries none:
```json
{
"number": 4280,
"title": "fix(kafka): keep the pre-delete hook's credentials as long as its Job",
"url": "https://github.com/cozystack/cozystack/pull/4280",
"labels": [],
"backport_prs": [
{"number": 4421, "state": "CLOSED", "url": "..."},
{"number": 4456, "state": "MERGED", "url": "..."}
]
}
```
A `duplicates` record, where `label` is the request the original was audited under on this line and is empty when it is not a candidate here:
```json
{
"number": 4254,
"title": "feat(kubevirt): expose migration configuration through platform values",
"url": "https://github.com/cozystack/cozystack/pull/4254",
"label": "kind/backport",
"kind": "open-twice",
"backport_prs": [
{"number": 4322, "state": "OPEN", "url": "...", "draft": true},
{"number": 4339, "state": "OPEN", "url": "..."},
{"number": 4402, "state": "OPEN", "url": "..."}
]
}
```
## Limits
A `MISSING`, `partial` or `unverified` verdict is a prompt to check, not proof of absence. A hand backport squashed or reworded without `-x` reads as `partial` or `unverified` until someone attests it complete, and as `MISSING` when nothing links it to its original; confirm at diff level before redoing the work. And the audit reports *that* something is missing, never *why* — a run of `MISSING` entries clustered in time usually means the bot itself was failing during that window, which is worth checking before cherry-picking them one by one.
The audit reads labels as they stand when it runs. A label added afterwards is seen only by the next run, and until then its PR is either absent or, if someone already backported it, listed as `UNLABELLED` rather than audited, so run it right before the cut rather than once at the start of the day.
A label is always read against the release lines as they stood when the PR merged, however much later the label was added, because nothing the audit lists records when that happened. A PR merged on 2026-07-01 and labelled `kind/backport` a month later, after `v1.6.0` published, is audited as a `release-1.5` candidate. [`backport.yaml`](../../.github/workflows/backport.yaml) runs again when the label is added and resolves its targets at that moment, so it opens the backport against `release-1.6`. When the line has moved on between the merge and the label, the two therefore disagree: the audit expects the change on the merge-time line and reports it `MISSING` there unless it arrived some other way, and lists the bot's backport under `UNLABELLED` on the newer one. `TestCandidateLabel` pins the audit's side of this.
A backport PR that names its original neither by the bot's head branch nor in a `Backport of` phrase links to nothing, so it is invisible to both cross-checks: it is not listed as `UNLABELLED` and does not count towards a `DUPLICATE`. For a labelled original the branch history can still prove the change landed, through an identical commit subject or an `-x` cherry-pick reference. For an unlabelled one those are not read at all, so a backport of an unlabelled PR that only a cherry-pick trailer points to, or nothing does, appears nowhere in the report.
cmd/backport-audit/main.go
-1569
View File
File diff suppressed because it is too large. Load diff
cmd/backport-audit/main_test.go
-1496
View File
File diff suppressed because it is too large. Load diff
hack/label-workflows-contract.bats
+1 -1
View File
@@ -214,7 +214,7 @@ assert_labels_defined() {
count="$(stale_exempt_list exempt-issue-labels | wc -l | tr -d ' ')"
[ "$count" -eq 12 ]
count="$(stale_exempt_list exempt-pr-labels | wc -l | tr -d ' ')"
[ "$count" -eq 4 ]
[ "$count" -eq 2 ]
# The renewal path: frozen exempts both; an accepted issue or a held PR is
# deliberate, not stale.
hack/release-freeze-contract.bats
+11 -423
View File
@@ -1,28 +1,22 @@
#!/usr/bin/env bats
# Contract for the rc freeze, the backport target resolution that depends on it,
# the triggering and concurrency rules that decide whether a backport runs at
# all, and the agreement between those rules and cmd/backport-audit, which
# gates a cut by reproducing them. Like promote-gate-contract.bats, these tests pin executable/structural
# workflow lines rather than prose: a commented-out gate or a guard demoted to a
# comment must never satisfy the contract.
# Contract for the rc freeze and the release-line build it dispatches. Like
# promote-gate-contract.bats, these tests pin executable/structural workflow
# lines rather than prose: a commented-out gate or a guard demoted to a comment
# must never satisfy the contract.
#
# Neither workflow can be exercised by a PR lane. cut-prerelease.yaml runs only
# on workflow_dispatch from main or a release line, and its whole purpose is a
# one-way, write-once side effect — the first vX.Y.0-rc.N freezes the line, and
# there is no dry-run mode and no undo short of deleting the branch by hand.
# backport.yaml runs only on a merged main-targeted PR. So every invariant below
# is one that first reports a slip at a real release, on the one commit nobody
# wants to be debugging: getting the freeze condition wrong re-opens a frozen
# line silently, and getting the comparator wrong aims every backport at the
# wrong branch. Cheap to pin here, expensive to discover there.
# No PR lane exercises them. cut-prerelease.yaml runs only on workflow_dispatch
# from main or a release line, and its whole purpose is a one-way, write-once
# side effect — the first vX.Y.0-rc.N freezes the line, and there is no dry-run
# mode and no undo short of deleting the branch by hand. So every invariant
# below is one that first reports a slip at a real release, on the one commit
# nobody wants to be debugging: getting the freeze condition wrong re-opens a
# frozen line silently. Cheap to pin here, expensive to discover there.
load test_helper
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
CUT="$REPO_ROOT/.github/workflows/cut-prerelease.yaml"
BACKPORT="$REPO_ROOT/.github/workflows/backport.yaml"
AUDIT="$REPO_ROOT/cmd/backport-audit/main.go"
# Strip YAML comments. POSIX `grep` only: the unit-test runner has no ripgrep.
# grep exits 1 when nothing is selected (legitimate for an all-comment block)
@@ -44,24 +38,6 @@ code_lines() {
[ "$rc" -le 1 ]
}
# Strip JavaScript comments. backport.yaml's logic lives in a github-script
# `script: |` block, where comments start with `//` and survive code_lines
# untouched — and that block's comments name the very APIs these tests assert
# are gone, so filtering on `#` alone would let a comment satisfy a pin.
script_lines() {
local rc=0
grep -v '^[[:space:]]*//' || rc=$?
[ "$rc" -le 1 ]
}
# Body of one job, up to the next top-level job key.
job_block() {
awk -v job=" $1:" '
$0 == job { inside = 1; next }
/^ [a-z0-9_-]+:$/ { inside = 0 }
inside' "$2"
}
# Body of one `- name:` step, up to the next step at the same indent.
step_block() {
awk -v want=" - name: $1" '
@@ -78,7 +54,6 @@ step_line() {
@test "workflows under contract exist" {
[ -f "$CUT" ]
[ -f "$BACKPORT" ]
}
# ── the freeze condition ─────────────────────────────────────────────────────
@@ -274,390 +249,3 @@ step_line() {
[ -n "$guard" ] && [ -n "$checkout" ]
[ "$guard" -lt "$checkout" ]
}
# ── backport target resolution ───────────────────────────────────────────────
# Both assertions below run over script_lines: this logic sits in a
# github-script block whose comments discuss getLatestRelease and min-1 by name.
@test "backport targets come from enumerated branches, not the latest release" {
block="$(job_block prepare "$BACKPORT")"
[ -n "$block" ]
# Enumerate real branches. getLatestRelease returns the newest published
# STABLE, which throughout a freeze window still names the PREVIOUS line — so
# a `backport` on a fix for the release being stabilised would land on the
# wrong branch and miss the release it was written for.
printf '%s\n' "$block" | script_lines | grep -qF 'github.paginate(github.rest.repos.listBranches'
count="$(printf '%s\n' "$block" | script_lines | grep -cF 'getLatestRelease' || true)"
[ "${count:-0}" -eq 0 ]
# Only branches matching release-<major>.<minor> qualify, so per-release
# staging branches (release-1.6.1) and rc branches never become targets.
printf '%s\n' "$block" | script_lines | grep -qF 'release-(\d+)\.(\d+)$'
# previous is the second-newest EXISTING line, never an arithmetic guess. The
# old min-1 derivation named a non-existent branch whenever a minor was
# skipped.
printf '%s\n' "$block" | script_lines | grep -qF 'lines.length > 1 ? lines[1].name'
count="$(printf '%s\n' "$block" | script_lines | grep -cE 'parseInt\(min\)[[:space:]]*-[[:space:]]*1|min[[:space:]]*-[[:space:]]*1' || true)"
[ "${count:-0}" -eq 0 ]
# Asking for a previous line when only one exists must fail, not silently
# backport nowhere.
printf '%s\n' "$block" | script_lines | grep -qF 'core.setFailed'
}
@test "release lines sort numerically descending so 1.10 outranks 1.9" {
block="$(job_block prepare "$BACKPORT")"
[ -n "$block" ]
# Numeric descending on major then minor is the ONLY reason release-1.10
# ranks above release-1.9. A slip to the default lexicographic sort compares
# the names as strings, puts "release-1.9" first, and aims every backport at
# a stale line — with nothing going red.
printf '%s\n' "$block" | script_lines | grep -qF '.sort((a, b) => (b.maj - a.maj) || (b.min - a.min))'
# The comparands must be numbers, not the captured strings.
printf '%s\n' "$block" | script_lines | grep -qF 'maj: Number(m[1]), min: Number(m[2])'
# And the head of that sort is what `backport` uses.
printf '%s\n' "$block" | script_lines | grep -qF 'lines[0].name'
}
# ── skip a target whose backport already merged ─────────────────────────────
# A second label event on a PR whose first backport already merged (e.g.
# backport-previous added after backport merged) re-enters this job for the
# already-delivered target. The cherry-pick is then empty, which the action's
# draft_commit_conflicts handling misreads as a real conflict and reports as a
# failed backport on the original PR — this guard skips the re-run instead.
@test "the guard checks merged, not just closed, PRs on the deterministic branch" {
block="$(job_block backport "$BACKPORT")"
[ -n "$block" ]
# The branch name the action itself would have used. Checking anything else
# answers a different question than "did THIS backport already land".
printf '%s\n' "$block" | script_lines | grep -qF 'backport-${context.payload.pull_request.number}-to-${targetBranch}'
# Closed alone is not enough: a closed-without-merge PR is a stale or
# abandoned attempt, not proof the change reached the target. Loosening this
# to any closed PR would skip a real retry after a conflict was closed
# unmerged.
printf '%s\n' "$block" | script_lines | grep -qF "state: 'closed'"
printf '%s\n' "$block" | script_lines | grep -qF 'p.merged_at !== null'
}
@test "both the checkout and the backport-action step are gated on the guard" {
block="$(job_block backport "$BACKPORT")"
[ -n "$block" ]
# Losing the gate on either step still runs the action against a target that
# already has the change, reproducing the false comment the guard exists to
# prevent.
count="$(printf '%s\n' "$block" | code_lines | grep -cF "steps.guard.outputs.already_merged != 'true'" || true)"
[ "${count:-0}" -eq 2 ]
}
@test "the guard runs before the checkout and the backport-action step" {
guard="$(step_line 'Check for existing merged backport' "$BACKPORT")"
checkout="$(step_line 'Checkout repository' "$BACKPORT")"
create="$(step_line 'Create back‑port PR' "$BACKPORT")"
[ -n "$guard" ] && [ -n "$checkout" ] && [ -n "$create" ]
[ "$guard" -lt "$checkout" ]
[ "$guard" -lt "$create" ]
}
# Mirrors release-changelog-behaviour.bats's convention for a github-script
# snippet: extract the decision expression verbatim in shape and execute it
# under node, because a grep for "merged_at" cannot tell an `.some()` from an
# `.every()` or a flipped comparison. Two cases are the ones that matter and
# are NOT interchangeable: the first-ever run (branch never existed, so the
# API returns an empty list) and a real retry (someone closed a conflicting
# backport PR without merging it). Both must NOT skip, or either the very
# first backport attempt or a legitimate re-creation after a manual close
# never runs.
@test "the guard's merged decision is correct on empty, closed-unmerged and closed-merged lists" {
command -v node >/dev/null || { echo "node unavailable; skipping"; return 0; }
tmp="$(mktemp -d)"
cat > "$tmp/guard.js" <<'JS'
function alreadyMerged(prs) {
return prs.some(p => p.merged_at !== null);
}
const cases = {
'first-run-empty': [],
'closed-unmerged': [{ merged_at: null }],
'closed-merged': [{ merged_at: '2026-01-01T00:00:00Z' }],
};
const lines = Object.entries(cases).map(([name, prs]) => name + '=' + alreadyMerged(prs));
require('fs').writeFileSync(1, lines.join('\n') + '\n');
JS
node "$tmp/guard.js" > "$tmp/out.txt" 2>"$tmp/err.txt" || {
echo "node failed:" >&2; cat "$tmp/err.txt" >&2; exit 1; }
want="first-run-empty=false
closed-unmerged=false
closed-merged=true"
got="$(cat "$tmp/out.txt")"
[ "$got" = "$want" ] || {
echo "guard decision table is wrong." >&2
echo "want:" >&2; printf '%s\n' "$want" | sed 's/^/ /' >&2
echo "got:" >&2; printf '%s\n' "$got" | sed 's/^/ /' >&2
exit 1
}
# The mirror must match the real guard's exact decision expression, not just
# the presence of "merged_at" somewhere in the step. Wrapped like the other
# tests in this file's final assertion: a bare failing command here, under a
# trap-installed EXIT handler, has been observed to abort the whole bats run
# instead of just this test on macOS.
block="$(job_block backport "$BACKPORT")"
[ -n "$block" ]
printf '%s\n' "$block" | script_lines | grep -qF 'prs.some(p => p.merged_at !== null)' || {
echo "backport.yaml's guard no longer uses prs.some(p => p.merged_at !== null)." >&2; exit 1; }
rm -rf "$tmp"
}
# ── backport concurrency ─────────────────────────────────────────────────────
# A run joins its concurrency group before `prepare`'s guard is evaluated, so
# nothing in that guard can keep a label event from disturbing the run already
# backporting the PR. The group key and `cancel-in-progress` close different
# halves of that and are pinned separately: either one alone still leaves a
# live failure, and neither reads load-bearing on its own.
@test "the trigger surface the concurrency key is built for does not widen" {
# Both halves of the key are written against one trigger delivering `closed` or
# `labeled`, and neither degrades safely if anything else arrives. The group key
# routes only `labeled` aside, so a new event lands in the main group;
# `cancel-in-progress` is positive, so it does not cancel there; and `prepare`
# does not qualify it. The result occupies the single pending slot, evicts a
# genuine backport request, and delivers nothing.
#
# Pinned as the whole surface rather than just the types list, because a SIBLING
# trigger reaches the same harm without touching `types:` at all: anything
# carrying a `pull_request` payload resolves the PR number and so lands in the
# same per-PR group. `pull_request_review` is the realistic one.
#
# Deliberately strict: this also fails on a widening that would be harmless,
# `workflow_dispatch` for instance, whose payload carries no `pull_request` so
# the key degenerates to a shared group holding no genuine request. Separating
# the harmless case needs a list of which triggers carry that payload, and that
# list drifts. A red here means read the block above, not that the pin is wrong.
triggers="$(awk '/^on:/{inside=1;next} /^[a-z]/{inside=0} inside && /^ [a-z_]+:/{print $1}' "$BACKPORT")"
[ -n "$triggers" ]
[ "$(printf '%s\n' "$triggers" | wc -l | tr -d ' ')" -eq 1 ]
printf '%s\n' "$triggers" | grep -qF 'pull_request_target:'
printf '%s\n' "$(code_lines < "$BACKPORT")" | grep -qF 'types: [closed, labeled]'
}
@test "a label event that requests no backport gets its own concurrency group" {
line="$(code_lines < "$BACKPORT" | grep '^ group: backport-')"
[ -n "$line" ]
# The per-PR discriminator, first: without it every PR's backport shares one
# group, and since a merge still cancels in that group, merging one PR kills
# another's in-flight backport. The assertions below all describe how a label
# event is separated from a merge WITHIN one PR, and every one of them stays
# green with the PR number deleted, so the widest failure of the three is the
# one nothing else here would notice.
printf '%s\n' "$line" | grep -qF 'backport-${{ github.workflow }}-${{ github.event.pull_request.number }}${{'
count="$(printf '%s\n' "$line" | grep -o "github\.event\.action == 'labeled'" | wc -l | tr -d ' ')"
[ "${count:-0}" -eq 1 ]
# The split is on the label NAMES, not on the action alone. Routing EVERY label
# event aside would move `backport` and `backport-previous` out of the group
# holding the run they have to queue behind, and one request could then evict
# the other.
#
# Pinned as the literal names, not as a count of `label.name != '`. That count
# measures the operator and says nothing about the operands, so renaming one
# name here and not in `prepare` — the half-finished rename — left it at two
# and stayed green, while a real `backport-previous` event took the `-label`
# branch and became free to cherry-pick alongside the merge run.
# Including the conjunction that joins the action check to the name checks.
# `&&` binds tighter than `||` in GitHub expressions, so flipping this one
# operator reads as `action == 'labeled' || (name != … && name != …)`, and on a
# `closed` event `github.event.label` is absent, both name checks are true, and
# EVERY event takes the suffix. A constant suffix is exactly as vacuous as no
# split at all. Every operand here was pinned before this line existed; the
# operator joining them was not.
#
# TRANSITIONAL: four names, not the original pair, because backport.yaml
# accepts both spellings while the org-level dosubot still applies the legacy
# ones. The namespaced names are genuine backport requests, so they belong on
# this side of the split exactly as the legacy ones do — routing them aside
# would hand a `kind/backport` event its own group and let it cherry-pick
# alongside the merge run, which is the failure the pin above describes. When
# dosubot's PR labelling is switched off and backport.yaml drops its legacy
# arms, this literal and the count below go back to the pair.
printf '%s\n' "$line" | grep -qF "github.event.action == 'labeled' && github.event.label.name != 'backport' && github.event.label.name != 'backport-previous' && github.event.label.name != 'kind/backport' && github.event.label.name != 'kind/backport-previous'"
# And exactly four of them. The literal above is a substring check, so appending
# a fifth exclusion satisfies it unchanged — and a fifth exclusion is not
# cosmetic: the named label stops being routed aside, lands in the main group,
# takes its single pending slot, skips in `prepare`, and evicts a genuine
# backport request while delivering nothing. Same harm as widening `types:`.
# Presence and count answer different questions; this test needs both.
count="$(printf '%s\n' "$line" | grep -o "github\.event\.label\.name != '" | wc -l | tr -d ' ')"
[ "${count:-0}" -eq 4 ]
# The suffix the condition selects, not only the condition. Asserting the
# operands alone leaves the whole split deletable — collapsing the tail to
# `&& '' || ''` puts every label back in the main group with the condition
# still reading correctly above it. Inverting it to `&& '' || '-label'` is
# worse than deleting it, because that moves the two backport requests aside
# and leaves the irrelevant labels sharing the group with the run in flight,
# which is the arrangement this file exists to prevent. Both survive every
# other assertion here.
printf '%s\n' "$line" | grep -qF "&& '-label' || ''"
}
@test "a labeled event queues behind the backport in flight instead of cancelling it" {
# Labels set with the default GITHUB_TOKEN start no run, but labels set by
# third-party GitHub Apps do, and they arrive in bursts. Under a plain `true`
# each one killed the backport the merge had just started, leaving a cancelled
# run next to a green one and the work repeated by whichever label came last.
# With `prepare` now narrowed, no burst label requalifies to repeat it,
# so dropping this arm does not restore the old noisy delivery — it loses the
# backport. `cancel-in-progress: true` is the obvious thing for the next reader
# to normalise this back to, so pin its absence too.
count="$(code_lines < "$BACKPORT" | grep -cF " cancel-in-progress: \${{ github.event.action == 'closed' }}" || true)"
[ "${count:-0}" -eq 1 ]
# Positively, not as `!= 'labeled'`. The two are equivalent only while `types:`
# holds exactly `closed` and `labeled`; the negative form hands cancellation to
# any third type added later, in the main group, which is the behaviour this
# arm removes.
count="$(code_lines < "$BACKPORT" | grep -c "cancel-in-progress:.*!= 'labeled'" || true)"
[ "${count:-0}" -eq 0 ]
count="$(code_lines < "$BACKPORT" | grep -c '^ cancel-in-progress: true$' || true)"
[ "${count:-0}" -eq 0 ]
}
@test "both backport jobs bound how long a queued request can wait" {
# These ceilings exist because of the queuing above, not for their own sake. A
# cancelling key disposed of a stuck run by killing it; queuing makes the next
# genuine request wait behind it instead, and an unbounded job on the 6-hour
# default turns one wedged job into a six-hour hole in the release line.
# Nothing else in this file makes them look load-bearing, so a tidy-up that
# drops them reads as harmless.
#
# Scope, since the test name says "how long a queued request can wait": what is
# bounded here is a stuck JOB. A run can wedge at the RUN level with every job
# already finished, and `timeout-minutes` is job-level with no run-level
# equivalent, so that state holds the group until the run is cancelled. These
# ceilings do not reach it and are not meant to.
# Asserted as a bound, not as presence. The hazard is the SIZE of the wait, so
# a pin that only checks the key exists is satisfied by `timeout-minutes: 355`,
# which restores the hole to within five minutes of the default it was written
# against.
#
# Bounded on BOTH sides, because both directions break it. Too high restores
# the hole; too low cancels a real backport that was only waiting for a runner,
# and an upper-bound-only assertion stays green at `timeout-minutes: 1`.
#
# The floor is judgement, not measurement, and says so. `timeout-minutes`
# bounds execution, and every execution measured on this workflow has been
# seconds, so no observation argues for any particular floor. No maximum is
# quoted deliberately: a superlative over a growing set goes stale on the next
# slower run. What the floor guards is a ceiling set low enough to cancel a
# cherry-pick much larger than anything seen yet; ten minutes is far above
# every leg measured. Per JOB, not per run, and not over the runner queue,
# which no `timeout-minutes` reaches.
for job in prepare backport; do
block="$(job_block "$job" "$BACKPORT" | code_lines)"
[ -n "$block" ]
minutes="$(printf '%s\n' "$block" | grep -o '^ timeout-minutes: [0-9][0-9]*' | grep -o '[0-9][0-9]*$')"
[ -n "$minutes" ]
[ "$minutes" -ge 10 ]
[ "$minutes" -le 30 ]
done
}
@test "each backport trigger reads only the labels it is entitled to" {
block="$(job_block prepare "$BACKPORT" | code_lines)"
[ -n "$block" ]
# The cumulative label set answers for the merge and for nothing else. Left
# ungated it re-enters this job on every later unrelated label — an automated
# size/* or kind/* — for a merged PR still carrying `backport`, redoing a
# backport that has already been delivered.
#
# TRANSITIONAL: four reads, not the original pair. backport.yaml accepts the
# namespaced and the legacy spelling of each label while the org-level dosubot
# still applies the legacy ones; when its PR labelling is switched off and the
# legacy arms are dropped, this literal and the count below go back to two.
printf '%s\n' "$block" | grep -qF "(github.event.action == 'closed' && (contains(github.event.pull_request.labels.*.name, 'backport') || contains(github.event.pull_request.labels.*.name, 'backport-previous') || contains(github.event.pull_request.labels.*.name, 'kind/backport') || contains(github.event.pull_request.labels.*.name, 'kind/backport-previous')))"
# And the guard reads that set exactly four times, all of them inside the gated
# disjunct above, so it cannot be joined by an ungated one that quietly
# restores the old behaviour while the assertion above stays green.
#
# Counted as OCCURRENCES, not lines. `grep -c` counts matching lines, and all
# four reads live on one line here, so it answers 1 and keeps answering 1 when a
# fifth read is appended to that same line — which is exactly the restoration
# this is meant to catch. Only an append on a NEW line would have moved it.
count="$(printf '%s\n' "$block" | grep -o 'contains(github\.event\.pull_request\.labels' | wc -l | tr -d ' ')"
[ "${count:-0}" -eq 4 ]
# A label event answers for the label it carries. Both spellings of each name,
# for the transitional reason given above.
printf '%s\n' "$block" | grep -qF "(github.event.action == 'labeled' && (github.event.label.name == 'backport' || github.event.label.name == 'backport-previous' || github.event.label.name == 'kind/backport' || github.event.label.name == 'kind/backport-previous'))"
# The two conjuncts the label logic hangs off, which the label assertions above
# would not miss. `base.ref == 'main'` is what stops the bot backporting its own
# output: its PRs target release-X.Y, so they cannot satisfy it however often a
# labeler re-applies `backport` to a `[Backport release-X.Y]` title. docs/release.md
# calls that architectural protection, which is only true while the line is here.
# With their trailing conjunctions, for the reason the group-key test spells
# out: flipping either `&&` to `||` turns the guard into a disjunction that
# fires on every merge to main and on labels applied to open PRs. That one does
# not produce a wrong backport, because the `backport` job re-checks both
# independently, but it runs the job when nothing asked for it.
# In BOTH jobs, because docs/release.md rests the architectural protection on
# both `if:` blocks. Losing either copy alone is survivable (the other still
# skips, and `backport` needs `prepare`), so this pins a duplicate rather than
# a hole — but an untested duplicate is how a duplicate stops being one.
for job in prepare backport; do
b="$(job_block "$job" "$BACKPORT" | code_lines)"
[ -n "$b" ]
printf '%s\n' "$b" | grep -qF "github.event.pull_request.base.ref == 'main' &&"
printf '%s\n' "$b" | grep -qF 'github.event.pull_request.merged == true &&'
done
}
@test "the audit accepts exactly the backport labels the bot acts on" {
# cmd/backport-audit answers "has everything labelled for this line landed on
# it" and exits non-zero while anything has not, so docs/release.md gates a
# patch cut on it. It reaches its candidates with `gh pr list --label`, which
# matches a label by exact name.
#
# Pin the two sets equal rather than pinning either one's contents: the
# audit must not miss a spelling the bot honours, and must not keep one the
# bot has retired. Both directions matter, because the legacy names are
# transitional and the two files have to drop them together.
[ -f "$AUDIT" ]
# Every name the bot tests for, from the label-detection script.
bot="$(grep -o "labels\.includes('[^']*')" "$BACKPORT" | sed "s/.*('//;s/')//" | sort -u)"
[ -n "$bot" ]
# Every backport label spelled as a Go string literal in the audit. Its own
# prose quotes these names with backticks, not double quotes, so a comment
# cannot satisfy this.
audit="$(grep -o '"\(kind/\)\{0,1\}backport\(-previous\)\{0,1\}"' "$AUDIT" | tr -d '"' | sort -u)"
[ -n "$audit" ]
[ "$bot" = "$audit" ]
# Both requests are represented, so the sets cannot agree by both having lost
# the same half.
printf '%s\n' "$audit" | grep -qx 'kind/backport'
printf '%s\n' "$audit" | grep -qx 'kind/backport-previous'
}