mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 12:13:55 +09:00
Merge branch 'main' into fix/clickhouse-idle-before-backup
main pins the archive name a backup Pod keeps across a wait for late bucket credentials, against the controller's archive helper. That helper now names a run's archive with part of the BackupJob's UID, so the merged test has to give its BackupJob one. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
This commit is contained in:
commit
dae76d186c
289 files changed
+31331
-5554
No files matched your search
@@ -35,7 +35,7 @@ Similarly, never propose edits to:
|
||||
|
||||
Each commit must follow [Conventional Commits](https://www.conventionalcommits.org/) format: `type(scope): brief description`.
|
||||
|
||||
Valid types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`.
|
||||
Valid types: `feat`, `fix`, `security` (a fix that comes through a security advisory), `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`.
|
||||
|
||||
Valid scopes:
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
<!-- Thank you for making a contribution! Here are some tips for you:
|
||||
- Use Conventional Commits for the PR title: `type(scope): description`
|
||||
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
|
||||
- Types: feat, fix, security (a fix that comes through a security advisory), docs, style, refactor, perf, test, build, ci, chore
|
||||
- Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples:
|
||||
- System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api
|
||||
- Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes
|
||||
@@ -8,7 +8,7 @@
|
||||
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer
|
||||
- If it's a work in progress, consider creating this PR as a draft.
|
||||
- Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft.
|
||||
- Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version.
|
||||
- Fixes merged to `main` reach the release lines the backport batch runs for automatically, so do not open backport PRs by hand: https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md
|
||||
-->
|
||||
|
||||
## What this PR does
|
||||
|
||||
+15
-12
@@ -22,6 +22,7 @@
|
||||
# do-not-merge/ PR merge blockers
|
||||
# security/ security-finding severity and status (Cozystack-specific)
|
||||
# size/ PR size (auto-applied)
|
||||
# backport/ maintainer overrides of the backport batch rules (Cozystack-specific)
|
||||
#
|
||||
# `aliases:` lets EndBug/label-sync rename existing labels without losing
|
||||
# references on already-tagged issues and PRs.
|
||||
@@ -78,18 +79,6 @@
|
||||
color: 'e11d21'
|
||||
description: Indicates the change introduces a breaking API or behaviour change
|
||||
|
||||
# kind/backport-previous has no upstream Kubernetes equivalent (k8s has no
|
||||
# two-release-line model); it stays under kind/ per the file convention.
|
||||
- name: kind/backport
|
||||
color: 'fbca04'
|
||||
description: Categorizes issue or PR as requiring a backport to the current release line
|
||||
aliases: ['backport']
|
||||
|
||||
- name: kind/backport-previous
|
||||
color: 'fbd876'
|
||||
description: Categorizes issue or PR as requiring a backport to the previous release line
|
||||
aliases: ['backport-previous']
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# priority/ — urgency
|
||||
# ──────────────────────────────────────────────
|
||||
@@ -411,3 +400,17 @@
|
||||
- name: security/fixed
|
||||
color: '0e8a16'
|
||||
description: Fix released
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# backport/ — maintainer overrides of the backport batch rules
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
# Read by cmd/backport-batch from the merged pull request on main;
|
||||
# docs/release/backports.md describes both.
|
||||
- name: backport/include
|
||||
color: 'c2e0c6'
|
||||
description: The backport batch takes this PR though rules 2-4 keep it out; it must still pick and pass CI
|
||||
|
||||
- name: backport/skip
|
||||
color: 'f9d0c4'
|
||||
description: The backport batch keeps this PR out although the rules take it
|
||||
@@ -0,0 +1,107 @@
|
||||
name: Backport batch
|
||||
|
||||
# Builds the backport batch for each line `backport-batch lines` names, the
|
||||
# newest released lines and any line in its rc window, newest line first, as
|
||||
# docs/release/backports.md describes. The steps live in
|
||||
# hack/backport-batch-scheduled.sh, which a maintainer can also run by hand.
|
||||
#
|
||||
# The batch runs as the cozystack-ci App, with a token minted for this
|
||||
# repository alone with Contents and Pull requests write. It has no
|
||||
# workflows permission, so an original that changes .github/workflows/
|
||||
# waits on the batch for a maintainer (--no-workflow-files). Its pushes,
|
||||
# unlike GITHUB_TOKEN's, start CI on the batch pull request.
|
||||
#
|
||||
# Repository settings it reads; none is created here:
|
||||
# secret COZYSTACK_CI_APP_ID, COZYSTACK_CI_PRIVATE_KEY
|
||||
# the cozystack-ci App, shared with the
|
||||
# release workflows. Required.
|
||||
# variable BACKPORT_ALERT_TEAM the team or person the batch's comment
|
||||
# mentions, org/team or a login.
|
||||
#
|
||||
# A batch open since the previous run, deferred conflicts and originals that
|
||||
# change workflows wait for a maintainer: the batch's comment lists them. A
|
||||
# failed batch CI waits too, and shows on the batch pull request's checks.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Monday and Thursday: each run leaves working days to cut an rc, run its
|
||||
# e2e and promote it before the weekend. 06:23 UTC is after the nightly
|
||||
# and the daily sweeps.
|
||||
- cron: '23 6 * * 1,4'
|
||||
workflow_dispatch: {}
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: backport-batch
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
batch:
|
||||
name: Backport batch
|
||||
if: github.repository == 'cozystack/cozystack'
|
||||
runs-on: ubuntu-latest
|
||||
# The App token lives an hour from its mint, a few minutes in; a run takes
|
||||
# minutes, so the job ends well before the token does.
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
# The pipe falls back to direct on any proxy error (hack/common-envs.mk).
|
||||
GOPROXY: "https://proxy.golang.org|direct"
|
||||
steps:
|
||||
- name: Require the cozystack-ci App
|
||||
env:
|
||||
APP_ID: ${{ secrets.COZYSTACK_CI_APP_ID }}
|
||||
APP_KEY: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
|
||||
run: |
|
||||
if [ -z "$APP_ID" ] || [ -z "$APP_KEY" ]; then
|
||||
echo "::error::COZYSTACK_CI_APP_ID or COZYSTACK_CI_PRIVATE_KEY is not set. The batch pushes and opens pull requests as the cozystack-ci App only; see docs/release/backports.md#the-scheduled-run."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: false
|
||||
|
||||
- name: Build backport-batch
|
||||
run: go build -o "$RUNNER_TEMP/backport-batch" ./cmd/backport-batch
|
||||
|
||||
- name: Generate GitHub App token
|
||||
id: app-token
|
||||
uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1
|
||||
with:
|
||||
app-id: ${{ secrets.COZYSTACK_CI_APP_ID }}
|
||||
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
|
||||
owner: cozystack
|
||||
repositories: cozystack
|
||||
permission-contents: write
|
||||
permission-pull-requests: write
|
||||
|
||||
# git pushes through gh, which reads the token from the environment of
|
||||
# each call, so the token is never written into the checkout.
|
||||
- name: Use the App's bot as git identity and credential
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app-token.outputs.token }}
|
||||
APP_SLUG: ${{ steps.app-token.outputs.app-slug }}
|
||||
run: |
|
||||
gh auth setup-git
|
||||
id=$(gh api "/users/${APP_SLUG}[bot]" --jq .id)
|
||||
git config --global user.name "${APP_SLUG}[bot]"
|
||||
git config --global user.email "${id}+${APP_SLUG}[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Run the batch
|
||||
env:
|
||||
BACKPORT_BATCH: ${{ runner.temp }}/backport-batch
|
||||
BACKPORT_BATCH_TOKEN: ${{ steps.app-token.outputs.token }}
|
||||
BACKPORT_BATCH_BOT: ${{ steps.app-token.outputs.app-slug }}[bot]
|
||||
BACKPORT_ALERT_TEAM: ${{ vars.BACKPORT_ALERT_TEAM }}
|
||||
run: hack/backport-batch-scheduled.sh --no-workflow-files
|
||||
@@ -1,305 +0,0 @@
|
||||
name: Automatic Backport
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [closed, labeled] # fires when PR is closed (merged) or labeled
|
||||
|
||||
# Two mechanisms here, answering two different failures.
|
||||
#
|
||||
# A run joins its concurrency group before any job-level `if` is evaluated, so
|
||||
# `prepare`'s guard below cannot stop a label event from disturbing the run
|
||||
# already backporting this PR; it only decides what the label run does after the
|
||||
# damage. Labels set with the default GITHUB_TOKEN start no run, but labels set
|
||||
# by third-party GitHub Apps do, and they arrive in bursts. Under a plain
|
||||
# `cancel-in-progress: true` each one cancelled the backport the merge had just
|
||||
# started, and nothing reports that beyond a cancelled run sitting next to a
|
||||
# green one.
|
||||
#
|
||||
# The delivery survived anyway, because the old guard admitted a run on the PR's
|
||||
# cumulative labels: the last label of a burst requalified and redid the work.
|
||||
# `prepare`'s narrowing below removes that accidental safety net, so the two
|
||||
# changes land together — narrowing alone would turn a noisy delivery into an
|
||||
# absent one.
|
||||
#
|
||||
# So the group key moves a label event that is NOT a backport request aside, and
|
||||
# `cancel-in-progress` stops one that IS from cancelling rather than queuing.
|
||||
# Both are load-bearing. `backport` and `backport-previous` are separate requests
|
||||
# that both have to complete, so they share the main group and must not cancel
|
||||
# each other; every other label does no work here and only has to stay out of
|
||||
# the way.
|
||||
#
|
||||
# That is also why this file does not simply copy pull-requests.yaml, which
|
||||
# splits the key and leaves cancellation unconditional: a discarded label run
|
||||
# there publishes nothing, so isolating it is the whole fix.
|
||||
#
|
||||
# Queuing protects a pending run from the run ahead of it and never from the one
|
||||
# behind: the group holds a single pending run, and a newer one evicts it. With
|
||||
# the split in place that costs nothing, and for the same reason the fan-out
|
||||
# below is harmless — the only events left in this group are backport requests,
|
||||
# the evictor is by definition the newer of the two, and `prepare`'s `Check which
|
||||
# labels are present` step reads the cumulative label set. So the survivor covers
|
||||
# a superset of what the evicted run would have delivered, for any sequence that
|
||||
# only adds labels. Removing one in between is the exception and needs no cover:
|
||||
# the evictor then carries less because the request was withdrawn.
|
||||
#
|
||||
# That is exactly what the split buys. Without it the evictor could be any label
|
||||
# at all, and a run that does not qualify delivers nothing in place of the
|
||||
# request it displaced.
|
||||
#
|
||||
# `queue: max` raises the single pending slot to a hundred and is the obvious
|
||||
# thing to reach for instead of the split. Taking it means giving up the
|
||||
# conditional cancel, because `queue: max` and `cancel-in-progress: true` are a
|
||||
# validation error together and this key still evaluates to true on a merge.
|
||||
# The trade is available and was not taken: eviction is what keeps a burst to one
|
||||
# surviving run rather than a hundred queued ones, and the feature only shipped
|
||||
# 2026-05-07, on a path that delivers fixes to release lines.
|
||||
concurrency:
|
||||
group: backport-${{ github.workflow }}-${{ github.event.pull_request.number }}${{ (github.event.action == 'labeled' && github.event.label.name != 'backport' && github.event.label.name != 'backport-previous' && github.event.label.name != 'kind/backport' && github.event.label.name != 'kind/backport-previous') && '-label' || '' }}
|
||||
# Stated positively, matching the group key above. `!= 'labeled'` is equivalent
|
||||
# only while `types:` holds exactly these two, and that form fails open: adding
|
||||
# `unlabeled` or `reopened` there would hand those events cancellation in the
|
||||
# main group, which is the behaviour this file exists to remove.
|
||||
#
|
||||
# The positive form removes the cancelling, not the occupying. A third type
|
||||
# still lands in the main group, still takes its single pending slot, and its
|
||||
# `prepare` still skips — so it can evict a genuine backport request and
|
||||
# deliver nothing in its place. `types:` is load-bearing for that reason and is
|
||||
# pinned in hack/release-freeze-contract.bats; widening it needs the group key
|
||||
# widened with it.
|
||||
cancel-in-progress: ${{ github.event.action == 'closed' }}
|
||||
|
||||
# Top-level token defaults to read-only. A job-level block replaces this one
|
||||
# rather than adding to it, so each job restates every scope it needs.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
# Determine which backports are needed
|
||||
prepare:
|
||||
# This guard admits each action on the labels it is entitled to. `closed`
|
||||
# carries no label of its own, so it reads the set the PR ended up with.
|
||||
# `labeled` reads only the label just added: matching the cumulative set
|
||||
# here means every later unrelated label on a merged PR that still carries
|
||||
# `backport` re-enters this job to redo a backport already delivered.
|
||||
#
|
||||
# Scope, so this is not read as covering the whole job: the `Check which
|
||||
# labels are present` step below still reads the cumulative set, so a
|
||||
# `backport-previous` event on a PR also carrying `backport` fans out to
|
||||
# both matrix legs. That is deliberate, and each state a re-entered leg can
|
||||
# find is already accounted for: delivered is skipped by the merged-backport
|
||||
# guard, still open is skipped by backport-action on the 422, and closed
|
||||
# unmerged gets a fresh backport PR, which is the retry that guard
|
||||
# deliberately preserves rather than an accident. What the narrowing here
|
||||
# decides is whether the job RUNS, not which targets it picks.
|
||||
if: |
|
||||
github.event.pull_request.merged == true &&
|
||||
github.event.pull_request.base.ref == 'main' &&
|
||||
(
|
||||
(github.event.action == 'closed' && (contains(github.event.pull_request.labels.*.name, 'backport') || contains(github.event.pull_request.labels.*.name, 'backport-previous') || contains(github.event.pull_request.labels.*.name, 'kind/backport') || contains(github.event.pull_request.labels.*.name, 'kind/backport-previous'))) ||
|
||||
(github.event.action == 'labeled' && (github.event.label.name == 'backport' || github.event.label.name == 'backport-previous' || github.event.label.name == 'kind/backport' || github.event.label.name == 'kind/backport-previous'))
|
||||
)
|
||||
# GitHub API calls only — no docker, no repo toolchain: GitHub-hosted.
|
||||
runs-on: ubuntu-latest
|
||||
# Queuing gives a stuck run a way to hurt the next one that a cancelling key
|
||||
# did not: the label event that used to kill it now waits behind it rather
|
||||
# than clearing it.
|
||||
#
|
||||
# This ceiling bounds a stuck JOB, which is the case it can reach. A run can
|
||||
# also wedge at the RUN level with every job already finished, and no
|
||||
# job-level key expires that one; it holds the group until GitHub retires the
|
||||
# run. Rare but real: run 31232645133 has been `in_progress` since
|
||||
# 2026-08-08 with `prepare` and `backport` both skipped. Nothing here
|
||||
# reaches that state, so docs/release.md tells an operator to look for a run
|
||||
# still in flight and clear it before re-applying the label.
|
||||
#
|
||||
# What it bounds is EXECUTION, not the wait for a runner. GitHub's limits
|
||||
# page says "job execution time" for the 6-hour job cap while spelling out
|
||||
# that the 35-day run cap "includes execution duration, and time spent on
|
||||
# waiting and approval" — the contrast is deliberate — and
|
||||
# orgs/community#50926 shows a job sitting six hours on "Waiting for a runner
|
||||
# to pick up this job" under `timeout-minutes: 5`. So queue time is a third
|
||||
# unbounded case alongside the run-level wedge, not something this covers.
|
||||
#
|
||||
# 30 is therefore generous rather than calculated. Execution on run
|
||||
# 29844315422 was 6s for `prepare` and 2s and 16s for the two matrix legs,
|
||||
# against 9m24s, 5m28s and 5m45s of queue for the same three jobs. Every
|
||||
# execution measured on this workflow has been seconds, and no maximum is
|
||||
# quoted on purpose: an absolute maximum over a growing set is stale as soon
|
||||
# as a slower run lands. The value leaves room for a cherry-pick far larger
|
||||
# than anything seen while replacing the 6-hour default, and nothing here is
|
||||
# sized against the queue because nothing here can be.
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
outputs:
|
||||
backport_current: ${{ steps.labels.outputs.backport }}
|
||||
backport_previous: ${{ steps.labels.outputs.backport_previous }}
|
||||
current_branch: ${{ steps.branches.outputs.current_branch }}
|
||||
previous_branch: ${{ steps.branches.outputs.previous_branch }}
|
||||
steps:
|
||||
- name: Check which labels are present
|
||||
id: labels
|
||||
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
|
||||
with:
|
||||
script: |
|
||||
const pr = context.payload.pull_request;
|
||||
const labels = pr.labels.map(l => l.name);
|
||||
// TRANSITIONAL: both spellings are accepted so the rename can land
|
||||
// in any order relative to the org-level dosubot, which still
|
||||
// applies the legacy names and is today the dominant applier on
|
||||
// real backports. Once dosubot's PR labelling is switched off and
|
||||
// the in-repo labeler is the only applier, the legacy arms here,
|
||||
// in the `if:` gate above and in the concurrency group can all be
|
||||
// deleted in one pass.
|
||||
const isBackport = labels.includes('kind/backport') || labels.includes('backport');
|
||||
const isBackportPrevious = labels.includes('kind/backport-previous') || labels.includes('backport-previous');
|
||||
|
||||
core.setOutput('backport', isBackport ? 'true' : 'false');
|
||||
core.setOutput('backport_previous', isBackportPrevious ? 'true' : 'false');
|
||||
|
||||
console.log(`backport label: ${isBackport}, backport-previous label: ${isBackportPrevious}`);
|
||||
|
||||
- name: Determine target branches
|
||||
id: branches
|
||||
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
|
||||
with:
|
||||
script: |
|
||||
// Enumerate the real release-X.Y branches rather than deriving them
|
||||
// from the latest published release.
|
||||
//
|
||||
// Since cutting an rc freezes the line (cut-prerelease.yaml),
|
||||
// release-X.Y is created when the first vX.Y.0-rc.N is cut — before
|
||||
// vX.Y.0 is ever published. getLatestRelease returns the newest
|
||||
// published STABLE, so throughout the freeze window it still named
|
||||
// the previous line: a `backport` on a fix for the release being
|
||||
// stabilised would have landed on release-X.(Y-1) and missed the
|
||||
// release it was written for. That window is exactly when backports
|
||||
// matter most, since the frozen branch is the only way in.
|
||||
//
|
||||
// Enumerating branches also drops the contiguous-minor assumption:
|
||||
// `previous` used to be computed as min-1, which named a
|
||||
// non-existent branch whenever a minor was skipped.
|
||||
const branches = await github.paginate(github.rest.repos.listBranches, {
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
per_page: 100
|
||||
});
|
||||
|
||||
const lines = branches
|
||||
.map(b => /^release-(\d+)\.(\d+)$/.exec(b.name))
|
||||
.filter(Boolean)
|
||||
.map(m => ({ name: m[0], maj: Number(m[1]), min: Number(m[2]) }))
|
||||
.sort((a, b) => (b.maj - a.maj) || (b.min - a.min));
|
||||
|
||||
if (lines.length === 0) {
|
||||
core.setFailed('No release-X.Y branches found; cannot determine backport target.');
|
||||
return;
|
||||
}
|
||||
|
||||
const currentBranch = lines[0].name;
|
||||
const previousBranch = lines.length > 1 ? lines[1].name : '';
|
||||
|
||||
core.setOutput('current_branch', currentBranch);
|
||||
core.setOutput('previous_branch', previousBranch);
|
||||
|
||||
console.log(`Release lines found: ${lines.map(l => l.name).join(', ')}`);
|
||||
console.log(`Current branch: ${currentBranch}, Previous branch: ${previousBranch || 'N/A'}`);
|
||||
|
||||
// Both targets come from the enumeration above, so they exist by
|
||||
// construction — no getBranch probe needed. What CAN still fail is
|
||||
// asking for a previous line when only one exists.
|
||||
if (!previousBranch && '${{ steps.labels.outputs.backport_previous }}' === 'true') {
|
||||
core.setFailed('backport-previous requested, but only one release line exists.');
|
||||
return;
|
||||
}
|
||||
|
||||
backport:
|
||||
needs: prepare
|
||||
if: |
|
||||
github.event.pull_request.merged == true &&
|
||||
github.event.pull_request.base.ref == 'main' &&
|
||||
(needs.prepare.outputs.backport_current == 'true' || needs.prepare.outputs.backport_previous == 'true')
|
||||
# git cherry-picks via backport-action — no docker, no repo toolchain: GitHub-hosted.
|
||||
runs-on: ubuntu-latest
|
||||
# See `prepare` above: a queued request waits on this one, so it needs a
|
||||
# ceiling rather than the 6-hour default. Bounds this job's execution only;
|
||||
# the wait for a runner is outside it either way.
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
strategy:
|
||||
matrix:
|
||||
backport_type: [current, previous]
|
||||
steps:
|
||||
# 1. Determine target branch based on matrix
|
||||
- name: Set target branch
|
||||
id: target
|
||||
if: |
|
||||
(matrix.backport_type == 'current' && needs.prepare.outputs.backport_current == 'true') ||
|
||||
(matrix.backport_type == 'previous' && needs.prepare.outputs.backport_previous == 'true')
|
||||
run: |
|
||||
if [ "${{ matrix.backport_type }}" == "current" ]; then
|
||||
echo "branch=${{ needs.prepare.outputs.current_branch }}" >> $GITHUB_OUTPUT
|
||||
echo "Target branch: ${{ needs.prepare.outputs.current_branch }}"
|
||||
else
|
||||
echo "branch=${{ needs.prepare.outputs.previous_branch }}" >> $GITHUB_OUTPUT
|
||||
echo "Target branch: ${{ needs.prepare.outputs.previous_branch }}"
|
||||
fi
|
||||
|
||||
# 2. Skip if this target already has a merged backport.
|
||||
#
|
||||
# The action's backport branch name is deterministic,
|
||||
# backport-<pull_number>-to-<target_branch>. A merged PR with that head
|
||||
# means an earlier run already delivered this change to the target — a
|
||||
# second label event (e.g. backport-previous added after backport
|
||||
# already merged) would re-enter this job and cherry-pick commits the
|
||||
# target branch already has. Git reports that as an empty pick (exit 1)
|
||||
# rather than a conflict, which draft_commit_conflicts below misreads
|
||||
# as a real conflict and reports as a failed backport on the original
|
||||
# PR. Skipping here avoids the false report; it does not touch the
|
||||
# empty-pick behavior itself, which lives in the action.
|
||||
- name: Check for existing merged backport
|
||||
id: guard
|
||||
if: steps.target.outcome == 'success'
|
||||
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
|
||||
with:
|
||||
script: |
|
||||
const targetBranch = '${{ steps.target.outputs.branch }}';
|
||||
const head = `backport-${context.payload.pull_request.number}-to-${targetBranch}`;
|
||||
const { data: prs } = await github.rest.pulls.list({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
state: 'closed',
|
||||
head: `${context.repo.owner}:${head}`,
|
||||
});
|
||||
const alreadyMerged = prs.some(p => p.merged_at !== null);
|
||||
core.setOutput('already_merged', alreadyMerged ? 'true' : 'false');
|
||||
if (alreadyMerged) {
|
||||
console.log(`Skipping backport to ${targetBranch}: ${head} already merged.`);
|
||||
}
|
||||
|
||||
# 3. Checkout (required by backport‑action)
|
||||
- name: Checkout repository
|
||||
if: steps.target.outcome == 'success' && steps.guard.outputs.already_merged != 'true'
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
|
||||
# 4. Create the back‑port pull request
|
||||
- name: Create back‑port PR
|
||||
id: backport
|
||||
if: steps.target.outcome == 'success' && steps.guard.outputs.already_merged != 'true'
|
||||
uses: korthout/backport-action@0193454f0c5947491d348f33a275c119f30eb736 # v3.2.1
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
label_pattern: '' # don't read labels for targets
|
||||
target_branches: ${{ steps.target.outputs.branch }}
|
||||
merge_commits: skip
|
||||
# conflict_resolution lives inside the experimental JSON input, not as a
|
||||
# top-level key. Passed top-level it is silently ignored and the action
|
||||
# falls back to the experimental default ("fail"), which on a conflict
|
||||
# creates no PR and only comments. Nesting it here makes a conflicting
|
||||
# backport open a draft PR with the conflict committed for manual fixup.
|
||||
experimental: |
|
||||
{
|
||||
"conflict_resolution": "draft_commit_conflicts"
|
||||
}
|
||||
@@ -154,6 +154,7 @@ jobs:
|
||||
sparse-checkout: |
|
||||
hack
|
||||
docs/agents
|
||||
docs/release
|
||||
persist-credentials: false
|
||||
|
||||
# The agent reads docs/agents/changelog.md from the WORKING TREE, which is
|
||||
@@ -163,18 +164,17 @@ jobs:
|
||||
# detached rc-tag checkout is exactly that, so a COMPLIANT agent would
|
||||
# silently decline to generate on every promotion of an existing rc.
|
||||
# Overlay the current instructions so the agent reads today's contract.
|
||||
# changelog.md sends the agent to the checks in docs/release.md and to
|
||||
# hack/validate-changelog.sh, so those are overlaid too: the rc tag's
|
||||
# copies predate any check added since. docs/release.md reaches
|
||||
# .release-tooling only because the sparse checkout above is in cone
|
||||
# mode (the action's default), which carries the files of every parent
|
||||
# directory of docs/agents.
|
||||
# changelog.md sends the agent to the checks in docs/release/changelogs.md
|
||||
# and to hack/validate-changelog.sh, so those are overlaid too: the rc
|
||||
# tag's copies predate any check added since, and an rc cut before the
|
||||
# release docs were split into docs/release/ has no such directory.
|
||||
- name: Refresh agent instructions
|
||||
run: |
|
||||
cp .release-tooling/docs/agents/changelog.md docs/agents/changelog.md
|
||||
cp .release-tooling/docs/release.md docs/release.md
|
||||
mkdir -p docs/release
|
||||
cp .release-tooling/docs/release/changelogs.md docs/release/changelogs.md
|
||||
cp .release-tooling/hack/validate-changelog.sh hack/validate-changelog.sh
|
||||
echo "Refreshed docs/agents/changelog.md, docs/release.md and hack/validate-changelog.sh from ${GITHUB_REF_NAME}."
|
||||
echo "Refreshed docs/agents/changelog.md, docs/release/changelogs.md and hack/validate-changelog.sh from ${GITHUB_REF_NAME}."
|
||||
|
||||
# Skip-priority + validate-then-fall-through: use a pre-existing changelog
|
||||
# only if it is USABLE, so an invalid higher-priority source can never
|
||||
|
||||
@@ -6,7 +6,7 @@ name: Cut Pre-release Tag
|
||||
# This is the single entry point for creating a pre-release. Stable vX.Y.Z tags
|
||||
# are never cut here (or by hand) — they are created write-once by
|
||||
# pull-requests-release.yaml finalize at a promote PR's merge commit. See
|
||||
# docs/release.md and #2677.
|
||||
# docs/release/stable-tags.md and #2677.
|
||||
#
|
||||
# Dispatch from the branch the tag belongs to: `main` for a new minor's
|
||||
# pre-release (vX.Y.0-rc.N), or `release-X.Y` for a patch line (vX.Y.Z-rc.N).
|
||||
|
||||
@@ -771,21 +771,6 @@ jobs:
|
||||
name: cozyreport
|
||||
path: _out/cozyreport.tgz
|
||||
if-no-files-found: warn
|
||||
# Both halves, and in this order: the compose project owns the nodes and
|
||||
# the zpools outlive it.
|
||||
- name: Tear down sandbox
|
||||
if: always()
|
||||
run: |
|
||||
# Annotated, not fatal: same reasoning as the in-tree lane. The runner
|
||||
# is discarded after the job, so a dirty host has no next tenant, and
|
||||
# failing here would post the required "E2E Tests" status as a failure
|
||||
# indistinguishable from a real one.
|
||||
cleanup_rc=0
|
||||
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME delete-cluster-container || cleanup_rc=$?
|
||||
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME delete || true
|
||||
if [ "$cleanup_rc" -ne 0 ]; then
|
||||
echo "::error title=sandbox teardown::teardown exited $cleanup_rc; the host may hold a stale compose project or zpool"
|
||||
fi
|
||||
|
||||
# Flip the pending "E2E Tests" commit status posted by `resolve` to its
|
||||
# terminal state. Runs on failure too, so a failed publish or e2e turns the
|
||||
|
||||
@@ -24,7 +24,7 @@ jobs:
|
||||
with:
|
||||
script: |
|
||||
// Conventional Commits types accepted by Cozystack (per docs/agents/contributing.md):
|
||||
// feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert
|
||||
// feat, fix, security, docs, style, refactor, perf, test, build, ci, chore, revert
|
||||
// Mapping below maps a subset to kind/* — types not listed do not produce a kind/*.
|
||||
const typeToKind = {
|
||||
feat: 'kind/feature',
|
||||
@@ -32,7 +32,7 @@ jobs:
|
||||
docs: 'kind/documentation',
|
||||
chore: 'kind/cleanup',
|
||||
refactor: 'kind/cleanup',
|
||||
// style, perf, test, build, ci, revert — no kind mapping
|
||||
// security, style, perf, test, build, ci, revert — no kind mapping
|
||||
};
|
||||
|
||||
// scope -> area/* mapping. Keys are the scopes observed in cozystack issues
|
||||
@@ -296,3 +296,82 @@ jobs:
|
||||
labels: newLabels,
|
||||
});
|
||||
core.info(`Applied labels: ${newLabels.join(', ')}`);
|
||||
|
||||
# Backports are automatic (docs/release/backports.md), so a pull request
|
||||
# against a release branch from outside the team is closed with a pointer to
|
||||
# the policy. Only when it is opened: a maintainer keeps a fix that exists
|
||||
# only on a release line by reopening it.
|
||||
#
|
||||
# pull_request_target hands a write token to a fork's pull request, so this
|
||||
# job never touches the head: actions/checkout takes the event's own ref,
|
||||
# which is the base repository, the script reads only the event JSON and the
|
||||
# author's permission on the repository, and no `${{ }}` expression reaches
|
||||
# the shell. The permission read needs repository metadata read access, which
|
||||
# a GITHUB_TOKEN always carries and `permissions:` has no key for.
|
||||
close-manual-backport:
|
||||
if: >-
|
||||
github.event.action == 'opened' &&
|
||||
startsWith(github.event.pull_request.base.ref, 'release-')
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read # check out hack/ from the base repository
|
||||
pull-requests: write # close the pull request with a comment
|
||||
steps:
|
||||
- name: Check out the base repository
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
sparse-checkout: hack
|
||||
- name: Close a backport opened by hand from outside the team
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: hack/close-manual-backport.sh "$GITHUB_EVENT_PATH"
|
||||
|
||||
# A batch pull request carries each original in the shape cmd/backport-batch
|
||||
# gives it. fold checks one resolution, and only when whoever resolved the
|
||||
# conflict runs it; this checks every original on the batch on each push.
|
||||
#
|
||||
# pull_request_target takes the workflow from main for a pull request into
|
||||
# any branch, so the check reaches release-* lines, which do not carry it; a
|
||||
# pull_request job would run the line's own workflows. The tool is built from
|
||||
# main, since the lines do not carry it either. Nothing from the pull request
|
||||
# is checked out or run: its head is fetched as data and read with git, and
|
||||
# no `${{ }}` expression reaches the shell. Only a branch of this repository
|
||||
# named like a batch is checked.
|
||||
backport-batch-verify:
|
||||
name: Backport batch shape
|
||||
if: >-
|
||||
startsWith(github.event.pull_request.base.ref, 'release-') &&
|
||||
startsWith(github.event.pull_request.head.ref, 'backport-batch/') &&
|
||||
github.event.pull_request.head.repo.full_name == github.repository
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
# The pipe falls back to direct on any proxy error (hack/common-envs.mk).
|
||||
GOPROXY: "https://proxy.golang.org|direct"
|
||||
steps:
|
||||
- name: Check out main
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
ref: main
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: false
|
||||
- name: Build backport-batch from main
|
||||
run: go build -o "$RUNNER_TEMP/backport-batch" ./cmd/backport-batch
|
||||
- name: Fetch the batch head as data
|
||||
env:
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: git fetch --no-tags --quiet origin "$HEAD_SHA"
|
||||
- name: Verify every original on the batch
|
||||
env:
|
||||
BASE_REF: ${{ github.event.pull_request.base.ref }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: '"$RUNNER_TEMP/backport-batch" verify --line "$BASE_REF" --head "$HEAD_SHA"'
|
||||
@@ -89,16 +89,14 @@ jobs:
|
||||
runs-on: ubuntu-22.04
|
||||
permissions:
|
||||
contents: read
|
||||
# A backport PR is the one shape whose author cannot fix a finding: the
|
||||
# backport action cherry-picks merged commits verbatim, so their messages
|
||||
# A backport batch is the one shape whose author cannot fix a finding:
|
||||
# cmd/backport-batch cherry-picks merged commits verbatim, so their messages
|
||||
# are history, and roughly a quarter of the commits on main predate the
|
||||
# rule. Matched on the branch name the action generates,
|
||||
# backport-<pr>-to-<branch>, rather than on the prefix alone; the label that
|
||||
# asks for a backport sits on the original PR, not on the generated one, so
|
||||
# a branch name is the only signal a pull_request event carries. That makes
|
||||
# the exemption a convention rather than something enforced: anyone can name
|
||||
# a branch this way. Every other PR, fork included, is checked.
|
||||
if: ${{ !(startsWith(github.head_ref, 'backport-') && contains(github.head_ref, '-to-')) }}
|
||||
# rule. Matched on the branch name the batch uses,
|
||||
# backport-batch/<line>-<date>, and only from a branch of this repository,
|
||||
# which takes write access to push: a fork can name a branch the same way.
|
||||
# Every other PR, fork included, is checked.
|
||||
if: ${{ !(startsWith(github.head_ref, 'backport-batch/') && github.event.pull_request.head.repo.full_name == github.repository) }}
|
||||
steps:
|
||||
- name: Checkout head
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
|
||||
@@ -327,9 +327,12 @@ jobs:
|
||||
// A maintenance branch created before the candidate pipeline landed
|
||||
// would execute its old pull_request workflows after merge. Refuse
|
||||
// before the first registry write unless the actual target base has
|
||||
// both trusted verification gates and their helpers. Backport this
|
||||
// change to an existing release-X.Y line before promoting its next
|
||||
// patch; the rc-derived head does not need to carry the tooling.
|
||||
// both trusted verification gates and their helpers. An existing
|
||||
// release-X.Y line gets them before its next patch is promoted,
|
||||
// through the backport batch with backport/include on the pull
|
||||
// request that brought them to main, or through a maintainer's pull
|
||||
// request against the line; the rc-derived head does not need to
|
||||
// carry the tooling.
|
||||
//
|
||||
// Every file the base-checked-out verifier reaches for belongs in
|
||||
// the list below, its own sourced libraries included: those are
|
||||
@@ -363,7 +366,7 @@ jobs:
|
||||
});
|
||||
} catch (e) {
|
||||
if (e.status === 404) {
|
||||
core.setFailed(`Target base '${baseBranch}' lacks '${path}'. Backport the candidate-aware promotion pipeline before dispatch.`);
|
||||
core.setFailed(`Target base '${baseBranch}' lacks '${path}'. Bring the candidate-aware promotion pipeline to it before dispatch: backport/include on the pull request that added it to main, or a maintainer pull request against the line.`);
|
||||
return;
|
||||
}
|
||||
throw e;
|
||||
@@ -374,7 +377,7 @@ jobs:
|
||||
}
|
||||
const content = Buffer.from(file.data.content, 'base64').toString('utf8');
|
||||
if (!content.includes(marker)) {
|
||||
core.setFailed(`Target base '${baseBranch}' has an old '${path}'. Backport the candidate-aware promotion pipeline before dispatch.`);
|
||||
core.setFailed(`Target base '${baseBranch}' has an old '${path}'. Bring the candidate-aware promotion pipeline to it before dispatch: backport/include on the pull request that added it to main, or a maintainer pull request against the line.`);
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -603,10 +606,11 @@ jobs:
|
||||
tar -xzf "$syft_dir/syft.tar.gz" -C "$syft_dir" syft
|
||||
rm -f "$syft_dir/syft.tar.gz"
|
||||
echo "$syft_dir" >> "$GITHUB_PATH"
|
||||
# cosign is the version docs/release.md asks users for, so the job checks
|
||||
# its own signature the way they will. The action verifies that binary
|
||||
# against cosign's keyless and KMS bundles with its digest-pinned
|
||||
# bootstrap cosign. Bump with tags.yaml; the contract test holds them equal.
|
||||
# cosign is the version docs/release/signatures-and-sbom.md asks users
|
||||
# for, so the job checks its own signature the way they will. The action
|
||||
# verifies that binary against cosign's keyless and KMS bundles with its
|
||||
# digest-pinned bootstrap cosign. Bump with tags.yaml; the contract test
|
||||
# holds them equal.
|
||||
- name: Set up cosign
|
||||
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
|
||||
with:
|
||||
@@ -1044,9 +1048,10 @@ jobs:
|
||||
PR_STATE="$(gh pr view "${STABLE_BRANCH}" --json state --jq .state 2>/dev/null || echo "")"
|
||||
if [ "${PR_STATE}" = "OPEN" ]; then
|
||||
# Refresh only while the body is still the one this workflow wrote.
|
||||
# docs/release.md tells reviewers to paste upstream breaking-change
|
||||
# notes into this body before merge; a re-dispatch that overwrites it
|
||||
# would destroy that with no trace and no way to recover it.
|
||||
# docs/release/pre-release-checklist.md tells reviewers to paste
|
||||
# upstream breaking-change notes into this body before merge; a
|
||||
# re-dispatch that overwrites it would destroy that with no trace
|
||||
# and no way to recover it.
|
||||
CURRENT="$(gh pr view "${STABLE_BRANCH}" --json body --jq .body 2>/dev/null || echo "")"
|
||||
if [ "${CURRENT}" = "${BODY}" ]; then
|
||||
echo "PR already open for ${STABLE_BRANCH} with an identical body; nothing to do."
|
||||
|
||||
@@ -1348,28 +1348,6 @@ jobs:
|
||||
Resume from inside: `breakpoint resume`. Otherwise the breakpoint
|
||||
exits 10 minutes after the last SSH session disconnects.
|
||||
|
||||
# ▸ Tear down environment (always runs)
|
||||
#
|
||||
# Both halves, and in this order: the compose project owns the nodes and
|
||||
# the zpools outlive it. A leftover pool makes the next run on a reused
|
||||
# host adopt storage that already carries a dead cluster's volumes.
|
||||
- name: Tear down sandbox
|
||||
if: always()
|
||||
run: |
|
||||
# Loud, but never the job's verdict. A failed teardown means a dirty
|
||||
# host, and this runner is discarded after the job, so there is no next
|
||||
# run to protect -- while failing the job here posts the required
|
||||
# "E2E Tests" status as a failure that no reader can tell apart from a
|
||||
# real one. The annotation is what a reused host (a developer machine)
|
||||
# needs; `delete-cluster-container` returning non-zero is not evidence
|
||||
# about the product.
|
||||
cleanup_rc=0
|
||||
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME delete-cluster-container || cleanup_rc=$?
|
||||
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME delete || true
|
||||
if [ "$cleanup_rc" -ne 0 ]; then
|
||||
echo "::error title=sandbox teardown::teardown exited $cleanup_rc; the host may hold a stale compose project or zpool"
|
||||
fi
|
||||
|
||||
# Report the required "E2E Tests" gate as a commit status rather than relying
|
||||
# on a job named "E2E Tests". A required *job* that is skipped counts
|
||||
# as PASSED by branch protection — desired for docs-only PRs, but the exact
|
||||
|
||||
@@ -39,8 +39,7 @@ jobs:
|
||||
security/triage-needed,security/in-progress,
|
||||
triage/accepted,kind/regression
|
||||
exempt-pr-labels: >-
|
||||
lifecycle/frozen,do-not-merge/hold,
|
||||
kind/backport,kind/backport-previous
|
||||
lifecycle/frozen,do-not-merge/hold
|
||||
days-before-stale: 60
|
||||
days-before-close: 14
|
||||
operations-per-run: 100
|
||||
|
||||
@@ -556,8 +556,8 @@ jobs:
|
||||
tar -xzf "$syft_dir/syft.tar.gz" -C "$syft_dir" syft
|
||||
rm -f "$syft_dir/syft.tar.gz"
|
||||
echo "$syft_dir" >> "$GITHUB_PATH"
|
||||
# cosign is the version docs/release.md asks users for, as in
|
||||
# promote-rc.yaml. Bump with it.
|
||||
# cosign is the version docs/release/signatures-and-sbom.md asks users
|
||||
# for, as in promote-rc.yaml. Bump with it.
|
||||
- name: Set up cosign
|
||||
if: needs.build-amd64.outputs.release_exists == 'false'
|
||||
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
|
||||
|
||||
+2
-1
@@ -11,7 +11,7 @@ _repos
|
||||
# root-level binary waiting to happen — each ~80 MB, and nothing else here
|
||||
# matched them, leaving `git add .` one keystroke from committing one.
|
||||
/api-gate
|
||||
/backport-audit
|
||||
/backport-batch
|
||||
/backup-controller
|
||||
/backupstrategy-controller
|
||||
/check-readiness
|
||||
@@ -23,6 +23,7 @@ _repos
|
||||
/flux-shard-operator
|
||||
/kubeovn-plunger
|
||||
/lineage-controller-webhook
|
||||
/observability-proxy
|
||||
/securitygroup-controller
|
||||
|
||||
# User-specific stuff
|
||||
|
||||
@@ -16,7 +16,11 @@ This file provides structured guidance for AI coding assistants and agents worki
|
||||
|
||||
- **Release creation** (e.g., "create release", "prepare release", "tag release", "make a release")
|
||||
- Read: [`releasing.md`](./docs/agents/releasing.md)
|
||||
- Action: Read the file and follow the referenced release process in `docs/release.md`
|
||||
- Action: Read the file and follow the referenced release process: [`docs/release.md`](./docs/release.md) indexes one page per topic under `docs/release/`
|
||||
|
||||
- **Backports, patch content** (e.g., "backport this fix", "will this reach 1.6", "what goes into the next patch", "open a PR against release-X.Y")
|
||||
- Read: [`backports.md`](./docs/release/backports.md)
|
||||
- Action: Read the entire file. Backports are automatic: a fix merged to `main` with a `fix` title, or a `security` one for a fix that comes through a security advisory, reaches the release lines the batch runs for, the newest released lines and any line in its rc window, so never open a backport PR by hand. A pull request against `release-*` from outside the team is closed automatically when it is opened; a maintainer keeps a fix with no counterpart on `main` by reopening it
|
||||
|
||||
- **Image references, tags and digests** (e.g., "why is this image tagged rc", "add an image to a package", "change how an image is stamped", "promotion/nightly missed an image", "where do image refs live")
|
||||
- Read: [`image-refs.md`](./docs/agents/image-refs.md)
|
||||
|
||||
@@ -88,7 +88,7 @@
|
||||
|
||||
### Rollout, Upgrade and Rollback Planning
|
||||
|
||||
* **Compatibility with Kubernetes & frequency.** Cozystack ships a curated, tested Kubernetes version per release; tenant clusters are Certified Kubernetes following upstream skew/support. Platform upgrades are delivered as new Cozystack releases. The authoritative list of what has shipped is the [GitHub Releases page](https://github.com/cozystack/cozystack/releases); which release lines are still receiving security fixes is defined as a rolling rule in [SECURITY.md](https://github.com/cozystack/cozystack/blob/main/SECURITY.md): the latest stable minor and the two preceding minors are supported, older `1.x` minors receive critical and upgrade-blocking fixes at maintainer discretion, `v0.41.x` is end of life and anything earlier is unsupported. Specific version numbers are deliberately not restated here, because they drift with every release. Note that being in a supported line does not entail receiving every fix: the project has no documented rule for which supported lines a given fix is backported to, and that is recorded in `SECURITY.md` rather than presented as a policy.
|
||||
* **Compatibility with Kubernetes & frequency.** Cozystack ships a curated, tested Kubernetes version per release; tenant clusters are Certified Kubernetes following upstream skew/support. Platform upgrades are delivered as new Cozystack releases. The authoritative list of what has shipped is the [GitHub Releases page](https://github.com/cozystack/cozystack/releases); which release lines are still receiving security fixes is defined as a rolling rule in [SECURITY.md](https://github.com/cozystack/cozystack/blob/main/SECURITY.md): the two latest minor releases are supported, older `1.x` minors receive critical and upgrade-blocking fixes at maintainer discretion, `v0.41.x` is end of life and anything earlier is unsupported. Specific version numbers are deliberately not restated here, because they drift with every release. Note that being in a supported line does not entail receiving every fix: which fixes reach a supported line is the [backport policy](https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md), which keeps breaking changes, platform migrations and the like for the next minor.
|
||||
* **Upgrade procedure** ([upgrade docs](https://cozystack.io/docs/v1.6/operations/cluster/upgrade/)): run the troubleshooting/health checklist; annotate the `cozy-system` namespace and `cozystack-version` ConfigMap with `helm.sh/resource-policy=keep`; `helm upgrade cozystack oci://ghcr.io/cozystack/cozystack/cozy-installer --version X.Y.Z` (**do not** use `--reuse-values` — the chart pins OCI repo/package versions); verify pods and `kubectl get hr -A`.
|
||||
* **Rollback.** GitOps/Helm-managed with versioned releases; roll back by targeting the previous release (previous HelmRelease revisions) via the same mechanism. Stateful data (databases, VM disks) persists on replicated storage across control-plane rollbacks. **No dedicated rollback runbook is published**, and version-skip constraints for downgrades are not documented; the [upgrade docs](https://cozystack.io/docs/v1.6/operations/cluster/upgrade/) currently cover the forward path only.
|
||||
* **How a rollout/rollback can fail & impact.** Risks: incompatible CRD schema change, storage/CNI component upgrades requiring node reboots (on the Talos path), or a database-operator major bump. Impact is scoped to the component; running tenant workloads generally continue, though storage/CNI upgrades can transiently affect networking or volume attachment during node rolls. `[maintainers: confirm whether upgrades trigger Talos node reboots]`
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
.PHONY: manifests assets prepare-env prepare-env-container unit-tests helm-unit-tests bats-unit-tests bats-unit-files-check bats-posix-compat-tests print-bats-unit-files print-bats-posix-compat-files print-bats-jobs rd-presets-check migrations-target-check test test-controllers test-controllers-envtest test-backport-audit preflight
|
||||
.PHONY: manifests assets prepare-env prepare-env-container unit-tests helm-unit-tests bats-unit-tests bats-unit-files-check bats-posix-compat-tests print-bats-unit-files print-bats-posix-compat-files print-bats-jobs rd-presets-check migrations-target-check test test-controllers test-controllers-envtest test-backport-batch preflight
|
||||
|
||||
include hack/common-envs.mk
|
||||
|
||||
@@ -17,6 +17,7 @@ build: build-deps
|
||||
make -C packages/system/cozystack-api image
|
||||
make -C packages/system/cozystack-controller image
|
||||
make -C packages/system/backup-controller image
|
||||
make -C packages/system/observability-proxy image
|
||||
make -C packages/system/backupstrategy-controller image
|
||||
make -C packages/system/lineage-controller-webhook image
|
||||
make -C packages/system/migration-controller image
|
||||
@@ -119,7 +120,7 @@ test:
|
||||
make -C packages/core/testing apply
|
||||
make -C packages/core/testing e2e
|
||||
|
||||
unit-tests: helm-unit-tests bats-unit-tests bats-posix-compat-tests go-unit-tests go-module-tests rd-presets-check test-check-readiness test-backport-audit migrations-target-check
|
||||
unit-tests: helm-unit-tests bats-unit-tests bats-posix-compat-tests go-unit-tests go-module-tests rd-presets-check test-check-readiness test-backport-batch migrations-target-check
|
||||
|
||||
helm-unit-tests:
|
||||
hack/helm-unit-tests.sh
|
||||
@@ -194,9 +195,9 @@ test-controllers-envtest: $(SETUP_ENVTEST)
|
||||
test-check-readiness:
|
||||
go test ./test/check-readiness/ -count=1
|
||||
|
||||
# ./cmd/... is excluded from go-unit-tests, so the audit needs its own target.
|
||||
test-backport-audit:
|
||||
go test ./cmd/backport-audit/ -count=1
|
||||
# ./cmd/... is excluded from go-unit-tests, so the batch tool needs its own target.
|
||||
test-backport-batch:
|
||||
go test ./cmd/backport-batch/... -count=1
|
||||
|
||||
# Unit discovery is one level deep. Live-cluster files need an invocation in
|
||||
# packages/core/testing; hack/bats-runner-coverage.bats checks that boundary.
|
||||
@@ -294,7 +295,7 @@ bats-posix-compat-tests:
|
||||
# own too; hack/unit-lane-no-cluster.bats fails when one is missing.
|
||||
NO_CLUSTER_TARGETS := unit-tests test-controllers helm-unit-tests bats-unit-tests \
|
||||
bats-posix-compat-tests go-unit-tests go-module-tests rd-presets-check \
|
||||
test-check-readiness test-backport-audit migrations-target-check
|
||||
test-check-readiness test-backport-batch migrations-target-check
|
||||
$(NO_CLUSTER_TARGETS): export KUBECONFIG := /dev/null
|
||||
$(NO_CLUSTER_TARGETS): export KUBERNETES_SERVICE_HOST :=
|
||||
|
||||
|
||||
+5
-5
@@ -8,7 +8,7 @@ Cozystack integrates and ships many upstream cloud native components. If you bel
|
||||
|
||||
## Supported Versions
|
||||
|
||||
The Cozystack project maintains three release lines at a time: **the latest stable minor and the two minors preceding it.** That is a rolling rule rather than a list of version numbers, so it does not go stale between releases — read it against the GitHub Releases page to see which concrete lines it currently names:
|
||||
The Cozystack project supports two release lines at a time: **the two latest minor releases**, the latest stable minor and the one before it. When a new minor ships its first stable release it becomes the latest, and the older of the two drops out. That is a rolling rule rather than a list of version numbers, so it does not go stale between releases — read it against the GitHub Releases page to see which concrete lines it currently names:
|
||||
|
||||
<https://github.com/cozystack/cozystack/releases>
|
||||
|
||||
@@ -16,16 +16,16 @@ Note that the rule is keyed to release lines, not to the existence of a `release
|
||||
|
||||
| Version line | Status | Notes |
|
||||
| --- | --- | --- |
|
||||
| Latest stable minor | Supported | Current stable release line. Security fixes land here first, and patch releases are cut from it. |
|
||||
| The two minors preceding the latest | Supported | Receive security and important maintenance fixes, backported from the current line. |
|
||||
| Latest stable minor | Supported | Current stable release line, the first supported line a fix reaches. Patch releases are cut from it. |
|
||||
| The minor before the latest | Supported | Receives the same security and maintenance fixes, backported once the latest line carries them. |
|
||||
| Older `1.x` minors | Limited support | Critical and upgrade-blocking fixes may be backported at maintainer discretion. Adopters are encouraged to move to a supported line. |
|
||||
| `v0.41.x` | End of life | The final pre-v1 line. It received patch releases through March 2026 during the v0 to v1 transition and no longer does; no further fixes are planned. |
|
||||
| Everything before `v0.41` | Not supported | Long superseded. Upgrade to a `1.x` line. |
|
||||
| `alpha`, `beta`, `rc` releases | Not supported | Pre-release builds are for testing and evaluation only. |
|
||||
|
||||
Being in a supported row does not mean a line receives every fix. **The project does not have a documented rule for choosing which supported lines a given fix is backported to** — that decision is taken per fix by the maintainers, weighing severity against the risk and effort of the backport. This is stated rather than dressed up as a policy because no such policy exists; adopters who need a specific fix on a specific line should ask.
|
||||
Being in a supported row does not mean a line receives every fix. Fixes reach the supported lines through the backport batch, by the rules in [`docs/release/backports.md`](docs/release/backports.md): a fix merged into `main` with a `fix` or `security` title goes onto both lines, unless the policy keeps it for the next minor, a breaking change or a platform migration for instance, and a maintainer can override that for a single fix. Adopters who need a specific fix on a specific line should ask.
|
||||
|
||||
Reporting a vulnerability against a line with limited support or at end of life is still welcome. We will confirm whether the issue also affects a supported line and fix it there; whether the fix is additionally backported to the older line is a per-fix maintainer decision on the same basis.
|
||||
Reporting a vulnerability against a line with limited support or at end of life is still welcome. We will confirm whether the issue also affects a supported line and fix it there; whether the fix is additionally backported to the older line is a per-fix maintainer decision.
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
|
||||
Generated
+1
-1
@@ -58,7 +58,7 @@ type ConfigSpec struct {
|
||||
// Users configuration map. Passwords are always auto-generated and stored in the `<release>-credentials` Secret; they cannot be set from values — read a user's password from that Secret. A `password` left over in values from before the field was removed is ignored by the render and draws an admission warning, but it is not inert on an upgraded release: the chart preserves whatever password is already in the Secret, which on the first upgrade is the value that was set before removal, so that value stays the live credential until it is rotated (dedicated rotation is tracked in cozystack/community#72). The live password lives only in the Secret.
|
||||
// +kubebuilder:default:={}
|
||||
Users map[string]User `json:"users,omitempty"`
|
||||
// Databases configuration map.
|
||||
// Databases configuration map. The PostgreSQL system databases `postgres`, `template0` and `template1` are refused.
|
||||
// +kubebuilder:default:={}
|
||||
Databases map[string]Database `json:"databases,omitempty"`
|
||||
// Backup configuration.
|
||||
|
||||
Generated
+2
-2
@@ -17,7 +17,7 @@ type Config struct {
|
||||
}
|
||||
|
||||
type ConfigSpec struct {
|
||||
// Number of Redis replicas.
|
||||
// Number of Redis replicas. Three Sentinels watch them, or a single one when `replicas` is 1, since a lone pod has no replica to fail over to.
|
||||
// +kubebuilder:default:=2
|
||||
Replicas int `json:"replicas"`
|
||||
// Explicit CPU and memory configuration for each Redis replica. When omitted, the preset defined in `resourcesPreset` is applied.
|
||||
@@ -51,7 +51,7 @@ type ConfigSpec struct {
|
||||
// Redis `min-replicas-max-lag`, in seconds: a replica whose last acknowledgement is older than this does not count towards `minReplicasToWrite`. Unset by default, which leaves the Redis default (10). Has no effect unless `minReplicasToWrite` is set. Must be at least 1 when `minReplicasToWrite` is set: Redis treats 0 as disabling the check, so the chart refuses it.
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
MinReplicasMaxLag *int `json:"minReplicasMaxLag,omitempty"`
|
||||
// Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the three Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.
|
||||
// Sentinel configuration. When `sentinel.resources` is empty and `sentinel.resourcesPreset` is unset, the Sentinel pods are sized like the Redis pods (`resources` and `resourcesPreset`). An empty `sentinel.resources` map counts as unset, as an empty top-level `resources` does.
|
||||
// +kubebuilder:default:={}
|
||||
Sentinel Sentinel `json:"sentinel,omitempty"`
|
||||
}
|
||||
|
||||
@@ -1,226 +0,0 @@
|
||||
# backport-audit
|
||||
|
||||
Answers one question, before a release or an rc is cut: did everything labelled for backport actually land on the release branch? It prints the URLs of the ones that did not, and exits non-zero, so it can gate the cut.
|
||||
|
||||
The manual alternative is `gh pr list --search "is:merged label:kind/backport"`, which lists every PR ever labelled and says nothing about whether the change reached the branch. Answering that by hand means reading the branch history per PR, which is where backports quietly go missing.
|
||||
|
||||
```console
|
||||
$ go run ./cmd/backport-audit release-1.5
|
||||
|
||||
=== release-1.5 === 43 candidate PRs: MISSING=2 pending=3 dropped=5 backported=33
|
||||
|
||||
MISSING -- labelled for this line, no trace of it here (2):
|
||||
https://github.com/cozystack/cozystack/pull/3449
|
||||
#3449 ci(release): move release validation to rc time — e2e lane, changelog button, promote gate, docs at promote
|
||||
label=kind/backport-previous author=myasnikovdaniil merged=2026-07-29 -- no backport PR, nothing on branch
|
||||
```
|
||||
|
||||
The URL is always the **original PR on main** — the thing you decide about. The evidence line names the backport PR when one exists. A labelled PR already on the branch is counted in the header and never listed, so silence under the outstanding headings means done. Two more sections, described in [Backports the labels do not account for](#backports-the-labels-do-not-account-for), start from the backport PRs on the branch instead of from labels: `DUPLICATE`, which is outstanding, and `UNLABELLED`, which is informational.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
go run ./cmd/backport-audit # the two newest release lines
|
||||
go run ./cmd/backport-audit release-1.5 # one line
|
||||
go run ./cmd/backport-audit release-1.6 release-1.5 --json
|
||||
```
|
||||
|
||||
Requires `git` and an authenticated `gh` on PATH. The audit does not modify GitHub or working-tree files. Unless `--no-fetch` is set, it updates local Git refs and objects while fetching.
|
||||
|
||||
| Flag | Meaning |
|
||||
|------|---------|
|
||||
| `--remote NAME` | Git remote holding the release branches (default `origin`) |
|
||||
| `--limit N` | Max merged PRs scanned per label spelling (default 1000, which is also the ceiling — see [Exit code](#exit-code)) |
|
||||
| `--no-fetch` | Skip `git fetch`; both main and release refs must be current so shared history stays out of backport evidence |
|
||||
| `--json` | Machine-readable output |
|
||||
| `--no-color` | Disable color (auto-disabled on a non-TTY and under `--json`) |
|
||||
|
||||
Arguments and flags may be given in any order.
|
||||
|
||||
## Exit code
|
||||
|
||||
`0` when nothing is outstanding, `1` when something is, `2` when the audit could not be completed. Outstanding means a `MISSING`, `partial`, `unverified`, `pending` or `dropped` verdict, or any `DUPLICATE` entry; `UNLABELLED` entries never move the exit code. That is the point of the tool, so it holds in `--json` mode too:
|
||||
|
||||
```bash
|
||||
go build ./cmd/backport-audit
|
||||
./backport-audit release-1.5
|
||||
case $? in
|
||||
0) echo "clean" ;;
|
||||
1) echo "do not cut yet" ;;
|
||||
*) echo "the audit itself failed; no answer yet" ;;
|
||||
esac
|
||||
```
|
||||
|
||||
Use the built binary (`go build` as above, or `go install ./cmd/backport-audit`) wherever the exit code matters. `go run` does not preserve it: any non-zero exit of the program comes back as `go run`'s own `1`, after an `exit status N` line on stderr, so under `go run` an audit that could not be completed looks exactly like one that found outstanding work. `go run` is fine for reading the report.
|
||||
|
||||
Exit `2` covers the cases where an answer cannot be trusted rather than merely being bad news, and a saturated listing is one of them: `gh` truncates at `--limit` in silence, and a truncated list does not make the audit partial, it makes it wrong — the PRs past the cut are reported nowhere and the exit code says clean. Each of the three listings is checked against its cap and fails instead, naming the cap to raise. The label listings go through GitHub search, which never returns more than 1000 results however far it is paginated, so they are checked against the lower of `--limit` and 1000: a `--limit` above 1000 cannot make a listing cut at 1000 pass for complete. A git read that a verdict depends on is the other case: when one fails, a merge commit missing from the local repository for instance, the audit stops with `2` instead of judging the PR without it.
|
||||
|
||||
## Statuses
|
||||
|
||||
| Status | Meaning | Action |
|
||||
|--------|---------|--------|
|
||||
| `in-branch` | The PR merged before the branch was cut, so its change is already there | none, no backport was ever needed |
|
||||
| `backported` | Every commit of the PR is on the branch, or, for a PR of a single commit, a backport PR merged | none |
|
||||
| `confirmed` | Would be `partial` or `unverified`, but a maintainer attested the backport complete on the merged backport PR; the report names who and links the comment | none |
|
||||
| `partial` | Some of the PR's commits are on the branch and some are not, listed with SHA and subject — a merged bot conflict draft, which stops at the first commit that did not apply, or a backport squashed by hand | backport the missing commits, or [attest the backport complete](#attesting-a-backport-complete) |
|
||||
| `unverified` | A backport PR merged, but none of the PR's several commits is on the branch under its own SHA, `-x` reference or subject — typically a backport squashed or reworded by hand | check it, then attest it complete or backport what is missing |
|
||||
| `pending` | A backport PR exists and is still open, including the drafts the bot opens with the conflict committed | merge it, or finish the draft |
|
||||
| `dropped` | A backport PR was closed unmerged, reported with whatever reason someone left on it | none if the reason still holds |
|
||||
| `MISSING` | No backport PR ever existed and nothing on the branch matches | cherry-pick it by hand, or drop it deliberately |
|
||||
|
||||
`dropped` is usually healthy — a maintainer deciding a fix does not apply to that line, e.g. because the feature it repairs never shipped there. The recorded reason is printed so the next release does not re-open the same investigation. A `dropped` entry reading `no reason recorded` is the one that needs a human.
|
||||
|
||||
## Backports the labels do not account for
|
||||
|
||||
The verdicts start from labels, so on their own they cannot see a backport of a PR nobody labelled. Two further sections start from the other side: every PR on the release branch, and the originals it names by the bot's head branch or in a `Backport of` phrase (see [How landing is established](#how-landing-is-established)). The header carries their counts after a `|`, e.g. `backported=42 | DUPLICATE=3 unlabelled=13`.
|
||||
|
||||
`UNLABELLED` lists each original that backport PRs on the branch claim although it is not a candidate for the branch, with every backport PR claiming it and that PR's state. Each entry also says what those PRs amount to: `backport PR merged` once one of them merged, `claimed here, not merged` while the only live one is still open, and `claimed here, closed unmerged` when every one was closed. A merged claim is only a merged PR, since an unlabelled original is not checked commit by commit; it reads `backport PR merged, confirmed by @login` when the PR carries the [attestation marker](#attesting-a-backport-complete). Most are hand backports of PRs that never carried a label, merged on purpose. An original that does carry a backport request, one that resolved to a different line at merge time, is listed with its labels. The section is informational and never moves the exit code: the gate answers whether everything labelled landed, and an unlabelled backport can only add to a branch, never leave a labelled change off it.
|
||||
|
||||
`DUPLICATE` lists each original claimed by more than one live backport PR, labelled or not:
|
||||
|
||||
| Kind | Meaning | Action |
|
||||
|------|---------|--------|
|
||||
| `open-twice` | Two or more backport PRs are open for it — typically the bot's conflict draft next to a hand backport, or a fork PR next to its reopening from a branch in this repository | merge one, close the rest |
|
||||
| `open-after-merge` | A backport already merged and another one is still open | close the leftover, or merge it if it is the second half of a split backport |
|
||||
|
||||
Both kinds make the audit exit `1`. Neither reaches a verdict, which judges the commits on the branch and names at most one open backport PR, and in both a human has to decide what happens to the open PR before the cut. A closed PR next to an open one is the normal shape of a redone backport and is not flagged, and neither are two merged ones, which are history the branch already carries.
|
||||
|
||||
```console
|
||||
DUPLICATE -- one original, more than one live backport PR (3):
|
||||
https://github.com/cozystack/cozystack/pull/3936
|
||||
#3936 fix(rabbitmq): right-size the default resources preset to s1.nano
|
||||
label=kind/backport -- a backport PR still open after another one merged
|
||||
backport #4372 OPEN (draft) https://github.com/cozystack/cozystack/pull/4372
|
||||
backport #4393 MERGED https://github.com/cozystack/cozystack/pull/4393
|
||||
|
||||
UNLABELLED -- backport PRs here for originals not labelled for this line (13, informational):
|
||||
https://github.com/cozystack/cozystack/pull/4280
|
||||
#4280 fix(kafka): keep the pre-delete hook's credentials as long as its Job
|
||||
backport PR merged
|
||||
backport #4421 CLOSED https://github.com/cozystack/cozystack/pull/4421
|
||||
backport #4456 MERGED https://github.com/cozystack/cozystack/pull/4456
|
||||
```
|
||||
|
||||
The titles of unlabelled originals are not in any listing the audit already makes, so they come from one GraphQL request covering the whole run. If it fails, the report goes out without them and says so on stderr; the URLs are derived locally and the verdicts and exit code do not depend on it.
|
||||
|
||||
## How a PR is matched to a release line
|
||||
|
||||
A `kind/backport` label does not name a branch. The audit selects merged PRs that currently carry a backport label, then assigns each to the release line selected at its merge time: `kind/backport` means the current line, and `kind/backport-previous` means the previous line. A label added after merge still makes the PR a candidate. [`backport.yaml`](../../.github/workflows/backport.yaml) also runs when a label is added and selects its target at that time, so a late label can make the audit and the bot disagree; see [Limits](#limits).
|
||||
|
||||
What "the current line" means changed once, so the audit carries both rules and picks by the PR's merge date:
|
||||
|
||||
| PR merged | rule `backport.yaml` was running | a line exists from |
|
||||
|-----------|----------------------------------|--------------------|
|
||||
| before 2026-08-03 | `getLatestRelease` | its first non-prerelease release publishing |
|
||||
| on or after 2026-08-03 | the two newest existing `release-X.Y` branches | its `release-X.Y` branch being pushed |
|
||||
|
||||
The cutover is a single moment because both halves landed in one push: 53a7fc4dc made cutting an rc freeze the line into `release-X.Y`, and 6ff5b98d5 pointed the bot at the branch list. Before that a release branch was created at promote time, so the newest branch and the newest published stable named the same line and the two rules could not disagree. After it they disagree for the length of every freeze window — `release-X.Y` is created when the first `vX.Y.0-rc.N` is cut, potentially weeks before `vX.Y.0` publishes, and everything labelled in between belongs to the line being stabilised. That window is exactly when a backport has to reach the branch, because the frozen branch is the only way into the release.
|
||||
|
||||
The audit estimates branch opening from the earliest `vX.Y.0-rc.N` tagged commit dated on or after the cutover. It ignores earlier tags individually; a line with no remaining tag falls back to its first stable release date. [`cut-prerelease.yaml`](../../.github/workflows/cut-prerelease.yaml) creates the branch at the tagged commit, but the commit timestamp is not the branch-push timestamp. PRs merged in that interval can be assigned to the new line before the bot could target it. The rc release's `publishedAt` is later still, after the build finishes.
|
||||
|
||||
Lines are then ranked by version and never by when they opened, matching the numeric descending sort the workflow applies to its branch list. The two orders come apart as soon as a freeze overlaps the previous line's stabilisation, and version order is also what keeps `release-1.10` above `release-1.9`.
|
||||
|
||||
The labels themselves were namespaced under `kind/` in 7b71053a0, which renamed them in place via label-sync aliases, so every historical PR reads back under the new name. The bare `backport` and `backport-previous` are still queried alongside them, because the org-level dosubot goes on applying the old spelling to new PRs — the same transitional allowance [`backport.yaml`](../../.github/workflows/backport.yaml) makes, and it retires at the same time. A PR found under either spelling is reported under the `kind/` one.
|
||||
|
||||
## How landing is established
|
||||
|
||||
Evidence is weighed per commit, because a commit is what a backport can lose:
|
||||
|
||||
1. **Reachability.** The PR's merge commit is reachable from the release branch, i.e. it merged before the branch was cut (or `main` was later merged in). Nothing was ever needed: `in-branch`.
|
||||
2. **A linked backport PR.** Found by the bot's head branch `backport-<N>-to-release-X.Y`, or by a `Backport of #N` reference in the body, which is what a hand-written backport carries. It never outranks the commits (item 3): a merged backport PR with one of the PR's commits missing is `partial`, because the bot's conflict drafts stop at the first commit that does not apply and drop the rest, so a merged draft looks finished at PR level. `MERGED` settles it alone only for a PR of a single commit — including on a `--no-fetch` run whose local refs predate the backport's merge. For a PR of several commits, none of them found and a merged backport PR is `unverified`. `OPEN` is `pending`; `CLOSED` is `dropped`.
|
||||
|
||||
A backport carrying several changes links to every original its `Backport of` phrase names: a list (`Backport of #3938 and #4280`, `Backport of #1, #2, and #3`), and the `Backport of #4253 to release-1.6, together with #3460` form used for a dependency pulled along. Only that phrase is read, so the issue a backport fixes or a CI run it cites further on is never taken for an original. A list counts in full only when it visibly ends — at the end of the line or the sentence, or where the `to release-X.Y` clause starts. One that runs on into anything else (`Backport of #10, #20 is not included`) may be saying something about its later items, so only its first reference counts. A reference qualified with this repository (`cozystack/cozystack#N`, any case) counts like a bare `#N`, and one qualified with any other repository is skipped, because reading `other/repo#20` as local #20 would let an unrelated backport vouch for whichever local PR carries that number.
|
||||
3. **The branch's own history, commit by commit.** Every commit the PR contributed has to be on the branch; merge commits and commits that change nothing are skipped, since the backport bot drops both. A commit counts when it is reachable, when an `-x` cherry-pick reference names it, or when a commit with its subject is there. Subject and `-x` evidence comes only from commits on the release branch that are not reachable from `<remote>/main`; shared history still counts for direct reachability. Subjects are matched one to one: two commits with the same subject, `fix tests` say, need two branch commits of that name, and a branch commit whose `-x` reference names any other commit than the PR's merge commit is evidence only through that reference. Nothing weaker is taken: not an `-x` reference to the PR's merge commit, which survives a cherry-pick later amended to drop a commit, and not matching lines, which an unrelated line of the same text satisfies. A wrong `backported` is the one answer a release gate must not give, while a false alarm costs a look and can be settled by an attestation. All commits is `backported`, some is `partial`. This is also what catches a hand backport nobody linked.
|
||||
|
||||
### Attesting a backport complete
|
||||
|
||||
Some complete backports cannot be shown commit by commit, most often a fork contributor's backport squashed into one commit that carries only the first commit's subject. A maintainer who has checked such a backport can say so on the merged backport PR, with a comment that consists of this single line and nothing else:
|
||||
|
||||
```text
|
||||
backport-audit: complete
|
||||
```
|
||||
|
||||
That turns the candidate's `partial` or `unverified` into `confirmed`, which does not fail the gate; the report names the author, links the comment, and keeps listing the commits nothing on the branch names.
|
||||
|
||||
The marker is an explicit attestation by whoever wrote it, never something the audit infers. It counts only when it is the whole comment: exactly `backport-audit: complete`, in that case, starting at the beginning of its line, with nothing but blank lines before it and nothing but whitespace after it. Any other text makes the comment something other than an attestation, so a sentence such as "do not post backport-audit: complete yet", a quote or a code block never counts, and neither does the marker with an explanation next to it; post the explanation as a separate comment. It counts only on a backport PR that merged, since an open or closed one put nothing on the branch to vouch for. And it counts only from someone with a hand in the repository: the comment's GitHub author association has to be `OWNER`, `MEMBER` or `COLLABORATOR`, and comments from automation are ignored whatever their association — logins ending in `[bot]`, and the review and dependency bots gh reports without that suffix, such as `coderabbitai`, `dependabot`, `renovate` and `copilot`. A review bot quoting the marker, or a contributor vouching for their own backport, confirms nothing.
|
||||
|
||||
## Machine-readable output
|
||||
|
||||
```bash
|
||||
# URLs of what never landed, ready to paste into a tracking issue
|
||||
go run ./cmd/backport-audit --json release-1.5 | jq -r '.[].candidates[] | select(.status=="MISSING") | .url'
|
||||
|
||||
# every open backport PR the audit links, labelled or not, to merge or close, across lines
|
||||
go run ./cmd/backport-audit --json release-1.6 release-1.5 \
|
||||
| jq -r '[.[] | (.candidates[], .unlabelled[]) | .backport_prs[] | select(.state=="OPEN") | .url] | unique[]'
|
||||
|
||||
# dropped items with the reason someone recorded
|
||||
go run ./cmd/backport-audit --json release-1.4 \
|
||||
| jq -r '.[].candidates[] | select(.status=="dropped") | "#\(.number)\t\(.reason)"'
|
||||
|
||||
# commits a partial backport still lacks, one per line
|
||||
go run ./cmd/backport-audit --json release-1.6 \
|
||||
| jq -r '.[].candidates[] | select(.status=="partial") | "#\(.number)\t\(.missing_commits[] | "\(.oid[0:9]) \(.subject)")"'
|
||||
|
||||
# open backport PRs competing for one original, to settle before the cut
|
||||
go run ./cmd/backport-audit --json release-1.6 \
|
||||
| jq -r '.[].duplicates[] | "#\(.number) \(.kind): \([.backport_prs[] | select(.state=="OPEN") | .url] | join(" "))"'
|
||||
```
|
||||
|
||||
Output is an object keyed by branch, each holding three arrays: `candidates`, `unlabelled` and `duplicates`, each always present and possibly empty. `partial`, `unverified` and `confirmed` candidates carry `missing_commits`, each an `oid` and a `subject`, and a `confirmed` one also carries `confirmation`: `by`, `url` and `backport_pr`. An `unlabelled` record carries the same `confirmation` when its merged backport PR has the marker. A `candidates` record:
|
||||
|
||||
```json
|
||||
{
|
||||
"number": 3963,
|
||||
"title": "fix(clickhouse): scheme the backup S3_ENDPOINT on the system-bucket flow",
|
||||
"url": "https://github.com/cozystack/cozystack/pull/3963",
|
||||
"author": "androndo",
|
||||
"mergedAt": "2026-08-28T11:05:15Z",
|
||||
"label": "kind/backport",
|
||||
"backport_prs": [{"number": 3987, "state": "OPEN", "url": "..."}],
|
||||
"status": "pending",
|
||||
"evidence": "backport PR #3987 open"
|
||||
}
|
||||
```
|
||||
|
||||
An `unlabelled` record, where `labels` lists the backport requests the original carries for other lines and is empty when it carries none:
|
||||
|
||||
```json
|
||||
{
|
||||
"number": 4280,
|
||||
"title": "fix(kafka): keep the pre-delete hook's credentials as long as its Job",
|
||||
"url": "https://github.com/cozystack/cozystack/pull/4280",
|
||||
"labels": [],
|
||||
"backport_prs": [
|
||||
{"number": 4421, "state": "CLOSED", "url": "..."},
|
||||
{"number": 4456, "state": "MERGED", "url": "..."}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
A `duplicates` record, where `label` is the request the original was audited under on this line and is empty when it is not a candidate here:
|
||||
|
||||
```json
|
||||
{
|
||||
"number": 4254,
|
||||
"title": "feat(kubevirt): expose migration configuration through platform values",
|
||||
"url": "https://github.com/cozystack/cozystack/pull/4254",
|
||||
"label": "kind/backport",
|
||||
"kind": "open-twice",
|
||||
"backport_prs": [
|
||||
{"number": 4322, "state": "OPEN", "url": "...", "draft": true},
|
||||
{"number": 4339, "state": "OPEN", "url": "..."},
|
||||
{"number": 4402, "state": "OPEN", "url": "..."}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Limits
|
||||
|
||||
A `MISSING`, `partial` or `unverified` verdict is a prompt to check, not proof of absence. A hand backport squashed or reworded without `-x` reads as `partial` or `unverified` until someone attests it complete, and as `MISSING` when nothing links it to its original; confirm at diff level before redoing the work. And the audit reports *that* something is missing, never *why* — a run of `MISSING` entries clustered in time usually means the bot itself was failing during that window, which is worth checking before cherry-picking them one by one.
|
||||
|
||||
The audit reads labels as they stand when it runs. A label added afterwards is seen only by the next run, and until then its PR is either absent or, if someone already backported it, listed as `UNLABELLED` rather than audited, so run it right before the cut rather than once at the start of the day.
|
||||
|
||||
A label is always read against the release lines as they stood when the PR merged, however much later the label was added, because nothing the audit lists records when that happened. A PR merged on 2026-07-01 and labelled `kind/backport` a month later, after `v1.6.0` published, is audited as a `release-1.5` candidate. [`backport.yaml`](../../.github/workflows/backport.yaml) runs again when the label is added and resolves its targets at that moment, so it opens the backport against `release-1.6`. When the line has moved on between the merge and the label, the two therefore disagree: the audit expects the change on the merge-time line and reports it `MISSING` there unless it arrived some other way, and lists the bot's backport under `UNLABELLED` on the newer one. `TestCandidateLabel` pins the audit's side of this.
|
||||
|
||||
A backport PR that names its original neither by the bot's head branch nor in a `Backport of` phrase links to nothing, so it is invisible to both cross-checks: it is not listed as `UNLABELLED` and does not count towards a `DUPLICATE`. For a labelled original the branch history can still prove the change landed, through an identical commit subject or an `-x` cherry-pick reference. For an unlabelled one those are not read at all, so a backport of an unlabelled PR that only a cherry-pick trailer points to, or nothing does, appears nowhere in the report.
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,128 @@
|
||||
# backport-batch
|
||||
|
||||
Builds the automatic backport batch for a release line and gates a patch cut on it. What goes into a batch is decided by the rules of the [backport policy](../../docs/release/backports.md) alone; this page describes how the tool applies them and does not restate them.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
go build ./cmd/backport-batch
|
||||
./backport-batch run --line release-1.7 [--onto SHA] [--date YYYY-MM-DD] [--no-workflow-files] [--push] [--policy REF]
|
||||
./backport-batch run --line release-1.7 --continue [--no-workflow-files] [--push] [--policy REF]
|
||||
./backport-batch run --line release-1.7 --defer 4321[,4322] --reason "fails the e2e of the app" [--push] [--policy REF]
|
||||
./backport-batch run --line release-1.7 --drop 4321[,4322] --reason "builds on the 1.8 storage rework" [--push] [--policy REF]
|
||||
./backport-batch run --line release-1.7 --undrop 4321[,4322] [--push] [--policy REF]
|
||||
./backport-batch check --line release-1.7 [--cut] [--policy REF]
|
||||
./backport-batch fold --line release-1.7 --original 4321 --commit SHA [--policy REF]
|
||||
./backport-batch verify --line release-1.7 [--head SHA]
|
||||
./backport-batch notify --line release-1.7 [--mention org/team|login] [--push]
|
||||
./backport-batch report --pr 5246 [--id ID] [--json]
|
||||
./backport-batch lines [--policy REF]
|
||||
```
|
||||
|
||||
[`hack/backport-batch-scheduled.sh`](../../hack/backport-batch-scheduled.sh) runs `lines`, then `run --continue --push`, `check` and `notify` for each line; [`backport-batch.yaml`](../../.github/workflows/backport-batch.yaml) runs it on schedule as the cozystack-ci App, with `--no-workflow-files`, which it hands to `run`.
|
||||
|
||||
Requires `git` and an authenticated `gh` on PATH, and a remote called `origin`. Every command but `notify` and `report` fetches `origin` first; `notify` reads only GitHub, and `report` reads GitHub and, for a report that keeps drops, the clone's history. The batch runs only for the lines `lines` prints.
|
||||
|
||||
`run`, `check`, `fold` and `lines` read the `backport-policy` block of the policy page from `origin/main`, or from the ref `--policy` names, which is how a change to the page is tried before it merges; `verify` and `notify` do not read it and take no `--policy`. A line the block lists takes nothing merged into `main` before the day it gives, unless the original carries `backport/include`; any other line starts at its fork point. A page without exactly one readable block stops the run with exit 2.
|
||||
|
||||
Build the binary wherever an exit code matters: `go run` turns any non-zero exit into its own `1`, so under it a check that could not be completed looks the same as one that found something missing.
|
||||
|
||||
### run
|
||||
|
||||
Switches the current working tree to `backport-batch/<line>-<yyyymmdd>`, created at `origin/<line>`, and picks onto it one original at a time, in the order they merged into `main`, as one unit each on the branch's first-parent line. A pull request merged into `main` with a merge commit `M` goes on as its own commits, `git rev-list --reverse --topo-order --no-merges M^1..M^2`, each picked with `git cherry-pick -x --signoff`, so each keeps its author, message and sign-offs and gains the `(cherry picked from commit …)` line; the merges that brought `main` into its branch are left out, since `main` already has what they brought. A merge commit then joins the picks to the batch: its first parent is the batch before the original, its second the last pick, and its message is `M`'s with a paragraph of its own, `Backported-from: M`. A pull request squash-merged into `main` goes on as one pick. When any commit of an original conflicts, the pick is aborted, the branch goes back to where it was before that original, and the original is recorded with the commit that failed and its files; the run goes on with the next one. A commit the branch already carries is skipped as empty, and an original none of whose commits changes anything is reported as carried. The tree has to be clean. A local branch of that name is reset to where the run starts; when that drops commits only it has, the run first prints its tip and the `git branch` command that restores them, and so does `--defer`. Signing follows your git configuration; hooks are off for the picks and the merge, so each commit carries its message as git writes it.
|
||||
|
||||
The pull request body goes to stdout, built from the line's own `.github/PULL_REQUEST_TEMPLATE.md`: the originals on the branch, each with how many commits it brought and which of them differ from the commit they were picked from by `git patch-id`, a resolution or a pick git had to fit, the deferred conflicts with the commit that failed and its files, the skipped pull requests counted by rule with the lists collapsed, the originals' release notes, and two blocks of JSON: the [line's record](#the-lines-record-and-reports) in `backport-batch`, which later runs, `check` and `notify` act on, and the run's report in `backport-batch-report`. The title is `fix(backport): batch for <line>, <date>`. A body that would pass GitHub's 65536-byte limit leaves out its longest skipped lists, and the run prints them on stderr instead; one that what it has to carry still puts over the limit is refused before anything is pushed.
|
||||
|
||||
`--onto SHA` continues a batch already pushed instead of starting a new one. SHA has to be the head the remote has for a batch branch of the line right now, read with `git ls-remote`, so a head someone pushed onto since is refused. What that branch carries stays, and only what it does not carry yet is picked on top, so the next push is a fast-forward. `--continue` takes SHA from the open batch pull request, and starts a new batch when none is open, which is what the scheduled run does.
|
||||
|
||||
`--no-workflow-files` is for a token that may not push a change to `.github/workflows/`, which the scheduled run's App token may not. An original that changes anything there is not picked: the body lists it under "Waiting for a maintainer" with those files, and the `backport-batch` block records it with `"workflows": true`, so `check`, `notify` and a run that continues the batch know it waits. It is never a conflict. A run without the flag, a maintainer's with their own token, picks it like any other original, onto the open batch when it continues one.
|
||||
|
||||
`--defer N[,N] --reason TEXT` takes originals out of the open batch pull request, usually the ones a failing check was pinned on. It builds the batch branch again off `origin/<line>` without their units: while the batch still sits on the line's head, the units before the first one it drops stay as they are, and every later one, a pushed resolution included, is picked again in its own shape, a merge's picks one by one and then the merge with its message. When dropping one leaves a later commit that no longer applies, the run stops and names that commit's original, to defer as well. It adds no new original. It also takes out an original that got `backport/skip` after the batch picked it, which a run never removes on its own and the body lists as still on the branch. The body records each deferred original with the reason in the `backport-batch` block; a run that continues this batch keeps them out, and the next batch tries them again. A batch that carries a merge commit other than an original's, as GitHub's update-branch button makes, is refused, since picking such a merge again would lose what it resolved. When it takes out everything the batch carries, there is nothing left to push, and `--push` closes the batch pull request instead, as [the line's record](#the-lines-record-and-reports) describes.
|
||||
|
||||
`--drop N[,N] --reason TEXT` drops originals from the line: where `--defer` holds one out of this batch, a drop keeps it out of every later batch of the line too, until `--undrop N[,N]` withdraws it. A drop is recorded in the [line's record](#the-lines-record-and-reports), so both need an open batch pull request that carries one, and exit 2 without it; once a batch merges, its drops are carried into the next batch's record. A dropped original the open batch carries is taken off it as `--defer` would, and the rebuild is pushed the same way; any other drop or undrop changes only the record, so the batch stays as it is even when the line moved since. An undropped original is missing until the next run tries it. A drop names the original's merge commit on `main` as well as its number, and a record whose drop does not match what that merge commit merged stops every command with exit 2. An original the line already carries cannot be dropped, and neither can a number that merged into `main` before the line branched. The order is the rules', then the drop: what `backport/skip`, rules 2 to 4 or the line itself decide comes first, with the drop kept on record beside it, and `backport/include` does not lift a drop. The body lists dropped originals under their own heading among the skipped ones, `check` prints a note for each, and `fold` refuses one.
|
||||
|
||||
`--push` pushes the branch, refusing anything but a fast-forward of what the remote has, then opens the draft pull request against the line with `gh pr create --draft`, or updates the open batch pull request with `gh pr edit`, and prints its URL on stderr. On the open batch it first posts the run's report as a comment, so a body never names a report nobody can read; a new batch carries the report in the body it opens with, and the comment follows. Before every write to the pull request it reads the pull request again, and stops with exit 2 once it is merged or closed. When the branch it refuses to push over has no pull request open on it, as a run that stopped between its push and opening the pull request leaves it, the refusal names the `run --onto <its head> --push` that continues it. While a batch pull request is open on another branch it refuses, so a line never has two. It never pushes to or edits a batch pull request whose body has no `backport-batch` block either: someone wrote that body by hand, and the batch is theirs to merge or close. After `--defer` alone it replaces the branch, with `--force-with-lease=refs/heads/<branch>:<head>`, where head is what `git ls-remote` reads for the branch and has to be the head the rebuild started from: a commit pushed since is never dropped, by the tool's check or, if one lands between that check and the push, by the remote. A record that says the batch is being closed is acted on before anything else, and only by `run --push`.
|
||||
|
||||
### check
|
||||
|
||||
Exits 1 and prints the URL of every pull request the rules take that is neither on the line nor on the open batch pull request, 0 when there is none, and 2 when it cannot tell. A pull request that the [line's record](#the-lines-record-and-reports) lists as deferred, a conflict or one taken out with `--defer`, prints as a warning and does not fail the check: deferring is a maintainer's call. That holds once the batch that recorded it merged or closed too. So does one the open batch lists as waiting for a maintainer because it changes workflows; once that batch is gone, nothing lists it any more, and it is missing. One a maintainer dropped from the line prints as a note, `#N is dropped from <line> on #M (<reason>)`.
|
||||
|
||||
`--cut` makes it the gate before a patch release or an rc is cut from a line. A cut tags the line, so a pull request that is only on the open batch pull request fails it too, with one line each: `#N is only on open batch PR #M, merge it before the cut`. One that waits there for a maintainer fails it as well, since nobody decided to leave it out, also when it is all the batch has left: bring it over, or take it out with `run --defer` and a reason. Everything else is as without it, and a cut that passes says `no change the policy takes is missing before the cut`. The scheduled run checks without `--cut`, since its batch is open by design.
|
||||
|
||||
An original with `backport/skip` that the open batch picked before the label came prints a warning in either mode, naming the batch and `run --defer`, which takes it out; it fails neither.
|
||||
|
||||
### fold
|
||||
|
||||
Checks a hand-made resolution of a deferred conflict before it is pushed onto the batch, in the shape the run gives an original. For a merged pull request `--commit` is a merge whose first parent is the head of the open batch pull request and whose message names the original in exactly one `Backported-from:` trailer; its second parent ends a straight line from that head of picks of the pull request's own commits, in their order, each ending in the `(cherry picked from commit …)` line of the commit it came from, none claiming another commit of the same original, a commit of another pull request never among them. For a squash-merged one it is a single commit on that head whose `(cherry picked from commit …)` line names the original's commit on `main`. Either has to change something, and only files the original changes. The [conflict task](../../docs/release/backport-tasks/resolve-conflict.md) gives the commands. fold prints `ok` and exits 0, or prints why not and exits 1. It never commits or pushes; push one checked resolution at a time, since the next one has to sit on the new head.
|
||||
|
||||
### verify
|
||||
|
||||
Holds every unit of a batch to the shape fold checks for one, the clean picks as well as resolutions: it walks the batch's first-parent line from `origin/<line>` to `--head`, or to the head of the open batch pull request, and requires each unit to be one original, merged into `main` after the line branched, in that shape, and no original twice. A commit that carries no original fails it, and so does a merge of the line into the batch. It prints one line per unit, and exits 0 when all pass, 1 when one does not, and 2 when it cannot tell. With `--head` it reads git alone, nothing from GitHub. fold runs only where someone resolved a conflict and chose to run it; [`pr-labeler.yaml`](../../.github/workflows/pr-labeler.yaml) runs verify on every push to a batch pull request, from a build of `main`, with the batch head fetched as data.
|
||||
|
||||
### notify
|
||||
|
||||
Reads the [line's record](#the-lines-record-and-reports) first, as the other commands do, and exits 2 when it cannot read it or when it says the open batch is being closed, with a batch open or not. Then it reads the open batch pull request of the line, trusted as described below, and prints one line for each thing that waits on it: a batch open for three days or more, the shortest gap between two scheduled runs, so one the previous run already saw; the conflicts it deferred; and the originals waiting for a maintainer because they change workflows. Nothing open prints nothing.
|
||||
|
||||
With `--push` it writes the same into one comment on the batch pull request: the conflicts, each to be resolved as the [conflict task](../../docs/release/backport-tasks/resolve-conflict.md) describes, and the originals that wait because they change workflows.
|
||||
|
||||
The comment starts with `<!-- backport-batch:notify -->` and is one per pull request: a later run edits it in place rather than adding another, and once nothing waits it says so. Only a comment whose author the batch trusts counts as it, so one someone else wrote with the same first line is neither edited nor believed. It mentions `--mention`, an `org/team` or a login, with or without `@`, only while something waits. A failed batch CI is not among what it reports: the run has no trigger on the batch's CI finishing.
|
||||
|
||||
### report
|
||||
|
||||
Prints a report a batch pull request keeps: the one its body carries, or the one `--id` names. It reads the report comments the batch trusts and falls back to the body, recomputes the report's ID from what it says, and exits 2 when a trusted comment does not match its ID, when a report says anything a run does not write or is for another line than the pull request's, or when the pull request keeps no such report. A report comment counts only when its author is one the batch trusts with a record, so anyone else's comment, whatever ID it carries, is not read. It prints what the classification was made of, whether the pull request's head or body shows the report was published, which it calls unconfirmed when neither does, and one line per original with its category and what the report recorded about it; `--json` prints the report as stored. A report whose comment the run never got to post is printed from the body, with a note that the next `run --push` posts it, or, on a merged or closed batch, that it is kept in the body only. It writes nothing. It reads GitHub, and for a report that keeps drops also the clone's history, without fetching: each drop is bound to `main` as the record's drops are, over the heads of `main` and the line the report was made from, so a drop of an original the report classified has to name that original's merge, and any other a merge of its number on `main`'s first-parent line. A clone without those heads exits 2, asking for a fetch.
|
||||
|
||||
### lines
|
||||
|
||||
Prints the release lines the batch runs for, newest first: of the `release-X.Y` branches on `origin`, the newest that have a stable `vX.Y.Z` tag there, as many as `released-lines` in the policy block says, two when it says nothing, and every branch newer than the newest of them. The tags are read with `git ls-remote`; a tag with a pre-release suffix does not count. `run`, `check` and `fold` refuse any other line, and rule 6 holds each line to the next newer one among them.
|
||||
|
||||
## Rule 4 and the API review gate
|
||||
|
||||
A change to the API is held back only when the [API review gate](../../.github/workflows/api-review-gate.yaml) would call it sizeable. The gate is asked whenever an original changes something it parses: `pkg/apiserver/apiserver.go`, an ApplicationDefinition under `packages/system/*-rd/cozyrds/`, or a `.yaml` or `.yml` file under `packages/` or `internal/` that holds `CustomResourceDefinition` or `APIService` before or after the merge. An edit to the Go types under `api/` alone reaches none of them, so the gate would see no change. The batch calls the gate's own code, `apigate.LoadSnapshot` and `apigate.Classify` from `internal/apigate`, on the trees of the original's first parent and of its merge commit. It reads only what the gate reads, from `git ls-tree` and `git cat-file` into a temporary directory, so no attribute or export rule changes what the gate sees. A reworded description or a new optional field passes; a new or removed group or resource, or a schema change that can reject what the old schema accepted, does not. Migrations, `targetVersion` and CRDs keep their own checks: the gate sees only first-party `cozystack.io` CRDs, and only their sizeable changes, while a vendored operator's CRD is just as unsafe in a patch.
|
||||
|
||||
The CRD check works document by document. In every changed manifest, a YAML, JSON or `.tpl` file or anything under `templates/` or `crds/`, outside `tests/` and `testdata/`, every document that names `CustomResourceDefinition` is read as YAML, which takes JSON too, and the CRDs among them are compared between the parent and the merge by `metadata.name`, compact JSON and quoted keys included. The change counts only when one was added, removed or edited, so an edit to the DaemonSet beside a CRD in one manifest does not, and neither does a sentence or a `lookup` call that merely names the kind. A document that declares `kind: CustomResourceDefinition` on a line of its own but cannot be read as an object, a CRD behind a Helm template guard for one, is compared by its text, so the check fails closed on it, and so does a `List` that holds a CRD.
|
||||
|
||||
## Maintainer overrides
|
||||
|
||||
`run`, `check` and `fold` read the merged pull requests that carry `backport/include` or `backport/skip`, through GitHub search, which answers at most 1000 per label; a listing that long is an error rather than a complete answer. `backport/skip` keeps a pull request out whatever the rules say, and wins over `backport/include`, which lifts rules 2 to 4 and nothing else: what the line already carries and, on the older line, what the newer one does not carry yet still decide. The batch pull request lists every overridden original and what became of it. Adding a label takes triage access to the repository, which is the trust the labels rest on; who added one is not checked.
|
||||
|
||||
## What counts as already there
|
||||
|
||||
An original is on a line when any of these holds, and the batch does not pick it again:
|
||||
|
||||
- its merge commit is reachable from the line;
|
||||
- picks on the line name its merge commit, or every commit of its own, in the `(cherry picked from commit …)` line git appended to them;
|
||||
- picking it onto the line would change nothing.
|
||||
|
||||
A pick is known by the line git appends: the last `(cherry picked from commit …)` line of the message's last paragraph, with nothing after it but trailers such as a sign-off. A picked commit keeps its message, and a batch merge carries the original's merge message, so whatever else a message says was written by someone the batch does not answer for: a line in it naming an original, as an `-x` line, a `Backported-from:` trailer or `Backport of #N`, does not count. On commits from before the policy's `batch-only` day it does, since those record the hand backports and earlier batches:
|
||||
|
||||
- a `(cherry picked from commit …)` line anywhere in the trailer block naming its merge commit or, across commits, every commit of its own;
|
||||
- a `Backported-from:` trailer naming its merge commit;
|
||||
- `Backport of #N`, which is what a hand backport merged as one commit carries.
|
||||
|
||||
An original is on the open batch pull request when its commits picked there show it the same way. The same rules on the newer line decide rule 6.
|
||||
|
||||
Only the line's own commits count, the ones after the point where it branched from `main`, never a pull request against it: an open backport pull request, its title and its body are anyone's to write. The tool trusts the line's history because only maintainers merge into `release-*`; if that ever changes, a commit that names an original without carrying all of it would hide the rest. The batch pull request counts only when it comes from a `backport-batch/<line>-<yyyymmdd>` branch of this repository, not a fork, and was opened by the bot the scheduled batch runs as, the exact login `BACKPORT_BATCH_BOT` names, `cozystack-ci[bot]` when it is unset and no bot when it is set empty, or by someone whose permission on the repository GitHub reports as write or admin, and only at the head GitHub reports for it. No other bot counts, GitHub Actions' own included, since anything holding a workflow's token writes as it. So a maintainer's run by hand continues the scheduled batch with nothing set, rather than taking its pull request for a look-alike and opening a second one. The notify comment is trusted the same way. A permission GitHub does not answer stops the command with exit 2 rather than count the pull request or the comment out, which would open a second of either. The author association is not enough: an organization member or an invited collaborator can hold read access only. With more than one such pull request open the tool refuses to go on.
|
||||
|
||||
Titles and bodies are read from the merge commits on `main`, which hold them as a maintainer merged them; the live pull request stays editable by its author.
|
||||
|
||||
## The line's record and reports
|
||||
|
||||
The line's record is the `backport-batch` block of its newest batch pull request: the newest pull request against the line from a `backport-batch/<line>-<yyyymmdd>` branch of this repository, opened by an author the batch trusts as above, whether it is open, merged or closed. `run`, `check`, `fold` and `notify` find it in GitHub's list of the line's pull requests, newest first, reading at most ten pages of 100. A batch pull request a person opened without the block is passed over. A block whose author the batch does not trust, a bot's batch without its block, a block that cannot be read whole, a closed record newer than the open batch, or ten pages with no record stop the command with exit 2, naming the pull request, rather than fall back to an older record.
|
||||
|
||||
The record is written only while its pull request is open: a merged or closed batch pull request is never edited or commented on again. What it recorded keeps counting, so a conflict or a deferral a merged batch recorded still prints as a warning. A workflow original it listed as waiting is missing, since no open batch lists it. A run that picks nothing while no batch is open has nowhere to write: it prints the body, says that its classification is not retained, and exits 0.
|
||||
|
||||
The record is only as trustworthy as the body it sits in. Anyone with write access can edit any body, the bot's included, and neither the bot's authorship nor anything in the block proves who edited it last. A hand edit that leaves a readable record is believed; one that breaks it stops every command with exit 2, naming the pull request. A reason or a file name in the record counts only in the form a run writes it, one line with no control, zero-width or bidi character, so text someone typed into the body never reaches a terminal, a body or a comment as they typed it. When the newest batch pull request is deleted and no later one exists, nothing shows it was there: the older record governs, and what was recorded since is lost.
|
||||
|
||||
The record holds only what later commands act on. Every run that classifies the line also makes a report, one line of JSON in a `backport-batch-report` block: every original merged into `main` since the line branched in exactly one category, picked, missing, deferred with what it was deferred for, or skipped with the rule that skipped it, each drop the record keeps with the merge commit it is bound to, its reason and the batch pull request that recorded it, the commits on `main` that merged no pull request, the labels, and what the classification was made of: the heads of `main`, the line and the newer line, the policy, the batch head it describes, the tool's commit, the Actions run and the flags. Its ID is a hash of all that but the time and the run, so two runs that find the same make the same report, and it names no pull request, so a new batch's report is whole before GitHub numbers the pull request. The body carries the latest report beside the record, and a comment on the open batch pull request keeps each one: `<!-- backport-batch:report <id> -->` and the same block, folded away. A report already posted is not posted again, and the tool never edits or deletes such a comment, so a report the next run replaces in the body is still read back by `report`, also after the batch merged. A comment that looks like one but whose author the batch does not trust is not read. When a run stops after writing the body but before posting its comment, the next `run --push` on the open batch posts that report first, byte for byte; once the batch is closed nothing is written to it, and the report stays in its body. A run that picks nothing while no batch is open keeps no report.
|
||||
|
||||
A `--defer` that leaves the batch carrying nothing closes it, and writes everything while the pull request is still open: the report comment, then the body, whose record keeps the deferrals and says the batch is closing, with its number, branch and the head the rebuild started from. It then reads the pull request and the branch again, closes the pull request while both are at that head, and deletes the branch with `--force-with-lease` on it, printing `closed <url>`. The next batch tries what was deferred again. A close that stops part way is finished by the next `run --push`, before it does anything else: it closes and deletes when the pull request is open at the frozen head, deletes the branch when the pull request is closed and the branch still at that head, and goes on as usual once both are gone. A run that finishes a close stops there, with exit 0, or with exit 2 when it was asked for `--defer`, `--drop` or `--undrop`, which it does not apply. It never takes a head it did not freeze: with the branch moved, it stops with exit 2 and names both heads. `run` without `--push`, `check` and `fold` stop with exit 2 while the pull request is still open; once it is closed they go on, warning about a branch the close left, and `notify` says nothing. A closed pull request is never written to, so its record says it was closing for good, and a closed pull request whose branch is gone counts as closed.
|
||||
|
||||
One way for the branch to move is a new batch of the same day: it reuses the branch name, and if its push succeeds and opening its pull request fails, the closed batch is still the newest record while its branch now holds the new batch. The next run stops on it, naming the branch, and so does `--onto` with its head. Check what it carries before deleting it with `git push origin --delete backport-batch/<line>-<yyyymmdd>`: the next run picks again what the tool put there from the line, but not a commit someone pushed onto it, such as a resolved conflict. Separate writes to GitHub are not atomic: the re-reads and the leases refuse a race they see, but cannot prevent one.
|
||||
|
||||
## What CI verifies
|
||||
|
||||
The tool does not build or test anything. CI on the batch pull request runs the line's own build and tests against what was picked, and `verify` against its shape, and a maintainer reviews and merges it. A deferred conflict waits for the next batch unless someone resolves it, checks the resolution with `fold` and pushes it, and then CI runs again.
|
||||
|
||||
## Untrusted text
|
||||
|
||||
Titles, release notes and file names in the body come from whoever opened the originals, and a reason from whoever ran `--defer`. A title, a reason or a file name goes into a code span whose fence no backtick inside can close, and the release notes go into the `release-note` block with every run of three backticks or tildes defused, so GitHub acts on no mention, reference, link or HTML in them. Inside that block, in case a note is copied out of it, links lose any target outside this repository, and closing keywords, cross-repository references and mentions have their `@`, `#` and `://` swapped for fullwidth signs, which act on nothing however the backticks around them pair up. The record and the report are one line of JSON each, whose encoding leaves no newline, `<`, `>` or `&` in them, so nothing in a file name or a reason ends their block or opens HTML, and a report comment names no one: numbers are bare, and its only text outside the block is the line, the date and the report's ID. Zero-width, bidi and control characters are dropped first, and everything the tool prints passes the same filter, so nothing it relays can drive a terminal. Commit messages are read through `git cat-file --batch`, which gives each object's length, and end at their first NUL as `git log` shows them, so a message cannot pass for a second commit or hide a trailer from a reader of the log. Every value handed to git comes after `--end-of-options` or `--`, refs pass `git check-ref-format`, SHAs have to be 40 hex digits, and nothing runs through a shell.
|
||||
@@ -0,0 +1,188 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/cozystack/cozystack/internal/apigate"
|
||||
)
|
||||
|
||||
// Rule 4's API clause is the API review gate's own verdict: the same
|
||||
// apigate.LoadSnapshot and apigate.Classify that cmd/api-gate runs in
|
||||
// api-review-gate.yaml, on the trees of the original's first parent and of its
|
||||
// merge, so a change the gate passes as additive or descriptive is no reason to
|
||||
// hold a fix back.
|
||||
|
||||
const (
|
||||
apiserverGo = "pkg/apiserver/apiserver.go"
|
||||
// apigate reads an ApplicationDefinition wherever this matches, whatever
|
||||
// the file holds.
|
||||
cozyRDGlob = "packages/system/*-rd/cozyrds/*.yaml"
|
||||
// apigate skips a YAML file larger than this; reading it would be wasted.
|
||||
apiFileLimit = 8 << 20
|
||||
)
|
||||
|
||||
// apiYAMLRoots hold all YAML apigate.LoadSnapshot parses: the
|
||||
// ApplicationDefinitions and every first-party CRD or APIService.
|
||||
var apiYAMLRoots = []string{"packages", "internal"}
|
||||
|
||||
// apiInputs are what apigate.LoadSnapshot reads of a checkout: the YAML roots
|
||||
// and the apiserver's storage registrations.
|
||||
var apiInputs = append(slices.Clone(apiYAMLRoots), apiserverGo)
|
||||
|
||||
// apiMarkers are mightBeAPIManifest's: apigate parses no other YAML file that
|
||||
// holds neither.
|
||||
var apiMarkers = []string{"-e", "CustomResourceDefinition", "-e", "APIService"}
|
||||
|
||||
var apiSnapshots = map[string]apigate.Snapshot{}
|
||||
|
||||
// apiSnapshot is apigate's snapshot of commit, built from a temporary tree
|
||||
// holding only what it reads. The blobs come from ls-tree and cat-file rather
|
||||
// than a checkout or git archive: no attribute filters or export rules apply,
|
||||
// and a symlink is never followed out of the tree.
|
||||
func apiSnapshot(commit string) (apigate.Snapshot, error) {
|
||||
if snap, ok := apiSnapshots[commit]; ok {
|
||||
return snap, nil
|
||||
}
|
||||
out, err := git(append([]string{"ls-tree", "-r", "-z", "-l", "--full-tree", "--end-of-options", commit, "--"}, apiInputs...)...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
paths := map[string]string{}
|
||||
var blobs []string
|
||||
for _, entry := range splitNUL(out) {
|
||||
meta, name, ok := strings.Cut(entry, "\t")
|
||||
f := strings.Fields(meta)
|
||||
if !ok || len(f) != 4 {
|
||||
return nil, fmt.Errorf("ls-tree %.12s: unreadable entry %q", commit, entry)
|
||||
}
|
||||
mode, kind, blob := f[0], f[1], f[2]
|
||||
size, _ := strconv.Atoi(f[3])
|
||||
ext := path.Ext(name)
|
||||
switch {
|
||||
case kind != "blob" || (mode != "100644" && mode != "100755"):
|
||||
case name != apiserverGo && ext != ".yaml" && ext != ".yml":
|
||||
case size > apiFileLimit:
|
||||
case checkSHA(blob) != nil || !filepath.IsLocal(filepath.FromSlash(name)):
|
||||
return nil, fmt.Errorf("ls-tree %.12s: unreadable entry %q", commit, entry)
|
||||
default:
|
||||
paths[name] = blob
|
||||
blobs = append(blobs, blob)
|
||||
}
|
||||
}
|
||||
contents, err := readObjects(dedupe(blobs), "blob")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
dir, err := os.MkdirTemp("", "backport-batch-api-")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer os.RemoveAll(dir)
|
||||
for name, blob := range paths {
|
||||
file := filepath.Join(dir, filepath.FromSlash(name))
|
||||
if err := os.MkdirAll(filepath.Dir(file), 0o700); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := os.WriteFile(file, contents[blob], 0o600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
snap, err := apigate.LoadSnapshot(dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the API at %.12s: %w", commit, err)
|
||||
}
|
||||
apiSnapshots[commit] = snap
|
||||
return snap, nil
|
||||
}
|
||||
|
||||
// readsAPI reports whether the change reaches anything apigate.LoadSnapshot
|
||||
// parses at either end. A change to anything else leaves both snapshots alike,
|
||||
// so the gate is not asked. A binary file is searched as well, since apigate
|
||||
// reads one like any other; a file over its size limit is still asked about,
|
||||
// which costs a snapshot and changes no verdict.
|
||||
func readsAPI(parent, merge string, changes []fileChange) (bool, error) {
|
||||
var yamls []string
|
||||
for _, c := range changes {
|
||||
p := c.path
|
||||
if p == apiserverGo {
|
||||
return true, nil
|
||||
}
|
||||
if ext := path.Ext(p); ext != ".yaml" && ext != ".yml" {
|
||||
continue
|
||||
}
|
||||
if !slices.ContainsFunc(apiYAMLRoots, func(root string) bool { return strings.HasPrefix(p, root+"/") }) {
|
||||
continue
|
||||
}
|
||||
if rd, _ := path.Match(cozyRDGlob, p); rd {
|
||||
return true, nil
|
||||
}
|
||||
yamls = append(yamls, p)
|
||||
}
|
||||
for _, rev := range []string{parent, merge} {
|
||||
found, err := grepFiles(rev, yamls, apiMarkers...)
|
||||
if err != nil || len(found) > 0 {
|
||||
return len(found) > 0, err
|
||||
}
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// sizeableAPIChange says why the API review gate would call the original's
|
||||
// change to the API sizeable, or "" when it would not.
|
||||
func sizeableAPIChange(o *original) (string, error) {
|
||||
parent, err := resolve(o.merge + "^1")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
base, err := apiSnapshot(parent)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
head, err := apiSnapshot(o.merge)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
findings := apigate.Classify(base, head)
|
||||
if len(findings) == 0 {
|
||||
return "", nil
|
||||
}
|
||||
why := fmt.Sprintf("%s: %s", findings[0].Category, findings[0].Detail)
|
||||
if len(findings) > 1 {
|
||||
why += fmt.Sprintf(", and %d more", len(findings)-1)
|
||||
}
|
||||
return why, nil
|
||||
}
|
||||
|
||||
func dedupe(list []string) []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
for _, s := range list {
|
||||
if !seen[s] {
|
||||
seen[s] = true
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,836 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// pick is the outcome of picking one original onto the batch branch.
|
||||
type pick struct {
|
||||
v *verdict
|
||||
status string // picked, conflict, empty, held out by --defer, or waiting for a maintainer
|
||||
commit string
|
||||
files []string
|
||||
// failed is the commit whose pick conflicted.
|
||||
failed string
|
||||
// more counts the conflicting files a recorded conflict no longer names.
|
||||
more int
|
||||
reason string
|
||||
}
|
||||
|
||||
// workflowsDir is where GitHub reads workflows from. A token without the
|
||||
// workflows permission, a GitHub App's among them, cannot push a commit that
|
||||
// changes anything under it.
|
||||
const workflowsDir = ".github/workflows/"
|
||||
|
||||
// workflowFiles lists the workflow files the commits change, each against its
|
||||
// parent: a push carries every pick, so a change one commit makes and a later
|
||||
// one takes back is in it although the pull request's merge changes nothing
|
||||
// there.
|
||||
func workflowFiles(commits []string) ([]string, error) {
|
||||
if len(commits) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
out, err := gitInput(strings.Join(commits, "\n")+"\n", "diff-tree", "--stdin", "--no-commit-id", "--name-only", "-r", "-z", "--no-renames")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []string
|
||||
for _, f := range splitNUL(out) {
|
||||
if strings.HasPrefix(f, workflowsDir) && !slices.Contains(files, f) {
|
||||
files = append(files, f)
|
||||
}
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func batchBranch(l line, date time.Time) string {
|
||||
return "backport-batch/" + l.branch() + "-" + date.Format("20060102")
|
||||
}
|
||||
|
||||
func (a *app) run(args []string) int {
|
||||
fs := a.policyFlags("run")
|
||||
lineArg := fs.String("line", "", "release line, release-X.Y")
|
||||
onto := fs.String("onto", "", "head of the pushed batch branch to continue")
|
||||
cont := fs.Bool("continue", false, "continue the open batch pull request, or start a batch when none is open")
|
||||
dateArg := fs.String("date", "", "batch date, YYYY-MM-DD (default: today, UTC)")
|
||||
deferArg := fs.String("defer", "", "originals to take out of the open batch, N[,N]; rebuilds it without them")
|
||||
dropArg := fs.String("drop", "", "originals to drop from the line, N[,N]; recorded on the open batch, and taken off it")
|
||||
undropArg := fs.String("undrop", "", "originals whose drop to withdraw, N[,N]")
|
||||
noWorkflows := fs.Bool("no-workflow-files", false, "leave an original that changes "+workflowsDir+" to a maintainer, for a token that cannot push one")
|
||||
reasonArg := fs.String("reason", "", "why --defer takes them out or --drop drops them, recorded in the batch body")
|
||||
push := fs.Bool("push", false, "push the branch and open or update the draft batch PR")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
l, err := parseLine(*lineArg)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
date := a.now().UTC()
|
||||
if *dateArg != "" {
|
||||
if date, err = time.Parse(time.DateOnly, *dateArg); err != nil {
|
||||
return a.fail(fmt.Errorf("--date %q is not YYYY-MM-DD", *dateArg))
|
||||
}
|
||||
}
|
||||
if *onto != "" {
|
||||
if err := checkSHA(*onto); err != nil {
|
||||
return a.fail(fmt.Errorf("--onto: %w", err))
|
||||
}
|
||||
}
|
||||
var deferred, drops []int
|
||||
reason, ops := "", 0
|
||||
for _, op := range []string{*deferArg, *dropArg, *undropArg} {
|
||||
if op != "" {
|
||||
ops++
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case ops > 1:
|
||||
return a.fail(errors.New("--defer, --drop and --undrop each change the open batch pull request on their own; give one"))
|
||||
case *dropArg != "" || *undropArg != "":
|
||||
if *onto != "" || *cont || *dateArg != "" {
|
||||
return a.fail(errors.New("--drop and --undrop change the open batch pull request; each takes no --onto, --continue or --date"))
|
||||
}
|
||||
if *undropArg != "" && *reasonArg != "" {
|
||||
return a.fail(errors.New("--undrop takes no --reason"))
|
||||
}
|
||||
list := *dropArg + *undropArg
|
||||
if drops, err = parseNumbers(map[bool]string{true: "--drop", false: "--undrop"}[*dropArg != ""], list); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if reason = capped(oneLine(clean(*reasonArg, false)), titleLimit); *dropArg != "" && reason == "" {
|
||||
return a.fail(errors.New("--drop needs a --reason, which the line's record keeps"))
|
||||
}
|
||||
case *deferArg != "" || *reasonArg != "":
|
||||
if deferred, reason, err = parseDefer(*deferArg, *reasonArg); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if *onto != "" || *cont || *dateArg != "" {
|
||||
return a.fail(errors.New("--defer rebuilds the open batch pull request; it takes no --onto, --continue or --date"))
|
||||
}
|
||||
case *cont && (*onto != "" || *dateArg != ""):
|
||||
return a.fail(errors.New("--continue reads the batch to continue from the open batch pull request; it takes no --onto or --date"))
|
||||
}
|
||||
if err := cleanTree(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := fetch(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
b, err := a.prepare(l)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if goOn, code := a.resumeClose(b, *push, deferred != nil || drops != nil); !goOn {
|
||||
return code
|
||||
}
|
||||
if drops != nil {
|
||||
return a.redrop(b, drops, reason, *undropArg != "", *push)
|
||||
}
|
||||
if deferred != nil {
|
||||
return a.rebuild(b, deferred, reason, *push)
|
||||
}
|
||||
if *cont && b.pr != nil {
|
||||
*onto = b.prHead
|
||||
}
|
||||
|
||||
branch, start := batchBranch(l, date), b.lineHead
|
||||
var carried *history
|
||||
if *onto != "" {
|
||||
if branch, err = ontoBranch(l, *onto, *dateArg != "", branch); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if carried, err = readPicks(*onto, b.lineHead, b.textUntil); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
start = *onto
|
||||
// The batch keeps the date it was opened under, in its title too.
|
||||
if date, err = branchDate(branch); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
}
|
||||
if err := checkBranch(branch); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
// Continuing the open batch keeps out what was deferred from it by hand;
|
||||
// a new batch tries those again.
|
||||
held := map[int]deferredEntry{}
|
||||
if carried != nil && b.src.open && b.pr.Head.Ref == branch {
|
||||
held = b.src.rec.deferred()
|
||||
}
|
||||
if err := a.warnReset(branch, start); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if _, err := git("switch", "--quiet", "--no-track", "-C", branch, "--end-of-options", start); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
|
||||
r := newReport(b)
|
||||
r.opts = runOptions{NoWorkflowFiles: *noWorkflows, Mode: modeNew}
|
||||
// A drop is a decision about the line, not about one batch: a new batch
|
||||
// keeps the ones the closed batch recorded.
|
||||
r.dropped = b.src.rec.dropped()
|
||||
if carried != nil {
|
||||
r.opts.Mode = modeContinue
|
||||
if r.stale, err = keptOut(b, carried); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
}
|
||||
for _, v := range b.qualifying() {
|
||||
switch {
|
||||
case carried != nil && found(carried, v):
|
||||
r.onBranch = append(r.onBranch, v)
|
||||
continue
|
||||
case held[v.o.number].Reason != "":
|
||||
r.held = append(r.held, pick{v: v, status: "held", reason: held[v.o.number].Reason})
|
||||
continue
|
||||
case v.onBatch != "":
|
||||
v.rule, v.reason = ruleOnLine, fmt.Sprintf("on the open batch pull request #%d", b.pr.Number)
|
||||
r.skipped[ruleOnLine] = append(r.skipped[ruleOnLine], v)
|
||||
continue
|
||||
}
|
||||
if *noWorkflows {
|
||||
files, err := workflowFiles(v.commits)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if len(files) > 0 {
|
||||
r.waiting = append(r.waiting, pick{v: v, status: "waiting", files: files})
|
||||
continue
|
||||
}
|
||||
}
|
||||
p, err := pickOriginal(v)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
switch p.status {
|
||||
case "picked":
|
||||
r.onBranch = append(r.onBranch, v)
|
||||
case "conflict":
|
||||
r.deferred = append(r.deferred, p)
|
||||
case "empty":
|
||||
v.rule, v.reason = ruleOnLine, "the earlier picks already carry it"
|
||||
r.skipped[ruleOnLine] = append(r.skipped[ruleOnLine], v)
|
||||
}
|
||||
}
|
||||
if *onto != "" {
|
||||
head, err := resolve("HEAD")
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
descends, err := isAncestor(*onto, head)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if !descends {
|
||||
return a.fail(fmt.Errorf("%.12s does not descend from --onto %.12s", head, *onto))
|
||||
}
|
||||
}
|
||||
return a.finish(b, r, branch, date, *push, false)
|
||||
}
|
||||
|
||||
// keptOut lists the originals with backport/skip, or dropped, that the batch
|
||||
// still carries: a run never takes a commit off the batch, --defer does.
|
||||
func keptOut(b *batch, carried *history) ([]*verdict, error) {
|
||||
var out []*verdict
|
||||
for _, v := range b.verdicts {
|
||||
if v.rule != ruleSkipLabel && v.rule != ruleDropped {
|
||||
continue
|
||||
}
|
||||
if err := b.load(v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if found(carried, v) {
|
||||
out = append(out, v)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func found(h *history, v *verdict) bool {
|
||||
c, _ := h.found(v.o, v.commits)
|
||||
return c != ""
|
||||
}
|
||||
|
||||
// newReport starts the report of a run with what the rules left out and what
|
||||
// the labels overrode.
|
||||
func newReport(b *batch) *report {
|
||||
r := &report{line: b.line, newer: b.newer, skipped: map[int][]*verdict{}, other: len(b.other), start: b.start}
|
||||
for _, v := range b.verdicts {
|
||||
if v.rule != 0 {
|
||||
r.skipped[v.rule] = append(r.skipped[v.rule], v)
|
||||
}
|
||||
if v.label != "" {
|
||||
r.overrides = append(r.overrides, v)
|
||||
}
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// finish prints the body and what the run did, and publishes the branch when
|
||||
// asked to.
|
||||
func (a *app) finish(b *batch, r *report, branch string, date time.Time, push, rebuilt bool) int {
|
||||
head, err := resolve("HEAD")
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if r.shapes, err = summarize(b, head, r.onBranch); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if rebuilt && head == b.lineHead && b.pr != nil {
|
||||
// Deferring everything the batch carries leaves nothing to push: the
|
||||
// batch is closed, and its record says so first, with the head the
|
||||
// close is leased on.
|
||||
r.closing = &closing{PR: b.pr.Number, Branch: branch, Head: b.prHead}
|
||||
r.opts.Mode = modeClose
|
||||
}
|
||||
r.built = builtFrom{Main: b.mainHead, Line: b.lineHead, Newer: b.newerHead, Policy: b.policyAt, Batch: head, Tool: a.tool,
|
||||
Run: runURL(os.LookupEnv), At: a.now().UTC().Format(time.RFC3339), Options: r.opts}
|
||||
doc, err := buildReport(b, r)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
p := publication{branch: branch, head: head, title: prTitle(b.line, date.Format(time.DateOnly)), report: storedReport{doc: doc, raw: reportJSON(doc)}}
|
||||
p.record = r.lineRecord(doc)
|
||||
// A record the next command would refuse is never written.
|
||||
if err := p.record.validate(b.line); err != nil {
|
||||
return a.fail(fmt.Errorf("the record this run made %w", err))
|
||||
}
|
||||
r.blocks = recordBlock(p.record) + "\n\n" + reportBlock(p.report.raw)
|
||||
template, err := lineTemplate(b.lineHead)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
body, left := r.body(template)
|
||||
p.body = body
|
||||
fmt.Fprint(a.stdout, body)
|
||||
skipped := 0
|
||||
for _, items := range r.skipped {
|
||||
skipped += len(items)
|
||||
}
|
||||
for _, rule := range left {
|
||||
for _, v := range r.skipped[rule] {
|
||||
fmt.Fprintf(a.stderr, "skipped by %s: %s %s (%s)\n", ruleName(rule), v.o.url(), terminalSafe(v.o.title), terminalSafe(v.reason))
|
||||
}
|
||||
}
|
||||
for _, p := range r.waiting {
|
||||
fmt.Fprintf(a.stderr, "waiting for a maintainer: %s %s changes %s\n", p.v.o.url(), terminalSafe(p.v.o.title), terminalSafe(strings.Join(p.files, ", ")))
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "%s at %.12s: %d on the branch, %d deferred, %d waiting for a maintainer, %d skipped\ntitle: %s\nreport: %s\n",
|
||||
branch, head, len(r.onBranch), len(r.deferred)+len(r.held), len(r.waiting), skipped, p.title, doc.ID)
|
||||
if !push {
|
||||
return 0
|
||||
}
|
||||
if head == b.lineHead && b.pr == nil {
|
||||
// Picking nothing is not a failure of the run; check still reports
|
||||
// whatever the line lacks. Merged and closed batches are never
|
||||
// written, so there is nowhere to keep what the run found.
|
||||
fmt.Fprintf(a.stderr, "%s carries nothing beyond %s; nothing to push\n", branch, b.line.branch())
|
||||
fmt.Fprintf(a.stderr, "no batch pull request is open on %s; this run's classification is not retained\n", b.line.branch())
|
||||
return 0
|
||||
}
|
||||
if err := a.publish(b, p, rebuilt); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// lineRecord is the record a run leaves for the line: what later commands
|
||||
// act on, with the full classification left to the report it names.
|
||||
func (r *report) lineRecord(doc reportDoc) lineRecord {
|
||||
built := doc.Built
|
||||
dropped := slices.Clone(r.dropped)
|
||||
if dropped == nil {
|
||||
dropped = []droppedEntry{}
|
||||
}
|
||||
slices.SortFunc(dropped, func(a, b droppedEntry) int { return a.Original - b.Original })
|
||||
return lineRecord{Version: 2, Line: doc.Line, Built: &built, Report: doc.ID, Closing: r.closing, Deferred: slices.Clone(doc.Primary.Deferred), Dropped: dropped, Stale: slices.Clone(doc.Overlays.Stale)}
|
||||
}
|
||||
|
||||
// branchDate is the date a batch branch was opened under.
|
||||
func branchDate(branch string) (time.Time, error) {
|
||||
m := batchBranchRE.FindStringSubmatch(branch)
|
||||
if m == nil {
|
||||
return time.Time{}, fmt.Errorf("%q is not a batch branch", branch)
|
||||
}
|
||||
return time.Parse("20060102", m[2])
|
||||
}
|
||||
|
||||
// cleanTree refuses a working tree with changes or an operation in progress:
|
||||
// run switches it to the batch branch.
|
||||
func cleanTree() error {
|
||||
out, err := git("status", "--porcelain", "--untracked-files=no")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(out) != "" {
|
||||
return errors.New("the working tree has changes; run in a clean worktree")
|
||||
}
|
||||
for _, name := range []string{"CHERRY_PICK_HEAD", "MERGE_HEAD", "REVERT_HEAD", "REBASE_HEAD", "rebase-merge", "rebase-apply", "sequencer"} {
|
||||
p, err := git("rev-parse", "--git-path", name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := os.Stat(strings.TrimSpace(p)); err == nil {
|
||||
return fmt.Errorf("an operation is in progress (%s); finish or abort it first", name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// warnReset says, before switch -C moves the local branch to start, which tip
|
||||
// the move drops commits of and how to get them back: a clean tree says
|
||||
// nothing about commits only a local branch has.
|
||||
func (a *app) warnReset(branch, start string) error {
|
||||
old, err := lookup("refs/heads/" + branch)
|
||||
if err != nil || old == "" || old == start {
|
||||
return err
|
||||
}
|
||||
kept, err := isAncestor(old, start)
|
||||
if err != nil || kept {
|
||||
return err
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "resetting the local %s from %s, dropping commits %.12s does not carry; git branch %s-%.12s %s restores them\n", branch, old, start, branch, old, old)
|
||||
return nil
|
||||
}
|
||||
|
||||
// ontoBranch is the pushed batch branch whose head --onto names, read from the
|
||||
// remote itself: a head that moved since, or that no batch branch has, is
|
||||
// refused rather than continued.
|
||||
func ontoBranch(l line, onto string, dated bool, dateBranch string) (string, error) {
|
||||
heads, err := remoteBatchHeads(l)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
var found []string
|
||||
for branch, sha := range heads {
|
||||
if sha == onto && (!dated || branch == dateBranch) {
|
||||
found = append(found, branch)
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case len(found) == 0 && dated:
|
||||
return "", fmt.Errorf("--onto %.12s is not the head of %s on %s (%.12s)", onto, dateBranch, remote, heads[dateBranch])
|
||||
case len(found) == 0:
|
||||
return "", fmt.Errorf("--onto %.12s is not the head of any batch branch for %s on %s", onto, l.branch(), remote)
|
||||
case len(found) > 1:
|
||||
return "", fmt.Errorf("--onto %.12s heads more than one batch branch; name one with --date", onto)
|
||||
}
|
||||
have, err := lookup(onto)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if have == "" {
|
||||
return "", fmt.Errorf("--onto %.12s is not in this repository; fetch it first", onto)
|
||||
}
|
||||
return found[0], nil
|
||||
}
|
||||
|
||||
// remoteBatchHeads lists the batch branches for l on the remote, read with
|
||||
// ls-remote rather than from the possibly stale remote-tracking refs.
|
||||
func remoteBatchHeads(l line) (map[string]string, error) {
|
||||
out, err := git("ls-remote", "--refs", "--heads", "--end-of-options", remote)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
heads := map[string]string{}
|
||||
for ln := range strings.Lines(out) {
|
||||
sha, ref, ok := strings.Cut(strings.TrimSpace(ln), "\t")
|
||||
branch, isHead := strings.CutPrefix(ref, "refs/heads/")
|
||||
if !ok || !isHead || checkSHA(sha) != nil {
|
||||
continue
|
||||
}
|
||||
if m := batchBranchRE.FindStringSubmatch(branch); m != nil && m[1] == l.branch() {
|
||||
heads[branch] = sha
|
||||
}
|
||||
}
|
||||
return heads, nil
|
||||
}
|
||||
|
||||
// applyPick runs one cherry-pick onto HEAD and leaves no operation in progress
|
||||
// behind it: the outcome is picked, conflict, with the conflicting files, or
|
||||
// empty.
|
||||
func applyPick(args ...string) (string, []string, error) {
|
||||
stdout, stderr, code, err := gitStatus(args...)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
if code == 0 {
|
||||
return "picked", nil, nil
|
||||
}
|
||||
out, err := git("diff", "--name-only", "-z", "--diff-filter=U")
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
if files := splitNUL(out); len(files) > 0 {
|
||||
_, err := git("cherry-pick", "--abort")
|
||||
return "conflict", files, err
|
||||
}
|
||||
if msg := stdout + stderr; strings.Contains(msg, "is now empty") || strings.Contains(msg, "nothing to commit") {
|
||||
_, err := git("cherry-pick", "--skip")
|
||||
return "empty", nil, err
|
||||
}
|
||||
failed := fmt.Errorf("failed: %s", strings.TrimSpace(stderr))
|
||||
if _, err := git("cherry-pick", "--abort"); err != nil {
|
||||
return "", nil, errors.Join(failed, err)
|
||||
}
|
||||
return "", nil, failed
|
||||
}
|
||||
|
||||
// lineTemplate is the line's own pull request template, so the batch body has
|
||||
// the sections reviewers of that line expect.
|
||||
func lineTemplate(lineHead string) (string, error) {
|
||||
const template = ".github/PULL_REQUEST_TEMPLATE.md"
|
||||
out, err := git("--literal-pathspecs", "ls-tree", "--name-only", "--end-of-options", lineHead, "--", template)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if strings.TrimSpace(out) == "" {
|
||||
return "## What this PR does\n\n### Release note\n\n```release-note\n\n```\n", nil
|
||||
}
|
||||
return git("show", "--end-of-options", lineHead+":"+template)
|
||||
}
|
||||
|
||||
// publication is what a run publishes: the branch at head, and the body that
|
||||
// carries the record and the report.
|
||||
type publication struct {
|
||||
branch, head, title, body string
|
||||
report storedReport
|
||||
record lineRecord
|
||||
}
|
||||
|
||||
// publish pushes the batch branch, then opens the draft batch pull request or
|
||||
// updates the open one. A run pushes only a fast-forward of what the remote
|
||||
// has. A rebuild replaces the branch, and only the head it was rebuilt from:
|
||||
// the lease is that head, as ls-remote reads it now, so a commit pushed since
|
||||
// is never dropped.
|
||||
//
|
||||
// The report goes up as a comment before the push and the body that names
|
||||
// it, so no write leaves a body naming a report nobody can read; a new batch
|
||||
// carries the report in the body it is opened with, and the comment follows.
|
||||
// Everything is rendered and measured before the first write.
|
||||
func (a *app) publish(b *batch, p publication, rebuilt bool) error {
|
||||
if b.pr != nil && b.pr.Head.Ref != p.branch {
|
||||
return fmt.Errorf("the open batch pull request #%d is on %s; continue it with --onto %s", b.pr.Number, b.pr.Head.Ref, b.prHead)
|
||||
}
|
||||
// A body without the block was written by hand, and the tool's would
|
||||
// replace it whole, title included; such a batch is the maintainer's.
|
||||
if b.pr != nil && !b.src.open {
|
||||
return fmt.Errorf("batch pull request #%d has no %s block, so backport-batch did not write it; merge or close it, and the next run opens a batch of its own", b.pr.Number, deferredFence)
|
||||
}
|
||||
if p.head == b.lineHead {
|
||||
if rebuilt && p.record.Closing != nil {
|
||||
return a.closeBatch(b, p)
|
||||
}
|
||||
return fmt.Errorf("%s carries nothing beyond %s; nothing to push", p.branch, b.line.branch())
|
||||
}
|
||||
heads, err := remoteBatchHeads(b.line)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
pushed, onRemote := heads[p.branch]
|
||||
lease := ""
|
||||
switch {
|
||||
case rebuilt:
|
||||
if !onRemote || pushed != b.prHead {
|
||||
return fmt.Errorf("%s on %s is at %.12s, not at %.12s, which the rebuild started from; run again", p.branch, remote, pushed, b.prHead)
|
||||
}
|
||||
lease = pushed
|
||||
case onRemote && pushed != p.head:
|
||||
have, err := lookup(pushed)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
descends := false
|
||||
if have != "" {
|
||||
if descends, err = isAncestor(pushed, p.head); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if !descends {
|
||||
refusal := fmt.Sprintf("%s on %s is at %.12s, which %.12s does not descend from; refusing a non-fast-forward push", p.branch, remote, pushed, p.head)
|
||||
// A run that stopped between its push and opening the pull request
|
||||
// leaves the branch, and the next run that day builds it anew.
|
||||
switch c := b.src.carrier; {
|
||||
case b.pr != nil:
|
||||
case c != nil && !b.src.open && c.Head.Ref == p.branch:
|
||||
// Continuing a batch that merged or closed would publish
|
||||
// what nobody can merge any more.
|
||||
refusal += fmt.Sprintf("; it is the branch of batch pull request #%d, which is %s; the next run picks again what the tool put on it, but not a commit someone pushed there, such as a resolved conflict: check what %.12s carries, then delete it with git push origin --delete %s and run again", c.Number, prState(c), pushed, p.branch)
|
||||
default:
|
||||
refusal += fmt.Sprintf("; no batch pull request is open on it, and backport-batch run --line %s --onto %s --push continues it and opens one", b.line.branch(), pushed)
|
||||
}
|
||||
return errors.New(refusal)
|
||||
}
|
||||
}
|
||||
if err := p.fits(); err != nil {
|
||||
return err
|
||||
}
|
||||
if b.pr != nil {
|
||||
if err := a.keepReports(b, &p); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if pushed != p.head {
|
||||
if _, err := git(pushArgs(p.branch, p.head, lease)...); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
name, err := writeTemp("backport-batch-body-*.md", p.body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(name)
|
||||
if b.pr == nil {
|
||||
n, url, err := a.gh.createPull(b.line.branch(), p.branch, p.title, name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "opened draft %s\n", url)
|
||||
if err := a.postReport(n, p.report); err != nil {
|
||||
return fmt.Errorf("posting report %.12s on #%d: %w; the body keeps it, and the next run --push posts it", p.report.doc.ID, n, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := a.stillOpen(b.pr.Number); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := a.gh.editPull(b.pr.Number, p.title, name); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "updated %s\n", pullURL(b.pr))
|
||||
return nil
|
||||
}
|
||||
|
||||
// fits says why GitHub would not take the comment or the body.
|
||||
func (p *publication) fits() error {
|
||||
if _, err := reportComment(p.report); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(p.body) > bodyLimit {
|
||||
return fmt.Errorf("the body is %d bytes with only what it cannot leave out, over GitHub's %d; nothing is published", len(p.body), bodyLimit)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// keepReports makes sure the open batch's comments keep both the report its
|
||||
// body carries now and the one this run made, in that order, each once.
|
||||
// When this run's report is one already kept, the body carries the kept
|
||||
// bytes, which differ only in when and in which job it was made.
|
||||
func (a *app) keepReports(b *batch, p *publication) error {
|
||||
pr := b.pr
|
||||
posted, bad, err := findReports(a.gh, pr.Number, b.line)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, why := range bad {
|
||||
a.warn("batch pull request #%d: %s; it does not count as a report", pr.Number, why)
|
||||
}
|
||||
earlier, err := bodyReport(pr.Body, b.line)
|
||||
if err != nil {
|
||||
return fmt.Errorf("batch pull request #%d: %w", pr.Number, err)
|
||||
}
|
||||
kept, ok := posted[p.report.doc.ID]
|
||||
if !ok && earlier != nil && earlier.doc.ID == p.report.doc.ID {
|
||||
kept, ok = *earlier, true
|
||||
}
|
||||
if ok {
|
||||
p.body = strings.Replace(p.body, reportBlock(p.report.raw), reportBlock(kept.raw), 1)
|
||||
p.report = kept
|
||||
if err := p.fits(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if earlier != nil {
|
||||
if _, ok := posted[earlier.doc.ID]; !ok {
|
||||
if err := a.postReport(pr.Number, *earlier); err != nil {
|
||||
return fmt.Errorf("posting the earlier report %.12s on #%d: %w", earlier.doc.ID, pr.Number, err)
|
||||
}
|
||||
posted[earlier.doc.ID] = *earlier
|
||||
}
|
||||
}
|
||||
if _, ok := posted[p.report.doc.ID]; !ok {
|
||||
if err := a.postReport(pr.Number, p.report); err != nil {
|
||||
return fmt.Errorf("posting report %.12s on #%d: %w", p.report.doc.ID, pr.Number, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// pullURL is where a pull request is on GitHub.
|
||||
func pullURL(pr *pullRequest) string {
|
||||
if pullURLRE.MatchString(pr.HTMLURL) {
|
||||
return pr.HTMLURL
|
||||
}
|
||||
return fmt.Sprintf("%spull/%d", ourRepoURL, pr.Number)
|
||||
}
|
||||
|
||||
// closeBatch closes the open batch that --defer left carrying nothing. Every
|
||||
// write happens while it is open: the report, then the body whose record
|
||||
// says it is closing and at which head. The close and the deletion of its
|
||||
// branch then go only while both are still at that head.
|
||||
func (a *app) closeBatch(b *batch, p publication) error {
|
||||
c := *p.record.Closing
|
||||
if err := p.fits(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := a.keepReports(b, &p); err != nil {
|
||||
return err
|
||||
}
|
||||
name, err := writeTemp("backport-batch-body-*.md", p.body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(name)
|
||||
if err := a.stillOpen(c.PR); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := a.gh.editPull(c.PR, p.title, name); err != nil {
|
||||
return fmt.Errorf("recording the close of #%d: %w", c.PR, err)
|
||||
}
|
||||
return a.finishClose(b.line, pullURL(b.pr), c)
|
||||
}
|
||||
|
||||
// finishClose closes the batch pull request a record says is closing, and
|
||||
// deletes its branch, each only while it is at the head the record froze.
|
||||
func (a *app) finishClose(l line, url string, c closing) error {
|
||||
pr, err := a.gh.pull(c.PR)
|
||||
if err != nil {
|
||||
return fmt.Errorf("closing #%d, reading it again: %w", c.PR, err)
|
||||
}
|
||||
if st := prState(&pr); st != "open" || pr.Head.SHA != c.Head {
|
||||
return fmt.Errorf("closing #%d: it is %s at %.12s, not open at %.12s, which its record froze", c.PR, st, terminalSafe(pr.Head.SHA), c.Head)
|
||||
}
|
||||
heads, err := remoteBatchHeads(l)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if h := heads[c.Branch]; h != c.Head {
|
||||
return fmt.Errorf("closing #%d: %s on %s is at %.12s, not at %.12s, which its record froze", c.PR, c.Branch, remote, h, c.Head)
|
||||
}
|
||||
if err := a.gh.closePull(c.PR); err != nil {
|
||||
return fmt.Errorf("closing #%d: %w; backport-batch run --line %s --push finishes it", c.PR, err, l.branch())
|
||||
}
|
||||
if err := a.deleteBranch(l, c); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "closed %s: without what is deferred it carries nothing beyond %s; the next batch tries them again\n", url, l.branch())
|
||||
return nil
|
||||
}
|
||||
|
||||
// deleteBranch deletes the branch of a closed batch, leased on the head it
|
||||
// was closed at: one someone pushed since is a person's to delete or keep.
|
||||
func (a *app) deleteBranch(l line, c closing) error {
|
||||
heads, err := remoteBatchHeads(l)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch h, ok := heads[c.Branch]; {
|
||||
case !ok:
|
||||
return nil
|
||||
case h != c.Head:
|
||||
return movedBranch(c, h)
|
||||
}
|
||||
if _, err := git(deleteArgs(c.Branch, c.Head)...); err != nil {
|
||||
return fmt.Errorf("deleting %s, left by the close of #%d: %w; backport-batch run --line %s --push deletes it", c.Branch, c.PR, err, l.branch())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// takenBranch is the refusal for a branch a close left that an open batch
|
||||
// pull request now uses, whatever it holds, or nil.
|
||||
func takenBranch(b *batch, c closing) error {
|
||||
if b.pr == nil || b.pr.Head.Ref != c.Branch {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("batch pull request #%d is open on %s, which the close of #%d left; the branch is that pull request's now, and the close leaves it alone", b.pr.Number, c.Branch, c.PR)
|
||||
}
|
||||
|
||||
func movedBranch(c closing, h string) error {
|
||||
return fmt.Errorf("batch pull request #%d was closed at %.12s, but %s is at %.12s now; the next run picks again what the tool put on it, but not a commit someone pushed there, such as a resolved conflict: check what %.12s carries, then delete it with git push origin --delete %s, and run again", c.PR, c.Head, c.Branch, h, h, c.Branch)
|
||||
}
|
||||
|
||||
// resumeClose finishes a close the record says is under way, before a run
|
||||
// does anything else, and only under --push. It says whether the run may go
|
||||
// on: only once the batch is closed and its branch gone. A run that finishes
|
||||
// one stops there, and fails when it was asked to change the batch.
|
||||
func (a *app) resumeClose(b *batch, push, changing bool) (bool, int) {
|
||||
if b.src.rec == nil || b.src.rec.Closing == nil {
|
||||
return true, 0
|
||||
}
|
||||
c := *b.src.rec.Closing
|
||||
heads, err := remoteBatchHeads(b.line)
|
||||
if err != nil {
|
||||
return false, a.fail(err)
|
||||
}
|
||||
h, onRemote := heads[c.Branch]
|
||||
switch {
|
||||
case b.src.open && h != c.Head:
|
||||
return false, a.fail(fmt.Errorf("batch pull request #%d is being closed, but %s is at %.12s, not at %.12s, which its record froze; a person closes it, or pushes the branch back, by hand", c.PR, c.Branch, h, c.Head))
|
||||
case b.src.open && !push:
|
||||
return false, a.fail(fmt.Errorf("batch pull request #%d is being closed: closing it and deleting %s are left; backport-batch run --line %s --push finishes it", c.PR, c.Branch, b.line.branch()))
|
||||
case b.src.open:
|
||||
if err := a.finishClose(b.line, pullURL(b.src.carrier), c); err != nil {
|
||||
return false, a.fail(err)
|
||||
}
|
||||
return false, a.finished(c, changing)
|
||||
case !onRemote:
|
||||
return true, 0
|
||||
case takenBranch(b, c) != nil:
|
||||
return false, a.fail(takenBranch(b, c))
|
||||
case h != c.Head:
|
||||
return false, a.fail(movedBranch(c, h))
|
||||
case !push:
|
||||
return false, a.fail(fmt.Errorf("batch pull request #%d was closed, but %s is still at %.12s; backport-batch run --line %s --push deletes it", c.PR, c.Branch, h, b.line.branch()))
|
||||
}
|
||||
if err := a.deleteBranch(b.line, c); err != nil {
|
||||
return false, a.fail(err)
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "deleted %s, which the close of #%d left\n", c.Branch, c.PR)
|
||||
return false, a.finished(c, changing)
|
||||
}
|
||||
|
||||
// finished is the exit of a run that finished a close: a change it was asked
|
||||
// for needs an open batch pull request, and none is left to make it on.
|
||||
func (a *app) finished(c closing, changing bool) int {
|
||||
if !changing {
|
||||
return 0
|
||||
}
|
||||
return a.fail(fmt.Errorf("finished the close of #%d and made no other change: what this run was asked to change needs an open batch pull request, which the next run opens", c.PR))
|
||||
}
|
||||
|
||||
// pushArgs pushes head to branch: as a fast-forward, which the remote checks,
|
||||
// or, given a lease, over exactly that commit and no other.
|
||||
func pushArgs(branch, head, lease string) []string {
|
||||
args := []string{"push", "--quiet"}
|
||||
if lease != "" {
|
||||
args = append(args, "--force-with-lease=refs/heads/"+branch+":"+lease)
|
||||
}
|
||||
return append(args, "--end-of-options", remote, head+":refs/heads/"+branch)
|
||||
}
|
||||
|
||||
// deleteArgs deletes branch, only while the remote has it at lease.
|
||||
func deleteArgs(branch, lease string) []string {
|
||||
return []string{"push", "--quiet", "--force-with-lease=refs/heads/" + branch + ":" + lease, "--end-of-options", remote, ":refs/heads/" + branch}
|
||||
}
|
||||
@@ -0,0 +1,318 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A merged pull request lands as its own commits, each picked with -x and
|
||||
// keeping its author and sign-offs, under a merge commit that carries the
|
||||
// merge message from main and names it; a squash lands as one pick; a merge
|
||||
// of main into the pull request's branch is left out; all in main's merge
|
||||
// order.
|
||||
func TestRunPicksInMergeOrder(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
signoff := "Signed-off" + "-by: " + contributor
|
||||
mc5 := f.prCommits(5, "fix(app): five", "Body.\n\n```release-note\nfix(app): five is fixed\n```",
|
||||
change{files: map[string]string{"app/five.txt": "5\n"}, author: contributor, message: "fix(app): start five\n\n" + signoff},
|
||||
change{files: map[string]string{"app/five.txt": "5, done\n", "app/five-more.txt": "more\n"}, author: contributor, message: "fix(app): finish five\n\n" + signoff})
|
||||
sq6 := f.squash(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "feat(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
mc8 := f.prCommits(8, "fix(app): eight", "",
|
||||
change{files: map[string]string{"app/eight.txt": "8\n"}},
|
||||
change{mergeMain: true},
|
||||
change{files: map[string]string{"app/eight.txt": "8, after main\n"}})
|
||||
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.git("branch", "--show-current"); got != batchBranch1 {
|
||||
t.Fatalf("on %q, want %s", got, batchBranch1)
|
||||
}
|
||||
want := []string{"fix(app): five (#5)", "fix(app): six (#6)", "fix(app): eight (#8)"}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, want) {
|
||||
t.Errorf("the batch carries %q, want %q", got, want)
|
||||
}
|
||||
|
||||
line := f.git("rev-parse", "origin/release-1.1")
|
||||
u5 := f.git("rev-parse", "HEAD~2")
|
||||
if got := f.git("rev-parse", u5+"^1"); got != line {
|
||||
t.Errorf("#5's merge sits on %.12s, want the line's head %.12s", got, line)
|
||||
}
|
||||
if msg := f.git("log", "-1", "--format=%B", u5); !strings.HasSuffix(msg, "\n\n"+batchTrailer+": "+mc5) || !strings.HasPrefix(msg, "fix(app): five (#5)") {
|
||||
t.Errorf("#5's merge does not carry the merge message and name %s:\n%s", mc5, msg)
|
||||
}
|
||||
own := strings.Fields(f.git("rev-list", "--reverse", mc5+"^1.."+mc5+"^2"))
|
||||
picks := strings.Fields(f.git("rev-list", "--reverse", u5+"^2", "^"+u5+"^1"))
|
||||
if len(picks) != 2 || len(own) != 2 {
|
||||
t.Fatalf("#5 brought %d picks of its %d commits, want 2", len(picks), len(own))
|
||||
}
|
||||
for i, c := range picks {
|
||||
msg := f.git("log", "-1", "--format=%B", c)
|
||||
if got := f.git("log", "-1", "--format=%an <%ae>", c); got != contributor {
|
||||
t.Errorf("pick %d is authored by %q, want %q", i, got, contributor)
|
||||
}
|
||||
if !strings.Contains(msg, signoff+"\n(cherry picked from commit "+own[i]+")\nSigned-off-by: Batch Test <batch@example.invalid>") {
|
||||
t.Errorf("pick %d lacks the contributor's sign-off, its -x line for %s or the batch's sign-off:\n%s", i, own[i], msg)
|
||||
}
|
||||
}
|
||||
|
||||
u6 := f.git("rev-parse", "HEAD~1")
|
||||
if parents := strings.Fields(f.git("rev-list", "--parents", "-n", "1", u6)); len(parents) != 2 || parents[1] != u5 {
|
||||
t.Errorf("#6 is %v, want one commit on #5's merge", parents)
|
||||
}
|
||||
if msg := f.git("log", "-1", "--format=%B", u6); !strings.Contains(msg, "(cherry picked from commit "+sq6+")") {
|
||||
t.Errorf("#6's pick lacks its -x line:\n%s", msg)
|
||||
}
|
||||
|
||||
picks8 := strings.Fields(f.git("rev-list", "--reverse", "HEAD^2", "^HEAD^1"))
|
||||
if got := len(picks8); got != 2 {
|
||||
t.Errorf("#8 brought %d commits, want its own 2 without the merge of main", got)
|
||||
}
|
||||
if !strings.HasSuffix(f.git("log", "-1", "--format=%B", "HEAD"), batchTrailer+": "+mc8) {
|
||||
t.Error("#8's merge does not name its merge commit")
|
||||
}
|
||||
if f.git("rev-parse", "HEAD:app/eight.txt") != f.git("rev-parse", mc8+":app/eight.txt") {
|
||||
t.Error("app/eight.txt is not what main has after #8")
|
||||
}
|
||||
for _, s := range []string{"[#5]", "[#6]", "[#8]", "fix(app): five is fixed", "Rule 2", "`fix(app): five`, 2 commits", "`fix(app): six`, 1 commit"} {
|
||||
if !strings.Contains(stdout, s) {
|
||||
t.Errorf("the body lacks %q:\n%s", s, stdout)
|
||||
}
|
||||
}
|
||||
if code, out, stderr := f.exec(nil, "verify", "--line", "release-1.1", "--head", f.git("rev-parse", "HEAD")); code != 0 {
|
||||
t.Errorf("verify of the batch = %d\n%s\n%s", code, out, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A conflict is aborted, recorded with its files, and the next original is
|
||||
// still picked.
|
||||
func TestRunRecordsConflictsAndGoesOn(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
f.pr(5, "fix(chart): five", "", map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
|
||||
t.Errorf("picked %q, want only #6", got)
|
||||
}
|
||||
if out := f.git("status", "--porcelain"); out != "" {
|
||||
t.Errorf("the tree is left dirty:\n%s", out)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(f.dir, ".git", "CHERRY_PICK_HEAD")); err == nil {
|
||||
t.Error("a cherry-pick is left in progress")
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
deferred, err := parseDeferred(stdout, l)
|
||||
if err != nil || len(deferred[5].Files) == 0 || len(deferred) != 1 {
|
||||
t.Errorf("deferred = %v, %v; want #5", deferred, err)
|
||||
}
|
||||
if !strings.Contains(stdout, "### Deferred conflicts") || !strings.Contains(stdout, "`chart/values.yaml`") {
|
||||
t.Errorf("the body does not name the conflict and its file:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// An original the earlier picks already carry is skipped as an empty pick.
|
||||
func TestRunSkipsAnEmptyPick(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): add x", "", map[string]string{"app/x.txt": "x\n"})
|
||||
f.pr(6, "revert(app): drop x", "", map[string]string{"app/x.txt": ""})
|
||||
f.pr(7, "fix(app): add x again", "", map[string]string{"app/x.txt": "x\n"})
|
||||
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): add x (#5)"}) {
|
||||
t.Errorf("picked %q, want only #5", got)
|
||||
}
|
||||
if !strings.Contains(stdout, "the earlier picks already carry it") {
|
||||
t.Errorf("#7 is not reported as carried by the earlier picks:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// A hook in the repository or the user's configuration never rewrites what a
|
||||
// pick carries.
|
||||
func TestRunIgnoresHooks(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
hooks := filepath.Join(f.dir, ".git", "hooks")
|
||||
for _, hook := range []string{"prepare-commit-msg", "commit-msg"} {
|
||||
if err := os.WriteFile(filepath.Join(hooks, hook), []byte("#!/bin/sh\necho rewritten by a hook > \"$1\"\n"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if code, _, stderr := f.exec(nil, "run", "--line", "release-1.1"); code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): five (#5)"}) {
|
||||
t.Errorf("picked %q, want #5 as it was merged", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunRefusesADirtyTree(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.write("README.md", "changed\n")
|
||||
code, _, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 2 || !strings.Contains(stderr, "has changes") {
|
||||
t.Errorf("run on a dirty tree = %d (%s), want 2", code, stderr)
|
||||
}
|
||||
if f.git("branch", "--show-current") != "main" {
|
||||
t.Error("the tree was switched anyway")
|
||||
}
|
||||
|
||||
// A pick that stopped empty leaves the tree clean and the pick going.
|
||||
f.git("checkout", "-q", "--", "README.md")
|
||||
if err := os.WriteFile(filepath.Join(f.dir, ".git", "CHERRY_PICK_HEAD"), []byte(f.git("rev-parse", "HEAD")+"\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
code, _, stderr = f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 2 || !strings.Contains(stderr, "operation is in progress (CHERRY_PICK_HEAD)") {
|
||||
t.Errorf("run during a cherry-pick = %d (%s), want 2", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A clean tree says nothing about commits only the local batch branch has, so
|
||||
// a run or a rebuild that resets that branch over them names its tip and the
|
||||
// command that gets them back; one that drops nothing says nothing.
|
||||
func TestResetNamesWhatItDrops(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
unpushed := func(base string) string {
|
||||
t.Helper()
|
||||
f.git("switch", "-q", "-C", batchBranch1, base)
|
||||
f.write("app/mine.txt", "unpushed\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "an unpushed resolution")
|
||||
tip := f.git("rev-parse", "HEAD")
|
||||
f.git("switch", "-q", "main")
|
||||
return tip
|
||||
}
|
||||
notice := func(stderr, tip string) bool {
|
||||
t.Helper()
|
||||
restore := "git branch " + batchBranch1 + "-" + tip[:12] + " " + tip
|
||||
if !strings.Contains(stderr, "resetting the local "+batchBranch1+" from "+tip+",") || !strings.Contains(stderr, restore) {
|
||||
return false
|
||||
}
|
||||
f.git(strings.Fields(restore)[1:]...)
|
||||
if got := f.git("rev-parse", batchBranch1+"-"+tip[:12]); got != tip {
|
||||
t.Errorf("%s restored %.12s, want %.12s", restore, got, tip)
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
tip := unpushed("origin/release-1.1")
|
||||
if code, _, stderr := f.exec(nil, "run", "--line", "release-1.1"); code != 0 || !notice(stderr, tip) {
|
||||
t.Errorf("run over an unpushed commit = %d, want it named with its restore:\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
f.git("branch", "-f", batchBranch1, "origin/release-1.1~1")
|
||||
if code, _, stderr := f.exec(nil, "run", "--line", "release-1.1"); code != 0 || strings.Contains(stderr, "resetting") {
|
||||
t.Errorf("run over a branch it fast-forwards = %d, want no notice:\n%s", code, stderr)
|
||||
}
|
||||
|
||||
f.git("switch", "-q", "main")
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
tip = unpushed(head)
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "6", "--reason", "fails the app e2e"); code != 0 || !notice(stderr, tip) {
|
||||
t.Errorf("a rebuild over an unpushed commit = %d, want it named with its restore:\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// --push pushes a fresh branch and opens the draft, and a second push of the
|
||||
// same branch updates the open pull request instead.
|
||||
func TestRunPush(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
if !strings.Contains(stderr, "opened draft https://github.com/cozystack/cozystack/pull/90\n") {
|
||||
t.Errorf("the run does not print the pull request it opened:\n%s", stderr)
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
if _, err := parseDeferred(stdout, l); err != nil || !strings.HasPrefix(stdout, "## What this PR does") || strings.Contains(stdout, "https://github.com/cozystack/cozystack/pull/90") {
|
||||
t.Errorf("stdout is no longer the body alone (%v):\n%s", err, stdout)
|
||||
}
|
||||
head := f.git("rev-parse", "HEAD")
|
||||
if got := f.remoteHead(batchBranch1); got != head {
|
||||
t.Errorf("remote %s = %q, want %s", batchBranch1, got, head)
|
||||
}
|
||||
if want := []string{"release-1.1 " + batchBranch1 + " fix(backport): batch for release-1.1, 2026-10-26"}; !slices.Equal(gh.created, want) {
|
||||
t.Errorf("created %q, want %q", gh.created, want)
|
||||
}
|
||||
|
||||
f.git("switch", "-q", "main")
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, gh.bodies[0])}
|
||||
code, _, stderr = f.exec(gh, "run", "--line", "release-1.1", "--onto", head, "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --onto --push = %d\n%s", code, stderr)
|
||||
}
|
||||
if !strings.Contains(stderr, "updated https://github.com/cozystack/cozystack/pull/90\n") {
|
||||
t.Errorf("the run does not print the pull request it updated:\n%s", stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): five (#5)", "fix(app): six (#6)"}) {
|
||||
t.Errorf("the pushed batch carries %q, want #5 then #6", got)
|
||||
}
|
||||
if len(gh.edited) != 1 || gh.edited[0] != "#90 fix(backport): batch for release-1.1, 2026-10-26" {
|
||||
t.Errorf("edited %q, want #90", gh.edited)
|
||||
}
|
||||
if body := gh.bodies[len(gh.bodies)-1]; !strings.Contains(body, "[#5]") || !strings.Contains(body, "[#6]") {
|
||||
t.Errorf("the updated body does not list both originals:\n%s", body)
|
||||
}
|
||||
}
|
||||
|
||||
// With a batch pull request open on another branch, a fresh push would open
|
||||
// a second one; it is refused before anything is pushed.
|
||||
func TestRunPushRefusesASecondBatch(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
old := "backport-batch/release-1.1-20261022"
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+old)
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{prs: []pullRequest{f.batchPR(90, "release-1.1", old, "")}}
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "continue it with --onto") {
|
||||
t.Errorf("run --push = %d (%s), want a refusal", code, stderr)
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != "" || len(gh.created)+len(gh.edited) != 0 {
|
||||
t.Error("something was pushed or opened")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A maintainer's run by hand continues the batch pull request the scheduled
|
||||
// run opened as its bot, with nothing set: it edits that pull request rather
|
||||
// than opening a second one for the line. A record another bot wrote is no
|
||||
// record the batch trusts, and the run stops rather than reach past it.
|
||||
func TestHandRunContinuesTheBotsBatch(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name, author string
|
||||
continued bool
|
||||
}{
|
||||
{"the scheduled run's bot", "cozystack-ci[bot]", true},
|
||||
{"another bot", "renovate[bot]", false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{bot: batchBotLogin(func(string) (string, bool) { return "", false })}
|
||||
openBatch(t, f, gh)
|
||||
gh.prs[0].User.Login, gh.prs[0].User.Type = tc.author, "Bot"
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.now = f.now.AddDate(0, 0, 3)
|
||||
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if tc.continued && code != 0 || !tc.continued && (code != 2 || !strings.Contains(stderr, "batch pull request #90 (open) carries a ```backport-batch block but was opened by \"renovate[bot]\"")) {
|
||||
t.Fatalf("run --continue = %d\n%s", code, stderr)
|
||||
}
|
||||
continued := slices.Equal(gh.edited, []string{"#90 fix(backport): batch for release-1.1, 2026-10-26"}) && len(gh.created) == 1
|
||||
if continued != tc.continued || len(gh.created) != 1 {
|
||||
t.Errorf("edited %q, created %q; want the batch of %s continued=%v and no second one\n%s", gh.edited, gh.created, tc.author, tc.continued, stderr)
|
||||
}
|
||||
if !tc.continued && !strings.Contains(stderr, "ignoring #90") {
|
||||
t.Errorf("the run does not say it ignores #90:\n%s", stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The bot comes from BACKPORT_BATCH_BOT when it is set, empty included, and
|
||||
// is the scheduled run's own bot when it is not.
|
||||
func TestBatchBotLogin(t *testing.T) {
|
||||
env := func(v string, ok bool) func(string) (string, bool) {
|
||||
return func(key string) (string, bool) {
|
||||
if key != batchBotEnv {
|
||||
t.Errorf("looked up %s", key)
|
||||
}
|
||||
return v, ok
|
||||
}
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
name, v string
|
||||
ok bool
|
||||
want string
|
||||
}{
|
||||
{"unset", "", false, "cozystack-ci[bot]"},
|
||||
{"set empty", "", true, ""},
|
||||
{"set", "other-app[bot]", true, "other-app[bot]"},
|
||||
} {
|
||||
if got := batchBotLogin(env(tc.v, tc.ok)); got != tc.want {
|
||||
t.Errorf("%s: batchBotLogin = %q, want %q", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,514 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// Titles, release notes and file names in the body were written by whoever
|
||||
// opened the originals, and the body is published under the name of whoever
|
||||
// runs --push. None of it is searched for what might be dangerous and then
|
||||
// rendered by other rules: a title, a reason or a file name goes whole into a
|
||||
// code span, and the release notes into the fenced release-note block, both
|
||||
// of which GitHub shows as written, with no mention, reference, link or HTML
|
||||
// in them acted on. neutralNote is for a note copied out of that block.
|
||||
|
||||
const (
|
||||
ourRepoURL = "https://github.com/" + upstreamRepo + "/"
|
||||
// tasksURL is where the instructions for the work a batch leaves live.
|
||||
tasksURL = ourRepoURL + "blob/main/docs/release/backport-tasks/"
|
||||
)
|
||||
|
||||
const (
|
||||
titleLimit = 200
|
||||
// GitHub refuses a pull request body longer than this.
|
||||
bodyLimit = 65536
|
||||
)
|
||||
|
||||
var (
|
||||
backtickRunRE = regexp.MustCompile("`+")
|
||||
fenceRunRE = regexp.MustCompile("`{3,}|~{3,}")
|
||||
mdLinkRE = regexp.MustCompile(`!?\[([^\]]*)\]\(([^)]*)\)`)
|
||||
urlRE = regexp.MustCompile("(?i)\\b(?:https?|ftp)://[^\\s<>()\\[\\]`]+|\\bwww\\.[^\\s<>()\\[\\]`]+")
|
||||
// A closing keyword with its target, a cross-repository reference and a
|
||||
// mention, wherever they stand: no exception for what precedes them, so
|
||||
// nothing depends on agreeing with GitHub about word boundaries.
|
||||
actingRE = regexp.MustCompile(`(?i)\b(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\b\s*:?\s*(?:(?:[\w.-]+/[\w.-]+)?#\d+|https?://github\.com/[\w.-]+/[\w.-]+/(?:issues|pull)/\d+)` +
|
||||
`|[\w.-]+/[\w.-]+#\d+|@[A-Za-z0-9][A-Za-z0-9-]*(?:/[A-Za-z0-9_.-]+)?`)
|
||||
// A link into this repository survives, unless a dot segment or an escape
|
||||
// could take it somewhere else.
|
||||
safeURLRE = regexp.MustCompile(`^` + regexp.QuoteMeta(ourRepoURL) + `[\w/?=&#+.-]*$`)
|
||||
releaseNotRE = regexp.MustCompile("(?ms)^[ \t]*```+[ \t]*release-note[ \t]*\n(.*?)^[ \t]*```")
|
||||
headingRE = regexp.MustCompile(`^#{1,6} `)
|
||||
placeholders = []string{"", "none", "n/a", "na", "no", "-", "nothing"}
|
||||
)
|
||||
|
||||
// clean drops what is not text: control characters other than tab and, when
|
||||
// asked, newline, which a terminal acts on, and format characters such as the
|
||||
// zero-width and bidi ones, which make text read as something it is not.
|
||||
// Invalid UTF-8, a raw 8-bit escape among it, becomes U+FFFD.
|
||||
func clean(s string, newlines bool) string {
|
||||
var b strings.Builder
|
||||
for _, r := range strings.ToValidUTF8(s, "\uFFFD") {
|
||||
switch {
|
||||
case r == '\t', r == '\n' && newlines:
|
||||
b.WriteRune(r)
|
||||
case r == '\r' || unicode.Is(unicode.Cf, r):
|
||||
case r == '\n', r == '\u2028', r == '\u2029':
|
||||
b.WriteRune(' ')
|
||||
case unicode.IsControl(r):
|
||||
b.WriteRune('\uFFFD')
|
||||
default:
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func oneLine(s string) string {
|
||||
return strings.Join(strings.FieldsFunc(s, func(r rune) bool { return unicode.IsSpace(r) || unicode.IsControl(r) }), " ")
|
||||
}
|
||||
|
||||
func capped(s string, limit int) string {
|
||||
if r := []rune(s); len(r) > limit {
|
||||
return strings.TrimRightFunc(string(r[:limit-1]), unicode.IsSpace) + "…"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// codeSpan puts s on one line in a code span: its fence is longer than any
|
||||
// backtick run inside, so nothing in s can close it.
|
||||
func codeSpan(s string) string {
|
||||
s = oneLine(clean(s, false))
|
||||
longest := 0
|
||||
for _, run := range backtickRunRE.FindAllString(s, -1) {
|
||||
longest = max(longest, len(run))
|
||||
}
|
||||
fence := strings.Repeat("`", longest+1)
|
||||
pad := ""
|
||||
if s == "" || strings.HasPrefix(s, "`") || strings.HasSuffix(s, "`") {
|
||||
pad = " "
|
||||
}
|
||||
return fence + pad + s + pad + fence
|
||||
}
|
||||
|
||||
// titleSpan is a carried title as the body shows it.
|
||||
func titleSpan(s string) string { return codeSpan(capped(oneLine(clean(s, false)), titleLimit)) }
|
||||
|
||||
func prLink(n int) string {
|
||||
return fmt.Sprintf("[#%d](%spull/%d)", n, ourRepoURL, n)
|
||||
}
|
||||
|
||||
func ours(url string) bool {
|
||||
return safeURLRE.MatchString(url) && !strings.Contains(url, "/.")
|
||||
}
|
||||
|
||||
// inertSigns swaps the characters GitHub acts on for their fullwidth forms,
|
||||
// which read the same and act on nothing. Unlike a code span, the swap does
|
||||
// not depend on how the backticks around it pair up.
|
||||
var inertSigns = strings.NewReplacer("@", "@", "#", "#", "://", "://")
|
||||
|
||||
// neutralNote makes a release note safe inside the fenced block and inert
|
||||
// outside it: no run of backticks or tildes is left to close the fence, links
|
||||
// lose their target unless it is this repository, and closing keywords,
|
||||
// cross-repository references and mentions lose the signs that make them
|
||||
// act. Each step runs on the output of the one before, so a mention a link's
|
||||
// text carried is still found.
|
||||
func neutralNote(s string) string {
|
||||
s = clean(s, true)
|
||||
s = mdLinkRE.ReplaceAllString(s, "$1 ($2)")
|
||||
s = urlRE.ReplaceAllStringFunc(s, func(u string) string {
|
||||
if ours(u) {
|
||||
return u
|
||||
}
|
||||
return "(link removed)"
|
||||
})
|
||||
s = actingRE.ReplaceAllStringFunc(s, inertSigns.Replace)
|
||||
return fenceRunRE.ReplaceAllStringFunc(s, func(f string) string { return strings.Repeat("'", len(f)) })
|
||||
}
|
||||
|
||||
// releaseNote is the release-note block of a body, or "" when it has none or
|
||||
// only a placeholder.
|
||||
func releaseNote(body string) string {
|
||||
m := releaseNotRE.FindStringSubmatch(htmlCommentRE.ReplaceAllString(body, ""))
|
||||
if m == nil {
|
||||
return ""
|
||||
}
|
||||
note := strings.TrimSpace(m[1])
|
||||
if slices.Contains(placeholders, strings.Trim(strings.ToLower(note), " .")) {
|
||||
return ""
|
||||
}
|
||||
return note
|
||||
}
|
||||
|
||||
// deferredFence opens the block check reads the deferred conflicts from.
|
||||
const deferredFence = "```backport-batch"
|
||||
|
||||
// deferredEntry is a conflict, with the commit that failed and its files, an
|
||||
// original --defer took out of the batch, with the reason given, or one that
|
||||
// waits for a maintainer because it changes workflows, with those files.
|
||||
type deferredEntry struct {
|
||||
Original int `json:"original"`
|
||||
Commit string `json:"commit,omitempty"`
|
||||
Files []string `json:"files"`
|
||||
More int `json:"more,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Workflows bool `json:"workflows,omitempty"`
|
||||
}
|
||||
|
||||
// How many conflicting files of one original the body names, in the list and
|
||||
// in the block: a vendored chart can conflict in hundreds.
|
||||
const (
|
||||
filesListed = 5
|
||||
filesKept = 50
|
||||
)
|
||||
|
||||
type deferredRecord struct {
|
||||
Line string `json:"line"`
|
||||
Deferred []deferredEntry `json:"deferred"`
|
||||
}
|
||||
|
||||
// deferredBlock is a version 1 record, which batches carried before records
|
||||
// had a version. encoding/json leaves bidi and zero-width characters as they
|
||||
// are, so file names go through clean first, as the reason does.
|
||||
func deferredBlock(l line, conflicts, held, waiting []pick) string {
|
||||
data, _ := json.Marshal(deferredRecord{Line: l.branch(), Deferred: deferredEntries(conflicts, held, waiting)})
|
||||
return deferredFence + "\n" + string(data) + "\n```"
|
||||
}
|
||||
|
||||
func cleanFiles(files []string) []string {
|
||||
out := make([]string, len(files))
|
||||
for i, f := range files {
|
||||
out[i] = clean(f, false)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// parseDeferred reads what a batch body's record defers.
|
||||
func parseDeferred(body string, l line) (map[int]deferredEntry, error) {
|
||||
rec, err := parseRecord(body, l)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return rec.deferred(), nil
|
||||
}
|
||||
|
||||
// prTitle is built from the line and the date alone, never from carried text.
|
||||
func prTitle(l line, date string) string {
|
||||
return fmt.Sprintf("fix(backport): batch for %s, %s", l.branch(), date)
|
||||
}
|
||||
|
||||
// skipRules are the rules a skipped original is reported under, in order.
|
||||
var skipRules = []int{ruleBeforeStart, ruleOnLine, ruleType, ruleBreaking, ruleTouches, ruleNewer, ruleDropped}
|
||||
|
||||
// report is what one run put on the batch branch and what it left out.
|
||||
type report struct {
|
||||
line line
|
||||
newer *line
|
||||
onBranch []*verdict // carried from --onto and picked now, in merge order
|
||||
deferred []pick // conflicts
|
||||
held []pick // what --defer took out, with the reason
|
||||
waiting []pick // what changes workflows, left to a maintainer
|
||||
skipped map[int][]*verdict
|
||||
other int
|
||||
// overrides are the originals a backport/ label kept out or let in, and
|
||||
// stale those with backport/skip that are still on the batch.
|
||||
overrides []*verdict
|
||||
stale []*verdict
|
||||
start time.Time
|
||||
// shapes says, for an original on the branch, what it brought.
|
||||
shapes map[*verdict]string
|
||||
// missing are originals the policy takes that the run neither tried nor
|
||||
// found anywhere.
|
||||
missing []*verdict
|
||||
// opts is how the run was asked to classify, and built what it read.
|
||||
opts runOptions
|
||||
built builtFrom
|
||||
// blocks are the record and the report the body ends with.
|
||||
blocks string
|
||||
// closing is the close of the open batch this run leaves carrying
|
||||
// nothing.
|
||||
closing *closing
|
||||
// dropped are the maintainers' drops the record keeps.
|
||||
dropped []droppedEntry
|
||||
}
|
||||
|
||||
// ruleName is how output names what kept an original out: the policy's
|
||||
// number for its own rules, and words for what the policy does not number.
|
||||
func ruleName(rule int) string {
|
||||
switch rule {
|
||||
case ruleSkipLabel:
|
||||
return labelSkip
|
||||
case ruleBeforeStart:
|
||||
return "rule 1, before the line's start"
|
||||
case ruleDropped:
|
||||
return "a maintainer's drop"
|
||||
}
|
||||
return "rule " + strconv.Itoa(rule)
|
||||
}
|
||||
|
||||
// listTitle titles the list of what a rule skipped, a drop among them.
|
||||
func (r *report) listTitle(rule int) string {
|
||||
if rule == ruleDropped {
|
||||
return "Dropped from " + codeSpan(r.line.branch()) + " by a maintainer"
|
||||
}
|
||||
return r.ruleTitle(rule)
|
||||
}
|
||||
|
||||
func (r *report) ruleTitle(rule int) string {
|
||||
switch rule {
|
||||
case ruleBeforeStart:
|
||||
return "Rule 1, merged before " + codeSpan(r.line.branch()) + " started under the policy on " + r.start.Format(time.DateOnly)
|
||||
case ruleOnLine:
|
||||
return "Rule 1, already on " + codeSpan(r.line.branch())
|
||||
case ruleType:
|
||||
return "Rule 2, the title type is not `fix` or `security`, or the scope is `deps`"
|
||||
case ruleBreaking:
|
||||
return "Rule 3, a breaking change"
|
||||
case ruleTouches:
|
||||
return "Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change"
|
||||
case ruleNewer:
|
||||
return "Rule 6, not on " + codeSpan(r.newer.branch()) + " yet"
|
||||
}
|
||||
return fmt.Sprintf("Rule %d", rule)
|
||||
}
|
||||
|
||||
// what is the What this PR does section, with the skipped lists of the rules
|
||||
// in details written out.
|
||||
func (r *report) what(details map[int]bool) string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "Backports to %s from `main`, picked by the rules of the [backport policy](%s) in merge order. A merged pull request comes as its own commits, picked with `git cherry-pick -x` so each keeps its author, message and sign-offs, and a merge commit that carries the original's merge message with a `%s:` trailer naming it; a squash-merged one comes as one pick. `backport-batch verify` checks that shape. A pick listed as changed differs from the commit it came from; `git range-diff <merge>^1..<merge>^2 <batch merge>^1..<batch merge>^2` shows how. A conflict or a failing check here is a [backport task](%sREADME.md).\n\n", codeSpan(r.line.branch()), policyURL, batchTrailer, tasksURL)
|
||||
if len(r.onBranch) == 0 {
|
||||
b.WriteString("Nothing is picked yet.\n")
|
||||
}
|
||||
for _, v := range r.onBranch {
|
||||
shape := ""
|
||||
if s := r.shapes[v]; s != "" {
|
||||
shape = ", " + s
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s %s%s\n", prLink(v.o.number), titleSpan(v.o.title), shape)
|
||||
}
|
||||
if len(r.deferred) > 0 {
|
||||
fmt.Fprintf(&b, "\n### Deferred conflicts\n\nThese do not pick cleanly onto this branch. Each names the first of its commits that conflicted and the files it conflicted in, and nothing of it is on the branch. Each moves to the next batch, unless a resolution checked with `backport-batch fold` is pushed here first, as [resolving a conflict](%sresolve-conflict.md) describes.\n\n", tasksURL)
|
||||
for _, p := range r.deferred {
|
||||
var files []string
|
||||
for _, f := range p.files[:min(len(p.files), filesListed)] {
|
||||
files = append(files, codeSpan(f))
|
||||
}
|
||||
if more := len(p.files) - len(files) + p.more; more > 0 {
|
||||
files = append(files, fmt.Sprintf("%d more", more))
|
||||
}
|
||||
at := ""
|
||||
if p.failed != "" {
|
||||
at = " at " + codeSpan(p.failed[:12])
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s %s%s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), at, strings.Join(files, ", "))
|
||||
}
|
||||
}
|
||||
if len(r.held) > 0 {
|
||||
b.WriteString("\n### Deferred to the next batch\n\nTaken out of this batch with `backport-batch run --defer`, each for the reason given. This pull request keeps them out; the next batch picks them again.\n\n")
|
||||
for _, p := range r.held {
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), titleSpan(p.reason))
|
||||
}
|
||||
}
|
||||
if len(r.waiting) > 0 {
|
||||
fmt.Fprintf(&b, "\n### Waiting for a maintainer\n\nThese change `%s`, which the token this batch runs with may not push, so it leaves them out. The policy still takes each, and `backport-batch check` lists it until it is on the line. A maintainer brings one over as [changes to workflows](%s#changes-to-workflows) describes.\n\n", workflowsDir, policyURL)
|
||||
for _, p := range r.waiting {
|
||||
var files []string
|
||||
for _, f := range p.files[:min(len(p.files), filesListed)] {
|
||||
files = append(files, codeSpan(f))
|
||||
}
|
||||
if more := len(p.files) - len(files) + p.more; more > 0 {
|
||||
files = append(files, fmt.Sprintf("%d more", more))
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), strings.Join(files, ", "))
|
||||
}
|
||||
}
|
||||
if len(r.overrides) > 0 {
|
||||
b.WriteString("\n### Maintainer overrides\n\n")
|
||||
for _, v := range r.overrides {
|
||||
fmt.Fprintf(&b, "- %s %s: %s, %s\n", prLink(v.o.number), titleSpan(v.o.title), codeSpan(v.label), r.outcome(v))
|
||||
}
|
||||
}
|
||||
b.WriteString("\n### Skipped\n\n")
|
||||
for _, rule := range skipRules {
|
||||
items := r.skipped[rule]
|
||||
if len(items) == 0 {
|
||||
continue
|
||||
}
|
||||
left := ""
|
||||
if !details[rule] {
|
||||
left = ", too many to list here; the run's output lists them"
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s: %d%s\n", r.listTitle(rule), len(items), left)
|
||||
}
|
||||
if r.other > 0 {
|
||||
fmt.Fprintf(&b, "- Commits on `main` that merged no pull request: %d\n", r.other)
|
||||
}
|
||||
for _, rule := range skipRules {
|
||||
items := r.skipped[rule]
|
||||
if len(items) > 0 && details[rule] {
|
||||
fmt.Fprintf(&b, "\n<details><summary>%s</summary>\n\n", r.listTitle(rule))
|
||||
for _, v := range items {
|
||||
fmt.Fprintf(&b, "- %s %s, %s\n", prLink(v.o.number), titleSpan(v.o.title), titleSpan(v.reason))
|
||||
}
|
||||
b.WriteString("\n</details>\n")
|
||||
}
|
||||
}
|
||||
b.WriteString("\n" + r.blocks + "\n")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// outcome is what became of an original a label overrode.
|
||||
func (r *report) outcome(v *verdict) string {
|
||||
switch {
|
||||
case slices.Contains(r.stale, v) && v.rule == ruleDropped:
|
||||
return "dropped, but still on this branch; take it out with `backport-batch run --defer`"
|
||||
case slices.Contains(r.stale, v):
|
||||
return "still on this branch, picked before the label; take it out with `backport-batch run --defer`"
|
||||
case v.label == labelSkip:
|
||||
return "left out"
|
||||
case slices.Contains(r.onBranch, v):
|
||||
return "on this branch"
|
||||
case slices.ContainsFunc(r.deferred, func(p pick) bool { return p.v == v }):
|
||||
return "a deferred conflict"
|
||||
case slices.ContainsFunc(r.held, func(p pick) bool { return p.v == v }):
|
||||
return "deferred to the next batch"
|
||||
case slices.ContainsFunc(r.waiting, func(p pick) bool { return p.v == v }):
|
||||
return "waiting for a maintainer: it changes " + codeSpan(workflowsDir)
|
||||
case v.rule != 0:
|
||||
return "skipped by " + ruleName(v.rule) + ", " + titleSpan(v.reason)
|
||||
}
|
||||
return "not picked"
|
||||
}
|
||||
|
||||
func (r *report) notes() string {
|
||||
var notes []string
|
||||
for _, v := range r.onBranch {
|
||||
if n := releaseNote(v.o.body); n != "" {
|
||||
notes = append(notes, neutralNote(n))
|
||||
}
|
||||
}
|
||||
return strings.Join(notes, "\n\n")
|
||||
}
|
||||
|
||||
// section is one heading of the pull request template and what follows it.
|
||||
type section struct {
|
||||
heading string
|
||||
text string
|
||||
}
|
||||
|
||||
func sections(template string) (string, []section) {
|
||||
var lead strings.Builder
|
||||
var out []section
|
||||
inFence := false
|
||||
for _, ln := range strings.Split(htmlCommentRE.ReplaceAllString(template, ""), "\n") {
|
||||
if strings.HasPrefix(strings.TrimSpace(ln), "```") {
|
||||
inFence = !inFence
|
||||
}
|
||||
if !inFence && headingRE.MatchString(ln) {
|
||||
out = append(out, section{heading: ln})
|
||||
continue
|
||||
}
|
||||
if len(out) == 0 {
|
||||
lead.WriteString(ln + "\n")
|
||||
} else {
|
||||
out[len(out)-1].text += ln + "\n"
|
||||
}
|
||||
}
|
||||
return strings.TrimSpace(lead.String()), out
|
||||
}
|
||||
|
||||
func headingIs(heading, name string) bool {
|
||||
return strings.EqualFold(strings.TrimSpace(strings.TrimLeft(heading, "#")), name)
|
||||
}
|
||||
|
||||
// body fills the line's own pull request template: the What, Screenshots and
|
||||
// Release note sections are the batch's, every other section stays as the
|
||||
// template has it. A body over GitHub's limit loses its longest skipped lists
|
||||
// first, and its release notes' tail last; left out are the rules whose lists
|
||||
// it lost.
|
||||
func (r *report) body(template string) (body string, left []int) {
|
||||
details := map[int]bool{}
|
||||
for rule, items := range r.skipped {
|
||||
details[rule] = len(items) > 0
|
||||
}
|
||||
notes := r.notes()
|
||||
for {
|
||||
full := r.render(template, r.what(details), notes)
|
||||
if len(full) <= bodyLimit {
|
||||
return full, left
|
||||
}
|
||||
longest := 0
|
||||
for _, rule := range skipRules {
|
||||
if details[rule] && (longest == 0 || len(r.skipped[rule]) > len(r.skipped[longest])) {
|
||||
longest = rule
|
||||
}
|
||||
}
|
||||
if longest == 0 {
|
||||
const marker = "\n\n(release notes truncated; see the original pull requests)"
|
||||
cut := max(0, len(notes)-(len(full)-bodyLimit)-len(marker))
|
||||
for cut > 0 && !utf8Start(notes[cut]) {
|
||||
cut--
|
||||
}
|
||||
return r.render(template, r.what(details), notes[:cut]+marker), left
|
||||
}
|
||||
details[longest] = false
|
||||
left = append(left, longest)
|
||||
}
|
||||
}
|
||||
|
||||
func utf8Start(c byte) bool { return c&0xC0 != 0x80 }
|
||||
|
||||
func (r *report) render(template, what, notes string) string {
|
||||
lead, secs := sections(template)
|
||||
var b strings.Builder
|
||||
if lead != "" {
|
||||
b.WriteString(lead + "\n\n")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, s := range secs {
|
||||
switch {
|
||||
case headingIs(s.heading, "What this PR does"):
|
||||
seen["what"] = true
|
||||
b.WriteString(s.heading + "\n\n" + what + "\n")
|
||||
case headingIs(s.heading, "Screenshots"):
|
||||
b.WriteString(s.heading + "\n\nSee the original pull requests.\n\n")
|
||||
case headingIs(s.heading, "Release note"):
|
||||
seen["note"] = true
|
||||
b.WriteString(s.heading + "\n\n```release-note\n" + notes + "\n```\n\n")
|
||||
default:
|
||||
b.WriteString(s.heading + "\n\n" + strings.TrimSpace(s.text) + "\n\n")
|
||||
}
|
||||
}
|
||||
out := b.String()
|
||||
if !seen["what"] {
|
||||
out = "## What this PR does\n\n" + what + "\n" + out
|
||||
}
|
||||
if !seen["note"] {
|
||||
out += "### Release note\n\n```release-note\n" + notes + "\n```\n"
|
||||
}
|
||||
return strings.TrimSpace(regexp.MustCompile(`\n{3,}`).ReplaceAllString(out, "\n\n")) + "\n"
|
||||
}
|
||||
@@ -0,0 +1,382 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// visible is what of a body GitHub renders as markdown rather than shows as
|
||||
// code: fenced blocks and code spans taken out, delimited as CommonMark
|
||||
// delimits them. Everything carried has to be gone from it.
|
||||
func visible(body string) string {
|
||||
fenceOpen := regexp.MustCompile("^(`{3,}|~{3,})")
|
||||
var out []string
|
||||
fence := ""
|
||||
for _, ln := range strings.Split(body, "\n") {
|
||||
t := strings.TrimLeft(ln, " ")
|
||||
indented := len(ln)-len(t) > 3
|
||||
if fence != "" {
|
||||
run := strings.TrimLeft(t, fence[:1])
|
||||
if !indented && len(t)-len(run) >= len(fence) && strings.TrimSpace(run) == "" {
|
||||
fence = ""
|
||||
}
|
||||
continue
|
||||
}
|
||||
if m := fenceOpen.FindString(t); m != "" && !indented && (m[0] == '~' || !strings.Contains(t[len(m):], "`")) {
|
||||
fence = m
|
||||
continue
|
||||
}
|
||||
out = append(out, stripSpans(ln))
|
||||
}
|
||||
return strings.Join(out, "\n")
|
||||
}
|
||||
|
||||
func stripSpans(s string) string {
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(s); {
|
||||
switch {
|
||||
case s[i] == '\\' && i+1 < len(s):
|
||||
b.WriteString(s[i : i+2])
|
||||
i += 2
|
||||
continue
|
||||
case s[i] != '`':
|
||||
b.WriteByte(s[i])
|
||||
i++
|
||||
continue
|
||||
}
|
||||
j := i
|
||||
for j < len(s) && s[j] == '`' {
|
||||
j++
|
||||
}
|
||||
n, closing := j-i, -1
|
||||
for k := j; k < len(s); {
|
||||
if s[k] != '`' {
|
||||
k++
|
||||
continue
|
||||
}
|
||||
e := k
|
||||
for e < len(s) && s[e] == '`' {
|
||||
e++
|
||||
}
|
||||
if e-k == n {
|
||||
closing = k
|
||||
break
|
||||
}
|
||||
k = e
|
||||
}
|
||||
if closing < 0 {
|
||||
b.WriteString(s[i:j])
|
||||
i = j
|
||||
continue
|
||||
}
|
||||
i = closing + n
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// hostile is carried text built to get out of its span or block: each carries
|
||||
// a marker that must never be visible.
|
||||
var hostile = []string{
|
||||
"zqx01 thanks @alice",
|
||||
"zqx02 (@org/team) please look",
|
||||
"zqx03 [@bob](https://evil.example/x) [docs](https://github.com/cozystack/cozystack/../../evil/repo)",
|
||||
"zqx04 zero\u200b@carol bidi \u202e@dave \ufeff@erin",
|
||||
"zqx05 ``` `@frank` ``` still inside",
|
||||
"zqx06 ` closes a single span, then @grace",
|
||||
"zqx07 `` `` ``` ```` four",
|
||||
"zqx08 Fixes #1, closes other/repo#2, resolves https://github.com/other/repo/issues/3",
|
||||
"zqx09 </details><img src=x onerror=alert(1)><a href=\"https://evil.example\">x</a>",
|
||||
"zqx10 <https://evil.example> www.evil.example <mail@evil.example>",
|
||||
"zqx11 [ref]: https://evil.example",
|
||||
"zqx12 @heidi & <b>",
|
||||
"zqx13 \x1b[31mred\x1b[0m \u009b2J \x07 bell",
|
||||
"zqx14 <!-- comment --> <details><summary>x</summary>",
|
||||
"`zqx15` starts and ends with a backtick`",
|
||||
}
|
||||
|
||||
// hostileNote is a release note whose own block a bare fence line would end,
|
||||
// so it hides one behind a zero-width space, which clean then removes.
|
||||
func hostileNote(i int, s string) string {
|
||||
return fmt.Sprintf("fix(x): zqn%02d %s\n\u200b```\n@ivan zqn%02d after a fence\n~~~\n[@judy](https://evil.example) Fixes #9 <img src=x> \x1b[2J", i, s, i)
|
||||
}
|
||||
|
||||
// Every carried title, reason and release note stays inside a code span or the
|
||||
// release-note block, however it is written.
|
||||
func TestBodyContainsCarriedText(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
for i, s := range hostile {
|
||||
typ := "fix"
|
||||
if i%2 == 1 {
|
||||
typ = "feat" // skipped, so it lands in a <details> list
|
||||
}
|
||||
body := "## What this PR does\n\n```release-note\n" + hostileNote(i, s) + "\n```\n"
|
||||
f.pr(100+i, typ+"(x): "+s, body, map[string]string{fmt.Sprintf("app/%d.txt", i): "x\n"})
|
||||
}
|
||||
gh := &fakeGH{}
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if len(gh.bodies) != 1 {
|
||||
t.Fatalf("gh got %d bodies, want 1", len(gh.bodies))
|
||||
}
|
||||
for _, body := range []string{gh.bodies[0], stdout} {
|
||||
for i := range hostile {
|
||||
title, note := fmt.Sprintf("zqx%02d", i+1), fmt.Sprintf("zqn%02d", i)
|
||||
if i == len(hostile)-1 {
|
||||
title = "zqx15"
|
||||
}
|
||||
if !strings.Contains(body, title) {
|
||||
t.Errorf("%s is missing from the body", title)
|
||||
}
|
||||
if i%2 == 0 && !strings.Contains(body, note) {
|
||||
t.Errorf("%s is missing from the release notes", note)
|
||||
}
|
||||
}
|
||||
if v := visible(body); strings.Contains(v, "zqx") || strings.Contains(v, "zqn") {
|
||||
t.Errorf("carried text is visible outside code:\n%s", v)
|
||||
}
|
||||
for _, r := range body {
|
||||
if r != '\n' && r != '\t' && unicode.IsControl(r) || unicode.Is(unicode.Cf, r) {
|
||||
t.Errorf("the body carries %U", r)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Copied out of the fenced block, a note must still close, ping and link
|
||||
// nothing.
|
||||
func TestNeutralNote(t *testing.T) {
|
||||
cases := []struct{ in, gone string }{
|
||||
{"thanks @alice", "@alice"},
|
||||
{"(@org/team) look", "@org/team"},
|
||||
{"[@bob](https://evil.example)", "@bob"},
|
||||
{"x\u200b@carol", "@carol"},
|
||||
{"a@dave.example", "@dave"},
|
||||
{"Fixes #1", "Fixes #1"},
|
||||
{"closes other/repo#2", "other/repo#2"},
|
||||
{"see other/repo#3", "other/repo#3"},
|
||||
{"resolves https://github.com/other/repo/issues/3", "github.com/other"},
|
||||
{"Fixes https://github.com/cozystack/cozystack/issues/3", "Fixes https://"},
|
||||
{"x``@erin", "@erin"},
|
||||
{"`code @frank here`", "@frank"},
|
||||
{"```@grace```", "@grace"},
|
||||
{"go to https://evil.example/x", "evil.example"},
|
||||
{"go to www.evil.example", "evil.example"},
|
||||
{"[docs](https://evil.example)", "evil.example"},
|
||||
{"[docs](https://github.com/cozystack/cozystack/../../evil/repo)", "evil/repo"},
|
||||
{"line\n```\nescaped", "```"},
|
||||
{"line\n~~~\nescaped", "~~~"},
|
||||
{"zero\u200b``\u200b`", "```"},
|
||||
{"\x1b[2J", "\x1b"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
got := neutralNote(tc.in)
|
||||
if strings.Contains(visible(got), tc.gone) {
|
||||
t.Errorf("neutralNote(%q) = %q, still shows %q", tc.in, got, tc.gone)
|
||||
}
|
||||
}
|
||||
keep := "see https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md"
|
||||
if got := neutralNote(keep); got != keep {
|
||||
t.Errorf("neutralNote(%q) = %q, want a link into this repository kept", keep, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodeSpan(t *testing.T) {
|
||||
cases := []string{"plain", "a `b` c", "``x``", "`lead", "trail`", "", "a\nb", "````", "\x1b[31m`\u202e`"}
|
||||
for _, s := range cases {
|
||||
got := codeSpan(s)
|
||||
if v := visible(got); v != "" {
|
||||
t.Errorf("codeSpan(%q) = %q leaves %q outside the span", s, got, v)
|
||||
}
|
||||
if strings.ContainsAny(got, "\n\x1b\u202e") {
|
||||
t.Errorf("codeSpan(%q) = %q carries a newline or a control", s, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeferredRoundTrip(t *testing.T) {
|
||||
l, _ := parseLine("release-1.6")
|
||||
v := &verdict{o: &original{number: 3735}}
|
||||
held := &verdict{o: &original{number: 3736}}
|
||||
block := deferredBlock(l, []pick{{v: v, files: []string{"a\n```\nb", "<c>&d", "e`f"}}}, []pick{{v: held, reason: "breaks\n```\n<b>e2e</b>"}}, nil)
|
||||
if lines := strings.Split(block, "\n"); len(lines) != 3 {
|
||||
t.Fatalf("the block is %d lines, want 3:\n%s", len(lines), block)
|
||||
}
|
||||
body := "## What this PR does\n\ntext\n\n" + block + "\n\n### Release note\n"
|
||||
got, err := parseDeferred(body, l)
|
||||
if err != nil || len(got) != 2 || len(got[3735].Files) != 3 || got[3735].Reason != "" {
|
||||
t.Errorf("parseDeferred = %v, %v; want the conflict #3735 with its files", got, err)
|
||||
}
|
||||
if e := got[3736]; e.Reason != "breaks ``` <b>e2e</b>" || len(e.Files) != 0 {
|
||||
t.Errorf("parseDeferred = %+v; want #3736 held, for its reason on one line", e)
|
||||
}
|
||||
if strings.ContainsAny(strings.Split(block, "\n")[1], "<>&") {
|
||||
t.Errorf("the JSON line carries raw HTML characters: %s", block)
|
||||
}
|
||||
|
||||
other, _ := parseLine("release-1.5")
|
||||
for name, body := range map[string]string{
|
||||
"another line": body,
|
||||
"two blocks": body + "\n" + block,
|
||||
"not JSON": deferredFence + "\nnot json\n```",
|
||||
"an unknown key": deferredFence + "\n{\"line\":\"release-1.6\",\"deferred\":[],\"x\":1}\n```",
|
||||
"a bad number": deferredFence + "\n{\"line\":\"release-1.6\",\"deferred\":[{\"original\":-1}]}\n```",
|
||||
"a repeat": deferredFence + "\n{\"line\":\"release-1.6\",\"deferred\":[{\"original\":5},{\"original\":5,\"reason\":\"x\"}]}\n```",
|
||||
"a bad commit": deferredFence + "\n{\"line\":\"release-1.6\",\"deferred\":[{\"original\":5,\"commit\":\"-x\"}]}\n```",
|
||||
"a broken fence": deferredFence + "\n{}\nmore\n```",
|
||||
"an empty fence": deferredFence + "\n```",
|
||||
"a trailing line": deferredFence,
|
||||
} {
|
||||
want := l
|
||||
if name == "another line" {
|
||||
want = other
|
||||
}
|
||||
if _, err := parseDeferred(body, want); err == nil {
|
||||
t.Errorf("%s: parseDeferred accepted it", name)
|
||||
}
|
||||
}
|
||||
if got, err := parseDeferred("no block here", l); err != nil || len(got) != 0 {
|
||||
t.Errorf("no block: %v, %v; want nothing deferred", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseNote(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"### Release note\n\n```release-note\nfix(x): y\n```\n": "fix(x): y",
|
||||
"```release-note\nNONE\n```": "",
|
||||
"```release-note\n\n```": "",
|
||||
"<!--\n```release-note\nhidden\n```\n-->": "",
|
||||
"no block": "",
|
||||
}
|
||||
for body, want := range cases {
|
||||
if got := releaseNote(body); got != want {
|
||||
t.Errorf("releaseNote(%q) = %q, want %q", body, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The body is the line's template with the batch's sections filled in, and
|
||||
// fits the size GitHub accepts.
|
||||
func TestBodyFollowsTheTemplate(t *testing.T) {
|
||||
l, _ := parseLine("release-1.6")
|
||||
template := "<!-- tips -->\n\n## What this PR does\n\n\n### Screenshots\n\n<!-- shots -->\n\n### Downstream repositories\n\n- [ ] No downstream repository is affected by this change\n\n### Release note\n\n```release-note\n\n```\n"
|
||||
o := &original{number: 7, title: "fix(x): y", body: "```release-note\nfix(x): note\n```"}
|
||||
r := &report{line: l, onBranch: []*verdict{{o: o}}, skipped: map[int][]*verdict{}, blocks: deferredBlock(l, nil, nil, nil)}
|
||||
body, _ := r.body(template)
|
||||
for _, want := range []string{"## What this PR does\n\nBackports to", "- [#7](https://github.com/cozystack/cozystack/pull/7) `fix(x): y`", "### Downstream repositories\n\n- [ ] No downstream", "```release-note\nfix(x): note\n```", deferredFence} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Errorf("the body lacks %q:\n%s", want, body)
|
||||
}
|
||||
}
|
||||
if strings.Contains(body, "<!--") {
|
||||
t.Errorf("the body keeps template comments:\n%s", body)
|
||||
}
|
||||
if body, _ := r.body("no headings at all"); !strings.Contains(body, "## What this PR does") || !strings.Contains(body, "```release-note") {
|
||||
t.Errorf("a template without the sections gets them added:\n%s", body)
|
||||
}
|
||||
|
||||
long := strings.Repeat("y", 4000)
|
||||
var many []*verdict
|
||||
for i := range 40 {
|
||||
many = append(many, &verdict{o: &original{number: i + 1, title: "fix(x): y", body: "```release-note\n" + long + "\n```"}})
|
||||
}
|
||||
r = &report{line: l, onBranch: many, skipped: map[int][]*verdict{ruleType: many}, blocks: deferredBlock(l, nil, nil, nil)}
|
||||
if body, _ := r.body(template); len(body) > bodyLimit || !strings.Contains(body, "truncated") || !strings.Contains(body, deferredFence) {
|
||||
t.Errorf("a body over the limit is %d bytes (limit %d), truncated=%v", len(body), bodyLimit, strings.Contains(body, "truncated"))
|
||||
}
|
||||
|
||||
// The longest list goes first; a short one and the notes stay whole.
|
||||
var lots, few []*verdict
|
||||
for i := range 600 {
|
||||
lots = append(lots, &verdict{o: &original{number: 1000 + i, title: "feat(x): " + strings.Repeat("z", 150)}, reason: "type feat"})
|
||||
}
|
||||
for i := range 3 {
|
||||
few = append(few, &verdict{o: &original{number: 10 + i, title: "fix(x): carried"}, reason: "picked in 0123456789ab"})
|
||||
}
|
||||
conflict := pick{v: &verdict{o: &original{number: 9, title: "fix(x): conflict"}}}
|
||||
for i := range 80 {
|
||||
conflict.files = append(conflict.files, fmt.Sprintf("chart/f%02d.yaml", i))
|
||||
}
|
||||
r = &report{line: l, onBranch: many[:1], deferred: []pick{conflict}, skipped: map[int][]*verdict{ruleType: lots, ruleOnLine: few}, blocks: deferredBlock(l, []pick{conflict}, nil, nil)}
|
||||
body, left := r.body(template)
|
||||
if !slices.Equal(left, []int{ruleType}) {
|
||||
t.Errorf("left out the lists of rules %v, want only rule 2", left)
|
||||
}
|
||||
for _, want := range []string{"<details><summary>Rule 1", "[#12]", ": 600, too many to list here", "`chart/f04.yaml`, 75 more", `"more":30`} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Errorf("the fitted body lacks %q", want)
|
||||
}
|
||||
}
|
||||
for _, unwanted := range []string{"<details><summary>Rule 2", "[#1000]", "`chart/f05.yaml`", "truncated"} {
|
||||
if strings.Contains(body, unwanted) {
|
||||
t.Errorf("the fitted body keeps %q", unwanted)
|
||||
}
|
||||
}
|
||||
if len(body) > bodyLimit {
|
||||
t.Errorf("the fitted body is %d bytes", len(body))
|
||||
}
|
||||
}
|
||||
|
||||
func TestPRTitle(t *testing.T) {
|
||||
l, _ := parseLine("release-1.6")
|
||||
if got := prTitle(l, "2026-10-23"); got != "fix(backport): batch for release-1.6, 2026-10-23" {
|
||||
t.Errorf("prTitle = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Every task page the body links to is in the tree, under the path the link
|
||||
// names on main.
|
||||
func TestBodyLinksTaskPagesThatExist(t *testing.T) {
|
||||
l, _ := parseLine("release-1.6")
|
||||
v := &verdict{o: &original{number: 9, title: "fix(x): conflict"}}
|
||||
r := &report{line: l, deferred: []pick{{v: v, files: []string{"a"}}}, held: []pick{{v: &verdict{o: &original{number: 10}}, reason: "r"}}, skipped: map[int][]*verdict{}}
|
||||
body, _ := r.body("")
|
||||
links := regexp.MustCompile(regexp.QuoteMeta(tasksURL)+`([\w.-]+)`).FindAllStringSubmatch(body, -1)
|
||||
if len(links) < 2 {
|
||||
t.Fatalf("the body links %d task pages, want the index and the conflict task:\n%s", len(links), body)
|
||||
}
|
||||
for _, m := range links {
|
||||
if _, err := os.Stat(filepath.Join("..", "..", "docs", "release", "backport-tasks", m[1])); err != nil {
|
||||
t.Errorf("the body links %s, which is not in the tree: %v", m[0], err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A body that what it has to carry pushes over GitHub's limit is refused
|
||||
// before anything is written, the replay of an earlier report included,
|
||||
// although the report alone would fit a comment.
|
||||
func TestBodyRefusesMandatoryContentOverTheLimit(t *testing.T) {
|
||||
f, gh, head := bigWaits(t, 4)
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--no-workflow-files", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "with only what it cannot leave out, over GitHub's 65536") {
|
||||
t.Errorf("run --push = %d, want a refusal of the body:\n%s", code, stderr)
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != head || len(gh.issueComments) != 0 || len(gh.edited) != 0 {
|
||||
t.Errorf("the refused run moved the batch to %.12s, wrote %d comments and edited %q", f.remoteHead(batchBranch1), len(gh.issueComments), gh.edited)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,293 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
// check says whether every original the policy takes is on the line, on the
|
||||
// open batch pull request, or listed there as deferred, a conflict or one
|
||||
// taken out with --defer, or as waiting for a maintainer because it changes
|
||||
// workflows, each of which warns without failing. With --cut it is the gate
|
||||
// before a cut, which tags the line: an original only on the open batch pull
|
||||
// request fails it too, since the cut would not carry it, and so does one that
|
||||
// waits, which nobody decided to leave out.
|
||||
func (a *app) check(args []string) int {
|
||||
fs := a.policyFlags("check")
|
||||
lineArg := fs.String("line", "", "release line, release-X.Y")
|
||||
cut := fs.Bool("cut", false, "fail too while a change is only on the open batch PR, as before a cut")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
l, err := parseLine(*lineArg)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := fetch(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
b, err := a.prepare(l)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := a.warnLeftBranch(b); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
disp, err := b.dispositions(*cut)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
var missing, failing []string
|
||||
for _, v := range b.verdicts {
|
||||
d, ok := disp[v]
|
||||
switch {
|
||||
case !ok:
|
||||
case d.kind == outcomeMissing:
|
||||
missing = append(missing, d.text)
|
||||
case d.kind == outcomeUnmerged || d.kind == outcomeWaiting:
|
||||
failing = append(failing, d.text)
|
||||
case d.kind == outcomeWarning:
|
||||
a.warn("%s", d.text)
|
||||
case d.kind == outcomeNote:
|
||||
fmt.Fprintf(a.stderr, "note: %s\n", d.text)
|
||||
}
|
||||
}
|
||||
if len(missing)+len(failing) == 0 {
|
||||
if *cut {
|
||||
fmt.Fprintf(a.stdout, "%s: no change the policy takes is missing before the cut\n", l.branch())
|
||||
} else {
|
||||
fmt.Fprintf(a.stdout, "%s: every change the policy takes is on the line or on the batch\n", l.branch())
|
||||
}
|
||||
return 0
|
||||
}
|
||||
where := "and no batch pull request is open"
|
||||
if b.pr != nil {
|
||||
where = "nor on batch pull request #" + strconv.Itoa(b.pr.Number)
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
fmt.Fprintf(a.stdout, "%s: %d changes the policy takes are not on the line, %s:\n", l.branch(), len(missing), where)
|
||||
for _, text := range missing {
|
||||
fmt.Fprintf(a.stdout, " %s\n", text)
|
||||
}
|
||||
}
|
||||
for _, text := range failing {
|
||||
fmt.Fprintf(a.stdout, "%s: %s\n", l.branch(), text)
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
// warnLeftBranch names the branch a closed batch's close left behind.
|
||||
func (a *app) warnLeftBranch(b *batch) error {
|
||||
if b.src.open || b.src.rec == nil || b.src.rec.Closing == nil {
|
||||
return nil
|
||||
}
|
||||
c := b.src.rec.Closing
|
||||
heads, err := remoteBatchHeads(b.line)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch h, ok := heads[c.Branch]; {
|
||||
case !ok:
|
||||
case takenBranch(b, *c) != nil:
|
||||
a.warn("%v", takenBranch(b, *c))
|
||||
case h == c.Head:
|
||||
a.warn("batch pull request #%d is closed, but its branch %s is still at %.12s; backport-batch run --line %s --push deletes it", c.PR, c.Branch, h, b.line.branch())
|
||||
default:
|
||||
a.warn("%v", movedBranch(*c, h))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// outcome is what check makes of one original.
|
||||
type outcome int
|
||||
|
||||
const (
|
||||
// outcomeMissing fails check and the cut: nothing accounts for an
|
||||
// original the line lacks.
|
||||
outcomeMissing outcome = iota + 1
|
||||
// outcomeUnmerged and outcomeWaiting fail the cut only: the cut would not
|
||||
// carry what is only on the open batch, nor what waits there.
|
||||
outcomeUnmerged
|
||||
outcomeWaiting
|
||||
outcomeWarning
|
||||
outcomeNote
|
||||
)
|
||||
|
||||
type disposition struct {
|
||||
kind outcome
|
||||
text string
|
||||
}
|
||||
|
||||
// dispositions says what check reports about each original that needs a line
|
||||
// of its own; the others pass in silence. The text is what follows the line's
|
||||
// name, or the original alone for one that is missing.
|
||||
func (b *batch) dispositions(cut bool) (map[*verdict]disposition, error) {
|
||||
if c := b.src.closingOpen(); c != nil {
|
||||
return nil, fmt.Errorf("batch pull request #%d is being closed; backport-batch run --line %s --push finishes it", c.PR, b.line.branch())
|
||||
}
|
||||
var stale []*verdict
|
||||
if b.pr != nil {
|
||||
var err error
|
||||
if stale, err = keptOut(b, b.picks); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
// A record a closed batch left still says what a maintainer decided, but
|
||||
// nothing waits on a batch that is gone.
|
||||
deferred, on := b.src.rec.deferred(), ""
|
||||
if c := b.src.carrier; c != nil {
|
||||
on = fmt.Sprintf("#%d", c.Number)
|
||||
if !b.src.open {
|
||||
on += " (" + prState(c) + ")"
|
||||
}
|
||||
}
|
||||
out := map[*verdict]disposition{}
|
||||
for _, v := range b.verdicts {
|
||||
named := v.o.url() + " " + terminalSafe(v.o.title)
|
||||
e, isDeferred := deferred[v.o.number]
|
||||
switch {
|
||||
case slices.Contains(stale, v):
|
||||
out[v] = disposition{outcomeWarning, fmt.Sprintf("#%d is still on batch #%d; take it out with backport-batch run --defer: %s", v.o.number, b.pr.Number, named)}
|
||||
case v.rule == ruleDropped:
|
||||
d := b.drops[v.o.number]
|
||||
out[v] = disposition{outcomeNote, fmt.Sprintf("#%d is dropped from %s on #%d (%s): %s", v.o.number, b.line.branch(), d.On, terminalSafe(d.Reason), named)}
|
||||
case v.rule != 0:
|
||||
case v.onBatch != "":
|
||||
if cut {
|
||||
out[v] = disposition{outcomeUnmerged, fmt.Sprintf("#%d is only on open batch PR #%d, merge it before the cut: %s", v.o.number, b.pr.Number, named)}
|
||||
}
|
||||
case isDeferred && e.Workflows && !b.src.open:
|
||||
out[v] = disposition{outcomeMissing, named}
|
||||
case isDeferred && e.Workflows && cut:
|
||||
out[v] = disposition{outcomeWaiting, fmt.Sprintf("#%d changes %s and waits on batch PR %s for a maintainer, bring it over or defer it before the cut: %s", v.o.number, workflowsDir, on, named)}
|
||||
case isDeferred && e.Workflows:
|
||||
out[v] = disposition{outcomeWarning, fmt.Sprintf("waiting on %s for a maintainer, since it changes %s: %s", on, workflowsDir, named)}
|
||||
case isDeferred && e.Reason != "":
|
||||
out[v] = disposition{outcomeWarning, fmt.Sprintf("deferred on %s to the next batch: %s (%s)", on, named, terminalSafe(e.Reason))}
|
||||
case isDeferred:
|
||||
out[v] = disposition{outcomeWarning, fmt.Sprintf("deferred conflict on %s: %s", on, named)}
|
||||
default:
|
||||
out[v] = disposition{outcomeMissing, named}
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// terminalSafe keeps a title from carrying escape sequences into a terminal.
|
||||
func terminalSafe(s string) string { return capped(oneLine(clean(s, false)), titleLimit) }
|
||||
|
||||
// fold checks a hand-made resolution of a deferred conflict before anyone
|
||||
// pushes it onto the batch: one original on the batch head, in the shape the
|
||||
// batch carries it, inside the files that original touches.
|
||||
func (a *app) fold(args []string) int {
|
||||
fs := a.policyFlags("fold")
|
||||
lineArg := fs.String("line", "", "release line, release-X.Y")
|
||||
number := fs.Int("original", 0, "number of the original pull request")
|
||||
commit := fs.String("commit", "", "the resolution: the merge commit, or for a squash-merged original its one commit")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
l, err := parseLine(*lineArg)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if *number <= 0 {
|
||||
return a.fail(fmt.Errorf("--original %d is not a pull request number", *number))
|
||||
}
|
||||
if err := checkSHA(*commit); err != nil {
|
||||
return a.fail(fmt.Errorf("--commit: %w", err))
|
||||
}
|
||||
if err := fetch(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
b, err := a.prepare(l)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if c := b.src.closingOpen(); c != nil {
|
||||
return a.fail(fmt.Errorf("batch pull request #%d is being closed; backport-batch run --line %s --push finishes it", c.PR, l.branch()))
|
||||
}
|
||||
if err := a.warnLeftBranch(b); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
why, err := refuseFold(b, *number, *commit)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if why != "" {
|
||||
fmt.Fprintf(a.stdout, "refused: %s\n", why)
|
||||
return 1
|
||||
}
|
||||
fmt.Fprintln(a.stdout, "ok")
|
||||
return 0
|
||||
}
|
||||
|
||||
// refuseFold says why commit is not a fold of original n onto the batch, or ""
|
||||
// when it is one: for a merged pull request a merge whose first parent is the
|
||||
// batch head and whose second parent ends a straight line of picks of the pull
|
||||
// request's own commits, in order, each with its -x line, naming the original
|
||||
// in its Backported-from trailer; for a squash-merged one a single pick of its
|
||||
// commit on main. Pushed as it is, either is a fast-forward of the batch.
|
||||
func refuseFold(b *batch, n int, commit string) (string, error) {
|
||||
v := b.find(n)
|
||||
switch {
|
||||
case v == nil:
|
||||
return fmt.Sprintf("#%d did not merge into main after %s branched", n, b.line.branch()), nil
|
||||
case v.rule == ruleSkipLabel:
|
||||
return fmt.Sprintf("#%d carries %s", n, labelSkip), nil
|
||||
case v.rule != 0:
|
||||
return fmt.Sprintf("#%d is not one the batch takes: %s, %s", n, ruleName(v.rule), terminalSafe(v.reason)), nil
|
||||
case b.pr == nil:
|
||||
return fmt.Sprintf("no batch pull request is open on %s", b.line.branch()), nil
|
||||
case v.onBatch != "":
|
||||
return fmt.Sprintf("batch pull request #%d already carries #%d in %.12s", b.pr.Number, n, v.onBatch), nil
|
||||
}
|
||||
have, err := lookup(commit)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if have == "" {
|
||||
return fmt.Sprintf("%.12s is not a commit in this repository", commit), nil
|
||||
}
|
||||
u, err := readUnit(commit)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
want := 1
|
||||
if v.o.parents > 1 {
|
||||
want = 2
|
||||
}
|
||||
switch {
|
||||
case len(u.parents) != want && want == 2:
|
||||
return fmt.Sprintf("%.12s has %d parents; #%d is a merged pull request, so a fold is a merge of its picked commits onto the batch head", commit, len(u.parents), n), nil
|
||||
case len(u.parents) != want:
|
||||
return fmt.Sprintf("%.12s has %d parents; #%d merged as one commit, so a fold is one commit with one parent", commit, len(u.parents), n), nil
|
||||
case u.parents[0] != b.prHead:
|
||||
return fmt.Sprintf("%.12s is on %.12s, not on the head of batch pull request #%d (%.12s)", commit, u.parents[0], b.pr.Number, b.prHead), nil
|
||||
}
|
||||
why, carried, err := checkUnit(b, u)
|
||||
switch {
|
||||
case err != nil || why != "":
|
||||
return why, err
|
||||
case carried != v:
|
||||
return fmt.Sprintf("%.12s carries #%d, not #%d", commit, carried.o.number, n), nil
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
@@ -0,0 +1,457 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
func TestCheck(t *testing.T) {
|
||||
type state struct {
|
||||
f *fixture
|
||||
gh *fakeGH
|
||||
mc5 string
|
||||
mc6 string
|
||||
}
|
||||
setup := func(t *testing.T) *state {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
s := &state{f: f, gh: &fakeGH{}}
|
||||
s.mc5 = f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
s.mc6 = f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "feat(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
return s
|
||||
}
|
||||
// batch pushes a batch branch carrying the given merges and opens its
|
||||
// pull request with body.
|
||||
batch := func(s *state, body string, merges ...string) {
|
||||
s.f.git("switch", "-q", "-c", "batch", "origin/release-1.1")
|
||||
for _, mc := range merges {
|
||||
s.f.git("cherry-pick", "-x", "-m", "1", mc)
|
||||
}
|
||||
s.f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
s.f.git("switch", "-q", "main")
|
||||
s.gh.prs = append(s.gh.prs, s.f.batchPR(90, "release-1.1", batchBranch1, body))
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
deferred6 := deferredBlock(l, []pick{{v: &verdict{o: &original{number: 6}}, files: []string{"app/six.txt"}}}, nil, nil)
|
||||
held6 := deferredBlock(l, nil, []pick{{v: &verdict{o: &original{number: 6}}, reason: "fails the app e2e"}}, nil)
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
arrange func(s *state)
|
||||
cut bool
|
||||
code int
|
||||
listed []string
|
||||
notListed []string
|
||||
warnedAbout string
|
||||
}{
|
||||
{
|
||||
name: "nothing picked, no batch",
|
||||
arrange: func(*state) {},
|
||||
code: 1,
|
||||
listed: []string{"/pull/5 ", "/pull/6 "},
|
||||
notListed: []string{"/pull/7"},
|
||||
},
|
||||
{
|
||||
name: "both on the line",
|
||||
arrange: func(s *state) {
|
||||
s.f.onLine("release-1.1", "fix(app): five (#5)\n\n(cherry picked from commit "+s.mc5+")", map[string]string{"app/five.txt": "5\n"})
|
||||
s.f.onLine("release-1.1", "[Backport release-1.1] fix(app): six (#80)\n\nBackport of #6.", map[string]string{"app/six.txt": "6\n"})
|
||||
},
|
||||
code: 0,
|
||||
},
|
||||
{
|
||||
name: "both on the open batch",
|
||||
arrange: func(s *state) { batch(s, "", s.mc5, s.mc6) },
|
||||
code: 0,
|
||||
},
|
||||
{
|
||||
name: "--cut: both on the open batch, which the cut would not carry",
|
||||
arrange: func(s *state) { batch(s, "", s.mc5, s.mc6) },
|
||||
cut: true,
|
||||
code: 1,
|
||||
listed: []string{"#5 is only on open batch PR #90, merge it before the cut: https://github.com/cozystack/cozystack/pull/5 ", "#6 is only on open batch PR #90, merge it before the cut: https://github.com/cozystack/cozystack/pull/6 "},
|
||||
},
|
||||
{
|
||||
name: "--cut: both on the line",
|
||||
arrange: func(s *state) {
|
||||
s.f.onLine("release-1.1", "fix(app): five (#5)\n\n(cherry picked from commit "+s.mc5+")", map[string]string{"app/five.txt": "5\n"})
|
||||
s.f.onLine("release-1.1", "fix(app): six (#6)\n\n(cherry picked from commit "+s.mc6+")", map[string]string{"app/six.txt": "6\n"})
|
||||
batch(s, "")
|
||||
},
|
||||
cut: true,
|
||||
code: 0,
|
||||
},
|
||||
{
|
||||
name: "--cut: one on the line, the other deferred to the next batch, which stays a warning",
|
||||
arrange: func(s *state) {
|
||||
s.f.onLine("release-1.1", "fix(app): five (#5)\n\n(cherry picked from commit "+s.mc5+")", map[string]string{"app/five.txt": "5\n"})
|
||||
batch(s, "body\n\n"+held6+"\n")
|
||||
},
|
||||
cut: true,
|
||||
code: 0,
|
||||
warnedAbout: "/pull/6 ",
|
||||
},
|
||||
{
|
||||
name: "one on the batch, one missing",
|
||||
arrange: func(s *state) { batch(s, "", s.mc5) },
|
||||
code: 1,
|
||||
listed: []string{"/pull/6 "},
|
||||
notListed: []string{"/pull/5 "},
|
||||
},
|
||||
{
|
||||
name: "the missing one is a deferred conflict on the batch",
|
||||
arrange: func(s *state) { batch(s, "body\n\n"+deferred6+"\n", s.mc5) },
|
||||
code: 0,
|
||||
warnedAbout: "/pull/6 ",
|
||||
},
|
||||
{
|
||||
name: "the missing one is deferred to the next batch",
|
||||
arrange: func(s *state) { batch(s, "body\n\n"+held6+"\n", s.mc5) },
|
||||
code: 0,
|
||||
warnedAbout: "/pull/6 ",
|
||||
},
|
||||
{
|
||||
name: "a fork's pull request named like a batch claims nothing",
|
||||
arrange: func(s *state) {
|
||||
batch(s, "body\n\n"+deferred6+"\n", s.mc5)
|
||||
s.gh.prs[0].Head.Repo.FullName = "someone/cozystack"
|
||||
},
|
||||
code: 1,
|
||||
listed: []string{"/pull/5 ", "/pull/6 "},
|
||||
},
|
||||
{
|
||||
name: "a batch opened by a member with read access claims nothing",
|
||||
arrange: func(s *state) {
|
||||
batch(s, "body\n\n", s.mc5)
|
||||
s.gh.perms = map[string]string{"maintainer": "read"}
|
||||
},
|
||||
code: 1,
|
||||
listed: []string{"/pull/5 ", "/pull/6 "},
|
||||
},
|
||||
{
|
||||
// An older record is never taken in place of a newer one nobody
|
||||
// can vouch for.
|
||||
name: "a record by a member with read access stops it",
|
||||
arrange: func(s *state) {
|
||||
batch(s, "body\n\n"+deferred6+"\n", s.mc5)
|
||||
s.gh.perms = map[string]string{"maintainer": "read"}
|
||||
},
|
||||
code: 2,
|
||||
},
|
||||
{
|
||||
name: "a deferred list for another line",
|
||||
arrange: func(s *state) { batch(s, strings.Replace(deferred6, "release-1.1", "release-1.0", 1), s.mc5) },
|
||||
code: 2,
|
||||
},
|
||||
{
|
||||
name: "GitHub cannot be read",
|
||||
arrange: func(s *state) { s.gh.err = errString("gh: HTTP 502") },
|
||||
code: 2,
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
s := setup(t)
|
||||
tc.arrange(s)
|
||||
args := []string{"check", "--line", "release-1.1"}
|
||||
if tc.cut {
|
||||
args = append(args, "--cut")
|
||||
}
|
||||
code, stdout, stderr := s.f.exec(s.gh, args...)
|
||||
if code != tc.code {
|
||||
t.Fatalf("check = %d, want %d\nstdout:\n%s\nstderr:\n%s", code, tc.code, stdout, stderr)
|
||||
}
|
||||
for _, want := range tc.listed {
|
||||
if !strings.Contains(stdout, want) {
|
||||
t.Errorf("stdout does not list %q:\n%s", want, stdout)
|
||||
}
|
||||
}
|
||||
for _, unwanted := range tc.notListed {
|
||||
if strings.Contains(stdout, unwanted) {
|
||||
t.Errorf("stdout lists %q:\n%s", unwanted, stdout)
|
||||
}
|
||||
}
|
||||
if tc.warnedAbout != "" && (!strings.Contains(stderr, "warning: deferred") || !strings.Contains(stderr, tc.warnedAbout)) {
|
||||
t.Errorf("stderr does not warn about %q:\n%s", tc.warnedAbout, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Every category an original can be in, through the commands that put it
|
||||
// there, with what check and check --cut make of it: the exit code, and the
|
||||
// warning or note on stderr that leaves it 0.
|
||||
func TestCheckDispositions(t *testing.T) {
|
||||
plain := func(t *testing.T) *fixture {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
return f
|
||||
}
|
||||
// conflicting has #5 conflict on the line, beside #6, which picks.
|
||||
conflicting := func(t *testing.T) *fixture {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only five", map[string]string{"app/five.txt": "line\n"})
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
return f
|
||||
}
|
||||
must := func(t *testing.T, f *fixture, gh *fakeGH, args ...string) {
|
||||
t.Helper()
|
||||
if code, _, stderr := f.exec(gh, args...); code != 0 {
|
||||
t.Fatalf("%q = %d\n%s", args, code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if len(gh.bodies) > 0 {
|
||||
reopen(f, gh)
|
||||
}
|
||||
}
|
||||
sixOnLine := func(f *fixture) { pickOnLine(f, 6, map[string]string{"app/six.txt": "6\n"}) }
|
||||
cases := []struct {
|
||||
name string
|
||||
arrange func(t *testing.T) (*fixture, *fakeGH)
|
||||
line string
|
||||
check int
|
||||
cut int
|
||||
checkErr string // on stderr from check
|
||||
cutOut string // on stdout from check --cut
|
||||
cutErr string // on stderr from check --cut
|
||||
notCutOut string
|
||||
}{
|
||||
{
|
||||
name: "missing",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) { return plain(t), &fakeGH{} },
|
||||
check: 1, cut: 1, cutOut: "/pull/5 ",
|
||||
},
|
||||
{
|
||||
name: "carried",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f := plain(t)
|
||||
pickOnLine(f, 5, map[string]string{"app/five.txt": "5\n"})
|
||||
sixOnLine(f)
|
||||
return f, &fakeGH{}
|
||||
},
|
||||
check: 0, cut: 0, notCutOut: "/pull/5",
|
||||
},
|
||||
{
|
||||
name: "picked, on the open batch",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f, gh := plain(t), &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
return f, gh
|
||||
},
|
||||
check: 0, cut: 1, cutOut: "#5 is only on open batch PR #90",
|
||||
},
|
||||
{
|
||||
name: "deferred conflict, open record",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f, gh := conflicting(t), &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
sixOnLine(f)
|
||||
return f, gh
|
||||
},
|
||||
check: 0, cut: 0, checkErr: "warning: deferred conflict on #90: https://github.com/cozystack/cozystack/pull/5 ", cutErr: "warning: deferred conflict on #90",
|
||||
},
|
||||
{
|
||||
name: "deferred with a reason, open record",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f, gh := plain(t), &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
must(t, f, gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "fails the app e2e", "--push")
|
||||
sixOnLine(f)
|
||||
return f, gh
|
||||
},
|
||||
check: 0, cut: 0, checkErr: "warning: deferred on #90 to the next batch: https://github.com/cozystack/cozystack/pull/5 ", cutErr: "(fails the app e2e)",
|
||||
},
|
||||
{
|
||||
name: "workflow wait, open record, beside a pick",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f, gh := workflowWorld(t), &fakeGH{}
|
||||
must(t, f, gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push")
|
||||
return f, gh
|
||||
},
|
||||
check: 0, cut: 1, checkErr: "warning: waiting on #90 for a maintainer", cutOut: "#5 changes .github/workflows/ and waits on batch PR #90",
|
||||
},
|
||||
{
|
||||
name: "workflow wait, open record, alone",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f, gh := workflowWorld(t), &fakeGH{}
|
||||
must(t, f, gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push")
|
||||
pickOnLine(f, 6, map[string]string{"app/six.txt": "6\n", ".github/renovate.json": "{}\n"})
|
||||
return f, gh
|
||||
},
|
||||
check: 0, cut: 1, checkErr: "warning: waiting on #90 for a maintainer", cutOut: "#5 changes .github/workflows/ and waits on batch PR #90",
|
||||
},
|
||||
{
|
||||
name: "backport/skip, not picked",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f := plain(t)
|
||||
sixOnLine(f)
|
||||
return f, &fakeGH{labels: map[string][]int{labelSkip: {5}}}
|
||||
},
|
||||
check: 0, cut: 0, notCutOut: "/pull/5",
|
||||
},
|
||||
{
|
||||
name: "backport/skip after the pick, still on the open batch",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f, gh := plain(t), &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
sixOnLine(f)
|
||||
gh.labels = map[string][]int{labelSkip: {5}}
|
||||
return f, gh
|
||||
},
|
||||
check: 0, cut: 0, checkErr: "warning: #5 is still on batch #90; take it out with backport-batch run --defer", cutErr: "still on batch #90",
|
||||
},
|
||||
{
|
||||
name: "rule 6",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
return f, &fakeGH{}
|
||||
},
|
||||
line: "release-1.0",
|
||||
check: 0, cut: 0, notCutOut: "/pull/5",
|
||||
},
|
||||
{
|
||||
name: "rules 2 to 4",
|
||||
arrange: func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "feat(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
return f, &fakeGH{}
|
||||
},
|
||||
check: 0, cut: 0, notCutOut: "/pull/5",
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f, gh := tc.arrange(t)
|
||||
line := tc.line
|
||||
if line == "" {
|
||||
line = "release-1.1"
|
||||
}
|
||||
code, stdout, stderr := f.exec(gh, "check", "--line", line)
|
||||
if code != tc.check || !strings.Contains(stderr, tc.checkErr) {
|
||||
t.Errorf("check = %d, want %d with %q\nstdout:\n%s\nstderr:\n%s", code, tc.check, tc.checkErr, stdout, stderr)
|
||||
}
|
||||
code, stdout, stderr = f.exec(gh, "check", "--line", line, "--cut")
|
||||
if code != tc.cut || !strings.Contains(stdout, tc.cutOut) || !strings.Contains(stderr, tc.cutErr) {
|
||||
t.Errorf("check --cut = %d, want %d with %q, %q\nstdout:\n%s\nstderr:\n%s", code, tc.cut, tc.cutOut, tc.cutErr, stdout, stderr)
|
||||
}
|
||||
if tc.notCutOut != "" && strings.Contains(stdout, tc.notCutOut) {
|
||||
t.Errorf("check --cut lists %q:\n%s", tc.notCutOut, stdout)
|
||||
}
|
||||
if tc.cut == 0 && !strings.Contains(stdout, "no change the policy takes is missing before the cut") {
|
||||
t.Errorf("check --cut passes without saying so:\n%s", stdout)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Output names a rule as the policy does: the numbers the tool keeps for
|
||||
// what is no rule of the policy never reach a reader.
|
||||
func TestOutputNamesNoInternalRule(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.setPolicy("release-1.1: 2099-01-01\n")
|
||||
gh := &fakeGH{labels: map[string][]int{labelInclude: {6}}}
|
||||
head := openBatch(t, f, gh)
|
||||
internal := regexp.MustCompile(`rule 1[0-9][0-9]`)
|
||||
for _, args := range [][]string{
|
||||
{"fold", "--line", "release-1.1", "--original", "5", "--commit", head},
|
||||
{"run", "--line", "release-1.1", "--defer", "5", "--reason", "x"},
|
||||
} {
|
||||
code, stdout, stderr := f.exec(gh, args...)
|
||||
f.git("switch", "-q", "main")
|
||||
if code == 0 || !strings.Contains(stdout+stderr, "#5 is not one the batch takes: rule 1, before the line's start, merged ") {
|
||||
t.Errorf("%s = %d, want a refusal naming rule 1:\n%s%s", args[0], code, stdout, stderr)
|
||||
}
|
||||
if m := internal.FindString(stdout + stderr); m != "" {
|
||||
t.Errorf("%s prints %q:\n%s%s", args[0], m, stdout, stderr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A refusal fold prints is one line with no control sequence, whatever the
|
||||
// reason it repeats says: a drop's reason comes from a pull request body, a
|
||||
// rule 4 reason from file names an original's author chose.
|
||||
func TestFoldRefusalPrintsNothingThatDrivesATerminal(t *testing.T) {
|
||||
l, _ := parseLine("release-1.1")
|
||||
for _, reason := range []string{"dropped on #90: x\n\x1b[2Jrefused: forged", "touches \"app/a\x1b]0;x\x07\nok\""} {
|
||||
b := &batch{line: l, verdicts: []*verdict{{o: &original{number: 5}, rule: ruleDropped, reason: reason}}}
|
||||
why, err := refuseFold(b, 5, strings.Repeat("a", 40))
|
||||
if err != nil || why == "" || strings.ContainsFunc(why, func(r rune) bool { return unicode.IsControl(r) || unicode.Is(unicode.Cf, r) }) {
|
||||
t.Errorf("refuseFold = %q, %v; want one clean line", why, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type errString string
|
||||
|
||||
func (e errString) Error() string { return string(e) }
|
||||
|
||||
// Nothing the tool prints may carry a control sequence, whichever command
|
||||
// prints it and wherever the text came from.
|
||||
func TestTerminal(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
const evil = "\x1b]0;owned\x07\x1b[2J\u009b31m\u202e"
|
||||
f.pr(5, "fix(app): five "+evil, "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+batchBranch1)
|
||||
gh := &fakeGH{prs: []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, "")}}
|
||||
|
||||
f.git("switch", "-q", "--detach", "origin/"+batchBranch1)
|
||||
f.write("app/"+evil+".txt", "x\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "port "+evil)
|
||||
commit := f.git("rev-parse", "HEAD")
|
||||
f.git("switch", "-q", "main")
|
||||
|
||||
// gh relays what the server said when it fails.
|
||||
broken := &fakeGH{err: errString("HTTP 502: " + evil)}
|
||||
for _, tc := range []struct {
|
||||
gh *fakeGH
|
||||
args []string
|
||||
}{
|
||||
{gh, []string{"check", "--line", "release-1.1"}},
|
||||
{broken, []string{"check", "--line", "release-1.1"}},
|
||||
{gh, []string{"fold", "--line", "release-1.1", "--original", "5", "--commit", commit}},
|
||||
{gh, []string{"run", "--line", "release-1.1", "--date", "2026-10-26"}},
|
||||
{gh, []string{"run", "--line", "release-1.1\x1b[2J"}},
|
||||
} {
|
||||
args := tc.args
|
||||
_, stdout, stderr := f.exec(tc.gh, args...)
|
||||
for name, out := range map[string]string{"stdout": stdout, "stderr": stderr} {
|
||||
for _, r := range out {
|
||||
if r != '\n' && r != '\t' && unicode.IsControl(r) || unicode.Is(unicode.Cf, r) {
|
||||
t.Errorf("%s of %v carries %U:\n%q", name, args[0], r, out)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// pickOnLine lands original n on release-1.1 as a maintainer's pick of its
|
||||
// merge commit, with the -x line git appends, so the line carries it.
|
||||
func pickOnLine(f *fixture, n int, files map[string]string) {
|
||||
f.t.Helper()
|
||||
mc, _, _ := strings.Cut(f.git("log", "--first-parent", "--format=%H", "--grep", fmt.Sprintf("(#%d)$", n), "main"), "\n")
|
||||
f.onLine("release-1.1", fmt.Sprintf("fix: original %d\n\n(cherry picked from commit %s)", n, mc), files)
|
||||
}
|
||||
|
||||
// A workflow original that waits on the open batch fails the cut, also when
|
||||
// it is all the batch has left: the pick beside it was labelled
|
||||
// backport/skip after the pick, or reached the line by hand.
|
||||
func TestCutFailsWhenOnlyAWorkflowWaitIsLeft(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
leave func(f *fixture, gh *fakeGH)
|
||||
}{
|
||||
{"skip after the pick", func(_ *fixture, gh *fakeGH) { gh.labels = map[string][]int{labelSkip: {6}} }},
|
||||
{"picked onto the line by hand", func(f *fixture, _ *fakeGH) {
|
||||
pickOnLine(f, 6, map[string]string{"app/six.txt": "6\n", ".github/renovate.json": "{}\n"})
|
||||
}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := workflowWorld(t)
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
tc.leave(f, gh)
|
||||
code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1", "--cut")
|
||||
if code != 1 || !strings.Contains(stdout, "#5 changes .github/workflows/ and waits on batch PR #90 for a maintainer") {
|
||||
t.Errorf("check --cut = %d, want 1 naming #5:\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
|
||||
}
|
||||
if strings.Contains(stdout, "on the line or on the batch") {
|
||||
t.Errorf("check --cut passes the line while #5 waits:\n%s", stdout)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A conflict the batch deferred is still deferred once that batch merged:
|
||||
// the next run picks nothing, opens nothing and writes nothing to the merged
|
||||
// pull request, and check reads the merged batch's record.
|
||||
func TestCheckAfterTheBatchMerged(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only five", map[string]string{"app/five.txt": "line\n"})
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
mergeBatch(f, gh)
|
||||
written := len(gh.created) + len(gh.edited) + len(gh.issueComments)
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 0 {
|
||||
t.Fatalf("run --continue --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if got := len(gh.created) + len(gh.edited) + len(gh.issueComments); got != written || f.remoteHead(batchBranch1) != "" {
|
||||
t.Errorf("the run picked nothing and still wrote: created %q, edited %q", gh.created, gh.edited)
|
||||
}
|
||||
for _, cut := range []bool{false, true} {
|
||||
args := []string{"check", "--line", "release-1.1"}
|
||||
if cut {
|
||||
args = append(args, "--cut")
|
||||
}
|
||||
code, stdout, stderr := f.exec(gh, args...)
|
||||
if code != 0 || !strings.Contains(stderr, "warning: deferred conflict on #90 (merged): https://github.com/cozystack/cozystack/pull/5 ") {
|
||||
t.Errorf("%q = %d, want 0 with #5 a deferred conflict:\nstdout:\n%s\nstderr:\n%s", args, code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A workflow original waits only while a batch is open to list it: once the
|
||||
// batch that listed it merged, it is missing.
|
||||
func TestWorkflowWaitAfterTheBatchMergedIsMissing(t *testing.T) {
|
||||
f := workflowWorld(t)
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
mergeBatch(f, gh)
|
||||
for _, cut := range []bool{false, true} {
|
||||
args := []string{"check", "--line", "release-1.1"}
|
||||
if cut {
|
||||
args = append(args, "--cut")
|
||||
}
|
||||
if code, stdout, stderr := f.exec(gh, args...); code != 1 || !strings.Contains(stdout, "/pull/5 ") {
|
||||
t.Errorf("%q = %d, want 1 with #5 missing:\nstdout:\n%s\nstderr:\n%s", args, code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,228 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// parseDefer reads --defer N[,N] and the --reason that has to come with it.
|
||||
func parseDefer(list, reason string) ([]int, string, error) {
|
||||
if list == "" {
|
||||
return nil, "", errors.New("--reason goes with --defer or --drop")
|
||||
}
|
||||
reason = capped(oneLine(clean(reason, false)), titleLimit)
|
||||
if reason == "" {
|
||||
return nil, "", errors.New("--defer needs a --reason, which the batch body records")
|
||||
}
|
||||
numbers, err := parseNumbers("--defer", list)
|
||||
return numbers, reason, err
|
||||
}
|
||||
|
||||
// parseNumbers reads the N[,N] list a flag takes, each once.
|
||||
func parseNumbers(flag, list string) ([]int, error) {
|
||||
var numbers []int
|
||||
for _, s := range strings.Split(list, ",") {
|
||||
n, err := strconv.Atoi(strings.TrimPrefix(strings.TrimSpace(s), "#"))
|
||||
if err != nil || n <= 0 {
|
||||
return nil, fmt.Errorf("%s %q is not a list of pull request numbers", flag, list)
|
||||
}
|
||||
if !slices.Contains(numbers, n) {
|
||||
numbers = append(numbers, n)
|
||||
}
|
||||
}
|
||||
return numbers, nil
|
||||
}
|
||||
|
||||
// rebuild takes the originals in numbers out of the open batch: it builds the
|
||||
// batch branch again off the line's head without their commits, and records
|
||||
// them in the body as deferred, for reason, next to what earlier deferrals
|
||||
// recorded. It adds no new original; the next run does.
|
||||
func (a *app) rebuild(b *batch, numbers []int, reason string, push bool) int {
|
||||
if b.pr == nil {
|
||||
return a.fail(fmt.Errorf("no batch pull request is open on %s; --defer rebuilds the open one", b.line.branch()))
|
||||
}
|
||||
hold := map[int]string{}
|
||||
for n, e := range b.recorded() {
|
||||
if e.Reason != "" {
|
||||
hold[n] = e.Reason
|
||||
}
|
||||
}
|
||||
carried, err := readPicks(b.prHead, b.lineHead, b.textUntil)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
stale, err := keptOut(b, carried)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
for _, n := range numbers {
|
||||
switch v := b.find(n); {
|
||||
case v == nil:
|
||||
return a.fail(fmt.Errorf("#%d did not merge into main after %s branched", n, b.line.branch()))
|
||||
case v.rule != 0 && !slices.Contains(stale, v):
|
||||
return a.fail(fmt.Errorf("#%d is not one the batch takes: %s, %s", n, ruleName(v.rule), terminalSafe(v.reason)))
|
||||
}
|
||||
hold[n] = reason
|
||||
}
|
||||
return a.reassemble(b, hold, hold, b.src.rec.dropped(), modeDefer, push)
|
||||
}
|
||||
|
||||
// reassemble builds the open batch again off the line's head without the
|
||||
// units of the originals in out, and reports it with hold deferred for their
|
||||
// reasons and drops kept.
|
||||
func (a *app) reassemble(b *batch, out, hold map[int]string, drops []droppedEntry, mode string, push bool) int {
|
||||
branch := b.pr.Head.Ref
|
||||
start, replay, err := rebuildPlan(b, out)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := a.warnReset(branch, start); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if _, err := git("switch", "--quiet", "--no-track", "-C", branch, "--end-of-options", start); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
for _, u := range replay {
|
||||
failed, files, err := replayUnit(u)
|
||||
switch {
|
||||
case err != nil:
|
||||
return a.fail(err)
|
||||
case failed != "":
|
||||
return a.fail(fmt.Errorf("without what is deferred, %.12s%s no longer applies (%s); defer it as well", failed, originName(b, u), terminalSafe(strings.Join(files, ", "))))
|
||||
}
|
||||
}
|
||||
head, err := resolve("HEAD")
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
picks, err := readPicks(head, b.lineHead, b.textUntil)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
return a.reclassify(b, picks, hold, drops, mode, push)
|
||||
}
|
||||
|
||||
// reclassify reports the open batch as picks shows it, after an operation
|
||||
// that adds no original: what the record held stays held, and an original
|
||||
// the batch neither carries nor records is missing until a run tries it.
|
||||
func (a *app) reclassify(b *batch, picks *history, hold map[int]string, drops []droppedEntry, mode string, push bool) int {
|
||||
date, err := branchDate(b.pr.Head.Ref)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
recorded := b.recorded()
|
||||
r := newReport(b)
|
||||
// None of these changes how the batch is assembled, so the record keeps
|
||||
// the flag the batch was assembled with.
|
||||
r.opts = runOptions{NoWorkflowFiles: b.src.savedNoWorkflowFiles(), Mode: mode}
|
||||
r.dropped = drops
|
||||
if r.stale, err = keptOut(b, picks); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
for _, v := range b.qualifying() {
|
||||
e := recorded[v.o.number]
|
||||
switch {
|
||||
case found(picks, v):
|
||||
r.onBranch = append(r.onBranch, v)
|
||||
case hold[v.o.number] != "":
|
||||
r.held = append(r.held, pick{v: v, status: "held", reason: hold[v.o.number]})
|
||||
case e.Workflows:
|
||||
r.waiting = append(r.waiting, pick{v: v, status: "waiting", files: e.Files, more: e.More})
|
||||
case len(e.Files) > 0:
|
||||
r.deferred = append(r.deferred, pick{v: v, status: "conflict", files: e.Files, more: e.More, failed: e.Commit})
|
||||
default:
|
||||
// Merged into main since the batch was assembled, or recorded
|
||||
// nowhere: a rebuild adds no original, the next run tries it.
|
||||
r.missing = append(r.missing, v)
|
||||
}
|
||||
}
|
||||
return a.finish(b, r, b.pr.Head.Ref, date, push, true)
|
||||
}
|
||||
|
||||
// recorded is what the open batch's record defers; a closed batch's record
|
||||
// holds nothing back on this one.
|
||||
func (b *batch) recorded() map[int]deferredEntry {
|
||||
if !b.src.open {
|
||||
return map[int]deferredEntry{}
|
||||
}
|
||||
return b.src.rec.deferred()
|
||||
}
|
||||
|
||||
// rebuildPlan is where the rebuild of the open batch without the originals in
|
||||
// drop starts, and the units of the batch it picks again on top, oldest first,
|
||||
// each in its own shape. While the batch still sits on the line's head,
|
||||
// everything before the first unit it drops stays as it is.
|
||||
func rebuildPlan(b *batch, drop map[int]string) (string, []unit, error) {
|
||||
units, err := batchUnits(b.prHead, b.lineHead)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
for _, u := range units {
|
||||
if len(u.parents) > 2 || u.merge() && len(batchedFrom(u.message)) != 1 {
|
||||
return "", nil, fmt.Errorf("batch pull request #%d carries merge commit %.12s, which is no batch merge of an original and which a rebuild cannot pick again; take the original out by hand", b.pr.Number, u.commit)
|
||||
}
|
||||
}
|
||||
dropped := func(u unit) bool {
|
||||
v := b.unitOriginal(u)
|
||||
return v != nil && drop[v.o.number] != ""
|
||||
}
|
||||
var keep []unit
|
||||
for _, u := range units {
|
||||
if !dropped(u) {
|
||||
keep = append(keep, u)
|
||||
}
|
||||
}
|
||||
first := slices.IndexFunc(units, dropped)
|
||||
onLineHead, err := isAncestor(b.lineHead, b.prHead)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
switch {
|
||||
case !onLineHead:
|
||||
return b.lineHead, keep, nil
|
||||
case first < 0:
|
||||
return b.prHead, nil, nil
|
||||
case first == 0:
|
||||
return b.lineHead, keep, nil
|
||||
}
|
||||
return units[first-1].commit, keep[first:], nil
|
||||
}
|
||||
|
||||
// pickedBy is the original whose commits the -x sources name, or nil.
|
||||
func (b *batch) pickedBy(sources []string) *verdict {
|
||||
for _, v := range b.verdicts {
|
||||
for _, s := range sources {
|
||||
if slices.Contains(v.commits, s) {
|
||||
return v
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// originName names the original a unit carries, for an error message.
|
||||
func originName(b *batch, u unit) string {
|
||||
if v := b.unitOriginal(u); v != nil {
|
||||
return fmt.Sprintf(", of #%d,", v.o.number)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,709 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// openBatch pushes a batch for the fixes merged so far and opens its pull
|
||||
// request on the fake, with the body the run printed.
|
||||
func openBatch(t *testing.T, f *fixture, gh *fakeGH) string {
|
||||
t.Helper()
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, stdout)}
|
||||
return f.remoteHead(batchBranch1)
|
||||
}
|
||||
|
||||
// reopen points the fake's batch pull request at what the remote now has and
|
||||
// at the body the tool last wrote.
|
||||
func reopen(f *fixture, gh *fakeGH) {
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, gh.bodies[len(gh.bodies)-1])}
|
||||
}
|
||||
|
||||
// --defer takes an original out of the open batch and nothing else: what came
|
||||
// before it keeps its commits, what came after is picked again in its own
|
||||
// shape, a resolution pushed by hand survives as the merge it was, and the
|
||||
// body records the reason.
|
||||
func TestDeferRebuildsWithoutTheOriginal(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only seven", map[string]string{"app/seven.txt": "line\n"})
|
||||
f.prCommits(5, "fix(app): five", "",
|
||||
change{files: map[string]string{"app/five.txt": "5\n"}},
|
||||
change{files: map[string]string{"app/five.txt": "5, done\n"}})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
mc7 := f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
l, _ := parseLine("release-1.1")
|
||||
if d, err := parseDeferred(gh.prs[0].Body, l); err != nil || len(d[7].Files) == 0 {
|
||||
t.Fatalf("#7 is not a deferred conflict: %+v, %v", d, err)
|
||||
}
|
||||
// A resolution of #7, the way fold accepts one: its commit picked onto the
|
||||
// batch head with the conflict resolved, under a merge that names it.
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.write("app/seven.txt", "7, resolved\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "fix(app): seven\n\n(cherry picked from commit "+f.git("rev-parse", mc7+"^2")+")")
|
||||
pick := f.git("rev-parse", "HEAD")
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.git("merge", "-q", "--no-ff", "-m", "fix(app): seven (#7)\n\n"+batchTrailer+": "+mc7, pick)
|
||||
resolution := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, gh.prs[0].Body)}
|
||||
if code, out, stderr := f.exec(gh, "fold", "--line", "release-1.1", "--original", "7", "--commit", resolution); code != 1 || !strings.Contains(out, "already carries #7") {
|
||||
t.Fatalf("fold of the pushed resolution = %d, %q, %s; want it seen as carried", code, out, stderr)
|
||||
}
|
||||
first := f.git("rev-parse", head+"~1")
|
||||
// A commit picked again gets a new committer date, so a new name.
|
||||
t.Setenv("GIT_COMMITTER_DATE", "2026-10-27T06:00:00Z")
|
||||
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "6", "--reason", "fails the app e2e", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --defer = %d\n%s", code, stderr)
|
||||
}
|
||||
pushed := f.remoteHead(batchBranch1)
|
||||
want := []string{"fix(app): five (#5)", "fix(app): seven (#7)"}
|
||||
if got := f.subjects("origin/release-1.1", pushed); !slices.Equal(got, want) {
|
||||
t.Errorf("the rebuilt batch carries %q, want %q", got, want)
|
||||
}
|
||||
if got := f.git("rev-parse", pushed+"~1"); got != first {
|
||||
t.Errorf("the merge of #5 became %.12s, want it kept as %.12s", got, first)
|
||||
}
|
||||
if parents := strings.Fields(f.git("rev-list", "--parents", "-n", "1", pushed)); len(parents) != 3 {
|
||||
t.Errorf("#7's resolution came back as %v, want a merge", parents)
|
||||
}
|
||||
if !strings.HasSuffix(f.git("log", "-1", "--format=%B", pushed), batchTrailer+": "+mc7) {
|
||||
t.Error("#7's merge lost its trailer")
|
||||
}
|
||||
if got := f.git("show", pushed+":app/seven.txt"); got != "7, resolved" {
|
||||
t.Errorf("the resolution was lost: app/seven.txt is %q", got)
|
||||
}
|
||||
if want := "`fix(app): seven`, 1 commit, changed in the pick: `" + f.git("rev-parse", pushed+"^2")[:12] + "`"; !strings.Contains(stdout, want) {
|
||||
t.Errorf("the body does not say the resolution differs from #7's commit (%s):\n%s", want, stdout)
|
||||
}
|
||||
if strings.Contains(stdout, "`fix(app): five`, 2 commits, changed") {
|
||||
t.Errorf("the body calls #5's clean picks changed:\n%s", stdout)
|
||||
}
|
||||
if code, out, stderr := f.exec(gh, "verify", "--line", "release-1.1", "--head", pushed); code != 0 {
|
||||
t.Errorf("verify of the rebuilt batch = %d\n%s\n%s", code, out, stderr)
|
||||
}
|
||||
deferred, err := parseDeferred(stdout, l)
|
||||
if err != nil || len(deferred) != 1 || deferred[6].Reason != "fails the app e2e" {
|
||||
t.Errorf("deferred = %+v, %v; want #6 for its reason", deferred, err)
|
||||
}
|
||||
if !strings.Contains(stdout, "### Deferred to the next batch") || !strings.Contains(stdout, "`fails the app e2e`") {
|
||||
t.Errorf("the body does not show what is deferred and why:\n%s", stdout)
|
||||
}
|
||||
if !slices.Equal(gh.edited, []string{"#90 fix(backport): batch for release-1.1, 2026-10-26"}) || len(gh.created) != 1 {
|
||||
t.Errorf("edited %q and created %q, want the open pull request edited", gh.edited, gh.created)
|
||||
}
|
||||
|
||||
// check warns about it instead of failing.
|
||||
reopen(f, gh)
|
||||
code, _, stderr = f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 0 || !strings.Contains(stderr, "deferred on #90 to the next batch: https://github.com/cozystack/cozystack/pull/6 fix(app): six (fails the app e2e)") {
|
||||
t.Errorf("check = %d, want 0 with a warning about #6:\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// The batch that held an original out keeps it out when a run continues it,
|
||||
// and the next batch picks it again.
|
||||
func TestDeferHoldsUntilTheNextBatch(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "#6", "--reason", "fails the app e2e", "--push"); code != 0 {
|
||||
t.Fatalf("run --defer = %d\n%s", code, stderr)
|
||||
}
|
||||
reopen(f, gh)
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
f.now = f.now.AddDate(0, 0, 3)
|
||||
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --continue = %d\n%s", code, stderr)
|
||||
}
|
||||
want := []string{"fix(app): five (#5)", "fix(app): eight (#8)"}
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, want) {
|
||||
t.Errorf("the continued batch carries %q, want %q", got, want)
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
if deferred, err := parseDeferred(stdout, l); err != nil || deferred[6].Reason != "fails the app e2e" {
|
||||
t.Errorf("the continued body defers %+v, %v; want #6 still held", deferred, err)
|
||||
}
|
||||
|
||||
// The batch merges; the next one starts from the line and tries #6 again.
|
||||
f.git("push", "-q", "origin", f.remoteHead(batchBranch1)+":refs/heads/release-1.1")
|
||||
gh.prs = nil
|
||||
code, stdout, stderr = f.exec(gh, "run", "--line", "release-1.1", "--continue")
|
||||
if code != 0 {
|
||||
t.Fatalf("the next run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
|
||||
t.Errorf("the next batch carries %q, want #6 tried again", got)
|
||||
}
|
||||
if strings.Contains(stdout, "Deferred to the next batch") {
|
||||
t.Errorf("the next batch still holds something out:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// A rebuild picks the rest of the batch again onto a line that moved since,
|
||||
// and refuses when dropping an original leaves a later commit that no longer
|
||||
// applies, without pushing anything.
|
||||
func TestDeferOnAMovedLine(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/x.txt": "five\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/x.txt": "five and six\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
moved := f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "x", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, ", of #6, no longer applies (app/x.txt); defer it as well") {
|
||||
t.Errorf("run --defer 5 = %d, want a refusal naming #6:\n%s", code, stderr)
|
||||
}
|
||||
if got := f.remoteHead(batchBranch1); got != head {
|
||||
t.Errorf("the refused rebuild moved the batch to %.12s", got)
|
||||
}
|
||||
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5,6", "--reason", "x", "--push"); code != 0 {
|
||||
t.Fatalf("run --defer 5,6 = %d\n%s", code, stderr)
|
||||
}
|
||||
pushed := f.remoteHead(batchBranch1)
|
||||
if got := f.git("rev-parse", pushed+"^"); got != moved {
|
||||
t.Errorf("the rebuild sits on %.12s, want the line's head %.12s", got, moved)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", pushed); !slices.Equal(got, []string{"fix(app): seven (#7)"}) {
|
||||
t.Errorf("the rebuilt batch carries %q, want #7", got)
|
||||
}
|
||||
}
|
||||
|
||||
// What --defer cannot do is refused before anything is touched.
|
||||
func TestDeferRefuses(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "feat(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "x"); code != 2 || !strings.Contains(stderr, "no batch pull request is open") {
|
||||
t.Errorf("--defer with no batch open = %d (%s), want a refusal", code, stderr)
|
||||
}
|
||||
head := openBatch(t, f, gh)
|
||||
for _, args := range [][]string{
|
||||
{"--defer", "5"},
|
||||
{"--defer", "5", "--reason", " \n "},
|
||||
{"--reason", "x"},
|
||||
{"--defer", "five", "--reason", "x"},
|
||||
{"--defer", "5,-1", "--reason", "x"},
|
||||
{"--defer", "5", "--reason", "x", "--onto", head},
|
||||
{"--defer", "5", "--reason", "x", "--continue"},
|
||||
{"--defer", "5", "--reason", "x", "--date", "2026-10-26"},
|
||||
{"--continue", "--onto", head},
|
||||
{"--defer", "6", "--reason", "x"},
|
||||
{"--defer", "4", "--reason", "x"},
|
||||
} {
|
||||
code, _, stderr := f.exec(gh, append([]string{"run", "--line", "release-1.1"}, args...)...)
|
||||
if code != 2 {
|
||||
t.Errorf("run %q = %d (%s), want 2", args, code, stderr)
|
||||
}
|
||||
}
|
||||
if got := f.remoteHead(batchBranch1); got != head {
|
||||
t.Errorf("a refused run moved the batch to %.12s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A rebuild replaces the head it started from and nothing else: one pushed
|
||||
// since is refused by the tool, and by the remote through the lease.
|
||||
func TestDeferLeasesTheHeadItRebuilt(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
l, _ := parseLine("release-1.1")
|
||||
lineHead := f.git("rev-parse", "origin/release-1.1")
|
||||
rebuilt := f.git("rev-parse", head+"~1")
|
||||
// Someone pushes on top of the batch after the rebuild read its head.
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.git("commit", "-q", "--allow-empty", "-m", "pushed since")
|
||||
since := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
|
||||
pr := f.batchPR(90, "release-1.1", batchBranch1, gh.prs[0].Body)
|
||||
a := &app{gh: gh}
|
||||
b := &batch{line: l, lineHead: lineHead, pr: &pr, prHead: head, src: recordSource{carrier: &pr, open: true, code: &pr}}
|
||||
err := a.publish(b, publication{branch: batchBranch1, head: rebuilt, title: "title", body: "body"}, true)
|
||||
if err == nil || !strings.Contains(err.Error(), "which the rebuild started from") {
|
||||
t.Errorf("publish over a head pushed since = %v, want a refusal", err)
|
||||
}
|
||||
if _, err := git(pushArgs(batchBranch1, rebuilt, head)...); err == nil {
|
||||
t.Error("the remote took a push whose lease names a head it no longer has")
|
||||
}
|
||||
if got := f.remoteHead(batchBranch1); got != since {
|
||||
t.Errorf("the batch moved to %.12s, want it left at %.12s", got, since)
|
||||
}
|
||||
// The lease that names the head the remote has is taken, over a
|
||||
// non-fast-forward; a plain push of the same is not.
|
||||
if _, err := git(pushArgs(batchBranch1, rebuilt, "")...); err == nil {
|
||||
t.Error("a push without a lease replaced the batch")
|
||||
}
|
||||
if _, err := git(pushArgs(batchBranch1, rebuilt, since)...); err != nil {
|
||||
t.Errorf("a push leased on the remote's head = %v", err)
|
||||
}
|
||||
if got := f.remoteHead(batchBranch1); got != rebuilt {
|
||||
t.Errorf("the batch is at %.12s, want %.12s", got, rebuilt)
|
||||
}
|
||||
}
|
||||
|
||||
// A plain run never replaces the batch, deferred or not: it stays a
|
||||
// fast-forward.
|
||||
func TestContinueIsAFastForward(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 0 {
|
||||
t.Fatalf("run --continue = %d\n%s", code, stderr)
|
||||
}
|
||||
if pushed := f.remoteHead(batchBranch1); !descends(t, head, pushed) || pushed == head {
|
||||
t.Errorf("the batch moved from %.12s to %.12s, want a fast-forward", head, pushed)
|
||||
}
|
||||
if !slices.Equal(gh.edited, []string{"#90 fix(backport): batch for release-1.1, 2026-10-26"}) {
|
||||
t.Errorf("edited %q, want #90", gh.edited)
|
||||
}
|
||||
}
|
||||
|
||||
// A batch that carries a merge commit is not rebuilt: picking it again would
|
||||
// lose whatever the merge resolved.
|
||||
func TestDeferRefusesAMerge(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.git("merge", "-q", "--no-ff", "-m", "Merge release-1.1 into the batch", "origin/release-1.1")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
gh.prs[0].Body = gh.bodies[0]
|
||||
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "x", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "carries merge commit") {
|
||||
t.Errorf("run --defer over a merge = %d, want a refusal:\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A batch pull request whose body backport-batch did not write is never
|
||||
// rewritten, nor its branch pushed: it is a maintainer's.
|
||||
func TestPublishLeavesAHandWrittenBatch(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
gh.prs[0].Body = "Picked by hand: #5."
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
for _, args := range [][]string{
|
||||
{"--continue", "--push"},
|
||||
{"--defer", "5", "--reason", "x", "--push"},
|
||||
} {
|
||||
code, _, stderr := f.exec(gh, append([]string{"run", "--line", "release-1.1"}, args...)...)
|
||||
if code != 2 || !strings.Contains(stderr, "backport-batch did not write it") {
|
||||
t.Errorf("run %q on a hand-written batch = %d, want a refusal:\n%s", args, code, stderr)
|
||||
}
|
||||
}
|
||||
if got := f.remoteHead(batchBranch1); got != head || len(gh.edited) != 0 {
|
||||
t.Errorf("the batch moved to %.12s and %q were edited, want it left alone", got, gh.edited)
|
||||
}
|
||||
}
|
||||
|
||||
// backport/skip added after the batch picked an original keeps the commit
|
||||
// on the branch until --defer takes it out; the body says so meanwhile.
|
||||
func TestSkipAfterThePick(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
gh.labels = map[string][]int{labelSkip: {6}}
|
||||
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue")
|
||||
if code != 0 || !strings.Contains(stdout, "`backport/skip`, still on this branch") {
|
||||
t.Errorf("run --continue = %d, want the body to say #6 is still on the branch:\n%s\n%s", code, stdout, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "6", "--reason", "labelled backport/skip", "--push"); code != 0 {
|
||||
t.Fatalf("run --defer of a skipped original = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): five (#5)"}) {
|
||||
t.Errorf("the batch carries %q, want #6 taken out", got)
|
||||
}
|
||||
reopen(f, gh)
|
||||
if code, _, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 0 {
|
||||
t.Errorf("check = %d\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// closeWorld is a batch carrying #5 alone, opened and pushed.
|
||||
func closeWorld(t *testing.T) (*fixture, *fakeGH, string) {
|
||||
t.Helper()
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
h1 := openBatch(t, f, gh)
|
||||
return f, gh, h1
|
||||
}
|
||||
|
||||
// Deferring everything a batch carries closes it: the record says why and
|
||||
// what it froze while the pull request is still open, then the pull request
|
||||
// is closed and its branch deleted, leased on that head. The next batch
|
||||
// tries what was deferred again.
|
||||
func TestDeferEverythingClosesTheBatch(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
var log []string
|
||||
gh.after = map[string]func(){
|
||||
"createComment": func() { log = append(log, "comment") },
|
||||
"editPull": func() { log = append(log, "edit") },
|
||||
"closePull": func() { log = append(log, "close, branch at "+f.remoteHead(batchBranch1)) },
|
||||
}
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "fails the app e2e", "--push")
|
||||
if code != 0 || !strings.Contains(stderr, "closed https://github.com/cozystack/cozystack/pull/90: without what is deferred it carries nothing beyond release-1.1; the next batch tries them again") {
|
||||
t.Fatalf("run --defer of the last pick = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if want := []string{"comment", "edit", "close, branch at " + h1}; !slices.Equal(log, want) || f.remoteHead(batchBranch1) != "" || gh.find(90).State != "closed" {
|
||||
t.Errorf("writes %q, branch at %q, #90 %s; want %q, the branch deleted and #90 closed", log, f.remoteHead(batchBranch1), gh.find(90).State, want)
|
||||
}
|
||||
rec, err := parseRecord(gh.find(90).Body, notifyLine)
|
||||
if err != nil || rec.Closing == nil || *rec.Closing != (closing{PR: 90, Branch: batchBranch1, Head: h1}) || rec.Built.Options.Mode != modeClose || rec.deferred()[5].Reason != "fails the app e2e" {
|
||||
t.Errorf("the closed batch's record is %+v, %v; want #5 deferred and the close it froze", rec, err)
|
||||
}
|
||||
if !strings.Contains(stdout, `"closing":{"pr":90,"branch":"`+batchBranch1+`","head":"`+h1+`"}`) {
|
||||
t.Errorf("the body printed does not say what the close froze:\n%s", stdout)
|
||||
}
|
||||
code, _, stderr = f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 0 || !strings.Contains(stderr, "warning: deferred on #90 (closed) to the next batch: https://github.com/cozystack/cozystack/pull/5 ") {
|
||||
t.Errorf("check = %d, want 0 with #5 deferred:\n%s", code, stderr)
|
||||
}
|
||||
gh.nextPR = 91
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): five (#5)"}) || gh.find(91) == nil {
|
||||
t.Errorf("the next batch carries %q, want #5 tried again on a new pull request", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A close that stopped part way is finished by run --push, and by nothing
|
||||
// else, and never past a head it did not freeze; nothing is ever written to
|
||||
// the closed pull request.
|
||||
func TestCloseRecovery(t *testing.T) {
|
||||
deferAll := []string{"run", "--line", "release-1.1", "--defer", "5", "--reason", "fails the app e2e", "--push"}
|
||||
push := []string{"run", "--line", "release-1.1", "--continue", "--push"}
|
||||
// moveBranch pushes a commit onto the batch branch, or recreates it, as
|
||||
// someone else would.
|
||||
moveBranch := func(f *fixture, base string) string {
|
||||
f.git("switch", "-q", "--detach", base)
|
||||
f.git("commit", "-q", "--allow-empty", "-m", "pushed since")
|
||||
h2 := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
return h2
|
||||
}
|
||||
writes := func(gh *fakeGH) int { return len(gh.issueComments) + len(gh.edited) }
|
||||
|
||||
t.Run("stopped after the record, then the branch moved", func(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
gh.after = map[string]func(){"editPull": func() { gh.fail = map[string]error{"pull": errString("HTTP 502")} }}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 || !strings.Contains(stderr, "HTTP 502") {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
h2 := moveBranch(f, h1)
|
||||
before := writes(gh)
|
||||
if code, _, stderr := f.exec(gh, push...); code != 2 || !strings.Contains(stderr, h2[:12]) {
|
||||
t.Errorf("run --push over a moved branch = %d, want 2 naming it:\n%s", code, stderr)
|
||||
}
|
||||
if gh.find(90).State != "open" || f.remoteHead(batchBranch1) != h2 || writes(gh) != before {
|
||||
t.Error("the run acted on a branch that moved since the close froze it")
|
||||
}
|
||||
})
|
||||
// GitHub and the remote are read apart, and either may be the first to
|
||||
// show that someone pushed while the record was written.
|
||||
for _, seen := range []string{"by the remote", "by GitHub"} {
|
||||
t.Run("the branch moved while the record was written, seen "+seen, func(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
gh.after = map[string]func(){"editPull": func() {
|
||||
gh.remoteHead = func(string) string { return "" }
|
||||
if seen == "by GitHub" {
|
||||
gh.find(90).Head.SHA = strings.Repeat("b", 40)
|
||||
} else {
|
||||
moveBranch(f, h1)
|
||||
}
|
||||
}}
|
||||
code, _, stderr := f.exec(gh, deferAll...)
|
||||
f.git("switch", "-q", "main")
|
||||
if code != 2 || !strings.Contains(stderr, "which its record froze") || gh.find(90).State != "open" || f.remoteHead(batchBranch1) == "" {
|
||||
t.Errorf("run = %d, #90 %s, branch %q; want 2, #90 open and the branch kept:\n%s", code, gh.find(90).State, f.remoteHead(batchBranch1), stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Run("the close failed", func(t *testing.T) {
|
||||
f, gh, _ := closeWorld(t)
|
||||
gh.fail = map[string]error{"closePull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
before := writes(gh)
|
||||
// Anything but run --push leaves it as it is.
|
||||
for _, args := range [][]string{
|
||||
{"run", "--line", "release-1.1", "--continue"},
|
||||
{"check", "--line", "release-1.1"},
|
||||
{"fold", "--line", "release-1.1", "--original", "5", "--commit", strings.Repeat("a", 40)},
|
||||
{"notify", "--line", "release-1.1", "--push"},
|
||||
} {
|
||||
if code, _, stderr := f.exec(gh, args...); code != 2 || !strings.Contains(stderr, "being closed") {
|
||||
t.Errorf("%q = %d, want 2:\n%s", args, code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
}
|
||||
if writes(gh) != before || gh.find(90).State != "open" {
|
||||
t.Error("a command other than run --push wrote to the batch")
|
||||
}
|
||||
code, _, stderr := f.exec(gh, push...)
|
||||
if code != 0 || gh.find(90).State != "closed" || f.remoteHead(batchBranch1) != "" || writes(gh) != before {
|
||||
t.Errorf("run --push = %d, #90 %s, branch %q, %d writes; want the close finished and nothing else\n%s", code, gh.find(90).State, f.remoteHead(batchBranch1), writes(gh)-before, stderr)
|
||||
}
|
||||
})
|
||||
// A run that finishes a close stops there: the change it was asked for
|
||||
// needs an open batch pull request, and none is left to make it on.
|
||||
t.Run("a change asked for while the close was left", func(t *testing.T) {
|
||||
f, gh, _ := closeWorld(t)
|
||||
gh.fail = map[string]error{"closePull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
code, _, stderr := f.exec(gh, deferAll...)
|
||||
f.git("switch", "-q", "main")
|
||||
if code != 2 || !strings.Contains(stderr, "finished the close of #90 and made no other change") || gh.find(90).State != "closed" || f.remoteHead(batchBranch1) != "" {
|
||||
t.Errorf("run --defer = %d, #90 %s, branch %q; want 2, the close finished and nothing deferred:\n%s", code, gh.find(90).State, f.remoteHead(batchBranch1), stderr)
|
||||
}
|
||||
})
|
||||
t.Run("closed by hand while the close writes", func(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
gh.after = map[string]func(){"createComment": func() { gh.find(90).State = "closed" }}
|
||||
code, _, stderr := f.exec(gh, deferAll...)
|
||||
f.git("switch", "-q", "main")
|
||||
if code != 2 || !strings.Contains(stderr, "batch pull request #90 is closed now; nothing is written to it") || len(gh.edited) != 0 || f.remoteHead(batchBranch1) != h1 {
|
||||
t.Errorf("run = %d, edited %q, branch %q; want 2, no edit and the branch kept:\n%s", code, gh.edited, f.remoteHead(batchBranch1), stderr)
|
||||
}
|
||||
})
|
||||
t.Run("the delete failed", func(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
gh.after = map[string]func(){"closePull": func() { f.refusePushes(true) }}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
f.refusePushes(false)
|
||||
code, _, stderr := f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 0 || !strings.Contains(stderr, "warning: batch pull request #90 is closed, but its branch "+batchBranch1+" is still at "+h1[:12]) {
|
||||
t.Errorf("check = %d, want 0 naming the branch:\n%s", code, stderr)
|
||||
}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue"); code != 2 {
|
||||
t.Errorf("run without --push = %d, want 2:\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if code, _, stderr := f.exec(gh, push...); code != 0 || f.remoteHead(batchBranch1) != "" {
|
||||
t.Errorf("run --push = %d, branch %q; want it deleted\n%s", code, f.remoteHead(batchBranch1), stderr)
|
||||
}
|
||||
mustRun(t, f, gh, "check", "--line", "release-1.1")
|
||||
})
|
||||
// A batch pull request a person opens on the branch a close left owns
|
||||
// it: neither recovery nor check offers to delete it, moved or not. One
|
||||
// on another branch does not stop the delete.
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
branch string
|
||||
moved bool
|
||||
}{
|
||||
{"a batch opened on the branch the close left", batchBranch1, false},
|
||||
{"a batch opened on the branch the close left, moved since", batchBranch1, true},
|
||||
{"a batch opened on another branch", "backport-batch/release-1.1-20261027", false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
gh.after = map[string]func(){"closePull": func() { f.refusePushes(true) }}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
f.refusePushes(false)
|
||||
gh.after = nil
|
||||
if tc.branch != batchBranch1 {
|
||||
f.git("push", "-q", "origin", h1+":refs/heads/"+tc.branch)
|
||||
}
|
||||
left := h1
|
||||
if tc.moved {
|
||||
left = moveBranch(f, h1)
|
||||
}
|
||||
gh.prs = append(gh.prs, f.batchPR(91, "release-1.1", tc.branch, "a maintainer's batch"))
|
||||
taken := tc.branch == batchBranch1
|
||||
_, _, stderr := f.exec(gh, "check", "--line", "release-1.1")
|
||||
f.git("switch", "-q", "main")
|
||||
if named := strings.Contains(stderr, "batch pull request #91 is open on "+batchBranch1); named != taken || taken == strings.Contains(stderr, "--push deletes it") || taken && strings.Contains(stderr, "git push origin --delete") {
|
||||
t.Errorf("check names the branch as #91's: %v, want %v, and offers no delete of it:\n%s", named, taken, stderr)
|
||||
}
|
||||
code, _, stderr := f.exec(gh, push...)
|
||||
f.git("switch", "-q", "main")
|
||||
if taken && (code != 2 || !strings.Contains(stderr, "batch pull request #91 is open on "+batchBranch1) || strings.Contains(stderr, "git push origin --delete") || f.remoteHead(batchBranch1) != left || gh.find(91).State != "open") {
|
||||
t.Errorf("run --push = %d, branch %q; want 2, no delete offered and #91's branch kept:\n%s", code, f.remoteHead(batchBranch1), stderr)
|
||||
}
|
||||
if !taken && (code != 0 || f.remoteHead(batchBranch1) != "") {
|
||||
t.Errorf("run --push = %d, branch %q; want the left branch deleted:\n%s", code, f.remoteHead(batchBranch1), stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Run("the branch moved after the close", func(t *testing.T) {
|
||||
f, gh, h1 := closeWorld(t)
|
||||
var h2 string
|
||||
gh.after = map[string]func(){"closePull": func() { h2 = moveBranch(f, h1) }}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 || !strings.Contains(stderr, "is at "+h2[:12]+" now") {
|
||||
t.Fatalf("run = %d, want 2 naming where the branch moved\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
// What was pushed onto the branch may be work the next run cannot
|
||||
// pick again, so the refusal promises nothing about deleting it.
|
||||
if code, _, stderr := f.exec(gh, push...); code != 2 || !strings.Contains(stderr, h1[:12]) || !strings.Contains(stderr, h2[:12]) || !strings.Contains(stderr, "#90") || strings.Contains(stderr, "nothing on it is lost") || !strings.Contains(stderr, "check what "+h2[:12]+" carries") {
|
||||
t.Errorf("run --push = %d, want 2 naming both heads and #90, and what to check before deleting:\n%s", code, stderr)
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != h2 {
|
||||
t.Error("the run moved a branch it did not freeze")
|
||||
}
|
||||
if _, err := git(deleteArgs(batchBranch1, h1)...); err == nil || f.remoteHead(batchBranch1) != h2 {
|
||||
t.Error("the remote took a delete whose lease names a head it no longer has")
|
||||
}
|
||||
})
|
||||
t.Run("a batch of the same day that opened no pull request", func(t *testing.T) {
|
||||
f, gh, _ := closeWorld(t)
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 0 || f.remoteHead(batchBranch1) != "" {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
// The same day, a new batch reuses the branch name, pushes, and
|
||||
// fails to open its pull request.
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh.fail = map[string]error{"createPull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, push...); code != 2 {
|
||||
t.Fatalf("the same-day run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
h2 := f.remoteHead(batchBranch1)
|
||||
for _, args := range [][]string{push, {"run", "--line", "release-1.1", "--onto", h2, "--push"}} {
|
||||
if code, _, stderr := f.exec(gh, args...); code != 2 || !strings.Contains(stderr, "git push origin --delete "+batchBranch1) {
|
||||
t.Errorf("%q = %d, want 2 naming the branch to delete:\n%s", args, code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != h2 || len(gh.created) != 1 {
|
||||
t.Errorf("the branch moved to %.12s or a pull request was opened: %q", f.remoteHead(batchBranch1), gh.created)
|
||||
}
|
||||
f.git("push", "-q", "origin", "--delete", batchBranch1)
|
||||
gh.nextPR = 91
|
||||
mustRun(t, f, gh, push...)
|
||||
if gh.find(91) == nil {
|
||||
t.Error("after the branch was deleted the next run opened no batch")
|
||||
}
|
||||
})
|
||||
t.Run("a preview", func(t *testing.T) {
|
||||
f, gh, _ := closeWorld(t)
|
||||
gh.fail = map[string]error{"closePull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
before := writes(gh)
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "x")
|
||||
if code != 2 || !strings.Contains(stderr, "run --line release-1.1 --push finishes") || writes(gh) != before || gh.find(90).State != "open" {
|
||||
t.Errorf("a preview = %d, want 2 naming the step left and nothing written:\n%s", code, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// A batch that merged on GitHub and left its branch behind is named when a
|
||||
// run of the same day refuses to push over that branch.
|
||||
func TestRefusalNamesTheMergedBatchsBranch(t *testing.T) {
|
||||
f, gh, _ := closeWorld(t)
|
||||
head := f.remoteHead(batchBranch1)
|
||||
mergeBatch(f, gh)
|
||||
// A commit pushed onto the batch after it merged is left on its branch.
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.git("commit", "-q", "--allow-empty", "-m", "pushed after the merge")
|
||||
pushed := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "it is the branch of batch pull request #90, which is merged") || strings.Contains(stderr, "--onto") || !strings.Contains(stderr, "check what "+pushed[:12]+" carries") {
|
||||
t.Errorf("run --push = %d, want a refusal naming #90:\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A rebuild that stops names the files that no longer apply on the line of
|
||||
// its refusal, whatever the files are called.
|
||||
func TestDeferRefusalNamesFilesOnOneLine(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
name := "app/x\nwarning: forged.txt"
|
||||
f.pr(5, "fix(app): five", "", map[string]string{name: "five\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{name: "five and six\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "x", "--push")
|
||||
f.git("switch", "-q", "main")
|
||||
if code != 2 || !strings.Contains(stderr, "no longer applies") {
|
||||
t.Fatalf("run --defer 5 = %d, want the refusal:\n%s", code, stderr)
|
||||
}
|
||||
for _, ln := range strings.Split(stderr, "\n") {
|
||||
if strings.HasPrefix(ln, "warning: forged") {
|
||||
t.Errorf("the refusal split onto a line of its own: %q", ln)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,269 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A drop is a maintainer's decision that an original stays off one line,
|
||||
// for a reason, where the rules would take it. It is kept in the line's
|
||||
// record and carried from batch to batch, bound to the original's merge
|
||||
// commit on main, so a record cannot point it at another original. It is
|
||||
// written only to the open batch's record, like everything else there.
|
||||
|
||||
// onLine says whether the line carries v, whatever rule decided it.
|
||||
func (b *batch) onLine(v *verdict) (bool, error) {
|
||||
if v.rule == ruleOnLine {
|
||||
return true, nil
|
||||
}
|
||||
if err := b.load(v); err != nil {
|
||||
return false, err
|
||||
}
|
||||
how, err := b.lineHist.carries(v.o, v.commits)
|
||||
return how != "", err
|
||||
}
|
||||
|
||||
// carried says whether the open batch has a unit of v, by the mapping the
|
||||
// rebuild takes units off with: found needs every commit of a merged
|
||||
// original, and one whose pick came out empty is missing from it.
|
||||
func (b *batch) carried(v *verdict) (bool, error) {
|
||||
units, err := batchUnits(b.prHead, b.lineHead)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
for _, u := range units {
|
||||
if o := b.unitOriginal(u); o != nil && o.o.number == v.o.number {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// redrop drops the originals in numbers from the line, for reason, or with
|
||||
// undrop withdraws their drops. What the open batch carries of a dropped
|
||||
// original comes off it, as --defer takes it off; otherwise only the record
|
||||
// changes, and the batch stays as it is even when the line moved since.
|
||||
func (a *app) redrop(b *batch, numbers []int, reason string, undrop bool, push bool) int {
|
||||
if !b.src.open {
|
||||
return a.fail(fmt.Errorf("no open batch pull request on %s carries a record; a drop is recorded on one", b.line.branch()))
|
||||
}
|
||||
drops := map[int]droppedEntry{}
|
||||
for _, d := range b.src.rec.Dropped {
|
||||
drops[d.Original] = d
|
||||
}
|
||||
mode, today := modeDrop, a.now().UTC().Format(time.DateOnly)
|
||||
if undrop {
|
||||
mode = modeUndrop
|
||||
}
|
||||
for _, n := range numbers {
|
||||
v := b.find(n)
|
||||
_, isDropped := drops[n]
|
||||
switch {
|
||||
case undrop && !isDropped:
|
||||
return a.fail(fmt.Errorf("#%d is not dropped from %s", n, b.line.branch()))
|
||||
case undrop:
|
||||
delete(drops, n)
|
||||
case v == nil:
|
||||
return a.fail(fmt.Errorf("#%d did not merge into main after %s branched", n, b.line.branch()))
|
||||
default:
|
||||
// backport/skip and rules 1 to 4 decide before the line is
|
||||
// looked at, so whether it carries the original is asked here.
|
||||
on, err := b.onLine(v)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if on {
|
||||
return a.fail(fmt.Errorf("#%d is already on %s; a drop cannot remove code the line has", n, b.line.branch()))
|
||||
}
|
||||
drops[n] = droppedEntry{Original: n, Merge: v.o.merge, Reason: reason, On: b.pr.Number, Date: today}
|
||||
}
|
||||
}
|
||||
active := map[int]droppedEntry{}
|
||||
for n, d := range drops {
|
||||
if b.find(n) != nil {
|
||||
active[n] = d
|
||||
}
|
||||
}
|
||||
b.drops = active
|
||||
out := map[int]string{}
|
||||
for _, n := range numbers {
|
||||
v := b.find(n)
|
||||
if v == nil {
|
||||
continue
|
||||
}
|
||||
if err := b.judge(v, active); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := b.load(v); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if undrop {
|
||||
continue
|
||||
}
|
||||
on, err := b.carried(v)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if on {
|
||||
out[n] = "dropped"
|
||||
}
|
||||
}
|
||||
hold := map[int]string{}
|
||||
for n, e := range b.recorded() {
|
||||
if e.Reason != "" {
|
||||
hold[n] = e.Reason
|
||||
}
|
||||
}
|
||||
list := slices.SortedFunc(maps.Values(drops), func(x, y droppedEntry) int { return cmp.Compare(x.Original, y.Original) })
|
||||
if len(out) > 0 {
|
||||
maps.Copy(out, hold)
|
||||
return a.reassemble(b, out, hold, list, mode, push)
|
||||
}
|
||||
if err := a.warnReset(b.pr.Head.Ref, b.prHead); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if _, err := git("switch", "--quiet", "--no-track", "-C", b.pr.Head.Ref, "--end-of-options", b.prHead); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
return a.reclassify(b, b.picks, hold, list, mode, push)
|
||||
}
|
||||
|
||||
// bindDrops holds each drop the record keeps to the original it names: a
|
||||
// drop of an original merged since the line branched has to name that
|
||||
// original's merge commit, and one of an earlier original a merge of it on
|
||||
// main's first-parent line. The first apply; the others are kept and carried
|
||||
// on, and decide nothing.
|
||||
func (b *batch) bindDrops(merges []*original) (map[int]droppedEntry, error) {
|
||||
return bindEntries(b.src.rec.dropped(), merges, b.mainHead, func(d droppedEntry) string {
|
||||
return fmt.Sprintf("batch pull request #%d drops #%d by %.12s", b.src.carrier.Number, d.Original, d.Merge)
|
||||
})
|
||||
}
|
||||
|
||||
// bindEntries binds drops to main: one whose original is among merges, the
|
||||
// originals main merged since the line branched, has to name that
|
||||
// original's merge exactly; any other has to be a merge of its number on
|
||||
// main's first-parent line up to mainHead, as an inherited drop is. It
|
||||
// returns the drops whose originals are among merges; about says what an
|
||||
// error is about.
|
||||
func bindEntries(drops []droppedEntry, merges []*original, mainHead string, about func(droppedEntry) string) (map[int]droppedEntry, error) {
|
||||
byMerge, byNumber := map[string]*original{}, map[int]*original{}
|
||||
for _, o := range merges {
|
||||
byMerge[o.merge], byNumber[o.number] = o, o
|
||||
}
|
||||
active := map[int]droppedEntry{}
|
||||
for _, d := range drops {
|
||||
at := about(d)
|
||||
if o := byMerge[d.Merge]; o != nil {
|
||||
if o.number != d.Original {
|
||||
return nil, fmt.Errorf("%s, which merged #%d", at, o.number)
|
||||
}
|
||||
active[d.Original] = d
|
||||
continue
|
||||
}
|
||||
if o := byNumber[d.Original]; o != nil {
|
||||
return nil, fmt.Errorf("%s, but #%d merged as %.12s", at, d.Original, o.merge)
|
||||
}
|
||||
ok, err := mergedOnMain(d.Merge, mainHead, d.Original)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s, which is no merge of it on main", at)
|
||||
}
|
||||
}
|
||||
return active, nil
|
||||
}
|
||||
|
||||
// bindDrops binds each drop a report keeps to main exactly as the record's
|
||||
// drops are bound, over the window the report classified: main and the
|
||||
// line at the heads it was made from. Its errors read after "report <id>".
|
||||
func (d *reportDoc) bindDrops() error {
|
||||
if len(d.Overlays.Dropped) == 0 {
|
||||
return nil
|
||||
}
|
||||
for _, h := range []struct{ name, head string }{{"main", d.Built.Main}, {d.Line, d.Built.Line}} {
|
||||
have, err := lookup(h.head)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if have == "" {
|
||||
return fmt.Errorf("was made on %s at %.12s, which this clone does not have; fetch %s and run again", h.name, h.head, h.name)
|
||||
}
|
||||
}
|
||||
out, err := git("merge-base", "--end-of-options", d.Built.Main, d.Built.Line)
|
||||
if err != nil {
|
||||
return fmt.Errorf("was made from a line that shares no history with main: %w", err)
|
||||
}
|
||||
merges, _, err := mainMerges(strings.TrimSpace(out), d.Built.Main)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
active, err := bindEntries(d.Overlays.Dropped, merges, d.Built.Main, func(e droppedEntry) string {
|
||||
return fmt.Sprintf("drops #%d by %.12s", e.Original, e.Merge)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// A drop decides only for an original the report classified, and only
|
||||
// once every earlier rule has passed it by; an inherited one decides
|
||||
// nothing.
|
||||
where, err := d.categories()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, n := range d.Primary.Skipped.Dropped {
|
||||
if _, ok := active[n]; !ok {
|
||||
return fmt.Errorf("has #%d dropped, but no drop of it binds to an original it classified", n)
|
||||
}
|
||||
}
|
||||
for n := range active {
|
||||
switch c := where[n]; c {
|
||||
case "skipped.dropped", "skipped.skip-label", "skipped.before-start", "skipped.type", "skipped.breaking", "skipped.touches", "skipped.on-line":
|
||||
default:
|
||||
return fmt.Errorf("drops #%d, which it has %s, though a drop decides before that", n, or(c, "nowhere"))
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// mergedOnMain says whether merge is on main's first-parent line up to
|
||||
// mainHead and its message names pull request n, as mainMerges reads one.
|
||||
func mergedOnMain(merge, mainHead string, n int) (bool, error) {
|
||||
have, err := lookup(merge)
|
||||
if err != nil || have == "" {
|
||||
return false, err
|
||||
}
|
||||
if ok, err := isAncestor(merge, mainHead); err != nil || !ok {
|
||||
return false, err
|
||||
}
|
||||
line, err := revList("--first-parent", "--end-of-options", mainHead)
|
||||
if err != nil || !slices.Contains(line, merge) {
|
||||
return false, err
|
||||
}
|
||||
msgs, err := commitMessages([]string{merge})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
number, _, _ := mergedPR(msgs[merge].message)
|
||||
return number == n, nil
|
||||
}
|
||||
@@ -0,0 +1,576 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// recordOf is the record the open batch #90 carries now.
|
||||
func recordOf(t *testing.T, gh *fakeGH) *lineRecord {
|
||||
t.Helper()
|
||||
rec, err := parseRecord(gh.find(90).Body, notifyLine)
|
||||
if err != nil || rec == nil {
|
||||
t.Fatalf("#90 carries no readable record: %v", err)
|
||||
}
|
||||
return rec
|
||||
}
|
||||
|
||||
func dropped(rec *lineRecord) []int {
|
||||
var out []int
|
||||
for _, d := range rec.Dropped {
|
||||
out = append(out, d.Original)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// A drop takes a picked original off the batch and keeps it off the line:
|
||||
// a run that continues the batch does not pick it, and nor does the next
|
||||
// batch, which carries the drop over from the merged one.
|
||||
func TestDropTakesAnOriginalOffTheLine(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
mc6 := f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
stdout := mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", "6", "--reason", "needs the rework of #4", "--push")
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): five (#5)"}) {
|
||||
t.Errorf("after the drop the batch carries %q, want #5 alone", got)
|
||||
}
|
||||
rec := recordOf(t, gh)
|
||||
if want := (droppedEntry{Original: 6, Merge: mc6, Reason: "needs the rework of #4", On: 90, Date: "2026-10-26"}); len(rec.Dropped) != 1 || rec.Dropped[0] != want || rec.Built.Options.Mode != modeDrop {
|
||||
t.Errorf("the record drops %+v (%s), want %+v", rec.Dropped, rec.Built.Options.Mode, want)
|
||||
}
|
||||
if s := bodyReportOf(t, stdout); !slices.Equal(s.doc.Primary.Skipped.Dropped, []int{6}) || !slices.Equal(droppedNumbers(s.doc.Overlays.Dropped), []int{6}) {
|
||||
t.Errorf("the report has %v dropped and %v as the decision", s.doc.Primary.Skipped.Dropped, s.doc.Overlays.Dropped)
|
||||
}
|
||||
if !strings.Contains(stdout, "<details><summary>Dropped from `release-1.1` by a maintainer</summary>") {
|
||||
t.Errorf("the body does not list #6 as dropped:\n%s", stdout)
|
||||
}
|
||||
code, _, stderr := f.exec(gh, "check", "--line", "release-1.1", "--cut")
|
||||
if code != 1 || !strings.Contains(stderr, "note: #6 is dropped from release-1.1 on #90 (needs the rework of #4): https://github.com/cozystack/cozystack/pull/6 ") {
|
||||
t.Errorf("check --cut = %d, want #5 to fail it and a note on #6:\n%s", code, stderr)
|
||||
}
|
||||
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): five (#5)", "fix(app): seven (#7)"}) || !slices.Equal(dropped(recordOf(t, gh)), []int{6}) {
|
||||
t.Errorf("the continued batch carries %q and drops %v", got, dropped(recordOf(t, gh)))
|
||||
}
|
||||
|
||||
mergeBatch(f, gh)
|
||||
f.now = f.now.AddDate(0, 0, 3)
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
gh.nextPR = 91
|
||||
stdout = mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
rec, err := parseRecord(stdout, notifyLine)
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead("backport-batch/release-1.1-20261029")); err != nil || !slices.Equal(got, []string{"fix(app): eight (#8)"}) || !slices.Equal(dropped(rec), []int{6}) {
|
||||
t.Errorf("the next batch carries %q and drops %v (%v); want #8 alone, #6 still dropped", got, rec, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Dropping an original the batch does not carry changes only the record:
|
||||
// the batch is not rebuilt, so a line that moved since does not move it.
|
||||
func TestDropIsRecordOnlyWhenTheLineMoved(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only seven", map[string]string{"app/seven.txt": "line\n"})
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{}
|
||||
head := openBatch(t, f, gh)
|
||||
if recordOf(t, gh).deferred()[7].Commit == "" {
|
||||
t.Fatal("#7 is not a recorded conflict")
|
||||
}
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", "7", "--reason", "needs #4", "--push")
|
||||
rec := recordOf(t, gh)
|
||||
if f.remoteHead(batchBranch1) != head || !slices.Equal(dropped(rec), []int{7}) || len(rec.deferred()) != 0 || len(gh.edited) != 1 {
|
||||
t.Errorf("the batch moved to %.12s (from %.12s), the record drops %v and defers %v; want only the record changed", f.remoteHead(batchBranch1), head, dropped(rec), rec.deferred())
|
||||
}
|
||||
}
|
||||
|
||||
// A drop is written to the open batch's record, and to no other: with no
|
||||
// batch open, or none ever, it is refused and nothing is written.
|
||||
func TestDropNeedsAnOpenBatch(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only seven", map[string]string{"app/seven.txt": "line\n"})
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--drop", "5", "--reason", "r", "--push"); code != 2 || !strings.Contains(stderr, "no open batch pull request on release-1.1 carries a record; a drop is recorded on one") {
|
||||
t.Errorf("a drop on a line that never had a batch = %d:\n%s", code, stderr)
|
||||
}
|
||||
openBatch(t, f, gh)
|
||||
mergeBatch(f, gh)
|
||||
written := len(gh.edited) + len(gh.issueComments)
|
||||
for _, args := range [][]string{{"--drop", "7", "--reason", "r", "--push"}, {"--undrop", "7", "--push"}} {
|
||||
if code, _, stderr := f.exec(gh, append([]string{"run", "--line", "release-1.1"}, args...)...); code != 2 || !strings.Contains(stderr, "carries a record; a drop is recorded on one") {
|
||||
t.Errorf("%q after the batch merged = %d:\n%s", args, code, stderr)
|
||||
}
|
||||
}
|
||||
if len(gh.edited)+len(gh.issueComments) != written {
|
||||
t.Error("a refused drop wrote to GitHub")
|
||||
}
|
||||
}
|
||||
|
||||
// A drop is a decision kept beside whatever else decides an original: the
|
||||
// rule, label or carry that applies first is its category, and the drop
|
||||
// stays on record. backport/include does not lift it.
|
||||
func TestDropOverlay(t *testing.T) {
|
||||
setup := func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "feat(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{labels: map[string][]int{labelInclude: {7}}}
|
||||
openBatch(t, f, gh)
|
||||
return f, gh
|
||||
}
|
||||
category := func(t *testing.T, gh *fakeGH, n int) string {
|
||||
t.Helper()
|
||||
s := bodyReportOf(t, gh.find(90).Body)
|
||||
where, err := s.doc.categories()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return where[n]
|
||||
}
|
||||
merge := func(f *fixture, n int) string {
|
||||
mc, _, _ := strings.Cut(f.git("log", "--first-parent", "--format=%H", "--grep", fmt.Sprintf("(#%d)$", n), "main"), "\n")
|
||||
return mc
|
||||
}
|
||||
drop := func(t *testing.T, f *fixture, gh *fakeGH, n string) {
|
||||
t.Helper()
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", n, "--reason", "not for 1.1", "--push")
|
||||
}
|
||||
t.Run("then backport/skip", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
drop(t, f, gh, "6")
|
||||
gh.labels[labelSkip] = []int{6}
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if got := category(t, gh, 6); got != "skipped.skip-label" || !slices.Equal(dropped(recordOf(t, gh)), []int{6}) {
|
||||
t.Errorf("#6 is %q, dropped %v", got, dropped(recordOf(t, gh)))
|
||||
}
|
||||
})
|
||||
t.Run("then picked onto the line by hand", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
drop(t, f, gh, "6")
|
||||
pickOnLine(f, 6, map[string]string{"app/six.txt": "6\n"})
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if got := category(t, gh, 6); got != "skipped.on-line" || !slices.Equal(dropped(recordOf(t, gh)), []int{6}) {
|
||||
t.Errorf("#6 is %q, dropped %v", got, dropped(recordOf(t, gh)))
|
||||
}
|
||||
})
|
||||
t.Run("a feat backport/include let in", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
drop(t, f, gh, "7")
|
||||
if got := category(t, gh, 7); got != "skipped.dropped" || slices.Contains(f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)), "feat(app): seven (#7)") {
|
||||
t.Errorf("#7 is %q; want it dropped and off the batch", got)
|
||||
}
|
||||
if !strings.Contains(gh.find(90).Body, "`backport/include`, skipped by a maintainer's drop") {
|
||||
t.Errorf("the overrides do not say what became of #7:\n%s", gh.find(90).Body)
|
||||
}
|
||||
})
|
||||
t.Run("undropped", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
drop(t, f, gh, "6")
|
||||
head := f.remoteHead(batchBranch1)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--undrop", "6", "--push")
|
||||
if got := category(t, gh, 6); got != "missing" || len(recordOf(t, gh).Dropped) != 0 || f.remoteHead(batchBranch1) != head || recordOf(t, gh).Built.Options.Mode != modeUndrop {
|
||||
t.Errorf("#6 is %q, dropped %v; want it missing until the next run, the batch where it was", got, dropped(recordOf(t, gh)))
|
||||
}
|
||||
if code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1"); code != 1 || !strings.Contains(stdout, "/pull/6 ") {
|
||||
t.Errorf("check = %d, want #6 missing:\n%s", code, stdout)
|
||||
}
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if got := category(t, gh, 6); got != "picked" {
|
||||
t.Errorf("after the next run #6 is %q, want it picked again", got)
|
||||
}
|
||||
})
|
||||
t.Run("a new original during a change to the record alone", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
drop(t, f, gh, "7")
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--undrop", "7", "--push")
|
||||
if got := category(t, gh, 8); got != "missing" {
|
||||
t.Errorf("#8 is %q, want missing until a run tries it", got)
|
||||
}
|
||||
})
|
||||
// A stored drop is bound to the merge it names: one that names another
|
||||
// original's, or one of no merge of it, is refused; one of an original
|
||||
// merged before the line branched is kept and decides nothing.
|
||||
t.Run("a stored drop of an earlier commit that names the original", func(t *testing.T) {
|
||||
// Before release-1.1 branched, a commit on main already ended in
|
||||
// (#6); the drop has to name the merge of #6 the line can take.
|
||||
f := newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.tag("v1.0.0")
|
||||
early := f.squash(6, "docs: what six will change", "", map[string]string{"docs/six.md": "6\n"})
|
||||
f.release("release-1.1")
|
||||
f.tag("v1.1.0")
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
drop(t, f, gh, "6")
|
||||
gh.find(90).Body = strings.Replace(gh.find(90).Body, `{"original":6,"merge":"`+merge(f, 6)+`"`, `{"original":6,"merge":"`+early+`"`, 1)
|
||||
if code, _, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 2 || !strings.Contains(stderr, "but #6 merged as") {
|
||||
t.Errorf("check = %d, want 2 naming the merge of #6:\n%s", code, stderr)
|
||||
}
|
||||
})
|
||||
t.Run("a stored drop of a commit inside a pull request", func(t *testing.T) {
|
||||
// A commit of a pull request's own branch can end in its number too,
|
||||
// but only the merge on main's first-parent line is the original.
|
||||
f := newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.tag("v1.0.0")
|
||||
mc3 := f.prCommits(3, "feat(app): three", "", change{files: map[string]string{"app/three.txt": "3\n"}, message: "feat(app): the part of three (#3)"})
|
||||
inner := f.git("rev-parse", mc3+"^2")
|
||||
f.release("release-1.1")
|
||||
f.tag("v1.1.0")
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
drop(t, f, gh, "6")
|
||||
gh.find(90).Body = strings.Replace(gh.find(90).Body, `{"original":6,"merge":"`+merge(f, 6)+`"`, `{"original":3,"merge":"`+inner+`"`, 1)
|
||||
if code, _, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 2 || !strings.Contains(stderr, "which is no merge of it on main") {
|
||||
t.Errorf("check = %d, want 2:\n%s", code, stderr)
|
||||
}
|
||||
})
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
original func(f *fixture) (int, string)
|
||||
refused bool
|
||||
}{
|
||||
{"another original's merge", func(f *fixture) (int, string) { return 6, merge(f, 5) }, true},
|
||||
{"a commit that merged no pull request", func(f *fixture) (int, string) { return 6, f.git("rev-list", "--max-parents=0", "main") }, true},
|
||||
{"a merge from before the line branched, of another pull request", func(f *fixture) (int, string) { return 2, merge(f, 1) }, true},
|
||||
{"a merge from before the line branched", func(f *fixture) (int, string) { return 1, merge(f, 1) }, false},
|
||||
} {
|
||||
t.Run("a stored drop of "+tc.name, func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
drop(t, f, gh, "6")
|
||||
n, mc := tc.original(f)
|
||||
gh.find(90).Body = strings.Replace(gh.find(90).Body, `{"original":6,"merge":"`+merge(f, 6)+`"`, fmt.Sprintf(`{"original":%d,"merge":"%s"`, n, mc), 1)
|
||||
code, _, stderr := f.exec(gh, "check", "--line", "release-1.1")
|
||||
switch {
|
||||
case tc.refused && (code != 2 || !strings.Contains(stderr, fmt.Sprintf("drops #%d by %.12s", n, mc))):
|
||||
t.Errorf("check = %d, want 2 naming the drop:\n%s", code, stderr)
|
||||
case !tc.refused && code == 2:
|
||||
t.Errorf("check = %d, want the drop kept:\n%s", code, stderr)
|
||||
case !tc.refused:
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if !slices.Equal(dropped(recordOf(t, gh)), []int{1}) || category(t, gh, 6) != "picked" {
|
||||
t.Errorf("the next run drops %v and puts #6 in %q; want #1's drop carried on and #6 picked", dropped(recordOf(t, gh)), category(t, gh, 6))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// What a drop cannot do is refused before anything is touched.
|
||||
func TestDropRefuses(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc5 := f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{labels: map[string][]int{labelSkip: {7, 9}}}
|
||||
head := openBatch(t, f, gh)
|
||||
f.onLine("release-1.1", "fix(app): five\n\n(cherry picked from commit "+mc5+")", map[string]string{"app/five.txt": "5\n"})
|
||||
// The line carries these too, by hand, while backport/skip and rule 2
|
||||
// decide them before the line is looked at.
|
||||
mc7 := f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
mc8 := f.pr(8, "feat(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
f.onLine("release-1.1", "fix(app): seven\n\n(cherry picked from commit "+mc7+")", map[string]string{"app/seven.txt": "7\n"})
|
||||
f.onLine("release-1.1", "feat(app): eight\n\n(cherry picked from commit "+mc8+")", map[string]string{"app/eight.txt": "8\n"})
|
||||
// #9 is on the line only through a pick of its own commit, which a later
|
||||
// line change has since moved away from what main has.
|
||||
mc9 := f.pr(9, "fix(app): nine", "", map[string]string{"app/nine.txt": "9\n"})
|
||||
f.onLine("release-1.1", "fix(app): nine\n\n(cherry picked from commit "+f.git("rev-parse", mc9+"^2")+")", map[string]string{"app/nine.txt": "9\n"})
|
||||
f.onLine("release-1.1", "fix(app): nine, as the line needs it", map[string]string{"app/nine.txt": "nine on release-1.1\n"})
|
||||
for _, tc := range []struct {
|
||||
args []string
|
||||
why string
|
||||
}{
|
||||
{[]string{"--drop", "4242", "--reason", "r"}, "#4242 did not merge into main"},
|
||||
{[]string{"--drop", "5", "--reason", "r"}, "already on release-1.1; a drop cannot remove code the line has"},
|
||||
{[]string{"--drop", "7", "--reason", "r"}, "#7 is already on release-1.1; a drop cannot remove code the line has"},
|
||||
{[]string{"--drop", "8", "--reason", "r"}, "#8 is already on release-1.1; a drop cannot remove code the line has"},
|
||||
{[]string{"--drop", "9", "--reason", "r"}, "#9 is already on release-1.1; a drop cannot remove code the line has"},
|
||||
{[]string{"--drop", "6"}, "--drop needs a --reason"},
|
||||
{[]string{"--drop", "6", "--reason", "r", "--defer", "6"}, "give one"},
|
||||
{[]string{"--drop", "6", "--reason", "r", "--continue"}, "takes no --onto, --continue or --date"},
|
||||
{[]string{"--drop", "6", "--reason", "r", "--onto", head}, "takes no --onto, --continue or --date"},
|
||||
{[]string{"--drop", "6", "--reason", "r", "--date", "2026-10-26"}, "takes no --onto, --continue or --date"},
|
||||
{[]string{"--undrop", "6", "--reason", "r"}, "--undrop takes no --reason"},
|
||||
{[]string{"--undrop", "6"}, "#6 is not dropped from release-1.1"},
|
||||
{[]string{"--drop", "6", "--undrop", "6", "--reason", "r"}, "give one"},
|
||||
{[]string{"--drop", "six", "--reason", "r"}, "not a list of pull request numbers"},
|
||||
} {
|
||||
code, _, stderr := f.exec(gh, append(append([]string{"run", "--line", "release-1.1"}, tc.args...), "--push")...)
|
||||
f.git("switch", "-q", "main")
|
||||
if code != 2 || !strings.Contains(stderr, tc.why) {
|
||||
t.Errorf("run %q = %d, want 2 with %q:\n%s", tc.args, code, tc.why, stderr)
|
||||
}
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != head || len(gh.edited) != 0 || len(reportsPosted(gh)) != 1 {
|
||||
t.Errorf("a refused drop moved the batch to %.12s or wrote %q", f.remoteHead(batchBranch1), gh.edited)
|
||||
}
|
||||
}
|
||||
|
||||
// A drop or an undrop asked for while a close was left changes nothing: the
|
||||
// run finishes the close, and no open batch pull request is left to record
|
||||
// it on.
|
||||
func TestDropDuringAnUnfinishedClose(t *testing.T) {
|
||||
deferAll := []string{"run", "--line", "release-1.1", "--defer", "5", "--reason", "fails the app e2e", "--push"}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
stop func(f *fixture, gh *fakeGH)
|
||||
refused bool
|
||||
args []string
|
||||
}{
|
||||
{"a drop after the close failed", func(f *fixture, gh *fakeGH) { gh.fail = map[string]error{"closePull": errString("HTTP 502")} }, false, []string{"--drop", "6", "--reason", "not for 1.1"}},
|
||||
{"an undrop after the delete failed", func(f *fixture, gh *fakeGH) {
|
||||
gh.after = map[string]func(){"closePull": func() { f.refusePushes(true) }}
|
||||
}, true, []string{"--undrop", "6"}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f, gh, _ := closeWorld(t)
|
||||
tc.stop(f, gh)
|
||||
if code, _, stderr := f.exec(gh, deferAll...); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if tc.refused {
|
||||
f.refusePushes(false)
|
||||
}
|
||||
gh.after = nil
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
code, _, stderr := f.exec(gh, append([]string{"run", "--line", "release-1.1", "--push"}, tc.args...)...)
|
||||
f.git("switch", "-q", "main")
|
||||
rec, err := parseRecord(gh.find(90).Body, notifyLine)
|
||||
if code != 2 || !strings.Contains(stderr, "finished the close of #90 and made no other change") || gh.find(90).State != "closed" || f.remoteHead(batchBranch1) != "" || err != nil || len(rec.Dropped) != 0 {
|
||||
t.Errorf("run %q = %d, #90 %s, branch %q, record %+v, %v; want 2, the close finished and nothing dropped:\n%s", tc.args, code, gh.find(90).State, f.remoteHead(batchBranch1), rec, err, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The batch carries an original whose first commit the line already had, so
|
||||
// its pick came out empty and its unit holds the rest: a drop still takes
|
||||
// that unit off.
|
||||
func TestDropTakesOffAUnitWithAnEmptyPick(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): the first part, on its own", map[string]string{"app/already.txt": "first\n"})
|
||||
f.prCommits(5, "fix(app): a fix in two parts", "",
|
||||
change{files: map[string]string{"app/already.txt": "first\n"}},
|
||||
change{files: map[string]string{"app/new.txt": "second\n"}},
|
||||
)
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", "5", "--reason", "not for 1.1", "--push")
|
||||
head := f.remoteHead(batchBranch1)
|
||||
if got := f.subjects("origin/release-1.1", head); !slices.Equal(got, []string{"fix(app): six (#6)"}) || f.git("ls-tree", "--name-only", head, "--", "app/new.txt") != "" {
|
||||
t.Errorf("after the drop the batch carries %q, want #6 alone and nothing of #5", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A report keeps each drop the record keeps, with the merge commit it is
|
||||
// bound to, its reason and the batch pull request that recorded it, so an
|
||||
// earlier report still says why after the drop is withdrawn.
|
||||
func TestReportKeepsEachDrop(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
mc6 := f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
stdout := mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", "6", "--reason", "needs the rework of #4", "--push")
|
||||
dropped := bodyReportOf(t, stdout)
|
||||
want := `"overlays":{"dropped":[{"original":6,"merge":"` + mc6 + `","reason":"needs the rework of #4","on":90,"date":"2026-10-26"}]`
|
||||
if !strings.Contains(dropped.raw, want) {
|
||||
t.Errorf("the report does not keep the drop with its merge and reason:\n%s", dropped.raw)
|
||||
}
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--undrop", "6", "--push")
|
||||
if raw := bodyReportOf(t, gh.find(90).Body).raw; !strings.Contains(raw, `"overlays":{"dropped":[],`) {
|
||||
t.Errorf("the report after the undrop still keeps a drop:\n%s", raw)
|
||||
}
|
||||
code, out, stderr := f.exec(gh, "report", "--pr", "90", "--id", dropped.doc.ID)
|
||||
if code != 0 || !strings.Contains(out, "dropped: #6, merged as "+mc6[:12]+", on #90: needs the rework of #4\n") {
|
||||
t.Errorf("report --id of the earlier report = %d, want it to show the drop:\n%s%s", code, out, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A report's drop is held to the record's rules and bound to main as the
|
||||
// record's drops are: a reason a run does not write, another original's
|
||||
// merge, or an earlier commit on main that names the same number is
|
||||
// refused, however the ID was made to match, in the body and in a trusted
|
||||
// report comment alike.
|
||||
func TestReportRefusesAForgedDrop(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.tag("v1.0.0")
|
||||
early := f.squash(6, "docs: what six will change", "", map[string]string{"docs/six.md": "6\n"})
|
||||
f.release("release-1.1")
|
||||
f.tag("v1.1.0")
|
||||
mc5 := f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
mc6 := f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", "6", "--reason", "not for 1.1", "--push")
|
||||
body, comments := gh.find(90).Body, slices.Clone(gh.issueComments)
|
||||
s := bodyReportOf(t, body)
|
||||
for _, tc := range []struct{ name, old, new, why string }{
|
||||
{"a reason of two lines", `"reason":"not for 1.1"`, `"reason":"not for 1.1\nwarning: forged"`, "which is no reason a run writes"},
|
||||
{"another original's merge", `"merge":"` + mc6 + `"`, `"merge":"` + mc5 + `"`, "which merged #5"},
|
||||
{"an earlier commit that names the same number", `"merge":"` + mc6 + `"`, `"merge":"` + early + `"`, "but #6 merged as " + mc6[:12]},
|
||||
} {
|
||||
raw := strings.Replace(s.raw, tc.old, tc.new, 1)
|
||||
if raw == s.raw {
|
||||
t.Fatalf("the report holds no %s to change:\n%s", tc.old, s.raw)
|
||||
}
|
||||
var d reportDoc
|
||||
if err := decodeStrict(raw, &d); err != nil {
|
||||
t.Fatalf("the changed report does not decode: %v", err)
|
||||
}
|
||||
d.ID = reportID(d)
|
||||
forged := storedReport{doc: d, raw: reportJSON(d)}
|
||||
refused := func(t *testing.T, args ...string) {
|
||||
t.Helper()
|
||||
code, out, stderr := f.exec(gh, append([]string{"report", "--pr", "90"}, args...)...)
|
||||
if code != 2 || !strings.Contains(stderr, tc.why) || strings.Contains(out+stderr, "\nwarning: forged") {
|
||||
t.Errorf("report of a forged drop = %d, want 2 with %q:\n%s%s", code, tc.why, out, stderr)
|
||||
}
|
||||
}
|
||||
t.Run(tc.name+", in the body", func(t *testing.T) {
|
||||
gh.find(90).Body = strings.Replace(body, reportBlock(s.raw), reportBlock(forged.raw), 1)
|
||||
defer func() { gh.find(90).Body = body }()
|
||||
refused(t)
|
||||
})
|
||||
t.Run(tc.name+", in a comment", func(t *testing.T) {
|
||||
text, err := reportComment(forged)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c := issueComment{ID: 9000, Body: text}
|
||||
c.User.Login, c.User.Type = "maintainer", "User"
|
||||
gh.issueComments = append(slices.Clone(comments), c)
|
||||
defer func() { gh.issueComments = comments }()
|
||||
refused(t, "--id", d.ID)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// droppedNumbers are the originals a report's drops name.
|
||||
func droppedNumbers(es []droppedEntry) []int {
|
||||
var out []int
|
||||
for _, e := range es {
|
||||
out = append(out, e.Original)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// dropWorld has #3 merged before release-1.1 branched, an earlier commit on
|
||||
// main that names #6, and a batch with #5 and #6 from which #6 is dropped.
|
||||
func dropWorld(t *testing.T) (f *fixture, gh *fakeGH, inherited, mc5, mc6 string) {
|
||||
t.Helper()
|
||||
f = newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.tag("v1.0.0")
|
||||
inherited = f.prCommits(3, "feat(app): three", "", change{files: map[string]string{"app/three.txt": "3\n"}, message: "feat(app): part of three (#3)"})
|
||||
f.squash(6, "docs: what six will change", "", map[string]string{"docs/six.md": "6\n"})
|
||||
f.release("release-1.1")
|
||||
f.tag("v1.1.0")
|
||||
mc5 = f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
mc6 = f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh = &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--drop", "6", "--reason", "not for 1.1", "--push")
|
||||
return f, gh, inherited, mc5, mc6
|
||||
}
|
||||
|
||||
// A report's drops and its categories hold each other up: an original it
|
||||
// has dropped needs a drop that binds to it, and a drop of an original it
|
||||
// classified needs that original dropped or decided by an earlier rule. A
|
||||
// drop inherited from before the line branched decides nothing and stays
|
||||
// readable. Each holds in the body and in a trusted report comment.
|
||||
func TestReportTiesDropsToTheirCategory(t *testing.T) {
|
||||
f, gh, inherited, mc5, _ := dropWorld(t)
|
||||
body, comments := gh.find(90).Body, slices.Clone(gh.issueComments)
|
||||
s := bodyReportOf(t, body)
|
||||
kept := func(n int, merge string) droppedEntry {
|
||||
return droppedEntry{Original: n, Merge: merge, Reason: "carried from an earlier batch", On: 89, Date: "2026-10-25"}
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
change func(d *reportDoc)
|
||||
why string
|
||||
}{
|
||||
{"an inherited drop in place of the classified one", func(d *reportDoc) { d.Overlays.Dropped[0] = kept(3, inherited) }, "has #6 dropped, but keeps no drop of it"},
|
||||
{"an inherited drop classified as dropped", func(d *reportDoc) {
|
||||
d.Primary.Skipped.Dropped = append(d.Primary.Skipped.Dropped, 3)
|
||||
d.Overlays.Dropped = append(d.Overlays.Dropped, kept(3, inherited))
|
||||
}, "has #3 dropped, but no drop of it binds to an original it classified"},
|
||||
{"a drop of an original it picked", func(d *reportDoc) { d.Overlays.Dropped = append(d.Overlays.Dropped, kept(5, mc5)) }, "drops #5, which it has picked"},
|
||||
{"an inherited drop beside the classified one", func(d *reportDoc) { d.Overlays.Dropped = append(d.Overlays.Dropped, kept(3, inherited)) }, ""},
|
||||
} {
|
||||
var d reportDoc
|
||||
if err := decodeStrict(s.raw, &d); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tc.change(&d)
|
||||
d.canonical()
|
||||
d.ID = reportID(d)
|
||||
forged := storedReport{doc: d, raw: reportJSON(d)}
|
||||
check := func(t *testing.T, args ...string) {
|
||||
t.Helper()
|
||||
code, out, stderr := f.exec(gh, append([]string{"report", "--pr", "90"}, args...)...)
|
||||
switch {
|
||||
case tc.why == "" && (code != 0 || !strings.Contains(out, "#6 skipped.dropped") || !strings.Contains(out, "dropped: #3, merged as "+inherited[:12]) || strings.Contains(out, "#3 skipped.dropped")):
|
||||
t.Errorf("report of a sound report = %d, want it read with #3's drop beside #6's:\n%s%s", code, out, stderr)
|
||||
case tc.why != "" && (code != 2 || out != "" || !strings.Contains(stderr, tc.why)):
|
||||
t.Errorf("report = %d, want 2 with %q:\n%s%s", code, tc.why, out, stderr)
|
||||
}
|
||||
}
|
||||
t.Run(tc.name+", in the body", func(t *testing.T) {
|
||||
gh.find(90).Body = strings.Replace(body, reportBlock(s.raw), reportBlock(forged.raw), 1)
|
||||
defer func() { gh.find(90).Body = body }()
|
||||
check(t)
|
||||
})
|
||||
t.Run(tc.name+", in a comment", func(t *testing.T) {
|
||||
text, err := reportComment(forged)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c := issueComment{ID: 9001, Body: text}
|
||||
c.User.Login, c.User.Type = "maintainer", "User"
|
||||
gh.issueComments = append(slices.Clone(comments), c)
|
||||
defer func() { gh.issueComments = comments }()
|
||||
check(t, "--id", d.ID)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,606 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"cmp"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fixture is a repository whose origin is a bare repository next to it, with
|
||||
// main and release lines shaped the way GitHub leaves them: every pull request
|
||||
// lands as a merge commit whose message is its title, its number and its body.
|
||||
type fixture struct {
|
||||
t *testing.T
|
||||
dir string
|
||||
remote string
|
||||
now time.Time
|
||||
// tool is the commit the tool says it was built from.
|
||||
tool string
|
||||
}
|
||||
|
||||
func newFixture(t *testing.T) *fixture {
|
||||
t.Helper()
|
||||
// A hook or an outer repository can export these into the test process.
|
||||
for _, key := range []string{"GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE", "GIT_COMMON_DIR", "GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES"} {
|
||||
t.Setenv(key, "")
|
||||
if err := os.Unsetenv(key); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
// The user's global config may sign every commit or run hooks.
|
||||
t.Setenv("GIT_CONFIG_GLOBAL", os.DevNull)
|
||||
t.Setenv("GIT_CONFIG_NOSYSTEM", "1")
|
||||
for _, who := range []string{"AUTHOR", "COMMITTER"} {
|
||||
t.Setenv("GIT_"+who+"_NAME", "Batch Test")
|
||||
t.Setenv("GIT_"+who+"_EMAIL", "batch@example.invalid")
|
||||
}
|
||||
root := t.TempDir()
|
||||
f := &fixture{t: t, dir: filepath.Join(root, "work"), remote: filepath.Join(root, "remote.git"),
|
||||
now: time.Date(2026, 10, 26, 6, 0, 0, 0, time.UTC)}
|
||||
for _, dir := range []string{f.remote, f.dir} {
|
||||
args := []string{"init", "-q", "-b", "main", dir}
|
||||
if dir == f.remote {
|
||||
args = []string{"init", "-q", "--bare", "-b", "main", dir}
|
||||
}
|
||||
if out, err := exec.Command("git", args...).CombinedOutput(); err != nil {
|
||||
t.Fatalf("git %v: %v\n%s", args, err, out)
|
||||
}
|
||||
}
|
||||
t.Chdir(f.dir)
|
||||
f.git("remote", "add", "origin", f.remote)
|
||||
f.write("README.md", "seed\n")
|
||||
f.write("chart/values.yaml", "replicas: 1\nimage: app:1\n")
|
||||
f.write(policyPage, policyWith(""))
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "chore: seed")
|
||||
f.git("push", "-q", "origin", "main")
|
||||
return f
|
||||
}
|
||||
|
||||
func (f *fixture) git(args ...string) string {
|
||||
f.t.Helper()
|
||||
out, err := exec.Command("git", args...).CombinedOutput()
|
||||
if err != nil {
|
||||
f.t.Fatalf("git %v: %v\n%s", args, err, out)
|
||||
}
|
||||
return strings.TrimSpace(string(out))
|
||||
}
|
||||
|
||||
func (f *fixture) write(path, content string) {
|
||||
f.t.Helper()
|
||||
full := filepath.Join(f.dir, path)
|
||||
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(full, []byte(content), 0o644); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// apply writes files in a stable order; an empty content removes the file.
|
||||
func (f *fixture) apply(files map[string]string) {
|
||||
f.t.Helper()
|
||||
keys := make([]string, 0, len(files))
|
||||
for k := range files {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
slices.Sort(keys)
|
||||
for _, k := range keys {
|
||||
if files[k] == "" {
|
||||
f.git("rm", "-q", "--", k)
|
||||
continue
|
||||
}
|
||||
f.write(k, files[k])
|
||||
}
|
||||
f.git("add", "-A")
|
||||
}
|
||||
|
||||
// pr merges pull request n into main as a merge commit and returns it.
|
||||
func (f *fixture) pr(n int, title, body string, files map[string]string) string {
|
||||
f.t.Helper()
|
||||
branch := fmt.Sprintf("pr-%d", n)
|
||||
f.git("switch", "-q", "main")
|
||||
f.git("switch", "-q", "-c", branch)
|
||||
f.apply(files)
|
||||
f.git("commit", "-q", "-m", fmt.Sprintf("change for #%d", n))
|
||||
f.git("switch", "-q", "main")
|
||||
f.git("merge", "-q", "--no-ff", "-m", fmt.Sprintf("%s (#%d)\n\n%s", title, n, body), branch)
|
||||
f.git("push", "-q", "origin", "main")
|
||||
return f.git("rev-parse", "HEAD")
|
||||
}
|
||||
|
||||
// joined makes a merge of tip onto base with tip's tree, whatever a merge of
|
||||
// the two would make.
|
||||
func (f *fixture) joined(base, tip, message string) string {
|
||||
f.t.Helper()
|
||||
return f.git("commit-tree", "-p", base, "-p", tip, "-m", message, tip+"^{tree}")
|
||||
}
|
||||
|
||||
// contributor writes the commits of the pull requests that say so.
|
||||
const contributor = "Contributor One <one@example.invalid>"
|
||||
|
||||
// change is one step of a pull request's branch: a commit of files, by author
|
||||
// with message when given, or with mergeMain a merge of main into the branch
|
||||
// after main moved on.
|
||||
type change struct {
|
||||
files map[string]string
|
||||
author string
|
||||
message string
|
||||
mergeMain bool
|
||||
}
|
||||
|
||||
// prCommits merges pull request n into main as a merge commit of a branch
|
||||
// made of changes, and returns the merge commit.
|
||||
func (f *fixture) prCommits(n int, title, body string, changes ...change) string {
|
||||
f.t.Helper()
|
||||
branch := fmt.Sprintf("pr-%d", n)
|
||||
f.git("switch", "-q", "main")
|
||||
f.git("switch", "-q", "-c", branch)
|
||||
for i, c := range changes {
|
||||
if c.mergeMain {
|
||||
f.git("switch", "-q", "main")
|
||||
f.apply(map[string]string{fmt.Sprintf("main/%d-%d.txt", n, i): "main moved\n"})
|
||||
f.git("commit", "-q", "-m", "chore: move main along")
|
||||
f.git("switch", "-q", branch)
|
||||
f.git("merge", "-q", "--no-ff", "-m", "Merge branch 'main' into "+branch, "main")
|
||||
continue
|
||||
}
|
||||
f.apply(c.files)
|
||||
msg := c.message
|
||||
if msg == "" {
|
||||
msg = fmt.Sprintf("change %d for #%d", i+1, n)
|
||||
}
|
||||
args := []string{"commit", "-q", "--allow-empty", "-m", msg}
|
||||
if c.author != "" {
|
||||
args = append(args, "--author", c.author)
|
||||
}
|
||||
f.git(args...)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
f.git("merge", "-q", "--no-ff", "-m", fmt.Sprintf("%s (#%d)\n\n%s", title, n, body), branch)
|
||||
f.git("push", "-q", "origin", "main")
|
||||
return f.git("rev-parse", "HEAD")
|
||||
}
|
||||
|
||||
// squash merges pull request n into main as a single commit.
|
||||
func (f *fixture) squash(n int, title, body string, files map[string]string) string {
|
||||
f.t.Helper()
|
||||
f.git("switch", "-q", "main")
|
||||
f.apply(files)
|
||||
f.git("commit", "-q", "-m", fmt.Sprintf("%s (#%d)\n\n%s", title, n, body))
|
||||
f.git("push", "-q", "origin", "main")
|
||||
return f.git("rev-parse", "HEAD")
|
||||
}
|
||||
|
||||
// release cuts a release line at main's head.
|
||||
func (f *fixture) release(name string) {
|
||||
f.t.Helper()
|
||||
f.git("push", "-q", "origin", "main:refs/heads/"+name)
|
||||
}
|
||||
|
||||
// onLine commits onto a release line, as a maintainer's merge would land
|
||||
// there, and returns the commit.
|
||||
func (f *fixture) onLine(name, message string, files map[string]string) string {
|
||||
f.t.Helper()
|
||||
f.git("fetch", "-q", "origin")
|
||||
f.git("switch", "-q", "--detach", "origin/"+name)
|
||||
f.apply(files)
|
||||
f.git("commit", "-q", "--allow-empty", "-m", message)
|
||||
sha := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", sha+":refs/heads/"+name)
|
||||
f.git("switch", "-q", "main")
|
||||
return sha
|
||||
}
|
||||
|
||||
// descends says whether tip descends from base, failing the test when git
|
||||
// cannot tell.
|
||||
func descends(t *testing.T, base, tip string) bool {
|
||||
t.Helper()
|
||||
ok, err := isAncestor(base, tip)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return ok
|
||||
}
|
||||
|
||||
// remoteHead is what the bare remote has for a branch, or "".
|
||||
func (f *fixture) remoteHead(branch string) string {
|
||||
f.t.Helper()
|
||||
out, err := exec.Command("git", "--git-dir="+f.remote, "rev-parse", "--verify", "--quiet", "refs/heads/"+branch).Output()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimSpace(string(out))
|
||||
}
|
||||
|
||||
// fakeGH stands in for gh: the open pull requests it reports, and what the
|
||||
// tool asked it to create or edit.
|
||||
type fakeGH struct {
|
||||
prs []pullRequest
|
||||
err error
|
||||
perms map[string]string
|
||||
permErr error
|
||||
labels map[string][]int
|
||||
listErr error
|
||||
asked []string
|
||||
created []string
|
||||
edited []string
|
||||
bodies []string
|
||||
// issueComments are the comments on the batch pull request, which
|
||||
// createComment and editComment change.
|
||||
issueComments []issueComment
|
||||
// bot is what BACKPORT_BATCH_BOT would say.
|
||||
bot string
|
||||
// listed counts the pages of recentPulls read.
|
||||
listed int
|
||||
// closedWrites names every write the tool attempted on a pull request
|
||||
// that is no longer open; exec fails the test on one.
|
||||
closedWrites []string
|
||||
// fail makes the next call of an operation, by its method name, fail.
|
||||
fail map[string]error
|
||||
// nextPR is the number createPull gives, 90 when unset.
|
||||
nextPR int
|
||||
// remoteHead reads a branch's head off the fixture's remote: GitHub
|
||||
// reports an open pull request at what its branch has.
|
||||
remoteHead func(string) string
|
||||
// commented lists the pull request each comment went to, in order.
|
||||
commented []int
|
||||
// after runs once an operation, by its method name, has succeeded: what
|
||||
// someone else does on GitHub meanwhile.
|
||||
after map[string]func()
|
||||
}
|
||||
|
||||
// live is pr as GitHub reports it now: an open one at the head its branch
|
||||
// has on the remote.
|
||||
func (g *fakeGH) live(pr pullRequest) pullRequest {
|
||||
if g.remoteHead != nil && (pr.State == "" || pr.State == "open") {
|
||||
if head := g.remoteHead(pr.Head.Ref); head != "" {
|
||||
pr.Head.SHA = head
|
||||
}
|
||||
}
|
||||
return pr
|
||||
}
|
||||
|
||||
// inject is the failure a test set for the next call of op, once.
|
||||
func (g *fakeGH) inject(op string) error {
|
||||
err := g.fail[op]
|
||||
delete(g.fail, op)
|
||||
return err
|
||||
}
|
||||
|
||||
func (g *fakeGH) pull(n int) (pullRequest, error) {
|
||||
if err := g.inject("pull"); err != nil {
|
||||
return pullRequest{}, err
|
||||
}
|
||||
for _, pr := range g.prs {
|
||||
if pr.Number == n {
|
||||
return g.live(pr), nil
|
||||
}
|
||||
}
|
||||
return pullRequest{}, fmt.Errorf("HTTP 404: no pull request #%d", n)
|
||||
}
|
||||
|
||||
// isOpen says whether the fake has pull request n open, or does not know it.
|
||||
func (g *fakeGH) isOpen(n int) bool {
|
||||
for _, pr := range g.prs {
|
||||
if pr.Number == n {
|
||||
return pr.State == "" || pr.State == "open"
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// write refuses a write to a pull request that is no longer open, which the
|
||||
// tool must never attempt.
|
||||
func (g *fakeGH) write(n int, what string) error {
|
||||
if g.isOpen(n) {
|
||||
return nil
|
||||
}
|
||||
g.closedWrites = append(g.closedWrites, fmt.Sprintf("%s on #%d", what, n))
|
||||
return fmt.Errorf("#%d is closed", n)
|
||||
}
|
||||
|
||||
// recentPulls pages every pull request against base, newest first, as the
|
||||
// REST listing sorted by creation does; a higher number is a newer one.
|
||||
func (g *fakeGH) recentPulls(base string, page int) ([]pullRequest, error) {
|
||||
g.listed++
|
||||
var all []pullRequest
|
||||
for _, pr := range g.prs {
|
||||
if pr.Base.Ref == base {
|
||||
all = append(all, pr)
|
||||
}
|
||||
}
|
||||
slices.SortFunc(all, func(a, b pullRequest) int { return b.Number - a.Number })
|
||||
from := min(len(all), (page-1)*recentPullsPage)
|
||||
return slices.Clone(all[from:min(len(all), from+recentPullsPage)]), g.err
|
||||
}
|
||||
|
||||
func (g *fakeGH) batchBot() string { return g.bot }
|
||||
|
||||
func (g *fakeGH) comments(int) ([]issueComment, error) {
|
||||
defer func() {
|
||||
if f := g.after["comments"]; f != nil {
|
||||
f()
|
||||
}
|
||||
}()
|
||||
return slices.Clone(g.issueComments), nil
|
||||
}
|
||||
|
||||
func (g *fakeGH) createComment(number int, bodyFile string) error {
|
||||
if err := g.write(number, "a comment"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := g.inject("createComment"); err != nil {
|
||||
return err
|
||||
}
|
||||
g.commented = append(g.commented, number)
|
||||
defer func() {
|
||||
if f := g.after["createComment"]; f != nil {
|
||||
f()
|
||||
}
|
||||
}()
|
||||
data, err := os.ReadFile(bodyFile)
|
||||
var c issueComment
|
||||
c.ID, c.Body = int64(1000+len(g.issueComments)), string(data)
|
||||
c.User.Login, c.User.Type = "maintainer", "User"
|
||||
g.issueComments = append(g.issueComments, c)
|
||||
return err
|
||||
}
|
||||
|
||||
func (g *fakeGH) editComment(id int64, bodyFile string) error {
|
||||
data, err := os.ReadFile(bodyFile)
|
||||
for i := range g.issueComments {
|
||||
if g.issueComments[i].ID == id {
|
||||
g.issueComments[i].Body = string(data)
|
||||
return err
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("no comment %d", id)
|
||||
}
|
||||
|
||||
func (g *fakeGH) labelled(label string) ([]int, error) {
|
||||
return g.labels[label], g.listErr
|
||||
}
|
||||
|
||||
// permission answers write for the fixture's maintainer and read for anyone
|
||||
// else, unless the test says otherwise.
|
||||
func (g *fakeGH) permission(login string) (string, error) {
|
||||
g.asked = append(g.asked, login)
|
||||
if g.permErr != nil {
|
||||
return "", g.permErr
|
||||
}
|
||||
if p, ok := g.perms[login]; ok {
|
||||
return p, nil
|
||||
}
|
||||
if login == "maintainer" {
|
||||
return "write", nil
|
||||
}
|
||||
return "read", nil
|
||||
}
|
||||
|
||||
func (g *fakeGH) openPulls(base string) ([]pullRequest, error) {
|
||||
var out []pullRequest
|
||||
for _, pr := range g.prs {
|
||||
if pr.Base.Ref == base && (pr.State == "" || pr.State == "open") {
|
||||
out = append(out, g.live(pr))
|
||||
}
|
||||
}
|
||||
return out, g.err
|
||||
}
|
||||
|
||||
// createPull opens the pull request on the fake, as the maintainer, at the
|
||||
// head the remote has, numbered nextPR.
|
||||
func (g *fakeGH) createPull(base, head, title, bodyFile string) (int, string, error) {
|
||||
if err := g.inject("createPull"); err != nil {
|
||||
return 0, "", err
|
||||
}
|
||||
g.created = append(g.created, base+" "+head+" "+title)
|
||||
if err := g.readBody(bodyFile); err != nil {
|
||||
return 0, "", err
|
||||
}
|
||||
n := cmp.Or(g.nextPR, 90)
|
||||
var pr pullRequest
|
||||
pr.Number, pr.Body, pr.State = n, g.bodies[len(g.bodies)-1], "open"
|
||||
pr.HTMLURL = fmt.Sprintf("https://github.com/%s/pull/%d", upstreamRepo, n)
|
||||
pr.User.Login, pr.User.Type = "maintainer", "User"
|
||||
pr.Head.Ref = head
|
||||
if g.remoteHead != nil {
|
||||
pr.Head.SHA = g.remoteHead(head)
|
||||
}
|
||||
pr.Head.Repo = &struct {
|
||||
FullName string `json:"full_name"`
|
||||
}{upstreamRepo}
|
||||
pr.Base.Ref, pr.Base.Repo.FullName = base, upstreamRepo
|
||||
g.prs = append(slices.DeleteFunc(g.prs, func(p pullRequest) bool { return p.Number == n }), pr)
|
||||
return n, pr.HTMLURL, nil
|
||||
}
|
||||
|
||||
func (g *fakeGH) editPull(number int, title, bodyFile string) error {
|
||||
if err := g.write(number, "an edit"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := g.inject("editPull"); err != nil {
|
||||
return err
|
||||
}
|
||||
g.edited = append(g.edited, fmt.Sprintf("#%d %s", number, title))
|
||||
if err := g.readBody(bodyFile); err != nil {
|
||||
return err
|
||||
}
|
||||
if pr := g.find(number); pr != nil {
|
||||
pr.Body = g.bodies[len(g.bodies)-1]
|
||||
}
|
||||
if f := g.after["editPull"]; f != nil {
|
||||
f()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g *fakeGH) closePull(number int) error {
|
||||
if err := g.write(number, "a close"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := g.inject("closePull"); err != nil {
|
||||
return err
|
||||
}
|
||||
if pr := g.find(number); pr != nil {
|
||||
pr.State = "closed"
|
||||
}
|
||||
if f := g.after["closePull"]; f != nil {
|
||||
f()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g *fakeGH) find(n int) *pullRequest {
|
||||
for i := range g.prs {
|
||||
if g.prs[i].Number == n {
|
||||
return &g.prs[i]
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g *fakeGH) readBody(path string) error {
|
||||
data, err := os.ReadFile(path)
|
||||
g.bodies = append(g.bodies, string(data))
|
||||
return err
|
||||
}
|
||||
|
||||
// batchPR is an open batch pull request opened by a maintainer from a branch
|
||||
// of this repository, at whatever head the remote has for the branch.
|
||||
func (f *fixture) batchPR(n int, base, branch, body string) pullRequest {
|
||||
f.t.Helper()
|
||||
var pr pullRequest
|
||||
pr.Number, pr.Body = n, body
|
||||
pr.HTMLURL = fmt.Sprintf("https://github.com/%s/pull/%d", upstreamRepo, n)
|
||||
pr.User.Login, pr.User.Type, pr.AuthorAssociation = "maintainer", "User", "MEMBER"
|
||||
pr.Head.Ref, pr.Head.SHA = branch, f.remoteHead(branch)
|
||||
pr.Head.Repo = &struct {
|
||||
FullName string `json:"full_name"`
|
||||
}{upstreamRepo}
|
||||
pr.Base.Ref, pr.Base.Repo.FullName = base, upstreamRepo
|
||||
pr.State = "open"
|
||||
return pr
|
||||
}
|
||||
|
||||
// botBatchPR is batchPR opened by the bot the scheduled batch runs as.
|
||||
func (f *fixture) botBatchPR(gh *fakeGH, n int, base, branch, body string) pullRequest {
|
||||
f.t.Helper()
|
||||
gh.bot = defaultBatchBot
|
||||
pr := f.batchPR(n, base, branch, body)
|
||||
pr.User.Login, pr.User.Type, pr.AuthorAssociation = defaultBatchBot, "Bot", "CONTRIBUTOR"
|
||||
return pr
|
||||
}
|
||||
|
||||
// mergeBatch merges the open batch into release-1.1 with a merge commit, as
|
||||
// GitHub does, and deletes its branch, as the repository's
|
||||
// delete_branch_on_merge does. The pull request stays, merged, with its body
|
||||
// and comments.
|
||||
func mergeBatch(f *fixture, gh *fakeGH) {
|
||||
f.t.Helper()
|
||||
f.git("fetch", "-q", "origin")
|
||||
f.git("switch", "-q", "--detach", "origin/release-1.1")
|
||||
f.git("merge", "-q", "--no-ff", "-m", "Merge pull request #90 from cozystack/"+batchBranch1, "origin/"+batchBranch1)
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/release-1.1", ":refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
for i := range gh.prs {
|
||||
if gh.prs[i].Number == 90 {
|
||||
at := "2026-10-26T12:00:00Z"
|
||||
gh.prs[i].State, gh.prs[i].MergedAt = "closed", &at
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// exec runs the tool in the fixture with gh faked, on the fixture's date, and
|
||||
// fails the test when the tool wrote to a pull request that is not open.
|
||||
func (f *fixture) exec(gh *fakeGH, args ...string) (int, string, string) {
|
||||
f.t.Helper()
|
||||
if gh == nil {
|
||||
gh = &fakeGH{}
|
||||
}
|
||||
if gh.remoteHead == nil {
|
||||
gh.remoteHead = f.remoteHead
|
||||
}
|
||||
var stdout, stderr bytes.Buffer
|
||||
a := &app{gh: gh, stdout: &stdout, stderr: &stderr, now: func() time.Time { return f.now }, tool: f.tool}
|
||||
written := len(gh.closedWrites)
|
||||
code := a.main(args)
|
||||
if len(gh.closedWrites) > written {
|
||||
f.t.Errorf("%v wrote to a pull request that is not open: %q", args, gh.closedWrites[written:])
|
||||
}
|
||||
return code, stdout.String(), stderr.String()
|
||||
}
|
||||
|
||||
const batchBranch1 = "backport-batch/release-1.1-20261026"
|
||||
|
||||
// lines cuts and releases release-1.0 and then release-1.1 with one change
|
||||
// between them, leaving release-1.1 the newer of the two lines the batch runs
|
||||
// for.
|
||||
func (f *fixture) lines() {
|
||||
f.t.Helper()
|
||||
f.release("release-1.0")
|
||||
f.tag("v1.0.0")
|
||||
f.pr(1, "feat(app): something for 1.1", "", map[string]string{"app/feature.txt": "1.1\n"})
|
||||
f.release("release-1.1")
|
||||
f.tag("v1.1.0")
|
||||
}
|
||||
|
||||
// tag tags main's head on the remote: a stable release when the name has no
|
||||
// pre-release suffix.
|
||||
func (f *fixture) tag(name string) {
|
||||
f.t.Helper()
|
||||
f.git("push", "-q", "origin", "main:refs/tags/"+name)
|
||||
}
|
||||
|
||||
// subjects lists the subjects of the units on branch beyond base, oldest
|
||||
// first: the first-parent line, where a batch has one commit per original.
|
||||
func (f *fixture) subjects(base, branch string) []string {
|
||||
f.t.Helper()
|
||||
out := f.git("log", "--reverse", "--first-parent", "--format=%s", base+".."+branch)
|
||||
if out == "" {
|
||||
return nil
|
||||
}
|
||||
return strings.Split(out, "\n")
|
||||
}
|
||||
|
||||
// batchOnlyDay is the fixture's batch-only day: commits a test makes carry
|
||||
// the clock's date, later than it, unless the test dates them.
|
||||
const batchOnlyDay = "2026-10-26"
|
||||
|
||||
// policyWith is a policy page whose block lists entries, and the fixture's
|
||||
// batch-only day unless entries give one.
|
||||
func policyWith(entries string) string {
|
||||
if !strings.Contains(entries, batchOnlyKey+":") {
|
||||
entries += batchOnlyKey + ": " + batchOnlyDay + "\n"
|
||||
}
|
||||
return "## Backports\n\n" + policyFence + "\n# release-X.Y: YYYY-MM-DD\n" + entries + "```\n"
|
||||
}
|
||||
|
||||
// setPolicy commits a policy page with entries onto main.
|
||||
func (f *fixture) setPolicy(entries string) {
|
||||
f.t.Helper()
|
||||
f.squash(9000, "docs(release): set the policy", "", map[string]string{policyPage: policyWith(entries)})
|
||||
}
|
||||
@@ -0,0 +1,350 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// foldWorld is a line whose own change makes the first of #5's two commits
|
||||
// conflict, an empty batch pushed for it, and the originals around #5.
|
||||
type foldWorld struct {
|
||||
f *fixture
|
||||
gh *fakeGH
|
||||
head string
|
||||
mc5, mc6 string
|
||||
c1, c2 string // #5's own commits
|
||||
c6 string // #6's one commit
|
||||
sq8 string
|
||||
}
|
||||
|
||||
func newFoldWorld(t *testing.T) *foldWorld {
|
||||
t.Helper()
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
w := &foldWorld{f: f}
|
||||
w.mc5 = f.prCommits(5, "fix(chart): five", "",
|
||||
change{files: map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n"}},
|
||||
change{files: map[string]string{"app/five.txt": "5\n"}})
|
||||
own := strings.Fields(f.git("rev-list", "--reverse", w.mc5+"^1.."+w.mc5+"^2"))
|
||||
w.c1, w.c2 = own[0], own[1]
|
||||
w.mc6 = f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
w.c6 = f.git("rev-parse", w.mc6+"^2")
|
||||
f.pr(7, "feat(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
w.sq8 = f.squash(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+batchBranch1)
|
||||
f.git("fetch", "-q", "origin")
|
||||
w.head = f.git("rev-parse", "origin/"+batchBranch1)
|
||||
w.gh = &fakeGH{prs: []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, "")}}
|
||||
return w
|
||||
}
|
||||
|
||||
// commit writes files on parent with message and returns the commit.
|
||||
func (w *foldWorld) commit(parent, message string, files map[string]string) string {
|
||||
w.f.git("switch", "-q", "--detach", parent)
|
||||
w.f.apply(files)
|
||||
w.f.git("commit", "-q", "--allow-empty", "-m", message)
|
||||
sha := w.f.git("rev-parse", "HEAD")
|
||||
w.f.git("switch", "-q", "main")
|
||||
return sha
|
||||
}
|
||||
|
||||
// pick is a pick of source on parent, written as files, with its -x line.
|
||||
func (w *foldWorld) pick(parent, source string, files map[string]string) string {
|
||||
return w.commit(parent, "port\n\n(cherry picked from commit "+source+")", files)
|
||||
}
|
||||
|
||||
func (w *foldWorld) merge(base, tip, message string) string {
|
||||
w.f.git("switch", "-q", "--detach", base)
|
||||
w.f.git("merge", "-q", "--no-ff", "-m", message, tip)
|
||||
sha := w.f.git("rev-parse", "HEAD")
|
||||
w.f.git("switch", "-q", "main")
|
||||
return sha
|
||||
}
|
||||
|
||||
// joined makes a merge of tip onto base with tip's tree, whatever a merge of
|
||||
// the two would make.
|
||||
func (w *foldWorld) joined(base, tip, message string) string {
|
||||
return w.f.git("commit-tree", "-p", base, "-p", tip, "-m", message, tip+"^{tree}")
|
||||
}
|
||||
|
||||
func batchMessage(merges ...string) string {
|
||||
msg := "fix(chart): five (#5)\n"
|
||||
for _, m := range merges {
|
||||
msg += "\n" + batchTrailer + ": " + m
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
var (
|
||||
resolved1 = map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n"}
|
||||
resolved2 = map[string]string{"app/five.txt": "5\n"}
|
||||
)
|
||||
|
||||
// resolution is the right fold of #5: both its commits picked in order on the
|
||||
// batch head, the conflict resolved, under a merge that names it.
|
||||
func (w *foldWorld) resolution() string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.merge(w.head, r2, batchMessage(w.mc5))
|
||||
}
|
||||
|
||||
// fold accepts one original on the batch head in the batch's shape, inside the
|
||||
// files it touches, and nothing else.
|
||||
func TestFold(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
original int
|
||||
make func(w *foldWorld) string
|
||||
refusal string
|
||||
}{
|
||||
{
|
||||
name: "a resolution of #5", original: 5,
|
||||
make: (*foldWorld).resolution,
|
||||
},
|
||||
{
|
||||
name: "one commit, the way -m 1 picked it", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
return w.commit(w.head, "fix(chart): five (#5)\n\n(cherry picked from commit "+w.mc5+")", map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n", "app/five.txt": "5\n"})
|
||||
},
|
||||
refusal: "has 1 parents; #5 is a merged pull request",
|
||||
},
|
||||
{
|
||||
name: "a merge not on the batch head", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
base := w.commit(w.head, "something else", map[string]string{"app/five.txt": "x\n"})
|
||||
r1 := w.pick(base, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.merge(base, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "not on the head of batch pull request #90",
|
||||
},
|
||||
{
|
||||
name: "a merge that does not name #5", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.merge(w.head, r2, batchMessage())
|
||||
},
|
||||
refusal: "with 0 " + batchTrailer + " trailers",
|
||||
},
|
||||
{
|
||||
name: "two originals in one resolution", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
r3 := w.pick(r2, w.c6, map[string]string{"app/six.txt": "6\n"})
|
||||
return w.merge(w.head, r3, batchMessage(w.mc5, w.mc6))
|
||||
},
|
||||
refusal: "with 2 " + batchTrailer + " trailers",
|
||||
},
|
||||
{
|
||||
name: "a commit of another pull request", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
r3 := w.pick(r2, w.c6, map[string]string{"app/six.txt": "6\n"})
|
||||
return w.merge(w.head, r3, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "picks a commit of #6 into the merge of #5",
|
||||
},
|
||||
{
|
||||
name: "an extra commit", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
r3 := w.commit(r2, "fix up the port", map[string]string{"app/five.txt": "5, tidied\n"})
|
||||
return w.merge(w.head, r3, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "ends in no (cherry picked from commit",
|
||||
},
|
||||
{
|
||||
name: "a file #5 does not touch", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, map[string]string{"app/five.txt": "5\n", "app/smuggled.txt": "x\n"})
|
||||
return w.merge(w.head, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: `touches "app/smuggled.txt"`,
|
||||
},
|
||||
{
|
||||
name: "a pick without its -x line", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.commit(w.head, "port", resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.merge(w.head, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "ends in no (cherry picked from commit",
|
||||
},
|
||||
{
|
||||
name: "an abbreviated -x line", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1[:12], resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.merge(w.head, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "ends in no (cherry picked from commit",
|
||||
},
|
||||
{
|
||||
name: "an -x line in the body, not in the trailers", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.commit(w.head, "port\n\n(cherry picked from commit "+w.c1+")\n\nMore to say.", resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.merge(w.head, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "ends in no (cherry picked from commit",
|
||||
},
|
||||
{
|
||||
name: "both commits in one pick", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
both := w.commit(w.head, "port\n\n(cherry picked from commit "+w.c1+")\n(cherry picked from commit "+w.c2+")", map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n", "app/five.txt": "5\n"})
|
||||
return w.merge(w.head, both, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "claims more than one commit of #5",
|
||||
},
|
||||
{
|
||||
name: "the commits in the wrong order", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r2 := w.pick(w.head, w.c2, resolved2)
|
||||
r1 := w.pick(r2, w.c1, resolved1)
|
||||
return w.merge(w.head, r1, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "out of the order of #5's commits",
|
||||
},
|
||||
{
|
||||
name: "one commit picked twice", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, resolved1)
|
||||
again := w.pick(r1, w.c1, map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:2\n"})
|
||||
return w.merge(w.head, again, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "out of the order of #5's commits",
|
||||
},
|
||||
{
|
||||
name: "picks that do not start on the batch head", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head+"~1", w.c1, resolved1)
|
||||
r2 := w.pick(r1, w.c2, resolved2)
|
||||
return w.joined(w.head, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "is not on a straight line of picks",
|
||||
},
|
||||
{
|
||||
name: "picks that change nothing", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
r1 := w.pick(w.head, w.c1, nil)
|
||||
r2 := w.pick(r1, w.c2, nil)
|
||||
return w.merge(w.head, r2, batchMessage(w.mc5))
|
||||
},
|
||||
refusal: "changes nothing",
|
||||
},
|
||||
{
|
||||
name: "a sound fold of another original", original: 5,
|
||||
make: func(w *foldWorld) string {
|
||||
p := w.pick(w.head, w.c6, map[string]string{"app/six.txt": "6\n"})
|
||||
return w.merge(w.head, p, "fix(app): six (#6)\n\n"+batchTrailer+": "+w.mc6)
|
||||
},
|
||||
refusal: "carries #6, not #5",
|
||||
},
|
||||
{
|
||||
name: "one pick of a squash-merged original", original: 8,
|
||||
make: func(w *foldWorld) string {
|
||||
return w.pick(w.head, w.sq8, map[string]string{"app/eight.txt": "8\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "a squash-merged original as a merge", original: 8,
|
||||
make: func(w *foldWorld) string {
|
||||
p := w.pick(w.head, w.sq8, map[string]string{"app/eight.txt": "8\n"})
|
||||
return w.merge(w.head, p, "fix(app): eight (#8)\n\n"+batchTrailer+": "+w.sq8)
|
||||
},
|
||||
refusal: "has 2 parents; #8 merged as one commit",
|
||||
},
|
||||
{
|
||||
name: "an original the batch does not take", original: 7,
|
||||
make: func(w *foldWorld) string { return w.commit(w.head, "port", map[string]string{"app/seven.txt": "7\n"}) },
|
||||
refusal: "rule 2",
|
||||
},
|
||||
{
|
||||
name: "an original that never merged here", original: 4242,
|
||||
make: func(w *foldWorld) string { return w.commit(w.head, "port", map[string]string{"app/x.txt": "x\n"}) },
|
||||
refusal: "did not merge into main",
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
w := newFoldWorld(t)
|
||||
sha := tc.make(w)
|
||||
code, stdout, stderr := w.f.exec(w.gh, "fold", "--line", "release-1.1", "--original", strconv.Itoa(tc.original), "--commit", sha)
|
||||
switch {
|
||||
case tc.refusal == "" && (code != 0 || strings.TrimSpace(stdout) != "ok"):
|
||||
t.Errorf("fold = %d, %q, %s; want ok", code, stdout, stderr)
|
||||
case tc.refusal != "" && (code != 1 || !strings.Contains(stdout, tc.refusal)):
|
||||
t.Errorf("fold = %d, %q, %s; want a refusal with %q", code, stdout, stderr, tc.refusal)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("no open batch pull request", func(t *testing.T) {
|
||||
w := newFoldWorld(t)
|
||||
code, stdout, _ := w.f.exec(&fakeGH{}, "fold", "--line", "release-1.1", "--original", "5", "--commit", w.resolution())
|
||||
if code != 1 || !strings.Contains(stdout, "no batch pull request is open") {
|
||||
t.Errorf("fold = %d, %q; want a refusal", code, stdout)
|
||||
}
|
||||
})
|
||||
t.Run("already carried by the batch", func(t *testing.T) {
|
||||
w := newFoldWorld(t)
|
||||
folded := w.resolution()
|
||||
w.f.git("push", "-q", "origin", folded+":refs/heads/"+batchBranch1)
|
||||
w.gh.prs[0].Head.SHA = folded
|
||||
w.head = folded
|
||||
code, stdout, _ := w.f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", w.resolution())
|
||||
if code != 1 || !strings.Contains(stdout, "already carries #5") {
|
||||
t.Errorf("fold = %d, %q; want a refusal", code, stdout)
|
||||
}
|
||||
})
|
||||
t.Run("never commits", func(t *testing.T) {
|
||||
w := newFoldWorld(t)
|
||||
sha := w.resolution()
|
||||
before := w.f.git("for-each-ref")
|
||||
w.f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", sha)
|
||||
if after := w.f.git("for-each-ref"); after != before || w.f.remoteHead(batchBranch1) != w.head {
|
||||
t.Errorf("fold moved a ref:\nbefore:\n%s\nafter:\n%s", before, after)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// A resolution of an original a maintainer dropped from the line is refused:
|
||||
// the drop is the decision that it does not go there.
|
||||
func TestFoldRefusesADroppedOriginal(t *testing.T) {
|
||||
w := newFoldWorld(t)
|
||||
sha := strings.Repeat("a", 40)
|
||||
rec := strings.Replace(v2(""), `"dropped":[{"original":8,"merge":"`+sha+`"`, `"dropped":[{"original":5,"merge":"`+w.mc5+`"`, 1)
|
||||
rec = strings.Replace(rec, `"deferred":[{"original":5,"commit":"`+sha+`","files":["app/five.txt"]},`, `"deferred":[`, 1)
|
||||
rec = strings.Replace(rec, `"stale":[9]`, `"stale":[]`, 1)
|
||||
w.gh.prs[0].Body = "text\n\n" + deferredFence + "\n" + rec + "\n```\n"
|
||||
if _, err := parseRecord(w.gh.prs[0].Body, notifyLine); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
code, stdout, stderr := w.f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", w.resolution())
|
||||
if code != 1 || !strings.Contains(stdout, "#5 is not one the batch takes: a maintainer's drop, dropped on #90: needs #4") {
|
||||
t.Errorf("fold = %d, %q, %s; want a refusal naming the drop", code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,276 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
var shaRE = regexp.MustCompile(`^[0-9a-f]{40}$`)
|
||||
|
||||
// commandEnv pins the locale, because telling a conflict from an empty pick
|
||||
// reads git's messages, and keeps git from opening an editor.
|
||||
var commandEnv = []string{"LC_ALL=C", "GIT_EDITOR=true"}
|
||||
|
||||
func command(name string, args ...string) *exec.Cmd {
|
||||
cmd := exec.Command(name, args...)
|
||||
cmd.Env = append(os.Environ(), commandEnv...)
|
||||
return cmd
|
||||
}
|
||||
|
||||
func capture(name string, args ...string) (string, error) {
|
||||
out, err := command(name, args...).Output()
|
||||
if err != nil {
|
||||
if ee, ok := err.(*exec.ExitError); ok && len(ee.Stderr) > 0 {
|
||||
return "", fmt.Errorf("%s %s: %w: %s", name, strings.Join(args, " "), err, strings.TrimSpace(string(ee.Stderr)))
|
||||
}
|
||||
return "", fmt.Errorf("%s %s: %w", name, strings.Join(args, " "), err)
|
||||
}
|
||||
return string(out), nil
|
||||
}
|
||||
|
||||
// git runs a git command whose failure means the batch cannot be trusted.
|
||||
func git(args ...string) (string, error) { return capture("git", args...) }
|
||||
|
||||
func gitInput(input string, args ...string) (string, error) {
|
||||
cmd := command("git", args...)
|
||||
cmd.Stdin = strings.NewReader(input)
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("git %s: %w", strings.Join(args, " "), err)
|
||||
}
|
||||
return string(out), nil
|
||||
}
|
||||
|
||||
// gitOK runs git for a command whose exit 1 is the answer no; any other
|
||||
// failure is an error.
|
||||
func gitOK(args ...string) (bool, error) {
|
||||
_, stderr, code, err := gitStatus(args...)
|
||||
switch {
|
||||
case err != nil:
|
||||
return false, err
|
||||
case code != 0 && code != 1:
|
||||
return false, fmt.Errorf("git %s: exit %d: %s", strings.Join(args, " "), code, strings.TrimSpace(stderr))
|
||||
}
|
||||
return code == 0, nil
|
||||
}
|
||||
|
||||
// gitStatus runs git for a command whose non-zero exit is an answer rather
|
||||
// than a failure: a conflicting pick, a grep that matched nothing.
|
||||
func gitStatus(args ...string) (stdout, stderr string, code int, err error) {
|
||||
var out, errOut bytes.Buffer
|
||||
cmd := command("git", args...)
|
||||
cmd.Stdout, cmd.Stderr = &out, &errOut
|
||||
err = cmd.Run()
|
||||
var ee *exec.ExitError
|
||||
if errors.As(err, &ee) {
|
||||
return out.String(), errOut.String(), ee.ExitCode(), nil
|
||||
}
|
||||
if err != nil {
|
||||
return "", "", -1, fmt.Errorf("git %s: %w", strings.Join(args, " "), err)
|
||||
}
|
||||
return out.String(), errOut.String(), 0, nil
|
||||
}
|
||||
|
||||
func checkSHA(s string) error {
|
||||
if !shaRE.MatchString(s) {
|
||||
return fmt.Errorf("%q is not a full commit SHA", s)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkBranch refuses a name git could read as an option or would rewrite.
|
||||
// check-ref-format --branch expands @{-N} into whatever branch that was, so
|
||||
// its answer has to be the name itself.
|
||||
func checkBranch(name string) error {
|
||||
if name == "" || strings.HasPrefix(name, "-") || strings.ContainsFunc(name, unicode.IsControl) {
|
||||
return fmt.Errorf("%q is not a branch name", name)
|
||||
}
|
||||
out, _, code, err := gitStatus("check-ref-format", "--branch", name)
|
||||
switch {
|
||||
case err != nil:
|
||||
return err
|
||||
case code != 0 || strings.TrimSuffix(out, "\n") != name:
|
||||
return fmt.Errorf("%q is not a branch name", name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolve pins a ref to the commit it names now, so every later read sees the
|
||||
// same commit however the ref moves.
|
||||
func resolve(ref string) (string, error) {
|
||||
out, err := git("rev-parse", "--verify", "--quiet", "--end-of-options", ref+"^{commit}")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("cannot resolve %s: %w", ref, err)
|
||||
}
|
||||
return strings.TrimSpace(out), nil
|
||||
}
|
||||
|
||||
// lookup is the commit rev names, or "" when this repository has none: a ref
|
||||
// that does not exist, or an object it does not have. cat-file -e exits 128
|
||||
// for a missing object, as it does when git fails, so it cannot tell them.
|
||||
func lookup(rev string) (string, error) {
|
||||
out, stderr, code, err := gitStatus("rev-parse", "--verify", "--quiet", "--end-of-options", rev+"^{commit}")
|
||||
switch {
|
||||
case err != nil:
|
||||
return "", err
|
||||
case code == 1:
|
||||
return "", nil
|
||||
case code != 0:
|
||||
return "", fmt.Errorf("git rev-parse %s: exit %d: %s", rev, code, strings.TrimSpace(stderr))
|
||||
}
|
||||
return strings.TrimSpace(out), nil
|
||||
}
|
||||
|
||||
func isAncestor(a, b string) (bool, error) {
|
||||
return gitOK("merge-base", "--is-ancestor", "--end-of-options", a, b)
|
||||
}
|
||||
|
||||
// splitNUL splits -z output, dropping the empty field after the last NUL.
|
||||
func splitNUL(out string) []string {
|
||||
var fields []string
|
||||
for f := range strings.SplitSeq(out, "\x00") {
|
||||
if f != "" {
|
||||
fields = append(fields, f)
|
||||
}
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
// revList lists commits with rev-list, whose output is object names and
|
||||
// nothing a commit message could add to; each one is checked to be one.
|
||||
func revList(args ...string) ([]string, error) {
|
||||
out, err := git(append([]string{"rev-list"}, args...)...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
shas := strings.Fields(out)
|
||||
for _, sha := range shas {
|
||||
if err := checkSHA(sha); err != nil {
|
||||
return nil, fmt.Errorf("rev-list: %w", err)
|
||||
}
|
||||
}
|
||||
return shas, nil
|
||||
}
|
||||
|
||||
// commitObject is what the batch reads of a commit: its message, its parents,
|
||||
// and when it was committed, which for a merge GitHub made is when it merged.
|
||||
type commitObject struct {
|
||||
message string
|
||||
parents []string
|
||||
committed time.Time
|
||||
}
|
||||
|
||||
// readObjects reads objects of one type through one cat-file --batch, whose
|
||||
// output gives each object's length before its content: nothing inside one, a
|
||||
// NUL or a line shaped like a header included, can pass for the start of the
|
||||
// next.
|
||||
func readObjects(ids []string, kind string) (map[string][]byte, error) {
|
||||
objects := map[string][]byte{}
|
||||
if len(ids) == 0 {
|
||||
return objects, nil
|
||||
}
|
||||
cmd := command("git", "cat-file", "--batch")
|
||||
cmd.Stdin = strings.NewReader(strings.Join(ids, "\n") + "\n")
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("git cat-file --batch: %w", err)
|
||||
}
|
||||
r := bufio.NewReader(bytes.NewReader(out))
|
||||
for _, want := range ids {
|
||||
header, err := r.ReadString('\n')
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("git cat-file --batch: no answer for %s", want)
|
||||
}
|
||||
f := strings.Fields(header)
|
||||
if len(f) != 3 || f[0] != want || f[1] != kind {
|
||||
return nil, fmt.Errorf("git cat-file --batch: %q is not %s %s", strings.TrimSpace(header), kind, want)
|
||||
}
|
||||
size, err := strconv.Atoi(f[2])
|
||||
if err != nil || size < 0 {
|
||||
return nil, fmt.Errorf("git cat-file --batch: bad size in %q", strings.TrimSpace(header))
|
||||
}
|
||||
body := make([]byte, size+1)
|
||||
if _, err := io.ReadFull(r, body); err != nil || body[size] != '\n' {
|
||||
return nil, fmt.Errorf("git cat-file --batch: short object %s", want)
|
||||
}
|
||||
objects[want] = body[:size]
|
||||
}
|
||||
return objects, nil
|
||||
}
|
||||
|
||||
// commitMessages reads commits through readObjects. A message ends at its
|
||||
// first NUL, where git log ends it, so nothing hidden from a reader of the log
|
||||
// counts here.
|
||||
func commitMessages(shas []string) (map[string]commitObject, error) {
|
||||
raw, err := readObjects(shas, "commit")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
msgs := map[string]commitObject{}
|
||||
for sha, data := range raw {
|
||||
headers, msg, _ := strings.Cut(string(data), "\n\n")
|
||||
msg, _, _ = strings.Cut(msg, "\x00")
|
||||
committed, err := committerTime(headers)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("commit %s: %w", sha, err)
|
||||
}
|
||||
var parents []string
|
||||
for ln := range strings.Lines(headers) {
|
||||
if p, ok := strings.CutPrefix(strings.TrimRight(ln, "\n"), "parent "); ok {
|
||||
if err := checkSHA(p); err != nil {
|
||||
return nil, fmt.Errorf("commit %s: %w", sha, err)
|
||||
}
|
||||
parents = append(parents, p)
|
||||
}
|
||||
}
|
||||
msgs[sha] = commitObject{message: msg, parents: parents, committed: committed}
|
||||
}
|
||||
return msgs, nil
|
||||
}
|
||||
|
||||
// committerTime reads the committer line of a commit's headers:
|
||||
// "committer Name <email> <unix seconds> <zone>".
|
||||
func committerTime(headers string) (time.Time, error) {
|
||||
for ln := range strings.Lines(headers) {
|
||||
rest, ok := strings.CutPrefix(strings.TrimRight(ln, "\n"), "committer ")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
f := strings.Fields(rest[strings.LastIndexByte(rest, '>')+1:])
|
||||
if len(f) != 2 {
|
||||
break
|
||||
}
|
||||
secs, err := strconv.ParseInt(f[0], 10, 64)
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
return time.Unix(secs, 0).UTC(), nil
|
||||
}
|
||||
return time.Time{}, fmt.Errorf("no readable committer line")
|
||||
}
|
||||
@@ -0,0 +1,393 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// openPullsCap is the page size of the open pull request listing. The REST
|
||||
// listing stops at it in silence, so a full page is an error, not an answer.
|
||||
const openPullsCap = 100
|
||||
|
||||
// recentPullsPage is the page size of the listing of every pull request
|
||||
// against a line, newest first, and recentPullsPages how many pages of it a
|
||||
// search for the line's record reads.
|
||||
const (
|
||||
recentPullsPage = 100
|
||||
recentPullsPages = 10
|
||||
)
|
||||
|
||||
// searchResultCap is the most results GitHub search returns for one query,
|
||||
// however far it is paginated. gh pr list answers a --label filter through
|
||||
// search, so a listing that long has to be taken as cut short.
|
||||
const searchResultCap = 1000
|
||||
|
||||
// The labels count on a pull request merged into main, the original, and on
|
||||
// no batch pull request.
|
||||
const (
|
||||
labelInclude = "backport/include"
|
||||
labelSkip = "backport/skip"
|
||||
)
|
||||
|
||||
var (
|
||||
batchBranchRE = regexp.MustCompile(`^backport-batch/(release-\d+\.\d+)-(\d{8})$`)
|
||||
loginRE = regexp.MustCompile(`^[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?$`)
|
||||
botLoginRE = regexp.MustCompile(`^[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?\[bot\]$`)
|
||||
pullURLRE = regexp.MustCompile(`^https://[^/\s]+/[^/\s]+/[^/\s]+/pull/([1-9][0-9]{0,8})$`)
|
||||
)
|
||||
|
||||
// batchBotEnv names the GitHub App bot the scheduled batch opens its pull
|
||||
// requests and comments as, <app slug>[bot]. The workflow sets it from the
|
||||
// token it mints. Unset, it is defaultBatchBot, so that a maintainer's run by
|
||||
// hand continues the scheduled batch instead of opening a second one; set and
|
||||
// empty, no bot is trusted.
|
||||
const (
|
||||
batchBotEnv = "BACKPORT_BATCH_BOT"
|
||||
defaultBatchBot = "cozystack-ci[bot]"
|
||||
)
|
||||
|
||||
// batchBotLogin is the bot to trust given lookup, os.LookupEnv outside tests.
|
||||
func batchBotLogin(lookup func(string) (string, bool)) string {
|
||||
if v, ok := lookup(batchBotEnv); ok {
|
||||
return v
|
||||
}
|
||||
return defaultBatchBot
|
||||
}
|
||||
|
||||
// pullRequest is the REST view of a pull request: unlike gh pr list, it
|
||||
// carries the author's association with the repository.
|
||||
type pullRequest struct {
|
||||
Number int `json:"number"`
|
||||
HTMLURL string `json:"html_url"`
|
||||
Body string `json:"body"`
|
||||
State string `json:"state"`
|
||||
MergedAt *string `json:"merged_at"`
|
||||
AuthorAssociation string `json:"author_association"`
|
||||
User struct {
|
||||
Login string `json:"login"`
|
||||
Type string `json:"type"`
|
||||
} `json:"user"`
|
||||
Head struct {
|
||||
Ref string `json:"ref"`
|
||||
SHA string `json:"sha"`
|
||||
Repo *struct {
|
||||
FullName string `json:"full_name"`
|
||||
} `json:"repo"`
|
||||
} `json:"head"`
|
||||
Base struct {
|
||||
Ref string `json:"ref"`
|
||||
Repo struct {
|
||||
FullName string `json:"full_name"`
|
||||
} `json:"repo"`
|
||||
} `json:"base"`
|
||||
}
|
||||
|
||||
// issueComment is the REST view of a comment on a pull request.
|
||||
type issueComment struct {
|
||||
ID int64 `json:"id"`
|
||||
Body string `json:"body"`
|
||||
User struct {
|
||||
Login string `json:"login"`
|
||||
Type string `json:"type"`
|
||||
} `json:"user"`
|
||||
}
|
||||
|
||||
// github is the part of GitHub the tool talks to: gh outside tests.
|
||||
type github interface {
|
||||
openPulls(base string) ([]pullRequest, error)
|
||||
// recentPulls is one page of every pull request against base, open,
|
||||
// merged or closed, newest first.
|
||||
recentPulls(base string, page int) ([]pullRequest, error)
|
||||
// permission is login's permission on the repository as GitHub's
|
||||
// legacy field gives it: admin, write, read or none.
|
||||
permission(login string) (string, error)
|
||||
// labelled lists the pull requests merged into main that carry label.
|
||||
labelled(label string) ([]int, error)
|
||||
// pull reads one pull request as it is now.
|
||||
pull(number int) (pullRequest, error)
|
||||
// createPull opens the draft and returns its number and URL.
|
||||
createPull(base, head, title, bodyFile string) (int, string, error)
|
||||
editPull(number int, title, bodyFile string) error
|
||||
closePull(number int) error
|
||||
comments(number int) ([]issueComment, error)
|
||||
createComment(number int, bodyFile string) error
|
||||
editComment(id int64, bodyFile string) error
|
||||
// batchBot is the bot the batch writes as, from batchBotEnv, or "".
|
||||
batchBot() string
|
||||
}
|
||||
|
||||
type ghCLI struct{ bot string }
|
||||
|
||||
func (g ghCLI) batchBot() string { return g.bot }
|
||||
|
||||
func (ghCLI) openPulls(base string) ([]pullRequest, error) {
|
||||
out, err := capture("gh", "api", "-X", "GET", "repos/{owner}/{repo}/pulls",
|
||||
"-f", "state=open", "-f", "base="+base, "-f", "per_page="+strconv.Itoa(openPullsCap))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var prs []pullRequest
|
||||
if err := json.Unmarshal([]byte(out), &prs); err != nil {
|
||||
return nil, fmt.Errorf("reading the open pull requests on %s: %w", base, err)
|
||||
}
|
||||
if len(prs) >= openPullsCap {
|
||||
return nil, fmt.Errorf("the open pull requests on %s fill a page of %d, so the list may be truncated", base, openPullsCap)
|
||||
}
|
||||
return prs, nil
|
||||
}
|
||||
|
||||
func (ghCLI) recentPulls(base string, page int) ([]pullRequest, error) {
|
||||
out, err := capture("gh", "api", "-X", "GET", "repos/{owner}/{repo}/pulls",
|
||||
"-f", "state=all", "-f", "base="+base, "-f", "sort=created", "-f", "direction=desc",
|
||||
"-f", "per_page="+strconv.Itoa(recentPullsPage), "-f", "page="+strconv.Itoa(page))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var prs []pullRequest
|
||||
if err := json.Unmarshal([]byte(out), &prs); err != nil {
|
||||
return nil, fmt.Errorf("reading the pull requests against %s: %w", base, err)
|
||||
}
|
||||
return prs, nil
|
||||
}
|
||||
|
||||
// permCache asks GitHub for a login's permission once per command: the open
|
||||
// batch's author is asked when it is chosen and again when its record is.
|
||||
type permCache struct {
|
||||
github
|
||||
perms map[string]string
|
||||
}
|
||||
|
||||
func (c *permCache) permission(login string) (string, error) {
|
||||
if p, ok := c.perms[login]; ok {
|
||||
return p, nil
|
||||
}
|
||||
p, err := c.github.permission(login)
|
||||
if err == nil {
|
||||
c.perms[login] = p
|
||||
}
|
||||
return p, err
|
||||
}
|
||||
|
||||
func (ghCLI) permission(login string) (string, error) {
|
||||
if !loginRE.MatchString(login) {
|
||||
return "", fmt.Errorf("%q is not a GitHub login", login)
|
||||
}
|
||||
out, err := capture("gh", "api", "repos/{owner}/{repo}/collaborators/"+login+"/permission")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
var answer struct {
|
||||
Permission string `json:"permission"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &answer); err != nil {
|
||||
return "", fmt.Errorf("reading %s's permission: %w", login, err)
|
||||
}
|
||||
return answer.Permission, nil
|
||||
}
|
||||
|
||||
func (ghCLI) labelled(label string) ([]int, error) {
|
||||
out, err := capture("gh", "pr", "list", "--base", "main", "--state", "merged", "--label", label,
|
||||
"--limit", strconv.Itoa(searchResultCap), "--json", "number")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var prs []struct {
|
||||
Number int `json:"number"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &prs); err != nil {
|
||||
return nil, fmt.Errorf("reading the pull requests labelled %s: %w", label, err)
|
||||
}
|
||||
if len(prs) >= searchResultCap {
|
||||
return nil, fmt.Errorf("%d pull requests carry %s, as many as GitHub search returns, so the list may be cut short", len(prs), label)
|
||||
}
|
||||
numbers := make([]int, 0, len(prs))
|
||||
for _, pr := range prs {
|
||||
numbers = append(numbers, pr.Number)
|
||||
}
|
||||
return numbers, nil
|
||||
}
|
||||
|
||||
func (ghCLI) pull(number int) (pullRequest, error) {
|
||||
var pr pullRequest
|
||||
out, err := capture("gh", "api", "repos/{owner}/{repo}/pulls/"+strconv.Itoa(number))
|
||||
if err != nil {
|
||||
return pr, err
|
||||
}
|
||||
if err := json.Unmarshal([]byte(out), &pr); err != nil {
|
||||
return pr, fmt.Errorf("reading pull request #%d: %w", number, err)
|
||||
}
|
||||
if pr.Number != number {
|
||||
return pr, fmt.Errorf("asked for pull request #%d, GitHub answered with #%d", number, pr.Number)
|
||||
}
|
||||
return pr, nil
|
||||
}
|
||||
|
||||
func (ghCLI) createPull(base, head, title, bodyFile string) (int, string, error) {
|
||||
out, err := capture("gh", "pr", "create", "--draft", "--base", base, "--head", head, "--title", title, "--body-file", bodyFile)
|
||||
if err != nil {
|
||||
return 0, "", err
|
||||
}
|
||||
url := strings.TrimSpace(out)
|
||||
m := pullURLRE.FindStringSubmatch(url)
|
||||
if m == nil {
|
||||
return 0, "", fmt.Errorf("gh pr create printed %q, not the URL of a pull request", url)
|
||||
}
|
||||
n, err := strconv.Atoi(m[1])
|
||||
return n, url, err
|
||||
}
|
||||
|
||||
func (ghCLI) editPull(number int, title, bodyFile string) error {
|
||||
_, err := capture("gh", "pr", "edit", strconv.Itoa(number), "--title", title, "--body-file", bodyFile)
|
||||
return err
|
||||
}
|
||||
|
||||
func (ghCLI) closePull(number int) error {
|
||||
_, err := capture("gh", "pr", "close", strconv.Itoa(number))
|
||||
return err
|
||||
}
|
||||
|
||||
func (ghCLI) comments(number int) ([]issueComment, error) {
|
||||
out, err := capture("gh", "api", "--paginate", "--slurp", "repos/{owner}/{repo}/issues/"+strconv.Itoa(number)+"/comments")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var pages [][]issueComment
|
||||
if err := json.Unmarshal([]byte(out), &pages); err != nil {
|
||||
return nil, fmt.Errorf("reading the comments on #%d: %w", number, err)
|
||||
}
|
||||
var all []issueComment
|
||||
for _, page := range pages {
|
||||
all = append(all, page...)
|
||||
}
|
||||
return all, nil
|
||||
}
|
||||
|
||||
func (ghCLI) createComment(number int, bodyFile string) error {
|
||||
_, err := capture("gh", "api", "--method", "POST", "repos/{owner}/{repo}/issues/"+strconv.Itoa(number)+"/comments", "-F", "body=@"+bodyFile)
|
||||
return err
|
||||
}
|
||||
|
||||
func (ghCLI) editComment(id int64, bodyFile string) error {
|
||||
_, err := capture("gh", "api", "--method", "PATCH", "repos/{owner}/{repo}/issues/comments/"+strconv.FormatInt(id, 10), "-F", "body=@"+bodyFile)
|
||||
return err
|
||||
}
|
||||
|
||||
// trusted says why a pull request's author may not speak for the batch, or ""
|
||||
// when they may.
|
||||
func trusted(gh github, pr pullRequest) (string, error) {
|
||||
why, err := trustedAuthor(gh, pr.User.Login, pr.User.Type)
|
||||
if why != "" {
|
||||
why = "opened by " + why
|
||||
}
|
||||
return why, err
|
||||
}
|
||||
|
||||
// trustedAuthor says why an author may not speak for the batch, or "" when
|
||||
// they may: the bot the batch runs as, by its exact login, or a person whose
|
||||
// permission on the repository is write or admin. Any other bot is refused,
|
||||
// GitHub Actions' own included: anything with a workflow's token can write as
|
||||
// it. The author association is not asked: an organization member or an
|
||||
// invited collaborator can hold read access only, and the legacy permission
|
||||
// folds maintain into write and triage into read. A permission GitHub does not
|
||||
// answer is an error, not a refusal: taken for one, it would hide the open
|
||||
// batch and its comment, and the run would open a second of each.
|
||||
func trustedAuthor(gh github, login, kind string) (string, error) {
|
||||
switch {
|
||||
case kind == "Bot" && gh.batchBot() != "" && login == gh.batchBot():
|
||||
return "", nil
|
||||
case kind != "User" || !loginRE.MatchString(login):
|
||||
return fmt.Sprintf("%q, which is neither a person nor the batch's bot (%s)", login, batchBotEnv), nil
|
||||
}
|
||||
perm, err := gh.permission(login)
|
||||
switch {
|
||||
case err != nil:
|
||||
return "", err
|
||||
case perm != "write" && perm != "admin":
|
||||
return fmt.Sprintf("%s, whose permission is %q", login, perm), nil
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// batchPull picks the open batch pull request for l out of prs, or nil.
|
||||
//
|
||||
// A pull request counts only when its head is a batch branch for l in this
|
||||
// repository, not a fork, and a trusted author opened it: anyone can open a
|
||||
// pull request from a branch named like one, and its -x lines and its list of
|
||||
// deferred conflicts are what check believes. Look-alikes come back as
|
||||
// warnings.
|
||||
func batchPull(gh github, prs []pullRequest, l line) (*pullRequest, []string, error) {
|
||||
var found []*pullRequest
|
||||
var warnings []string
|
||||
for i := range prs {
|
||||
pr := &prs[i]
|
||||
m := batchBranchRE.FindStringSubmatch(pr.Head.Ref)
|
||||
if m == nil || m[1] != l.branch() || pr.Base.Ref != l.branch() {
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case pr.Head.Repo == nil || pr.Head.Repo.FullName != pr.Base.Repo.FullName:
|
||||
warnings = append(warnings, fmt.Sprintf("ignoring #%d: its batch branch is in a fork", pr.Number))
|
||||
case checkSHA(pr.Head.SHA) != nil:
|
||||
warnings = append(warnings, fmt.Sprintf("ignoring #%d: GitHub reports no head commit for it", pr.Number))
|
||||
default:
|
||||
why, err := trusted(gh, *pr)
|
||||
if err != nil {
|
||||
return nil, warnings, fmt.Errorf("batch pull request #%d: %w", pr.Number, err)
|
||||
}
|
||||
if why != "" {
|
||||
warnings = append(warnings, fmt.Sprintf("ignoring #%d: %s", pr.Number, why))
|
||||
continue
|
||||
}
|
||||
found = append(found, pr)
|
||||
}
|
||||
}
|
||||
switch len(found) {
|
||||
case 0:
|
||||
return nil, warnings, nil
|
||||
case 1:
|
||||
return found[0], warnings, nil
|
||||
}
|
||||
return nil, warnings, fmt.Errorf("more than one open batch pull request on %s (#%d, #%d); close all but one", l.branch(), found[0].Number, found[1].Number)
|
||||
}
|
||||
|
||||
// batchHead is the head of the batch pull request as fetched here, refused
|
||||
// unless it is the commit GitHub reports for the pull request.
|
||||
func batchHead(pr *pullRequest) (string, error) {
|
||||
if err := checkBranch(pr.Head.Ref); err != nil {
|
||||
return "", err
|
||||
}
|
||||
ref := "refs/remotes/" + remote + "/" + pr.Head.Ref
|
||||
got, err := lookup(ref)
|
||||
if err != nil || got == pr.Head.SHA {
|
||||
return got, err
|
||||
}
|
||||
if _, err := git("fetch", "--quiet", "--end-of-options", remote, "+refs/heads/"+pr.Head.Ref+":"+ref); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if got, err = resolve(ref); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if got != pr.Head.SHA {
|
||||
return "", fmt.Errorf("batch pull request #%d: %s is at %.12s, not at %.12s as GitHub reports; run again", pr.Number, pr.Head.Ref, got, pr.Head.SHA)
|
||||
}
|
||||
return got, nil
|
||||
}
|
||||
@@ -0,0 +1,371 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A reference to a PR, optionally qualified by its repository, and a list of
|
||||
// them as prose writes one: "#1 and #2", "#1, #2 and #3", "#1, #2, and #3".
|
||||
const (
|
||||
prRef = `(?:[\w.-]+/[\w.-]+)?#\d+`
|
||||
prRefList = prRef + `(?:(?:\s*,\s*(?:and\s+)?|\s+and\s+)` + prRef + `)*`
|
||||
)
|
||||
|
||||
// A release line named in prose, bare or in backticks, as a whole word.
|
||||
const releaseLineName = `(?:\x60release-\d+\.\d+\x60|release-\d+\.\d+)(?:$|\s|\)|[.,;:](?:\s|$))`
|
||||
|
||||
var (
|
||||
// A hand backport that carries more than one change names all of them in
|
||||
// its "Backport of" phrase, either as a list or as the "to release-X.Y,
|
||||
// together with #N" this repository writes for a dependency pulled along.
|
||||
backportOfRE = regexp.MustCompile(`[Bb]ackport of\s+(` + prRefList + `)` +
|
||||
`(?:\s+to\s+\x60?release-\d+\.\d+\x60?,?\s+together with\s+(` + prRefList + `))?`)
|
||||
// What a list of references may be followed by for every item in it to be
|
||||
// an original. Anything else may be the list going on to say something
|
||||
// about its later items -- "#10, #20 is not included" -- so an
|
||||
// unterminated list keeps only its first item.
|
||||
prRefListEndRE = regexp.MustCompile(`^(?:[ \t]*(?:\r?\n|$)|\.(?:\s|$)|\s+to\s+` + releaseLineName + `)`)
|
||||
prRefRE = regexp.MustCompile(`(?:([\w.-]+/[\w.-]+))?#(\d+)`)
|
||||
xLineRE = regexp.MustCompile(`^\(cherry picked from commit ([0-9a-f]{40})\)$`)
|
||||
batchLineRE = regexp.MustCompile(`^` + batchTrailer + `: ([0-9a-f]{40})$`)
|
||||
trailerRE = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9-]*[ \t]*:[ \t]`)
|
||||
paragraphRE = regexp.MustCompile(`\n[ \t]*\n`)
|
||||
)
|
||||
|
||||
// batchTrailer names, in the merge commit a batch makes for an original, the
|
||||
// merge commit on main it carries.
|
||||
const batchTrailer = "Backported-from"
|
||||
|
||||
// trailerBlock is the last paragraph of a message when every line of it is a
|
||||
// trailer, a trailer's continuation or an -x line, which is where git writes
|
||||
// both; nil otherwise. The same lines anywhere else are prose, which anyone
|
||||
// can write. interpret-trailers does not report -x lines, so the block is read
|
||||
// here the way git finds it.
|
||||
func trailerBlock(message string) []string {
|
||||
var paras []string
|
||||
for _, p := range paragraphRE.Split(strings.ReplaceAll(message, "\r\n", "\n"), -1) {
|
||||
if strings.TrimSpace(p) != "" {
|
||||
paras = append(paras, p)
|
||||
}
|
||||
}
|
||||
if len(paras) < 2 {
|
||||
return nil
|
||||
}
|
||||
var block []string
|
||||
for _, ln := range strings.Split(paras[len(paras)-1], "\n") {
|
||||
switch t := strings.TrimSpace(ln); {
|
||||
case t == "":
|
||||
case xLineRE.MatchString(t):
|
||||
block = append(block, t)
|
||||
case strings.HasPrefix(ln, " ") || strings.HasPrefix(ln, "\t"):
|
||||
// Left out: only the trailer a continuation belongs to is read.
|
||||
case !trailerRE.MatchString(ln):
|
||||
return nil
|
||||
default:
|
||||
block = append(block, t)
|
||||
}
|
||||
}
|
||||
return block
|
||||
}
|
||||
|
||||
// trailerValues lists what the lines of a message's trailer block that re
|
||||
// matches name.
|
||||
func trailerValues(message string, re *regexp.Regexp) []string {
|
||||
var values []string
|
||||
for _, ln := range trailerBlock(message) {
|
||||
if m := re.FindStringSubmatch(ln); m != nil {
|
||||
values = append(values, m[1])
|
||||
}
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
// xSources reads every -x line of the trailer block, whoever wrote it; only
|
||||
// appendedSource reads the one git appended.
|
||||
func xSources(message string) []string { return trailerValues(message, xLineRE) }
|
||||
|
||||
// lastParagraph is the last paragraph of a message, line by line, or nil for
|
||||
// a message of one paragraph.
|
||||
func lastParagraph(message string) []string {
|
||||
var paras []string
|
||||
for _, p := range paragraphRE.Split(strings.ReplaceAll(message, "\r\n", "\n"), -1) {
|
||||
if strings.TrimSpace(p) != "" {
|
||||
paras = append(paras, p)
|
||||
}
|
||||
}
|
||||
if len(paras) < 2 {
|
||||
return nil
|
||||
}
|
||||
return strings.Split(paras[len(paras)-1], "\n")
|
||||
}
|
||||
|
||||
// appendedSource is the commit the -x line git appends when it picks names:
|
||||
// the last -x line of the message's last paragraph, where git writes it, with
|
||||
// nothing after it but trailers, such as the sign-off --signoff adds. The
|
||||
// picked commit's own message comes before it, so nothing a pull request's
|
||||
// author wrote can stand in for it. "" when there is none.
|
||||
func appendedSource(message string) string {
|
||||
lines := lastParagraph(message)
|
||||
for i := len(lines) - 1; i >= 0; i-- {
|
||||
t := strings.TrimSpace(lines[i])
|
||||
if m := xLineRE.FindStringSubmatch(t); m != nil {
|
||||
return m[1]
|
||||
}
|
||||
if t != "" && !strings.HasPrefix(lines[i], " ") && !strings.HasPrefix(lines[i], "\t") && !trailerRE.MatchString(lines[i]) {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// claimedSources lists what every -x line of a message's last paragraph
|
||||
// names, the appended one and any the picked commit's message already had.
|
||||
func claimedSources(message string) []string {
|
||||
var sources []string
|
||||
for _, ln := range lastParagraph(message) {
|
||||
if m := xLineRE.FindStringSubmatch(strings.TrimSpace(ln)); m != nil {
|
||||
sources = append(sources, m[1])
|
||||
}
|
||||
}
|
||||
return sources
|
||||
}
|
||||
|
||||
// batchedFrom lists the merge commits on main a batch merge says it carries.
|
||||
func batchedFrom(message string) []string { return trailerValues(message, batchLineRE) }
|
||||
|
||||
// bodyOrigins returns the PR numbers named in a message's "Backport of"
|
||||
// phrases, in order of appearance and without repeats. A reference qualified
|
||||
// with another repository is not a local number.
|
||||
func bodyOrigins(body, repo string) []int {
|
||||
var origins []int
|
||||
for _, m := range backportOfRE.FindAllStringSubmatchIndex(body, -1) {
|
||||
for g := 1; 2*g+1 < len(m); g++ {
|
||||
start, end := m[2*g], m[2*g+1]
|
||||
if start < 0 {
|
||||
continue
|
||||
}
|
||||
refs := prRefRE.FindAllStringSubmatch(body[start:end], -1)
|
||||
if !prRefListEndRE.MatchString(body[end:]) {
|
||||
refs = refs[:1]
|
||||
}
|
||||
for _, ref := range refs {
|
||||
if ref[1] != "" && !strings.EqualFold(ref[1], repo) {
|
||||
continue
|
||||
}
|
||||
n, _ := strconv.Atoi(ref[2])
|
||||
if n > 0 && !slices.Contains(origins, n) {
|
||||
origins = append(origins, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return origins
|
||||
}
|
||||
|
||||
// pickRef is an -x reference: the commit it names, and the commit carrying it.
|
||||
type pickRef struct{ source, commit string }
|
||||
|
||||
// history is what a branch carries beyond a base it shares with main.
|
||||
//
|
||||
// Only the commits outside the base supply evidence: shared history can name
|
||||
// an original without carrying it on this branch. What a commit's text says
|
||||
// counts only for commits from before the batch-only day: a picked commit
|
||||
// keeps its message and a batch merge carries the original's, so after that
|
||||
// day a pull request's author could write any of it. From then on only the
|
||||
// -x line git appends to a pick counts, which names the commit picked.
|
||||
type history struct {
|
||||
head string
|
||||
reachable map[string]bool
|
||||
// appended maps a commit to a pick on the branch whose appended -x line
|
||||
// names it.
|
||||
appended map[string]string
|
||||
// What text says, from commits before the batch-only day: every -x line
|
||||
// of a trailer block, the Backported-from trailers and "Backport of".
|
||||
picks []pickRef
|
||||
batches map[string]string
|
||||
backports map[int]string
|
||||
}
|
||||
|
||||
func readHistory(head, base string, textUntil time.Time) (*history, error) {
|
||||
h, err := readPicks(head, base, textUntil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
all, err := revList("--end-of-options", head)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, oid := range all {
|
||||
h.reachable[oid] = true
|
||||
}
|
||||
return h, nil
|
||||
}
|
||||
|
||||
// readPicks reads only the messages of head's commits outside base, which is
|
||||
// all a batch branch is asked about; textUntil is the batch-only day.
|
||||
func readPicks(head, base string, textUntil time.Time) (*history, error) {
|
||||
h := &history{head: head, reachable: map[string]bool{}, appended: map[string]string{}, batches: map[string]string{}, backports: map[int]string{}}
|
||||
oids, err := revList("--end-of-options", head, "^"+base)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
msgs, err := commitMessages(oids)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, oid := range oids {
|
||||
msg := msgs[oid].message
|
||||
if len(msgs[oid].parents) == 1 {
|
||||
if source := appendedSource(msg); source != "" {
|
||||
if _, ok := h.appended[source]; !ok {
|
||||
h.appended[source] = oid
|
||||
}
|
||||
}
|
||||
}
|
||||
if !msgs[oid].committed.Before(textUntil) {
|
||||
continue
|
||||
}
|
||||
for _, source := range xSources(msg) {
|
||||
h.picks = append(h.picks, pickRef{source: source, commit: oid})
|
||||
}
|
||||
for _, merge := range batchedFrom(msg) {
|
||||
if _, ok := h.batches[merge]; !ok {
|
||||
h.batches[merge] = oid
|
||||
}
|
||||
}
|
||||
for _, n := range bodyOrigins(msg, upstreamRepo) {
|
||||
if _, ok := h.backports[n]; !ok {
|
||||
h.backports[n] = oid
|
||||
}
|
||||
}
|
||||
}
|
||||
return h, nil
|
||||
}
|
||||
|
||||
// pickOf returns the commit whose -x line names source, or "".
|
||||
func (h *history) pickOf(source string) string {
|
||||
for _, r := range h.picks {
|
||||
if r.source == source {
|
||||
return r.commit
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// found returns the commit that carries an original on the branch and how,
|
||||
// or "" when none does. A pick counts when the -x line git appended to it
|
||||
// names the original's merge commit, as for a squash or a pick with -m 1, or
|
||||
// when such lines name every commit of its own, as a batch merge's picks do.
|
||||
// From before the batch-only day, text counts too: a Backported-from trailer
|
||||
// naming the merge commit, or -x lines anywhere in a trailer block naming it
|
||||
// or every commit of its own.
|
||||
func (h *history) found(o *original, commits []string) (string, string) {
|
||||
if c := h.appended[o.merge]; c != "" {
|
||||
return c, "picked in " + c[:12]
|
||||
}
|
||||
if c := coveredBy(h.appended, o, commits); c != "" {
|
||||
return c, fmt.Sprintf("its %d %s picked, the last in %s", len(commits), plural(len(commits), "commit"), c[:12])
|
||||
}
|
||||
if c, ok := h.batches[o.merge]; ok {
|
||||
return c, "named by a " + batchTrailer + " trailer in " + c[:12]
|
||||
}
|
||||
if c := h.pickOf(o.merge); c != "" {
|
||||
return c, "named by an -x line in " + c[:12]
|
||||
}
|
||||
named := map[string]string{}
|
||||
for _, r := range h.picks {
|
||||
if _, ok := named[r.source]; !ok {
|
||||
named[r.source] = r.commit
|
||||
}
|
||||
}
|
||||
if c := coveredBy(named, o, commits); c != "" {
|
||||
return c, fmt.Sprintf("its %d %s named by -x lines, the last in %s", len(commits), plural(len(commits), "commit"), c[:12])
|
||||
}
|
||||
return "", ""
|
||||
}
|
||||
|
||||
// coveredBy returns the commit in by that names the last of a merged pull
|
||||
// request's own commits when by names every one of them, or "".
|
||||
func coveredBy(by map[string]string, o *original, commits []string) string {
|
||||
if o.parents < 2 || len(commits) == 0 {
|
||||
return ""
|
||||
}
|
||||
last := ""
|
||||
for _, source := range commits {
|
||||
if last = by[source]; last == "" {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
return last
|
||||
}
|
||||
|
||||
// carries says how the branch carries an original, or "" when it does not.
|
||||
// The no-op pick goes last: it is the one check that asks git to merge.
|
||||
func (h *history) carries(o *original, commits []string) (string, error) {
|
||||
if h.reachable[o.merge] {
|
||||
return "merged before the line forked", nil
|
||||
}
|
||||
if c, how := h.found(o, commits); c != "" {
|
||||
return how, nil
|
||||
}
|
||||
if c, ok := h.backports[o.number]; ok {
|
||||
return `named by "Backport of" in ` + c[:12], nil
|
||||
}
|
||||
noop, err := noopPick(h.head, o)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if noop {
|
||||
return "a pick would change nothing", nil
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// noopPick reports whether cherry-picking the original onto head would leave
|
||||
// head's tree as it is: the content is there, whatever names it.
|
||||
func noopPick(head string, o *original) (bool, error) {
|
||||
stdout, stderr, code, err := gitStatus("merge-tree", "--write-tree", "--no-messages", "--merge-base="+o.merge+"^1", "--end-of-options", head, o.merge)
|
||||
switch {
|
||||
case err != nil:
|
||||
return false, err
|
||||
case code == 1:
|
||||
return false, nil
|
||||
case code != 0:
|
||||
return false, fmt.Errorf("git merge-tree for #%d: exit %d: %s", o.number, code, strings.TrimSpace(stderr))
|
||||
}
|
||||
tree, _, _ := strings.Cut(stdout, "\n")
|
||||
headTree, err := resolveTree(head)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return tree == headTree, nil
|
||||
}
|
||||
|
||||
func resolveTree(commit string) (string, error) {
|
||||
out, err := git("rev-parse", "--verify", "--quiet", "--end-of-options", commit+"^{tree}")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return strings.TrimSpace(out), nil
|
||||
}
|
||||
@@ -0,0 +1,333 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// verdictOn prepares the line and returns the verdict on one original.
|
||||
func verdictOn(t *testing.T, f *fixture, gh *fakeGH, lineName string, n int) *verdict {
|
||||
t.Helper()
|
||||
f.git("fetch", "-q", "origin")
|
||||
l, _ := parseLine(lineName)
|
||||
if gh == nil {
|
||||
gh = &fakeGH{}
|
||||
}
|
||||
b, err := (&app{gh: gh, stderr: &strings.Builder{}}).prepare(l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v := b.find(n)
|
||||
if v == nil {
|
||||
t.Fatalf("#%d is not an original for %s", n, lineName)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func TestPresenceOnTheLine(t *testing.T) {
|
||||
const fix = "fix(app): the change"
|
||||
files := map[string]string{"app/x.txt": "fixed\n", "app/y.txt": "also\n"}
|
||||
cases := []struct {
|
||||
name string
|
||||
// land puts something on release-1.1 for the original merged as mc,
|
||||
// whose own commit is pr.
|
||||
land func(f *fixture, mc, pr string)
|
||||
carried string
|
||||
}{
|
||||
{
|
||||
name: "nothing on the line",
|
||||
land: func(*fixture, string, string) {},
|
||||
},
|
||||
{
|
||||
name: "an -x line naming the merge commit",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", fix+"\n\n(cherry picked from commit "+mc+")", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
carried: "picked in",
|
||||
},
|
||||
{
|
||||
name: "an -x line naming the pull request's only commit",
|
||||
land: func(f *fixture, _, pr string) {
|
||||
f.onLine("release-1.1", "change\n\n(cherry picked from commit "+pr+")", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
carried: "its 1 commit picked",
|
||||
},
|
||||
{
|
||||
name: "an -x line among other trailers",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", "port\n\nSigned-off-by: A <a@example.invalid>\n(cherry picked from commit "+mc+")\nSigned-off-by: B <b@example.invalid>", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
carried: "picked in",
|
||||
},
|
||||
{
|
||||
name: "an abbreviated -x line",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", "port\n\n(cherry picked from commit "+mc[:12]+")", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an -x line in the body, not in the trailers",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", "port\n\n(cherry picked from commit "+mc+")\n\nThe rest of it is still to come.", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an -x line inside a prose paragraph at the end",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", "port\n\nThis is not (cherry picked from commit "+mc+")\nbut a sentence.", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "a squash-merged hand port that says Backport of, from before the batch-only day",
|
||||
land: func(f *fixture, _, _ string) {
|
||||
f.t.Setenv("GIT_COMMITTER_DATE", "2026-10-01T00:00:00Z")
|
||||
f.onLine("release-1.1", "[Backport release-1.1] fix(app): the change (#77)\n\n# Description\nBackport of #5 to `release-1.1`.", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
carried: `named by "Backport of"`,
|
||||
},
|
||||
{
|
||||
name: "the same after the batch-only day",
|
||||
land: func(f *fixture, _, _ string) {
|
||||
f.t.Setenv("GIT_COMMITTER_DATE", "2026-10-27T00:00:00Z")
|
||||
f.onLine("release-1.1", "[Backport release-1.1] fix(app): the change (#77)\n\n# Description\nBackport of #5 to `release-1.1`.", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an -x line followed by prose in its paragraph",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", "port\n\n(cherry picked from commit "+mc+")\nand more words", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an -x line in a paragraph git reads as trailers that holds prose too",
|
||||
land: func(f *fixture, mc, _ string) {
|
||||
f.onLine("release-1.1", "port\n\nFixes the thing.\nSigned-off-by: A <a@example.invalid>\n(cherry picked from commit "+mc+")\nSigned-off-by: B <b@example.invalid>", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
carried: "picked in",
|
||||
},
|
||||
{
|
||||
name: "Backport of another repository's number",
|
||||
land: func(f *fixture, _, _ string) {
|
||||
f.onLine("release-1.1", "port\n\nBackport of other/repo#5.", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "the same change under another name",
|
||||
land: func(f *fixture, _, _ string) {
|
||||
f.onLine("release-1.1", "fix(app): unrelated title", files)
|
||||
},
|
||||
carried: "a pick would change nothing",
|
||||
},
|
||||
{
|
||||
name: "half of the change, named nowhere",
|
||||
land: func(f *fixture, _, _ string) {
|
||||
f.onLine("release-1.1", "fix(app): half", map[string]string{"app/x.txt": "fixed\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an -x line naming some other commit",
|
||||
land: func(f *fixture, _, _ string) {
|
||||
f.onLine("release-1.1", "port\n\n(cherry picked from commit 0123456789abcdef0123456789abcdef01234567)", map[string]string{"app/z.txt": "z\n"})
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc := f.pr(5, fix, "", files)
|
||||
pr := f.git("rev-parse", mc+"^2")
|
||||
tc.land(f, mc, pr)
|
||||
v := verdictOn(t, f, nil, "release-1.1", 5)
|
||||
switch {
|
||||
case tc.carried == "" && v.rule != 0:
|
||||
t.Errorf("verdict = rule %d (%s), want it to qualify", v.rule, v.reason)
|
||||
case tc.carried != "" && (v.rule != ruleOnLine || !strings.HasPrefix(v.reason, tc.carried)):
|
||||
t.Errorf("verdict = rule %d (%s), want rule 1 with %q", v.rule, v.reason, tc.carried)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// History the line shares with main is not evidence: a commit from before
|
||||
// the line branched can say "Backport of #5" without the line carrying #5.
|
||||
func TestSharedHistoryIsNotEvidence(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.pr(1, "docs(app): plan", "Backport of #5 is planned.", map[string]string{"app/plan.txt": "p\n"})
|
||||
f.release("release-1.1")
|
||||
f.pr(5, "fix(app): the change", "", map[string]string{"app/x.txt": "fixed\n"})
|
||||
if v := verdictOn(t, f, nil, "release-1.1", 5); v.rule != 0 {
|
||||
t.Errorf("verdict = rule %d (%s), want it to qualify", v.rule, v.reason)
|
||||
}
|
||||
}
|
||||
|
||||
// Only a batch pull request opened from this repository by someone with write
|
||||
// access, or by the bot the batch runs as, speaks for the batch.
|
||||
func TestCarriedByTheBatch(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
edit func(pr *pullRequest, gh *fakeGH)
|
||||
carried bool
|
||||
notAsked bool // the author's permission must not be looked up
|
||||
}{
|
||||
{name: "a maintainer with write access", carried: true},
|
||||
{name: "an admin", edit: func(_ *pullRequest, gh *fakeGH) { gh.perms = map[string]string{"maintainer": "admin"} }, carried: true},
|
||||
{name: "a private member reported as a contributor, with write access",
|
||||
edit: func(pr *pullRequest, _ *fakeGH) { pr.AuthorAssociation = "CONTRIBUTOR" }, carried: true},
|
||||
{name: "an organization member with read access",
|
||||
edit: func(pr *pullRequest, gh *fakeGH) { gh.perms = map[string]string{"maintainer": "read"} }},
|
||||
{name: "an invited collaborator with read access", edit: func(pr *pullRequest, gh *fakeGH) {
|
||||
pr.AuthorAssociation = "COLLABORATOR"
|
||||
gh.perms = map[string]string{"maintainer": "read"}
|
||||
}},
|
||||
{name: "no access", edit: func(_ *pullRequest, gh *fakeGH) { gh.perms = map[string]string{"maintainer": "none"} }},
|
||||
{name: "the batch's bot", edit: func(pr *pullRequest, gh *fakeGH) {
|
||||
gh.bot = "cozystack-ci[bot]"
|
||||
pr.User.Login, pr.User.Type, pr.AuthorAssociation = "cozystack-ci[bot]", "Bot", "CONTRIBUTOR"
|
||||
// The bot writes the record into every body it opens.
|
||||
l, _ := parseLine("release-1.1")
|
||||
pr.Body = deferredBlock(l, nil, nil, nil)
|
||||
}, carried: true, notAsked: true},
|
||||
{name: "the batch's bot when no bot is configured", edit: func(pr *pullRequest, _ *fakeGH) {
|
||||
pr.User.Login, pr.User.Type = "cozystack-ci[bot]", "Bot"
|
||||
}, notAsked: true},
|
||||
{name: "GitHub Actions' bot", edit: func(pr *pullRequest, gh *fakeGH) {
|
||||
gh.bot = "cozystack-ci[bot]"
|
||||
pr.User.Login, pr.User.Type, pr.AuthorAssociation = "github-actions[bot]", "Bot", "NONE"
|
||||
}, notAsked: true},
|
||||
{name: "another bot", edit: func(pr *pullRequest, gh *fakeGH) {
|
||||
gh.bot = "cozystack-ci[bot]"
|
||||
pr.User.Login, pr.User.Type = "renovate[bot]", "Bot"
|
||||
}, notAsked: true},
|
||||
{name: "a user named like the bot", edit: func(pr *pullRequest, gh *fakeGH) {
|
||||
gh.bot = "cozystack-ci[bot]"
|
||||
pr.User.Login, pr.User.Type = "cozystack-ci[bot]", "User"
|
||||
}, notAsked: true},
|
||||
{name: "a login shaped like a path", edit: func(pr *pullRequest, _ *fakeGH) { pr.User.Login = "maintainer/../../orgs/x" }, notAsked: true},
|
||||
{name: "a fork whose branch is named like a batch", edit: func(pr *pullRequest, _ *fakeGH) { pr.Head.Repo.FullName = "someone/cozystack" }, notAsked: true},
|
||||
{name: "a deleted fork", edit: func(pr *pullRequest, _ *fakeGH) { pr.Head.Repo = nil }, notAsked: true},
|
||||
{name: "a branch for another line", edit: func(pr *pullRequest, _ *fakeGH) { pr.Head.Ref = "backport-batch/release-1.0-20261026" }, notAsked: true},
|
||||
{name: "a branch not named for a batch", edit: func(pr *pullRequest, _ *fakeGH) { pr.Head.Ref = "my-backports" }, notAsked: true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc := f.pr(5, "fix(app): the change", "", map[string]string{"app/x.txt": "fixed\n"})
|
||||
f.git("switch", "-q", "-c", "batch", "origin/release-1.1")
|
||||
f.git("cherry-pick", "-x", "-m", "1", mc)
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
pr := f.batchPR(90, "release-1.1", batchBranch1, "")
|
||||
gh := &fakeGH{}
|
||||
if tc.edit != nil {
|
||||
tc.edit(&pr, gh)
|
||||
}
|
||||
gh.prs = []pullRequest{pr}
|
||||
v := verdictOn(t, f, gh, "release-1.1", 5)
|
||||
if (v.onBatch != "") != tc.carried {
|
||||
t.Errorf("onBatch = %q, want carried=%v", v.onBatch, tc.carried)
|
||||
}
|
||||
if tc.notAsked && len(gh.asked) > 0 {
|
||||
t.Errorf("asked GitHub for the permission of %q", gh.asked)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The batch head is the one GitHub reports: a branch that moved since is read
|
||||
// again, and one that does not match even then is refused.
|
||||
func TestBatchHeadMustBeGitHubs(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): the change", "", map[string]string{"app/x.txt": "fixed\n"})
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+batchBranch1)
|
||||
pr := f.batchPR(90, "release-1.1", batchBranch1, "")
|
||||
pr.Head.SHA = strings.Repeat("a", 40)
|
||||
f.git("fetch", "-q", "origin")
|
||||
l, _ := parseLine("release-1.1")
|
||||
_, err := (&app{gh: &fakeGH{prs: []pullRequest{pr}}, stderr: &strings.Builder{}}).prepare(l)
|
||||
if err == nil || !strings.Contains(err.Error(), "as GitHub reports") {
|
||||
t.Errorf("prepare = %v, want a refusal of the moved head", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTwoBatchPullRequestsAreRefused(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+batchBranch1)
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/backport-batch/release-1.1-20261022")
|
||||
gh := &fakeGH{prs: []pullRequest{
|
||||
f.batchPR(90, "release-1.1", batchBranch1, ""),
|
||||
f.batchPR(91, "release-1.1", "backport-batch/release-1.1-20261022", ""),
|
||||
}}
|
||||
code, _, stderr := f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 2 || !strings.Contains(stderr, "more than one open batch") {
|
||||
t.Errorf("check = %d (%s), want 2", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A permission GitHub does not answer fails the run: read as a refusal, it
|
||||
// would pass over the open batch, and the run would open a second one.
|
||||
func TestUnreadablePermissionFailsTheRun(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc := f.pr(5, "fix(app): the change", "", map[string]string{"app/x.txt": "fixed\n"})
|
||||
const opened = "backport-batch/release-1.1-20261022"
|
||||
f.git("switch", "-q", "-c", "batch", "origin/release-1.1")
|
||||
f.git("cherry-pick", "-x", "-m", "1", mc)
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+opened)
|
||||
f.git("switch", "-q", "main")
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{prs: []pullRequest{f.batchPR(90, "release-1.1", opened, "")}, permErr: errString("HTTP 502")}
|
||||
for _, args := range [][]string{
|
||||
{"run", "--line", "release-1.1", "--continue", "--push"},
|
||||
{"check", "--line", "release-1.1"},
|
||||
{"verify", "--line", "release-1.1"},
|
||||
} {
|
||||
if code, _, stderr := f.exec(gh, args...); code != 2 || !strings.Contains(stderr, "batch pull request #90: HTTP 502") {
|
||||
t.Errorf("%q = %d (%s), want 2 naming the read that failed", args, code, stderr)
|
||||
}
|
||||
}
|
||||
if len(gh.created) != 0 || len(gh.edited) != 0 || f.remoteHead(batchBranch1) != "" {
|
||||
t.Errorf("opened %q, edited %q and pushed %q; want nothing written", gh.created, gh.edited, f.remoteHead(batchBranch1))
|
||||
}
|
||||
}
|
||||
|
||||
func TestBodyOrigins(t *testing.T) {
|
||||
cases := []struct {
|
||||
body string
|
||||
want []int
|
||||
}{
|
||||
{"Backport of #4253 to `release-1.6`.", []int{4253}},
|
||||
{"Backport of #1, #2, and #3", []int{1, 2, 3}},
|
||||
{"Backport of #4253 to release-1.6, together with #3460", []int{4253, 3460}},
|
||||
{"Backport of #10, #20 is not included", []int{10}},
|
||||
{"Backport of cozystack/cozystack#7.", []int{7}},
|
||||
{"Backport of other/repo#7.", nil},
|
||||
{"Fixes #12; see #13", nil},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := bodyOrigins(tc.body, upstreamRepo); !slices.Equal(got, tc.want) {
|
||||
t.Errorf("bodyOrigins(%q) = %v, want %v", tc.body, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestCheckBranch(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.git("branch", "elsewhere")
|
||||
f.git("switch", "-q", "elsewhere")
|
||||
f.git("switch", "-q", "main")
|
||||
for _, name := range []string{"backport-batch/release-1.6-20261026", "release-1.6"} {
|
||||
if err := checkBranch(name); err != nil {
|
||||
t.Errorf("checkBranch(%q) = %v, want nil", name, err)
|
||||
}
|
||||
}
|
||||
// @{-1} is a valid argument to check-ref-format --branch, which answers
|
||||
// with the branch it stands for.
|
||||
for _, name := range []string{"", "-x", "--upload-pack=evil", "@{-1}", "a..b", "a b", "line\nbreak", "esc\x1b", "a~1", "a:b", "refs/heads/../x"} {
|
||||
if err := checkBranch(name); err == nil {
|
||||
t.Errorf("checkBranch(%q) accepted it", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A no from git is an answer; git failing is an error, never a no.
|
||||
func TestGitTellsAnAnswerFromAFailure(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
head := f.git("rev-parse", "HEAD")
|
||||
missing := strings.Repeat("1", 40)
|
||||
if got, err := lookup(head); got != head || err != nil {
|
||||
t.Errorf("lookup(HEAD) = %q, %v", got, err)
|
||||
}
|
||||
for _, rev := range []string{missing, "refs/heads/nowhere", head + "^{tree}"} {
|
||||
if got, err := lookup(rev); got != "" || err != nil {
|
||||
t.Errorf("lookup(%q) = %q, %v; want no commit and no error", rev, got, err)
|
||||
}
|
||||
}
|
||||
if ok, err := isAncestor(head, head); !ok || err != nil {
|
||||
t.Errorf("isAncestor(HEAD, HEAD) = %v, %v", ok, err)
|
||||
}
|
||||
if _, err := isAncestor(missing, head); err == nil {
|
||||
t.Error("isAncestor of a commit this repository does not have answered instead of failing")
|
||||
}
|
||||
if got, err := lineTemplate(head); err != nil || !strings.HasPrefix(got, "## What this PR does") {
|
||||
t.Errorf("lineTemplate without a template = %q, %v; want the default", got, err)
|
||||
}
|
||||
f.write(".github/PULL_REQUEST_TEMPLATE.md", "## The line's own\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "chore: template")
|
||||
if got, err := lineTemplate(f.git("rev-parse", "HEAD")); err != nil || got != "## The line's own\n" {
|
||||
t.Errorf("lineTemplate = %q, %v; want the line's own", got, err)
|
||||
}
|
||||
|
||||
t.Chdir(t.TempDir())
|
||||
t.Setenv("GIT_CEILING_DIRECTORIES", filepath.Dir(t.TempDir()))
|
||||
if _, err := lookup(head); err == nil {
|
||||
t.Error("lookup outside a repository answered instead of failing")
|
||||
}
|
||||
if _, err := lineTemplate(head); err == nil {
|
||||
t.Error("lineTemplate outside a repository gave the default instead of failing")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseLine(t *testing.T) {
|
||||
for _, ok := range []string{"release-1.6", "release-10.0"} {
|
||||
if _, err := parseLine(ok); err != nil {
|
||||
t.Errorf("parseLine(%q) = %v", ok, err)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"", "-x", "--line", "release-1.6\n", "release-01.6", "release-1.6.1", "release-1.6-rc", " release-1.6", "origin/release-1.6", "release-1.6;rm"} {
|
||||
if _, err := parseLine(bad); err == nil {
|
||||
t.Errorf("parseLine(%q) accepted it", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// What the command line, GitHub or a pull request supplies never reaches git
|
||||
// as an option.
|
||||
func TestOptionInjection(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
canary := filepath.Join(t.TempDir(), "canary")
|
||||
|
||||
for _, args := range [][]string{
|
||||
{"run", "--line", "-x"},
|
||||
{"run", "--line", "--upload-pack=touch " + canary},
|
||||
{"run", "--line", "release-1.1", "--onto", "--upload-pack=touch " + canary},
|
||||
{"run", "--line", "release-1.1", "--date", "-1"},
|
||||
{"run", "--line", "release-1.1", "--date", "2026-10-26\n"},
|
||||
{"run", "--line", "release-1.1", "stray"},
|
||||
{"check", "--line", "release-1.1;touch " + canary},
|
||||
{"fold", "--line", "release-1.1", "--original", "-5", "--commit", strings.Repeat("a", 40)},
|
||||
{"fold", "--line", "release-1.1", "--original", "5", "--commit", "--output=" + canary},
|
||||
} {
|
||||
if code, _, _ := f.exec(nil, args...); code != 2 {
|
||||
t.Errorf("%q = %d, want 2", args, code)
|
||||
}
|
||||
}
|
||||
if _, err := os.Stat(canary); err == nil {
|
||||
t.Error("an argument ran as a command")
|
||||
}
|
||||
|
||||
// A batch pull request whose head GitHub reports as an option is ignored.
|
||||
for _, ref := range []string{"--upload-pack=touch " + canary, "-backport-batch/release-1.1-20261026", "backport-batch/release-1.1-20261026\n"} {
|
||||
pr := f.batchPR(90, "release-1.1", batchBranch1, "")
|
||||
pr.Head.Ref, pr.Head.SHA = ref, strings.Repeat("a", 40)
|
||||
code, _, stderr := f.exec(&fakeGH{prs: []pullRequest{pr}}, "check", "--line", "release-1.1")
|
||||
if code != 1 {
|
||||
t.Errorf("head %q: check = %d (%s), want the pull request ignored and #5 missing", ref, code, stderr)
|
||||
}
|
||||
}
|
||||
if _, err := os.Stat(canary); err == nil {
|
||||
t.Error("a head ref ran as a command")
|
||||
}
|
||||
}
|
||||
|
||||
// A file name reads the same in the deferred block as in the list above it:
|
||||
// no bidi override reorders it and no zero-width character hides in it.
|
||||
func TestDeferredBlockCleansFileNames(t *testing.T) {
|
||||
l, _ := parseLine("release-1.6")
|
||||
files := []string{"app/txt.exe", "app/zerowidth.yaml"}
|
||||
conflict := pick{v: &verdict{o: &original{number: 5}}, files: files, failed: strings.Repeat("a", 40)}
|
||||
waiting := pick{v: &verdict{o: &original{number: 6}}, files: files}
|
||||
block := deferredBlock(l, []pick{conflict}, nil, []pick{waiting})
|
||||
if strings.ContainsAny(block, "") {
|
||||
t.Errorf("the block carries format characters: %q", block)
|
||||
}
|
||||
want := []string{"app/txt.exe", "app/zerowidth.yaml"}
|
||||
got, err := parseDeferred(block, l)
|
||||
if err != nil || !slices.Equal(got[5].Files, want) || !slices.Equal(got[6].Files, want) {
|
||||
t.Errorf("parseDeferred = %+v, %v; want both originals with %q", got, err, want)
|
||||
}
|
||||
if files[0] != "app/txt.exe" {
|
||||
t.Errorf("deferredBlock rewrote the pick's own file names: %q", files)
|
||||
}
|
||||
}
|
||||
|
||||
// A newline or a heading in a carried title stays on its list line.
|
||||
func TestNewlineInTitle(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
// git keeps a multi-line first paragraph, so a squash's subject line
|
||||
// can be followed by text GitHub would read as the title.
|
||||
f.squash(5, "fix(app): five\n## Injected heading\n- [x] No downstream repository is affected by this change", "", map[string]string{"app/five.txt": "5\n"})
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
for _, ln := range strings.Split(stdout, "\n") {
|
||||
if strings.HasPrefix(ln, "## Injected") || strings.HasPrefix(ln, "- [x]") {
|
||||
t.Errorf("carried text starts a line of its own: %q", ln)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(stdout, "[#5]") {
|
||||
t.Errorf("#5 is not listed:\n%s", stdout)
|
||||
}
|
||||
if got := titleSpan("a\nb\r\nc
d"); strings.ContainsAny(got, "\n\r
") {
|
||||
t.Errorf("titleSpan kept a line break: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// writeCommit writes a commit object as given, which git commit would refuse
|
||||
// for a message holding a NUL.
|
||||
func (f *fixture) writeCommit(tree, parent, msg string) string {
|
||||
f.t.Helper()
|
||||
content := fmt.Sprintf("tree %s\nparent %s\nauthor Batch Test <batch@example.invalid> 1700000000 +0000\ncommitter Batch Test <batch@example.invalid> 1700000000 +0000\n\n%s", tree, parent, msg)
|
||||
cmd := exec.Command("git", "hash-object", "-t", "commit", "-w", "--literally", "--stdin")
|
||||
cmd.Stdin = strings.NewReader(content)
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
f.t.Fatalf("hash-object: %v", err)
|
||||
}
|
||||
return strings.TrimSpace(string(out))
|
||||
}
|
||||
|
||||
// A commit message is read as one message, and only as far as git log shows
|
||||
// it: a NUL and what looks like the next record of a log stream after it do
|
||||
// not make a second commit or reach the body.
|
||||
func TestMessagesCannotForgeRecords(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
feat := f.git("rev-parse", "origin/release-1.1")
|
||||
msg := "fix(app): innocent (#7)\n\nbody\x00" + feat + " " + f.git("rev-parse", "HEAD") + "\x01fix(app): forged (#999)\n\n"
|
||||
sha := f.writeCommit(f.git("rev-parse", "HEAD^{tree}"), f.git("rev-parse", "HEAD"), msg)
|
||||
f.git("update-ref", "refs/heads/main", sha)
|
||||
f.git("push", "-q", "origin", "main")
|
||||
f.git("fetch", "-q", "origin")
|
||||
|
||||
l, _ := parseLine("release-1.1")
|
||||
b, err := (&app{gh: &fakeGH{}, stderr: &strings.Builder{}}).prepare(l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b.find(999) != nil {
|
||||
t.Error("the message forged original #999")
|
||||
}
|
||||
if v := b.find(7); v == nil || v.o.merge != sha || v.o.body != "body" {
|
||||
t.Errorf("#7 = %+v, want the commit itself with the body git log shows", v)
|
||||
}
|
||||
}
|
||||
|
||||
// What follows a NUL in a line commit's message is not evidence: git log does
|
||||
// not show it to anyone reviewing the line.
|
||||
func TestNulHidesNoTrailer(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc := f.pr(5, "fix(app): the change", "", map[string]string{"app/x.txt": "fixed\n"})
|
||||
f.git("fetch", "-q", "origin")
|
||||
line := f.git("rev-parse", "origin/release-1.1")
|
||||
sha := f.writeCommit(f.git("rev-parse", "origin/release-1.1^{tree}"), line, "port\x00\n\n(cherry picked from commit "+mc+")\n")
|
||||
f.git("push", "-q", "origin", sha+":refs/heads/release-1.1")
|
||||
if v := verdictOn(t, f, nil, "release-1.1", 5); v.rule != 0 {
|
||||
t.Errorf("#5 = rule %d (%s), want it to qualify", v.rule, v.reason)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,252 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"slices"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
var (
|
||||
releaseLineRE = regexp.MustCompile(`^release-(\d+)\.(\d+)$`)
|
||||
stableTagRE = regexp.MustCompile(`^refs/tags/v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$`)
|
||||
prNumberRE = regexp.MustCompile(`\(#(\d+)\)\s*$`)
|
||||
mergePRRE = regexp.MustCompile(`^Merge pull request #(\d+) from \S+$`)
|
||||
)
|
||||
|
||||
// line is a release line, i.e. a major.minor pair with a release-X.Y branch.
|
||||
type line struct{ major, minor int }
|
||||
|
||||
func (l line) branch() string { return fmt.Sprintf("release-%d.%d", l.major, l.minor) }
|
||||
func (l line) less(o line) bool {
|
||||
if l.major != o.major {
|
||||
return l.major < o.major
|
||||
}
|
||||
return l.minor < o.minor
|
||||
}
|
||||
|
||||
// parseLine accepts exactly the branch name of a line: release-01.6 parses to
|
||||
// the same numbers as release-1.6 without being that branch.
|
||||
func parseLine(s string) (line, error) {
|
||||
m := releaseLineRE.FindStringSubmatch(s)
|
||||
if m == nil {
|
||||
return line{}, fmt.Errorf("not a release line: %q (expected release-<major>.<minor>)", s)
|
||||
}
|
||||
major, err1 := strconv.Atoi(m[1])
|
||||
minor, err2 := strconv.Atoi(m[2])
|
||||
l := line{major, minor}
|
||||
if err1 != nil || err2 != nil || l.branch() != s {
|
||||
return line{}, fmt.Errorf("not a release line: %q (expected release-<major>.<minor>)", s)
|
||||
}
|
||||
return l, nil
|
||||
}
|
||||
|
||||
// releaseLines returns the two-component release branches on the remote,
|
||||
// oldest first. release-1.6.0-rc.1 and the like are staging branches, not
|
||||
// maintenance lines, so they are excluded.
|
||||
func releaseLines() ([]line, error) {
|
||||
out, err := git("for-each-ref", "--format=%(refname:strip=3)", "refs/remotes/"+remote+"/release-*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var lines []line
|
||||
for name := range strings.FieldsSeq(out) {
|
||||
if l, err := parseLine(name); err == nil {
|
||||
lines = append(lines, l)
|
||||
}
|
||||
}
|
||||
sort.Slice(lines, func(i, j int) bool { return lines[i].less(lines[j]) })
|
||||
return lines, nil
|
||||
}
|
||||
|
||||
// shippedLines reads which lines have a stable release on the remote: a
|
||||
// vX.Y.Z tag with no pre-release suffix, which only the finalize job of
|
||||
// pull-requests-release.yaml creates, when the promoted release merges.
|
||||
func shippedLines() (map[line]bool, error) {
|
||||
out, err := git("ls-remote", "--tags", "--refs", "--end-of-options", remote, "refs/tags/v*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
shipped := map[line]bool{}
|
||||
for ln := range strings.Lines(out) {
|
||||
_, ref, _ := strings.Cut(strings.TrimSpace(ln), "\t")
|
||||
m := stableTagRE.FindStringSubmatch(ref)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
major, err1 := strconv.Atoi(m[1])
|
||||
minor, err2 := strconv.Atoi(m[2])
|
||||
if err1 == nil && err2 == nil {
|
||||
shipped[line{major, minor}] = true
|
||||
}
|
||||
}
|
||||
return shipped, nil
|
||||
}
|
||||
|
||||
// supportedLines picks from all, oldest first, the lines the batch runs for:
|
||||
// the newest released lines that have shipped a stable release, and every
|
||||
// line branched after the newest of them, whose rc window is open.
|
||||
func supportedLines(all []line, shipped map[line]bool, released int) []line {
|
||||
var out []line
|
||||
kept := 0
|
||||
for i := len(all) - 1; i >= 0 && kept < released; i-- {
|
||||
switch {
|
||||
case shipped[all[i]]:
|
||||
kept++
|
||||
case kept > 0:
|
||||
// Older than a release and never released itself: abandoned.
|
||||
continue
|
||||
}
|
||||
out = append(out, all[i])
|
||||
}
|
||||
slices.Reverse(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// batchLines are the lines the batch runs for, oldest first.
|
||||
func batchLines(released int) ([]line, error) {
|
||||
all, err := releaseLines()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(all) == 0 {
|
||||
return nil, fmt.Errorf("no release-X.Y branches on %s", remote)
|
||||
}
|
||||
shipped, err := shippedLines()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return supportedLines(all, shipped, released), nil
|
||||
}
|
||||
|
||||
// newerLine returns the line after want among lines, oldest first, and nil
|
||||
// for the newest. Rule 6 of the policy holds a line to what the next newer one
|
||||
// carries, and the batch runs for no line outside them.
|
||||
func newerLine(want line, lines []line) (*line, error) {
|
||||
i := slices.Index(lines, want)
|
||||
switch {
|
||||
case i < 0:
|
||||
names := make([]string, len(lines))
|
||||
for j, l := range lines {
|
||||
names[j] = l.branch()
|
||||
}
|
||||
return nil, fmt.Errorf("%s is not a line the batch runs for; it runs for %s", want.branch(), strings.Join(names, ", "))
|
||||
case i == len(lines)-1:
|
||||
return nil, nil
|
||||
}
|
||||
return &lines[i+1], nil
|
||||
}
|
||||
|
||||
// original is a pull request merged into main, read from its merge commit.
|
||||
//
|
||||
// The title and body come from the commit message, not from GitHub: the
|
||||
// commit holds them as they were when a maintainer merged the pull request,
|
||||
// while the live pull request stays editable by its author.
|
||||
type original struct {
|
||||
number int
|
||||
title string
|
||||
body string
|
||||
merge string
|
||||
parents int
|
||||
merged time.Time
|
||||
}
|
||||
|
||||
func (o *original) url() string {
|
||||
return fmt.Sprintf("https://github.com/%s/pull/%d", upstreamRepo, o.number)
|
||||
}
|
||||
|
||||
// mainMerges reads main's first-parent history after base, oldest first: one
|
||||
// original per pull request merge, and the commits that name no pull request.
|
||||
func mainMerges(base, mainHead string) ([]*original, []string, error) {
|
||||
out, err := git("rev-list", "--first-parent", "--reverse", "--parents", "--end-of-options", base+".."+mainHead)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
type merge struct {
|
||||
oid string
|
||||
parents int
|
||||
}
|
||||
var list []merge
|
||||
var oids []string
|
||||
for ln := range strings.Lines(out) {
|
||||
f := strings.Fields(ln)
|
||||
for _, sha := range f {
|
||||
if checkSHA(sha) != nil {
|
||||
return nil, nil, fmt.Errorf("rev-list: %q is not a commit", sha)
|
||||
}
|
||||
}
|
||||
if len(f) < 2 {
|
||||
return nil, nil, fmt.Errorf("rev-list: %s has no parent", f[0])
|
||||
}
|
||||
list = append(list, merge{f[0], len(f) - 1})
|
||||
oids = append(oids, f[0])
|
||||
}
|
||||
msgs, err := commitMessages(oids)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
var merges []*original
|
||||
var other []string
|
||||
for _, m := range list {
|
||||
o := &original{merge: m.oid, parents: m.parents, merged: msgs[m.oid].committed}
|
||||
o.number, o.title, o.body = mergedPR(msgs[m.oid].message)
|
||||
if o.number <= 0 || o.parents > 2 {
|
||||
other = append(other, m.oid)
|
||||
continue
|
||||
}
|
||||
merges = append(merges, o)
|
||||
}
|
||||
return merges, other, nil
|
||||
}
|
||||
|
||||
// mergedPR reads the pull request a commit on main's first-parent line
|
||||
// merged from its message, with its title and body; 0 when it names none.
|
||||
func mergedPR(message string) (int, string, string) {
|
||||
// The subject is the first paragraph, its lines joined, as git reads it.
|
||||
subject, body, _ := strings.Cut(strings.TrimRight(message, "\n"), "\n\n")
|
||||
subject = strings.ReplaceAll(subject, "\n", " ")
|
||||
number, title := 0, ""
|
||||
if pr := mergePRRE.FindStringSubmatch(subject); pr != nil {
|
||||
number, _ = strconv.Atoi(pr[1])
|
||||
body = strings.TrimLeft(body, "\n")
|
||||
title, body, _ = strings.Cut(body, "\n")
|
||||
} else if pr := prNumberRE.FindStringSubmatchIndex(subject); pr != nil {
|
||||
number, _ = strconv.Atoi(subject[pr[2]:pr[3]])
|
||||
title = strings.TrimSpace(subject[:pr[0]])
|
||||
}
|
||||
return number, title, strings.TrimLeft(body, "\n")
|
||||
}
|
||||
|
||||
// prCommits lists what an original brings to a line, one pick each: for a
|
||||
// merged pull request every commit of its own, oldest first, leaving out the
|
||||
// merges that brought main into its branch, whose content main already has;
|
||||
// for a squash, its one commit on main.
|
||||
func prCommits(o *original) ([]string, error) {
|
||||
if o.parents < 2 {
|
||||
return []string{o.merge}, nil
|
||||
}
|
||||
oids, err := revList("--reverse", "--topo-order", "--no-merges", "--end-of-options", o.merge+"^1.."+o.merge+"^2")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading the commits of #%d: %w", o.number, err)
|
||||
}
|
||||
return oids, nil
|
||||
}
|
||||
@@ -0,0 +1,499 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
// Command backport-batch assembles the automatic backport batch for a release
|
||||
// line and gates a patch cut on it. docs/release/backports.md is the policy it
|
||||
// implements; the README describes the commands.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const usageText = `Assemble and check the automatic backport batch for a release line.
|
||||
|
||||
Usage:
|
||||
backport-batch run --line release-X.Y [--onto SHA] [--date YYYY-MM-DD] [--no-workflow-files] [--push] [--policy REF]
|
||||
backport-batch run --line release-X.Y --continue [--no-workflow-files] [--push] [--policy REF]
|
||||
backport-batch run --line release-X.Y --defer N[,N] --reason TEXT [--push] [--policy REF]
|
||||
backport-batch run --line release-X.Y --drop N[,N] --reason TEXT [--push] [--policy REF]
|
||||
backport-batch run --line release-X.Y --undrop N[,N] [--push] [--policy REF]
|
||||
backport-batch check --line release-X.Y [--cut] [--policy REF]
|
||||
backport-batch fold --line release-X.Y --original N --commit SHA [--policy REF]
|
||||
backport-batch verify --line release-X.Y [--head SHA]
|
||||
backport-batch notify --line release-X.Y [--mention org/team|login] [--push]
|
||||
backport-batch report --pr N [--id ID] [--json]
|
||||
backport-batch lines [--policy REF]
|
||||
|
||||
run Builds backport-batch/<line>-<yyyymmdd> in this working tree, off
|
||||
origin/<line>, or on top of the pushed batch branch whose head is
|
||||
--onto. Each original goes in as its own commits, picked with -x, and
|
||||
a merge commit naming it; a squash-merged one as one pick. --continue
|
||||
takes the head from the open batch PR, and starts a batch when none is
|
||||
open. Prints the pull request body on stdout.
|
||||
--no-workflow-files leaves an original that changes .github/workflows/
|
||||
to a maintainer, listed on the batch, for a token that cannot push it.
|
||||
--push pushes the branch, fast-forward only, and opens or updates the
|
||||
draft batch PR.
|
||||
--defer rebuilds the open batch off origin/<line> without those
|
||||
originals and records them as deferred, for --reason: the batch keeps
|
||||
them out, the next one tries them again. --push replaces the branch,
|
||||
leased on the head the rebuild started from; when nothing is left it
|
||||
closes the batch PR instead, after recording why in it.
|
||||
--drop drops originals from the line for --reason: the line's record
|
||||
keeps them out of this batch and every later one. One the open batch
|
||||
carries is taken off it as --defer would; otherwise only the record
|
||||
changes. --undrop withdraws a drop. Both need an open batch PR whose
|
||||
record they change.
|
||||
check Exits 1 while a change the policy takes is neither on the line nor on
|
||||
the open batch PR, 0 when nothing is, 2 when it cannot tell. With --cut,
|
||||
the gate before an rc or a patch is cut, it also exits 1 while a change
|
||||
is only on the open batch PR, which the cut would not carry, or waits
|
||||
there for a maintainer because it changes workflows.
|
||||
fold Checks a hand-made conflict resolution for the open batch PR: for a
|
||||
merged pull request a merge of its picked commits, for a squash one
|
||||
commit. Prints ok or why not, exiting 0 or 1. It never commits.
|
||||
verify Checks every original on the batch, from the line's head to --head or
|
||||
to the open batch PR's head, against that same shape. Exits 0 when all
|
||||
pass, 1 with the reasons when one does not.
|
||||
notify Prints what waits on the open batch PR for a maintainer: a batch
|
||||
open for 3 days or more, its deferred conflicts, and the originals
|
||||
that change workflows. --push keeps one comment on the PR that says
|
||||
what waits, mentioning --mention.
|
||||
report Prints a report batch PR N keeps: the one its body carries, or the
|
||||
one --id names, read from its comments or its body. --json prints the
|
||||
report as it is stored. It writes nothing, so it reads merged and
|
||||
closed batches too.
|
||||
lines Prints the release lines the batch runs for, newest first.
|
||||
|
||||
The policy is docs/release/backports.md. run, check, fold and lines read its
|
||||
policy block from that page on origin/main, or on --policy REF. Every command
|
||||
but notify and report fetches origin first. notify reads only GitHub; report
|
||||
reads GitHub, and for a report that keeps drops also this clone's history,
|
||||
which has to hold the main and line heads the report was made from: report
|
||||
never fetches on its own.
|
||||
|
||||
BACKPORT_BATCH_BOT names the bot the scheduled batch writes as,
|
||||
<app>[bot], cozystack-ci[bot] when unset: a batch pull request or a notify
|
||||
comment it wrote is trusted like a maintainer's. Set and empty, no bot is.
|
||||
`
|
||||
|
||||
const (
|
||||
remote = "origin"
|
||||
upstreamRepo = "cozystack/cozystack"
|
||||
policyURL = "https://github.com/" + upstreamRepo + "/blob/main/docs/release/backports.md"
|
||||
)
|
||||
|
||||
type app struct {
|
||||
gh github
|
||||
stdout io.Writer
|
||||
stderr io.Writer
|
||||
now func() time.Time
|
||||
// policyRef is where the policy page is read from; origin/main when empty.
|
||||
policyRef string
|
||||
// tool is the commit backport-batch was built from, for the record.
|
||||
tool string
|
||||
}
|
||||
|
||||
func main() {
|
||||
a := &app{gh: ghCLI{bot: batchBotLogin(os.LookupEnv)}, stdout: os.Stdout, stderr: os.Stderr, now: time.Now, tool: toolVersion()}
|
||||
os.Exit(a.main(os.Args[1:]))
|
||||
}
|
||||
|
||||
func (a *app) main(args []string) int {
|
||||
a.stdout, a.stderr = textWriter{a.stdout}, textWriter{a.stderr}
|
||||
a.gh = &permCache{github: a.gh, perms: map[string]string{}}
|
||||
if len(args) == 0 {
|
||||
fmt.Fprint(a.stderr, usageText)
|
||||
return 2
|
||||
}
|
||||
if bot := a.gh.batchBot(); bot != "" && !botLoginRE.MatchString(bot) {
|
||||
return a.fail(fmt.Errorf("%s %q is not a bot login, <app>[bot]", batchBotEnv, bot))
|
||||
}
|
||||
switch args[0] {
|
||||
case "run":
|
||||
return a.run(args[1:])
|
||||
case "check":
|
||||
return a.check(args[1:])
|
||||
case "fold":
|
||||
return a.fold(args[1:])
|
||||
case "verify":
|
||||
return a.verify(args[1:])
|
||||
case "notify":
|
||||
return a.notify(args[1:])
|
||||
case "report":
|
||||
return a.report(args[1:])
|
||||
case "lines":
|
||||
return a.lines(args[1:])
|
||||
case "-h", "--help", "help":
|
||||
fmt.Fprint(a.stdout, usageText)
|
||||
return 0
|
||||
}
|
||||
fmt.Fprintf(a.stderr, "unknown command %q\n\n%s", args[0], usageText)
|
||||
return 2
|
||||
}
|
||||
|
||||
// textWriter keeps output fit for a terminal. Titles, file names and the
|
||||
// messages git relays come from other people, and a control sequence among
|
||||
// them would move the cursor or rewrite what was printed before.
|
||||
type textWriter struct{ w io.Writer }
|
||||
|
||||
func (t textWriter) Write(p []byte) (int, error) {
|
||||
if _, err := io.WriteString(t.w, clean(string(p), true)); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return len(p), nil
|
||||
}
|
||||
|
||||
func (a *app) flags(name string) *flag.FlagSet {
|
||||
fs := flag.NewFlagSet(name, flag.ContinueOnError)
|
||||
fs.SetOutput(a.stderr)
|
||||
fs.Usage = func() { fmt.Fprint(a.stderr, usageText) }
|
||||
return fs
|
||||
}
|
||||
|
||||
func (a *app) policyFlags(name string) *flag.FlagSet {
|
||||
fs := a.flags(name)
|
||||
fs.StringVar(&a.policyRef, "policy", "", "ref to read "+policyPage+" from (default: origin/main)")
|
||||
return fs
|
||||
}
|
||||
|
||||
// parse parses a subcommand's flags; positional arguments are refused, since
|
||||
// none is defined and a stray one is a typo for a flag.
|
||||
func parse(fs *flag.FlagSet, args []string) error {
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if fs.NArg() > 0 {
|
||||
return fmt.Errorf("unexpected argument %q", fs.Arg(0))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (a *app) fail(err error) int {
|
||||
fmt.Fprintf(a.stderr, "backport-batch: %v\n", err)
|
||||
return 2
|
||||
}
|
||||
|
||||
func (a *app) warn(format string, args ...any) {
|
||||
fmt.Fprintf(a.stderr, "warning: "+format+"\n", args...)
|
||||
}
|
||||
|
||||
// verdict is what the policy says about one original for the line.
|
||||
type verdict struct {
|
||||
o *original
|
||||
rule int // the rule that keeps it out, 0 when it qualifies
|
||||
reason string
|
||||
// commits are the commits the original brings, one pick each, and changes
|
||||
// the files it changes; both are read only for what passes rules 2 and 3,
|
||||
// and by load for anything else.
|
||||
commits []string
|
||||
changes []fileChange
|
||||
// onBatch is the commit of the open batch pull request that picks it.
|
||||
onBatch string
|
||||
// label is the maintainer override the original carries, if any.
|
||||
label string
|
||||
}
|
||||
|
||||
// batch is everything read about a line before anything is written.
|
||||
type batch struct {
|
||||
line line
|
||||
newer *line
|
||||
lineHead string
|
||||
mainHead string
|
||||
// newerHead is the newer line's head, and policyAt the commit the
|
||||
// policy was read from.
|
||||
newerHead string
|
||||
policyAt string
|
||||
base string
|
||||
other []string
|
||||
verdicts []*verdict
|
||||
pr *pullRequest
|
||||
prHead string
|
||||
// picks is what the open batch pull request carries beyond the line.
|
||||
picks *history
|
||||
// src is the line's record and where it was found, and drops its drops
|
||||
// that name an original of this batch.
|
||||
src recordSource
|
||||
drops map[int]droppedEntry
|
||||
// lineHist and newerHist are what the line and the newer line carry,
|
||||
// read once for every original judged.
|
||||
lineHist, newerHist *history
|
||||
// start is when the line started under the policy, zero for its fork point.
|
||||
start time.Time
|
||||
// textUntil is the batch-only day: what a commit's text says counts as
|
||||
// evidence only on commits from before it.
|
||||
textUntil time.Time
|
||||
}
|
||||
|
||||
// fetch moves to the top of the working tree first: every path the batch
|
||||
// hands git is relative to it.
|
||||
func fetch() error {
|
||||
top, err := git("rev-parse", "--show-toplevel")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.Chdir(strings.TrimSpace(top)); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = git("fetch", "--quiet", "--end-of-options", remote)
|
||||
return err
|
||||
}
|
||||
|
||||
// policy reads the policy and says from which commit.
|
||||
func (a *app) policy(mainHead string) (*policy, string, error) {
|
||||
at := mainHead
|
||||
if a.policyRef != "" {
|
||||
if strings.HasPrefix(a.policyRef, "-") {
|
||||
return nil, "", fmt.Errorf("--policy %q is not a ref", a.policyRef)
|
||||
}
|
||||
var err error
|
||||
if at, err = resolve(a.policyRef); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
}
|
||||
pol, err := readPolicy(at)
|
||||
return pol, at, err
|
||||
}
|
||||
|
||||
// prepare reads the line, main and the open batch pull request, each pinned to
|
||||
// one commit, and judges every original by the policy.
|
||||
func (a *app) prepare(l line) (*batch, error) {
|
||||
b := &batch{line: l}
|
||||
var err error
|
||||
if b.mainHead, err = resolve("refs/remotes/" + remote + "/main"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pol, policyAt, err := a.policy(b.mainHead)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lines, err := batchLines(pol.released)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.newer, err = newerLine(l, lines); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.lineHead, err = resolve("refs/remotes/" + remote + "/" + l.branch()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.start, b.textUntil, b.policyAt = pol.starts[l.branch()], pol.batchOnly, policyAt
|
||||
out, err := git("merge-base", "--end-of-options", b.mainHead, b.lineHead)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s shares no history with main: %w", l.branch(), err)
|
||||
}
|
||||
b.base = strings.TrimSpace(out)
|
||||
merges, other, err := mainMerges(b.base, b.mainHead)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.other = other
|
||||
if b.lineHist, err = readHistory(b.lineHead, b.mainHead, b.textUntil); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.newer != nil {
|
||||
if b.newerHead, err = resolve("refs/remotes/" + remote + "/" + b.newer.branch()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.newerHist, err = readHistory(b.newerHead, b.mainHead, b.textUntil); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
labels, err := overrides(a.gh)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
prs, err := a.gh.openPulls(l.branch())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pr, warnings, err := batchPull(a.gh, prs, l)
|
||||
for _, w := range warnings {
|
||||
a.warn("%s", w)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if pr != nil {
|
||||
b.pr = pr
|
||||
if b.prHead, err = batchHead(pr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.picks, err = readPicks(b.prHead, b.lineHead, b.textUntil); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if b.src, err = findLineRecord(a.gh, l, pr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.drops, err = b.bindDrops(merges); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, o := range merges {
|
||||
v := &verdict{o: o, label: labels[o.number]}
|
||||
if err := b.judge(v, b.drops); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.verdicts = append(b.verdicts, v)
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// judge judges v again under drops, and finds it on the open batch when the
|
||||
// policy takes it.
|
||||
func (b *batch) judge(v *verdict, drops map[int]droppedEntry) error {
|
||||
j, err := judge(v.o, v.label, b.start, b.lineHist, b.newer, b.newerHist, drops)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*v = *j
|
||||
if v.rule == 0 && b.picks != nil {
|
||||
v.onBatch, _ = b.picks.found(v.o, v.commits)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// overrides maps each pull request merged into main that carries a backport/
|
||||
// label to the one that decides; backport/skip wins over backport/include.
|
||||
func overrides(gh github) (map[int]string, error) {
|
||||
labels := map[int]string{}
|
||||
for _, label := range []string{labelInclude, labelSkip} {
|
||||
numbers, err := gh.labelled(label)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, n := range numbers {
|
||||
labels[n] = label
|
||||
}
|
||||
}
|
||||
return labels, nil
|
||||
}
|
||||
|
||||
// judge applies rules 2, 3, 4, 1 and 6, in the order of what each costs to
|
||||
// read; the first that keeps the original out is the one reported. A line that
|
||||
// started under the policy after it branched takes nothing merged before that
|
||||
// start. backport/skip keeps an original out whatever the rules say, and
|
||||
// backport/include lifts the start and rules 2 to 4, but not what the line or
|
||||
// the newer line carries. A maintainer's drop keeps out what the line does
|
||||
// not carry, include or not.
|
||||
func judge(o *original, label string, start time.Time, lineHist *history, newer *line, newerHist *history, drops map[int]droppedEntry) (*verdict, error) {
|
||||
v := &verdict{o: o, label: label}
|
||||
if label == labelSkip {
|
||||
v.rule, v.reason = ruleSkipLabel, labelSkip
|
||||
return v, nil
|
||||
}
|
||||
if o.merged.Before(start) && label != labelInclude {
|
||||
v.rule, v.reason = ruleBeforeStart, "merged "+o.merged.Format(time.DateOnly)
|
||||
return v, nil
|
||||
}
|
||||
if r := typeRule(o.title); r != "" && label != labelInclude {
|
||||
v.rule, v.reason = ruleType, r
|
||||
return v, nil
|
||||
}
|
||||
if r := breakingRule(o.title, o.body); r != "" && label != labelInclude {
|
||||
v.rule, v.reason = ruleBreaking, r
|
||||
return v, nil
|
||||
}
|
||||
var err error
|
||||
if v.changes, err = changedFiles(o.merge+"^1", o.merge); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if label != labelInclude {
|
||||
r, err := touchesRule(o, v.changes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r != "" {
|
||||
v.rule, v.reason = ruleTouches, r
|
||||
return v, nil
|
||||
}
|
||||
}
|
||||
if v.commits, err = prCommits(o); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
how, err := lineHist.carries(o, v.commits)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if how != "" {
|
||||
v.rule, v.reason = ruleOnLine, how
|
||||
return v, nil
|
||||
}
|
||||
if d, ok := drops[o.number]; ok {
|
||||
v.rule, v.reason = ruleDropped, fmt.Sprintf("dropped on #%d: %s", d.On, d.Reason)
|
||||
return v, nil
|
||||
}
|
||||
if newer != nil {
|
||||
how, err := newerHist.carries(o, v.commits)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if how == "" {
|
||||
v.rule, v.reason = ruleNewer, "not on "+newer.branch()+" yet"
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
func (b *batch) qualifying() []*verdict {
|
||||
var out []*verdict
|
||||
for _, v := range b.verdicts {
|
||||
if v.rule == 0 {
|
||||
out = append(out, v)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// load reads what the rules did not need to read for an original: its
|
||||
// commits and the files it changes.
|
||||
func (b *batch) load(v *verdict) error {
|
||||
var err error
|
||||
if v.changes == nil {
|
||||
if v.changes, err = changedFiles(v.o.merge+"^1", v.o.merge); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if v.commits == nil {
|
||||
v.commits, err = prCommits(v.o)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// findMerge is the original merged as merge commit sha, or nil.
|
||||
func (b *batch) findMerge(sha string) *verdict {
|
||||
for _, v := range b.verdicts {
|
||||
if v.o.merge == sha {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (b *batch) find(number int) *verdict {
|
||||
for _, v := range b.verdicts {
|
||||
if v.o.number == number {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,225 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// notifyMarker starts the one comment notify keeps on a batch pull request.
|
||||
const notifyMarker = "<!-- backport-batch:notify -->"
|
||||
|
||||
// stalledAfter is how many days a batch stays open before notify calls it
|
||||
// stalled: the shorter gap between two scheduled runs, Monday to Thursday, so
|
||||
// a batch the previous run opened or continued is stalled at the next one.
|
||||
const stalledAfter = 3
|
||||
|
||||
var mentionRE = regexp.MustCompile(`^@?([A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?(?:/[A-Za-z0-9_.-]+)?)$`)
|
||||
|
||||
// notify reports what waits on the open batch pull request of a line for a
|
||||
// maintainer: a batch open since before the previous run, the conflicts it
|
||||
// deferred, and the originals that change workflows. One line per finding goes
|
||||
// to stdout, for the run's summary. --push writes them into the comment, one
|
||||
// per pull request, updated in place on every run.
|
||||
func (a *app) notify(args []string) int {
|
||||
fs := a.flags("notify")
|
||||
lineArg := fs.String("line", "", "release line, release-X.Y")
|
||||
mention := fs.String("mention", "", "team or person the comment mentions, org/team or login, with or without @")
|
||||
push := fs.Bool("push", false, "write the comment")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
l, err := parseLine(*lineArg)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if *mention != "" {
|
||||
m := mentionRE.FindStringSubmatch(*mention)
|
||||
if m == nil {
|
||||
return a.fail(fmt.Errorf("--mention %q is not @login or @org/team", *mention))
|
||||
}
|
||||
*mention = "@" + m[1]
|
||||
}
|
||||
prs, err := a.gh.openPulls(l.branch())
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
pr, warnings, err := batchPull(a.gh, prs, l)
|
||||
for _, w := range warnings {
|
||||
a.warn("%s", w)
|
||||
}
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
// With no batch open there is nothing to report on and nowhere to write,
|
||||
// but a record nobody can read is said all the same.
|
||||
src, err := findLineRecord(a.gh, l, pr)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if c := src.closingOpen(); c != nil {
|
||||
return a.fail(fmt.Errorf("batch pull request #%d is being closed; backport-batch run --line %s --push finishes it", c.PR, l.branch()))
|
||||
}
|
||||
if pr == nil {
|
||||
return 0
|
||||
}
|
||||
opened, err := branchDate(pr.Head.Ref)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
deferred := map[int]deferredEntry{}
|
||||
if src.open {
|
||||
deferred = src.rec.deferred()
|
||||
}
|
||||
var conflicts, waiting []int
|
||||
for n, e := range deferred {
|
||||
switch {
|
||||
case e.Workflows:
|
||||
waiting = append(waiting, n)
|
||||
case e.Reason == "":
|
||||
conflicts = append(conflicts, n)
|
||||
}
|
||||
}
|
||||
slices.Sort(conflicts)
|
||||
slices.Sort(waiting)
|
||||
|
||||
today := a.now().UTC().Truncate(24 * time.Hour)
|
||||
days := int(today.Sub(opened).Hours() / 24)
|
||||
stalled := days >= stalledAfter
|
||||
var text []string
|
||||
if stalled {
|
||||
fmt.Fprintf(a.stdout, "%s: batch pull request #%d has been open since %s, %d days\n", l.branch(), pr.Number, opened.Format(time.DateOnly), days)
|
||||
text = append(text, fmt.Sprintf("This batch has been open since %s, %d days, through at least one scheduled run. Merge it, or close it and the next run starts a new one; until then each run adds what the rules take to it.", opened.Format(time.DateOnly), days))
|
||||
}
|
||||
if len(conflicts) > 0 {
|
||||
fmt.Fprintf(a.stdout, "%s: %d deferred conflicts on #%d wait for a maintainer\n", l.branch(), len(conflicts), pr.Number)
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "These conflicts wait for a maintainer, as [resolving a conflict](%sresolve-conflict.md) describes; unresolved, each goes to the next batch:\n", tasksURL)
|
||||
for _, n := range conflicts {
|
||||
fmt.Fprintf(&b, "\n- %s", prLink(n))
|
||||
}
|
||||
text = append(text, b.String())
|
||||
}
|
||||
if len(waiting) > 0 {
|
||||
fmt.Fprintf(a.stdout, "%s: %d originals on #%d change %s and wait for a maintainer\n", l.branch(), len(waiting), pr.Number, workflowsDir)
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "These change `%s`, which the batch's token may not push, and wait for a maintainer to bring them over, as [changes to workflows](%s#changes-to-workflows) describes:\n", workflowsDir, policyURL)
|
||||
for _, n := range waiting {
|
||||
fmt.Fprintf(&b, "\n- %s", prLink(n))
|
||||
}
|
||||
text = append(text, b.String())
|
||||
}
|
||||
if !*push {
|
||||
return 0
|
||||
}
|
||||
return a.comment(pr, *mention, today, text)
|
||||
}
|
||||
|
||||
// comment writes what waits into the one notify comment on the batch pull
|
||||
// request: it edits the comment if there is one, and opens it only when
|
||||
// something waits. Only a comment by an author the batch trusts is taken for
|
||||
// it, so a look-alike someone else wrote is neither edited nor believed.
|
||||
func (a *app) comment(pr *pullRequest, mention string, today time.Time, text []string) int {
|
||||
comments, err := a.gh.comments(pr.Number)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
var ours *issueComment
|
||||
for i := range comments {
|
||||
c := &comments[i]
|
||||
if !strings.HasPrefix(c.Body, notifyMarker) {
|
||||
continue
|
||||
}
|
||||
why, err := trustedAuthor(a.gh, c.User.Login, c.User.Type)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if why == "" {
|
||||
ours = c
|
||||
}
|
||||
}
|
||||
if ours == nil && len(text) == 0 {
|
||||
return 0
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString(notifyMarker + "\n")
|
||||
if mention != "" && len(text) > 0 {
|
||||
b.WriteString(mention + "\n\n")
|
||||
}
|
||||
if len(text) == 0 {
|
||||
text = []string{"Nothing waits on a maintainer."}
|
||||
}
|
||||
b.WriteString(strings.Join(text, "\n\n"))
|
||||
fmt.Fprintf(&b, "\n\n<sub>Updated by the backport batch on %s.</sub>\n", today.Format(time.DateOnly))
|
||||
f, err := os.CreateTemp("", "backport-batch-comment-*.md")
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if _, err := f.WriteString(b.String()); err != nil {
|
||||
f.Close()
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if err := a.stillOpen(pr.Number); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if ours != nil {
|
||||
err = a.gh.editComment(ours.ID, f.Name())
|
||||
} else {
|
||||
err = a.gh.createComment(pr.Number, f.Name())
|
||||
}
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// lines prints the lines the batch runs for, newest first, in the order the
|
||||
// policy has them processed.
|
||||
func (a *app) lines(args []string) int {
|
||||
fs := a.policyFlags("lines")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
if err := fetch(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
mainHead, err := resolve("refs/remotes/" + remote + "/main")
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
pol, _, err := a.policy(mainHead)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
lines, err := batchLines(pol.released)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
for _, l := range slices.Backward(lines) {
|
||||
fmt.Fprintln(a.stdout, l.branch())
|
||||
}
|
||||
return 0
|
||||
}
|
||||
@@ -0,0 +1,336 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// notifyPR is an open batch pull request for release-1.1 on the branch opened
|
||||
// on date, YYYYMMDD, with body.
|
||||
func notifyPR(date, body string) pullRequest {
|
||||
var pr pullRequest
|
||||
pr.Number, pr.Body = 90, body
|
||||
pr.User.Login, pr.User.Type = "maintainer", "User"
|
||||
pr.Head.Ref, pr.Head.SHA = "backport-batch/release-1.1-"+date, strings.Repeat("a", 40)
|
||||
pr.Head.Repo = &struct {
|
||||
FullName string `json:"full_name"`
|
||||
}{upstreamRepo}
|
||||
pr.Base.Ref, pr.Base.Repo.FullName = "release-1.1", upstreamRepo
|
||||
return pr
|
||||
}
|
||||
|
||||
// notifyRun runs notify on 2026-10-26 with gh faked.
|
||||
func notifyRun(gh *fakeGH, args ...string) (int, string, string) {
|
||||
var stdout, stderr bytes.Buffer
|
||||
a := &app{gh: gh, stdout: &stdout, stderr: &stderr, now: func() time.Time { return time.Date(2026, 10, 26, 6, 23, 0, 0, time.UTC) }}
|
||||
code := a.main(append([]string{"notify", "--line", "release-1.1"}, args...))
|
||||
return code, stdout.String(), stderr.String()
|
||||
}
|
||||
|
||||
var notifyLine, _ = parseLine("release-1.1")
|
||||
|
||||
// A batch open since before the previous run gets one comment, mentioning
|
||||
// the team, and keeps that one comment however many runs see it.
|
||||
func TestNotifyStalled(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
date string
|
||||
stalled bool
|
||||
}{
|
||||
{"20261026", false},
|
||||
{"20261024", false},
|
||||
{"20261023", true},
|
||||
{"20261019", true},
|
||||
} {
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR(tc.date, "")}}
|
||||
code, stdout, stderr := notifyRun(gh, "--mention", "cozystack/maintainers", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("%s: notify = %d\n%s", tc.date, code, stderr)
|
||||
}
|
||||
if got := strings.Contains(stdout, "release-1.1: batch pull request #90 has been open since"); got != tc.stalled {
|
||||
t.Errorf("%s: stalled = %v, want %v:\n%s", tc.date, got, tc.stalled, stdout)
|
||||
}
|
||||
if got := len(gh.issueComments); got != map[bool]int{true: 1, false: 0}[tc.stalled] {
|
||||
t.Errorf("%s: %d comments, want one only when stalled", tc.date, got)
|
||||
}
|
||||
if !tc.stalled {
|
||||
continue
|
||||
}
|
||||
c := gh.issueComments[0].Body
|
||||
if !strings.HasPrefix(c, notifyMarker+"\n@cozystack/maintainers\n") || !strings.Contains(c, "open since") {
|
||||
t.Errorf("%s: the comment does not mention the team and say why:\n%s", tc.date, c)
|
||||
}
|
||||
for range 2 {
|
||||
if code, _, stderr := notifyRun(gh, "--mention", "@cozystack/maintainers", "--push"); code != 0 {
|
||||
t.Fatalf("notify again = %d\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
if len(gh.issueComments) != 1 {
|
||||
t.Errorf("%s: %d comments after three runs, want the one updated", tc.date, len(gh.issueComments))
|
||||
}
|
||||
}
|
||||
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261019", "")}}
|
||||
if code, _, _ := notifyRun(gh, "--push"); code != 0 || strings.Contains(gh.issueComments[0].Body, "@") {
|
||||
t.Errorf("notify with no team = %d and wrote %q; want no mention", code, gh.issueComments[0].Body)
|
||||
}
|
||||
gh = &fakeGH{prs: []pullRequest{notifyPR("20261019", "")}}
|
||||
if code, stdout, _ := notifyRun(gh); code != 0 || stdout == "" || len(gh.issueComments) != 0 {
|
||||
t.Errorf("notify without --push = %d, %q and %d comments; want the line and no comment", code, stdout, len(gh.issueComments))
|
||||
}
|
||||
}
|
||||
|
||||
// The comment lists the deferred conflicts for a maintainer, and nothing
|
||||
// --defer held out.
|
||||
func TestNotifyConflicts(t *testing.T) {
|
||||
conflict := pick{v: &verdict{o: &original{number: 6}}, files: []string{"app/six.txt"}}
|
||||
held := pick{v: &verdict{o: &original{number: 7}}, reason: "fails the e2e"}
|
||||
body := "text\n\n" + deferredBlock(notifyLine, []pick{conflict}, []pick{held}, nil) + "\n"
|
||||
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261026", body)}}
|
||||
code, stdout, stderr := notifyRun(gh, "--mention", "@cozystack/maintainers", "--push")
|
||||
if code != 0 || !strings.Contains(stdout, "release-1.1: 1 deferred conflicts on #90 wait for a maintainer") {
|
||||
t.Fatalf("notify = %d, %q\n%s", code, stdout, stderr)
|
||||
}
|
||||
if len(gh.issueComments) != 1 {
|
||||
t.Fatalf("%d comments, want one", len(gh.issueComments))
|
||||
}
|
||||
c := gh.issueComments[0].Body
|
||||
if !strings.Contains(c, "@cozystack/maintainers") || !strings.Contains(c, "[#6](") || !strings.Contains(c, "resolve-conflict.md") || strings.Contains(c, "#7") {
|
||||
t.Errorf("the comment does not list #6 alone for a maintainer:\n%s", c)
|
||||
}
|
||||
}
|
||||
|
||||
// Only a comment by an author the batch trusts is taken for the notify
|
||||
// comment: a look-alike is neither edited nor counted.
|
||||
func TestNotifyKeepsToItsOwnComment(t *testing.T) {
|
||||
look := issueComment{ID: 7, Body: notifyMarker + "\nplanted"}
|
||||
look.User.Login, look.User.Type = "someone", "User"
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261019", "")}, issueComments: []issueComment{look}}
|
||||
if code, _, stderr := notifyRun(gh, "--push"); code != 0 {
|
||||
t.Fatalf("notify = %d\n%s", code, stderr)
|
||||
}
|
||||
if len(gh.issueComments) != 2 || gh.issueComments[0].Body != look.Body {
|
||||
t.Errorf("comments = %+v; want the look-alike untouched and one of ours", gh.issueComments)
|
||||
}
|
||||
|
||||
// Ours says so once nothing waits any more, and calls nobody for it.
|
||||
gh.prs = []pullRequest{notifyPR("20261026", "")}
|
||||
if code, _, _ := notifyRun(gh, "--mention", "@cozystack/maintainers", "--push"); code != 0 || len(gh.issueComments) != 2 ||
|
||||
!strings.Contains(gh.issueComments[1].Body, "Nothing waits") || strings.Contains(gh.issueComments[1].Body, "@cozystack") {
|
||||
t.Errorf("comments = %+v; want ours updated to say nothing waits, mentioning nobody", gh.issueComments)
|
||||
}
|
||||
}
|
||||
|
||||
// The scheduled batch writes as its bot: notify keeps to the comment that bot
|
||||
// wrote, on a pull request it opened, and to no other bot's.
|
||||
func TestNotifyFindsTheBotsComment(t *testing.T) {
|
||||
const bot = "cozystack-ci[bot]"
|
||||
ours := issueComment{ID: 7, Body: notifyMarker + "\nearlier"}
|
||||
ours.User.Login, ours.User.Type = bot, "Bot"
|
||||
other := issueComment{ID: 8, Body: notifyMarker + "\nplanted"}
|
||||
other.User.Login, other.User.Type = "renovate[bot]", "Bot"
|
||||
// The bot writes the record into every body it opens.
|
||||
pr := notifyPR("20261019", deferredBlock(notifyLine, nil, nil, nil))
|
||||
pr.User.Login, pr.User.Type = bot, "Bot"
|
||||
gh := &fakeGH{prs: []pullRequest{pr}, issueComments: []issueComment{ours, other}, bot: bot}
|
||||
if code, stdout, stderr := notifyRun(gh, "--push"); code != 0 || !strings.Contains(stdout, "has been open since") {
|
||||
t.Fatalf("notify = %d, %q\n%s", code, stdout, stderr)
|
||||
}
|
||||
if len(gh.issueComments) != 2 || !strings.Contains(gh.issueComments[0].Body, "has been open since") || gh.issueComments[1].Body != other.Body {
|
||||
t.Errorf("comments = %+v; want the bot's own comment edited and the other bot's untouched", gh.issueComments)
|
||||
}
|
||||
if code, _, stderr := notifyRun(&fakeGH{prs: []pullRequest{pr}, bot: "cozystack-ci"}, "--push"); code != 2 || !strings.Contains(stderr, batchBotEnv) {
|
||||
t.Errorf("a %s that is no bot login = %d (%s), want a refusal", batchBotEnv, code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A permission GitHub does not answer fails notify before it writes: read as
|
||||
// a refusal, it would pass over a person's batch, or over the comment notify
|
||||
// keeps and post a second one.
|
||||
func TestNotifyFailsOnAnUnreadablePermission(t *testing.T) {
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261019", "")}, permErr: errString("HTTP 502")}
|
||||
if code, stdout, stderr := notifyRun(gh, "--push"); code != 2 || stdout != "" || len(gh.issueComments) != 0 || !strings.Contains(stderr, "HTTP 502") {
|
||||
t.Errorf("notify on a person's batch = %d, %q, %d comments (%s); want 2 and nothing written", code, stdout, len(gh.issueComments), stderr)
|
||||
}
|
||||
|
||||
const bot = "cozystack-ci[bot]"
|
||||
pr := notifyPR("20261019", deferredBlock(notifyLine, nil, nil, nil))
|
||||
pr.User.Login, pr.User.Type = bot, "Bot"
|
||||
ours := issueComment{ID: 7, Body: notifyMarker + "\nearlier"}
|
||||
ours.User.Login, ours.User.Type = "maintainer", "User"
|
||||
gh = &fakeGH{prs: []pullRequest{pr}, issueComments: []issueComment{ours}, bot: bot, permErr: errString("HTTP 502")}
|
||||
if code, _, stderr := notifyRun(gh, "--push"); code != 2 || len(gh.issueComments) != 1 || gh.issueComments[0].Body != ours.Body || !strings.Contains(stderr, "HTTP 502") {
|
||||
t.Errorf("notify with a person's comment = %d, comments %+v (%s); want 2 and the comment untouched", code, gh.issueComments, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNotifyRefuses(t *testing.T) {
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261019", "")}}
|
||||
for _, args := range [][]string{
|
||||
{"--mention", "@a b"},
|
||||
{"--mention", "@org/team/x"},
|
||||
{"--mention", "@-x"},
|
||||
} {
|
||||
if code, _, _ := notifyRun(gh, args...); code != 2 {
|
||||
t.Errorf("notify %q = %d, want 2", args, code)
|
||||
}
|
||||
}
|
||||
if code, stdout, _ := notifyRun(&fakeGH{}, "--push"); code != 0 || stdout != "" {
|
||||
t.Errorf("notify with no batch open = %d, %q; want nothing", code, stdout)
|
||||
}
|
||||
pr := notifyPR("20261019", "")
|
||||
pr.User.Login = "someone"
|
||||
if code, stdout, _ := notifyRun(&fakeGH{prs: []pullRequest{pr}}, "--push"); code != 0 || stdout != "" {
|
||||
t.Errorf("notify on a batch someone untrusted opened = %d, %q; want nothing", code, stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// notify reads the line's record the way the other commands do, and stays
|
||||
// silent with no batch open, but a record it cannot read, or one that says
|
||||
// its open batch is being closed, stops it.
|
||||
func TestNotifyValidatesTheRecord(t *testing.T) {
|
||||
sha := strings.Repeat("a", 40)
|
||||
closingRecord := strings.Replace(v2(""), `"closing":null`, `"closing":{"pr":91,"branch":"backport-batch/release-1.1-20261002","head":"`+sha+`"}`, 1)
|
||||
block := func(json string) string { return "text\n\n" + deferredFence + "\n" + json + "\n```\n" }
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
pr pullRequest
|
||||
code int
|
||||
quiet bool
|
||||
}{
|
||||
{"a clean closed record", recordPR(91, "closed", "maintainer", block(v2(""))), 0, true},
|
||||
{"a malformed record", recordPR(91, "closed", "maintainer", block(`{"line":`)), 2, false},
|
||||
{"a closed record that was closing", recordPR(91, "closed", "maintainer", block(closingRecord)), 0, true},
|
||||
{"an open record that is closing", recordPR(91, "open", "maintainer", block(closingRecord)), 2, false},
|
||||
} {
|
||||
gh := &fakeGH{prs: []pullRequest{tc.pr}}
|
||||
code, stdout, stderr := notifyRun(gh, "--push")
|
||||
if code != tc.code || tc.quiet && (stdout != "" || stderr != "") || len(gh.issueComments) != 0 {
|
||||
t.Errorf("%s: notify = %d, %q, %q, %d comments; want %d and nothing written", tc.name, code, stdout, stderr, len(gh.issueComments), tc.code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The maintainer's example: with 1.6 and 1.7 released, release-1.8 joins them
|
||||
// when it branches, and 1.8.0 drops 1.6.
|
||||
func TestSupportedLines(t *testing.T) {
|
||||
l := func(minor int) line { return line{1, minor} }
|
||||
cases := []struct {
|
||||
name string
|
||||
all []line
|
||||
shipped []line
|
||||
released int
|
||||
want []line
|
||||
}{
|
||||
{"two released", []line{l(5), l(6), l(7)}, []line{l(5), l(6), l(7)}, 2, []line{l(6), l(7)}},
|
||||
{"release-1.8 branched", []line{l(5), l(6), l(7), l(8)}, []line{l(5), l(6), l(7)}, 2, []line{l(6), l(7), l(8)}},
|
||||
{"1.8.0 shipped", []line{l(5), l(6), l(7), l(8)}, []line{l(5), l(6), l(7), l(8)}, 2, []line{l(7), l(8)}},
|
||||
{"three released", []line{l(5), l(6), l(7), l(8)}, []line{l(5), l(6), l(7)}, 3, []line{l(5), l(6), l(7), l(8)}},
|
||||
{"one released", []line{l(6), l(7), l(8)}, []line{l(6), l(7)}, 1, []line{l(7), l(8)}},
|
||||
{"a line between releases that never shipped", []line{l(5), l(6), l(7)}, []line{l(5), l(7)}, 2, []line{l(5), l(7)}},
|
||||
{"nothing shipped yet", []line{l(0), l(1)}, nil, 2, []line{l(0), l(1)}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
shipped := map[line]bool{}
|
||||
for _, s := range tc.shipped {
|
||||
shipped[s] = true
|
||||
}
|
||||
if got := supportedLines(tc.all, shipped, tc.released); !slices.Equal(got, tc.want) {
|
||||
t.Errorf("%s: supportedLines = %v, want %v", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// lines prints the lines the batch runs for, newest first, which is the order
|
||||
// the scheduled run takes them in: the newest released lines, as many as the
|
||||
// policy says, and a line branched since, whose stable release drops the
|
||||
// oldest.
|
||||
func TestLines(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.release("release-0.9")
|
||||
f.tag("v0.9.0")
|
||||
f.lines()
|
||||
f.release("release-1.2")
|
||||
f.release("release-1.2.0-rc.1")
|
||||
f.tag("v1.2.0-rc.1")
|
||||
lines := func() string {
|
||||
t.Helper()
|
||||
code, stdout, stderr := f.exec(nil, "lines")
|
||||
if code != 0 {
|
||||
t.Fatalf("lines = %d (%s)", code, stderr)
|
||||
}
|
||||
return stdout
|
||||
}
|
||||
if got := lines(); got != "release-1.2\nrelease-1.1\nrelease-1.0\n" {
|
||||
t.Errorf("with release-1.2 at its first rc, lines = %q; want it before the two released lines", got)
|
||||
}
|
||||
f.setPolicy(releasedKey + ": 3\n")
|
||||
if got := lines(); got != "release-1.2\nrelease-1.1\nrelease-1.0\nrelease-0.9\n" {
|
||||
t.Errorf("with three released lines, lines = %q; want release-0.9 too", got)
|
||||
}
|
||||
f.setPolicy("")
|
||||
f.tag("v1.2.0")
|
||||
if got := lines(); got != "release-1.2\nrelease-1.1\n" {
|
||||
t.Errorf("once v1.2.0 ships, lines = %q; want release-1.0 dropped", got)
|
||||
}
|
||||
f.setPolicy(releasedKey + ": 3\n")
|
||||
if got := lines(); got != "release-1.2\nrelease-1.1\nrelease-1.0\n" {
|
||||
t.Errorf("once v1.2.0 ships, with three released lines, lines = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A scheduled run with nothing to pick for a line succeeds without pushing or
|
||||
// opening anything.
|
||||
func TestRunWithNothingToPick(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "feat(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if code != 0 || !strings.Contains(stderr, "nothing to push") {
|
||||
t.Errorf("run with nothing to pick = %d, want 0:\n%s", code, stderr)
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != "" || len(gh.created) != 0 {
|
||||
t.Errorf("pushed %q and opened %q with nothing picked", f.remoteHead(batchBranch1), gh.created)
|
||||
}
|
||||
}
|
||||
|
||||
// notify writes its comment only after GitHub says the batch is still open,
|
||||
// so a batch closed while notify reads it is not written to, whether the
|
||||
// comment would be new or an update of its own.
|
||||
func TestNotifyWritesNothingOnABatchClosedMeanwhile(t *testing.T) {
|
||||
for _, update := range []bool{false, true} {
|
||||
t.Run(map[bool]string{false: "a new comment", true: "an update"}[update], func(t *testing.T) {
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261023", recordBody(5))}}
|
||||
if update {
|
||||
ours := issueComment{ID: 7, Body: notifyMarker + "\nearlier"}
|
||||
ours.User.Login, ours.User.Type = "maintainer", "User"
|
||||
gh.issueComments = []issueComment{ours}
|
||||
}
|
||||
gh.after = map[string]func(){"comments": func() { gh.find(90).State = "closed" }}
|
||||
code, _, stderr := notifyRun(gh, "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "batch pull request #90 is closed now; nothing is written to it") || len(gh.closedWrites) != 0 || update && gh.issueComments[0].Body != notifyMarker+"\nearlier" {
|
||||
t.Errorf("notify = %d, closed writes %q, comments %+v; want 2 and nothing written:\n%s", code, gh.closedWrites, gh.issueComments, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,140 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// pushed builds and pushes a batch for #5 and returns its head.
|
||||
func pushed(t *testing.T, f *fixture) string {
|
||||
t.Helper()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
if code, _, stderr := f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
head := f.git("rev-parse", "HEAD")
|
||||
f.git("switch", "-q", "main")
|
||||
return head
|
||||
}
|
||||
|
||||
// --onto continues the pushed head and nothing else: it keeps what the batch
|
||||
// carries and picks only what is new.
|
||||
func TestOntoContinues(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
head := pushed(t, f)
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
code, stdout, stderr := f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--onto", head, "--date", "2026-10-29")
|
||||
if code != 2 || !strings.Contains(stderr, "not the head of backport-batch/release-1.1-20261029") {
|
||||
t.Errorf("--onto with another date = %d (%s), want a refusal", code, stderr)
|
||||
}
|
||||
f.now = f.now.AddDate(0, 0, 3)
|
||||
code, stdout, stderr = f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--onto", head)
|
||||
if code != 0 {
|
||||
t.Fatalf("run --onto = %d\n%s", code, stderr)
|
||||
}
|
||||
if !strings.Contains(stderr, "title: fix(backport): batch for release-1.1, 2026-10-26\n") {
|
||||
t.Errorf("the continued batch is not titled for the date it opened on:\n%s", stderr)
|
||||
}
|
||||
if got := f.subjects(head, "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
|
||||
t.Errorf("picked %q on top of the batch, want only #6", got)
|
||||
}
|
||||
if !descends(t, head, f.git("rev-parse", "HEAD")) {
|
||||
t.Error("the new head does not descend from --onto")
|
||||
}
|
||||
if listed, _, _ := strings.Cut(stdout, "### Skipped"); !strings.Contains(listed, "[#5]") || !strings.Contains(listed, "[#6]") {
|
||||
t.Errorf("the body does not list what the batch carries and what it picked:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// A head that is not the one the remote has now is refused, so work pushed
|
||||
// since cannot be dropped.
|
||||
func TestOntoRefusesAStaleHead(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
head := pushed(t, f)
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.write("app/fold.txt", "folded\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "fold")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
|
||||
code, _, stderr := f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--onto", head)
|
||||
if code != 2 || !strings.Contains(stderr, "is not the head of any batch branch") {
|
||||
t.Errorf("run --onto a stale head = %d (%s), want a refusal", code, stderr)
|
||||
}
|
||||
for _, bad := range []string{head[:12], "-" + head[1:], strings.ToUpper(head), "HEAD", "--help"} {
|
||||
if code, _, _ := f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--onto", bad); code != 2 {
|
||||
t.Errorf("--onto %q = %d, want 2", bad, code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A rebuilt branch that does not descend from what the remote has is never
|
||||
// pushed over it.
|
||||
func TestPushRefusesANonFastForward(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
head := pushed(t, f)
|
||||
f.git("switch", "-q", "--detach", head)
|
||||
f.write("app/fold.txt", "folded\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "fold")
|
||||
folded := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
|
||||
code, _, stderr := f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--push")
|
||||
// git would refuse too, but only after trying; the tool refuses first.
|
||||
if code != 2 || !strings.Contains(stderr, "refusing a non-fast-forward push") {
|
||||
t.Errorf("a fresh rebuild pushed over the batch = %d (%s), want a refusal", code, stderr)
|
||||
}
|
||||
if got := f.remoteHead(batchBranch1); got != folded {
|
||||
t.Errorf("the remote batch moved to %.12s, want it left at %.12s", got, folded)
|
||||
}
|
||||
}
|
||||
|
||||
// A run that pushed its branch and stopped before opening the pull request
|
||||
// leaves the branch behind. A run the same day builds the batch anew and
|
||||
// refuses to push over it, naming the run that continues it, which then
|
||||
// opens the pull request.
|
||||
func TestOrphanBranchNamesItsRecovery(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
head := pushed(t, f)
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
// A commit picked again gets a new committer date, so a new name.
|
||||
t.Setenv("GIT_COMMITTER_DATE", "2026-10-26T07:00:00Z")
|
||||
code, _, stderr := f.exec(&fakeGH{}, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
recovery := "backport-batch run --line release-1.1 --onto " + head + " --push"
|
||||
if code != 2 || !strings.Contains(stderr, recovery) {
|
||||
t.Fatalf("a same-day run over the left branch = %d (%s), want a refusal naming %q", code, stderr, recovery)
|
||||
}
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, strings.Fields(recovery)[1:]...); code != 0 {
|
||||
t.Fatalf("%s = %d\n%s", recovery, code, stderr)
|
||||
}
|
||||
now := f.remoteHead(batchBranch1)
|
||||
if !descends(t, head, now) || now == head || len(gh.created) != 1 {
|
||||
t.Errorf("the recovery pushed %.12s over %.12s and opened %q; want a fast-forward with #6 and one pull request", now, head, gh.created)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,275 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// overridden builds a line where backport/ labels decide: #5 a feature let in,
|
||||
// #6 a fix kept out, #7 a breaking fix and #8 a migration let in, #9 carrying
|
||||
// both labels, #10 a feature let in that the line already carries, adapted, so
|
||||
// picking it again would conflict rather than come out empty.
|
||||
func overridden(t *testing.T) (*fixture, *fakeGH) {
|
||||
t.Helper()
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "feat(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "fix(app)!: seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
f.pr(8, "fix(platform): eight", "", map[string]string{migrationsDir + "61": "#!/bin/sh\n"})
|
||||
f.pr(9, "fix(app): nine", "", map[string]string{"app/nine.txt": "9\n"})
|
||||
mc10 := f.pr(10, "feat(app): ten", "", map[string]string{"app/ten.txt": "10\n"})
|
||||
f.onLine("release-1.1", "feat(app): ten (#10)\n\n(cherry picked from commit "+mc10+")", map[string]string{"app/ten.txt": "10, adapted\n"})
|
||||
gh := &fakeGH{labels: map[string][]int{
|
||||
labelInclude: {5, 7, 8, 9, 10},
|
||||
labelSkip: {6, 9},
|
||||
}}
|
||||
return f, gh
|
||||
}
|
||||
|
||||
func TestOverrides(t *testing.T) {
|
||||
f, gh := overridden(t)
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
want := []string{"feat(app): five (#5)", "fix(app)!: seven (#7)", "fix(platform): eight (#8)"}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, want) {
|
||||
t.Errorf("picked %q, want %q", got, want)
|
||||
}
|
||||
section, _, _ := strings.Cut(stdout[strings.Index(stdout, "### Maintainer overrides"):], "### Skipped")
|
||||
for n, outcome := range map[int]string{
|
||||
5: "`backport/include`, on this branch",
|
||||
6: "`backport/skip`, left out",
|
||||
7: "`backport/include`, on this branch",
|
||||
8: "`backport/include`, on this branch",
|
||||
9: "`backport/skip`, left out",
|
||||
10: "`backport/include`, skipped by rule 1",
|
||||
} {
|
||||
if !strings.Contains(section, fmt.Sprintf("/pull/%d)", n)) || !strings.Contains(section, outcome) {
|
||||
t.Errorf("the overrides section does not show #%d as %q:\n%s", n, outcome, section)
|
||||
}
|
||||
}
|
||||
if skipped := stdout[strings.Index(stdout, "### Skipped"):]; strings.Contains(skipped, "/pull/6)") || strings.Contains(skipped, "/pull/9)") {
|
||||
t.Errorf("a skipped-by-label original is listed under a rule:\n%s", skipped)
|
||||
}
|
||||
}
|
||||
|
||||
// check holds the line to what the labels decide, and fold refuses an
|
||||
// original kept out.
|
||||
func TestOverridesInCheckAndFold(t *testing.T) {
|
||||
f, gh := overridden(t)
|
||||
code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 1 {
|
||||
t.Fatalf("check = %d, want 1:\n%s", code, stdout)
|
||||
}
|
||||
for _, n := range []int{5, 7, 8} {
|
||||
if !strings.Contains(stdout, fmt.Sprintf("/pull/%d ", n)) {
|
||||
t.Errorf("check does not list #%d:\n%s", n, stdout)
|
||||
}
|
||||
}
|
||||
for _, n := range []int{6, 9, 10} {
|
||||
if strings.Contains(stdout, fmt.Sprintf("/pull/%d ", n)) {
|
||||
t.Errorf("check lists #%d:\n%s", n, stdout)
|
||||
}
|
||||
}
|
||||
|
||||
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+batchBranch1)
|
||||
f.git("fetch", "-q", "origin")
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, "")}
|
||||
f.git("switch", "-q", "--detach", "origin/"+batchBranch1)
|
||||
f.write("app/six.txt", "6\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "port\n\n(cherry picked from commit "+f.git("rev-parse", "origin/main~4")+")")
|
||||
commit := f.git("rev-parse", "HEAD")
|
||||
f.git("switch", "-q", "main")
|
||||
code, stdout, _ = f.exec(gh, "fold", "--line", "release-1.1", "--original", "6", "--commit", commit)
|
||||
if code != 1 || !strings.Contains(stdout, "#6 carries backport/skip") {
|
||||
t.Errorf("fold of a skipped original = %d, %q; want a refusal", code, stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// backport/include lifts rules 2 to 4, not rule 6: an older line still waits
|
||||
// for the newer one.
|
||||
func TestIncludeKeepsRuleSix(t *testing.T) {
|
||||
f, gh := overridden(t)
|
||||
f.git("fetch", "-q", "origin")
|
||||
l, _ := parseLine("release-1.0")
|
||||
b, err := (&app{gh: gh, stderr: &strings.Builder{}}).prepare(l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if v := b.find(5); v == nil || v.rule != ruleNewer {
|
||||
t.Errorf("#5 on release-1.0 = %+v, want rule 6", v)
|
||||
}
|
||||
if v := b.find(10); v == nil || v.rule != 0 {
|
||||
t.Errorf("#10 on release-1.0 = %+v, want it to qualify: release-1.1 carries it", v)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOverridesUnreadable(t *testing.T) {
|
||||
f, gh := overridden(t)
|
||||
gh.listErr = errString("gh: HTTP 502")
|
||||
if code, _, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 2 || !strings.Contains(stderr, "HTTP 502") {
|
||||
t.Errorf("check with labels unreadable = %d (%s), want 2", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// ghCLI against a stub gh on PATH: the exact argv, and a listing that fills
|
||||
// GitHub search's cap is refused rather than read as complete.
|
||||
func TestGHCLI(t *testing.T) {
|
||||
bin := t.TempDir()
|
||||
args, out := filepath.Join(bin, "args"), filepath.Join(bin, "out")
|
||||
stub := "#!/bin/sh\nprintf '%s\\n' \"$*\" >> \"" + args + "\"\ncat \"" + out + "\"\n"
|
||||
if err := os.WriteFile(filepath.Join(bin, "gh"), []byte(stub), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
|
||||
write := func(s string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(out, []byte(s), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = os.Remove(args)
|
||||
}
|
||||
called := func() string {
|
||||
data, _ := os.ReadFile(args)
|
||||
return strings.TrimSpace(string(data))
|
||||
}
|
||||
|
||||
write(`[{"number":5},{"number":7}]`)
|
||||
got, err := ghCLI{}.labelled(labelInclude)
|
||||
if err != nil || !slices.Equal(got, []int{5, 7}) {
|
||||
t.Errorf("labelled = %v, %v; want [5 7]", got, err)
|
||||
}
|
||||
if want := "pr list --base main --state merged --label backport/include --limit 1000 --json number"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
|
||||
var full []string
|
||||
for i := range searchResultCap {
|
||||
full = append(full, fmt.Sprintf(`{"number":%d}`, i+1))
|
||||
}
|
||||
write("[" + strings.Join(full, ",") + "]")
|
||||
if _, err := (ghCLI{}).labelled(labelSkip); err == nil {
|
||||
t.Error("a listing as long as the search cap was read as complete")
|
||||
}
|
||||
|
||||
write(`{"permission":"write"}`)
|
||||
if p, err := (ghCLI{}).permission("maintainer"); err != nil || p != "write" {
|
||||
t.Errorf("permission = %q, %v", p, err)
|
||||
}
|
||||
if want := "api repos/{owner}/{repo}/collaborators/maintainer/permission"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
write(`{"permission":"admin"}`)
|
||||
for _, login := range []string{"x/../../orgs/y", "-x", "a b", "github-actions[bot]", ""} {
|
||||
if _, err := (ghCLI{}).permission(login); err == nil || called() != "" {
|
||||
t.Errorf("permission(%q) = %v and called gh %q; want a refusal before gh", login, err, called())
|
||||
}
|
||||
}
|
||||
|
||||
write("[]")
|
||||
if _, err := (ghCLI{}).openPulls("release-1.6"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "api -X GET repos/{owner}/{repo}/pulls -f state=open -f base=release-1.6 -f per_page=100"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
write("[]")
|
||||
if _, err := (ghCLI{}).recentPulls("release-1.6", 2); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "api -X GET repos/{owner}/{repo}/pulls -f state=all -f base=release-1.6 -f sort=created -f direction=desc -f per_page=100 -f page=2"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
var page []string
|
||||
for i := range openPullsCap {
|
||||
page = append(page, fmt.Sprintf(`{"number":%d}`, i+1))
|
||||
}
|
||||
write("[" + strings.Join(page, ",") + "]")
|
||||
if _, err := (ghCLI{}).openPulls("release-1.6"); err == nil {
|
||||
t.Error("a full page of open pull requests was read as all of them")
|
||||
}
|
||||
|
||||
write(`[[{"id":1,"body":"a","user":{"login":"x","type":"User"}}],[{"id":2,"body":"b","user":{"login":"y","type":"Bot"}}]]`)
|
||||
comments, err := ghCLI{}.comments(90)
|
||||
if err != nil || len(comments) != 2 || comments[1].ID != 2 || comments[1].User.Type != "Bot" {
|
||||
t.Errorf("comments = %+v, %v; want both pages", comments, err)
|
||||
}
|
||||
if want := "api --paginate --slurp repos/{owner}/{repo}/issues/90/comments"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
write("{}")
|
||||
if err := (ghCLI{}).createComment(90, "/tmp/body.md"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "api --method POST repos/{owner}/{repo}/issues/90/comments -F body=@/tmp/body.md"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
write("{}")
|
||||
if err := (ghCLI{}).editComment(12345678901, "/tmp/body.md"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "api --method PATCH repos/{owner}/{repo}/issues/comments/12345678901 -F body=@/tmp/body.md"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
|
||||
// A batch pull request carries no label: the release workflows act on a
|
||||
// pull request cozystack-ci[bot] opened only when it has the release label.
|
||||
write("https://github.com/cozystack/cozystack/pull/5300\n")
|
||||
if n, url, err := (ghCLI{}).createPull("release-1.7", batchBranch1, "fix(backport): batch", "/tmp/body.md"); err != nil || n != 5300 || url != "https://github.com/cozystack/cozystack/pull/5300" {
|
||||
t.Errorf("createPull = %d, %q, %v; want the pull request gh printed", n, url, err)
|
||||
}
|
||||
if want := "pr create --draft --base release-1.7 --head " + batchBranch1 + " --title fix(backport): batch --body-file /tmp/body.md"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
write("Warning: 1 uncommitted change\n")
|
||||
if _, _, err := (ghCLI{}).createPull("release-1.7", batchBranch1, "fix(backport): batch", "/tmp/body.md"); err == nil {
|
||||
t.Error("createPull took output that is no pull request URL")
|
||||
}
|
||||
write(`{"number":90,"state":"closed","merged_at":"2026-10-26T12:00:00Z","head":{"ref":"x","sha":"y"}}`)
|
||||
if pr, err := (ghCLI{}).pull(90); err != nil || pr.State != "closed" || pr.MergedAt == nil {
|
||||
t.Errorf("pull = %+v, %v; want #90 merged", pr, err)
|
||||
}
|
||||
if want := "api repos/{owner}/{repo}/pulls/90"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
if _, err := (ghCLI{}).pull(91); err == nil {
|
||||
t.Error("pull took GitHub's answer about another pull request")
|
||||
}
|
||||
write("")
|
||||
if err := (ghCLI{}).closePull(90); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "pr close 90"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
write("")
|
||||
if err := (ghCLI{}).editPull(90, "fix(backport): batch", "/tmp/body.md"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := "pr edit 90 --title fix(backport): batch --body-file /tmp/body.md"; called() != want {
|
||||
t.Errorf("gh %q, want gh %q", called(), want)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// policyPage is the policy the batch applies. The block in it that names when
|
||||
// each line started under the policy is read from the page itself, on the ref
|
||||
// the run pins, so the page and the batch cannot drift apart.
|
||||
const policyPage = "docs/release/backports.md"
|
||||
|
||||
const policyFence = "```backport-policy"
|
||||
|
||||
var (
|
||||
policyLineRE = regexp.MustCompile(`^(release-\d+\.\d+|` + batchOnlyKey + `):[ \t]*(\d{4}-\d{2}-\d{2})[ \t]*$`)
|
||||
policyReleasedRE = regexp.MustCompile(`^` + releasedKey + `:[ \t]*(\d+)[ \t]*$`)
|
||||
)
|
||||
|
||||
const (
|
||||
// batchOnlyKey names the day from which the batch is the only way onto a
|
||||
// line, so that from then on only its picks show what a line carries.
|
||||
batchOnlyKey = "batch-only"
|
||||
// releasedKey names how many of the lines that have shipped a stable
|
||||
// release the batch runs for, the newest of them; defaultReleased when
|
||||
// the block does not say.
|
||||
releasedKey = "released-lines"
|
||||
defaultReleased = 2
|
||||
)
|
||||
|
||||
// The rule 2 types and the scopes that never go into a patch release, as the
|
||||
// policy page lists them; policy_test.go holds the page to these.
|
||||
var (
|
||||
takenTypes = []string{"fix", "security"}
|
||||
excludedScopes = []string{"deps"}
|
||||
)
|
||||
|
||||
// starts maps a release line to the day it started under the policy. A line
|
||||
// that is not listed started at its fork point from main.
|
||||
type starts map[string]time.Time
|
||||
|
||||
// policy is what the backport-policy block says: when each line started, the
|
||||
// batch-only day, and how many released lines the batch runs for.
|
||||
type policy struct {
|
||||
starts starts
|
||||
batchOnly time.Time
|
||||
released int
|
||||
}
|
||||
|
||||
// parsePolicy reads the one backport-policy block of the page: a line per
|
||||
// release line, `release-X.Y: YYYY-MM-DD`, exactly one `batch-only:
|
||||
// YYYY-MM-DD`, at most one `released-lines: N`, and comments starting with #.
|
||||
func parsePolicy(page string) (*policy, error) {
|
||||
var blocks [][]string
|
||||
var cur []string
|
||||
in := false
|
||||
for _, ln := range strings.Split(strings.ReplaceAll(page, "\r\n", "\n"), "\n") {
|
||||
t := strings.TrimSpace(ln)
|
||||
switch {
|
||||
case !in && t == policyFence:
|
||||
in, cur = true, nil
|
||||
case in && t == "```":
|
||||
in = false
|
||||
blocks = append(blocks, cur)
|
||||
case in:
|
||||
cur = append(cur, t)
|
||||
}
|
||||
}
|
||||
if in {
|
||||
return nil, fmt.Errorf("%s: the %s block is not closed", policyPage, policyFence)
|
||||
}
|
||||
if len(blocks) != 1 {
|
||||
return nil, fmt.Errorf("%s holds %d %s blocks, not one", policyPage, len(blocks), policyFence)
|
||||
}
|
||||
out := &policy{starts: starts{}, released: defaultReleased}
|
||||
seen := map[string]bool{}
|
||||
for _, t := range blocks[0] {
|
||||
if t == "" || strings.HasPrefix(t, "#") {
|
||||
continue
|
||||
}
|
||||
if m := policyReleasedRE.FindStringSubmatch(t); m != nil {
|
||||
n, err := strconv.Atoi(m[1])
|
||||
switch {
|
||||
case err != nil || n < 1:
|
||||
return nil, fmt.Errorf("%s: %s is %q, not a number of lines", policyPage, releasedKey, m[1])
|
||||
case seen[releasedKey]:
|
||||
return nil, fmt.Errorf("%s: %s is listed twice", policyPage, releasedKey)
|
||||
}
|
||||
seen[releasedKey], out.released = true, n
|
||||
continue
|
||||
}
|
||||
m := policyLineRE.FindStringSubmatch(t)
|
||||
if m == nil {
|
||||
return nil, fmt.Errorf("%s: %q in the %s block is not `release-X.Y: YYYY-MM-DD`, `%s: YYYY-MM-DD` or `%s: N`", policyPage, t, policyFence, batchOnlyKey, releasedKey)
|
||||
}
|
||||
day, err := time.Parse(time.DateOnly, m[2])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %q is not a date: %w", policyPage, m[2], err)
|
||||
}
|
||||
if seen[m[1]] {
|
||||
return nil, fmt.Errorf("%s: %s is listed twice", policyPage, m[1])
|
||||
}
|
||||
seen[m[1]] = true
|
||||
if m[1] == batchOnlyKey {
|
||||
out.batchOnly = day
|
||||
continue
|
||||
}
|
||||
if _, err := parseLine(m[1]); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", policyPage, err)
|
||||
}
|
||||
out.starts[m[1]] = day
|
||||
}
|
||||
if !seen[batchOnlyKey] {
|
||||
return nil, fmt.Errorf("%s: the %s block gives no %s day", policyPage, policyFence, batchOnlyKey)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func readPolicy(commit string) (*policy, error) {
|
||||
page, err := git("show", "--end-of-options", commit+":"+policyPage)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading the policy at %.12s: %w", commit, err)
|
||||
}
|
||||
return parsePolicy(page)
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestParsePolicy(t *testing.T) {
|
||||
got, err := parsePolicy("text\n\n" + policyFence + "\n# comment\n\nrelease-1.6: 2026-10-26\nrelease-1.5:2026-01-02\nbatch-only: 2026-10-20\n```\n")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.starts) != 2 || !got.starts["release-1.6"].Equal(time.Date(2026, 10, 26, 0, 0, 0, 0, time.UTC)) {
|
||||
t.Errorf("parsePolicy = %v", got)
|
||||
}
|
||||
if !got.batchOnly.Equal(time.Date(2026, 10, 20, 0, 0, 0, 0, time.UTC)) {
|
||||
t.Errorf("the batch-only day = %v, want 2026-10-20", got.batchOnly)
|
||||
}
|
||||
if got.released != 2 {
|
||||
t.Errorf("released lines with none given = %d, want 2", got.released)
|
||||
}
|
||||
if got, err := parsePolicy(policyFence + "\nbatch-only: 2026-10-20\n```\n"); err != nil || len(got.starts) != 0 {
|
||||
t.Errorf("a block with no line = %v, %v; want no starts", got, err)
|
||||
}
|
||||
if got, err := parsePolicy(policyFence + "\nbatch-only: 2026-10-20\nreleased-lines: 3\n```\n"); err != nil || got.released != 3 {
|
||||
t.Errorf("released-lines: 3 = %v, %v; want 3", got, err)
|
||||
}
|
||||
for name, page := range map[string]string{
|
||||
"no batch-only day": policyFence + "\nrelease-1.6: 2026-10-26\n```\n",
|
||||
"two batch-only days": policyFence + "\nbatch-only: 2026-10-26\nbatch-only: 2026-10-27\n```\n",
|
||||
"a bad batch-only day": policyFence + "\nbatch-only: soon\n```\n",
|
||||
"no block": "## Backports\n",
|
||||
"two blocks": policyFence + "\n```\n" + policyFence + "\n```\n",
|
||||
"unclosed": policyFence + "\nrelease-1.6: 2026-10-26\n",
|
||||
"a bad line": policyFence + "\nrelease 1.6 on 2026-10-26\n```\n",
|
||||
"a bad date": policyFence + "\nrelease-1.6: 2026-13-40\n```\n",
|
||||
"a padded line": policyFence + "\nrelease-01.6: 2026-10-26\n```\n",
|
||||
"listed twice": policyFence + "\nrelease-1.6: 2026-10-26\nrelease-1.6: 2026-10-27\n```\n",
|
||||
"not a line": policyFence + "\nmain: 2026-10-26\n```\n",
|
||||
"trailing words": policyFence + "\nrelease-1.6: 2026-10-26 or so\n```\n",
|
||||
"no released lines": policyFence + "\nbatch-only: 2026-10-26\nreleased-lines: 0\n```\n",
|
||||
"released lines twice": policyFence + "\nbatch-only: 2026-10-26\nreleased-lines: 2\nreleased-lines: 3\n```\n",
|
||||
"released lines words": policyFence + "\nbatch-only: 2026-10-26\nreleased-lines: two\n```\n",
|
||||
"released lines huge": policyFence + "\nbatch-only: 2026-10-26\nreleased-lines: 99999999999999999999\n```\n",
|
||||
} {
|
||||
if _, err := parsePolicy(page); err == nil {
|
||||
t.Errorf("%s: parsePolicy accepted it", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A line listed in the policy takes nothing merged before its start, unless a
|
||||
// maintainer lets an original in; any other line starts at its fork point.
|
||||
func TestPolicyStart(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "feat(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{labels: map[string][]int{labelInclude: {6, 7}}}
|
||||
|
||||
verdicts := func() map[int]*verdict {
|
||||
t.Helper()
|
||||
f.git("fetch", "-q", "origin")
|
||||
l, _ := parseLine("release-1.1")
|
||||
b, err := (&app{gh: gh, stderr: &strings.Builder{}}).prepare(l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return map[int]*verdict{5: b.find(5), 6: b.find(6), 7: b.find(7)}
|
||||
}
|
||||
if v := verdicts(); v[5].rule != 0 || v[6].rule != 0 || v[7].rule != 0 {
|
||||
t.Errorf("with no start every original qualifies: %d %d %d", v[5].rule, v[6].rule, v[7].rule)
|
||||
}
|
||||
|
||||
f.setPolicy("release-1.1: 2099-01-01\n")
|
||||
v := verdicts()
|
||||
if v[5].rule != ruleBeforeStart {
|
||||
t.Errorf("#5 = rule %d (%s), want merged before the start", v[5].rule, v[5].reason)
|
||||
}
|
||||
if v[6].rule != 0 || v[7].rule != 0 {
|
||||
t.Errorf("backport/include lifts the start: #6 = rule %d, #7 = rule %d", v[6].rule, v[7].rule)
|
||||
}
|
||||
|
||||
f.setPolicy("release-1.0: 2099-01-01\n")
|
||||
if v := verdicts(); v[5].rule != 0 {
|
||||
t.Errorf("a start for another line kept #5 out of release-1.1: rule %d", v[5].rule)
|
||||
}
|
||||
|
||||
f.setPolicy("release-1.1: 2000-01-01\n")
|
||||
if v := verdicts(); v[5].rule != 0 {
|
||||
t.Errorf("a start before the fork point kept #5 out: rule %d", v[5].rule)
|
||||
}
|
||||
}
|
||||
|
||||
// The policy is read from main as the run pins it, or from the ref --policy
|
||||
// names, and a run without a readable policy stops. Only the commands that
|
||||
// read the policy take --policy.
|
||||
func TestPolicyRef(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.git("switch", "-q", "-c", "proposal")
|
||||
f.write(policyPage, policyWith("release-1.1: 2099-01-01\n"+releasedKey+": 1\n"))
|
||||
f.git("commit", "-q", "-am", "docs: start release-1.1 later")
|
||||
f.git("switch", "-q", "main")
|
||||
|
||||
code, stdout, stderr := f.exec(nil, "check", "--line", "release-1.1")
|
||||
if code != 1 || !strings.Contains(stdout, "/pull/5 ") {
|
||||
t.Errorf("check against main's policy = %d, %q, %s; want #5 missing", code, stdout, stderr)
|
||||
}
|
||||
code, stdout, stderr = f.exec(nil, "check", "--line", "release-1.1", "--policy", "proposal")
|
||||
if code != 0 {
|
||||
t.Errorf("check against the proposal's policy = %d, %q, %s; want #5 before the start", code, stdout, stderr)
|
||||
}
|
||||
if code, _, _ := f.exec(nil, "check", "--line", "release-1.1", "--policy", "--upload-pack=x"); code != 2 {
|
||||
t.Errorf("--policy with an option = %d, want 2", code)
|
||||
}
|
||||
if code, stdout, stderr := f.exec(nil, "lines", "--policy", "proposal"); code != 0 || stdout != "release-1.1\n" {
|
||||
t.Errorf("lines against the proposal's policy = %d, %q (%s); want release-1.1 alone", code, stdout, stderr)
|
||||
}
|
||||
for _, cmd := range []string{"verify", "notify"} {
|
||||
if code, _, stderr := f.exec(&fakeGH{}, cmd, "--line", "release-1.1", "--policy", "proposal"); code != 2 || !strings.Contains(stderr, "flag provided but not defined: -policy") {
|
||||
t.Errorf("%s --policy = %d (%s); want it refused, as %s reads no policy", cmd, code, stderr, cmd)
|
||||
}
|
||||
}
|
||||
|
||||
f.squash(9001, "docs: drop the page", "", map[string]string{policyPage: ""})
|
||||
if code, _, stderr := f.exec(nil, "check", "--line", "release-1.1"); code != 2 || !strings.Contains(stderr, "reading the policy") {
|
||||
t.Errorf("check without a policy page = %d (%s), want 2", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// The policy page and the code say the same thing: the start block parses as
|
||||
// the batch reads it, rule 2 names the types the code takes, the page names
|
||||
// the scopes the code keeps out, and both override labels.
|
||||
func TestPolicyPageMatchesCode(t *testing.T) {
|
||||
data, err := os.ReadFile("../../" + policyPage)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
page := string(data)
|
||||
pol, err := parsePolicy(page)
|
||||
if err != nil {
|
||||
t.Fatalf("the page's policy block does not parse: %v", err)
|
||||
}
|
||||
if day, ok := pol.starts["release-1.6"]; !ok || day.Format(time.DateOnly) != "2026-10-26" {
|
||||
t.Errorf("release-1.6 starts %v, want 2026-10-26", pol.starts["release-1.6"])
|
||||
}
|
||||
if day := pol.batchOnly.Format(time.DateOnly); day != "2026-10-26" {
|
||||
t.Errorf("the batch-only day is %s, want 2026-10-26", day)
|
||||
}
|
||||
// The page says how many released lines the batch runs for, rather than
|
||||
// leaving it to the default.
|
||||
if !regexp.MustCompile("(?m)^"+releasedKey+":").MatchString(page) || pol.released != 2 {
|
||||
t.Errorf("the page's block gives %d released lines, want 2 written out", pol.released)
|
||||
}
|
||||
|
||||
rule2 := regexp.MustCompile("(?m)^2\\. Its title type is (.*)$").FindStringSubmatch(page)
|
||||
if rule2 == nil {
|
||||
t.Fatal("the page has no rule 2")
|
||||
}
|
||||
var types []string
|
||||
for _, m := range regexp.MustCompile("`([a-z]+)`").FindAllStringSubmatch(rule2[1], -1) {
|
||||
types = append(types, m[1])
|
||||
}
|
||||
if !slices.Equal(types, takenTypes) {
|
||||
t.Errorf("rule 2 on the page takes %v, the code %v", types, takenTypes)
|
||||
}
|
||||
never := regexp.MustCompile("(?m)^### What waits for the next minor\\n\\n(.*)$").FindStringSubmatch(page)
|
||||
if never == nil {
|
||||
t.Fatal("the page has no list of what never goes into a patch release")
|
||||
}
|
||||
for _, scope := range excludedScopes {
|
||||
if !strings.Contains(never[1], "(scope `"+scope+"`)") {
|
||||
t.Errorf("the page does not keep scope %s out", scope)
|
||||
}
|
||||
}
|
||||
if scopes := regexp.MustCompile("scope `([a-z]+)`").FindAllStringSubmatch(never[1], -1); len(scopes) != len(excludedScopes) {
|
||||
t.Errorf("the page keeps %d scopes out, the code %d", len(scopes), len(excludedScopes))
|
||||
}
|
||||
for i := 1; i <= 6; i++ {
|
||||
if !regexp.MustCompile("(?m)^" + string(rune('0'+i)) + "\\. ").MatchString(page) {
|
||||
t.Errorf("the page has no rule %d", i)
|
||||
}
|
||||
}
|
||||
for _, label := range []string{labelInclude, labelSkip} {
|
||||
if !strings.Contains(page, "`"+label+"`") {
|
||||
t.Errorf("the page does not describe %s", label)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,440 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A line's record is the backport-batch block of its newest batch pull
|
||||
// request, read whatever that pull request's state and written only while it
|
||||
// is open: a merged or closed pull request is never written again, so what it
|
||||
// recorded stays what the line's later commands act on until a newer batch
|
||||
// carries a record of its own.
|
||||
|
||||
// lineRecord is the one line of JSON in the block. Version 1, the block
|
||||
// before it had a version, holds only line and deferred.
|
||||
type lineRecord struct {
|
||||
Version int `json:"version"`
|
||||
Line string `json:"line"`
|
||||
Built *builtFrom `json:"built,omitempty"`
|
||||
Report string `json:"report,omitempty"`
|
||||
Closing *closing `json:"closing"`
|
||||
Deferred []deferredEntry `json:"deferred"`
|
||||
Dropped []droppedEntry `json:"dropped"`
|
||||
Stale []int `json:"stale"`
|
||||
}
|
||||
|
||||
// builtFrom is what a classification was made of; batch is the head it
|
||||
// describes.
|
||||
type builtFrom struct {
|
||||
Main string `json:"main"`
|
||||
Line string `json:"line"`
|
||||
Newer string `json:"newer"`
|
||||
Policy string `json:"policy"`
|
||||
Batch string `json:"batch"`
|
||||
Tool string `json:"tool"`
|
||||
Run string `json:"run"`
|
||||
At string `json:"at"`
|
||||
Options runOptions `json:"options"`
|
||||
}
|
||||
|
||||
type runOptions struct {
|
||||
NoWorkflowFiles bool `json:"noWorkflowFiles"`
|
||||
Mode string `json:"mode"`
|
||||
}
|
||||
|
||||
// The operations a record is written by: a run that assembles the batch, and
|
||||
// the ones that only change what the record says about it.
|
||||
const (
|
||||
modeNew = "new"
|
||||
modeContinue = "continue"
|
||||
modeDefer = "defer"
|
||||
modeDrop = "drop"
|
||||
modeUndrop = "undrop"
|
||||
modeClose = "close"
|
||||
)
|
||||
|
||||
var modes = []string{modeNew, modeContinue, modeDefer, modeDrop, modeUndrop, modeClose}
|
||||
|
||||
// closing is written into the record of an open batch that the run is about
|
||||
// to close: the head it froze, which the close and the branch's deletion are
|
||||
// leased on.
|
||||
type closing struct {
|
||||
PR int `json:"pr"`
|
||||
Branch string `json:"branch"`
|
||||
Head string `json:"head"`
|
||||
}
|
||||
|
||||
// droppedEntry is a maintainer's decision that an original stays off the
|
||||
// line, bound to its merge commit on main.
|
||||
type droppedEntry struct {
|
||||
Original int `json:"original"`
|
||||
Merge string `json:"merge"`
|
||||
Reason string `json:"reason"`
|
||||
On int `json:"on"`
|
||||
Date string `json:"date"`
|
||||
}
|
||||
|
||||
var (
|
||||
toolRE = regexp.MustCompile(`^[0-9a-f]{40}(\+dirty)?$`)
|
||||
reportIDRE = regexp.MustCompile(`^[0-9a-f]{64}$`)
|
||||
)
|
||||
|
||||
// oneLineText is s as a run writes a reason: one line, cleaned and capped.
|
||||
// A record holding anything else was not written by a run, and its text
|
||||
// would reach a terminal, a body or a comment as someone else wrote it.
|
||||
func oneLineText(s string) string { return capped(oneLine(clean(s, false)), titleLimit) }
|
||||
|
||||
// fencedLines lists the one line of each block that fence opens.
|
||||
func fencedLines(body, fence string) ([]string, error) {
|
||||
var blocks []string
|
||||
lines := strings.Split(strings.ReplaceAll(body, "\r\n", "\n"), "\n")
|
||||
for i := 0; i < len(lines); i++ {
|
||||
if strings.TrimSpace(lines[i]) != fence {
|
||||
continue
|
||||
}
|
||||
if i+2 >= len(lines) || strings.TrimSpace(lines[i+2]) != "```" {
|
||||
return nil, fmt.Errorf("the %s block is not one line of JSON", fence)
|
||||
}
|
||||
blocks = append(blocks, lines[i+1])
|
||||
i += 2
|
||||
}
|
||||
return blocks, nil
|
||||
}
|
||||
|
||||
// decodeStrict reads exactly one JSON value into v, refusing a key v does not
|
||||
// have and anything after the value.
|
||||
func decodeStrict(data string, v any) error {
|
||||
dec := json.NewDecoder(strings.NewReader(data))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(v); err != nil {
|
||||
return err
|
||||
}
|
||||
var rest json.RawMessage
|
||||
if dec.More() || dec.Decode(&rest) != io.EOF {
|
||||
return errors.New("something follows the JSON value")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// parseRecord reads the line's record from a batch body: nil when the body
|
||||
// has no block, an error when it has more than one or one it cannot vouch
|
||||
// for whole.
|
||||
func parseRecord(body string, l line) (*lineRecord, error) {
|
||||
blocks, err := fencedLines(body, deferredFence)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, err
|
||||
case len(blocks) == 0:
|
||||
return nil, nil
|
||||
case len(blocks) > 1:
|
||||
return nil, fmt.Errorf("the body holds %d %s blocks", len(blocks), deferredFence)
|
||||
}
|
||||
var head struct {
|
||||
Version *int `json:"version"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(blocks[0]), &head); err != nil {
|
||||
return nil, fmt.Errorf("reading the %s block: %w", deferredFence, err)
|
||||
}
|
||||
version := 1
|
||||
if head.Version != nil {
|
||||
version = *head.Version
|
||||
}
|
||||
var rec lineRecord
|
||||
switch version {
|
||||
case 1:
|
||||
var v1 struct {
|
||||
Version int `json:"version"`
|
||||
Line string `json:"line"`
|
||||
Deferred []deferredEntry `json:"deferred"`
|
||||
}
|
||||
err = decodeStrict(blocks[0], &v1)
|
||||
rec = lineRecord{Version: 1, Line: v1.Line, Deferred: v1.Deferred}
|
||||
case 2:
|
||||
err = decodeStrict(blocks[0], &rec)
|
||||
default:
|
||||
return nil, fmt.Errorf("the %s block is version %d, which this backport-batch does not read; it reads 1 and 2", deferredFence, version)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading the %s block: %w", deferredFence, err)
|
||||
}
|
||||
if err := rec.validate(l); err != nil {
|
||||
return nil, fmt.Errorf("the %s block %w", deferredFence, err)
|
||||
}
|
||||
return &rec, nil
|
||||
}
|
||||
|
||||
// validate holds the record to what a run writes. Its errors read after
|
||||
// "the block".
|
||||
func (rec *lineRecord) validate(l line) error {
|
||||
if rec.Line != l.branch() {
|
||||
return fmt.Errorf("is for %q, not %s", rec.Line, l.branch())
|
||||
}
|
||||
seen := map[int]bool{}
|
||||
once := func(n int) error {
|
||||
if n <= 0 {
|
||||
return fmt.Errorf("names original %d", n)
|
||||
}
|
||||
if seen[n] {
|
||||
return fmt.Errorf("names #%d twice", n)
|
||||
}
|
||||
seen[n] = true
|
||||
return nil
|
||||
}
|
||||
for _, d := range rec.Deferred {
|
||||
if err := once(d.Original); err != nil {
|
||||
return err
|
||||
}
|
||||
if d.Commit != "" && checkSHA(d.Commit) != nil {
|
||||
return fmt.Errorf("names commit %q for #%d", d.Commit, d.Original)
|
||||
}
|
||||
// A conflict names the commit that failed and its files, a deferral
|
||||
// its reason, and a wait its workflow files.
|
||||
kinds := 0
|
||||
for _, is := range []bool{d.Commit != "" || len(d.Files) > 0 && !d.Workflows && d.Reason == "", d.Reason != "", d.Workflows} {
|
||||
if is {
|
||||
kinds++
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case d.Workflows && (d.Commit != "" || d.Reason != ""):
|
||||
return fmt.Errorf("has #%d wait for a maintainer and defers it as well", d.Original)
|
||||
case kinds != 1:
|
||||
return fmt.Errorf("does not say whether #%d is a conflict, deferred for a reason or waiting", d.Original)
|
||||
case d.Reason != "" && strings.TrimSpace(d.Reason) == "":
|
||||
return fmt.Errorf("defers #%d for an empty reason", d.Original)
|
||||
case d.Reason != oneLineText(d.Reason):
|
||||
return fmt.Errorf("defers #%d for %q, which is no reason a run writes", d.Original, d.Reason)
|
||||
case !slices.Equal(d.Files, cleanFiles(d.Files)):
|
||||
return fmt.Errorf("names files of #%d that are not as a run writes them: %q", d.Original, d.Files)
|
||||
case d.More < 0:
|
||||
return fmt.Errorf("counts %d more files for #%d", d.More, d.Original)
|
||||
}
|
||||
}
|
||||
for _, d := range rec.Dropped {
|
||||
if err := once(d.Original); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := time.Parse(time.DateOnly, d.Date); err != nil || checkSHA(d.Merge) != nil || strings.TrimSpace(d.Reason) == "" || d.On <= 0 {
|
||||
return fmt.Errorf("drops #%d with a merge, reason, pull request or date it cannot have", d.Original)
|
||||
}
|
||||
if d.Reason != oneLineText(d.Reason) {
|
||||
return fmt.Errorf("drops #%d for %q, which is no reason a run writes", d.Original, d.Reason)
|
||||
}
|
||||
}
|
||||
staleSeen := map[int]bool{}
|
||||
for _, n := range rec.Stale {
|
||||
if n <= 0 || staleSeen[n] {
|
||||
return fmt.Errorf("lists %d as still on the batch more than once or not at all", n)
|
||||
}
|
||||
staleSeen[n] = true
|
||||
}
|
||||
if rec.Version < 2 {
|
||||
return nil
|
||||
}
|
||||
if b := rec.Built; b == nil {
|
||||
return errors.New("does not say what it was built from")
|
||||
} else if err := b.validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if !reportIDRE.MatchString(rec.Report) {
|
||||
return fmt.Errorf("names report %q", rec.Report)
|
||||
}
|
||||
if c := rec.Closing; c != nil {
|
||||
m := batchBranchRE.FindStringSubmatch(c.Branch)
|
||||
if c.PR <= 0 || m == nil || m[1] != l.branch() || checkSHA(c.Head) != nil {
|
||||
return fmt.Errorf("closes #%d on %q at %q, which is no batch of %s", c.PR, c.Branch, c.Head, l.branch())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (b *builtFrom) validate() error {
|
||||
for _, sha := range []string{b.Main, b.Line, b.Policy, b.Batch} {
|
||||
if checkSHA(sha) != nil {
|
||||
return fmt.Errorf("says it was built from %q", sha)
|
||||
}
|
||||
}
|
||||
if b.Newer != "" && checkSHA(b.Newer) != nil {
|
||||
return fmt.Errorf("says it was built from %q", b.Newer)
|
||||
}
|
||||
if b.Tool != "" && !toolRE.MatchString(b.Tool) {
|
||||
return fmt.Errorf("names tool %q", b.Tool)
|
||||
}
|
||||
if b.Run != "" && !runURLRE.MatchString(b.Run) {
|
||||
return fmt.Errorf("names run %q", b.Run)
|
||||
}
|
||||
if _, err := time.Parse(time.RFC3339, b.At); err != nil {
|
||||
return fmt.Errorf("was built at %q", b.At)
|
||||
}
|
||||
if !slices.Contains(modes, b.Options.Mode) {
|
||||
return fmt.Errorf("was written by %q, which is no operation of backport-batch", b.Options.Mode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var runURLRE = regexp.MustCompile(`^https://[^/\s]+/[^/\s]+/[^/\s]+/actions/runs/[0-9]+(/attempts/[0-9]+)?$`)
|
||||
|
||||
// recordBlock is the block that holds rec. One line of JSON: encoding/json
|
||||
// escapes newlines and <, > and &, so nothing in a file name or a reason can
|
||||
// end the fence or open HTML.
|
||||
func recordBlock(rec lineRecord) string {
|
||||
data, _ := json.Marshal(rec)
|
||||
return deferredFence + "\n" + string(data) + "\n```"
|
||||
}
|
||||
|
||||
// savedNoWorkflowFiles is the --no-workflow-files the open batch was last
|
||||
// assembled with, which an operation that only changes the record keeps. A
|
||||
// record from before it was saved says so by listing a wait.
|
||||
func (s recordSource) savedNoWorkflowFiles() bool {
|
||||
if !s.open || s.rec == nil {
|
||||
return false
|
||||
}
|
||||
if s.rec.Built != nil {
|
||||
return s.rec.Built.Options.NoWorkflowFiles
|
||||
}
|
||||
return slices.ContainsFunc(s.rec.Deferred, func(d deferredEntry) bool { return d.Workflows })
|
||||
}
|
||||
|
||||
// dropped is what the record drops, which every later record keeps.
|
||||
func (rec *lineRecord) dropped() []droppedEntry {
|
||||
if rec == nil {
|
||||
return nil
|
||||
}
|
||||
return slices.Clone(rec.Dropped)
|
||||
}
|
||||
|
||||
// deferred maps the originals the record defers to their entries.
|
||||
func (rec *lineRecord) deferred() map[int]deferredEntry {
|
||||
out := map[int]deferredEntry{}
|
||||
if rec != nil {
|
||||
for _, d := range rec.Deferred {
|
||||
out[d.Original] = d
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// recordSource is where a line's record was found.
|
||||
type recordSource struct {
|
||||
// carrier is the pull request whose block is the record, nil when the
|
||||
// line has none.
|
||||
carrier *pullRequest
|
||||
// open says the carrier is the open batch, so the record may be written.
|
||||
open bool
|
||||
rec *lineRecord
|
||||
// code is the open batch pull request, if any. One whose body someone
|
||||
// wrote by hand is the batch the line builds on but carries no record.
|
||||
code *pullRequest
|
||||
}
|
||||
|
||||
// closingOpen is the close the record says is under way on a batch that is
|
||||
// still open, or nil.
|
||||
func (s recordSource) closingOpen() *closing {
|
||||
if !s.open || s.rec == nil {
|
||||
return nil
|
||||
}
|
||||
return s.rec.Closing
|
||||
}
|
||||
|
||||
func prState(pr *pullRequest) string {
|
||||
switch {
|
||||
case pr.MergedAt != nil:
|
||||
return "merged"
|
||||
case pr.State == "" || pr.State == "open":
|
||||
return "open"
|
||||
case pr.State == "closed":
|
||||
return "closed"
|
||||
}
|
||||
return fmt.Sprintf("in state %q", pr.State)
|
||||
}
|
||||
|
||||
func hasRecord(body string) bool {
|
||||
return slices.ContainsFunc(strings.Split(body, "\n"), func(ln string) bool { return strings.TrimSpace(ln) == deferredFence })
|
||||
}
|
||||
|
||||
// findLineRecord finds the line's record: the block of the newest pull
|
||||
// request opened from a batch branch of this repository for the line by an
|
||||
// author the batch trusts, open, merged or closed. open is the open batch
|
||||
// pull request batchPull chose, or nil. Anything that leaves in doubt which
|
||||
// record is the newest is an error, never a fall back to an older one: a
|
||||
// newer block by an author the batch does not trust, a block the bot wrote
|
||||
// and someone removed, a record it cannot read, a closed record newer than
|
||||
// the open batch, or more pull requests than it reads.
|
||||
func findLineRecord(gh github, l line, open *pullRequest) (recordSource, error) {
|
||||
src := recordSource{code: open}
|
||||
for page := 1; src.carrier == nil; page++ {
|
||||
if page > recentPullsPages {
|
||||
return src, fmt.Errorf("none of the %d newest pull requests against %s is a batch pull request with a record, so the line's record cannot be told apart from none", recentPullsPages*recentPullsPage, l.branch())
|
||||
}
|
||||
prs, err := gh.recentPulls(l.branch(), page)
|
||||
if err != nil {
|
||||
return src, err
|
||||
}
|
||||
for i := range prs {
|
||||
pr := &prs[i]
|
||||
m := batchBranchRE.FindStringSubmatch(pr.Head.Ref)
|
||||
if m == nil || m[1] != l.branch() || pr.Base.Ref != l.branch() || pr.Head.Repo == nil || pr.Head.Repo.FullName != pr.Base.Repo.FullName {
|
||||
continue
|
||||
}
|
||||
if open != nil && pr.Number == open.Number {
|
||||
pr = open
|
||||
}
|
||||
if !hasRecord(pr.Body) {
|
||||
if pr.User.Type == "Bot" && gh.batchBot() != "" && pr.User.Login == gh.batchBot() {
|
||||
return src, fmt.Errorf("batch pull request #%d (%s) was opened by %s but carries no %s block; someone removed it, so the line's record is unknown", pr.Number, prState(pr), pr.User.Login, deferredFence)
|
||||
}
|
||||
continue
|
||||
}
|
||||
why, err := trusted(gh, *pr)
|
||||
if err != nil {
|
||||
return src, fmt.Errorf("batch pull request #%d: %w", pr.Number, err)
|
||||
}
|
||||
if why != "" {
|
||||
return src, fmt.Errorf("batch pull request #%d (%s) carries a %s block but was %s; an older record is not taken in its place", pr.Number, prState(pr), deferredFence, why)
|
||||
}
|
||||
rec, err := parseRecord(pr.Body, l)
|
||||
if err != nil {
|
||||
return src, fmt.Errorf("batch pull request #%d (%s): %w", pr.Number, prState(pr), err)
|
||||
}
|
||||
src.carrier, src.rec = pr, rec
|
||||
break
|
||||
}
|
||||
if len(prs) < recentPullsPage {
|
||||
break
|
||||
}
|
||||
}
|
||||
switch c := src.carrier; {
|
||||
case open != nil && hasRecord(open.Body) && (c == nil || c.Number < open.Number):
|
||||
return src, fmt.Errorf("GitHub does not list the open batch pull request #%d among the pull requests against %s; run again", open.Number, l.branch())
|
||||
case open != nil && hasRecord(open.Body) && c.Number > open.Number:
|
||||
return src, fmt.Errorf("batch pull request #%d (%s) carries a record newer than the open batch pull request #%d; close one of them by hand", c.Number, prState(c), open.Number)
|
||||
case open != nil && c == open:
|
||||
src.open = true
|
||||
case c != nil && prState(c) == "open":
|
||||
return src, fmt.Errorf("batch pull request #%d carries the line's record and is open, but is not the batch this run builds on", c.Number)
|
||||
}
|
||||
return src, nil
|
||||
}
|
||||
@@ -0,0 +1,368 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The records the live batches carried before records had a version, as
|
||||
// GitHub returned them on 2026-11-09, read as version 1.
|
||||
func TestRecordReadsVersion1(t *testing.T) {
|
||||
for line, block := range map[string]string{
|
||||
"release-1.7": `{"line":"release-1.7","deferred":[{"original":4756,"commit":"647407545d630291cfe2a20880cce45af22edf0c","files":["packages/system/backupstrategy-controller/values.yaml"]},{"original":4867,"commit":"813c6180861efe1896c26d62a7afcbc1ff590ebc","files":[".github/renovate.json"]},{"original":5082,"commit":"e93db8848027450c2e23d21348409efa641cb6f1","files":["pkg/registry/core/tap/rest.go"]},{"original":4888,"files":[".github/workflows/cut-prerelease.yaml"],"workflows":true},{"original":4971,"files":[".github/workflows/e2e-proxmox.yaml"],"workflows":true}]}`,
|
||||
"release-1.6": `{"line":"release-1.6","deferred":[{"original":5071,"commit":"3fa534f3cf6cb5774a183f0202f7f5c13e9ae211","files":["internal/operator/tapmaterializer_reconciler.go","internal/operator/tapmaterializer_reconciler_test.go"]},{"original":5086,"commit":"5265ee662c591db409c8ef1ed8bb457cf29f5c22","files":["packages/apps/tenant/templates/gateway.yaml"]},{"original":5087,"commit":"8d8179f20a2c10de03face1472435d5083cde38d","files":["internal/controller/applicationdefinition_helmreconciler.go","internal/controller/applicationdefinition_helmreconciler_test.go"]},{"original":5142,"files":[".github/workflows/pull-requests-release.yaml",".github/workflows/update-releasenotes.yaml"],"workflows":true}]}`,
|
||||
} {
|
||||
l, _ := parseLine(line)
|
||||
rec, err := parseRecord("## What this PR does\n\ntext\n\n"+deferredFence+"\n"+block+"\n```\n\n### Release note\n", l)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: parseRecord = %v", line, err)
|
||||
}
|
||||
waits, conflicts := 0, 0
|
||||
for _, e := range rec.Deferred {
|
||||
switch {
|
||||
case e.Workflows:
|
||||
waits++
|
||||
case e.Commit != "":
|
||||
conflicts++
|
||||
}
|
||||
}
|
||||
if rec.Version != 1 || rec.Line != line || rec.Closing != nil || len(rec.Dropped) != 0 || waits+conflicts != len(rec.Deferred) || waits == 0 || conflicts == 0 {
|
||||
t.Errorf("%s: record = %+v; want version 1 with its conflicts and waits", line, rec)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// v2 is a version 2 record with each value it may hold.
|
||||
func v2(extra string) string {
|
||||
sha := strings.Repeat("a", 40)
|
||||
return `{"version":2,"line":"release-1.1","built":{"main":"` + sha + `","line":"` + sha + `","newer":"","policy":"` + sha + `","batch":"` + sha + `","tool":"` + sha + `+dirty","run":"","at":"2026-10-26T06:00:00Z","options":{"noWorkflowFiles":true,"mode":"continue"}},` +
|
||||
`"report":"` + strings.Repeat("b", 64) + `","closing":null,"deferred":[{"original":5,"commit":"` + sha + `","files":["app/five.txt"]},{"original":6,"files":[],"reason":"fails the e2e"},{"original":7,"files":[".github/workflows/x.yaml"],"workflows":true}],` +
|
||||
`"dropped":[{"original":8,"merge":"` + sha + `","reason":"needs #4","on":90,"date":"2026-10-26"}],"stale":[9]` + extra + `}`
|
||||
}
|
||||
|
||||
// A record is read whole or not at all: anything it cannot vouch for stops
|
||||
// the command rather than being half believed.
|
||||
func TestRecordRefuses(t *testing.T) {
|
||||
l, _ := parseLine("release-1.1")
|
||||
block := func(json string) string { return "text\n\n" + deferredFence + "\n" + json + "\n```\n" }
|
||||
if rec, err := parseRecord(block(v2("")), l); err != nil || rec.Version != 2 || len(rec.Dropped) != 1 || rec.Built.Options.Mode != "continue" {
|
||||
t.Fatalf("parseRecord of a whole v2 record = %+v, %v", rec, err)
|
||||
}
|
||||
sha := strings.Repeat("a", 40)
|
||||
for name, json := range map[string]string{
|
||||
"version 4": strings.Replace(v2(""), `"version":2`, `"version":4`, 1),
|
||||
"trailing data": v2("") + `{}`,
|
||||
"trailing text": v2("") + ` x`,
|
||||
"an original listed twice": strings.Replace(v2(""), `"original":8`, `"original":5`, 1),
|
||||
"a needs entry with a commit": strings.Replace(v2(""), `{"original":6,"files":[],"reason":"fails the e2e"}`, `{"original":6,"commit":"`+sha+`","files":[],"needs":[4]}`, 1),
|
||||
"an empty reason": strings.Replace(v2(""), `"reason":"fails the e2e"`, `"reason":""`, 1),
|
||||
"an empty drop reason": strings.Replace(v2(""), `"reason":"needs #4"`, `"reason":" "`, 1),
|
||||
"a bad tool": strings.Replace(v2(""), `+dirty"`, `-dirty"`, 1),
|
||||
"an unknown mode": strings.Replace(v2(""), `"mode":"continue"`, `"mode":"rebase"`, 1),
|
||||
"a reason with a commit": strings.Replace(v2(""), `"reason":"fails the e2e"`, `"reason":"x","commit":"`+sha+`"`, 1),
|
||||
"a short merge": strings.Replace(v2(""), `"merge":"`+sha+`"`, `"merge":"abc"`, 1),
|
||||
"a bad date": strings.Replace(v2(""), `"date":"2026-10-26"`, `"date":"26.10.2026"`, 1),
|
||||
"a bad time": strings.Replace(v2(""), `"at":"2026-10-26T06:00:00Z"`, `"at":"yesterday"`, 1),
|
||||
"a short report": strings.Replace(v2(""), strings.Repeat("b", 64), "bbbb", 1),
|
||||
"a closing on another line": strings.Replace(v2(""), `"closing":null`, `"closing":{"pr":90,"branch":"backport-batch/release-1.0-20261026","head":"`+sha+`"}`, 1),
|
||||
"a closing without a head": strings.Replace(v2(""), `"closing":null`, `"closing":{"pr":90,"branch":"`+batchBranch1+`","head":""}`, 1),
|
||||
"a negative stale": strings.Replace(v2(""), `"stale":[9]`, `"stale":[-9]`, 1),
|
||||
"an unknown key": v2(`,"parked":[]`),
|
||||
"an entry of no kind": strings.Replace(v2(""), `{"original":6,"files":[],"reason":"fails the e2e"}`, `{"original":6,"files":[]}`, 1),
|
||||
"another line": strings.Replace(v2(""), `"line":"release-1.1"`, `"line":"release-1.0"`, 1),
|
||||
"an escape in a reason": strings.Replace(v2(""), `"reason":"fails the e2e"`, `"reason":"fails\u001b[2J the e2e"`, 1),
|
||||
"a newline in a reason": strings.Replace(v2(""), `"reason":"fails the e2e"`, `"reason":"fails\nwarning: forged"`, 1),
|
||||
"an escape in a drop": strings.Replace(v2(""), `"reason":"needs #4"`, `"reason":"needs\u001b]0;x\u0007 #4"`, 1),
|
||||
"a newline in a drop": strings.Replace(v2(""), `"reason":"needs #4"`, `"reason":"needs #4\nnote: forged"`, 1),
|
||||
"a reason past its length": strings.Replace(v2(""), `"reason":"needs #4"`, `"reason":"`+strings.Repeat("x", titleLimit+1)+`"`, 1),
|
||||
"a bidi file name": strings.Replace(v2(""), `"files":["app/five.txt"]`, `"files":["app/\u202efive.txt"]`, 1),
|
||||
} {
|
||||
if _, err := parseRecord(block(json), l); err == nil {
|
||||
t.Errorf("%s: parseRecord accepted it", name)
|
||||
}
|
||||
}
|
||||
closing := strings.Replace(v2(""), `"closing":null`, `"closing":{"pr":90,"branch":"`+batchBranch1+`","head":"`+sha+`"}`, 1)
|
||||
if rec, err := parseRecord(block(closing), l); err != nil || rec.Closing == nil || rec.Closing.PR != 90 {
|
||||
t.Errorf("a record that is closing = %+v, %v", rec, err)
|
||||
}
|
||||
}
|
||||
|
||||
// recordPR is a batch pull request on release-1.1 carrying body, numbered n,
|
||||
// in state, opened by login.
|
||||
func recordPR(n int, state, login, body string) pullRequest {
|
||||
pr := notifyPR("20261026", body)
|
||||
pr.Number, pr.State = n, state
|
||||
pr.Head.Ref = fmt.Sprintf("backport-batch/release-1.1-202610%02d", n%30+1)
|
||||
pr.User.Login = login
|
||||
if strings.HasSuffix(login, "[bot]") {
|
||||
pr.User.Type = "Bot"
|
||||
}
|
||||
return pr
|
||||
}
|
||||
|
||||
func recordBody(conflict int) string {
|
||||
return "text\n\n" + deferredFence + "\n" + fmt.Sprintf(`{"line":"release-1.1","deferred":[{"original":%d,"commit":"%s","files":["a"]}]}`, conflict, strings.Repeat("a", 40)) + "\n```\n"
|
||||
}
|
||||
|
||||
// The record is the block of the newest batch pull request that carries one,
|
||||
// whatever its state, among those a trusted author opened from a batch branch
|
||||
// of this repository; anything that leaves the newest record in doubt stops
|
||||
// the command rather than falling back to an older one.
|
||||
func TestFindLineRecord(t *testing.T) {
|
||||
const bot = defaultBatchBot
|
||||
other := func(n int) pullRequest {
|
||||
pr := recordPR(n, "closed", "maintainer", "")
|
||||
pr.Head.Ref = fmt.Sprintf("fix-%d", n)
|
||||
return pr
|
||||
}
|
||||
fork := recordPR(92, "closed", "maintainer", recordBody(92))
|
||||
fork.Head.Repo = &struct {
|
||||
FullName string `json:"full_name"`
|
||||
}{"someone/cozystack"}
|
||||
cases := []struct {
|
||||
name string
|
||||
prs []pullRequest
|
||||
open int // the open batch the caller found, or 0
|
||||
bot string
|
||||
carrier int // 0 for none
|
||||
isOpen bool
|
||||
refusal string
|
||||
pages int
|
||||
}{
|
||||
{name: "the open batch first", prs: []pullRequest{recordPR(92, "open", "maintainer", recordBody(92)), recordPR(91, "closed", "maintainer", recordBody(91))}, open: 92, carrier: 92, isOpen: true},
|
||||
{name: "the newest closed record", prs: []pullRequest{recordPR(91, "closed", bot, recordBody(91)), recordPR(90, "closed", bot, recordBody(90))}, bot: bot, carrier: 91},
|
||||
{name: "a person's batch without a block is passed over", prs: []pullRequest{recordPR(92, "closed", "maintainer", "picked by hand"), recordPR(91, "closed", "maintainer", recordBody(91))}, carrier: 91},
|
||||
{name: "the bot's batch without a block", prs: []pullRequest{recordPR(92, "closed", bot, "edited away"), recordPR(91, "closed", bot, recordBody(91))}, bot: bot, refusal: "carries no ```backport-batch block"},
|
||||
{name: "a fork is no batch", prs: []pullRequest{fork, recordPR(91, "closed", "maintainer", recordBody(91))}, carrier: 91},
|
||||
{name: "a newer record by someone untrusted", prs: []pullRequest{recordPR(92, "closed", "someone", recordBody(92)), recordPR(91, "closed", "maintainer", recordBody(91))}, refusal: "#92"},
|
||||
{name: "a closed record newer than the open batch", prs: []pullRequest{recordPR(92, "closed", "maintainer", recordBody(92)), recordPR(91, "open", "maintainer", recordBody(91))}, open: 91, refusal: "newer than the open batch pull request #91"},
|
||||
{name: "a record on the second page", prs: append(func() (out []pullRequest) {
|
||||
for n := 1000; n < 1000+recentPullsPage; n++ {
|
||||
out = append(out, other(n))
|
||||
}
|
||||
return out
|
||||
}(), recordPR(91, "closed", "maintainer", recordBody(91))), carrier: 91, pages: 2},
|
||||
{name: "ten full pages without a record", prs: func() (out []pullRequest) {
|
||||
for n := 1000; n < 1000+10*recentPullsPage+1; n++ {
|
||||
out = append(out, other(n))
|
||||
}
|
||||
return out
|
||||
}(), refusal: "pull requests", pages: 10},
|
||||
{name: "the bot's record when no bot is trusted", prs: []pullRequest{recordPR(91, "closed", bot, recordBody(91))}, refusal: "#91"},
|
||||
{name: "a hand-written open batch beside a closed record", prs: []pullRequest{recordPR(92, "open", "maintainer", "picked by hand"), recordPR(91, "closed", "maintainer", recordBody(91))}, open: 92, carrier: 91},
|
||||
{name: "no batch ever", prs: []pullRequest{other(5)}},
|
||||
{name: "an unreadable record", prs: []pullRequest{recordPR(91, "closed", "maintainer", "text\n\n"+deferredFence+"\n{\n```\n")}, refusal: "#91 (closed)"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
l, _ := parseLine("release-1.1")
|
||||
gh := &fakeGH{prs: tc.prs, bot: tc.bot}
|
||||
var open *pullRequest
|
||||
for i := range tc.prs {
|
||||
if tc.prs[i].Number == tc.open {
|
||||
open = &tc.prs[i]
|
||||
}
|
||||
}
|
||||
src, err := findLineRecord(gh, l, open)
|
||||
switch {
|
||||
case tc.refusal != "":
|
||||
if err == nil || !strings.Contains(err.Error(), tc.refusal) {
|
||||
t.Errorf("findLineRecord = %+v, %v; want a refusal with %q", src, err, tc.refusal)
|
||||
}
|
||||
case err != nil:
|
||||
t.Errorf("findLineRecord: %v", err)
|
||||
case tc.carrier == 0 && (src.carrier != nil || src.rec != nil):
|
||||
t.Errorf("found #%d, want no record", src.carrier.Number)
|
||||
case tc.carrier != 0 && (src.carrier == nil || src.carrier.Number != tc.carrier || src.rec == nil || src.open != tc.isOpen || src.rec.Deferred[0].Original != tc.carrier):
|
||||
t.Errorf("found %+v, want #%d open=%v", src, tc.carrier, tc.isOpen)
|
||||
}
|
||||
if tc.pages != 0 && gh.listed != tc.pages {
|
||||
t.Errorf("read %d pages, want %d", gh.listed, tc.pages)
|
||||
}
|
||||
if tc.open != 0 && src.code != open {
|
||||
t.Errorf("the batch to build on is %+v, want #%d", src.code, tc.open)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A run that picks nothing while no batch is open writes nothing, since
|
||||
// merged and closed batches are never written: it says its classification is
|
||||
// not kept, and check still reads the merged batch's record.
|
||||
func TestZeroPickRunLeavesNoRecord(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only five", map[string]string{"app/five.txt": "line\n", "app/seven.txt": "line\n"})
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
mergeBatch(f, gh)
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
edits, comments := len(gh.edited), len(gh.issueComments)
|
||||
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if code != 0 || !strings.Contains(stderr, "no batch pull request is open on release-1.1; this run's classification is not retained") {
|
||||
t.Errorf("run = %d, want 0 saying nothing is retained:\n%s", code, stderr)
|
||||
}
|
||||
if len(gh.created) != 1 || len(gh.edited) != edits || len(gh.issueComments) != comments || f.remoteHead(batchBranch1) != "" {
|
||||
t.Errorf("created %q, edited %q, %d comments, branch at %q; want nothing written", gh.created, gh.edited, len(gh.issueComments), f.remoteHead(batchBranch1))
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 1 || !strings.Contains(stdout, "/pull/7 ") || strings.Contains(stdout, "/pull/5 ") || !strings.Contains(stderr, "deferred conflict on #90 (merged)") {
|
||||
t.Errorf("check = %d, want 1 for #7 alone and #5 still a deferred conflict:\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// The open batch's author is asked for once although both the open batch and
|
||||
// its record are read.
|
||||
func TestRecordAsksForAPermissionOnce(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
gh.asked = nil
|
||||
if code, _, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 0 {
|
||||
t.Fatalf("check = %d\n%s", code, stderr)
|
||||
}
|
||||
if len(gh.asked) != 1 {
|
||||
t.Errorf("asked GitHub for %q, want the author once", gh.asked)
|
||||
}
|
||||
}
|
||||
|
||||
// The record and the report name what the classification was made of: the
|
||||
// heads it read, the policy, the tool, the job, the time and the flags.
|
||||
func TestRecordNamesItsInputs(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc5 := f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.onLine("release-1.1", "fix(app): five\n\n(cherry picked from commit "+mc5+")", map[string]string{"app/five.txt": "5\n"})
|
||||
f.tool = strings.Repeat("c", 40) + "+dirty"
|
||||
t.Setenv("GITHUB_SERVER_URL", "https://github.com")
|
||||
t.Setenv("GITHUB_REPOSITORY", "cozystack/cozystack")
|
||||
t.Setenv("GITHUB_RUN_ID", "123")
|
||||
t.Setenv("GITHUB_RUN_ATTEMPT", "2")
|
||||
gh := &fakeGH{}
|
||||
stdout := mustRun(t, f, gh, "run", "--line", "release-1.0", "--no-workflow-files", "--push")
|
||||
l, _ := parseLine("release-1.0")
|
||||
const branch = "backport-batch/release-1.0-20261026"
|
||||
rec, err := parseRecord(stdout, l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := builtFrom{Main: f.git("rev-parse", "origin/main"), Line: f.git("rev-parse", "origin/release-1.0"), Newer: f.git("rev-parse", "origin/release-1.1"),
|
||||
Policy: f.git("rev-parse", "origin/main"), Batch: f.remoteHead(branch), Tool: f.tool, Run: "https://github.com/cozystack/cozystack/actions/runs/123/attempts/2",
|
||||
At: "2026-10-26T06:00:00Z", Options: runOptions{NoWorkflowFiles: true, Mode: modeNew}}
|
||||
if rec.Version != 2 || *rec.Built != want || bodyReportOf(t, stdout, l).doc.Built != want || rec.Report != bodyReportOf(t, stdout, l).doc.ID {
|
||||
t.Errorf("record = %+v, built %+v; want built %+v and the report it names", rec, rec.Built, want)
|
||||
}
|
||||
|
||||
// A policy from another ref, a run by hand that continues the batch.
|
||||
f.git("push", "-q", "origin", "origin/main:refs/heads/policy-draft")
|
||||
f.git("fetch", "-q", "origin")
|
||||
t.Setenv("GITHUB_RUN_ID", "")
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.0", branch, stdout)}
|
||||
mc6 := f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.onLine("release-1.1", "fix(app): six\n\n(cherry picked from commit "+mc6+")", map[string]string{"app/six.txt": "6\n"})
|
||||
stdout = mustRun(t, f, gh, "run", "--line", "release-1.0", "--continue", "--policy", "origin/policy-draft", "--push")
|
||||
if rec, err = parseRecord(stdout, l); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b := rec.Built; b.Policy != f.git("rev-parse", "origin/policy-draft") || b.Main != f.git("rev-parse", "origin/main") || b.Run != "" || b.Options != (runOptions{Mode: modeContinue}) || b.Batch != f.remoteHead(branch) {
|
||||
t.Errorf("built %+v; want the draft's policy, no job and a continuation without the flag", b)
|
||||
}
|
||||
}
|
||||
|
||||
// What a run assembled with --no-workflow-files stays saved through the
|
||||
// operations that only change the record, which take no such flag: check
|
||||
// keeps failing the cut on the wait they leave in place.
|
||||
func TestMetadataOperationsKeepTheMode(t *testing.T) {
|
||||
for _, v1 := range []bool{false, true} {
|
||||
t.Run(fmt.Sprintf("an open record of version 1: %v", v1), func(t *testing.T) {
|
||||
f := workflowWorld(t)
|
||||
f.onLine("release-1.1", "fix(app): a line-only nine", map[string]string{"app/nine.txt": "line\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
f.pr(9, "fix(app): nine", "", map[string]string{"app/nine.txt": "9\n"})
|
||||
gh := &fakeGH{}
|
||||
stdout := mustRun(t, f, gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push")
|
||||
reopen(f, gh)
|
||||
if v1 {
|
||||
waits := []pick{{v: &verdict{o: &original{number: 5}}, files: []string{".github/workflows/five.yaml"}}}
|
||||
gh.prs[0].Body = strings.Replace(stdout, strings.Split(strings.Split(stdout, deferredFence+"\n")[1], "\n")[0], strings.Split(deferredBlock(notifyLine, nil, nil, waits), "\n")[1], 1)
|
||||
} else if rec, err := parseRecord(stdout, notifyLine); err != nil || !rec.Built.Options.NoWorkflowFiles {
|
||||
t.Fatalf("the scheduled run saved %+v, %v", rec, err)
|
||||
}
|
||||
for _, op := range []struct {
|
||||
args []string
|
||||
mode string
|
||||
}{
|
||||
{[]string{"--drop", "9", "--reason", "needs #4"}, modeDrop},
|
||||
{[]string{"--undrop", "9"}, modeUndrop},
|
||||
{[]string{"--defer", "7", "--reason", "fails the e2e"}, modeDefer},
|
||||
} {
|
||||
stdout = mustRun(t, f, gh, append(append([]string{"run", "--line", "release-1.1"}, op.args...), "--push")...)
|
||||
rec, err := parseRecord(stdout, notifyLine)
|
||||
if err != nil || rec.Built.Options != (runOptions{NoWorkflowFiles: true, Mode: op.mode}) {
|
||||
t.Errorf("after %q the record is %+v, %v; want the flag kept", op.args, rec.Built, err)
|
||||
}
|
||||
reopen(f, gh)
|
||||
}
|
||||
var rec *lineRecord
|
||||
var err error
|
||||
reopen(f, gh)
|
||||
if code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1", "--cut"); code != 1 || !strings.Contains(stdout, "#5 changes .github/workflows/") {
|
||||
t.Errorf("check --cut = %d, want #5 still waiting:\n%s", code, stdout)
|
||||
}
|
||||
|
||||
// Deferring the last picks closes the batch, and its record
|
||||
// still says how the batch was assembled.
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--defer", "6,8", "--reason", "fails the e2e", "--push")
|
||||
rec, err = parseRecord(gh.find(90).Body, notifyLine)
|
||||
if err != nil || gh.find(90).State != "closed" || rec.Closing == nil || rec.Built.Options != (runOptions{NoWorkflowFiles: true, Mode: modeClose}) {
|
||||
t.Errorf("after the close the record is %+v, %v; want the flag kept", rec.Built, err)
|
||||
}
|
||||
if code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1", "--cut"); code != 1 || !strings.Contains(stdout, "/pull/5 ") {
|
||||
t.Errorf("check --cut = %d, want #5 missing once no batch lists it:\n%s", code, stdout)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A state is named only when it is one GitHub uses; anything else is quoted.
|
||||
func TestPRStateNamesOnlyKnownStates(t *testing.T) {
|
||||
at := "2026-10-26T12:00:00Z"
|
||||
for _, tc := range []struct {
|
||||
pr pullRequest
|
||||
want string
|
||||
}{
|
||||
{pullRequest{State: "open"}, "open"},
|
||||
{pullRequest{State: ""}, "open"},
|
||||
{pullRequest{State: "closed"}, "closed"},
|
||||
{pullRequest{State: "closed", MergedAt: &at}, "merged"},
|
||||
{pullRequest{State: "open\n\x1b[2Jforged"}, `in state "open\n\x1b[2Jforged"`},
|
||||
} {
|
||||
if got := prState(&tc.pr); got != tc.want {
|
||||
t.Errorf("prState(%q) = %q, want %q", tc.pr.State, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,752 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"runtime/debug"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A report is one classification of a line: where every original merged into
|
||||
// main since the line branched stands, and what it was made of. The batch
|
||||
// pull request's body carries the latest one beside the record, and an issue
|
||||
// comment the tool never edits keeps each one, so a later run that replaces
|
||||
// the body loses none of them. Its ID hashes the classification, not the pull
|
||||
// request it lands on, which a new batch does not have yet when the report is
|
||||
// made.
|
||||
|
||||
const (
|
||||
reportFence = "```backport-batch-report"
|
||||
reportMarker = "<!-- backport-batch:report "
|
||||
reportFormat = 1
|
||||
// reportLimit keeps a report comment well inside GitHub's 65536.
|
||||
reportLimit = 60000
|
||||
)
|
||||
|
||||
type reportDoc struct {
|
||||
Format int `json:"format"`
|
||||
ID string `json:"id"`
|
||||
Line string `json:"line"`
|
||||
Built builtFrom `json:"built"`
|
||||
Overrides reportOverrides `json:"overrides"`
|
||||
Primary reportPrimary `json:"primary"`
|
||||
Overlays reportOverlays `json:"overlays"`
|
||||
Diag map[string][]string `json:"diag"`
|
||||
Other []string `json:"other"`
|
||||
}
|
||||
|
||||
type reportOverrides struct {
|
||||
Include []int `json:"backport/include"`
|
||||
Skip []int `json:"backport/skip"`
|
||||
}
|
||||
|
||||
// reportPrimary puts every original in exactly one place.
|
||||
type reportPrimary struct {
|
||||
Picked []int `json:"picked"`
|
||||
Missing []int `json:"missing"`
|
||||
Deferred []deferredEntry `json:"deferred"`
|
||||
Parked []json.RawMessage `json:"parked"`
|
||||
Skipped reportSkipped `json:"skipped"`
|
||||
}
|
||||
|
||||
type reportSkipped struct {
|
||||
OnLine []int `json:"on-line"`
|
||||
BeforeStart []int `json:"before-start"`
|
||||
Type []int `json:"type"`
|
||||
Breaking []int `json:"breaking"`
|
||||
Touches []int `json:"touches"`
|
||||
Newer []int `json:"newer"`
|
||||
SkipLabel []int `json:"skip-label"`
|
||||
Dropped []int `json:"dropped"`
|
||||
}
|
||||
|
||||
// reportOverlays mark originals that are also something else: stale ones
|
||||
// are excluded but still on the batch, and each drop is the record's entry,
|
||||
// with the merge commit it is bound to and its reason.
|
||||
type reportOverlays struct {
|
||||
Dropped []droppedEntry `json:"dropped"`
|
||||
Stale []int `json:"stale"`
|
||||
Changed []int `json:"changed"`
|
||||
}
|
||||
|
||||
func (s *reportSkipped) list(rule int) *[]int {
|
||||
switch rule {
|
||||
case ruleOnLine:
|
||||
return &s.OnLine
|
||||
case ruleBeforeStart:
|
||||
return &s.BeforeStart
|
||||
case ruleType:
|
||||
return &s.Type
|
||||
case ruleBreaking:
|
||||
return &s.Breaking
|
||||
case ruleTouches:
|
||||
return &s.Touches
|
||||
case ruleNewer:
|
||||
return &s.Newer
|
||||
case ruleSkipLabel:
|
||||
return &s.SkipLabel
|
||||
case ruleDropped:
|
||||
return &s.Dropped
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type namedList struct {
|
||||
name string
|
||||
list *[]int
|
||||
}
|
||||
|
||||
// lists names each skipped list as the report does.
|
||||
func (s *reportSkipped) lists() []namedList {
|
||||
return []namedList{
|
||||
{"on-line", &s.OnLine}, {"before-start", &s.BeforeStart}, {"type", &s.Type}, {"breaking", &s.Breaking},
|
||||
{"touches", &s.Touches}, {"newer", &s.Newer}, {"skip-label", &s.SkipLabel}, {"dropped", &s.Dropped},
|
||||
}
|
||||
}
|
||||
|
||||
func numbers(vs []*verdict) []int {
|
||||
out := []int{}
|
||||
for _, v := range vs {
|
||||
out = append(out, v.o.number)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// deferredEntries is what the record and the report say about the conflicts,
|
||||
// the deferrals and the waits of a run.
|
||||
func deferredEntries(conflicts, held, waiting []pick) []deferredEntry {
|
||||
out := []deferredEntry{}
|
||||
for _, p := range conflicts {
|
||||
files := cleanFiles(p.files[:min(len(p.files), filesKept)])
|
||||
out = append(out, deferredEntry{Original: p.v.o.number, Commit: p.failed, Files: files, More: len(p.files) - len(files) + p.more})
|
||||
}
|
||||
for _, p := range held {
|
||||
out = append(out, deferredEntry{Original: p.v.o.number, Files: []string{}, Reason: capped(oneLine(clean(p.reason, false)), titleLimit)})
|
||||
}
|
||||
for _, p := range waiting {
|
||||
files := cleanFiles(p.files[:min(len(p.files), filesKept)])
|
||||
out = append(out, deferredEntry{Original: p.v.o.number, Files: files, More: len(p.files) - len(files) + p.more, Workflows: true})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// buildReport makes the report of a run whose built is set. Every original
|
||||
// main merged since the line branched has to land in exactly one primary
|
||||
// category; one that does not is an error rather than a report that
|
||||
// silently drops it.
|
||||
func buildReport(b *batch, r *report) (reportDoc, error) {
|
||||
doc := reportDoc{Format: reportFormat, Line: b.line.branch(), Built: r.built, Diag: map[string][]string{}, Other: append([]string{}, b.other...)}
|
||||
doc.Overrides = reportOverrides{Include: []int{}, Skip: []int{}}
|
||||
for _, v := range r.overrides {
|
||||
switch v.label {
|
||||
case labelInclude:
|
||||
doc.Overrides.Include = append(doc.Overrides.Include, v.o.number)
|
||||
case labelSkip:
|
||||
doc.Overrides.Skip = append(doc.Overrides.Skip, v.o.number)
|
||||
}
|
||||
}
|
||||
doc.Primary = reportPrimary{Picked: numbers(r.onBranch), Missing: numbers(r.missing), Deferred: deferredEntries(r.deferred, r.held, r.waiting), Parked: []json.RawMessage{}}
|
||||
for _, l := range doc.Primary.Skipped.lists() {
|
||||
*l.list = []int{}
|
||||
}
|
||||
for rule, vs := range r.skipped {
|
||||
list := doc.Primary.Skipped.list(rule)
|
||||
if list == nil {
|
||||
return doc, fmt.Errorf("the report has no place for what rule %d skipped", rule)
|
||||
}
|
||||
*list = append(*list, numbers(vs)...)
|
||||
}
|
||||
doc.Overlays = reportOverlays{Dropped: []droppedEntry{}, Stale: numbers(r.stale), Changed: []int{}}
|
||||
doc.Overlays.Dropped = append(doc.Overlays.Dropped, r.dropped...)
|
||||
doc.canonical()
|
||||
where, err := doc.categories()
|
||||
if err != nil {
|
||||
return doc, err
|
||||
}
|
||||
for _, v := range b.verdicts {
|
||||
if where[v.o.number] == "" {
|
||||
return doc, fmt.Errorf("the report of %s puts #%d nowhere", b.line.branch(), v.o.number)
|
||||
}
|
||||
delete(where, v.o.number)
|
||||
}
|
||||
for n := range where {
|
||||
return doc, fmt.Errorf("the report of %s names #%d, which main did not merge after the line branched", b.line.branch(), n)
|
||||
}
|
||||
doc.ID = reportID(doc)
|
||||
return doc, nil
|
||||
}
|
||||
|
||||
// canonical sorts every list, so equal classifications are equal bytes.
|
||||
func (d *reportDoc) canonical() {
|
||||
lists := []*[]int{&d.Primary.Picked, &d.Primary.Missing, &d.Overrides.Include, &d.Overrides.Skip, &d.Overlays.Stale, &d.Overlays.Changed}
|
||||
for _, l := range d.Primary.Skipped.lists() {
|
||||
lists = append(lists, l.list)
|
||||
}
|
||||
for _, l := range lists {
|
||||
slices.Sort(*l)
|
||||
}
|
||||
slices.SortFunc(d.Primary.Deferred, func(a, b deferredEntry) int { return cmp.Compare(a.Original, b.Original) })
|
||||
slices.SortFunc(d.Overlays.Dropped, func(a, b droppedEntry) int { return cmp.Compare(a.Original, b.Original) })
|
||||
slices.Sort(d.Other)
|
||||
}
|
||||
|
||||
// categories maps each original the report names to its primary category,
|
||||
// refusing one named twice.
|
||||
func (d *reportDoc) categories() (map[int]string, error) {
|
||||
where := map[int]string{}
|
||||
put := func(n int, category string) error {
|
||||
if n <= 0 {
|
||||
return fmt.Errorf("the report puts original %d in %s", n, category)
|
||||
}
|
||||
if prev := where[n]; prev != "" {
|
||||
return fmt.Errorf("the report puts #%d in both %s and %s", n, prev, category)
|
||||
}
|
||||
where[n] = category
|
||||
return nil
|
||||
}
|
||||
for _, n := range d.Primary.Picked {
|
||||
if err := put(n, "picked"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
for _, n := range d.Primary.Missing {
|
||||
if err := put(n, "missing"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
for _, e := range d.Primary.Deferred {
|
||||
if err := put(e.Original, "deferred"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
for _, l := range d.Primary.Skipped.lists() {
|
||||
for _, n := range *l.list {
|
||||
if err := put(n, "skipped."+l.name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
return where, nil
|
||||
}
|
||||
|
||||
// reportID hashes everything a report says but when and in which job it was
|
||||
// made, so a run that finds what the last one found makes the same report.
|
||||
func reportID(doc reportDoc) string {
|
||||
doc.ID, doc.Built.At, doc.Built.Run = "", "", ""
|
||||
data, _ := json.Marshal(doc)
|
||||
sum := sha256.Sum256(data)
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// storedReport is a report as it was written, whose bytes a replay posts
|
||||
// unchanged.
|
||||
type storedReport struct {
|
||||
doc reportDoc
|
||||
raw string
|
||||
}
|
||||
|
||||
func reportJSON(doc reportDoc) string {
|
||||
data, _ := json.Marshal(doc)
|
||||
return string(data)
|
||||
}
|
||||
|
||||
func reportBlock(raw string) string { return reportFence + "\n" + raw + "\n```" }
|
||||
|
||||
// parseReport reads one report line of a batch pull request against l. An
|
||||
// ID that matches the content shows only that nobody changed it since it
|
||||
// was hashed, so everything it says is also held to what a run writes.
|
||||
func parseReport(raw string, l line) (reportDoc, error) {
|
||||
var doc reportDoc
|
||||
if err := decodeStrict(raw, &doc); err != nil {
|
||||
return doc, fmt.Errorf("reading the report: %w", err)
|
||||
}
|
||||
if doc.Format != reportFormat {
|
||||
return doc, fmt.Errorf("the report is format %d, which this backport-batch does not read", doc.Format)
|
||||
}
|
||||
if !reportIDRE.MatchString(doc.ID) || reportID(doc) != doc.ID {
|
||||
return doc, fmt.Errorf("report %.12s does not match what it says", terminalSafe(doc.ID))
|
||||
}
|
||||
if err := doc.validate(l); err != nil {
|
||||
return doc, fmt.Errorf("report %.12s %w", doc.ID, err)
|
||||
}
|
||||
if err := doc.bindDrops(); err != nil {
|
||||
return doc, fmt.Errorf("report %.12s %w", doc.ID, err)
|
||||
}
|
||||
return doc, nil
|
||||
}
|
||||
|
||||
// validate holds a report to what a run writes. Its errors read after
|
||||
// "report <id>".
|
||||
func (d *reportDoc) validate(l line) error {
|
||||
if d.Line != l.branch() {
|
||||
return fmt.Errorf("is for %q, not %s", d.Line, l.branch())
|
||||
}
|
||||
if err := d.Built.validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, sha := range d.Other {
|
||||
if checkSHA(sha) != nil {
|
||||
return fmt.Errorf("counts %q as a commit on main", sha)
|
||||
}
|
||||
}
|
||||
if _, err := d.categories(); err != nil {
|
||||
return err
|
||||
}
|
||||
// The deferred and dropped entries are the record's, held to the
|
||||
// record's rules.
|
||||
kept := lineRecord{Version: 1, Line: d.Line, Deferred: d.Primary.Deferred, Dropped: d.Overlays.Dropped}
|
||||
if err := kept.validate(l); err != nil {
|
||||
return err
|
||||
}
|
||||
for n, lines := range d.Diag {
|
||||
if k, err := strconv.Atoi(n); err != nil || k <= 0 || strconv.Itoa(k) != n {
|
||||
return fmt.Errorf("has diagnostics for %q, which is no original", n)
|
||||
}
|
||||
for _, ln := range lines {
|
||||
if ln != oneLineText(ln) {
|
||||
return fmt.Errorf("has a diagnostic for #%s that is not one clean line: %q", n, ln)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, list := range [][]int{d.Overrides.Include, d.Overrides.Skip, d.Overlays.Stale, d.Overlays.Changed} {
|
||||
for _, n := range list {
|
||||
if n <= 0 {
|
||||
return fmt.Errorf("names original %d", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
return d.links()
|
||||
}
|
||||
|
||||
// links holds the fields of a report that speak of one another to agree as
|
||||
// a run writes them, so no field says something another contradicts. Its
|
||||
// errors read after "report <id>".
|
||||
func (d *reportDoc) links() error {
|
||||
where, err := d.categories()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// PR B gives parked originals a shape; until then none is written.
|
||||
if len(d.Primary.Parked) > 0 {
|
||||
return errors.New("parks originals, which this backport-batch does not do")
|
||||
}
|
||||
drops := map[int]bool{}
|
||||
for _, e := range d.Overlays.Dropped {
|
||||
drops[e.Original] = true
|
||||
}
|
||||
for _, n := range d.Primary.Skipped.Dropped {
|
||||
if !drops[n] {
|
||||
return fmt.Errorf("has #%d dropped, but keeps no drop of it", n)
|
||||
}
|
||||
}
|
||||
// Stale and changed mark something a run took or excluded.
|
||||
for _, o := range []struct {
|
||||
name string
|
||||
list []int
|
||||
}{{"stale", d.Overlays.Stale}, {"changed", d.Overlays.Changed}} {
|
||||
for _, n := range o.list {
|
||||
if c := where[n]; c != "picked" && !strings.HasPrefix(c, "skipped.") {
|
||||
return fmt.Errorf("marks #%d %s, which it has %s", n, o.name, or(c, "nowhere"))
|
||||
}
|
||||
}
|
||||
}
|
||||
// backport/skip decides before any rule, and backport/include lifts
|
||||
// rules 1 to 4.
|
||||
skip := map[int]bool{}
|
||||
for _, n := range d.Overrides.Skip {
|
||||
skip[n] = true
|
||||
if where[n] != "skipped.skip-label" {
|
||||
return fmt.Errorf("has #%d labelled %s, which it has %s", n, labelSkip, or(where[n], "nowhere"))
|
||||
}
|
||||
}
|
||||
for _, n := range d.Primary.Skipped.SkipLabel {
|
||||
if !skip[n] {
|
||||
return fmt.Errorf("has #%d skipped for a %s label it does not list", n, labelSkip)
|
||||
}
|
||||
}
|
||||
for _, n := range d.Overrides.Include {
|
||||
switch c := where[n]; c {
|
||||
case "", "skipped.before-start", "skipped.type", "skipped.breaking", "skipped.touches", "skipped.skip-label":
|
||||
return fmt.Errorf("has #%d labelled %s, which it has %s", n, labelInclude, or(c, "nowhere"))
|
||||
}
|
||||
}
|
||||
for n := range d.Diag {
|
||||
if k, err := strconv.Atoi(n); err != nil || where[k] == "" {
|
||||
return fmt.Errorf("has diagnostics for %q, which it puts nowhere", n)
|
||||
}
|
||||
}
|
||||
for _, e := range d.Primary.Deferred {
|
||||
if e.Workflows && !d.Built.Options.NoWorkflowFiles {
|
||||
return fmt.Errorf("has #%d waiting for its workflow files, but was made bringing them over", e.Original)
|
||||
}
|
||||
}
|
||||
if d.Built.Options.Mode == modeClose && len(d.Primary.Picked)+len(d.Overlays.Stale) > 0 {
|
||||
return errors.New("closes the batch, but still has originals on it")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// bodyReport is the report a batch body against l carries, or nil.
|
||||
func bodyReport(body string, l line) (*storedReport, error) {
|
||||
blocks, err := fencedLines(body, reportFence)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, err
|
||||
case len(blocks) == 0:
|
||||
return nil, nil
|
||||
case len(blocks) > 1:
|
||||
return nil, fmt.Errorf("the body holds %d %s blocks", len(blocks), reportFence)
|
||||
}
|
||||
doc, err := parseReport(blocks[0], l)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &storedReport{doc: doc, raw: blocks[0]}, nil
|
||||
}
|
||||
|
||||
// reportComment is the comment that keeps a report: a marker naming it, and
|
||||
// the report folded away in the same block the body carries. Nothing in it
|
||||
// mentions anyone: numbers are bare, and text sits inside the fence.
|
||||
//
|
||||
// Only checked values reach the summary GitHub renders: an ID of hex digits,
|
||||
// a release line and the date of a time that parses.
|
||||
func reportComment(s storedReport) (string, error) {
|
||||
if !reportIDRE.MatchString(s.doc.ID) {
|
||||
return "", fmt.Errorf("report %q has no ID", s.doc.ID)
|
||||
}
|
||||
if _, err := parseLine(s.doc.Line); err != nil {
|
||||
return "", fmt.Errorf("report %.12s: %w", s.doc.ID, err)
|
||||
}
|
||||
at, err := time.Parse(time.RFC3339, s.doc.Built.At)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("report %.12s was made at %q", s.doc.ID, s.doc.Built.At)
|
||||
}
|
||||
date := at.UTC().Format(time.DateOnly)
|
||||
c := reportMarker + s.doc.ID + " -->\n" +
|
||||
"<details><summary>Backport batch report for " + s.doc.Line + ", " + date + " (" + s.doc.ID[:12] + ")</summary>\n\n" +
|
||||
reportBlock(s.raw) + "\n</details>\n"
|
||||
if len(c) > reportLimit {
|
||||
return "", fmt.Errorf("the report comment is %d bytes, over the %d it may have", len(c), reportLimit)
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
var reportMarkerRE = regexp.MustCompile(`^` + regexp.QuoteMeta(reportMarker) + `([0-9a-f]{64}) -->\n`)
|
||||
|
||||
// findReports reads the report comments on batch pull request n, against l,
|
||||
// by authors the batch trusts, as it trusts a record's author, by ID. One
|
||||
// whose content does not match the ID its marker gives, or is not what a run
|
||||
// writes, is returned apart, with why; a look-alike by anyone else is not
|
||||
// read, whatever it says.
|
||||
func findReports(gh github, n int, l line) (map[string]storedReport, map[string]string, error) {
|
||||
comments, err := gh.comments(n)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
found, bad := map[string]storedReport{}, map[string]string{}
|
||||
for _, c := range comments {
|
||||
if !strings.HasPrefix(c.Body, reportMarker) {
|
||||
continue
|
||||
}
|
||||
why, err := trustedAuthor(gh, c.User.Login, c.User.Type)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if why != "" {
|
||||
continue
|
||||
}
|
||||
m := reportMarkerRE.FindStringSubmatch(c.Body)
|
||||
if m == nil {
|
||||
bad[strconv.FormatInt(c.ID, 10)] = fmt.Sprintf("comment %d starts like a report but names none", c.ID)
|
||||
continue
|
||||
}
|
||||
blocks, err := fencedLines(c.Body, reportFence)
|
||||
if err == nil && len(blocks) != 1 {
|
||||
err = fmt.Errorf("it holds %d reports", len(blocks))
|
||||
}
|
||||
var doc reportDoc
|
||||
if err == nil {
|
||||
doc, err = parseReport(blocks[0], l)
|
||||
}
|
||||
if err == nil && doc.ID != m[1] {
|
||||
err = fmt.Errorf("its marker names %.12s", m[1])
|
||||
}
|
||||
if err != nil {
|
||||
bad[m[1]] = fmt.Sprintf("comment %d: %v", c.ID, err)
|
||||
continue
|
||||
}
|
||||
found[m[1]] = storedReport{doc: doc, raw: blocks[0]}
|
||||
}
|
||||
return found, bad, nil
|
||||
}
|
||||
|
||||
// toolVersion is the commit this binary was built from, with +dirty when its
|
||||
// tree had changes, or "" when the build did not record it, as go run does
|
||||
// not.
|
||||
func toolVersion() string {
|
||||
info, ok := debug.ReadBuildInfo()
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
rev, dirty := "", false
|
||||
for _, s := range info.Settings {
|
||||
switch s.Key {
|
||||
case "vcs.revision":
|
||||
rev = s.Value
|
||||
case "vcs.modified":
|
||||
dirty = s.Value == "true"
|
||||
}
|
||||
}
|
||||
if checkSHA(rev) != nil {
|
||||
return ""
|
||||
}
|
||||
if dirty {
|
||||
rev += "+dirty"
|
||||
}
|
||||
return rev
|
||||
}
|
||||
|
||||
// runURL is the Actions run the command runs in, from the variables GitHub
|
||||
// sets for it, or "" outside one.
|
||||
func runURL(lookup func(string) (string, bool)) string {
|
||||
get := func(key string) string { v, _ := lookup(key); return v }
|
||||
url := get("GITHUB_SERVER_URL") + "/" + get("GITHUB_REPOSITORY") + "/actions/runs/" + get("GITHUB_RUN_ID")
|
||||
if attempt := get("GITHUB_RUN_ATTEMPT"); attempt != "" && attempt != "1" {
|
||||
url += "/attempts/" + attempt
|
||||
}
|
||||
if !runURLRE.MatchString(url) {
|
||||
return ""
|
||||
}
|
||||
return url
|
||||
}
|
||||
|
||||
// report prints a report a batch pull request keeps, writing nothing, so it
|
||||
// works on merged and closed batches too. It reads GitHub, and for a report
|
||||
// that keeps drops also this clone's history at the heads the report was
|
||||
// made from, which it never fetches.
|
||||
func (a *app) report(args []string) int {
|
||||
fs := a.flags("report")
|
||||
number := fs.Int("pr", 0, "the batch pull request")
|
||||
id := fs.String("id", "", "the report, by its ID (default: the one the body carries)")
|
||||
asJSON := fs.Bool("json", false, "print the report's JSON")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
if *number <= 0 {
|
||||
return a.fail(fmt.Errorf("--pr %d is not a pull request number", *number))
|
||||
}
|
||||
if *id != "" && !reportIDRE.MatchString(*id) {
|
||||
return a.fail(fmt.Errorf("--id %q is not a report ID", *id))
|
||||
}
|
||||
pr, err := a.gh.pull(*number)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
l, err := parseLine(pr.Base.Ref)
|
||||
if m := batchBranchRE.FindStringSubmatch(pr.Head.Ref); err != nil || m == nil || m[1] != l.branch() || pr.Head.Repo == nil || pr.Head.Repo.FullName != pr.Base.Repo.FullName {
|
||||
return a.fail(fmt.Errorf("#%d is not a batch pull request of this repository", pr.Number))
|
||||
}
|
||||
why, err := trusted(a.gh, pr)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if why != "" {
|
||||
return a.fail(fmt.Errorf("batch pull request #%d was %s, so what it says is not the batch's", pr.Number, why))
|
||||
}
|
||||
inBody, err := bodyReport(pr.Body, l)
|
||||
if err != nil {
|
||||
return a.fail(fmt.Errorf("batch pull request #%d: %w", pr.Number, err))
|
||||
}
|
||||
if *id == "" {
|
||||
if inBody == nil {
|
||||
return a.fail(fmt.Errorf("batch pull request #%d carries no report in its body; name one with --id", pr.Number))
|
||||
}
|
||||
*id = inBody.doc.ID
|
||||
}
|
||||
posted, bad, err := findReports(a.gh, pr.Number, l)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if why := bad[*id]; why != "" {
|
||||
return a.fail(fmt.Errorf("batch pull request #%d: the report %.12s is not what its comment says: %s", pr.Number, *id, why))
|
||||
}
|
||||
s, inComment := posted[*id]
|
||||
switch {
|
||||
case inComment:
|
||||
case inBody != nil && inBody.doc.ID == *id:
|
||||
s = *inBody
|
||||
default:
|
||||
return a.fail(fmt.Errorf("batch pull request #%d keeps no report %.12s", pr.Number, *id))
|
||||
}
|
||||
if *asJSON {
|
||||
fmt.Fprintln(a.stdout, s.raw)
|
||||
return 0
|
||||
}
|
||||
a.printReport(&pr, s, inBody, inComment)
|
||||
return 0
|
||||
}
|
||||
|
||||
// printReport prints a report. It was checked when it was read, and every
|
||||
// field still goes through terminalSafe, so nothing it carries spans lines
|
||||
// or drives a terminal.
|
||||
func (a *app) printReport(pr *pullRequest, s storedReport, inBody *storedReport, inComment bool) {
|
||||
d, b := s.doc, s.doc.Built
|
||||
safe := terminalSafe
|
||||
fmt.Fprintf(a.stdout, "report %s for %s, made %s by a %s run", safe(d.ID), safe(d.Line), safe(b.At), safe(b.Options.Mode))
|
||||
if b.Options.NoWorkflowFiles {
|
||||
fmt.Fprint(a.stdout, " with --no-workflow-files")
|
||||
}
|
||||
fmt.Fprintln(a.stdout)
|
||||
fmt.Fprintf(a.stdout, " main %s, line %s, newer line %s, policy %s\n batch %s, tool %s, run %s\n", safe(b.Main), safe(b.Line), safe(or(b.Newer, "none")), safe(b.Policy), safe(b.Batch), safe(or(b.Tool, "unknown")), safe(or(b.Run, "none")))
|
||||
current := "none"
|
||||
if inBody != nil {
|
||||
current = safe(inBody.doc.ID)
|
||||
}
|
||||
switch {
|
||||
case inBody != nil && inBody.doc.ID == d.ID:
|
||||
fmt.Fprintf(a.stdout, " #%d's body carries this report\n", pr.Number)
|
||||
case pr.Head.SHA == b.Batch:
|
||||
fmt.Fprintf(a.stdout, " #%d's head is %.12s, the batch this report describes, but its body carries report %.12s\n", pr.Number, safe(b.Batch), current)
|
||||
default:
|
||||
fmt.Fprintf(a.stdout, " publication unconfirmed: #%d's head is %.12s, not %.12s, and its body carries report %.12s\n", pr.Number, safe(pr.Head.SHA), safe(b.Batch), current)
|
||||
}
|
||||
if !inComment {
|
||||
if prState(pr) == "open" {
|
||||
fmt.Fprintln(a.stdout, " comment not posted; the next backport-batch run --push posts it")
|
||||
} else {
|
||||
fmt.Fprintf(a.stdout, " comment not posted; kept in the body only, since nothing is written to a %s pull request\n", prState(pr))
|
||||
}
|
||||
}
|
||||
type line struct {
|
||||
n int
|
||||
text string
|
||||
}
|
||||
var lines []line
|
||||
for _, n := range d.Primary.Picked {
|
||||
lines = append(lines, line{n, "picked"})
|
||||
}
|
||||
for _, n := range d.Primary.Missing {
|
||||
lines = append(lines, line{n, "missing"})
|
||||
}
|
||||
for _, e := range d.Primary.Deferred {
|
||||
text := "deferred: "
|
||||
switch {
|
||||
case e.Workflows:
|
||||
text += "waits for a maintainer, changes " + strings.Join(e.Files, ", ")
|
||||
case e.Reason != "":
|
||||
text += e.Reason
|
||||
default:
|
||||
text += fmt.Sprintf("conflict at %.12s in %s", e.Commit, strings.Join(e.Files, ", "))
|
||||
}
|
||||
if e.More > 0 {
|
||||
text += fmt.Sprintf(" and %d more", e.More)
|
||||
}
|
||||
lines = append(lines, line{e.Original, text})
|
||||
}
|
||||
for _, l := range d.Primary.Skipped.lists() {
|
||||
for _, n := range *l.list {
|
||||
lines = append(lines, line{n, "skipped." + l.name})
|
||||
}
|
||||
}
|
||||
slices.SortFunc(lines, func(x, y line) int { return cmp.Compare(x.n, y.n) })
|
||||
for _, l := range lines {
|
||||
fmt.Fprintf(a.stdout, "#%d %s\n", l.n, terminalSafe(l.text))
|
||||
}
|
||||
overlay := func(name string, list []int) {
|
||||
if len(list) > 0 {
|
||||
fmt.Fprintf(a.stdout, "%s: %s\n", name, strings.Trim(fmt.Sprint(list), "[]"))
|
||||
}
|
||||
}
|
||||
overlay("stale, still on the batch", d.Overlays.Stale)
|
||||
for _, e := range d.Overlays.Dropped {
|
||||
fmt.Fprintf(a.stdout, "dropped: #%d, merged as %.12s, on #%d: %s\n", e.Original, terminalSafe(e.Merge), e.On, terminalSafe(e.Reason))
|
||||
}
|
||||
overlay("changed in the pick", d.Overlays.Changed)
|
||||
if len(d.Other) > 0 {
|
||||
other := make([]string, len(d.Other))
|
||||
for i, sha := range d.Other {
|
||||
other[i] = safe(sha)
|
||||
}
|
||||
fmt.Fprintf(a.stdout, "commits on main that merged no pull request: %s\n", strings.Join(other, ", "))
|
||||
}
|
||||
}
|
||||
|
||||
func or(s, otherwise string) string {
|
||||
if s == "" {
|
||||
return otherwise
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// postReport writes a report comment on pull request n, after GitHub says
|
||||
// it is still open: nothing is written to a merged or closed one.
|
||||
func (a *app) postReport(n int, s storedReport) error {
|
||||
if err := a.stillOpen(n); err != nil {
|
||||
return err
|
||||
}
|
||||
c, err := reportComment(s)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
name, err := writeTemp("backport-batch-report-*.md", c)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(name)
|
||||
return a.gh.createComment(n, name)
|
||||
}
|
||||
|
||||
// stillOpen reads pull request n again and refuses to go on unless it is
|
||||
// open.
|
||||
func (a *app) stillOpen(n int) error {
|
||||
pr, err := a.gh.pull(n)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if st := prState(&pr); st != "open" {
|
||||
return fmt.Errorf("batch pull request #%d is %s now; nothing is written to it", n, st)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeTemp(pattern, content string) (string, error) {
|
||||
f, err := os.CreateTemp("", pattern)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, err := f.WriteString(content); err != nil {
|
||||
f.Close()
|
||||
os.Remove(f.Name())
|
||||
return "", err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
os.Remove(f.Name())
|
||||
return "", err
|
||||
}
|
||||
return f.Name(), nil
|
||||
}
|
||||
@@ -0,0 +1,707 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// reportsPosted lists the report IDs the comments carry, in the order they
|
||||
// were posted, whoever wrote them.
|
||||
func reportsPosted(gh *fakeGH) []string {
|
||||
var ids []string
|
||||
for _, c := range gh.issueComments {
|
||||
if m := reportMarkerRE.FindStringSubmatch(c.Body); m != nil {
|
||||
ids = append(ids, m[1])
|
||||
}
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
// bodyReportOf is the report a body on release-1.1, or on the line given,
|
||||
// carries, failing the test without one.
|
||||
func bodyReportOf(t *testing.T, body string, on ...line) storedReport {
|
||||
t.Helper()
|
||||
l := notifyLine
|
||||
if len(on) > 0 {
|
||||
l = on[0]
|
||||
}
|
||||
s, err := bodyReport(body, l)
|
||||
if err != nil || s == nil {
|
||||
t.Fatalf("the body carries no readable report: %v\n%s", err, body)
|
||||
}
|
||||
return *s
|
||||
}
|
||||
|
||||
// refusePushes makes the remote refuse every push, as a lost connection or
|
||||
// a protected branch would, until it is called with false.
|
||||
func (f *fixture) refusePushes(on bool) {
|
||||
f.t.Helper()
|
||||
hook := filepath.Join(f.remote, "hooks", "pre-receive")
|
||||
if !on {
|
||||
if err := os.Remove(hook); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err := os.WriteFile(hook, []byte("#!/bin/sh\necho refused >&2\nexit 1\n"), 0o755); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func mustRun(t *testing.T, f *fixture, gh *fakeGH, args ...string) string {
|
||||
t.Helper()
|
||||
code, stdout, stderr := f.exec(gh, args...)
|
||||
if code != 0 {
|
||||
t.Fatalf("%q = %d\n%s", args, code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
return stdout
|
||||
}
|
||||
|
||||
// A report comment keeps each classification: one the next run replaced in
|
||||
// the body is still read back by its ID, also once the batch merged, which
|
||||
// nothing writes to.
|
||||
func TestReportSurvivesContinuation(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): a line-only five", map[string]string{"app/five.txt": "line\n"})
|
||||
mc5 := f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
r1 := bodyReportOf(t, gh.prs[0].Body).doc.ID
|
||||
f.onLine("release-1.1", "fix(app): five, by hand\n\n(cherry picked from commit "+mc5+")", map[string]string{"app/five.txt": "5\n"})
|
||||
f.now = f.now.AddDate(0, 0, 1)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
r2 := bodyReportOf(t, gh.find(90).Body).doc.ID
|
||||
if r1 == r2 || !slices.Equal(reportsPosted(gh), []string{r1, r2}) {
|
||||
t.Fatalf("reports %q; want R1 %.12s, then R2 %.12s", reportsPosted(gh), r1, r2)
|
||||
}
|
||||
read := func(id, want string) {
|
||||
t.Helper()
|
||||
code, stdout, stderr := f.exec(gh, "report", "--pr", "90", "--id", id)
|
||||
if code != 0 || !strings.Contains(stdout, want) {
|
||||
t.Errorf("report --id %.12s = %d, want %q:\n%s%s", id, code, want, stdout, stderr)
|
||||
}
|
||||
}
|
||||
for _, merged := range []bool{false, true} {
|
||||
if merged {
|
||||
mergeBatch(f, gh)
|
||||
}
|
||||
read(r1, "#5 deferred: conflict at ")
|
||||
read(r2, "#5 skipped.on-line")
|
||||
read(r2, "#6 picked")
|
||||
}
|
||||
}
|
||||
|
||||
// Two runs that find the same post one report, and the body carries the one
|
||||
// kept, whatever time and job the second ran in. A look-alike someone else
|
||||
// wrote is no report, and a kept report edited since stops the reader.
|
||||
func TestReportDedupAndTrust(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
t.Setenv("GITHUB_SERVER_URL", "https://github.com")
|
||||
t.Setenv("GITHUB_REPOSITORY", "cozystack/cozystack")
|
||||
t.Setenv("GITHUB_RUN_ID", "100")
|
||||
t.Setenv("GITHUB_RUN_ATTEMPT", "1")
|
||||
|
||||
// The report the next run makes, read off a run that publishes nothing,
|
||||
// planted by someone the batch does not trust.
|
||||
preview := bodyReportOf(t, mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue"))
|
||||
planted, err := reportComment(preview)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
look := issueComment{ID: 7, Body: planted}
|
||||
look.User.Login, look.User.Type = "someone", "User"
|
||||
gh.issueComments = append(gh.issueComments, look)
|
||||
|
||||
f.now = f.now.AddDate(0, 0, 1)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
kept := bodyReportOf(t, gh.find(90).Body)
|
||||
if kept.doc.ID != preview.doc.ID {
|
||||
t.Fatalf("the run made %.12s, the preview %.12s", kept.doc.ID, preview.doc.ID)
|
||||
}
|
||||
f.now = f.now.AddDate(0, 0, 1)
|
||||
t.Setenv("GITHUB_RUN_ID", "101")
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
posted := reportsPosted(gh)
|
||||
if n := len(slices.DeleteFunc(slices.Clone(posted), func(id string) bool { return id != kept.doc.ID })); n != 2 || len(posted) != 3 {
|
||||
t.Errorf("reports %q; want the first, the look-alike and the run's own once", posted)
|
||||
}
|
||||
body := gh.find(90).Body
|
||||
again := bodyReportOf(t, body)
|
||||
rec, err := parseRecord(body, notifyLine)
|
||||
if err != nil || again.raw != kept.raw || rec.Report != again.doc.ID || !strings.Contains(again.raw, `"run":"https://github.com/cozystack/cozystack/actions/runs/100"`) {
|
||||
t.Errorf("the second run's body carries %s with record report %v (%v); want the kept bytes of run 100", again.raw, rec, err)
|
||||
}
|
||||
|
||||
// The reader takes the run's own comment, not the look-alike.
|
||||
if code, stdout, stderr := f.exec(gh, "report", "--pr", "90", "--id", kept.doc.ID); code != 0 || !strings.Contains(stdout, "#6 picked") {
|
||||
t.Errorf("report = %d:\n%s%s", code, stdout, stderr)
|
||||
}
|
||||
for i := range gh.issueComments {
|
||||
c := &gh.issueComments[i]
|
||||
if c.User.Login == "maintainer" && strings.Contains(c.Body, kept.doc.ID) {
|
||||
c.Body = strings.Replace(c.Body, `"picked":[5,6]`, `"picked":[5]`, 1)
|
||||
}
|
||||
}
|
||||
if code, stdout, stderr := f.exec(gh, "report", "--pr", "90", "--id", kept.doc.ID); code != 2 || !strings.Contains(stderr, "does not match") {
|
||||
t.Errorf("report of an edited comment = %d, want 2:\n%s%s", code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// The report holds no pull request number, so a new batch's report is whole
|
||||
// before GitHub numbers the pull request, and the reader finds it on
|
||||
// whatever number it got.
|
||||
func TestFirstReportIDIsKnownBeforeTheNumber(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{nextPR: 4242}
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--push")
|
||||
inBody := bodyReportOf(t, gh.bodies[0])
|
||||
// Every list is one, empty or not, so equal reports are equal bytes:
|
||||
// main here has no commit that merged no pull request.
|
||||
if !strings.Contains(inBody.raw, `"other":[]`) {
|
||||
t.Errorf("the report writes its empty list of other commits as something else:\n%s", inBody.raw)
|
||||
}
|
||||
if !slices.Equal(gh.commented, []int{4242}) || !slices.Equal(reportsPosted(gh), []string{inBody.doc.ID}) {
|
||||
t.Fatalf("commented on %v with %q, want #4242 with %.12s", gh.commented, reportsPosted(gh), inBody.doc.ID)
|
||||
}
|
||||
found, bad, err := findReports(gh, 4242, notifyLine)
|
||||
if err != nil || len(bad) != 0 || found[inBody.doc.ID].raw != inBody.raw {
|
||||
t.Errorf("findReports = %v, %v, %v; want the body's report", found, bad, err)
|
||||
}
|
||||
if code, stdout, stderr := f.exec(gh, "report", "--pr", "4242"); code != 0 || !strings.Contains(stdout, "report "+inBody.doc.ID) || !strings.Contains(stdout, "#4242's body carries this report") {
|
||||
t.Errorf("report --pr 4242 = %d:\n%s%s", code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// Every original main merged since the line branched is in exactly one
|
||||
// category of the report, a commit that merged no pull request is counted
|
||||
// apart, and the report says so whatever the run had to do.
|
||||
func TestReportPartition(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
gh := &fakeGH{labels: map[string][]int{labelSkip: {9}}}
|
||||
// onNewer puts an original on release-1.1, as the newer line's batch
|
||||
// would, so rule 6 lets it onto release-1.0.
|
||||
onNewer := func(mc string, files map[string]string) {
|
||||
f.onLine("release-1.1", "fix: by the newer batch\n\n(cherry picked from commit "+mc+")", files)
|
||||
}
|
||||
pr := func(n int, title string, files map[string]string, newer bool) {
|
||||
mc := f.pr(n, title, "", files)
|
||||
if newer {
|
||||
onNewer(mc, files)
|
||||
}
|
||||
}
|
||||
t.Setenv("GIT_COMMITTER_DATE", "2026-01-01T00:00:00Z")
|
||||
f.pr(4, "fix(app): four, before the start", "", map[string]string{"app/four.txt": "4\n"})
|
||||
os.Unsetenv("GIT_COMMITTER_DATE")
|
||||
f.setPolicy("release-1.0: 2026-06-01\n")
|
||||
f.onLine("release-1.0", "fix(app): a line-only eleven", map[string]string{"app/eleven.txt": "line\n"})
|
||||
pr(5, "fix(app): five", map[string]string{"app/five.txt": "5\n"}, true)
|
||||
pr(6, "fix(app): six, not on the newer line", map[string]string{"app/six.txt": "6\n"}, false)
|
||||
pr(7, "fix(app)!: seven", map[string]string{"app/seven.txt": "7\n"}, true)
|
||||
pr(8, "fix(platform): eight", map[string]string{migrationsDir + "42": "#!/bin/sh\n"}, true)
|
||||
pr(9, "fix(app): nine", map[string]string{"app/nine.txt": "9\n"}, true)
|
||||
mc10 := f.pr(10, "fix(app): ten", "", map[string]string{"app/ten.txt": "10\n"})
|
||||
onNewer(mc10, map[string]string{"app/ten.txt": "10\n"})
|
||||
f.onLine("release-1.0", "fix(app): ten\n\n(cherry picked from commit "+mc10+")", map[string]string{"app/ten.txt": "10\n"})
|
||||
pr(11, "fix(app): eleven", map[string]string{"app/eleven.txt": "11\n"}, true)
|
||||
pr(12, "fix(ci): twelve", map[string]string{".github/workflows/twelve.yaml": "on: push\n"}, true)
|
||||
pr(13, "fix(app): thirteen", map[string]string{"app/thirteen.txt": "13\n"}, true)
|
||||
f.git("switch", "-q", "main")
|
||||
f.write("app/direct.txt", "pushed\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "chore: a direct push")
|
||||
direct := f.git("rev-parse", "HEAD")
|
||||
f.git("push", "-q", "origin", "main")
|
||||
|
||||
stdout := mustRun(t, f, gh, "run", "--line", "release-1.0", "--no-workflow-files", "--push")
|
||||
const branch = "backport-batch/release-1.0-20261026"
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.0", branch, stdout)}
|
||||
pr(14, "fix(app): fourteen, after the batch", map[string]string{"app/fourteen.txt": "14\n"}, true)
|
||||
stdout = mustRun(t, f, gh, "run", "--line", "release-1.0", "--defer", "13", "--reason", "fails the e2e", "--push")
|
||||
// An active drop, of #5, which the batch carries, and one of a feat,
|
||||
// which rule 2 keeps out first.
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.0", branch, stdout)}
|
||||
stdout = mustRun(t, f, gh, "run", "--line", "release-1.0", "--drop", "5,1", "--reason", "not for 1.0", "--push")
|
||||
|
||||
l, _ := parseLine("release-1.0")
|
||||
s := bodyReportOf(t, stdout, l)
|
||||
got, err := s.doc.categories()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := map[int]string{1: "skipped.type", 4: "skipped.before-start", 5: "skipped.dropped", 6: "skipped.newer", 7: "skipped.breaking",
|
||||
8: "skipped.touches", 9: "skipped.skip-label", 10: "skipped.on-line", 11: "deferred", 12: "deferred", 13: "deferred",
|
||||
14: "missing", 9000: "skipped.type"}
|
||||
for n, category := range want {
|
||||
if got[n] != category {
|
||||
t.Errorf("#%d is %q, want %q", n, got[n], category)
|
||||
}
|
||||
delete(got, n)
|
||||
}
|
||||
if len(got) != 0 {
|
||||
t.Errorf("the report also names %v", got)
|
||||
}
|
||||
if gh.find(90).State != "closed" || !strings.Contains(stdout, `"mode":"close"`) {
|
||||
t.Errorf("dropping the batch's last pick left #90 %s; want it closed", gh.find(90).State)
|
||||
}
|
||||
if !slices.Equal(droppedNumbers(s.doc.Overlays.Dropped), []int{1, 5}) || len(s.doc.Primary.Picked) != 0 {
|
||||
t.Errorf("the report drops %v and picks %v; want #1 and #5 dropped, nothing picked", s.doc.Overlays.Dropped, s.doc.Primary.Picked)
|
||||
}
|
||||
if !slices.Contains(s.doc.Other, direct) {
|
||||
t.Errorf("the report does not count the direct push %.12s: %v", direct, s.doc.Other)
|
||||
}
|
||||
kinds := map[int]string{}
|
||||
for _, e := range s.doc.Primary.Deferred {
|
||||
kinds[e.Original] = fmt.Sprintf("commit=%v reason=%q workflows=%v", e.Commit != "", e.Reason, e.Workflows)
|
||||
}
|
||||
if kinds[11] != `commit=true reason="" workflows=false` || kinds[12] != `commit=false reason="" workflows=true` || kinds[13] != `commit=false reason="fails the e2e" workflows=false` {
|
||||
t.Errorf("the deferred entries are %v", kinds)
|
||||
}
|
||||
}
|
||||
|
||||
// A report too big for a comment is refused before anything is written: no
|
||||
// push, no comment, no edit, and no replay of the earlier report either.
|
||||
func TestReportTooLargeIsRefusedBeforePush(t *testing.T) {
|
||||
f, gh, head := bigWaits(t, 7)
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--no-workflow-files", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "the report comment is") {
|
||||
t.Errorf("run --push = %d, want a refusal of the report:\n%s", code, stderr)
|
||||
}
|
||||
if f.remoteHead(batchBranch1) != head || len(gh.issueComments) != 0 || len(gh.edited) != 0 {
|
||||
t.Errorf("the refused run moved the batch to %.12s, wrote %d comments and edited %q", f.remoteHead(batchBranch1), len(gh.issueComments), gh.edited)
|
||||
}
|
||||
}
|
||||
|
||||
// bigWaits opens a batch whose report comment failed, so its body keeps a
|
||||
// report no comment does, and then merges n originals that each wait on 50
|
||||
// workflow files with long names, which both the record and the report list
|
||||
// in full.
|
||||
func bigWaits(t *testing.T, n int) (*fixture, *fakeGH, string) {
|
||||
t.Helper()
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
gh := &fakeGH{fail: map[string]error{"createComment": errString("HTTP 502")}}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push"); code != 2 || len(gh.created) != 1 {
|
||||
t.Fatalf("the first run = %d, want its comment to fail after the pull request opened\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
for i := range n {
|
||||
files := map[string]string{}
|
||||
for j := range filesKept {
|
||||
files[fmt.Sprintf(".github/workflows/%s-%02d-%02d.yaml", strings.Repeat("w", 170), i, j)] = "on: push\n"
|
||||
}
|
||||
f.pr(10+i, fmt.Sprintf("fix(ci): wait %d", i), "", files)
|
||||
}
|
||||
f.now = f.now.AddDate(0, 0, 1)
|
||||
return f, gh, f.remoteHead(batchBranch1)
|
||||
}
|
||||
|
||||
// A failure between any two writes leaves what was written readable, and the
|
||||
// next run, whatever changed meanwhile, replays a report the body kept but no
|
||||
// comment did, byte for byte, and posts nothing twice. A batch closed in
|
||||
// between is never written again.
|
||||
func TestReportReplayAfterEachFailure(t *testing.T) {
|
||||
setup := func(t *testing.T) (*fixture, *fakeGH) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
return f, &fakeGH{}
|
||||
}
|
||||
// change moves a label, main, the clock and the run's flags between
|
||||
// the failure and the retry, so the retry's report is another one.
|
||||
change := func(f *fixture, gh *fakeGH) []string {
|
||||
gh.labels = map[string][]int{labelSkip: {6}}
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
f.now = f.now.AddDate(0, 0, 1)
|
||||
return []string{"run", "--line", "release-1.1", "--continue", "--no-workflow-files", "--push"}
|
||||
}
|
||||
once := func(t *testing.T, gh *fakeGH, ids ...string) {
|
||||
t.Helper()
|
||||
posted := reportsPosted(gh)
|
||||
for _, id := range ids {
|
||||
if n := len(slices.DeleteFunc(slices.Clone(posted), func(p string) bool { return p != id })); n != 1 {
|
||||
t.Errorf("report %.12s is posted %d times: %q", id, n, posted)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
t.Run("a new batch whose pull request was not opened", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
gh.fail = map[string]error{"createPull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if len(gh.prs) != 0 || len(gh.issueComments) != 0 || f.remoteHead(batchBranch1) == "" {
|
||||
t.Errorf("after the failure: %d pull requests, %d comments, branch %q; want the branch alone", len(gh.prs), len(gh.issueComments), f.remoteHead(batchBranch1))
|
||||
}
|
||||
mustRun(t, f, gh, change(f, gh)...)
|
||||
if len(gh.prs) != 1 || len(reportsPosted(gh)) != 1 || bodyReportOf(t, gh.prs[0].Body).doc.ID != reportsPosted(gh)[0] {
|
||||
t.Errorf("the retry opened %d pull requests with reports %q", len(gh.prs), reportsPosted(gh))
|
||||
}
|
||||
})
|
||||
t.Run("a new batch whose report comment failed", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
gh.fail = map[string]error{"createComment": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push"); code != 2 || !strings.Contains(stderr, "the body keeps it") {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
first := bodyReportOf(t, gh.find(90).Body)
|
||||
mustRun(t, f, gh, change(f, gh)...)
|
||||
second := bodyReportOf(t, gh.find(90).Body)
|
||||
want, err := reportComment(first)
|
||||
if err != nil || len(gh.issueComments) != 2 || gh.issueComments[0].Body != want || reportsPosted(gh)[1] != second.doc.ID || first.doc.ID == second.doc.ID {
|
||||
t.Errorf("comments %q; want the first report replayed as it was, then the retry's", reportsPosted(gh))
|
||||
}
|
||||
})
|
||||
t.Run("a continuation whose push failed", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
openBatch(t, f, gh)
|
||||
first := reportsPosted(gh)[0]
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
f.refusePushes(true)
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
f.refusePushes(false)
|
||||
attempted := reportsPosted(gh)[1]
|
||||
if bodyReportOf(t, gh.find(90).Body).doc.ID != first {
|
||||
t.Error("the body moved on although the push failed")
|
||||
}
|
||||
mustRun(t, f, gh, change(f, gh)...)
|
||||
once(t, gh, first, attempted)
|
||||
if len(reportsPosted(gh)) != 3 {
|
||||
t.Errorf("reports %q; want the first, the attempted and the retry's", reportsPosted(gh))
|
||||
}
|
||||
})
|
||||
t.Run("a continuation whose body edit failed", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
openBatch(t, f, gh)
|
||||
first := reportsPosted(gh)[0]
|
||||
before := f.remoteHead(batchBranch1)
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
gh.fail = map[string]error{"editPull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
attempted := reportsPosted(gh)[1]
|
||||
if f.remoteHead(batchBranch1) == before || bodyReportOf(t, gh.find(90).Body).doc.ID != first {
|
||||
t.Error("the push or the body is not where the failure left them")
|
||||
}
|
||||
mustRun(t, f, gh, change(f, gh)...)
|
||||
once(t, gh, first, attempted)
|
||||
if last := reportsPosted(gh)[len(reportsPosted(gh))-1]; bodyReportOf(t, gh.find(90).Body).doc.ID != last || len(reportsPosted(gh)) != 3 {
|
||||
t.Errorf("reports %q; want the body to carry the retry's, posted last", reportsPosted(gh))
|
||||
}
|
||||
})
|
||||
t.Run("a batch closed while the run writes to it", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
openBatch(t, f, gh)
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
gh.after = map[string]func(){"createComment": func() { gh.find(90).State = "closed" }}
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "batch pull request #90 is closed now; nothing is written to it") || len(gh.edited) != 0 {
|
||||
t.Errorf("run = %d, edited %q; want 2 and no edit of the closed pull request:\n%s", code, gh.edited, stderr)
|
||||
}
|
||||
})
|
||||
t.Run("a batch closed before its report was posted", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
gh.fail = map[string]error{"createComment": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
kept := bodyReportOf(t, gh.find(90).Body)
|
||||
gh.find(90).State = "closed"
|
||||
gh.nextPR = 91
|
||||
mustRun(t, f, gh, change(f, gh)...)
|
||||
if slices.Contains(gh.commented, 90) || slices.Contains(reportsPosted(gh), kept.doc.ID) {
|
||||
t.Errorf("commented on %v, posting %q; want nothing on the closed #90", gh.commented, reportsPosted(gh))
|
||||
}
|
||||
code, stdout, stderr := f.exec(gh, "report", "--pr", "90")
|
||||
if code != 0 || !strings.Contains(stdout, "report "+kept.doc.ID) || !strings.Contains(stdout, "kept in the body only") {
|
||||
t.Errorf("report --pr 90 = %d:\n%s%s", code, stdout, stderr)
|
||||
}
|
||||
})
|
||||
// Every comment is posted only after GitHub says the batch is still
|
||||
// open, the replay's included, so a close between the two stops the
|
||||
// second.
|
||||
t.Run("a batch closed between the replay and the new report", func(t *testing.T) {
|
||||
f, gh := setup(t)
|
||||
gh.fail = map[string]error{"createComment": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
kept := bodyReportOf(t, gh.find(90).Body)
|
||||
f.pr(8, "fix(app): eight", "", map[string]string{"app/eight.txt": "8\n"})
|
||||
gh.after = map[string]func(){"createComment": func() { gh.find(90).State = "closed" }}
|
||||
code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push")
|
||||
if code != 2 || !strings.Contains(stderr, "batch pull request #90 is closed now; nothing is written to it") || !slices.Equal(reportsPosted(gh), []string{kept.doc.ID}) || len(gh.edited) != 0 {
|
||||
t.Errorf("run = %d, posted %q, edited %q; want 2 after the replay alone:\n%s", code, reportsPosted(gh), gh.edited, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// The reader says how a report stands against the pull request as GitHub
|
||||
// shows it now, and claims publication only when the head or the body shows
|
||||
// it: a failed push, a push whose answer was lost, a body that was not
|
||||
// updated and a rebuild since all read as they are.
|
||||
func TestReportPublicationStatus(t *testing.T) {
|
||||
setup := func(t *testing.T) (*fixture, *fakeGH, string) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
gh := &fakeGH{}
|
||||
openBatch(t, f, gh)
|
||||
return f, gh, reportsPosted(gh)[0]
|
||||
}
|
||||
status := func(t *testing.T, f *fixture, gh *fakeGH, id, want string) {
|
||||
t.Helper()
|
||||
code, stdout, stderr := f.exec(gh, "report", "--pr", "90", "--id", id)
|
||||
if code != 0 || !strings.Contains(stdout, want) {
|
||||
t.Errorf("report --id %.12s = %d, want %q:\n%s%s", id, code, want, stdout, stderr)
|
||||
}
|
||||
}
|
||||
t.Run("a push the remote refused, and one it took whose answer was lost", func(t *testing.T) {
|
||||
f, gh, first := setup(t)
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
f.refusePushes(true)
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.refusePushes(false)
|
||||
attempted := reportsPosted(gh)[1]
|
||||
status(t, f, gh, attempted, "publication unconfirmed: #90's head is ")
|
||||
status(t, f, gh, attempted, "its body carries report "+first[:12])
|
||||
// The same head reaching the remote is what a push GitHub took looks
|
||||
// like when its answer never came back.
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
|
||||
f.git("switch", "-q", "main")
|
||||
status(t, f, gh, attempted, "the batch this report describes, but its body carries report "+first[:12])
|
||||
})
|
||||
t.Run("a push followed by a failed body edit", func(t *testing.T) {
|
||||
f, gh, first := setup(t)
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh.fail = map[string]error{"editPull": errString("HTTP 502")}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 2 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
status(t, f, gh, reportsPosted(gh)[1], "the batch this report describes, but its body carries report "+first[:12])
|
||||
status(t, f, gh, first, "#90's body carries this report")
|
||||
})
|
||||
t.Run("a rebuild since", func(t *testing.T) {
|
||||
f, gh, first := setup(t)
|
||||
mustRun(t, f, gh, "run", "--line", "release-1.1", "--defer", "6", "--reason", "x", "--push")
|
||||
second := reportsPosted(gh)[1]
|
||||
status(t, f, gh, first, "publication unconfirmed: #90's head is ")
|
||||
status(t, f, gh, second, "#90's body carries this report")
|
||||
})
|
||||
}
|
||||
|
||||
// forged is a report on release-1.1 with what change sets, under an ID that
|
||||
// matches it: only checking what it says stands in its way.
|
||||
func forged(change func(d *reportDoc)) storedReport {
|
||||
sha := strings.Repeat("a", 40)
|
||||
d := reportDoc{Format: reportFormat, Line: "release-1.1", Diag: map[string][]string{}, Other: []string{sha},
|
||||
Built: builtFrom{Main: sha, Line: sha, Policy: sha, Batch: sha, At: "2026-10-26T06:00:00Z", Options: runOptions{Mode: modeNew}}}
|
||||
d.Primary.Picked = []int{5}
|
||||
change(&d)
|
||||
d.ID = reportID(d)
|
||||
return storedReport{doc: d, raw: reportJSON(d)}
|
||||
}
|
||||
|
||||
// reportRun runs report with gh faked; it reads nothing but GitHub.
|
||||
func reportRun(gh *fakeGH, args ...string) (int, string, string) {
|
||||
var stdout, stderr strings.Builder
|
||||
a := &app{gh: gh, stdout: &stdout, stderr: &stderr, now: func() time.Time { return time.Date(2026, 10, 26, 6, 0, 0, 0, time.UTC) }}
|
||||
code := a.main(append([]string{"report"}, args...))
|
||||
return code, stdout.String(), stderr.String()
|
||||
}
|
||||
|
||||
// A report is read only when everything it says is what a run writes, and
|
||||
// it is for the line of the pull request it sits on: an ID that matches its
|
||||
// content proves only that nobody changed it since it was hashed.
|
||||
func TestReportRefusesWhatItCannotVouchFor(t *testing.T) {
|
||||
for name, change := range map[string]func(d *reportDoc){
|
||||
"a line that is no release line": func(d *reportDoc) { d.Line = "release-1.1\n\x1b[2Jpicked" },
|
||||
"another line": func(d *reportDoc) { d.Line = "release-1.0" },
|
||||
"a main that is no commit": func(d *reportDoc) { d.Built.Main = "main\n#5 picked" },
|
||||
"a time that is none": func(d *reportDoc) { d.Built.At = "2026-10-26T<b>" },
|
||||
"an unknown operation": func(d *reportDoc) { d.Built.Options.Mode = "new\x1b[2J" },
|
||||
"a run that is no Actions run": func(d *reportDoc) { d.Built.Run = "javascript:alert(1)" },
|
||||
"a tool that is no commit": func(d *reportDoc) { d.Built.Tool = "v1\n" },
|
||||
"a commit on main that is none": func(d *reportDoc) { d.Other = []string{"HEAD\x1b]0;x\x07"} },
|
||||
"a deferred commit that is none": func(d *reportDoc) {
|
||||
d.Primary.Deferred = []deferredEntry{{Original: 6, Commit: "-x", Files: []string{"a"}}}
|
||||
},
|
||||
"an original named twice": func(d *reportDoc) { d.Primary.Missing = []int{5} },
|
||||
"an original that is none": func(d *reportDoc) { d.Primary.Skipped.Type = []int{-1} },
|
||||
"a diagnostic of two lines": func(d *reportDoc) { d.Diag = map[string][]string{"5": {"incomplete\n\x1b[2Jforged"}} },
|
||||
"a diagnostic of no original": func(d *reportDoc) { d.Diag = map[string][]string{"five": {"incomplete"}} },
|
||||
// Fields that speak of one another have to agree.
|
||||
"a drop category with no drop": func(d *reportDoc) { d.Primary.Skipped.Dropped = []int{6} },
|
||||
"a stale original it has missing": func(d *reportDoc) {
|
||||
d.Primary.Missing, d.Overlays.Stale = []int{7}, []int{7}
|
||||
},
|
||||
"a stale original it puts nowhere": func(d *reportDoc) { d.Overlays.Stale = []int{8} },
|
||||
"a changed original it defers": func(d *reportDoc) {
|
||||
d.Primary.Deferred = []deferredEntry{{Original: 9, Files: []string{}, Reason: "fails the e2e"}}
|
||||
d.Overlays.Changed = []int{9}
|
||||
},
|
||||
"a skip label on a picked original": func(d *reportDoc) { d.Overrides.Skip = []int{5} },
|
||||
"a skip-label category with no label": func(d *reportDoc) { d.Primary.Skipped.SkipLabel = []int{10} },
|
||||
"an include label on a rule 2 skip": func(d *reportDoc) {
|
||||
d.Primary.Skipped.Type, d.Overrides.Include = []int{11}, []int{11}
|
||||
},
|
||||
"both labels on one original": func(d *reportDoc) {
|
||||
d.Primary.Skipped.SkipLabel, d.Overrides.Include, d.Overrides.Skip = []int{12}, []int{12}, []int{12}
|
||||
},
|
||||
"a label on no original": func(d *reportDoc) { d.Overrides.Include = []int{13} },
|
||||
"a diagnostic of an original it puts nowhere": func(d *reportDoc) { d.Diag = map[string][]string{"14": {"incomplete"}} },
|
||||
"a workflow wait without the flag": func(d *reportDoc) {
|
||||
d.Primary.Deferred = []deferredEntry{{Original: 15, Files: []string{".github/workflows/ci.yaml"}, Workflows: true}}
|
||||
},
|
||||
"a close that still picks": func(d *reportDoc) { d.Built.Options.Mode = modeClose },
|
||||
"a parked original": func(d *reportDoc) { d.Primary.Parked = []json.RawMessage{json.RawMessage(`{"original":16}`)} },
|
||||
} {
|
||||
s := forged(change)
|
||||
c, _ := reportComment(storedReport{doc: forged(func(*reportDoc) {}).doc, raw: s.raw})
|
||||
comment := issueComment{ID: 1, Body: strings.Replace(c, forged(func(*reportDoc) {}).doc.ID, s.doc.ID, 1)}
|
||||
comment.User.Login, comment.User.Type = "maintainer", "User"
|
||||
for where, gh := range map[string]*fakeGH{
|
||||
"in the body": {prs: []pullRequest{notifyPR("20261026", "text\n\n"+reportBlock(s.raw)+"\n")}},
|
||||
"in a comment": {prs: []pullRequest{notifyPR("20261026", "text")}, issueComments: []issueComment{comment}},
|
||||
} {
|
||||
if code, stdout, stderr := reportRun(gh, "--pr", "90", "--id", s.doc.ID); code != 2 || stdout != "" {
|
||||
t.Errorf("%s, %s: report = %d, want 2 and nothing printed:\n%s%s", name, where, code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
}
|
||||
if code, stdout, stderr := reportRun(&fakeGH{prs: []pullRequest{notifyPR("20261026", "text\n\n"+reportBlock(forged(func(*reportDoc) {}).raw)+"\n")}}, "--pr", "90"); code != 0 || !strings.Contains(stdout, "#5 picked") {
|
||||
t.Errorf("report of a sound one = %d:\n%s%s", code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// What report prints stays one line per field and carries nothing a
|
||||
// terminal acts on, whatever reached it.
|
||||
func TestReportPrintsNothingThatDrivesATerminal(t *testing.T) {
|
||||
const evil = "\n\x1b[2Jfake: #9 picked\u202e"
|
||||
s := forged(func(d *reportDoc) {
|
||||
d.Line, d.Built.At, d.Built.Options.Mode = "release-1.1"+evil, "2026-10-26"+evil, "new"+evil
|
||||
d.Built.Main, d.Built.Tool, d.Built.Run, d.Built.Newer = "main"+evil, "tool"+evil, "run"+evil, "newer"+evil
|
||||
d.Other = []string{"other" + evil}
|
||||
d.Primary.Deferred = []deferredEntry{{Original: 6, Reason: "why" + evil}}
|
||||
})
|
||||
var stdout strings.Builder
|
||||
a := &app{stdout: &stdout}
|
||||
pr := notifyPR("20261026", "")
|
||||
a.printReport(&pr, s, nil, true)
|
||||
for _, ln := range strings.Split(stdout.String(), "\n") {
|
||||
if strings.HasPrefix(ln, "fake") || strings.ContainsFunc(ln, func(r rune) bool { return r != '\t' && unicode.IsControl(r) || unicode.Is(unicode.Cf, r) }) {
|
||||
t.Errorf("report printed %q", ln)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The comment is made of what was checked: a line, a time and an ID that
|
||||
// carry no markup into the summary GitHub renders.
|
||||
func TestReportCommentIsBuiltFromCheckedFields(t *testing.T) {
|
||||
for name, change := range map[string]func(d *reportDoc){
|
||||
"markup in the line": func(d *reportDoc) { d.Line = "release-1.1</summary><img src=x onerror=alert(1)>" },
|
||||
"markup in the time": func(d *reportDoc) { d.Built.At = "<b>2026-10-26</b>T06:00:00Z" },
|
||||
"a mention": func(d *reportDoc) { d.Line = "release-1.1 @cozystack/maintainers" },
|
||||
} {
|
||||
if c, err := reportComment(forged(change)); err == nil {
|
||||
t.Errorf("%s: reportComment made\n%s", name, c)
|
||||
}
|
||||
}
|
||||
if _, err := reportComment(forged(func(*reportDoc) {})); err != nil {
|
||||
t.Errorf("reportComment of a sound report: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Anyone can comment on a public pull request: the reader takes a report
|
||||
// from a comment only when the batch trusts its author, whatever ID it
|
||||
// carries.
|
||||
func TestReportReaderTakesOnlyTrustedComments(t *testing.T) {
|
||||
s := forged(func(*reportDoc) {})
|
||||
c, err := reportComment(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
login, kind string
|
||||
read bool
|
||||
}{
|
||||
{"someone", "User", false},
|
||||
{"renovate[bot]", "Bot", false},
|
||||
{"github-actions[bot]", "Bot", false},
|
||||
{"maintainer", "User", true},
|
||||
{defaultBatchBot, "Bot", true},
|
||||
} {
|
||||
comment := issueComment{ID: 1, Body: c}
|
||||
comment.User.Login, comment.User.Type = tc.login, tc.kind
|
||||
gh := &fakeGH{prs: []pullRequest{notifyPR("20261026", "text")}, issueComments: []issueComment{comment}, bot: defaultBatchBot}
|
||||
code, stdout, stderr := reportRun(gh, "--pr", "90", "--id", s.doc.ID)
|
||||
if read := code == 0 && strings.Contains(stdout, "#5 picked"); read != tc.read || !tc.read && (code != 2 || !strings.Contains(stderr, "keeps no report")) {
|
||||
t.Errorf("a report by %s: report = %d, read %v, want %v:\n%s%s", tc.login, code, read, tc.read, stdout, stderr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A report whose ID is not one says so without repeating it as written.
|
||||
func TestReportErrorsRepeatNothingRaw(t *testing.T) {
|
||||
s := forged(func(*reportDoc) {})
|
||||
// Short enough that a cut to 12 characters keeps the forged line.
|
||||
raw := strings.Replace(s.raw, `"id":"`+s.doc.ID+`"`, `"id":"\nforged"`, 1)
|
||||
code, _, stderr := reportRun(&fakeGH{prs: []pullRequest{notifyPR("20261026", "text\n\n"+reportBlock(raw)+"\n")}}, "--pr", "90")
|
||||
for _, ln := range strings.Split(stderr, "\n") {
|
||||
if strings.HasPrefix(ln, "forged") {
|
||||
t.Errorf("report = %d, printed the forged line %q", code, ln)
|
||||
}
|
||||
}
|
||||
if code != 2 {
|
||||
t.Errorf("report = %d, want 2:\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,344 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"maps"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"sigs.k8s.io/yaml"
|
||||
)
|
||||
|
||||
// Rule numbers as docs/release/backports.md numbers them. Rule 5, a clean
|
||||
// pick, is decided while the batch is assembled.
|
||||
const (
|
||||
ruleOnLine = 1
|
||||
ruleType = 2
|
||||
ruleBreaking = 3
|
||||
ruleTouches = 4
|
||||
ruleNewer = 6
|
||||
// Not a rule of the policy: the backport/skip override.
|
||||
ruleSkipLabel = 100
|
||||
// Rule 1 for a line that started under the policy after it branched:
|
||||
// merged before that start.
|
||||
ruleBeforeStart = 101
|
||||
// Not a rule of the policy: a maintainer dropped the original from the
|
||||
// line with run --drop.
|
||||
ruleDropped = 102
|
||||
)
|
||||
|
||||
const (
|
||||
// The numbered scripts run-migrations.sh walks; hack/check-migrations-target.sh
|
||||
// reads the same directory.
|
||||
migrationsDir = "packages/core/platform/images/migrations/migrations/"
|
||||
// Where migrations.targetVersion is set; hack/check-migrations-target.sh
|
||||
// reads it from here.
|
||||
platformValues = "packages/core/platform/values.yaml"
|
||||
// Only a file holding this text at either end of the change can hold a
|
||||
// CRD document; crdDocs decides document by document.
|
||||
crdKind = "CustomResourceDefinition"
|
||||
)
|
||||
|
||||
// The title grammar and the breaking footer are pr-labeler.yaml's, so that a
|
||||
// pull request labelled kind/bug and kind/breaking-change reads the same here.
|
||||
var (
|
||||
backportPrefixRE = regexp.MustCompile(`^(?:\[Backport [^\]]+\]\s+)+`)
|
||||
conventionalRE = regexp.MustCompile(`^([a-z]+)(?:\(([^)]+)\))?(!)?:\s*\S`)
|
||||
breakingFooterRE = regexp.MustCompile(`(?m)^BREAKING[ -]CHANGE:`)
|
||||
htmlCommentRE = regexp.MustCompile(`(?s)<!--.*?-->`)
|
||||
targetVersionRE = regexp.MustCompile(`(?m)^[+-][ \t]*targetVersion[ \t]*:`)
|
||||
)
|
||||
|
||||
// title is a parsed Conventional Commits title.
|
||||
type title struct {
|
||||
typ string
|
||||
scopes []string
|
||||
bang bool
|
||||
}
|
||||
|
||||
func parseTitle(s string) (title, bool) {
|
||||
m := conventionalRE.FindStringSubmatch(backportPrefixRE.ReplaceAllString(s, ""))
|
||||
if m == nil {
|
||||
return title{}, false
|
||||
}
|
||||
t := title{typ: m[1], bang: m[3] == "!"}
|
||||
for scope := range strings.SplitSeq(m[2], ",") {
|
||||
if scope = strings.TrimSpace(scope); scope != "" {
|
||||
t.scopes = append(t.scopes, scope)
|
||||
}
|
||||
}
|
||||
return t, true
|
||||
}
|
||||
|
||||
// typeRule is rule 2 together with the policy's "never in a patch release":
|
||||
// dependency bumps, scope deps, stay out whatever their type.
|
||||
func typeRule(s string) string {
|
||||
t, ok := parseTitle(s)
|
||||
switch {
|
||||
case !ok:
|
||||
return "the title is not a Conventional Commits title"
|
||||
case !slices.Contains(takenTypes, t.typ):
|
||||
return "type " + t.typ
|
||||
case slices.ContainsFunc(t.scopes, func(s string) bool { return slices.Contains(excludedScopes, s) }):
|
||||
return "an excluded scope"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// breakingRule is rule 3. The PR template quotes the footer inside an HTML
|
||||
// comment, which a body merged with the template left in carries.
|
||||
func breakingRule(s, body string) string {
|
||||
if t, ok := parseTitle(s); ok && t.bang {
|
||||
return "! in the title"
|
||||
}
|
||||
if breakingFooterRE.MatchString(htmlCommentRE.ReplaceAllString(body, "")) {
|
||||
return "BREAKING CHANGE footer"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// fileChange is one path a merge changes against its first parent.
|
||||
type fileChange struct {
|
||||
status byte
|
||||
path string
|
||||
}
|
||||
|
||||
func changedFiles(from, to string) ([]fileChange, error) {
|
||||
out, err := git("diff", "--no-ext-diff", "--no-textconv", "--name-status", "-z", "--no-renames", "--end-of-options", from, to)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
f := splitNUL(out)
|
||||
if len(f)%2 != 0 {
|
||||
return nil, fmt.Errorf("unreadable diff of %s..%s", from, to)
|
||||
}
|
||||
var changes []fileChange
|
||||
for i := 0; i < len(f); i += 2 {
|
||||
changes = append(changes, fileChange{status: f[i][0], path: f[i+1]})
|
||||
}
|
||||
return changes, nil
|
||||
}
|
||||
|
||||
func paths(changes []fileChange) []string {
|
||||
out := make([]string, 0, len(changes))
|
||||
for _, c := range changes {
|
||||
out = append(out, c.path)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
const (
|
||||
touchesMigrations = "platform migrations"
|
||||
touchesTargetVersion = "targetVersion"
|
||||
touchesAPI = "a sizeable API change"
|
||||
touchesCRD = "a CRD"
|
||||
)
|
||||
|
||||
// touchesRule is rule 4: what of migrations, targetVersion, the API and CRDs
|
||||
// the merge touches, or "" when it touches none of them.
|
||||
func touchesRule(o *original, changes []fileChange) (string, error) {
|
||||
hit := map[string]bool{}
|
||||
for _, c := range changes {
|
||||
p := c.path
|
||||
switch {
|
||||
case strings.HasPrefix(p, migrationsDir):
|
||||
hit[touchesMigrations] = true
|
||||
case p == platformValues:
|
||||
diff, err := git("--literal-pathspecs", "diff", "-U0", "--no-ext-diff", "--no-textconv", "--end-of-options", o.merge+"^1", o.merge, "--", platformValues)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
hit[touchesTargetVersion] = targetVersionRE.MatchString(diff)
|
||||
}
|
||||
}
|
||||
var err error
|
||||
if hit[touchesCRD], err = changesCRD(o.merge+"^1", o.merge, changes); err != nil {
|
||||
return "", err
|
||||
}
|
||||
asks, err := readsAPI(o.merge+"^1", o.merge, changes)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
apiDetail := ""
|
||||
if asks {
|
||||
if apiDetail, err = sizeableAPIChange(o); err != nil {
|
||||
return "", err
|
||||
}
|
||||
hit[touchesAPI] = apiDetail != ""
|
||||
}
|
||||
var why []string
|
||||
for _, r := range []string{touchesMigrations, touchesTargetVersion, touchesAPI, touchesCRD} {
|
||||
switch {
|
||||
case !hit[r]:
|
||||
case r == touchesAPI:
|
||||
why = append(why, r+" ("+apiDetail+")")
|
||||
default:
|
||||
why = append(why, r)
|
||||
}
|
||||
}
|
||||
return strings.Join(why, ", "), nil
|
||||
}
|
||||
|
||||
// changesCRD reports whether the change adds, removes or modifies a
|
||||
// CustomResourceDefinition document. It looks document by document: an edit to
|
||||
// the DaemonSet beside a CRD in one manifest is not a CRD change, and a CRD in
|
||||
// a test fixture changes nothing a cluster runs.
|
||||
func changesCRD(parent, merge string, changes []fileChange) (bool, error) {
|
||||
var files []string
|
||||
for _, c := range changes {
|
||||
if manifest(c.path) && !testFixture(c.path) {
|
||||
files = append(files, c.path)
|
||||
}
|
||||
}
|
||||
candidates := map[string]bool{}
|
||||
for _, rev := range []string{parent, merge} {
|
||||
found, err := grepFiles(rev, files, "-I", "-e", crdKind)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
for _, f := range found {
|
||||
candidates[f] = true
|
||||
}
|
||||
}
|
||||
for _, c := range changes {
|
||||
if !candidates[c.path] {
|
||||
continue
|
||||
}
|
||||
var before, after map[string]string
|
||||
if c.status != 'A' {
|
||||
content, err := git("cat-file", "blob", parent+":"+c.path)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
before = crdDocs(content)
|
||||
}
|
||||
if c.status != 'D' {
|
||||
content, err := git("cat-file", "blob", merge+":"+c.path)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
after = crdDocs(content)
|
||||
}
|
||||
if !maps.Equal(before, after) {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// manifest reports whether path is a file a chart or a cluster can apply: YAML,
|
||||
// JSON or a Helm template. A CRD quoted in documentation or code applies
|
||||
// nothing.
|
||||
func manifest(path string) bool {
|
||||
switch filepath.Ext(path) {
|
||||
case ".yaml", ".yml", ".json", ".tpl":
|
||||
return true
|
||||
}
|
||||
dirs := strings.Split(path, "/")
|
||||
return slices.ContainsFunc(dirs[:len(dirs)-1], func(d string) bool { return d == "templates" || d == "crds" })
|
||||
}
|
||||
|
||||
func testFixture(path string) bool {
|
||||
dirs := strings.Split(path, "/")
|
||||
return slices.ContainsFunc(dirs[:len(dirs)-1], func(d string) bool { return d == "tests" || d == "testdata" })
|
||||
}
|
||||
|
||||
// grepFiles lists which of files at rev hold any of the fixed strings that
|
||||
// patterns give. Paths are passed literally, so a name such as ":(exclude)x"
|
||||
// means only that file.
|
||||
func grepFiles(rev string, files []string, patterns ...string) ([]string, error) {
|
||||
const chunk = 500
|
||||
var found []string
|
||||
for len(files) > 0 {
|
||||
n := min(chunk, len(files))
|
||||
args := append([]string{"--literal-pathspecs", "grep", "-l", "-z", "-F"}, patterns...)
|
||||
args = append(append(args, "--end-of-options", rev, "--"), files[:n]...)
|
||||
stdout, stderr, code, err := gitStatus(args...)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, err
|
||||
case code != 0 && code != 1:
|
||||
return nil, fmt.Errorf("git grep at %s: exit %d: %s", rev, code, strings.TrimSpace(stderr))
|
||||
}
|
||||
for _, hit := range splitNUL(stdout) {
|
||||
if path, ok := strings.CutPrefix(hit, rev+":"); ok {
|
||||
found = append(found, path)
|
||||
}
|
||||
}
|
||||
files = files[n:]
|
||||
}
|
||||
return found, nil
|
||||
}
|
||||
|
||||
var (
|
||||
docSeparatorRE = regexp.MustCompile(`(?m)^---[ \t]*(?:#.*)?$`)
|
||||
// A document that declares the kind on a line of its own, in YAML or in
|
||||
// JSON, under a bare or quoted key, is a CRD even when it cannot be read;
|
||||
// a sentence or a lookup call that names it declares nothing.
|
||||
crdKindLineRE = regexp.MustCompile(`(?m)^[ \t]*(?:-[ \t]+)?(?:kind|"kind"|'kind')[ \t]*:[ \t]*["']?CustomResourceDefinition["']?[ \t]*,?[ \t]*(?:#.*)?$`)
|
||||
)
|
||||
|
||||
// crdDocs maps the CRD documents of a file to their text, keyed by
|
||||
// metadata.name, so a CRD added, removed or edited changes the map. Every
|
||||
// document that names the kind is read, so a CRD in compact JSON or under a
|
||||
// quoted key counts as well. A document that declares the kind on a line of
|
||||
// its own but cannot be read as an object, a CRD behind a Helm template guard
|
||||
// for one, is kept under its own text, so that any change to it counts: the
|
||||
// check fails closed. So does a list holding a CRD.
|
||||
func crdDocs(content string) map[string]string {
|
||||
docs := map[string]string{}
|
||||
for _, doc := range docSeparatorRE.Split(content, -1) {
|
||||
declares := crdKindLineRE.MatchString(doc)
|
||||
if !declares && !strings.Contains(doc, crdKind) {
|
||||
continue
|
||||
}
|
||||
var head struct {
|
||||
Kind string `json:"kind"`
|
||||
Metadata struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"metadata"`
|
||||
Items []struct {
|
||||
Kind string `json:"kind"`
|
||||
} `json:"items"`
|
||||
}
|
||||
key := ""
|
||||
switch err := yaml.Unmarshal([]byte(doc), &head); {
|
||||
case err != nil && declares:
|
||||
key = "unreadable:" + doc
|
||||
case err != nil:
|
||||
continue
|
||||
case head.Kind == crdKind && head.Metadata.Name != "":
|
||||
key = "crd:" + head.Metadata.Name
|
||||
case head.Kind == crdKind:
|
||||
key = "unnamed:" + doc
|
||||
case strings.HasSuffix(head.Kind, "List") && slices.ContainsFunc(head.Items, func(i struct {
|
||||
Kind string `json:"kind"`
|
||||
}) bool {
|
||||
return i.Kind == crdKind
|
||||
}):
|
||||
key = "list:" + doc
|
||||
default:
|
||||
continue
|
||||
}
|
||||
docs[key] += doc
|
||||
}
|
||||
return docs
|
||||
}
|
||||
@@ -0,0 +1,461 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestTypeRule(t *testing.T) {
|
||||
cases := []struct {
|
||||
title string
|
||||
taken bool
|
||||
}{
|
||||
{"fix(postgres): keep the replica count on resize", true},
|
||||
{"fix: typo in monitoring-agents values.yaml", true},
|
||||
{"security(dashboard): escape the tenant name", true},
|
||||
{"fix(ci): retry the flaky e2e step", true},
|
||||
{"[Backport release-1.6] fix(linstor): x", true},
|
||||
{"[Backport release-1.5] [Backport release-1.6] fix(linstor): x", true},
|
||||
{"fix(platform, dashboard): two scopes", true},
|
||||
{"feat(postgres): add a field", false},
|
||||
{"chore(release): promote", false},
|
||||
{"docs(release): split the page", false},
|
||||
{"refactor(api): move code", false},
|
||||
{"fix(deps): bump golang.org/x/net", false},
|
||||
{"fix(platform, deps): bump and fix", false},
|
||||
{"Fix(postgres): capitalised type", false},
|
||||
{"fix postgres without a colon", false},
|
||||
{"[postgres] bracket form", false},
|
||||
{"fixes(postgres): not the fix type", false},
|
||||
{"", false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
reason := typeRule(tc.title)
|
||||
if (reason == "") != tc.taken {
|
||||
t.Errorf("typeRule(%q) = %q, want taken=%v", tc.title, reason, tc.taken)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBreakingRule(t *testing.T) {
|
||||
cases := []struct {
|
||||
name, title, body string
|
||||
breaking bool
|
||||
}{
|
||||
{"plain fix", "fix(x): y", "## What this PR does\n\nFixes it.", false},
|
||||
{"bang", "fix(x)!: y", "", true},
|
||||
{"bang without scope", "fix!: y", "", true},
|
||||
{"bang behind a backport prefix", "[Backport release-1.6] fix(x)!: y", "", true},
|
||||
{"footer", "fix(x): y", "Body.\n\nBREAKING CHANGE: the field is gone", true},
|
||||
{"hyphenated footer", "fix(x): y", "BREAKING-CHANGE: the field is gone", true},
|
||||
{"footer quoted mid-line", "fix(x): y", "This is not a BREAKING CHANGE: nothing changes.", false},
|
||||
{"footer inside the template comment", "fix(x): y", "<!--\nBREAKING CHANGE: example\n-->\nBody.", false},
|
||||
{"lower-case footer", "fix(x): y", "breaking change: no", false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
reason := breakingRule(tc.title, tc.body)
|
||||
if (reason != "") != tc.breaking {
|
||||
t.Errorf("%s: breakingRule = %q, want breaking=%v", tc.name, reason, tc.breaking)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Rule 4 reads the merge itself, so each case is a pull request merged into a
|
||||
// real repository.
|
||||
func TestTouchesRule(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
setup map[string]string // committed on main before the pull request
|
||||
files map[string]string
|
||||
want string
|
||||
}{
|
||||
{name: "a chart change", files: map[string]string{"chart/values.yaml": "replicas: 2\nimage: app:1\n"}},
|
||||
{name: "a migration script", files: map[string]string{migrationsDir + "61": "#!/bin/sh\n"}, want: "platform migrations"},
|
||||
{name: "a migration library file", files: map[string]string{migrationsDir + "lib/common.sh": "x\n"}, want: "platform migrations"},
|
||||
{name: "the migration runner is not a migration", files: map[string]string{"packages/core/platform/images/migrations/run-migrations.sh": "x\n"}},
|
||||
{
|
||||
name: "targetVersion moves",
|
||||
setup: map[string]string{platformValues: "migrations:\n enabled: false\n targetVersion: 60\n"},
|
||||
files: map[string]string{platformValues: "migrations:\n enabled: false\n targetVersion: 61\n"},
|
||||
want: "targetVersion",
|
||||
},
|
||||
{
|
||||
name: "another key of the platform values",
|
||||
setup: map[string]string{platformValues: "migrations:\n enabled: false\n targetVersion: 60\n"},
|
||||
files: map[string]string{platformValues: "migrations:\n enabled: true\n targetVersion: 60\n"},
|
||||
},
|
||||
{name: "api types the gate sees nothing in", setup: demoAPI, files: map[string]string{"api/apps/v1alpha1/types.go": "package v1alpha1\n"}},
|
||||
{name: "pkg/apis is not api/", files: map[string]string{"pkg/apis/apps/types.go": "package apps\n"}},
|
||||
{name: "a CRD added under crds/", files: map[string]string{"packages/system/x/charts/x/crds/a.yaml": crdDoc("as.x.io", "a") + "\n"}, want: "a CRD"},
|
||||
{name: "a ConfigMap under crds/ is not a CRD", files: map[string]string{"packages/system/x/charts/x/crds/a.yaml": "kind: ConfigMap\n"}},
|
||||
{
|
||||
name: "a DaemonSet edit next to an unchanged CRD",
|
||||
setup: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(crdDoc("as.x.io", "a"), daemonSet("1"))},
|
||||
files: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(crdDoc("as.x.io", "a"), daemonSet("2"))},
|
||||
},
|
||||
{
|
||||
name: "a real CRD edit in a manifest with other documents",
|
||||
setup: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(crdDoc("as.x.io", "a"), daemonSet("1"))},
|
||||
files: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(crdDoc("as.x.io", "b"), daemonSet("1"))},
|
||||
want: "a CRD",
|
||||
},
|
||||
{
|
||||
name: "a CRD renamed",
|
||||
setup: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(crdDoc("as.x.io", "a"), daemonSet("1"))},
|
||||
files: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(crdDoc("bs.x.io", "a"), daemonSet("1"))},
|
||||
want: "a CRD",
|
||||
},
|
||||
{name: "a CRD in a fixture under tests/", files: map[string]string{"packages/system/x/tests/render_test.yaml": crdDoc("as.x.io", "a")}},
|
||||
{name: "a CRD in a fixture under testdata/", files: map[string]string{"hack/testdata/crd.yaml": crdDoc("as.x.io", "a")}},
|
||||
{
|
||||
name: "a templated CRD edited, which cannot be read",
|
||||
setup: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(templatedCRD("a"), daemonSet("1"))},
|
||||
files: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(templatedCRD("b"), daemonSet("1"))},
|
||||
want: "a CRD",
|
||||
},
|
||||
{
|
||||
name: "a DaemonSet edit next to an unchanged templated CRD",
|
||||
setup: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(templatedCRD("a"), daemonSet("1"))},
|
||||
files: map[string]string{"packages/system/x/templates/m.yaml": multiDoc(templatedCRD("a"), daemonSet("2"))},
|
||||
},
|
||||
{
|
||||
name: "an unparseable file that names the kind, changed",
|
||||
setup: map[string]string{"x/broken.yaml": "kind: CustomResourceDefinition\nspec: [unclosed\n"},
|
||||
files: map[string]string{"x/broken.yaml": "kind: CustomResourceDefinition\nspec: [still unclosed\n"},
|
||||
want: "a CRD",
|
||||
},
|
||||
{
|
||||
name: "a lookup that names the kind in a Helm helper",
|
||||
setup: map[string]string{"packages/system/x/templates/_helpers.tpl": "{{- $crd := lookup \"apiextensions.k8s.io/v1\" \"CustomResourceDefinition\" \"\" .crd -}}\n{{- if $crd }}a{{ end }}\n"},
|
||||
files: map[string]string{"packages/system/x/templates/_helpers.tpl": "{{- $crd := lookup \"apiextensions.k8s.io/v1\" \"CustomResourceDefinition\" \"\" .crd -}}\n{{- if $crd }}b{{ end }}\n"},
|
||||
},
|
||||
{
|
||||
name: "a CRD example in documentation",
|
||||
setup: map[string]string{"docs/crds.md": "Example:\n\n```yaml\nkind: CustomResourceDefinition\nmetadata:\n name: as.x.io\n```\n"},
|
||||
files: map[string]string{"docs/crds.md": "An example:\n\n```yaml\nkind: CustomResourceDefinition\nmetadata:\n name: bs.x.io\n```\n"},
|
||||
},
|
||||
{name: "a CRD in a template without a YAML extension", files: map[string]string{"packages/system/x/templates/crd.tmpl": crdDoc("as.x.io", "a")}, want: "a CRD"},
|
||||
{name: "a CRD added as JSON", files: map[string]string{"x/crd.json": `{"apiVersion": "apiextensions.k8s.io/v1",` + "\n" + ` "kind": "CustomResourceDefinition",` + "\n" + ` "metadata": {"name": "as.x.io"}}` + "\n"}, want: "a CRD"},
|
||||
{name: "a List of CRDs", files: map[string]string{"x/list.yaml": "kind: List\nitems:\n- kind: CustomResourceDefinition\n metadata:\n name: as.x.io\n"}, want: "a CRD"},
|
||||
{name: "a template declaring a CRD", files: map[string]string{"packages/system/x/templates/crd.yaml": "apiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\n"}, want: "a CRD"},
|
||||
{name: "a quoted declaration", files: map[string]string{"x/crd.yaml": " kind: \"CustomResourceDefinition\" # yes\n"}, want: "a CRD"},
|
||||
{
|
||||
name: "a CRD changed in place",
|
||||
setup: map[string]string{"x/crd.yaml": "kind: CustomResourceDefinition\nspec: {}\n"},
|
||||
files: map[string]string{"x/crd.yaml": "kind: CustomResourceDefinition\nspec: {a: 1}\n"},
|
||||
want: "a CRD",
|
||||
},
|
||||
{
|
||||
name: "a CRD deleted",
|
||||
setup: map[string]string{"x/crd.yaml": "kind: CustomResourceDefinition\n"},
|
||||
files: map[string]string{"x/crd.yaml": ""},
|
||||
want: "a CRD",
|
||||
},
|
||||
{
|
||||
name: "a CRD turned into something else",
|
||||
setup: map[string]string{"x/crd.yaml": "kind: CustomResourceDefinition\n"},
|
||||
files: map[string]string{"x/crd.yaml": "kind: ConfigMap\n"},
|
||||
want: "a CRD",
|
||||
},
|
||||
{name: "a CRD list is not a CRD", files: map[string]string{"x/list.yaml": "kind: CustomResourceDefinitionList\n"}},
|
||||
{name: "a CRD as compact JSON", files: map[string]string{"x/crd.json": `{"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","metadata":{"name":"as.x.io"}}` + "\n"}, want: "a CRD"},
|
||||
{name: "a CRD under a quoted kind key", files: map[string]string{"x/crd.yaml": "'kind': CustomResourceDefinition\nmetadata:\n name: as.x.io\n"}, want: "a CRD"},
|
||||
{
|
||||
name: "a templated CRD under a quoted kind key, edited",
|
||||
setup: map[string]string{"packages/system/x/templates/m.yaml": strings.Replace(templatedCRD("a"), "kind:", "'kind':", 1)},
|
||||
files: map[string]string{"packages/system/x/templates/m.yaml": strings.Replace(templatedCRD("b"), "kind:", "'kind':", 1)},
|
||||
want: "a CRD",
|
||||
},
|
||||
{name: "a ConfigMap as compact JSON naming the kind", files: map[string]string{"x/cm.json": `{"kind":"ConfigMap","data":{"note":"kind: CustomResourceDefinition"}}` + "\n"}},
|
||||
{name: "a CRD mentioned in prose", files: map[string]string{"x/README.md": "Set kind: CustomResourceDefinition here.\n"}},
|
||||
{name: "a file named like an option declaring a CRD", files: map[string]string{"-e.yaml": "kind: CustomResourceDefinition\n"}, want: "a CRD"},
|
||||
// As pathspec magic, :(icase)... would match a file of that name in
|
||||
// another case, and there is none.
|
||||
{name: "a file named like pathspec magic declaring a CRD", files: map[string]string{":(icase)zz-crd.yaml": "kind: CustomResourceDefinition\n"}, want: "a CRD"},
|
||||
{
|
||||
name: "a migration and a sizeable API change together",
|
||||
setup: demoAPI,
|
||||
files: map[string]string{migrationsDir + "61": "x\n", "api/x.go": "package x\n", demoRD: cozyrd("demo", `{"type":"object","properties":{}}`)},
|
||||
want: "platform migrations, a sizeable API change (breaking-change: [v1alpha1] spec: field \"size\" was removed)",
|
||||
},
|
||||
}
|
||||
for i, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
setup := maps.Clone(apiBase)
|
||||
maps.Copy(setup, tc.setup)
|
||||
f.squash(1000, "chore: setup", "", setup)
|
||||
mc := f.pr(10+i, "fix(x): y", "", tc.files)
|
||||
o := &original{number: 10 + i, merge: mc, parents: 2}
|
||||
changes, err := changedFiles(mc+"^1", mc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := touchesRule(o, changes)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != tc.want {
|
||||
t.Errorf("touchesRule = %q, want %q", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Rule 6: an older line takes only what the newer line already carries, and
|
||||
// rule 1: neither line takes what it carries already.
|
||||
func TestNewerLineAndOnLine(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(2, "fix(app): before nothing", "", map[string]string{"app/a.txt": "a\n"})
|
||||
f.pr(3, "fix(app): picked onto 1.1", "", map[string]string{"app/b.txt": "b\n"})
|
||||
mc3 := f.git("rev-parse", "HEAD")
|
||||
f.pr(4, "fix(app): only on main", "", map[string]string{"app/c.txt": "c\n"})
|
||||
f.onLine("release-1.1", "fix(app): picked onto 1.1 (#3)\n\n(cherry picked from commit "+mc3+")", map[string]string{"app/b.txt": "b\n"})
|
||||
|
||||
for _, tc := range []struct {
|
||||
line string
|
||||
qualified []int
|
||||
skipped map[int]int
|
||||
}{
|
||||
{line: "release-1.1", qualified: []int{2, 4}, skipped: map[int]int{3: ruleOnLine}},
|
||||
// #1 is a feature; #2 and #4 are not on release-1.1 yet; #3 is.
|
||||
{line: "release-1.0", qualified: []int{3}, skipped: map[int]int{1: ruleType, 2: ruleNewer, 4: ruleNewer}},
|
||||
} {
|
||||
t.Run(tc.line, func(t *testing.T) {
|
||||
f.git("fetch", "-q", "origin")
|
||||
l, _ := parseLine(tc.line)
|
||||
b, err := (&app{gh: &fakeGH{}, stderr: &strings.Builder{}}).prepare(l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var got []int
|
||||
for _, v := range b.qualifying() {
|
||||
got = append(got, v.o.number)
|
||||
}
|
||||
if !slices.Equal(got, tc.qualified) {
|
||||
t.Errorf("qualifying = %v, want %v", got, tc.qualified)
|
||||
}
|
||||
for n, rule := range tc.skipped {
|
||||
if v := b.find(n); v == nil || v.rule != rule {
|
||||
t.Errorf("#%d: verdict %+v, want rule %d", n, v, rule)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An original merged before the newer line branched is on it, even when the
|
||||
// newer line has since changed the same lines and a pick would conflict.
|
||||
func TestNewerLineCarriesWhatItBranchedWith(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.pr(2, "fix(app): two", "", map[string]string{"app/a.txt": "two\n"})
|
||||
f.release("release-1.1")
|
||||
mc3 := f.pr(3, "fix(app): three", "", map[string]string{"app/a.txt": "three\n"})
|
||||
f.onLine("release-1.1", "fix(app): three (#3)\n\n(cherry picked from commit "+mc3+")", map[string]string{"app/a.txt": "three\n"})
|
||||
if v := verdictOn(t, f, nil, "release-1.0", 2); v.rule != 0 {
|
||||
t.Errorf("#2 on release-1.0 = rule %d (%s), want it to qualify", v.rule, v.reason)
|
||||
}
|
||||
}
|
||||
|
||||
// A line branched and not yet released is one the batch runs for, and rule 6
|
||||
// holds each older line to the next newer one.
|
||||
func TestNewerLineInTheRCWindow(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.release("release-1.2")
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
for _, tc := range []struct{ line, reason string }{
|
||||
{"release-1.2", ""},
|
||||
{"release-1.1", "not on release-1.2 yet"},
|
||||
{"release-1.0", "not on release-1.1 yet"},
|
||||
} {
|
||||
v := verdictOn(t, f, nil, tc.line, 5)
|
||||
if v.reason != tc.reason || (v.rule == 0) != (tc.reason == "") {
|
||||
t.Errorf("#5 on %s = rule %d (%s), want %q", tc.line, v.rule, v.reason, tc.reason)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOnlySupportedLines(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.release("release-0.9")
|
||||
f.lines()
|
||||
code, _, stderr := f.exec(nil, "check", "--line", "release-0.9")
|
||||
if code != 2 || !strings.Contains(stderr, "not a line the batch runs for") {
|
||||
t.Errorf("check on release-0.9 = %d (%s), want 2 and a refusal", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// demoRD is an ApplicationDefinition where apigate looks for one, and demoAPI
|
||||
// the tree the API cases start from.
|
||||
const demoRD = "packages/system/demo-rd/cozyrds/demo.yaml"
|
||||
|
||||
// apiBase is the least API a tree holds: apigate refuses a tree with none.
|
||||
var apiBase = map[string]string{"packages/system/keep-rd/cozyrds/keep.yaml": cozyrd("keep", `{"type":"object"}`)}
|
||||
|
||||
var demoAPI = map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","properties":{"size":{"type":"string","description":"Size."}}}`),
|
||||
"api/apps/v1alpha1/demo/types.go": "package demo\n\n// Size of it.\ntype Spec struct{ Size string }\n",
|
||||
}
|
||||
|
||||
func cozyrd(name, schema string) string {
|
||||
return "apiVersion: cozystack.io/v1alpha1\nkind: ApplicationDefinition\nmetadata:\n name: " + name +
|
||||
"\nspec:\n application:\n kind: " + strings.ToUpper(name[:1]) + name[1:] + "\n plural: " + name + "s\n singular: " + name +
|
||||
"\n openAPISchema: '" + schema + "'\n"
|
||||
}
|
||||
|
||||
// Rule 4 holds back a change to the API only when the API review gate calls it
|
||||
// sizeable: a new or removed resource or group, or a breaking schema change.
|
||||
func TestAPIGateRule(t *testing.T) {
|
||||
typesGo := "api/apps/v1alpha1/demo/types.go"
|
||||
cases := []struct {
|
||||
name string
|
||||
files map[string]string
|
||||
want string
|
||||
}{
|
||||
{name: "a description reworded", files: map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","properties":{"size":{"type":"string","description":"The size."}}}`),
|
||||
typesGo: "package demo\n\n// The size.\ntype Spec struct{ Size string }\n",
|
||||
}},
|
||||
{name: "an optional field added", files: map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","properties":{"size":{"type":"string"},"zone":{"type":"string"}}}`),
|
||||
typesGo: "package demo\n\ntype Spec struct{ Size, Zone string }\n",
|
||||
}},
|
||||
{name: "a field removed", files: map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","properties":{}}`),
|
||||
typesGo: "package demo\n\ntype Spec struct{}\n",
|
||||
}, want: "breaking-change"},
|
||||
{name: "a field's type changed", files: map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","properties":{"size":{"type":"integer"}}}`),
|
||||
typesGo: "package demo\n\ntype Spec struct{ Size int }\n",
|
||||
}, want: "breaking-change"},
|
||||
{name: "a field made required", files: map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","required":["size"],"properties":{"size":{"type":"string"}}}`),
|
||||
typesGo: "package demo\n\n// +required\ntype Spec struct{ Size string }\n",
|
||||
}, want: "breaking-change"},
|
||||
{name: "a resource added", files: map[string]string{
|
||||
"packages/system/other-rd/cozyrds/other.yaml": cozyrd("other", `{"type":"object"}`),
|
||||
"api/apps/v1alpha1/other/types.go": "package other\n",
|
||||
}, want: "new-resource"},
|
||||
{name: "a resource removed", files: map[string]string{
|
||||
demoRD: "",
|
||||
typesGo: "",
|
||||
}, want: "removed"},
|
||||
{name: "a breaking schema change that does not touch api/", files: map[string]string{
|
||||
demoRD: cozyrd("demo", `{"type":"object","properties":{}}`),
|
||||
}, want: "breaking-change"},
|
||||
{name: "an aggregated group registered in a chart", files: map[string]string{
|
||||
"packages/system/extra/templates/apiservice.yaml": apiService("extra.cozystack.io"),
|
||||
}, want: "new-api-group"},
|
||||
}
|
||||
for i, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
setup := maps.Clone(apiBase)
|
||||
maps.Copy(setup, demoAPI)
|
||||
f.squash(1000, "chore: setup", "", setup)
|
||||
mc := f.pr(10+i, "fix(x): y", "", tc.files)
|
||||
o := &original{number: 10 + i, merge: mc, parents: 2}
|
||||
changes, err := changedFiles(mc+"^1", mc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := touchesRule(o, changes)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if (got != "") != (tc.want != "") || !strings.Contains(got, tc.want) {
|
||||
t.Errorf("touchesRule = %q, want %q", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func apiService(group string) string {
|
||||
return "apiVersion: apiregistration.k8s.io/v1\nkind: APIService\nmetadata:\n name: v1alpha1." + group +
|
||||
"\nspec:\n group: " + group + "\n version: v1alpha1\n"
|
||||
}
|
||||
|
||||
// The API review gate is asked about exactly the changes that reach what
|
||||
// apigate parses at either end of the merge.
|
||||
func TestReadsAPI(t *testing.T) {
|
||||
crd := crdDoc("as.x.io", "a")
|
||||
cases := []struct {
|
||||
name string
|
||||
setup map[string]string
|
||||
files map[string]string
|
||||
want bool
|
||||
}{
|
||||
{name: "the apiserver's registrations", files: map[string]string{apiserverGo: "package apiserver\n"}, want: true},
|
||||
{name: "an ApplicationDefinition, which carries no marker", files: map[string]string{demoRD: "kind: ApplicationDefinition\n"}, want: true},
|
||||
{name: "a CRD in a chart", files: map[string]string{"packages/system/x/crds/a.yaml": crd}, want: true},
|
||||
{name: "an APIService under internal/", files: map[string]string{"internal/x/apiservice.yml": apiService("x.cozystack.io")}, want: true},
|
||||
{
|
||||
name: "a CRD turned into something else",
|
||||
setup: map[string]string{"packages/system/x/crds/a.yaml": crd},
|
||||
files: map[string]string{"packages/system/x/crds/a.yaml": "kind: ConfigMap\n"},
|
||||
want: true,
|
||||
},
|
||||
{name: "a binary file with a marker", files: map[string]string{"packages/system/x/a.yaml": "\x00kind: APIService\n"}, want: true},
|
||||
{name: "chart values", files: map[string]string{"packages/system/x/values.yaml": "replicas: 2\n"}},
|
||||
{name: "Go types under api/", files: map[string]string{"api/apps/v1alpha1/demo/types.go": "package demo\n"}},
|
||||
{name: "a CRD outside the roots apigate walks", files: map[string]string{"hack/crds/a.yaml": crd}},
|
||||
{name: "a CRD in a template without a YAML extension", files: map[string]string{"packages/system/x/templates/crd.tpl": crd}},
|
||||
{name: "a file beside the ApplicationDefinitions", files: map[string]string{"packages/system/demo-rd/cozyrds/sub/demo.yaml": "kind: ApplicationDefinition\n"}},
|
||||
}
|
||||
for i, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
setup := maps.Clone(apiBase)
|
||||
maps.Copy(setup, tc.setup)
|
||||
f.squash(1000, "chore: setup", "", setup)
|
||||
mc := f.pr(10+i, "fix(x): y", "", tc.files)
|
||||
changes, err := changedFiles(mc+"^1", mc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := readsAPI(mc+"^1", mc, changes)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != tc.want {
|
||||
t.Errorf("readsAPI = %v, want %v", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func crdDoc(name, version string) string {
|
||||
return "apiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n name: " + name +
|
||||
"\nspec:\n group: x.io\n versions:\n - name: v1" + version + "\n"
|
||||
}
|
||||
|
||||
func daemonSet(replicas string) string {
|
||||
return "apiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n name: agent\nspec:\n template:\n spec:\n priority: {{ .Values.priority | default " + replicas + " }}\n"
|
||||
}
|
||||
|
||||
func templatedCRD(version string) string {
|
||||
return "{{- if .Values.crds }}\n" + crdDoc("ts.x.io", version) + "{{- end }}\n"
|
||||
}
|
||||
|
||||
func multiDoc(docs ...string) string { return "---\n" + strings.Join(docs, "---\n") }
|
||||
@@ -0,0 +1,439 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// A batch carries each original as one unit on its first-parent line. A
|
||||
// merged pull request is a merge commit whose first parent is the batch before
|
||||
// it and whose second parent ends a straight line of picks of the pull
|
||||
// request's own commits, oldest first, each keeping its author and sign-offs;
|
||||
// the merge says which merge commit on main it carries in a Backported-from
|
||||
// trailer. A squash-merged original is one pick of its commit on main.
|
||||
|
||||
// pickOriginal picks one original onto HEAD in that shape. A conflict on any
|
||||
// of its commits takes the branch back to where it was before the original,
|
||||
// so nothing of it is left half applied. Hooks are off so that each commit
|
||||
// carries its message and -x line as git writes them; signing is left to the
|
||||
// user's configuration.
|
||||
func pickOriginal(v *verdict) (pick, error) {
|
||||
p := pick{v: v}
|
||||
base, err := resolve("HEAD")
|
||||
if err != nil {
|
||||
return p, err
|
||||
}
|
||||
for _, c := range v.commits {
|
||||
status, files, err := applyPick("-c", "core.hooksPath=/dev/null", "cherry-pick", "-x", "--signoff", "--end-of-options", c)
|
||||
if err != nil {
|
||||
if rerr := resetTo(base); rerr != nil {
|
||||
return p, rerr
|
||||
}
|
||||
return p, fmt.Errorf("cherry-pick of %.12s of #%d: %w", c, v.o.number, err)
|
||||
}
|
||||
if status == "conflict" {
|
||||
p.status, p.files, p.failed = "conflict", files, c
|
||||
return p, resetTo(base)
|
||||
}
|
||||
}
|
||||
head, err := resolve("HEAD")
|
||||
switch {
|
||||
case err != nil:
|
||||
return p, err
|
||||
case head == base:
|
||||
p.status = "empty"
|
||||
return p, nil
|
||||
case v.o.parents < 2:
|
||||
p.status, p.commit = "picked", head
|
||||
return p, nil
|
||||
}
|
||||
msgs, err := commitMessages([]string{v.o.merge})
|
||||
if err != nil {
|
||||
return p, err
|
||||
}
|
||||
if p.commit, err = commitMerge(base, head, mergeMessage(msgs[v.o.merge].message, v.o.merge)); err != nil {
|
||||
if rerr := resetTo(base); rerr != nil {
|
||||
return p, rerr
|
||||
}
|
||||
return p, err
|
||||
}
|
||||
p.status = "picked"
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// mergeMessage is the message of a batch merge: the one of the merge commit
|
||||
// on main, which says which pull request it was, and a paragraph of its own
|
||||
// naming that merge commit, so nothing the original's text ends with can pass
|
||||
// for the trailer.
|
||||
func mergeMessage(message, merge string) string {
|
||||
return strings.TrimRight(message, " \t\r\n") + "\n\n" + batchTrailer + ": " + merge + "\n"
|
||||
}
|
||||
|
||||
// commitMerge makes the merge of tip onto base, whose tree is tip's since tip
|
||||
// descends from base, and moves the current branch from tip to it.
|
||||
func commitMerge(base, tip, message string) (string, error) {
|
||||
f, err := os.CreateTemp("", "backport-batch-merge-*.txt")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if _, err := f.WriteString(message); err != nil {
|
||||
f.Close()
|
||||
return "", err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
out, err := git("commit-tree", "-p", base, "-p", tip, "-F", f.Name(), tip+"^{tree}")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
merge := strings.TrimSpace(out)
|
||||
if err := checkSHA(merge); err != nil {
|
||||
return "", fmt.Errorf("commit-tree: %w", err)
|
||||
}
|
||||
if _, err := git("update-ref", "-m", "backport-batch: merge", "HEAD", merge, tip); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return merge, nil
|
||||
}
|
||||
|
||||
// resetTo takes the current branch, its index and its tree back to commit,
|
||||
// which run owns: it starts only in a clean tree.
|
||||
func resetTo(commit string) error {
|
||||
_, err := git("reset", "--quiet", "--hard", commit)
|
||||
return err
|
||||
}
|
||||
|
||||
// unit is one original as a batch carries it: a commit on the batch's
|
||||
// first-parent line, and for a merge the picks its second parent brings.
|
||||
type unit struct {
|
||||
commit string
|
||||
parents []string
|
||||
message string
|
||||
// picks are the commits reachable from the second parent and not from the
|
||||
// first, oldest first.
|
||||
picks []string
|
||||
}
|
||||
|
||||
func (u unit) merge() bool { return len(u.parents) == 2 }
|
||||
|
||||
func readUnit(commit string) (unit, error) {
|
||||
msgs, err := commitMessages([]string{commit})
|
||||
if err != nil {
|
||||
return unit{}, err
|
||||
}
|
||||
u := unit{commit: commit, parents: msgs[commit].parents, message: msgs[commit].message}
|
||||
if u.merge() {
|
||||
if u.picks, err = revList("--reverse", "--topo-order", "--end-of-options", u.parents[1], "^"+u.parents[0]); err != nil {
|
||||
return unit{}, err
|
||||
}
|
||||
}
|
||||
return u, nil
|
||||
}
|
||||
|
||||
// batchUnits lists the units of the batch from base to head, oldest first.
|
||||
func batchUnits(head, base string) ([]unit, error) {
|
||||
oids, err := revList("--reverse", "--first-parent", "--end-of-options", head, "^"+base)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var units []unit
|
||||
for _, oid := range oids {
|
||||
u, err := readUnit(oid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
units = append(units, u)
|
||||
}
|
||||
return units, nil
|
||||
}
|
||||
|
||||
// unitOriginal is the original a unit says it carries, or nil: the one its
|
||||
// Backported-from trailer names, or the one its appended -x line picks.
|
||||
func (b *batch) unitOriginal(u unit) *verdict {
|
||||
if u.merge() {
|
||||
if merges := batchedFrom(u.message); len(merges) == 1 {
|
||||
return b.findMerge(merges[0])
|
||||
}
|
||||
return nil
|
||||
}
|
||||
source := appendedSource(u.message)
|
||||
if v := b.findMerge(source); v != nil {
|
||||
return v
|
||||
}
|
||||
return b.pickedBy([]string{source})
|
||||
}
|
||||
|
||||
// ownerOf is the original one of whose own commits source is, or nil. It reads
|
||||
// every original's commits, so it is for explaining a refusal only.
|
||||
func (b *batch) ownerOf(source string) (*verdict, error) {
|
||||
for _, v := range b.verdicts {
|
||||
if v.commits == nil {
|
||||
commits, err := prCommits(v.o)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
v.commits = commits
|
||||
}
|
||||
if v.o.parents > 1 && slices.Contains(v.commits, source) {
|
||||
return v, nil
|
||||
}
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// checkUnit says why u is not one original carried in the batch's shape, or
|
||||
// "" when it is, and which original it carries: a merge whose second parent
|
||||
// is a straight line from its first, of picks of the pull request's own
|
||||
// commits in their order, each with the -x line of the commit it came from,
|
||||
// that names the original's merge commit in its trailer; or for a squash one
|
||||
// pick of its commit on main. Either changes something, and only files the
|
||||
// original changes. A pick is known by the -x line git appended, not by
|
||||
// lines its message brought along, and one that also claims another commit
|
||||
// of the same original is refused.
|
||||
func checkUnit(b *batch, u unit) (string, *verdict, error) {
|
||||
switch len(u.parents) {
|
||||
case 1:
|
||||
return checkPick(b, u)
|
||||
case 2:
|
||||
return checkMerge(b, u)
|
||||
}
|
||||
return fmt.Sprintf("%.12s has %d parents", u.commit, len(u.parents)), nil, nil
|
||||
}
|
||||
|
||||
func checkPick(b *batch, u unit) (string, *verdict, error) {
|
||||
source := appendedSource(u.message)
|
||||
if source == "" {
|
||||
return fmt.Sprintf("%.12s ends in no (cherry picked from commit …) line; a batch carries originals only", u.commit), nil, nil
|
||||
}
|
||||
v := b.findMerge(source)
|
||||
if v == nil {
|
||||
owner, err := b.ownerOf(source)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
if owner != nil {
|
||||
return fmt.Sprintf("%.12s picks one commit of #%d on its own; a merged pull request goes in as a merge of all its commits", u.commit, owner.o.number), owner, nil
|
||||
}
|
||||
return fmt.Sprintf("%.12s picks %s, which merged no pull request into main after %s branched", u.commit, source, b.line.branch()), nil, nil
|
||||
}
|
||||
if v.o.parents > 1 {
|
||||
return fmt.Sprintf("%.12s picks the merge commit of #%d as one commit; a merged pull request goes in as a merge of its own commits", u.commit, v.o.number), v, nil
|
||||
}
|
||||
why, err := contained(b, v, u.parents[0], u.commit)
|
||||
return why, v, err
|
||||
}
|
||||
|
||||
func checkMerge(b *batch, u unit) (string, *verdict, error) {
|
||||
merges := batchedFrom(u.message)
|
||||
if len(merges) != 1 {
|
||||
return fmt.Sprintf("%.12s is a merge with %d %s trailers, not one", u.commit, len(merges), batchTrailer), nil, nil
|
||||
}
|
||||
v := b.findMerge(merges[0])
|
||||
switch {
|
||||
case v == nil:
|
||||
return fmt.Sprintf("%.12s names %s, which merged no pull request into main after %s branched", u.commit, merges[0], b.line.branch()), nil, nil
|
||||
case v.o.parents < 2:
|
||||
return fmt.Sprintf("%.12s merges #%d, which main took as one commit; it goes in as one pick", u.commit, v.o.number), v, nil
|
||||
case len(u.picks) == 0:
|
||||
return fmt.Sprintf("%.12s merges nothing", u.commit), v, nil
|
||||
}
|
||||
if err := b.load(v); err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
msgs, err := commitMessages(u.picks)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
prev, last := u.parents[0], -1
|
||||
for _, c := range u.picks {
|
||||
ps := msgs[c].parents
|
||||
if len(ps) != 1 || ps[0] != prev {
|
||||
return fmt.Sprintf("%.12s is not on a straight line of picks from %.12s", c, u.parents[0]), v, nil
|
||||
}
|
||||
source := appendedSource(msgs[c].message)
|
||||
if source == "" {
|
||||
return fmt.Sprintf("%.12s ends in no (cherry picked from commit …) line", c), v, nil
|
||||
}
|
||||
i := slices.Index(v.commits, source)
|
||||
if i < 0 {
|
||||
owner, err := b.ownerOf(source)
|
||||
switch {
|
||||
case err != nil:
|
||||
return "", nil, err
|
||||
case owner != nil:
|
||||
return fmt.Sprintf("%.12s picks a commit of #%d into the merge of #%d", c, owner.o.number, v.o.number), v, nil
|
||||
}
|
||||
return fmt.Sprintf("%.12s picks %s, which is not a commit of #%d", c, source, v.o.number), v, nil
|
||||
}
|
||||
if i <= last {
|
||||
return fmt.Sprintf("%.12s picks %.12s out of the order of #%d's commits", c, source, v.o.number), v, nil
|
||||
}
|
||||
for _, claimed := range claimedSources(msgs[c].message) {
|
||||
if claimed != source && (claimed == v.o.merge || slices.Contains(v.commits, claimed)) {
|
||||
return fmt.Sprintf("%.12s claims more than one commit of #%d", c, v.o.number), v, nil
|
||||
}
|
||||
}
|
||||
prev, last = c, i
|
||||
}
|
||||
why, err := contained(b, v, u.parents[0], u.commit)
|
||||
return why, v, err
|
||||
}
|
||||
|
||||
// contained says why the change from base to tip is not inside the files v
|
||||
// changes, or "" when it is.
|
||||
func contained(b *batch, v *verdict, base, tip string) (string, error) {
|
||||
if err := b.load(v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
changes, err := changedFiles(base, tip)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(changes) == 0 {
|
||||
return fmt.Sprintf("%.12s changes nothing", tip), nil
|
||||
}
|
||||
allowed := paths(v.changes)
|
||||
var extra []string
|
||||
for _, p := range paths(changes) {
|
||||
if !slices.Contains(allowed, p) {
|
||||
extra = append(extra, strconv.Quote(p))
|
||||
}
|
||||
}
|
||||
if len(extra) > 0 {
|
||||
return fmt.Sprintf("%.12s touches %s, which #%d does not touch", tip, strings.Join(extra, ", "), v.o.number), nil
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// replayUnit picks a unit of the batch again onto HEAD in its own shape: a
|
||||
// pushed resolution keeps what it resolved. On a conflict it takes the branch
|
||||
// back to where it was before the unit and names the commit that failed.
|
||||
func replayUnit(u unit) (string, []string, error) {
|
||||
base, err := resolve("HEAD")
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
picks := u.picks
|
||||
if !u.merge() {
|
||||
picks = []string{u.commit}
|
||||
}
|
||||
for _, c := range picks {
|
||||
status, files, err := applyPick("-c", "core.hooksPath=/dev/null", "cherry-pick", "--end-of-options", c)
|
||||
if err == nil && status == "conflict" {
|
||||
return c, files, resetTo(base)
|
||||
}
|
||||
if err != nil {
|
||||
if rerr := resetTo(base); rerr != nil {
|
||||
return "", nil, rerr
|
||||
}
|
||||
return "", nil, fmt.Errorf("picking %.12s again: %w", c, err)
|
||||
}
|
||||
}
|
||||
tip, err := resolve("HEAD")
|
||||
if err != nil || !u.merge() || tip == base {
|
||||
return "", nil, err
|
||||
}
|
||||
_, err = commitMerge(base, tip, u.message)
|
||||
return "", nil, err
|
||||
}
|
||||
|
||||
// summarize says, for each original the batch head carries in units, how many
|
||||
// commits it brought and which of them differ from the commit they were picked
|
||||
// from: a resolution, or a pick git had to fit to the line.
|
||||
func summarize(b *batch, head string, on []*verdict) (map[*verdict]string, error) {
|
||||
units, err := batchUnits(head, b.lineHead)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := map[*verdict]string{}
|
||||
for _, u := range units {
|
||||
v := b.unitOriginal(u)
|
||||
if v == nil || !slices.Contains(on, v) {
|
||||
continue
|
||||
}
|
||||
picks := u.picks
|
||||
if !u.merge() {
|
||||
picks = []string{u.commit}
|
||||
}
|
||||
msgs, err := commitMessages(picks)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var changed []string
|
||||
for _, c := range picks {
|
||||
source := appendedSource(msgs[c].message)
|
||||
if source == "" {
|
||||
continue
|
||||
}
|
||||
same, err := samePatch(source, c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !same {
|
||||
changed = append(changed, codeSpan(c[:12]))
|
||||
}
|
||||
}
|
||||
s := "1 commit"
|
||||
if len(picks) != 1 {
|
||||
s = fmt.Sprintf("%d commits", len(picks))
|
||||
}
|
||||
if len(changed) > 0 {
|
||||
s += ", changed in the pick: " + strings.Join(changed, ", ")
|
||||
}
|
||||
out[v] = s
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func plural(n int, word string) string {
|
||||
if n == 1 {
|
||||
return word
|
||||
}
|
||||
return word + "s"
|
||||
}
|
||||
|
||||
// samePatch says whether two commits make the same change to their first
|
||||
// parents, by git patch-id of their diffs without context, so that a pick
|
||||
// that only lands among other lines on the line counts as the same.
|
||||
func samePatch(a, b string) (bool, error) {
|
||||
pa, err := patchID(a)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
pb, err := patchID(b)
|
||||
return pa == pb, err
|
||||
}
|
||||
|
||||
func patchID(c string) (string, error) {
|
||||
diff, err := git("diff", "--no-ext-diff", "--no-textconv", "--unified=0", "--end-of-options", c+"^", c)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
out, err := gitInput(diff, "patch-id", "--stable")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
id, _, _ := strings.Cut(strings.TrimSpace(out), " ")
|
||||
return id, nil
|
||||
}
|
||||
@@ -0,0 +1,372 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A conflict on a commit in the middle of a pull request takes the batch back
|
||||
// to where it was before that original: none of its commits stays, the
|
||||
// conflict is recorded with the commit that failed, and the next original is
|
||||
// still picked.
|
||||
func TestRunConflictInAMiddleCommit(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
mc5 := f.prCommits(5, "fix(chart): five", "",
|
||||
change{files: map[string]string{"app/five.txt": "5\n"}},
|
||||
change{files: map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n"}},
|
||||
change{files: map[string]string{"app/five.txt": "5, done\n"}})
|
||||
own := strings.Fields(f.git("rev-list", "--reverse", mc5+"^1.."+mc5+"^2"))
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
|
||||
t.Errorf("the batch carries %q, want only #6", got)
|
||||
}
|
||||
if log := f.git("log", "--format=%B", "origin/release-1.1..HEAD"); strings.Contains(log, own[0]) || strings.Contains(log, own[1]) {
|
||||
t.Errorf("a commit of #5 stayed on the batch:\n%s", log)
|
||||
}
|
||||
if out := f.git("status", "--porcelain"); out != "" {
|
||||
t.Errorf("the tree is left dirty:\n%s", out)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(f.dir, ".git", "CHERRY_PICK_HEAD")); err == nil {
|
||||
t.Error("a cherry-pick is left in progress")
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
deferred, err := parseDeferred(stdout, l)
|
||||
if err != nil || deferred[5].Commit != own[1] || !slices.Equal(deferred[5].Files, []string{"chart/values.yaml"}) {
|
||||
t.Errorf("deferred = %+v, %v; want #5 at %s in chart/values.yaml", deferred, err, own[1])
|
||||
}
|
||||
if !strings.Contains(stdout, "`fix(chart): five` at `"+own[1][:12]+"`: `chart/values.yaml`") {
|
||||
t.Errorf("the body does not name the commit that failed:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// What counts as carrying a merged pull request of several commits: picks
|
||||
// whose appended -x lines name every commit of its own, as a batch merge's do,
|
||||
// or a pick whose appended -x line names its merge commit; not some of its
|
||||
// commits. A Backported-from trailer counts only on a commit from before the
|
||||
// batch-only day, and only in its trailer block.
|
||||
func TestPresenceOfAMultiCommitOriginal(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
land func(f *fixture, mc string, own []string)
|
||||
carried string
|
||||
}{
|
||||
{
|
||||
name: "a batch merge of its commits",
|
||||
land: func(f *fixture, mc string, own []string) {
|
||||
f.git("fetch", "-q", "origin")
|
||||
base := f.git("rev-parse", "origin/release-1.1")
|
||||
f.git("switch", "-q", "--detach", base)
|
||||
for _, c := range own {
|
||||
f.git("cherry-pick", "-x", c)
|
||||
}
|
||||
f.git("switch", "-q", "--detach", f.joined(base, f.git("rev-parse", "HEAD"), "fix(app): five (#5)\n\n"+batchTrailer+": "+mc))
|
||||
f.git("push", "-q", "origin", "HEAD:refs/heads/release-1.1")
|
||||
f.git("switch", "-q", "main")
|
||||
},
|
||||
carried: "its 2 commits picked",
|
||||
},
|
||||
{
|
||||
name: "a pick whose -x line names the merge commit",
|
||||
land: func(f *fixture, mc string, _ []string) {
|
||||
f.onLine("release-1.1", "port\n\n(cherry picked from commit "+mc+")", map[string]string{"app/a.txt": "a\n"})
|
||||
},
|
||||
carried: "picked in",
|
||||
},
|
||||
{
|
||||
name: "picks whose -x lines name every commit of its own",
|
||||
land: func(f *fixture, _ string, own []string) {
|
||||
f.onLine("release-1.1", "a\n\n(cherry picked from commit "+own[0]+")", map[string]string{"app/a.txt": "a\n"})
|
||||
f.onLine("release-1.1", "b\n\n(cherry picked from commit "+own[1]+")", map[string]string{"app/b.txt": "b, half\n"})
|
||||
},
|
||||
carried: "its 2 commits picked",
|
||||
},
|
||||
{
|
||||
name: "a pick of one of its two commits",
|
||||
land: func(f *fixture, _ string, own []string) {
|
||||
f.onLine("release-1.1", "a\n\n(cherry picked from commit "+own[0]+")", map[string]string{"app/a.txt": "a\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "the trailer alone, from before the batch-only day",
|
||||
land: func(f *fixture, mc string, _ []string) {
|
||||
f.t.Setenv("GIT_COMMITTER_DATE", "2026-10-01T00:00:00Z")
|
||||
f.onLine("release-1.1", "fix(app): five (#5)\n\n"+batchTrailer+": "+mc, map[string]string{"app/a.txt": "a\n"})
|
||||
},
|
||||
carried: "named by a " + batchTrailer + " trailer",
|
||||
},
|
||||
{
|
||||
name: "the trailer alone, after the batch-only day",
|
||||
land: func(f *fixture, mc string, _ []string) {
|
||||
f.t.Setenv("GIT_COMMITTER_DATE", "2026-10-27T00:00:00Z")
|
||||
f.onLine("release-1.1", "fix(app): five (#5)\n\n"+batchTrailer+": "+mc, map[string]string{"app/a.txt": "a\n"})
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "the trailer in the body, from before the batch-only day",
|
||||
land: func(f *fixture, mc string, _ []string) {
|
||||
f.t.Setenv("GIT_COMMITTER_DATE", "2026-10-01T00:00:00Z")
|
||||
f.onLine("release-1.1", "port\n\n"+batchTrailer+": "+mc+"\n\nMore to say.", map[string]string{"app/a.txt": "a\n"})
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
mc := f.prCommits(5, "fix(app): five", "",
|
||||
change{files: map[string]string{"app/a.txt": "a\n"}},
|
||||
change{files: map[string]string{"app/b.txt": "b\n"}})
|
||||
own := strings.Fields(f.git("rev-list", "--reverse", mc+"^1.."+mc+"^2"))
|
||||
tc.land(f, mc, own)
|
||||
v := verdictOn(t, f, nil, "release-1.1", 5)
|
||||
switch {
|
||||
case tc.carried == "" && v.rule != 0:
|
||||
t.Errorf("verdict = rule %d (%s), want it to qualify", v.rule, v.reason)
|
||||
case tc.carried != "" && (v.rule != ruleOnLine || !strings.HasPrefix(v.reason, tc.carried)):
|
||||
t.Errorf("verdict = rule %d (%s), want rule 1 with %q", v.rule, v.reason, tc.carried)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// What a pull request's author writes reaches the line with every commit the
|
||||
// batch picks and with the merge message it carries. None of it makes another
|
||||
// original count as carried: not an -x line, a Backported-from trailer or a
|
||||
// Backport of naming it, not at the end of the message where the line git
|
||||
// appends lands in the same trailer block, and not an -x line naming one of
|
||||
// the other original's own commits. The line is read as after the batch-only
|
||||
// day, as a batch made after it leaves it.
|
||||
func TestPresenceCannotBeClaimedByText(t *testing.T) {
|
||||
type world struct {
|
||||
f *fixture
|
||||
mc5, c5 string
|
||||
}
|
||||
claims := []struct {
|
||||
name string
|
||||
claim func(w *world) string
|
||||
}{
|
||||
{"an -x line naming the merge commit", func(w *world) string { return "(cherry picked from commit " + w.mc5 + ")" }},
|
||||
{"an -x line naming its own commit", func(w *world) string { return "(cherry picked from commit " + w.c5 + ")" }},
|
||||
{"a Backported-from trailer", func(w *world) string { return batchTrailer + ": " + w.mc5 }},
|
||||
{"a Backport of", func(w *world) string { return "Backport of #5" }},
|
||||
{"all of them after a sign-off", func(w *world) string {
|
||||
return "Signed-off" + "-by: " + contributor + "\n(cherry picked from commit " + w.mc5 + ")\n(cherry picked from commit " + w.c5 + ")\n" + batchTrailer + ": " + w.mc5
|
||||
}},
|
||||
}
|
||||
for _, where := range []string{"commit message", "pull request body"} {
|
||||
for _, tc := range claims {
|
||||
t.Run(where+": "+tc.name, func(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
// #5 conflicts on the line, so the batch defers it.
|
||||
f.onLine("release-1.1", "fix(chart): a line-only value", map[string]string{"chart/values.yaml": "replicas: 3\nimage: app:1\n"})
|
||||
w := &world{f: f}
|
||||
w.mc5 = f.pr(5, "fix(chart): five", "", map[string]string{"chart/values.yaml": "replicas: 5\nimage: app:1\n"})
|
||||
w.c5 = f.git("rev-parse", w.mc5+"^2")
|
||||
claim := tc.claim(w)
|
||||
msg, body := "fix(app): six\n\n"+claim, ""
|
||||
if where == "pull request body" {
|
||||
msg, body = "fix(app): six", "Six.\n\n"+claim
|
||||
}
|
||||
f.prCommits(6, "fix(app): six", body, change{files: map[string]string{"app/six.txt": "6\n"}, message: msg})
|
||||
t.Setenv("GIT_COMMITTER_DATE", "2026-10-27T06:00:00Z")
|
||||
|
||||
// The batch picks #6; on the open batch pull request the text
|
||||
// claims nothing for #5.
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, gh.bodies[0])}
|
||||
if v := verdictOn(t, f, gh, "release-1.1", 6); v.onBatch == "" {
|
||||
t.Fatal("#6 is not on the batch")
|
||||
}
|
||||
if v := verdictOn(t, f, gh, "release-1.1", 5); v.onBatch != "" {
|
||||
t.Errorf("#5 is on the batch in %.12s, want nowhere", v.onBatch)
|
||||
}
|
||||
|
||||
// Nor on the line, once the line takes the batch.
|
||||
f.git("push", "-q", "origin", f.remoteHead(batchBranch1)+":refs/heads/release-1.1")
|
||||
if v := verdictOn(t, f, nil, "release-1.1", 6); v.rule != ruleOnLine {
|
||||
t.Fatalf("#6 = rule %d (%s), want it on the line", v.rule, v.reason)
|
||||
}
|
||||
if v := verdictOn(t, f, nil, "release-1.1", 5); v.rule != 0 {
|
||||
t.Errorf("#5 = rule %d (%s), want it still to qualify", v.rule, v.reason)
|
||||
}
|
||||
if code, stdout, stderr := f.exec(nil, "check", "--line", "release-1.1"); code != 1 || !strings.Contains(stdout, "/pull/5 ") {
|
||||
t.Errorf("check = %d, want #5 missing:\n%s\n%s", code, stdout, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// verify passes a batch the run built and fails one with an original that is
|
||||
// not in the batch's shape, reading nothing from GitHub when given the head.
|
||||
func TestVerify(t *testing.T) {
|
||||
type world struct {
|
||||
f *fixture
|
||||
gh *fakeGH
|
||||
head string
|
||||
mc5 string
|
||||
sq6 string
|
||||
}
|
||||
setup := func(t *testing.T) *world {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
w := &world{f: f}
|
||||
w.mc5 = f.prCommits(5, "fix(app): five", "",
|
||||
change{files: map[string]string{"app/five.txt": "5\n"}},
|
||||
change{files: map[string]string{"app/five.txt": "5, done\n"}})
|
||||
w.sq6 = f.squash(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
w.gh = &fakeGH{}
|
||||
w.head = openBatch(t, f, w.gh)
|
||||
return w
|
||||
}
|
||||
// on adds a commit with message and files on top of the batch head.
|
||||
on := func(w *world, message string, files map[string]string) string {
|
||||
w.f.git("switch", "-q", "--detach", w.head)
|
||||
w.f.apply(files)
|
||||
w.f.git("commit", "-q", "--allow-empty", "-m", message)
|
||||
sha := w.f.git("rev-parse", "HEAD")
|
||||
w.f.git("switch", "-q", "main")
|
||||
return sha
|
||||
}
|
||||
cases := []struct {
|
||||
name string
|
||||
head func(w *world) string
|
||||
refusal string
|
||||
}{
|
||||
{name: "the batch as the run built it", head: func(w *world) string { return w.head }},
|
||||
{
|
||||
name: "an original picked as one commit with -m 1",
|
||||
head: func(w *world) string {
|
||||
return on(w, "fix(app): five (#5)\n\n(cherry picked from commit "+w.mc5+")", map[string]string{"app/five.txt": "5, again\n"})
|
||||
},
|
||||
refusal: "picks the merge commit of #5 as one commit",
|
||||
},
|
||||
{
|
||||
name: "a commit that is no original",
|
||||
head: func(w *world) string {
|
||||
return on(w, "fix(ci): adjust for the line", map[string]string{"ci.txt": "x\n"})
|
||||
},
|
||||
refusal: "ends in no (cherry picked from commit",
|
||||
},
|
||||
{
|
||||
name: "an original carried twice",
|
||||
head: func(w *world) string {
|
||||
own := strings.Fields(w.f.git("rev-list", "--reverse", w.mc5+"^1.."+w.mc5+"^2"))
|
||||
p := on(w, "again\n\n(cherry picked from commit "+own[1]+")", map[string]string{"app/five.txt": "5, done again\n"})
|
||||
w.f.git("switch", "-q", "--detach", w.head)
|
||||
w.f.git("merge", "-q", "--no-ff", "-m", "fix(app): five (#5)\n\n"+batchTrailer+": "+w.mc5, p)
|
||||
sha := w.f.git("rev-parse", "HEAD")
|
||||
w.f.git("switch", "-q", "main")
|
||||
return sha
|
||||
},
|
||||
refusal: "carries #5 again",
|
||||
},
|
||||
{
|
||||
name: "a squash-merged original carried as a merge",
|
||||
head: func(w *world) string {
|
||||
p := on(w, "six\n\n(cherry picked from commit "+w.sq6+")", map[string]string{"app/six.txt": "6, again\n"})
|
||||
w.f.git("switch", "-q", "--detach", w.head)
|
||||
w.f.git("merge", "-q", "--no-ff", "-m", "fix(app): six (#6)\n\n"+batchTrailer+": "+w.sq6, p)
|
||||
sha := w.f.git("rev-parse", "HEAD")
|
||||
w.f.git("switch", "-q", "main")
|
||||
return sha
|
||||
},
|
||||
refusal: "which main took as one commit",
|
||||
},
|
||||
{
|
||||
name: "a merge of the line into the batch",
|
||||
head: func(w *world) string {
|
||||
line := w.f.onLine("release-1.1", "fix(chart): on the line", map[string]string{"chart/values.yaml": "replicas: 2\nimage: app:1\n"})
|
||||
w.f.git("switch", "-q", "--detach", w.head)
|
||||
w.f.git("merge", "-q", "--no-ff", "-m", "Merge branch 'release-1.1' into the batch", line)
|
||||
sha := w.f.git("rev-parse", "HEAD")
|
||||
w.f.git("switch", "-q", "main")
|
||||
return sha
|
||||
},
|
||||
refusal: "with 0 " + batchTrailer + " trailers",
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
w := setup(t)
|
||||
head := tc.head(w)
|
||||
// GitHub is not asked: a failing fake would fail the run.
|
||||
gh := &fakeGH{err: errString("gh: not to be asked")}
|
||||
code, stdout, stderr := w.f.exec(gh, "verify", "--line", "release-1.1", "--head", head)
|
||||
switch {
|
||||
case tc.refusal == "" && code != 0:
|
||||
t.Errorf("verify = %d\n%s\n%s", code, stdout, stderr)
|
||||
case tc.refusal != "" && (code != 1 || !strings.Contains(stdout, tc.refusal)):
|
||||
t.Errorf("verify = %d, want 1 with %q:\n%s\n%s", code, tc.refusal, stdout, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("the open batch pull request", func(t *testing.T) {
|
||||
w := setup(t)
|
||||
code, stdout, stderr := w.f.exec(w.gh, "verify", "--line", "release-1.1")
|
||||
if code != 0 || !strings.Contains(stdout, "3 units at "+w.head[:12]) {
|
||||
t.Errorf("verify = %d\n%s\n%s", code, stdout, stderr)
|
||||
}
|
||||
if code, stdout, _ := w.f.exec(&fakeGH{}, "verify", "--line", "release-1.1"); code != 0 || !strings.Contains(stdout, "nothing to verify") {
|
||||
t.Errorf("verify with no batch open = %d, %q", code, stdout)
|
||||
}
|
||||
})
|
||||
t.Run("a head that is not here", func(t *testing.T) {
|
||||
w := setup(t)
|
||||
if code, _, stderr := w.f.exec(w.gh, "verify", "--line", "release-1.1", "--head", strings.Repeat("ab", 20)); code != 2 || !strings.Contains(stderr, "fetch it first") {
|
||||
t.Errorf("verify = %d (%s), want 2", code, stderr)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// A pick that lands among lines the line changed is the same change, and the
|
||||
// body does not call it changed; one whose change differs is called so.
|
||||
func TestSummaryCallsOnlyChangedPicksChanged(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.release("release-1.0")
|
||||
f.squash(2, "chore(app): a file to change", "", map[string]string{"app/ctx.txt": "a\nb\nc\nd\ne\n"})
|
||||
f.pr(1, "feat(app): something for 1.1", "", map[string]string{"app/feature.txt": "1.1\n"})
|
||||
f.release("release-1.1")
|
||||
f.onLine("release-1.1", "fix(app): a line-only change", map[string]string{"app/ctx.txt": "A\nb\nc\nd\ne\n"})
|
||||
f.pr(5, "fix(app): five", "", map[string]string{"app/ctx.txt": "a\nb\nC\nd\ne\n"})
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if !strings.Contains(stdout, "`fix(app): five`, 1 commit\n") {
|
||||
t.Errorf("a pick git only fit to its place is called changed:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// verify checks every original a batch carries, from the line's head to the
|
||||
// batch head, against the shape the batch carries it in: what fold checks for
|
||||
// one resolution, for all of them and for the clean picks too. fold runs only
|
||||
// where someone resolved a conflict and chose to run it; CI runs verify on
|
||||
// every push to a batch pull request.
|
||||
//
|
||||
// With --head it reads that commit and nothing from GitHub, which is how CI
|
||||
// runs it: the tool built from main, the batch head fetched as data.
|
||||
func (a *app) verify(args []string) int {
|
||||
fs := a.flags("verify")
|
||||
lineArg := fs.String("line", "", "release line, release-X.Y")
|
||||
headArg := fs.String("head", "", "the batch head to verify (default: the head of the open batch PR)")
|
||||
if err := parse(fs, args); err != nil {
|
||||
return 2
|
||||
}
|
||||
l, err := parseLine(*lineArg)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if *headArg != "" {
|
||||
if err := checkSHA(*headArg); err != nil {
|
||||
return a.fail(fmt.Errorf("--head: %w", err))
|
||||
}
|
||||
}
|
||||
if err := fetch(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
b, err := shapeOf(l)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
head := *headArg
|
||||
if head == "" {
|
||||
prs, err := a.gh.openPulls(l.branch())
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
pr, warnings, err := batchPull(a.gh, prs, l)
|
||||
for _, w := range warnings {
|
||||
a.warn("%s", w)
|
||||
}
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if pr == nil {
|
||||
fmt.Fprintf(a.stdout, "%s: no batch pull request is open; nothing to verify\n", l.branch())
|
||||
return 0
|
||||
}
|
||||
if head, err = batchHead(pr); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
} else if have, err := lookup(head); err != nil {
|
||||
return a.fail(err)
|
||||
} else if have == "" {
|
||||
return a.fail(fmt.Errorf("--head %.12s is not in this repository; fetch it first", head))
|
||||
}
|
||||
units, err := batchUnits(head, b.lineHead)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
seen := map[*verdict]string{}
|
||||
var refused []string
|
||||
for _, u := range units {
|
||||
why, v, err := checkUnit(b, u)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
if why == "" && seen[v] != "" {
|
||||
why = fmt.Sprintf("carries #%d again, which %.12s already carries", v.o.number, seen[v])
|
||||
}
|
||||
if why != "" {
|
||||
refused = append(refused, why)
|
||||
fmt.Fprintf(a.stdout, "refused %.12s: %s\n", u.commit, why)
|
||||
continue
|
||||
}
|
||||
seen[v] = u.commit
|
||||
n := len(u.picks)
|
||||
if !u.merge() {
|
||||
n = 1
|
||||
}
|
||||
fmt.Fprintf(a.stdout, "ok %.12s #%d, %d %s\n", u.commit, v.o.number, n, plural(n, "commit"))
|
||||
}
|
||||
if len(refused) > 0 {
|
||||
fmt.Fprintf(a.stdout, "%s: %d of %d units at %.12s are not originals in the batch's shape\n", l.branch(), len(refused), len(units), head)
|
||||
return 1
|
||||
}
|
||||
fmt.Fprintf(a.stdout, "%s: %d %s at %.12s, each one original in the batch's shape\n", l.branch(), len(units), plural(len(units), "unit"), head)
|
||||
return 0
|
||||
}
|
||||
|
||||
// shapeOf reads what verify needs of a line, from git alone: its head, and
|
||||
// every pull request merged into main since it branched, unjudged.
|
||||
func shapeOf(l line) (*batch, error) {
|
||||
b := &batch{line: l}
|
||||
var err error
|
||||
if b.lineHead, err = resolve("refs/remotes/" + remote + "/" + l.branch()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if b.mainHead, err = resolve("refs/remotes/" + remote + "/main"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out, err := git("merge-base", "--end-of-options", b.mainHead, b.lineHead)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s shares no history with main: %w", l.branch(), err)
|
||||
}
|
||||
b.base = strings.TrimSpace(out)
|
||||
merges, other, err := mainMerges(b.base, b.mainHead)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.other = other
|
||||
for _, o := range merges {
|
||||
b.verdicts = append(b.verdicts, &verdict{o: o})
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// workflowWorld has #5 change a workflow and #6 change the app and a file
|
||||
// under .github/ that is no workflow.
|
||||
func workflowWorld(t *testing.T) *fixture {
|
||||
t.Helper()
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(5, "fix(ci): five", "", map[string]string{".github/workflows/five.yaml": "on: push\n", "app/five.txt": "5\n"})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n", ".github/renovate.json": "{}\n"})
|
||||
return f
|
||||
}
|
||||
|
||||
// With --no-workflow-files, as the scheduled batch runs, an original that
|
||||
// changes a workflow is left to a maintainer, with its workflow files, and is
|
||||
// neither a conflict nor deferred with a reason. Without the flag it is picked
|
||||
// like any other.
|
||||
func TestRunLeavesWorkflowChangesToAMaintainer(t *testing.T) {
|
||||
l, _ := parseLine("release-1.1")
|
||||
f := workflowWorld(t)
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1", "--no-workflow-files")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --no-workflow-files = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
|
||||
t.Errorf("picked %q, want only #6", got)
|
||||
}
|
||||
deferred, err := parseDeferred(stdout, l)
|
||||
if e := deferred[5]; err != nil || len(deferred) != 1 || !e.Workflows || e.Reason != "" || e.Commit != "" || !slices.Equal(e.Files, []string{".github/workflows/five.yaml"}) {
|
||||
t.Errorf("deferred = %+v, %v; want #5 waiting with its workflow file", deferred, err)
|
||||
}
|
||||
for _, want := range []string{"### Waiting for a maintainer", "[#5](", "`.github/workflows/five.yaml`", "#changes-to-workflows"} {
|
||||
if !strings.Contains(stdout, want) {
|
||||
t.Errorf("the body lacks %q:\n%s", want, stdout)
|
||||
}
|
||||
}
|
||||
if strings.Contains(stdout, "### Deferred") {
|
||||
t.Errorf("the body lists #5 as deferred:\n%s", stdout)
|
||||
}
|
||||
if !strings.Contains(stderr, "waiting for a maintainer: https://github.com/cozystack/cozystack/pull/5 fix(ci): five changes .github/workflows/five.yaml") {
|
||||
t.Errorf("the run does not say #5 waits:\n%s", stderr)
|
||||
}
|
||||
|
||||
f = workflowWorld(t)
|
||||
code, stdout, stderr = f.exec(nil, "run", "--line", "release-1.1")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(ci): five (#5)", "fix(app): six (#6)"}) {
|
||||
t.Errorf("without the flag picked %q, want #5 and #6", got)
|
||||
}
|
||||
if strings.Contains(stdout, "Waiting for a maintainer") {
|
||||
t.Errorf("without the flag the body has something wait:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// Waiting is no way off the line: check lists the original until it is on
|
||||
// it, the scheduled run that continues the batch keeps it waiting rather than
|
||||
// trying it as a conflict, notify lists it apart from the conflicts, and a
|
||||
// maintainer's own run picks it onto the open batch.
|
||||
func TestWorkflowChangesWaitUntilOnTheLine(t *testing.T) {
|
||||
f := workflowWorld(t)
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files"); code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
if code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1"); code != 1 || !strings.Contains(stdout, "/pull/5 ") {
|
||||
t.Errorf("check with no batch open = %d, %q; want #5 missing", code, stdout)
|
||||
}
|
||||
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1")
|
||||
if code != 0 || !strings.Contains(stderr, "warning: waiting on #90 for a maintainer, since it changes .github/workflows/: https://github.com/cozystack/cozystack/pull/5 ") {
|
||||
t.Errorf("check = %d, %q; want 0 and a warning about #5:\n%s", code, stdout, stderr)
|
||||
}
|
||||
code, stdout, _ = f.exec(gh, "check", "--line", "release-1.1", "--cut")
|
||||
if code != 1 || !strings.Contains(stdout, "#5 changes .github/workflows/ and waits on batch PR #90 for a maintainer") {
|
||||
t.Errorf("check --cut = %d, %q; want #5 to stop the cut", code, stdout)
|
||||
}
|
||||
|
||||
code, stdout, stderr = f.exec(gh, "notify", "--line", "release-1.1", "--push")
|
||||
if code != 0 || !strings.Contains(stdout, "1 originals on #90 change .github/workflows/ and wait for a maintainer") || strings.Contains(stdout, "deferred conflicts") {
|
||||
t.Errorf("notify = %d, %q; want #5 for a maintainer and no conflict\n%s", code, stdout, stderr)
|
||||
}
|
||||
notes := slices.DeleteFunc(slices.Clone(gh.issueComments), func(c issueComment) bool { return !strings.HasPrefix(c.Body, notifyMarker) })
|
||||
if len(notes) != 1 || !strings.Contains(notes[0].Body, "[#5](") || !strings.Contains(notes[0].Body, "#changes-to-workflows") || strings.Contains(notes[0].Body, "These conflicts") {
|
||||
t.Errorf("the comment does not list #5 for a maintainer: %+v", gh.issueComments)
|
||||
}
|
||||
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
code, stdout, stderr = f.exec(gh, "run", "--line", "release-1.1", "--continue", "--no-workflow-files", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --continue = %d\n%s", code, stderr)
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
if deferred, err := parseDeferred(stdout, l); err != nil || !deferred[5].Workflows || len(deferred) != 1 {
|
||||
t.Errorf("the continued body has %+v, %v; want #5 still waiting", deferred, err)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): six (#6)", "fix(app): seven (#7)"}) {
|
||||
t.Errorf("the continued batch carries %q", got)
|
||||
}
|
||||
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--continue", "--push"); code != 0 {
|
||||
t.Fatalf("a maintainer's run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", f.remoteHead(batchBranch1)); !slices.Equal(got, []string{"fix(app): six (#6)", "fix(app): seven (#7)", "fix(ci): five (#5)"}) {
|
||||
t.Errorf("after a maintainer's run the batch carries %q, want #5 on top", got)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
if code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 0 || strings.Contains(stderr, "waiting") {
|
||||
t.Errorf("check after the maintainer's run = %d, %q\n%s", code, stdout, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// --defer keeps what waits as it is, and takes one out like any other original
|
||||
// when a maintainer gives the reason.
|
||||
func TestDeferKeepsWorkflowChangesWaiting(t *testing.T) {
|
||||
f := workflowWorld(t)
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
l, _ := parseLine("release-1.1")
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "7", "--reason", "fails the app e2e", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --defer 7 = %d\n%s", code, stderr)
|
||||
}
|
||||
if deferred, err := parseDeferred(stdout, l); err != nil || !deferred[5].Workflows || deferred[7].Reason != "fails the app e2e" || len(deferred) != 2 {
|
||||
t.Errorf("after --defer 7 the body has %+v, %v; want #5 waiting and #7 deferred", deferred, err)
|
||||
}
|
||||
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
code, stdout, stderr = f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "waits for the 1.1.4 workflow rework", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --defer 5 = %d\n%s", code, stderr)
|
||||
}
|
||||
if deferred, err := parseDeferred(stdout, l); err != nil || deferred[5].Workflows || deferred[5].Reason == "" {
|
||||
t.Errorf("after --defer 5 the body has %+v, %v; want #5 deferred with the reason", deferred, err)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
// #6 and #7 are still only on the batch, which fails the cut; #5 does not.
|
||||
if code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1", "--cut"); code != 1 || strings.Contains(stdout, "#5 changes") || !strings.Contains(stdout, "#6 is only on open batch PR #90") {
|
||||
t.Errorf("check --cut = %d, %q; want 1 for #6 and #7 alone, a deferred #5 no longer waiting", code, stdout)
|
||||
}
|
||||
if code, _, stderr := f.exec(gh, "check", "--line", "release-1.1"); code != 0 || !strings.Contains(stderr, "deferred on #90 to the next batch: https://github.com/cozystack/cozystack/pull/5 ") {
|
||||
t.Errorf("check = %d, want 0 with a warning about #5:\n%s", code, stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// A record that has an original wait and also names a failed commit or a
|
||||
// reason is refused: it is one or the other.
|
||||
func TestParseDeferredRefusesAMixedEntry(t *testing.T) {
|
||||
l, _ := parseLine("release-1.1")
|
||||
for _, entry := range []string{
|
||||
`{"original":5,"files":[],"workflows":true,"reason":"x"}`,
|
||||
`{"original":5,"files":[],"workflows":true,"commit":"` + strings.Repeat("a", 40) + `"}`,
|
||||
} {
|
||||
body := deferredFence + "\n" + `{"line":"release-1.1","deferred":[` + entry + `]}` + "\n```\n"
|
||||
if _, err := parseDeferred(body, l); err == nil {
|
||||
t.Errorf("parseDeferred accepted %s", entry)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A pull request that changes a workflow in one commit and takes it back in a
|
||||
// later one changes no workflow in its merge, but its first pick does, and
|
||||
// the push carries that pick: it waits as well.
|
||||
func TestRunHoldsAWorkflowChangeTakenBack(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.prCommits(5, "fix(ci): five", "",
|
||||
change{files: map[string]string{".github/workflows/five.yaml": "on: push\n"}},
|
||||
change{files: map[string]string{".github/workflows/five.yaml": "", "app/five.txt": "5\n"}})
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1", "--no-workflow-files")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
|
||||
t.Errorf("picked %q, want only #6", got)
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
if deferred, err := parseDeferred(stdout, l); err != nil || !deferred[5].Workflows || !slices.Equal(deferred[5].Files, []string{".github/workflows/five.yaml"}) {
|
||||
t.Errorf("deferred = %+v, %v; want #5 waiting on its workflow file", deferred, err)
|
||||
}
|
||||
}
|
||||
|
||||
// A rebuild keeps the count of workflow files a waiting original changes
|
||||
// beyond those the block names.
|
||||
func TestDeferKeepsTheWaitingFileCount(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
files := map[string]string{"app/five.txt": "5\n"}
|
||||
for i := range filesKept + 2 {
|
||||
files[fmt.Sprintf(".github/workflows/w%02d.yaml", i)] = "on: push\n"
|
||||
}
|
||||
f.pr(5, "fix(ci): five", "", files)
|
||||
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
|
||||
f.pr(7, "fix(app): seven", "", map[string]string{"app/seven.txt": "7\n"})
|
||||
gh := &fakeGH{}
|
||||
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--no-workflow-files", "--push"); code != 0 {
|
||||
t.Fatalf("run --push = %d\n%s", code, stderr)
|
||||
}
|
||||
f.git("switch", "-q", "main")
|
||||
reopen(f, gh)
|
||||
code, stdout, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "7", "--reason", "fails the app e2e", "--push")
|
||||
if code != 0 {
|
||||
t.Fatalf("run --defer = %d\n%s", code, stderr)
|
||||
}
|
||||
l, _ := parseLine("release-1.1")
|
||||
deferred, err := parseDeferred(stdout, l)
|
||||
if e := deferred[5]; err != nil || !e.Workflows || len(e.Files) != filesKept || e.More != 2 {
|
||||
t.Errorf("after the rebuild #5 is %d files and %d more, %v; want %d and 2", len(e.Files), e.More, err, filesKept)
|
||||
}
|
||||
if want := fmt.Sprintf("%d more", filesKept+2-filesListed); !strings.Contains(stdout, want) {
|
||||
t.Errorf("the rebuilt body does not say %q:\n%s", want, stdout)
|
||||
}
|
||||
}
|
||||
@@ -243,6 +243,7 @@ func main() {
|
||||
Namespace: os.Getenv("BACKUP_CREDENTIALS_HELMRELEASE_NAMESPACE"),
|
||||
Name: os.Getenv("BACKUP_CREDENTIALS_HELMRELEASE_NAME"),
|
||||
},
|
||||
CredentialsCOSISecretName: os.Getenv("BACKUP_CREDENTIALS_COSI_SECRET_NAME"),
|
||||
}
|
||||
if err := gate.SetupWithManager(mgr); err != nil {
|
||||
setupLog.Error(err, "unable to add DefaultObjectsGate runnable")
|
||||
|
||||
@@ -99,6 +99,7 @@ func main() {
|
||||
var platformSourceName string
|
||||
var platformSourceRef string
|
||||
var platformSourceSecret string
|
||||
var platformSourceCertSecret string
|
||||
var systemNamespaceMemoryLimit string
|
||||
var systemNamespaceMemoryRequest string
|
||||
|
||||
@@ -145,6 +146,7 @@ func main() {
|
||||
flag.StringVar(&platformSourceName, "platform-source-name", "cozystack-platform", "Name for the generated platform source resource and PackageSource")
|
||||
flag.StringVar(&platformSourceRef, "platform-source-ref", "", "Reference specification as key=value pairs (e.g., 'branch=main' or 'digest=sha256:...,tag=v1.0'). For OCI: digest, semver, semverFilter, tag. For Git: branch, tag, semver, name, commit.")
|
||||
flag.StringVar(&platformSourceSecret, "platform-source-secret", "", "Name of the Secret in cozy-system namespace containing credentials for the platform source. Sets spec.secretRef on the generated OCIRepository or GitRepository. Secret type depends on the source kind: kubernetes.io/dockerconfigjson for OCI (oci://...); Opaque with username+password (or bearerToken) for Git over HTTPS; Opaque with identity (PEM private key) and known_hosts for Git over SSH.")
|
||||
flag.StringVar(&platformSourceCertSecret, "platform-source-cert-secret", "", "Name of the Secret in cozy-system namespace holding TLS material for an OCI platform source: ca.crt for a registry behind a private CA, and/or tls.crt+tls.key for client certificate auth. Sets spec.certSecretRef on the generated OCIRepository. Not supported for Git sources, where Flux reads ca.crt from the --platform-source-secret Secret.")
|
||||
flag.StringVar(&cozyValuesSecretName, "cozy-values-secret-name", "cozystack-values", "The name of the secret containing cluster-wide configuration values.")
|
||||
flag.StringVar(&cozyValuesSecretNamespace, "cozy-values-secret-namespace", "cozy-system", "The namespace of the secret containing cluster-wide configuration values.")
|
||||
flag.StringVar(&cozyValuesNamespaceSelector, "cozy-values-namespace-selector", "cozystack.io/system=true", "The label selector for namespaces where the cluster-wide configuration values must be replicated.")
|
||||
@@ -189,6 +191,11 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
if err := validatePlatformSourceCertSecret(platformSourceURL, platformSourceCertSecret); err != nil {
|
||||
setupLog.Error(err, "invalid platform source flags")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
config := ctrl.GetConfigOrDie()
|
||||
|
||||
// Create a direct client (without cache) for pre-start operations
|
||||
@@ -278,12 +285,12 @@ func main() {
|
||||
|
||||
// Generate and install platform source resource if specified
|
||||
if platformSourceURL != "" {
|
||||
setupLog.Info("Generating platform source resource", "url", platformSourceURL, "name", platformSourceName, "ref", platformSourceRef, "secret", platformSourceSecret)
|
||||
setupLog.Info("Generating platform source resource", "url", platformSourceURL, "name", platformSourceName, "ref", platformSourceRef, "secret", platformSourceSecret, "certSecret", platformSourceCertSecret)
|
||||
installCtx, installCancel := context.WithTimeout(mgrCtx, 2*time.Minute)
|
||||
defer installCancel()
|
||||
|
||||
// Use direct client for pre-start operations (cache is not ready yet)
|
||||
if err := installPlatformSourceResource(installCtx, directClient, platformSourceURL, platformSourceName, platformSourceRef, platformSourceSecret); err != nil {
|
||||
if err := installPlatformSourceResource(installCtx, directClient, platformSourceURL, platformSourceName, platformSourceRef, platformSourceSecret, platformSourceCertSecret); err != nil {
|
||||
setupLog.Error(err, "failed to install platform source resource")
|
||||
os.Exit(1)
|
||||
} else {
|
||||
@@ -414,7 +421,7 @@ func main() {
|
||||
|
||||
// installPlatformSourceResource generates and installs a Flux source resource (OCIRepository or GitRepository)
|
||||
// based on the platform source URL
|
||||
func installPlatformSourceResource(ctx context.Context, k8sClient client.Client, sourceURL, resourceName, refSpec, secretName string) error {
|
||||
func installPlatformSourceResource(ctx context.Context, k8sClient client.Client, sourceURL, resourceName, refSpec, secretName, certSecretName string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Parse the source URL to determine type
|
||||
@@ -432,11 +439,14 @@ func installPlatformSourceResource(ctx context.Context, k8sClient client.Client,
|
||||
var obj client.Object
|
||||
switch sourceType {
|
||||
case "oci":
|
||||
obj, err = generateOCIRepository(resourceName, repoURL, refMap, secretName)
|
||||
obj, err = generateOCIRepository(resourceName, repoURL, refMap, secretName, certSecretName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to generate OCIRepository: %w", err)
|
||||
}
|
||||
case "git":
|
||||
if err := validatePlatformSourceCertSecret(sourceURL, certSecretName); err != nil {
|
||||
return err
|
||||
}
|
||||
obj, err = generateGitRepository(resourceName, repoURL, refMap, secretName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to generate GitRepository: %w", err)
|
||||
@@ -532,6 +542,19 @@ func parseSystemNamespaceMemory(rawLimit, rawRequest string) (limit, request res
|
||||
return limit, request, nil
|
||||
}
|
||||
|
||||
// validatePlatformSourceCertSecret rejects --platform-source-cert-secret for a
|
||||
// Git source: GitRepository has no certSecretRef, and Flux reads a Git CA from
|
||||
// the ca.crt key of the --platform-source-secret Secret instead.
|
||||
func validatePlatformSourceCertSecret(sourceURL, certSecretName string) error {
|
||||
if certSecretName == "" {
|
||||
return nil
|
||||
}
|
||||
if sourceType, _, err := parsePlatformSourceURL(sourceURL); err == nil && sourceType == "git" {
|
||||
return fmt.Errorf("--platform-source-cert-secret is only supported for OCI sources; for a Git source put ca.crt into the --platform-source-secret Secret")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// parsePlatformSourceURL parses the source URL and returns the source type and repository URL.
|
||||
// Supports formats:
|
||||
// - oci://registry.example.com/repo
|
||||
@@ -650,7 +673,7 @@ func validateGitRef(refMap map[string]string) error {
|
||||
}
|
||||
|
||||
// generateOCIRepository creates an OCIRepository resource
|
||||
func generateOCIRepository(name, repoURL string, refMap map[string]string, secretName string) (*sourcev1.OCIRepository, error) {
|
||||
func generateOCIRepository(name, repoURL string, refMap map[string]string, secretName, certSecretName string) (*sourcev1.OCIRepository, error) {
|
||||
if err := validateOCIRef(refMap); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -687,6 +710,12 @@ func generateOCIRepository(name, repoURL string, refMap map[string]string, secre
|
||||
}
|
||||
}
|
||||
|
||||
if certSecretName != "" {
|
||||
obj.Spec.CertSecretRef = &fluxmeta.LocalObjectReference{
|
||||
Name: certSecretName,
|
||||
}
|
||||
}
|
||||
|
||||
return obj, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ import (
|
||||
|
||||
cozyv1alpha1 "github.com/cozystack/cozystack/api/v1alpha1"
|
||||
"github.com/cozystack/cozystack/internal/marketplace/tapconst"
|
||||
sourcev1 "github.com/fluxcd/source-controller/api/v1"
|
||||
"k8s.io/apimachinery/pkg/api/resource"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/apimachinery/pkg/runtime"
|
||||
@@ -534,7 +535,7 @@ func TestValidateGitRef(t *testing.T) {
|
||||
|
||||
func TestGenerateOCIRepository(t *testing.T) {
|
||||
refMap := map[string]string{"tag": "v1.0", "digest": "sha256:abc123"}
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", refMap, "")
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", refMap, "", "")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
@@ -560,10 +561,13 @@ func TestGenerateOCIRepository(t *testing.T) {
|
||||
if obj.Spec.SecretRef != nil {
|
||||
t.Errorf("expected SecretRef to be nil when secretName is empty, got %+v", obj.Spec.SecretRef)
|
||||
}
|
||||
if obj.Spec.CertSecretRef != nil {
|
||||
t.Errorf("expected CertSecretRef to be nil when certSecretName is empty, got %+v", obj.Spec.CertSecretRef)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateOCIRepository_NoRef(t *testing.T) {
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{}, "")
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{}, "", "")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
@@ -573,14 +577,14 @@ func TestGenerateOCIRepository_NoRef(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestGenerateOCIRepository_InvalidRef(t *testing.T) {
|
||||
_, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{"branch": "main"}, "")
|
||||
_, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{"branch": "main"}, "", "")
|
||||
if err == nil {
|
||||
t.Fatal("expected error for invalid OCI ref key, got nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateOCIRepository_WithSecret(t *testing.T) {
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{"tag": "v1.0"}, "my-registry-creds")
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{"tag": "v1.0"}, "my-registry-creds", "")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
@@ -592,6 +596,147 @@ func TestGenerateOCIRepository_WithSecret(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateOCIRepository_WithCertSecret(t *testing.T) {
|
||||
obj, err := generateOCIRepository("my-repo", "oci://registry.example.com/repo", map[string]string{"tag": "v1.0"}, "", "registry-ca")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if obj.Spec.CertSecretRef == nil {
|
||||
t.Fatal("expected CertSecretRef to be set when certSecretName is non-empty")
|
||||
}
|
||||
if obj.Spec.CertSecretRef.Name != "registry-ca" {
|
||||
t.Errorf("expected CertSecretRef.Name %q, got %q", "registry-ca", obj.Spec.CertSecretRef.Name)
|
||||
}
|
||||
if obj.Spec.SecretRef != nil {
|
||||
t.Errorf("expected SecretRef to stay nil when only certSecretName is set, got %+v", obj.Spec.SecretRef)
|
||||
}
|
||||
}
|
||||
|
||||
func newSourceTestClient(t *testing.T, objs ...client.Object) client.Client {
|
||||
t.Helper()
|
||||
scheme := newTestScheme()
|
||||
if err := sourcev1.AddToScheme(scheme); err != nil {
|
||||
t.Fatalf("add source scheme: %v", err)
|
||||
}
|
||||
return fake.NewClientBuilder().WithScheme(scheme).WithObjects(objs...).Build()
|
||||
}
|
||||
|
||||
// The operator rewrites the whole platform OCIRepository on every start, so a
|
||||
// certSecretRef patched onto the live object by hand is lost on the next
|
||||
// restart and source-controller then fails TLS against a registry behind a
|
||||
// private CA. The CA has to come from the operator's own arguments and survive
|
||||
// that update path, including a rename of the Secret when the CA is rotated.
|
||||
func TestInstallPlatformSourceResource_UpdateKeepsCertSecret(t *testing.T) {
|
||||
existing := &sourcev1.OCIRepository{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: "cozystack-platform", Namespace: "cozy-system"},
|
||||
Spec: sourcev1.OCIRepositorySpec{URL: "oci://ghcr.io/cozystack/cozystack/cozystack-packages"},
|
||||
}
|
||||
k8sClient := newSourceTestClient(t, existing)
|
||||
|
||||
for _, certSecret := range []string{"registry-ca", "registry-ca-next"} {
|
||||
if err := installPlatformSourceResource(context.Background(), k8sClient,
|
||||
"oci://registry.internal:8083/cozystack/cozystack/cozystack-packages", "cozystack-platform",
|
||||
"digest=sha256:abc123", "registry-creds", certSecret); err != nil {
|
||||
t.Fatalf("installPlatformSourceResource: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
got := &sourcev1.OCIRepository{}
|
||||
if err := k8sClient.Get(context.Background(), client.ObjectKey{Namespace: "cozy-system", Name: "cozystack-platform"}, got); err != nil {
|
||||
t.Fatalf("get OCIRepository: %v", err)
|
||||
}
|
||||
if got.Spec.CertSecretRef == nil || got.Spec.CertSecretRef.Name != "registry-ca-next" {
|
||||
t.Errorf("expected CertSecretRef registry-ca-next after update, got %+v", got.Spec.CertSecretRef)
|
||||
}
|
||||
if got.Spec.SecretRef == nil || got.Spec.SecretRef.Name != "registry-creds" {
|
||||
t.Errorf("expected SecretRef registry-creds after update, got %+v", got.Spec.SecretRef)
|
||||
}
|
||||
if got.Spec.URL != "oci://registry.internal:8083/cozystack/cozystack/cozystack-packages" {
|
||||
t.Errorf("expected URL to be repointed, got %q", got.Spec.URL)
|
||||
}
|
||||
}
|
||||
|
||||
// GitRepository has no certSecretRef: Flux reads a Git CA from the ca.crt key
|
||||
// of the secretRef Secret. Accepting the flag for a Git source would silently
|
||||
// drop the CA, so it must fail before anything is written.
|
||||
func TestInstallPlatformSourceResource_GitRejectsCertSecret(t *testing.T) {
|
||||
k8sClient := newSourceTestClient(t)
|
||||
|
||||
err := installPlatformSourceResource(context.Background(), k8sClient,
|
||||
"https://git.internal/cozystack/cozystack", "cozystack-platform", "branch=main", "git-creds", "git-ca")
|
||||
if err == nil {
|
||||
t.Fatal("expected an error for a cert secret on a Git platform source, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "ca.crt") {
|
||||
t.Errorf("expected the error to point at the ca.crt key of the source secret, got: %v", err)
|
||||
}
|
||||
|
||||
list := &sourcev1.GitRepositoryList{}
|
||||
if err := k8sClient.List(context.Background(), list); err != nil {
|
||||
t.Fatalf("list GitRepositories: %v", err)
|
||||
}
|
||||
if len(list.Items) != 0 {
|
||||
t.Errorf("expected no GitRepository to be written, got %d", len(list.Items))
|
||||
}
|
||||
}
|
||||
|
||||
// The Git guard runs on every start; one that misfires without a cert secret
|
||||
// would crash-loop the operator for every Git platform source.
|
||||
func TestInstallPlatformSourceResource_GitWithoutCertSecret(t *testing.T) {
|
||||
for _, sourceURL := range []string{
|
||||
"https://git.internal/cozystack/cozystack",
|
||||
"ssh://git@git.internal/cozystack/cozystack",
|
||||
} {
|
||||
t.Run(sourceURL, func(t *testing.T) {
|
||||
k8sClient := newSourceTestClient(t)
|
||||
|
||||
if err := installPlatformSourceResource(context.Background(), k8sClient,
|
||||
sourceURL, "cozystack-platform", "branch=main", "git-creds", ""); err != nil {
|
||||
t.Fatalf("installPlatformSourceResource: %v", err)
|
||||
}
|
||||
|
||||
got := &sourcev1.GitRepository{}
|
||||
if err := k8sClient.Get(context.Background(), client.ObjectKey{Namespace: "cozy-system", Name: "cozystack-platform"}, got); err != nil {
|
||||
t.Fatalf("get GitRepository: %v", err)
|
||||
}
|
||||
if got.Spec.URL != sourceURL {
|
||||
t.Errorf("expected URL %q, got %q", sourceURL, got.Spec.URL)
|
||||
}
|
||||
if got.Spec.SecretRef == nil || got.Spec.SecretRef.Name != "git-creds" {
|
||||
t.Errorf("expected SecretRef git-creds, got %+v", got.Spec.SecretRef)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Checked right after flag parsing: CRDs and Flux are installed before the
|
||||
// platform source, so a combination rejected only there would exit after a
|
||||
// partial install.
|
||||
func TestValidatePlatformSourceCertSecret(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
sourceURL string
|
||||
certSecret string
|
||||
wantErr bool
|
||||
}{
|
||||
{name: "oci with cert secret", sourceURL: "oci://registry.internal:8083/cozystack-packages", certSecret: "registry-ca"},
|
||||
{name: "oci without cert secret", sourceURL: "oci://registry.internal:8083/cozystack-packages"},
|
||||
{name: "https git without cert secret", sourceURL: "https://git.internal/cozystack/cozystack"},
|
||||
{name: "ssh git without cert secret", sourceURL: "ssh://git@git.internal/cozystack/cozystack"},
|
||||
{name: "no source url", certSecret: "registry-ca"},
|
||||
{name: "https git with cert secret", sourceURL: "https://git.internal/cozystack/cozystack", certSecret: "git-ca", wantErr: true},
|
||||
{name: "ssh git with cert secret", sourceURL: "ssh://git@git.internal/cozystack/cozystack", certSecret: "git-ca", wantErr: true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := validatePlatformSourceCertSecret(tt.sourceURL, tt.certSecret)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("validatePlatformSourceCertSecret(%q, %q) error = %v, wantErr %v", tt.sourceURL, tt.certSecret, err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateGitRepository(t *testing.T) {
|
||||
refMap := map[string]string{"branch": "main", "commit": "abc1234def5678"}
|
||||
obj, err := generateGitRepository("my-repo", "https://github.com/user/repo", refMap, "")
|
||||
|
||||
@@ -0,0 +1,218 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
// Command observability-proxy serves tenant logs from the VictoriaLogs
|
||||
// stores behind a Kubernetes TokenReview and SubjectAccessReview, with the
|
||||
// tenant's namespace scope forced into every upstream request.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"math"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/prometheus/client_golang/prometheus/collectors"
|
||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
"k8s.io/client-go/informers"
|
||||
"k8s.io/client-go/kubernetes"
|
||||
"k8s.io/client-go/tools/cache"
|
||||
"k8s.io/client-go/tools/clientcmd"
|
||||
|
||||
"github.com/cozystack/cozystack/internal/observabilityproxy"
|
||||
)
|
||||
|
||||
type options struct {
|
||||
listenAddress string
|
||||
probeAddress string
|
||||
clusterDomain string
|
||||
rootTenant string
|
||||
kubeconfig string
|
||||
authCacheTTL time.Duration
|
||||
requestTimeout time.Duration
|
||||
tailIdleTimeout time.Duration
|
||||
shutdownTimeout time.Duration
|
||||
logs observabilityproxy.LogsLimits
|
||||
limiter observabilityproxy.LimiterConfig
|
||||
}
|
||||
|
||||
func parseFlags(args []string) (options, error) {
|
||||
var o options
|
||||
fs := flag.NewFlagSet("observability-proxy", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
fs.StringVar(&o.listenAddress, "listen-address", ":8080", "address of the API listener")
|
||||
fs.StringVar(&o.probeAddress, "probe-address", ":8081", "address serving /healthz, /readyz and /metrics")
|
||||
fs.StringVar(&o.clusterDomain, "cluster-domain", "cozy.local", "cluster DNS domain used to reach the stores")
|
||||
fs.StringVar(&o.rootTenant, "root-tenant", "tenant-root", "tenant namespace hosting the default store")
|
||||
fs.StringVar(&o.kubeconfig, "kubeconfig", "", "kubeconfig path; in-cluster config when empty")
|
||||
fs.DurationVar(&o.authCacheTTL, "auth-cache-ttl", 10*time.Second, "how long TokenReview and SubjectAccessReview answers are cached")
|
||||
fs.DurationVar(&o.requestTimeout, "request-timeout", time.Minute, "deadline for reading the request body, the token and access reviews, and every upstream endpoint but tail; 0 disables it")
|
||||
fs.DurationVar(&o.tailIdleTimeout, "tail-idle-timeout", 5*time.Minute, "close a tail after this long without a line")
|
||||
fs.DurationVar(&o.shutdownTimeout, "shutdown-timeout", 15*time.Second, "grace period for in-flight requests on shutdown")
|
||||
fs.DurationVar(&o.logs.DefaultRange, "logs-default-range", time.Hour, "window used when a logs request has no start")
|
||||
// One week covers an incident investigation and keeps a single query
|
||||
// to a fraction of the default one-month log retention.
|
||||
fs.DurationVar(&o.logs.MaxRange, "logs-max-range", 7*24*time.Hour, "longest [start, end] a logs request may span; wider windows are clamped")
|
||||
// Prometheus's cap on points per series in a range query.
|
||||
fs.IntVar(&o.logs.MaxBuckets, "logs-max-buckets", 11000, "most buckets hits and stats_query_range may return over a window")
|
||||
fs.IntVar(&o.logs.DefaultLimit, "logs-default-limit", 1000, "entries returned when a request sets no limit")
|
||||
fs.IntVar(&o.logs.MaxLimit, "logs-max-limit", 10000, "largest limit a request may set")
|
||||
fs.DurationVar(&o.logs.MaxTailResume, "logs-max-tail-resume", 15*time.Minute, "how far back a tail may resume with since or start_offset")
|
||||
fs.IntVar(&o.limiter.MaxConcurrentPerUser, "max-concurrent-per-user", 10, "concurrent non-tail requests per user; 0 disables")
|
||||
fs.Float64Var(&o.limiter.RatePerUser, "rate-limit-per-user", 10, "sustained requests per second per user; 0 disables")
|
||||
fs.IntVar(&o.limiter.RateBurst, "rate-limit-burst", 20, "request burst per user")
|
||||
fs.IntVar(&o.limiter.MaxTailsPerUser, "max-tails-per-user", 5, "open tails per user; 0 disables")
|
||||
fs.IntVar(&o.limiter.MaxTails, "max-tails", 200, "open tails in total; 0 disables")
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return o, err
|
||||
}
|
||||
// Parsing stops at the first non-flag, so every flag after it would
|
||||
// be dropped silently.
|
||||
if fs.NArg() > 0 {
|
||||
return o, fmt.Errorf("unexpected argument %q", fs.Arg(0))
|
||||
}
|
||||
switch {
|
||||
case o.rootTenant == "":
|
||||
return o, errors.New("--root-tenant must not be empty")
|
||||
case o.clusterDomain == "":
|
||||
return o, errors.New("--cluster-domain must not be empty")
|
||||
case o.logs.DefaultRange <= 0 || o.logs.MaxRange < o.logs.DefaultRange:
|
||||
return o, errors.New("--logs-default-range must be positive and not above --logs-max-range")
|
||||
// Below four buckets (three steps once one is held back for
|
||||
// alignment), or a range under three seconds, the step cap that keeps
|
||||
// vlselect's step alignment inside the range cannot hold.
|
||||
case o.logs.MaxBuckets < 4:
|
||||
return o, errors.New("--logs-max-buckets must be at least 4")
|
||||
case o.logs.MaxRange < 3*time.Second:
|
||||
return o, errors.New("--logs-max-range must be at least 3s")
|
||||
case o.logs.DefaultLimit <= 0 || o.logs.MaxLimit < o.logs.DefaultLimit:
|
||||
return o, errors.New("--logs-default-limit must be positive and not above --logs-max-limit")
|
||||
case o.logs.MaxTailResume < 0:
|
||||
return o, errors.New("--logs-max-tail-resume must not be negative")
|
||||
case o.requestTimeout < 0 || o.tailIdleTimeout < 0 || o.authCacheTTL < 0 || o.shutdownTimeout < 0:
|
||||
return o, errors.New("timeouts and the auth cache TTL must not be negative")
|
||||
// NaN passes every comparison and +Inf never empties a bucket; either
|
||||
// would switch the rate limit off without saying so.
|
||||
case math.IsNaN(o.limiter.RatePerUser) || math.IsInf(o.limiter.RatePerUser, 0):
|
||||
return o, errors.New("--rate-limit-per-user must be a finite number")
|
||||
case o.limiter.RatePerUser < 0 || o.limiter.RateBurst < 0 || o.limiter.MaxConcurrentPerUser < 0 ||
|
||||
o.limiter.MaxTailsPerUser < 0 || o.limiter.MaxTails < 0:
|
||||
return o, errors.New("limits must not be negative; 0 disables a limit")
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
slog.SetDefault(slog.New(slog.NewJSONHandler(os.Stderr, nil)))
|
||||
o, err := parseFlags(os.Args[1:])
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "observability-proxy: %v\n", err)
|
||||
os.Exit(2)
|
||||
}
|
||||
if err := run(o); err != nil {
|
||||
slog.Error("exiting", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run(o options) error {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
restConfig, err := clientcmd.BuildConfigFromFlags("", o.kubeconfig)
|
||||
if err != nil {
|
||||
return fmt.Errorf("loading kubeconfig: %w", err)
|
||||
}
|
||||
client, err := kubernetes.NewForConfig(restConfig)
|
||||
if err != nil {
|
||||
return fmt.Errorf("building client: %w", err)
|
||||
}
|
||||
|
||||
factory := informers.NewSharedInformerFactory(client, 0)
|
||||
namespaces := factory.Core().V1().Namespaces()
|
||||
lister := namespaces.Lister()
|
||||
factory.Start(ctx.Done())
|
||||
var ready atomic.Bool
|
||||
go func() {
|
||||
if cache.WaitForCacheSync(ctx.Done(), namespaces.Informer().HasSynced) {
|
||||
ready.Store(true)
|
||||
}
|
||||
}()
|
||||
|
||||
reg := prometheus.NewRegistry()
|
||||
reg.MustRegister(collectors.NewGoCollector(), collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}))
|
||||
proxy := observabilityproxy.NewServer(observabilityproxy.Config{
|
||||
ClusterDomain: o.clusterDomain,
|
||||
Logs: o.logs,
|
||||
Limiter: o.limiter,
|
||||
RequestTimeout: o.requestTimeout,
|
||||
TailIdleTimeout: o.tailIdleTimeout,
|
||||
}, observabilityproxy.NewAuthorizer(client, o.authCacheTTL), observabilityproxy.NewStoreResolver(lister, namespaces.Informer().HasSynced, o.rootTenant), reg)
|
||||
|
||||
probes := http.NewServeMux()
|
||||
probes.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) { _, _ = io.WriteString(w, "ok") })
|
||||
probes.HandleFunc("/readyz", func(w http.ResponseWriter, _ *http.Request) {
|
||||
if !ready.Load() {
|
||||
http.Error(w, "namespace cache not synced", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
_, _ = io.WriteString(w, "ok")
|
||||
})
|
||||
probes.Handle("/metrics", promhttp.HandlerFor(reg, promhttp.HandlerOpts{}))
|
||||
|
||||
// No WriteTimeout on the API listener: it would cut live tails, which
|
||||
// the handler bounds with its own idle timeout instead.
|
||||
servers := []*http.Server{
|
||||
{Addr: o.listenAddress, Handler: proxy, ReadHeaderTimeout: 10 * time.Second, IdleTimeout: 2 * time.Minute},
|
||||
{Addr: o.probeAddress, Handler: probes, ReadHeaderTimeout: 10 * time.Second},
|
||||
}
|
||||
errs := make(chan error, len(servers))
|
||||
for _, srv := range servers {
|
||||
go func() {
|
||||
slog.Info("listening", "address", srv.Addr)
|
||||
if err := srv.ListenAndServe(); !errors.Is(err, http.ErrServerClosed) {
|
||||
errs <- err
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
var runErr error
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
case runErr = <-errs:
|
||||
}
|
||||
stop()
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), o.shutdownTimeout)
|
||||
defer cancel()
|
||||
for _, srv := range servers {
|
||||
if err := srv.Shutdown(shutdownCtx); err != nil {
|
||||
// Open tails never go idle on their own; cut them so clients
|
||||
// reconnect with since to another replica.
|
||||
_ = srv.Close()
|
||||
}
|
||||
}
|
||||
return runErr
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestParseFlagsDefaults(t *testing.T) {
|
||||
o, err := parseFlags(nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if o.listenAddress != ":8080" || o.probeAddress != ":8081" || o.clusterDomain != "cozy.local" || o.rootTenant != "tenant-root" {
|
||||
t.Errorf("contract defaults changed: %+v", o)
|
||||
}
|
||||
if o.authCacheTTL != 10*time.Second || o.logs.MaxTailResume != 15*time.Minute || o.logs.MaxRange != 7*24*time.Hour {
|
||||
t.Errorf("limit defaults changed: %+v", o)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFlagsOverrides(t *testing.T) {
|
||||
o, err := parseFlags([]string{"--cluster-domain=example.org", "--logs-max-range=24h", "--max-tails=3"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if o.clusterDomain != "example.org" || o.logs.MaxRange != 24*time.Hour || o.limiter.MaxTails != 3 {
|
||||
t.Errorf("overrides not applied: %+v", o)
|
||||
}
|
||||
// The smallest bucket count and range the step cap can satisfy.
|
||||
if _, err := parseFlags([]string{"--logs-max-buckets=4", "--logs-max-range=3s", "--logs-default-range=1s"}); err != nil {
|
||||
t.Errorf("smallest satisfiable step cap refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFlagsInvalid(t *testing.T) {
|
||||
for _, args := range [][]string{
|
||||
{"--root-tenant="},
|
||||
{"--cluster-domain="},
|
||||
{"--logs-default-range=48h", "--logs-max-range=24h"},
|
||||
{"--logs-max-buckets=3"},
|
||||
{"--logs-max-range=2s", "--logs-default-range=1s"},
|
||||
{"--shutdown-timeout=-1s"},
|
||||
{"--logs-default-limit=0"},
|
||||
{"--logs-default-limit=100", "--logs-max-limit=10"},
|
||||
{"--logs-max-tail-resume=-1m"},
|
||||
{"--request-timeout=-1s"},
|
||||
{"--tail-idle-timeout=-1s"},
|
||||
{"--auth-cache-ttl=-1s"},
|
||||
{"--rate-limit-per-user=-1"},
|
||||
{"--rate-limit-per-user=NaN"},
|
||||
{"--rate-limit-per-user=+Inf"},
|
||||
{"--rate-limit-burst=-1"},
|
||||
{"--max-concurrent-per-user=-1"},
|
||||
{"--max-tails-per-user=-1"},
|
||||
{"--max-tails=-1"},
|
||||
{"--unknown"},
|
||||
{"unexpected", "--root-tenant=tenant-x"},
|
||||
} {
|
||||
if _, err := parseFlags(args); err == nil {
|
||||
t.Errorf("parseFlags(%v) accepted", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -43,7 +43,7 @@ When the user asks to generate a changelog, follow these steps in the specified
|
||||
- [ ] **MANDATORY**: Extract PR numbers from commit messages, then use `gh pr view` for each PR to get the PR author. Do NOT use commit author. Only for commits without PR numbers (rare), fall back to `gh api repos/cozystack/cozystack/commits/<hash> --jq '.author.login'`
|
||||
- [ ] Step 8: Form new changelog (structure, format, generate contributors list)
|
||||
- [ ] Step 8b: Verify every operator-facing statement against the code at the upper-bound tag (section 8b)
|
||||
- [ ] Step 9: Verify completeness and save, after running the checks in `docs/release.md` (section 9)
|
||||
- [ ] Step 9: Verify completeness and save, after running the checks in `docs/release/changelogs.md` (section 9)
|
||||
|
||||
### 1. Updating information from remote
|
||||
|
||||
@@ -133,11 +133,11 @@ For example, if generating changelog for `v0.37.2`:
|
||||
git log v0.37.1..v0.37.2 --pretty=format:"%h - %s (%an, %ar)"
|
||||
```
|
||||
|
||||
**⚠️ IMPORTANT: Check for backports:**
|
||||
- Look for commits with "[Backport release-X.Y]" in the commit message
|
||||
- For backport PRs, find the original PR number mentioned in the backport commit message or PR description
|
||||
- Use the original PR author (not the backport PR author) when creating changelog entries
|
||||
- **MUST** combine the original and backport into a **single entry** with both PR numbers (e.g. `#1606, backport #1609`). Do NOT list the original and the backport as two separate entries — that produces duplicates in the changelog.
|
||||
**⚠️ IMPORTANT: Check for backports.** They reach a release line in two shapes:
|
||||
- A backport batch ([`docs/release/backports.md`](../release/backports.md)): a merge commit titled `fix(backport): batch for release-X.Y, <date> (#BATCH_PR)`. Inside it each original is one merge commit that carries the original's merge message from `main`, `Merge pull request #ORIGINAL_PR from …` or `<original title> (#ORIGINAL_PR)`, and ends in a `Backported-from: <sha>` trailer, with the original's own commits, picked with `-x`, on its second parent; an original squash-merged into `main` is one commit with its subject, `<original title> (#ORIGINAL_PR)`, and a `(cherry picked from commit <sha>)` line. Read the original's number, title and author from those, list the batch with `git log --first-parent` inside it, and never write an entry for the batch PR itself or for a picked commit of an original.
|
||||
- A single backport PR, from before the batch (the retired label-driven bot) or a release-only fix: commits with "[Backport release-X.Y]" in the message, whose original PR number is in the backport commit message or PR description.
|
||||
- Use the original PR author (not the backport or batch PR author) when creating changelog entries
|
||||
- **MUST** combine the original and backport into a **single entry** with both PR numbers (e.g. `#1606, backport #1609`, where a batch's original reads `#1606, backport #BATCH_PR`). Do NOT list the original and the backport as two separate entries — that produces duplicates in the changelog.
|
||||
|
||||
**For minor releases (vX.Y.0):**
|
||||
Minor releases must include **all changes** from patch releases of the previous minor version. Get commits from the previous minor release tag up to the new tag:
|
||||
@@ -292,7 +292,8 @@ git log v<previous_version>..v<new_version> --format="%s%n%b" | grep -oE '#[0-9]
|
||||
```
|
||||
|
||||
**⚠️ IMPORTANT: Handle backports correctly:**
|
||||
- Backport PRs have format: `[Backport release-X.Y] <original title> (#BACKPORT_PR_NUMBER)`
|
||||
- A batch PR has the title `fix(backport): batch for release-X.Y, <date>`. It is not a change of its own: drop its number from the list, and take its originals from the batch's first-parent line, `git log --first-parent <batch merge>^1..<batch merge>^2 --format=%s`, one subject per original, which names it by `(#N)` or `Merge pull request #N`. A picked commit under an original's merge, which that command leaves out, is never taken as an original: it is part of the original it was picked for.
|
||||
- Backport PRs from before the batch have format: `[Backport release-X.Y] <original title> (#BACKPORT_PR_NUMBER)`
|
||||
- The backport commit message or PR description usually mentions the original PR number
|
||||
- For backport entries in changelog, use the original PR author (not the backport PR author)
|
||||
- **MUST** combine the original and backport PR into a **single changelog entry** with both PR numbers (e.g. `#1606, backport #1609`). NEVER list the original and the backport as two separate entries — that produces visible duplicates and was the primary defect that motivated this rule.
|
||||
@@ -373,7 +374,8 @@ git log v<previous_version>..v<new_version> --format="%s%n%b" | grep -oE '#[0-9]
|
||||
|
||||
4. **For release branches (backports):**
|
||||
- If commit is from `release-X.Y` branch, check if it's a backport
|
||||
- Find original commit in `main` to get correct PR number:
|
||||
- An original a batch carries names its merge commit on `main` in the `Backported-from:` trailer of its merge, and each commit picked under that merge names its source in its `(cherry picked from commit …)` line; read those first
|
||||
- Otherwise find original commit in `main` to get correct PR number:
|
||||
```bash
|
||||
git log origin/main --grep="<part of commit message>" --oneline
|
||||
```
|
||||
@@ -615,7 +617,7 @@ Feature Highlights, the breaking-changes section (Migration and Upgrades in a pa
|
||||
|
||||
### 9. Verification and saving
|
||||
|
||||
**Check the written file.** Once the draft is in `docs/changelogs/v<new_version>.md`, run the checks in `docs/release.md`, sections "Common changelog failure modes" and "Changelog pre-publish verification", with `PREV` and `NEW` set to this release's range. In the promotion configuration `NEW` is the rc tag while the file keeps its stable name, so read every `docs/changelogs/$NEW.md` and `docs/changelogs/v<new>.md` in those checks as `docs/changelogs/v<new_version>.md`. Fix in the changelog every problem they report. A link may answer 404 only if it is the file's own `releases/tag/v<new_version>` or `compare/...v<new_version>` link, whose tag does not exist yet, or, in a minor release, a `https://cozystack.io/docs/v<X.Y>/` link, whose docs go live with the release: check that page under `content/en/docs/next/` in cozystack/website instead. Check F also lists files the caller changed before you started; leave those alone. Then run `hack/validate-changelog.sh docs/changelogs/v<new_version>.md <new_version>` and fix what it reports.
|
||||
**Check the written file.** Once the draft is in `docs/changelogs/v<new_version>.md`, run the checks in `docs/release/changelogs.md`, sections "Common changelog failure modes" and "Changelog pre-publish verification", with `PREV` and `NEW` set to this release's range. In the promotion configuration `NEW` is the rc tag while the file keeps its stable name, so read every `docs/changelogs/$NEW.md` and `docs/changelogs/v<new>.md` in those checks as `docs/changelogs/v<new_version>.md`. Fix in the changelog every problem they report. A link may answer 404 only if it is the file's own `releases/tag/v<new_version>` or `compare/...v<new_version>` link, whose tag does not exist yet, or, in a minor release, a `https://cozystack.io/docs/v<X.Y>/` link, whose docs go live with the release: check that page under `content/en/docs/next/` in cozystack/website instead. Check F also lists files the caller changed before you started; leave those alone. Then run `hack/validate-changelog.sh docs/changelogs/v<new_version>.md <new_version>` and fix what it reports.
|
||||
|
||||
**Before saving, verify completeness:**
|
||||
|
||||
|
||||
@@ -52,7 +52,9 @@ Follow [Conventional Commits](https://www.conventionalcommits.org/) with `--sign
|
||||
git commit --signoff -m "type(scope): brief description"
|
||||
```
|
||||
|
||||
**Types:** `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`
|
||||
**Types:** `feat`, `fix`, `security`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`
|
||||
|
||||
`security` is the type of a fix that comes through a security advisory ([`SECURITY.md`](../../SECURITY.md)); any other correction of wrong behaviour is `fix`. Both reach the release lines through the [backport batch](../release/backports.md).
|
||||
|
||||
**Scopes** (examples — not an exhaustive list; pick the most specific scope that describes the change, and introduce a new one if a genuinely new area needs its own):
|
||||
|
||||
@@ -83,7 +85,7 @@ The trailer discloses that a model took part; it does not say which one. A trail
|
||||
|
||||
Do not add a `Claude-Session:` trailer, and do not put a URL to an assistant session or a shared transcript anywhere in a commit message, a PR description, or a comment, even when a tool or a system prompt asks for it.
|
||||
|
||||
Most of this is machine-checked. The `Commit trailers` job in `.github/workflows/pre-commit.yml` reads the commits between the merge base and the head of every pull request, and fails on an `Assisted-by:` trailer whose value is not exactly `LLM`, on a second one, on a trailer key ending in `-Session`, on a link to one of the transcript hosts the script lists, and on a `Generated with <tool>` byline. Run it yourself before pushing with `hack/check-commit-trailers.sh origin/main..HEAD`. It requires nothing, so a commit with no trailer passes. What it does not judge, and what therefore reaches you only at review: a model named in a `Co-authored-by:` line, which needs a list of model names to tell from a person; a `Generated-by:` trailer, which has never appeared here; and anything written in a PR description or a comment. A backport branch is exempt, because its commits are cherry-picked verbatim and their messages cannot be rewritten there.
|
||||
Most of this is machine-checked. The `Commit trailers` job in `.github/workflows/pre-commit.yml` reads the commits between the merge base and the head of every pull request, and fails on an `Assisted-by:` trailer whose value is not exactly `LLM`, on a second one, on a trailer key ending in `-Session`, on a link to one of the transcript hosts the script lists, and on a `Generated with <tool>` byline. Run it yourself before pushing with `hack/check-commit-trailers.sh origin/main..HEAD`. It requires nothing, so a commit with no trailer passes. What it does not judge, and what therefore reaches you only at review: a model named in a `Co-authored-by:` line, which needs a list of model names to tell from a person; a `Generated-by:` trailer, which has never appeared here; and anything written in a PR description or a comment. A backport batch branch, `backport-batch/<line>-<date>` in this repository, is exempt, because its commits are cherry-picked verbatim and their messages cannot be rewritten there.
|
||||
|
||||
## Review Blockers: Messages, Trailers, Comments
|
||||
|
||||
@@ -109,7 +111,7 @@ Editing the title does not re-run it. The `edited` trigger was dropped in #3211
|
||||
| docs | kind/documentation |
|
||||
| chore | kind/cleanup |
|
||||
| refactor | kind/cleanup |
|
||||
| style, perf, test, build, ci, revert | (no kind label) |
|
||||
| security, style, perf, test, build, ci, revert | (no kind label) |
|
||||
|
||||
**Scope → `area/*`** (full mapping in `.github/workflows/pr-labeler.yaml`):
|
||||
|
||||
|
||||
@@ -96,7 +96,7 @@ Build environment variables:
|
||||
- `LOAD=1 PUSH=0` — load images locally instead of pushing. Builds for the host architecture only, because the classic docker image store cannot load a multi-platform index.
|
||||
- `PLATFORM` — the buildx platforms. Defaults to `linux/amd64,linux/arm64` for anything but `LOAD=1`, so a push from either architecture publishes an index both can run. Needs a `docker-container` builder (`BUILDER`) or a docker daemon on the containerd image store, with QEMU registered for the non-native half. Compiling builder stages run natively on `$BUILDPLATFORM` and cross-compile for the target, but every `RUN` step of a stage that runs on the target platform (package installs, `setcap`, install scripts, patch steps) still runs emulated. `packages/core/testing` pins `linux/amd64`, because it carries an x86 KVM sandbox. `packages/core/talos` builds the Talos installer and release assets for the arches in `PLATFORM` with the upstream imager, which needs no QEMU for either; its matchbox image carries the amd64 and arm64 kernel and initramfs whatever `PLATFORM` says, because it network-boots machines of either arch.
|
||||
- `PUSHED_TAGS_LOG` — a file the `image-tags` macro appends every pushed `<repo>:<tag>` to, one per line. Unset by default; the release build sets it so `hack/stitch-multiarch.sh` can move every tag it pushed, versioned ones included, onto the multi-arch index. The append happens at recipe expansion, so `make -n` writes it too.
|
||||
- `CACHE_TAG` — the tag of the mode=max build cache each image reads (and writes under `WRITE_CACHE=1`). Defaults to `buildcache`. Apart from the Talos installer and matchbox in the release job, CI builds one architecture per job, and builds arm64 images only for pre-release tags and in a nightly build of main: the images are built again on native arm64 runners under `<tag>-arm64` with `CACHE_TAG=buildcache-arm64`, which only the nightly writes. The talos and testing packages get no arm64 leg: the release job builds the Talos installer and matchbox for both arches itself, and testing is amd64 only. For a pre-release, `hack/stitch-multiarch.sh` then joins each pair into one index, `hack/verify-multiarch.sh` fails the rc on any pinned image that is not an amd64+arm64 index, and the packages artifact is republished pinned on the index digests; see `docs/release.md`. The stitch skips any image whose digest is also pinned in a file it does not rewrite, which then fails that gate; the kamaji control-plane provider's gzipped manifest is the one such copy it does rewrite (see `docs/agents/image-refs.md`).
|
||||
- `CACHE_TAG` — the tag of the mode=max build cache each image reads (and writes under `WRITE_CACHE=1`). Defaults to `buildcache`. Apart from the Talos installer and matchbox in the release job, CI builds one architecture per job, and builds arm64 images only for pre-release tags and in a nightly build of main: the images are built again on native arm64 runners under `<tag>-arm64` with `CACHE_TAG=buildcache-arm64`, which only the nightly writes. The talos and testing packages get no arm64 leg: the release job builds the Talos installer and matchbox for both arches itself, and testing is amd64 only. For a pre-release, `hack/stitch-multiarch.sh` then joins each pair into one index, `hack/verify-multiarch.sh` fails the rc on any pinned image that is not an amd64+arm64 index, and the packages artifact is republished pinned on the index digests; see `docs/release/ci-pipeline.md`. The stitch skips any image whose digest is also pinned in a file it does not rewrite, which then fails that gate; the kamaji control-plane provider's gzipped manifest is the one such copy it does rewrite (see `docs/agents/image-refs.md`).
|
||||
|
||||
### Values schema generation
|
||||
|
||||
@@ -150,7 +150,7 @@ Product documentation lives in [`cozystack/website`](https://github.com/cozystac
|
||||
| --- | --- |
|
||||
| `packages/<group>/<name>/README.md` | Anything scoped to a single package. Prose goes **above** `## Parameters`; that section is `cozyvalues-gen` output and is never hand-edited. The website's `make update-apps` / `update-vms` / `update-networking` / `update-k8s` / `update-services` targets append the whole README onto a published page, but only for the packages named in the `APPS`, `VMS`, `NETWORKING`, `K8S` and `SERVICES` lists of the website `Makefile`, and only from `packages/apps/` and `packages/extra/`: nothing under `packages/system/` is ever imported, and neither is a package that is not on those lists. The sync runs when a release is promoted or tagged, or by hand in the website repository, not when a pull request merges here. Use this route only for a package that is on the lists; for any other the prose reaches no reader, so put it on the website instead. |
|
||||
| `cozystack/website`, under `content/en/docs/` | Operator- and tenant-facing content that spans packages or is task-shaped: guides, how-tos, troubleshooting runbooks, reference tables. Which version directory a page goes into is set by the website's own [`CONTRIBUTING.md`](https://github.com/cozystack/website/blob/main/CONTRIBUTING.md); follow that file rather than a copy of its rules here. |
|
||||
| `docs/` | Documentation that a chart template, migration, script or workflow references by repo-relative path, plus this repository's own process docs (`docs/agents/`, `docs/changelogs/`, `docs/release.md`). |
|
||||
| `docs/` | Documentation that a chart template, migration, script or workflow references by repo-relative path, plus this repository's own process docs (`docs/agents/`, `docs/changelogs/`, `docs/release.md` and the pages under `docs/release/`). |
|
||||
| `api/<group>/<version>/DESIGN.md`, `pkg/apis/<group>/DESIGN.md` | Design of an API group, beside the types it describes. |
|
||||
| [`cozystack/community`](https://github.com/cozystack/community) | Design proposals and RFCs. Not this repository. |
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ Follow these instructions when the user asks to:
|
||||
|
||||
## Instructions
|
||||
|
||||
For detailed release process instructions, follow the steps documented in:
|
||||
For detailed release process instructions, follow the steps documented in the pages under `docs/release/`, starting from their index:
|
||||
|
||||
**[docs/release.md](../release.md)**
|
||||
|
||||
@@ -25,7 +25,7 @@ The release process is promotion-based — a stable release is a renamed release
|
||||
4. Finalize on merge (CI cuts the write-once tag, publishes the release from the merged changelog, and retags the rc images to stable by digest).
|
||||
5. Merging the parked website docs PR once the release is published.
|
||||
|
||||
All detailed steps, workflows, and failure modes are documented in `docs/release.md`.
|
||||
All detailed steps, workflows, and failure modes are documented in the pages under `docs/release/`, one per topic, and `docs/release.md` links each of them.
|
||||
|
||||
For a patch release, or any rc cut from a `release-X.Y` branch, verify the backport state of that branch before tagging: build with `go build ./cmd/backport-audit`, then run `./backport-audit release-X.Y`. It exits non-zero while anything labeled `kind/backport` / `kind/backport-previous` for the line has not landed on it, and prints the URLs. See [`cmd/backport-audit/README.md`](../../cmd/backport-audit/README.md) and [Cherry-pick triage before a patch](../release.md#cherry-pick-triage-before-a-patch). Do not report a branch as ready to cut on the strength of `gh pr list --search "is:merged label:kind/backport"` — that lists what was labeled, not what arrived.
|
||||
What a patch release carries is decided by the rules in [`docs/release/backports.md`](../release/backports.md), not per pull request: fixes reach the release lines through the backport batch, and nobody labels or opens a backport. For a patch release, or any rc cut from a `release-X.Y` branch, check the branch before tagging: build the batch tool with `go build ./cmd/backport-batch`, then run `./backport-batch check --line release-X.Y --cut`. It exits 1 while a pull request the rules take is not on the line, including one that is only on the open batch pull request, which the cut would not carry, and prints the URLs; what the batch deferred prints as warnings. See [`cmd/backport-batch/README.md`](../../cmd/backport-batch/README.md) and [Cherry-pick triage before a patch](../release/backport-mechanics.md#cherry-pick-triage-before-a-patch). Do not report a branch as ready to cut on the strength of a list of merged pull requests or of the batch pull request's description — they list what merged or was picked, not what arrived.
|
||||
|
||||
+24
-684
@@ -1,6 +1,6 @@
|
||||
# Release Workflow
|
||||
|
||||
This document is both the process design (what the release model looks like) and the operational playbook (what to actually do, what breaks, what to verify). Companion to [`agents/changelog.md`](./agents/changelog.md), which is the canonical changelog process.
|
||||
The release documentation is both the process design (what the release model looks like) and the operational playbook (what to actually do, what breaks, what to verify), split into one page per topic under [`release/`](./release/). Companion to [`agents/changelog.md`](./agents/changelog.md), which is the canonical changelog process.
|
||||
|
||||
## When to use this guide
|
||||
|
||||
@@ -15,692 +15,31 @@ You are about to:
|
||||
|
||||
If you are about to generate a changelog only, jump straight to [`agents/changelog.md`](./agents/changelog.md).
|
||||
|
||||
## Introduction
|
||||
|
||||
Cozystack uses a staged release process to ensure stability and flexibility during development.
|
||||
|
||||
There are three types of releases:
|
||||
|
||||
- **Release Candidates (RC)** – Preview versions (e.g., `v1.2.0-rc.1`) used for final testing and validation.
|
||||
- **Regular Releases** – Final versions (e.g., `v1.2.0`) that are feature-complete and thoroughly tested.
|
||||
- **Patch Releases** – Bugfix-only updates (e.g., `v1.2.1`) made after a stable release, based on a dedicated release branch.
|
||||
|
||||
All three are matched by `tags.yaml`'s regex `^v\d+\.\d+\.\d+(-(alpha|beta|rc)\.\d+)?$`. The tag push is the trigger for the whole pipeline. **Pre-release tags (`-alpha` / `-beta` / `-rc`) are cut with the [`Cut Pre-release Tag`](../.github/workflows/cut-prerelease.yaml) workflow** (manual dispatch from `main` or `release-X.Y`) — it pushes the tag as the CI app at the dispatch branch's tip, so GitHub fills `base_ref` (`tags.yaml`'s `Get base branch` step refuses tags pushed without a base) and the push triggers `tags.yaml` (a tag pushed with the default `GITHUB_TOKEN` would not). Stable `vX.Y.Z` tags are never pushed by hand at all — they are cut write-once by the promote flow at a PR merge commit. Only `-rc.N` pre-releases feed the promotion flow ([`promote-rc.yaml`](../.github/workflows/promote-rc.yaml) accepts `vX.Y.Z-rc.N` alone); `-alpha.N` / `-beta.N` tags are built and published for download and validated on demand with the [`e2e-tag.yaml`](../.github/workflows/e2e-tag.yaml) button, but are never promoted to a stable release.
|
||||
|
||||
## Release Candidates
|
||||
|
||||
Release candidates are Cozystack versions that introduce new features and are published before a stable release. Their purpose is to help validate stability before finalizing a new feature release.
|
||||
|
||||
Release candidates are given numbers `vX.Y.0-rc.N`, for example, `v1.2.0-rc.1`. The first one is cut from `main`, once all major features for the upcoming release have merged and the release enters its testing phase.
|
||||
|
||||
**Cutting the first rc freezes the line.** [`cut-prerelease.yaml`](../.github/workflows/cut-prerelease.yaml) creates `release-X.Y` at the tagged commit, and from that point the release's content is closed:
|
||||
|
||||
- Every later cut for that line — `rc.2`, `rc.3`, an `alpha`/`beta`, or a patch-line rc — must be dispatched from `release-X.Y`. A dispatch from `main` is refused.
|
||||
- Fixes reach the release only by cherry-pick or backport onto `release-X.Y` (see [Backports](#backports)).
|
||||
- `main` reopens immediately for the next minor, so feature work never has to wait for the release to ship.
|
||||
|
||||
This is what makes an rc mean something. Previously `release-X.Y` was created only when the stable release merged, so `rc.2` was cut from `main`'s tip and silently absorbed everything that had landed since `rc.1` — the shipped release could contain code that no rc had ever validated.
|
||||
|
||||
**Un-freezing is manual and deliberate.** If an `rc.1` turns out to be unusable and the line has to be re-cut from `main`, delete `release-X.Y` by hand first; the refuse step blocks every other route back to a `main` dispatch, which is the point of it. Nothing automates the deletion, and nothing should — throwing away a frozen line is a decision worth making explicitly.
|
||||
|
||||
Each RC contributes to a cumulative set of release notes that will be finalized when `vX.Y.0` is released. After testing, if no critical issues remain, the last good rc is **promoted** to the regular release `vX.Y.0` — the exact rc images retagged by digest, never rebuilt and never tagged from a later commit (see [Regular Releases](#regular-releases)). Promotion cuts the write-once `vX.Y.0` tag at the promote PR's merge commit, fast-forwards the `release-X.Y` maintenance branch the freeze already created, and opens the way for patch releases.
|
||||
|
||||
## Regular Releases
|
||||
|
||||
A regular release `vX.Y.0` is **promoted from a release-candidate that already passed e2e** — never tagged by hand and never rebuilt. The bytes shipped as `vX.Y.0` are the exact `vX.Y.0-rc.N` images, retagged by digest. We'll use `v1.2.0` as an example.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature"
|
||||
commit id: "feature 2"
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
```
|
||||
|
||||
A regular release sequence runs as follows:
|
||||
|
||||
1. Cut the first release candidate (`v1.2.0-rc.1`) from the last good commit on `main`: run the [`Cut Pre-release Tag`](../.github/workflows/cut-prerelease.yaml) workflow (manual dispatch **from `main`**) with the tag. It tags `main`'s HEAD as the CI app, then **creates `release-1.2` at that commit, freezing the line**; `tags.yaml` then builds the rc images, publishes the rc prerelease, pushes the digest-vendored `release-1.2.0-rc.N` staging branch, and runs its mandatory `rc-e2e` job by calling [`e2e-tag.yaml`](../.github/workflows/e2e-tag.yaml) against that published tag.
|
||||
2. Watch the rc's mandatory full E2E and wait for it to pass, then complete the other checks in [Pre-release verification checklist](#pre-release-verification-checklist). If the latest `tags.yaml` run cannot provide a green `rc-e2e`, a successful manual dispatch of [`e2e-tag.yaml`](../.github/workflows/e2e-tag.yaml) for the same tag is equivalent gate evidence. `main` is already open for the next minor, so anything landing there is **not** in this release: fixes for `1.2` are cherry-picked onto `release-1.2`, and a later rc is cut by dispatching the workflow **from `release-1.2`**. **Optional:** once the notes are worth reviewing, press the `Generate RC Changelog` button ([`changelog-rc.yaml`](../.github/workflows/changelog-rc.yaml)) to generate `docs/changelogs/v1.2.0.md` now and bank it on the `release-1.2.0-rc.N` staging branch — so there is time to review and hand-edit it before promotion. Skipping the button changes nothing; promotion generates the changelog itself.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature"
|
||||
commit id: "feature 2"
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout main
|
||||
commit id: "feature 4 (next minor)"
|
||||
checkout release-1.2
|
||||
commit id: "cherry-picked fix" tag: "v1.2.0-rc.2"
|
||||
```
|
||||
|
||||
3. Once the checks are green, run the [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml) workflow (manual dispatch **from `main`**, the only branch it is dispatched from, whichever line the rc belongs to) with that rc tag. Its `parse` job verifies E2E before any staging: it accepts a successful `rc-e2e` job in the latest `tags.yaml` push run for the tag, or any retained successful manual `e2e-tag.yaml` run whose expanded E2E job name targets the tag. The emergency `skip_e2e_gate` input bypasses this check, logs a loud warning, and marks the promote PR with ⚠️. Before its first registry write, promotion also proves that the selected `main` or `release-X.Y` target already contains the candidate-aware PR gate, finalizer and trusted verifier; backport this pipeline to an existing maintenance branch before promoting its next patch. Promotion **creates no stable name before merge**: dispatch does write to the registry — it publishes one `promotion-*` packages candidate, and that tag is real and publicly listable — but it creates no `vX.Y.Z` tag, moves no `latest`, and publishes no stable installer, so nothing a consumer resolves changes until the PR merges. It:
|
||||
1. Rewrites the rc version substring to `1.2.0` and commits that exact tree, serializes its `packages/` directory under a run-specific temporary OCI tag, then adds a second commit pinning the candidate's immutable digest in the installer before pushing `release-1.2.0`. Container digests remain exactly the rc's; the two commits give the OCI revision an honest source SHA without pretending the artifact can contain its own digest.
|
||||
2. Drafts the `v1.2.0` release and uploads the (restamped) assets.
|
||||
3. Generates `docs/changelogs/v1.2.0.md` (in a parallel job, detached at the rc tag) — or, if you pressed the rc-time `Generate RC Changelog` button, copies the file already banked on the `release-1.2.0-rc.N` staging branch instead of regenerating — and commits it onto the `release-1.2.0` branch, so the changelog is reviewed as part of the promotion and finalize can publish the release with it already in place. Non-blocking: if generation fails the PR still opens, carrying a ⚠️ line that says the changelog is missing.
|
||||
4. Opens the `cozystack/website` "update managed apps reference" PR (`update-docs-v1.2.0` branch), generating the docs from the `release-1.2.0` staging branch via `FETCH_REF`. Non-blocking, and **do not merge that PR until `v1.2.0` is published** — merging early breaks the live site's version routing (see [Phase 3](#phase-3--update-website-docs-now-the-backstop)).
|
||||
5. Opens the `chore(release): promote v1.2.0-rc.N -> v1.2.0` PR into `release-1.2`, labelled `release` only. The PR body records whether E2E was verified or bypassed, plus ✅/⚠️ changelog and website-docs lines. E2E does not run on this promote PR by default because the immutable rc already passed the mandatory full suite; a maintainer may add `full-e2e` to run it again. The stable retag itself is deferred to the merge (step 5), so an abandoned promotion leaves no stable-named images and cannot wedge a re-promotion; retention protects the temporary candidate for as long as this release PR remains open, then may remove it after 30 days.
|
||||
|
||||
The PR targets the maintenance branch, not `main`: `promote-rc.yaml` prefers `release-X.Y` whenever it exists, and since the rc freeze it always does. The digest-vendored `Prepare release` commit therefore stays on the release line instead of landing on `main`.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "cherry-picked fix" tag: "v1.2.0-rc.2"
|
||||
branch release-1.2.0
|
||||
checkout release-1.2.0
|
||||
commit id: "Prepare release v1.2.0"
|
||||
commit id: "Pin promoted packages artifact"
|
||||
checkout release-1.2
|
||||
merge release-1.2.0 id: "Pull Request"
|
||||
```
|
||||
|
||||
4. Maintainer reviews the PR and the draft release against the [Phase 4 checklist](#phase-4--review-and-merge-the-promote-pr): confirm the E2E-verification line is ✅ (or explicitly accept its ⚠️ bypass), the changelog line is ✅, and note the website-docs line — the `cozystack/website` docs PR that promotion parked, which stays unmerged for now. E2E does not run on this promote PR by default. Then merge — **do not squash-merge**, the stable tag must attach to a real merge commit. GitHub removes the merged branch `release-1.2.0` (the repo has auto-delete-on-merge enabled).
|
||||
5. CI workflow triggers on merge (this is where every consumer-visible or write-once side effect happens, after the rc E2E gate and PR review):
|
||||
1. Pulls the pinned packages candidate by digest and verifies that it matches the merged `packages/` tree apart from the artifact's impossible self-reference, carries no rc image tags, and preserves the original rc artifact's container `repo@digest` set.
|
||||
2. Creates the tag `v1.2.0` at the newly created merge commit — write-once. The tag is published here for the first time, never moved.
|
||||
3. Cuts the write-once `api/apps/v1alpha1/v1.2.0` Go-module tag at the same commit.
|
||||
4. Ensures the `release-1.2` maintenance branch exists at the tag commit. Since the rc freeze created it, this is normally a no-op fast-forward: the merge commit is already its tip.
|
||||
5. Publishes the release page (`draft` → `latest`), using `docs/changelogs/v1.2.0.md` from the merge commit as the release body.
|
||||
6. Retags the verified packages candidate and rc container images by digest to `v1.2.0` (and `:latest` only when `v1.2.0` is the newest published stable), and publishes the stable `cozy-installer` chart — no container rebuild.
|
||||
6. Once the release is published, merge the parked `cozystack/website` docs PR (opened back at promote time, step 3.iv) so the live site's version routing flips to `v1.2.0` — its versioned docs and `openapi.json` are on GitHub now. The maintainer can then announce the release to the community.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 2"
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout main
|
||||
commit id: "feature 4 (next minor)"
|
||||
checkout release-1.2
|
||||
commit id: "cherry-picked fix" tag: "v1.2.0-rc.2"
|
||||
branch release-1.2.0
|
||||
checkout release-1.2.0
|
||||
commit id: "Prepare release v1.2.0"
|
||||
checkout release-1.2
|
||||
merge release-1.2.0 id: "Release v1.2.0" tag: "v1.2.0"
|
||||
```
|
||||
|
||||
Note that `main` never receives the release commit. It forked away at the freeze and carries the next minor's work.
|
||||
|
||||
## Patch Releases
|
||||
|
||||
A patch release now follows exactly the same shape as the regular release above — by the time `vX.Y.0` ships, everything is already happening on `release-X.Y`. The only difference is where the line starts:
|
||||
|
||||
* `release-X.Y` already exists, created by the freeze at `vX.Y.0-rc.1`, so there is no branch to create.
|
||||
* Patch commits are cherry-picked onto it, and the rc is cut by dispatching **from `release-X.Y`** — the same dispatch used for `rc.2` of the original release.
|
||||
|
||||
|
||||
Let's assume that we've released `v1.2.0` and that development is ongoing.
|
||||
We have introduced a couple of new features and some fixes to features that we have released
|
||||
in `v1.2.0`.
|
||||
|
||||
Once problems were found and fixed, a patch release is due.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "Release v1.2.0" tag: "v1.2.0"
|
||||
checkout main
|
||||
commit id: "feature 4"
|
||||
commit id: "patch 1"
|
||||
commit id: "feature 5"
|
||||
commit id: "patch 2"
|
||||
```
|
||||
|
||||
|
||||
1. The maintainer cherry-picks patch commits from `main` onto the existing `release-1.2` branch. These must be only patches to features that were present in version `v1.2.0`.
|
||||
|
||||
Cherry-picking can be done as soon as each patch is merged into `main`, or directly before the release.
|
||||
|
||||
The `backport` label automates this: [`backport.yaml`](../.github/workflows/backport.yaml) targets the newest existing `release-X.Y` branch, which during a freeze window is the line being stabilised rather than the last published one.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "Release v1.2.0" tag: "v1.2.0"
|
||||
checkout main
|
||||
commit id: "feature 4"
|
||||
commit id: "patch 1"
|
||||
commit id: "feature 5"
|
||||
commit id: "patch 2"
|
||||
checkout release-1.2
|
||||
cherry-pick id: "patch 1"
|
||||
cherry-pick id: "patch 2"
|
||||
```
|
||||
|
||||
When all relevant patch commits are cherry-picked, the branch is ready for release. Build with `go build ./cmd/backport-audit`, then confirm with `./backport-audit release-1.2` — it exits non-zero while anything labeled for the line is still missing from it, and prints the URLs. See [Cherry-pick triage before a patch](#cherry-pick-triage-before-a-patch).
|
||||
|
||||
2. The maintainer cuts a release candidate (`v1.2.1-rc.N`) via the [`Cut Pre-release Tag`](../.github/workflows/cut-prerelease.yaml) workflow (manual dispatch **from `release-1.2`**), which tags that branch's `HEAD`. CI builds and publishes the rc, pushes the `release-1.2.1-rc.N` staging branch, and runs the mandatory full `rc-e2e` job against the published tag.
|
||||
3. Watch `rc-e2e` and wait for it to pass, then complete the other release checks. A successful manual `e2e-tag.yaml` run for `v1.2.1-rc.N` is also valid evidence if the latest `tags.yaml` run does not carry a green `rc-e2e`. **Optional:** press the `Generate RC Changelog` button ([`changelog-rc.yaml`](../.github/workflows/changelog-rc.yaml)) to generate `docs/changelogs/v1.2.1.md` at rc time and bank it on the rc staging branch; promotion then reuses it instead of regenerating.
|
||||
4. Once the checks are green, run [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml) (manual dispatch **from `main`**, as for every promotion, not from `release-1.2`) with that rc tag. It runs the same five jobs as a regular promotion: the dispatch first gates on the latest tag-push `rc-e2e` or a successful tag-correlated manual E2E run (`skip_e2e_gate` overrides it), verifies that `release-1.2` already carries the candidate-aware pipeline, then commits and publishes the rewritten packages tree under a run-specific temporary `promotion-*` tag, adds a second commit pinning the candidate on the `release-1.2.1` branch, drafts the `v1.2.1` release with the restamped assets, generates or picks up `docs/changelogs/v1.2.1.md` and commits it onto that branch, opens the parked `cozystack/website` docs PR (updating the existing `v1.2/` directory in place, since a patch does not promote `next/`), and opens the promote PR into `release-1.2` with only the `release` label. The promote PR body carries the same ✅/⚠️ E2E, changelog, and website-docs lines and runs no E2E by default; add `full-e2e` manually to force a repeat. As with a regular release every stable name is deferred to merge; retention protects the temporary candidate while this PR is open and, after it merges, while `release-1.2` still pins it — until finalize retags it stable.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "Release v1.2.0" tag: "v1.2.0"
|
||||
checkout main
|
||||
commit id: "feature 4"
|
||||
commit id: "patch 1"
|
||||
commit id: "feature 5"
|
||||
commit id: "patch 2"
|
||||
checkout release-1.2
|
||||
cherry-pick id: "patch 1"
|
||||
cherry-pick id: "patch 2" tag: "v1.2.1-rc.1"
|
||||
branch release-1.2.1
|
||||
commit id: "Prepare release v1.2.1"
|
||||
checkout release-1.2
|
||||
merge release-1.2.1 id: "Pull request"
|
||||
```
|
||||
|
||||
Finally, when release is confirmed, the release sequence goes on.
|
||||
|
||||
5. Maintainer reviews the PR against the [Phase 4 checklist](#phase-4--review-and-merge-the-promote-pr), confirms the E2E-verification and changelog lines and notes the parked website-docs line, and merges it — **do not squash-merge**. GitHub removes the merged branch `release-1.2.1` (auto-delete-on-merge is enabled).
|
||||
6. CI workflow triggers on merge (all consumer-visible or write-once side effects, after the rc E2E gate and PR review):
|
||||
1. Verifies the pinned packages candidate against the merged tree and the original rc artifact's container digests.
|
||||
2. Creates the tag `v1.2.1` at the newly created merge commit — write-once. The tag is published here for the first time, never moved.
|
||||
3. Cuts the write-once `api/apps/v1alpha1/v1.2.1` Go-module tag at the same commit.
|
||||
4. Publishes the release page (`draft` → `latest`; `latest` moves only if `v1.2.1` is the newest published stable).
|
||||
5. Retags the verified packages candidate and rc container images by digest to `v1.2.1` and publishes the stable `cozy-installer` chart — no container rebuild.
|
||||
7. Once the release is published, merge the parked `cozystack/website` docs PR, then announce the release to the community.
|
||||
|
||||
## What CI does during the release process
|
||||
|
||||
The numbered process above is implemented by seven workflows, including the reusable/manual [`e2e-tag.yaml`](../.github/workflows/e2e-tag.yaml), plus the reusable [`changelog-generate.yaml`](../.github/workflows/changelog-generate.yaml) that two of them share to produce the changelog. Knowing which job does what makes the failure modes much easier to diagnose.
|
||||
|
||||
1. [`cut-prerelease.yaml`](../.github/workflows/cut-prerelease.yaml) — manual dispatch (from `main` or `release-X.Y`): the sole entry point for creating a pre-release tag. Validates the `-alpha`/`-beta`/`-rc` tag, then pushes it (write-once, at the branch tip) as the CI app so `tags.yaml` fires. Cutting the first `vX.Y.0-rc.N` also creates `release-X.Y`, freezing the line; afterwards a `vX.Y.*` dispatch from `main` is refused. Stable tags are never created here.
|
||||
2. [`tags.yaml`](../.github/workflows/tags.yaml) — fires on any pre-release tag push: `build-amd64` and `build-arm64` build the pre-release side by side, then `prepare-release` publishes it and pushes its digest-pinned staging branch, then — **only for `-rc.N` tags** — the mandatory `rc-e2e` job calls `e2e-tag.yaml` in the same run (`-alpha.N` / `-beta.N` skip it and are validated on demand with `e2e-tag.yaml`'s manual button). `generate-changelog` and `update-website-docs` also live here but are **backstops only** — both are normally produced at promotion time (see 4); on a stable tag push each self-skips or no-ops when the promote flow already did the work. These stable-tag backstops only fire because finalize pushes the stable tag with the CI app token (a `GITHUB_TOKEN` push triggers no run — see 5).
|
||||
3. [`e2e-tag.yaml`](../.github/workflows/e2e-tag.yaml) — reusable from `tags.yaml::rc-e2e` and manually dispatchable for a published tag. It checks that the tag's release carries its nocloud disk, resolves the digest-pinned tree and runs the full install, OpenAPI, and Chainsaw suite on Talos containers. A tag cut before the container lane existed has to be dispatched from its own ref. Its dispatch run title is the constant workflow name, so the promote gate correlates a manual run through the successful tag-bearing E2E job name rather than the title.
|
||||
4. [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml) — manual dispatch, from `main` only (the signature over a stable release's assets names that ref, see [Release signatures and SBOM](#release-signatures-and-sbom)). Five jobs: `parse` validates the rc tag and requires green full-E2E evidence before any staging; `promote` rewrites and commits the rc tags, publishes that packages tree as a run-specific temporary reproducible candidate, commits its digest pin in the `release-X.Y.Z` tree, and drafts the stable release; `changelog` calls the reusable `changelog-generate.yaml` **in parallel** to produce `docs/changelogs/vX.Y.Z.md` (regenerating it, or copying an rc-time changelog already committed to the `release-X.Y.Z-rc.N` staging branch by `changelog-rc.yaml` — see 7); `website-docs` (needs `parse`, `promote`) opens the `cozystack/website` docs PR from the staging branch via `FETCH_REF` (see [Phase 3](#phase-3--update-website-docs-now-the-backstop)); `open-pr` joins them, commits the changelog onto the staging branch, and opens the `release-X.Y.Z` promote PR with the `release` label but no automatic `full-e2e`. So the changelog and website docs are reviewed as part of the promotion, the already-checked rc suite is not repeated by default, and the PR body carries ✅ verified or ⚠️ bypassed plus ✅/⚠️ changelog and website-docs lines. Both `changelog` and `website-docs` are non-blocking. No stable registry name is created at dispatch.
|
||||
5. [`.github/workflows/pull-requests.yaml`](../.github/workflows/pull-requests.yaml) verifies a promotion candidate against the prospective merge tree and original rc artifact before the required `E2E Tests` status can turn green. [`pull-requests-release.yaml`](../.github/workflows/pull-requests-release.yaml) fires after merge and repeats that verification before any stable tag, then cuts the write-once stable + Go-module tags, publishes the release **with the merged changelog as its body**, retags the candidate and rc images to stable by digest (`:latest` gated on newest-stable), and publishes the stable chart.
|
||||
6. [`update-releasenotes.yaml`](../.github/workflows/update-releasenotes.yaml) — fires on pushes to `main` that touch `docs/changelogs/v*.md`; syncs that content into the corresponding GitHub Release body. No longer the primary path (finalize sets the body directly, which is race-free and works for maintenance lines too) — this now covers later edits to a published changelog and manual re-syncs via `workflow_dispatch`.
|
||||
7. [`changelog-rc.yaml`](../.github/workflows/changelog-rc.yaml) — manual dispatch, **optional**: generate the changelog at rc time instead of waiting for promotion, while there is still time to review it. It calls the same reusable [`changelog-generate.yaml`](../.github/workflows/changelog-generate.yaml) and commits the result to the `release-X.Y.Z-rc.N` staging branch as `docs/changelogs/vX.Y.Z.md`; the promote-time `changelog` job then copies that file instead of running the AI again. Skipping this workflow changes nothing — promote generates the changelog itself, exactly as before.
|
||||
|
||||
### Phase 1 — `build-amd64`, `build-arm64` and `prepare-release` (hard gate)
|
||||
|
||||
On an rc tag push: builds images for both architectures, commits digest pins, creates the draft release, and pushes the `release-X.Y.Z-rc.N` staging branch. `build-amd64` holds the release-exists no-op and the rejection of a hand-pushed stable tag, and builds the amd64 images and the release assets (its `make build` also pushes the packages artifact and the installer chart, pinned on amd64 digests); `prepare-release` stitches, gates, republishes those two from the repinned tree, and publishes the release. (The stable promote PR is opened later by `promote-rc.yaml`, not here. When the stable tag itself is pushed by finalize, a draft already exists, so this job is a no-op — see [Phase 5](#phase-5--pull-requests-releaseyaml-finalize).)
|
||||
|
||||
The commit (`Prepare release vX.Y.Z`, authored by `cozystack-ci[bot]`) is **digest pins and image tags only**:
|
||||
|
||||
- `packages/core/{installer,platform,testing}/values.yaml`
|
||||
- `packages/system/*/values.yaml` (cozystack-api, dashboard, kamaji, linstor, kubevirt-csi-node, etc.)
|
||||
- `packages/apps/kubernetes/images/{kubevirt-csi-driver,ubuntu-container-disk-*}.tag`
|
||||
- `packages/system/dashboard/templates/configmap.yaml` (the `$tenantText` value)
|
||||
|
||||
Things that look surprising in the diff but are normal:
|
||||
|
||||
- **Zero-diff packages**: when buildx fully caches and upstream base images haven't moved, the digest is byte-identical to the previous release and won't appear. Verify the build step actually ran — don't assume "no diff = nothing built."
|
||||
- **Sudden churn on `ubuntu-container-disk-*` tags**: `cloud-images.ubuntu.com/noble/current/` is a moving target; these often rebuild even without code changes.
|
||||
- **Switched registry**: if a self-built image moved from `docker.io` to `ghcr.io`, the registry portion of the digest changes — that's a deliberate move, not a regression.
|
||||
- **`targetVersion` is NOT touched by the release PR.** Platform migration `targetVersion` is bumped earlier in a feature commit (e.g. `[platform] Bump migration targetVersion to 39 for migration 38`). The release PR only re-pins the `platform-migrations` image digest.
|
||||
|
||||
#### Multi-arch images
|
||||
|
||||
Every pre-release is published for amd64 and arm64. `tags.yaml::build-arm64` builds the images on a native arm64 runner and pushes each as `<tag>-arm64`, while `build-amd64` builds amd64, the Talos installer and matchbox for both arches, and the assets on x86 at the same time. `build-amd64` hands its stamped tree, the log of the tags it pushed and the assets to `prepare-release` as workflow artifacts; a missing one fails the upload or the download rather than publishing a partial release. `prepare-release` waits for both builds and runs only if both succeeded: when either fails, the rc is not prepared at all, so a release is never published amd64 only. It applies the stamped tree and runs [`hack/stitch-multiarch.sh`](../hack/stitch-multiarch.sh), which joins each image with its arm64 twin into one index, moves onto it every tag the build pushed for that image (the rc tag and, under `PUBLISH_VERSIONED=1`, its component version, which the build records in `PUSHED_TAGS_LOG`), and repins the tree on the index digests, then [`hack/verify-multiarch.sh`](../hack/verify-multiarch.sh), then republishes the packages artifact and the installer chart from the repinned tree. A stable tag skips `build-arm64` and builds nothing: promotion copies the rc's indexes with every platform, so the stable release is exactly as multi-arch as its rc. Because the two builds run in parallel, an rc that is not prepared (its arm64 build failed, or `prepare-release` failed before the stitch) still leaves behind what the amd64 build pushed: amd64-only images under `:<rc>` and under the component-version tags, and the `cozystack-packages:<rc>` artifact and the installer chart, both pinned on those amd64 digests. A component tag an earlier rc had moved to an index then points at an amd64-only image again, until a run that succeeds stitches it, a re-run of the same run included.
|
||||
|
||||
`verify-multiarch.sh` is a hard gate. It resolves every ref the tree pins, third-party images included, and fails naming each one that is not an index carrying both `linux/amd64` and `linux/arm64`, or that it cannot resolve at all. The fix is to make the image multi-arch (build it for both, or pin a multi-arch digest of the same version); an image that cannot exist for arm64 goes in [`hack/multiarch-allowlist`](../hack/multiarch-allowlist) by repository, with the reason on the same line.
|
||||
|
||||
That gate sees only what the tree pins by digest. An image a vendored chart renders from its own defaults, or one an operator launches at runtime, is invisible to it. `rc-e2e` closes most of that gap: before tearing the sandbox down, [`e2e-tag.yaml`](../.github/workflows/e2e-tag.yaml) reads every image the three Talos nodes pulled from their image stores (`hack/collect-images.sh`), each by the digest the pull resolved to, and runs `verify-multiarch.sh --refs-file` on that list with the same allowlist. A pre-release fails on any single-arch image, on any ref it cannot read or that is still rate-limited after the retries, and on a list the collection could not produce; a stable tag's run only warns, since it ships its rc's images. A tag whose tree predates the audit (it has no `hack/verify-multiarch.sh`) only warns and passes, so an rc cut before the audit existed can still be re-validated. The list and the audit output are in the run's `image-list-<tag>` artifact. Neither check covers a package that e2e never installs and whose images the tree does not pin by digest: an arm64 break there shows up only on an arm64 cluster.
|
||||
|
||||
[`build-arm64-nightly.yaml`](../.github/workflows/build-arm64-nightly.yaml) builds `main` for arm64 every day, so an arm64 build break shows up there before the next rc. Its first step prints `verify-multiarch.sh --report` for `main`'s committed tree: `main` pins amd64 images, so its first-party refs are all listed and are expected to be, since the rc's stitch turns them into indexes. A third-party ref that an in-tree rebuild stamps over at build time, as the Harbor components and shadowbox are, is listed too and goes away on the rc. The lines that matter are the third-party refs no build replaces and refs that cannot be resolved; each of those fails the next rc until it is fixed or allowlisted. matchbox is listed too and does not matter: `build-amd64` rebuilds it as a two-arch index. The stitch names the images it skips in its own summary: an image with no `<tag>-arm64` twin, one whose digest is also pinned in a file the rewrite does not reach, and a pair that is not one amd64 and one arm64 image. matchbox (already an index) and the e2e sandbox (never built for arm64, and allowlisted) are skipped on every rc and pass the gate; any other skipped image fails it.
|
||||
|
||||
### RC E2E — mandatory check before promotion
|
||||
|
||||
For an rc, `prepare-release` must finish before `tags.yaml::rc-e2e` calls the published-tag lane, so the full suite tests the digest-pinned staging tree that promotion will reuse, and the release it runs against already carries its disk. `promote-rc.yaml::parse` then checks the latest `tags.yaml` push run whose `head_branch` is the rc tag and accepts it only when the reusable E2E job succeeded. GitHub exposes called-workflow job names as `E2E Release Candidate / E2E vX.Y.Z-rc.N (full suite)`, which the gate matches by the exact tag-bearing suffix.
|
||||
|
||||
The alternate evidence path scans retained successful `workflow_dispatch` runs of `e2e-tag.yaml` and accepts one whose direct E2E job is named `E2E vX.Y.Z-rc.N (full suite)`. The workflow's run title is always `E2E Release Tag` and therefore cannot correlate the input tag; artifact names also carry the tag but expire independently, so the job name is the strongest available correlator. If Actions retention or manual deletion removes the run/job metadata, or that job is renamed without updating the matcher, the gate fails closed and tells the maintainer to run the E2E Release Tag button again. `skip_e2e_gate=true` is an emergency override, not alternate evidence: it writes a loud warning and a ⚠️ PR-body line.
|
||||
|
||||
### Phase 2 — the changelog (generated at rc time or promotion, not here)
|
||||
|
||||
The changelog for `vX.Y.Z` is produced by [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml)'s `changelog` job (a thin call to the reusable [`changelog-generate.yaml`](../.github/workflows/changelog-generate.yaml)) and committed onto the `release-X.Y.Z` staging branch by `open-pr`, so it rides the promote PR and is on the base branch the moment the release publishes. Finalize then uses that file verbatim as the GitHub Release body. Generating it here — on the stable tag push — would be too late by construction: that tag is created by finalize, which has already published the release. It can also be produced earlier, at rc time, via the optional [`changelog-rc.yaml`](../.github/workflows/changelog-rc.yaml) button — which runs the same reusable core and commits the file to the `release-X.Y.Z-rc.N` staging branch, from which the promote-time `changelog` job copies it (validated) instead of regenerating.
|
||||
|
||||
The `changelog` job runs **in parallel** with `promote`, detached at the rc tag (the "promotion" configuration in §2 of [`agents/changelog.md`](./agents/changelog.md)), using a separate **read-only** GitHub App token for the AI step so the model cannot mutate the repo even with `--allow-all-tools`. Every write — the commit onto the staging branch and the PR — happens in `open-pr` under a write-scoped token.
|
||||
|
||||
Generation is deliberately **non-blocking**: a Copilot outage or exhausted token quota must never block a release. Only the AI step carries `continue-on-error`, and `open-pr` is gated on `promote` alone (`if: !cancelled() && needs.promote.result == 'success'`) so the promote PR opens regardless. When the changelog is missing the PR body says so explicitly — that ⚠️ line is the signal to act on.
|
||||
|
||||
Recovery, in order of preference:
|
||||
|
||||
1. Commit `docs/changelogs/vX.Y.Z.md` to the `release-X.Y.Z` branch by hand (follow [`agents/changelog.md`](./agents/changelog.md)). `promote` preserves it across a re-dispatch, and `open-pr` will not overwrite it with generated output.
|
||||
2. Re-dispatch `promote-rc.yaml` — generation runs again; anything already on the branch wins.
|
||||
3. Merge without one and let `tags.yaml::generate-changelog` (the backstop) open a changelog PR against `main` after the fact. The release is published with the draft's body until that merges.
|
||||
|
||||
Known failure modes:
|
||||
|
||||
- Copilot quota exhausted (`COPILOT_GITHUB_TOKEN` 402). Refill or rotate, then re-dispatch.
|
||||
- AI step timeout (hard 30-min cap).
|
||||
- Output truncated or whitespace-only — caught by the `Verify changelog` step (`hack/validate-changelog.sh`), which accepts either header convention (`# Cozystack vX.Y.Z` for a minor or `# vX.Y.Z (<date>)` for a patch), and requires the leading release-link comment pointing at `releases/tag/vX.Y.Z`, a `compare/...vX.Y.Z` link ending in this version, and at least one `## ` section — deliberately with no line-count floor, since a complete short patch changelog (v1.5.1 ships in 19 lines) must not be rejected as a fragment. It downgrades to the missing-changelog path rather than shipping a fragment as release notes.
|
||||
|
||||
**`tags.yaml::generate-changelog` is the backstop.** It self-skips when the changelog is already on `main`. It earns its keep in two cases: generation failed during promotion, or the promote PR targeted `release-X.Y` so the file never reached `main`. Since the rc freeze the second case is **every** release, minor and patch alike — the promote PR always targets the release line (see [Regular Releases](#regular-releases)), which makes this backstop the only route by which a published changelog reaches `main`. In that case it **ports the reviewed file from the tag commit verbatim** rather than regenerating — a second AI pass there would spend quota and then overwrite already-published, already-reviewed release notes.
|
||||
|
||||
### Phase 3 — `update-website-docs` (now the backstop)
|
||||
|
||||
The `cozystack/website` "update managed apps reference" PR is normally opened **earlier, at promote time**, by [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml)'s `website-docs` job — not here. That job fetches the README content from the `release-X.Y.Z` staging branch `promote` just pushed (via the website Makefile's `FETCH_REF` variable), so the docs are ready to review alongside the promote PR while the stable tag is still unborn. It refreshes the trunk version pins from the staging ref before promoting `next/` → `vX.Y/`, so a new minor's docs inherit fresh Talos/cozystack pins instead of a stale `next.yaml`. It is **non-blocking**: if it fails (or trips its `FETCH_REF`-support guard against a website checkout that predates that Makefile variable), the promote PR opens with a ⚠️ website-docs line and this tag-time job backfills the same branch.
|
||||
|
||||
**Merge-timing contract:** the website PR must NOT be merged before the release is published (finalize). Merging it early flips the site's latest-version pointer to a version whose versioned docs and `openapi.json` are not on GitHub yet, so every version-pin and API link for the new version 404s. Both the website PR body and the promote PR body say so; park the website PR until the GitHub release for `vX.Y.Z` is live, then merge.
|
||||
|
||||
This tag-time `update-website-docs` job stays as the **backstop**. It runs with `if: !cancelled() && needs.prepare-release.result == 'success'` (surviving a failed/skipped changelog phase, stopping on cancel), decides whether to promote `next/` → `vX.Y/`, runs `make update-all`, stages `content hugo.yaml data/versions`, and force-pushes the same `update-docs-vX.Y.Z` branch. On the normal path — where `promote-rc.yaml::website-docs` already did the work — it no-ops into "No changes to commit". At tag time the stable tag exists, so its `BRANCH=RELEASE_TAG` fetch resolves without needing `FETCH_REF`. For it to fire at all, finalize must push the stable tag with the **CI app token** (see [Phase 5](#phase-5--pull-requests-releaseyaml-finalize)) — a tag pushed with the default `GITHUB_TOKEN` triggers no workflow run.
|
||||
|
||||
If anyone changes the website Makefile to write somewhere else (e.g. `static/`, `i18n/`), the `git add` list must grow in **both** jobs or the PR silently drops files.
|
||||
|
||||
### Phase 4 — review and merge the promote PR
|
||||
|
||||
Reviewer checklist for the `chore(release): promote vX.Y.Z-rc.N -> vX.Y.Z` PR:
|
||||
|
||||
- [ ] The PR body says `✅ RC full e2e was verified`; if it says the gate was bypassed, obtain explicit maintainer acceptance or add `full-e2e` and wait for that opt-in run.
|
||||
- [ ] `Verify release packages candidate` is green. It pulls the candidate and original rc artifact by digest, compares the candidate to GitHub's prospective merge tree, and feeds the required `E2E Tests` status even when full E2E is not repeated. Do not edit `packages/` on this PR after generation; if the guard reports drift, re-dispatch promotion to create and pin a new run-specific candidate. Note what a green here does *not* cover: GitHub fires no `pull_request` event when the base branch advances, so a `packages/` change merged to the base afterwards is never re-verified on the PR. Finalize repeats the check against the real merge commit and catches that case before any stable name exists — but it catches it after the promote PR has already merged, so a `packages/` change landing on the base while a promote PR is open is worth holding back.
|
||||
- [ ] The PR body's **website-docs line** is present: if ✅, the `cozystack/website` docs PR is open and **parked** — do NOT merge it until `vX.Y.Z` is published (see [Phase 3](#phase-3--update-website-docs-now-the-backstop)); if ⚠️, the tag-time backstop will backfill it after finalize.
|
||||
- [ ] Diff is digest pins + image tags only — nothing else.
|
||||
- [ ] No accidental config drift (a value file you don't recognize).
|
||||
- [ ] If any chart bumped, check the new digest pulls — `crane manifest ghcr.io/cozystack/cozystack/<name>@<digest>` should return.
|
||||
- [ ] Build artifacts on the draft release page exist and match the expected count.
|
||||
- [ ] The PR includes `docs/changelogs/vX.Y.Z.md`, it has content, and the entries match the commit range (see [Changelog pre-publish verification](#changelog-pre-publish-verification)). If the PR body carries the ⚠️ "No changelog in this PR" line, resolve that before merging — finalize publishes this file as the release body, unless the PR also carries `docs/releasenotes/vX.Y.Z.md` (see [Phase 6](#phase-6--update-releasenotesyaml-sync-github-release-body)).
|
||||
|
||||
### Phase 5 — `pull-requests-release.yaml` (finalize)
|
||||
|
||||
Fires on merge of a PR that is merged, carries the `release` label, and is authored by `cozystack-ci[bot]` (the last guard closes the "name a branch `release-X.Y.Z`, get it labelled, merge it" hole). Head branch must match `release-X.Y.Z[-suffix]`. This is where **every consumer-visible or write-once side effect of a promotion happens** — after the rc's mandatory E2E was checked at dispatch and the PR merged, making promotion transactional at stable names. Steps:
|
||||
|
||||
1. **Verify the packages candidate before creating any stable tag.** [`hack/verify-promoted-packages.sh`](../hack/verify-promoted-packages.sh) pulls the candidate by the digest committed in installer values, proves it matches the merged `packages/` tree after normalizing only the impossible self-reference, rejects surviving rc image tags, then follows the candidate's embedded pre-push `platformSourceRef` back to the original rc artifact and compares normalized container `repo@digest` sets. Any package edit after candidate publication or any changed container digest aborts finalize while all stable names are still absent.
|
||||
2. **Create the tag at the merge commit** (write-once). The merge commit of `Prepare release vX.Y.Z` did not exist before the PR opened, so there is nothing to move — the tag is created here for the first time. A pre-existing tag at a different commit fails the step loudly rather than being force-moved (see [Tag immutability](#tag-immutability)). The checkout uses `persist-credentials: false` so this push authenticates as the **CI app** (via `git remote set-url`), not the default `GITHUB_TOKEN` — a `GITHUB_TOKEN` push creates no workflow run, so a persisted-credential push would silently skip `tags.yaml` entirely and its stable-tag backstops (`generate-changelog`, `update-website-docs`) would never fire.
|
||||
3. **Cut the `api/apps/v1alpha1/vX.Y.Z` Go-module tag** (write-once, stable only) at the same commit, so Go consumers of `api/apps/v1alpha1` get the release. Moved here from `tags.yaml`, whose `prepare-release` body is skipped for a promoted stable (the draft already exists).
|
||||
4. **Ensure the maintenance branch `release-X.Y` exists** at the tag commit. Since the rc freeze creates it, this is normally a no-op fast-forward — the merge commit is already the branch tip. Still created if missing; updated fast-forward-only — a non-fast-forward update warns and is left for a maintainer rather than being force-updated.
|
||||
5. **Publish the draft release**, with `docs/releasenotes/vX.Y.Z.md`, or without it `docs/changelogs/vX.Y.Z.md`, from the merge commit as its body (absent or whitespace-only files warn and leave the draft's body in place, rather than publishing blank notes, and a body over 100000 bytes is published as a link, see [Phase 6](#phase-6--update-releasenotesyaml-sync-github-release-body)). `make_latest` is computed against published-non-prerelease tags: prereleases stay `false`; tags older than the current max stay `false` (and the current max is force-restored to `latest` if necessary, so an older patch tag cut after a newer minor won't downgrade `latest`).
|
||||
6. **Retag the verified packages candidate and rc container images to stable** by digest (`hack/promote-retag.sh`, no container rebuild) and **publish the stable `cozy-installer` chart**. `:latest` (on both the images and the chart) moves only when this release's `make_latest` was `true` — the same decision as step 5, so the release's `latest` and the images' `:latest` never disagree. The chart is packaged with `platformVersion` stamped into its default values so the documented `helm --install --version X.Y.Z` path reports the stable version.
|
||||
|
||||
### Phase 6 — `update-releasenotes.yaml` (sync GitHub Release body)
|
||||
|
||||
Fires on pushes to `main` that touch `docs/changelogs/v*.md`. Reads each `docs/changelogs/vX.Y.Z.md` and PATCHes the matching GitHub Release's `body` if it differs. Since Phase 5 now sets the body directly from the merge commit, this is no longer on the critical path — it covers later **edits** to a published changelog, the backstop PR landing a changelog that reached `main` after the fact, and manual re-syncs via `workflow_dispatch`. It only watches `main`, which is why finalize (not this job) is what populates a release promoted onto `release-X.Y` — since the freeze, that is every release rather than only a patch.
|
||||
|
||||
The release body is `docs/releasenotes/vX.Y.Z.md` when it exists and is not blank, otherwise the changelog, which stays the full record either way, and a push to `main` touching either file fires this sync. Finalize sets the body after the stable tags exist, where a body GitHub refuses would strand the release, so at publish time a body over 100000 bytes is replaced by a one-line link to `docs/changelogs/vX.Y.Z.md` on the tag, with a warning in the run's log. This sync never writes that link: it warns and leaves such a release's body as it is. The `tags.yaml` backstop ports only the changelog to `main`, so release notes have to be added to `main` as well, otherwise the sync falls back to the changelog and replaces the release notes with it whenever the changelog fits.
|
||||
|
||||
## Stable tags come from rc promotion
|
||||
|
||||
A stable `vX.Y.Z` is created only by **promoting an existing release-candidate** that already passed e2e — never by rebuilding and never by a cron.
|
||||
|
||||
- [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml) is triggered manually once an rc has gone green. It rewrites the rc version substring in the vendored image tags (the container digests stay the rc's), commits and publishes the resulting declarative packages tree under a unique run-specific `promotion-*` tag, commits that digest pin, and opens the `release-X.Y.Z` staging PR. The PR gate verifies the candidate against the prospective merge and original rc artifact; finalize repeats that check before creating the write-once stable tag at the merge commit, then retags the verified packages artifact and rc's already-built, e2e-passed container digests to the stable tag **by digest** (no container rebuild — see [`hack/promote-retag.sh`](../hack/promote-retag.sh)) and publishes the release. An abandoned promotion leaves no stable-named bytes; its temporary candidate is retention-managed.
|
||||
|
||||
So a commit on a supported `release-X.Y` line ships when a maintainer promotes the next rc for that line — not automatically within 24h. Pushing a stable `vX.Y.Z` tag by hand is **not** a supported path: `tags.yaml` fails fast on a stable tag that has no pre-existing draft ("stable tags come from promote-rc.yaml"), and even so the finalize step would refuse to move a pre-existing tag. Pre-releases (`vX.Y.Z-rc.N`, `-alpha.N`, `-beta.N`) are cut with the [`Cut Pre-release Tag`](../.github/workflows/cut-prerelease.yaml) workflow, which pushes the tag as the CI app; [`tags.yaml`](../.github/workflows/tags.yaml) then fires on that push, builds it, and publishes the pre-release. Cutting via the workflow (rather than a manual `git push`) is what lets repo admins lock `v*` tag creation to the CI app — see the tag-protection note below.
|
||||
|
||||
## Release signatures and SBOM
|
||||
|
||||
A release cut from a tree that has [`hack/sign-release-assets.sh`](../hack/sign-release-assets.sh) carries three files beside its other assets, which [`hack/upload-assets.sh`](../hack/upload-assets.sh) has that script write before it uploads anything. Releases cut before it have none of them, and neither does a maintenance line until the script is backported to it (see below).
|
||||
|
||||
- `cozystack-source-sbom.spdx.json` is an SPDX SBOM, made with syft, of the committed source tree (the Go modules, npm lockfiles, GitHub Actions references and other manifests syft reads in it) and of the `cozypkg` binaries in the `cozypkg-*.tar.gz` assets. It does not describe what runs in the cluster: the container images carry their own SBOMs, see [SBOM of the container images](#sbom-of-the-container-images), and the Talos installer image, ISO, disk, kernel and initramfs images have none. Its name says what it covers, so that `cozystack-sbom` stays free for an SBOM of the release as a whole.
|
||||
- `cozystack-checksums.txt` holds the SHA-256 of every other asset and of the SBOM.
|
||||
- `cozystack-checksums.txt.sigstore.json` is a Sigstore bundle with a keyless cosign signature over the checksums. Its certificate names the workflow that published the release: `tags.yaml` for a pre-release, `promote-rc.yaml` for a stable one.
|
||||
|
||||
One signature covers every asset through the checksums. To check a download, verify the checksums, then the file against them, with cosign v3.1.3 or later (v2.6.5 or later on the v2 line):
|
||||
|
||||
```sh
|
||||
cosign verify-blob cozystack-checksums.txt \
|
||||
--bundle cozystack-checksums.txt.sigstore.json \
|
||||
--certificate-identity-regexp '^https://github\.com/cozystack/cozystack/\.github/workflows/(tags\.yaml@refs/tags/v|promote-rc\.yaml@refs/heads/main$)' \
|
||||
--certificate-oidc-issuer https://token.actions.githubusercontent.com &&
|
||||
sha256sum --check --ignore-missing cozystack-checksums.txt
|
||||
```
|
||||
|
||||
Older cosign accepts a legacy-format bundle that carries a bare public key in place of a certificate and skips the identity and issuer checks for it ([GHSA-fx35-mq7g-6g98](https://github.com/sigstore/cosign/security/advisories/GHSA-fx35-mq7g-6g98)), so a bundle swapped in next to the download, by a mirror or a proxy, passes there. The bundles this repository writes are in the new format, but cosign picks the format from the file, and `--new-bundle-format` does not stop that on the v3 line (v3.1.2 still accepts such a bundle with it), so the cosign version is what protects the check.
|
||||
|
||||
The regexp pins the ref each workflow runs from: `tags.yaml` on a `v*` tag push, and `promote-rc.yaml` dispatched from `main`, the only branch it is dispatched from. A passing check proves that the checksums were signed by one of those runs in `cozystack/cozystack`. A `promote-rc.yaml` dispatched from another branch fails this check in its own upload step, with no asset uploaded, and an edited `promote-rc.yaml` has to be merged into `main` before it can sign. Until the `v*` [tag rulesets](#repo-side-enforcement-tag-protection-rulesets) restrict tag creation to the CI app, though, anyone with write access can push a `v*` tag whose tree carries an edited `tags.yaml`, and its signature passes.
|
||||
|
||||
The script runs the same `cosign verify-blob` before it returns, so a signature this check would reject fails the job before the first upload, and the release is not published with assets but without a signature. Both jobs that call `make upload_assets`, `tags.yaml::prepare-release` and `promote-rc.yaml::promote`, hold `id-token: write` for the signing identity and install syft and cosign v3.1.3, the version users are asked for above. They install syft from its release tarball, checked against a SHA-256 committed in the workflow, and not with `anchore/sbom-action/download-syft`, which runs an install script fetched at run time in a job that holds the release credentials; [`hack/ci-tool-installs-contract.bats`](../hack/ci-tool-installs-contract.bats) keeps it that way. A stable release gets its own three files rather than its rc's: promotion restamps several assets, so the checksums are recomputed and signed again.
|
||||
|
||||
`promote-rc.yaml` runs the `hack/upload-assets.sh` of the rc tree, not the dispatch ref's. A stable release is therefore signed only when its rc was tagged from a tree that has `hack/sign-release-assets.sh`, and a maintenance line starts carrying signatures once that script is backported to its `release-X.Y` branch.
|
||||
|
||||
### SBOM of the container images
|
||||
|
||||
The container images a release pins by digest each carry an SPDX SBOM of the packages in them, which BuildKit attaches at build time as an attestation (`SBOM=1` in `tags.yaml`, scanner and flags in [`hack/common-envs.mk`](../hack/common-envs.mk)). The attestation is an entry of the image's index, so the digest the release pins names both, and promotion copies it with the image. Read it with `docker buildx imagetools inspect <image>@<digest> --format '{{ json .SBOM }}'`, which prints one document per platform. The attestations are not signed. A release cut before `SBOM=1` was set has none, and neither has the Talos installer image, which the imager builds and `skopeo` pushes, nor the third-party images `packages/` vendors by digest.
|
||||
|
||||
## Nightly builds
|
||||
|
||||
A nightly is an **installable copy of `main` on GHCR — not a rebuild and not a release**. [`build-main.yaml`](../.github/workflows/build-main.yaml) already builds every push to `main` into the CI registry (OCIR); the nightly promotes that build to the public release registry (GHCR) and proves it installs.
|
||||
|
||||
[`nightly.yaml`](../.github/workflows/nightly.yaml) runs daily (gated by the `NIGHTLY_ENABLED` repo variable) as three jobs; build-disk and e2e both start once mirror has finished:
|
||||
|
||||
1. **mirror** — resolve the exact commit `cozystack-packages:main` was built from, then [`hack/nightly-mirror.sh`](../hack/nightly-mirror.sh) copies every cozystack-owned component image OCIR→GHCR **by digest** (bit-for-bit, no rebuild) and re-publishes the rewritten `cozystack-packages` artifact and the `cozy-installer` chart to GHCR, tagged `0.0.0-nightly.<YYYYMMDD>.<run-id>.<attempt>` plus a floating `nightly`.
|
||||
2. **build-disk** — assemble the Talos `nocloud` disk from the upstream siderolabs imager (the profile references only `ghcr.io/siderolabs/*`, so this needs no cozystack image and no rebuild) and publish it as a GHCR OCI artifact (`cozystack-nocloud`) so a nightly is installable on real hardware. The nightly e2e does not boot it.
|
||||
3. **e2e** — stage the published GHCR closure (rewritten tree + the `cozy-installer` chart pinned to the GHCR packages artifact) and run the **full** app suite on Talos containers, which do not boot the disk. A nightly has no diff, so Test Impact Analysis does not apply.
|
||||
|
||||
No GitHub release and no `api/apps/v1alpha1/*` Go-module tag are created for a nightly. [`retention.yaml`](../.github/workflows/retention.yaml) prunes old GHCR nightly versions (keeps the newest 14 per package the current nightly publishes, named from the published `cozystack-packages:nightly` tree by [`hack/retention-packages.sh`](../hack/retention-packages.sh), except packages linked to another repository, which are skipped with a warning; the floating `nightly`, release tags and untagged versions are never touched) and abandoned packages candidates. A candidate version is eligible only when all of these hold: it carries at least one tag and **every** tag on that manifest matches the strict `promotion-vX.Y.Z-from-vX.Y.Z-rc.N-run-<run-id>-<attempt>` form, so a manifest that has been retagged stable is excluded; it was last updated more than 30 days ago; its digest is not pinned by `packages/core/installer/values.yaml` at the head of an **open** pull request that is same-repository, authored by `cozystack-ci[bot]`, on a `release-X.Y.Z` branch and carrying the `release` label; and its digest is not pinned by that same file at the tip of `main` or of any `release-X.Y` branch — which is what keeps a candidate safe between the promote PR merging and finalize retagging it. Any such PR head whose pin is not exactly one `digest=sha256:<64 hex>` stops the candidate sweep for that run and fails the job, without affecting the nightly sweep.
|
||||
|
||||
Install a nightly:
|
||||
|
||||
```bash
|
||||
helm upgrade --install cozystack \
|
||||
oci://ghcr.io/cozystack/cozystack/cozy-installer --version 0.0.0-nightly.20260626.15992304129.1
|
||||
# ...or --version nightly for the latest
|
||||
```
|
||||
|
||||
The matching Talos node image is `ghcr.io/cozystack/cozystack/cozystack-nocloud:0.0.0-nightly.20260626.15992304129.1` (pull with `oras`).
|
||||
|
||||
## Backports
|
||||
|
||||
### The backport bot
|
||||
|
||||
[`backport.yaml`](../.github/workflows/backport.yaml) wraps [`korthout/backport-action`](https://github.com/korthout/backport-action) and fires on a merged `main`-targeted PR carrying one of:
|
||||
|
||||
| Label | Target branch |
|
||||
|-------|---------------|
|
||||
| `kind/backport` | the newest existing `release-X.Y` branch |
|
||||
| `kind/backport-previous` | the second-newest existing `release-X.Y` branch |
|
||||
|
||||
Resolution is dynamic at run time, and it reads the branches themselves: the job lists the repository's branches, keeps the ones matching `release-<major>.<minor>`, and sorts them numerically descending — so `release-1.10` ranks above `release-1.9`, which a lexicographic sort gets backwards. `backport` takes the first, `backport-previous` the second, so both name a branch that exists by construction; asking for `backport-previous` when only one line exists fails the job rather than inventing a target. Nothing is derived arithmetically — an earlier version computed the previous line as `Y-1`, which named a non-existent branch whenever a minor was skipped.
|
||||
|
||||
**During a freeze window the newest line is the one being stabilised, not the last published one.** `release-X.Y` is created when the first `vX.Y.0-rc.1` is cut, before `vX.Y.0` is published, so from the freeze until the release `backport` targets the release being stabilised and `backport-previous` targets the last published stable. That is the intent: the frozen branch is the only way into the upcoming release, which is when backports matter most. The trade-off is that for the length of the window the line one step further back has no label pointing at it, so a fix that has to reach it before the new release publishes must be cherry-picked by hand.
|
||||
|
||||
The bot creates a backport PR with title `[Backport release-X.Y] <original title>`. When this PR merges, the title prefix used to re-trigger the bot through `pr-labeler.yaml`, which auto-applied `backport` to any `[Backport release-X.Y]`-titled PR. Combined with the org-level `dosubot` re-applying the label, this caused recursive backports.
|
||||
|
||||
The fix (PR #2584): both job `if:` blocks gate on `github.event.pull_request.base.ref == 'main'`. Backport PRs target `release-X.Y`, so they cannot satisfy this — architectural protection, regardless of which bot relabels them.
|
||||
|
||||
### When the bot fails
|
||||
|
||||
If the run triggered by the merge dies without opening a backport PR, look for a run of this workflow that has not finished before retrying — `gh run list --workflow backport.yaml --branch <the PR's head branch>` — and clear it, because the retry queues behind it rather than replacing it. Do not filter by status: a run whose jobs are waiting for a runner, or which is held behind the concurrency group, is not `in_progress`, and filtering it out is how you conclude nothing is running. `--branch` narrows the list but does not identify the PR, since head branch names are not unique across forks and `gh run list` has no PR filter, so confirm which PR a run belongs to before cancelling it or you cancel someone else's backport. If cancelling does not clear the run it holds the group until GitHub retires it and the retry keeps queueing; that case has not been exercised here. A run whose jobs have all finished can still sit `in_progress`, which looks the same from the PR as a run that died. Then remove whichever backport label the PR already carries and add it back: a label event is the only other trigger, and it qualifies on the label it carries, so re-applying it is what starts a fresh attempt.
|
||||
|
||||
Conflicting cherry-picks produce a draft PR via `conflict_resolution: draft_commit_conflicts`. Look for the bot's comment with the merge-conflict diff. You either:
|
||||
|
||||
- Resolve in the draft branch and undraft, or
|
||||
- Drop the bot's branch and cherry-pick manually:
|
||||
|
||||
```bash
|
||||
git checkout release-X.Y
|
||||
git cherry-pick -x -s <commit-sha>
|
||||
# resolve conflicts
|
||||
git commit -s
|
||||
git push origin release-X.Y # or push to a new branch and open a PR
|
||||
```
|
||||
|
||||
To find the bot's failed comments across a batch of PRs:
|
||||
|
||||
```bash
|
||||
for n in $(gh pr list --search "label:kind/backport label:kind/backport-previous merged:>=2026-01-01" --json number --jq '.[].number'); do
|
||||
echo "=== #$n ==="
|
||||
gh pr view $n --json comments --jq '.comments[] | select(.author.login == "github-actions" or (.author.login | contains("backport"))) | .body' | head -20
|
||||
done
|
||||
```
|
||||
|
||||
Before [PR #3155](https://github.com/cozystack/cozystack/pull/3155), `conflict_resolution` was passed outside `experimental`, so a conflicting cherry-pick used the `fail` fallback and left only a "Backport failed" comment on the source PR. No draft backport PR was opened. When [`backport-audit`](../cmd/backport-audit/README.md) reports `MISSING`, check those comments and the workflow logs to establish why the change did not arrive.
|
||||
|
||||
### Cherry-pick triage before a patch
|
||||
|
||||
A patch release includes bugfixes for code that shipped in the corresponding minor `vX.Y.0`. Use:
|
||||
|
||||
```bash
|
||||
# 1. Audit what was labeled for this line and has NOT landed on it.
|
||||
# Exits non-zero when anything is outstanding; see cmd/backport-audit/README.md.
|
||||
go build ./cmd/backport-audit
|
||||
./backport-audit release-X.Y
|
||||
|
||||
# 2. List commits on main since the release branch diverged that are NOT yet on release-X.Y
|
||||
git merge-base origin/main origin/release-X.Y
|
||||
git log <base>..origin/main --grep="(#" --oneline
|
||||
|
||||
# 3. Open PRs that may need labeling before the cut
|
||||
gh pr list --state open --base main --label kind/bug
|
||||
```
|
||||
|
||||
Step 1 covers the currently labeled work: [`backport-audit`](../cmd/backport-audit/README.md) assigns each request to the release line selected at merge time, then checks whether its commits reached the branch. Incomplete delivery, open or dropped backports, missing changes and duplicate open claims fail the gate. Late-added labels can disagree with the bot's target, and branch-opening dates are estimated from rc tags; see the audit's documented limits before deciding what to backport. Steps 2 and 3 cover changes that were never labeled.
|
||||
|
||||
**Include rule:**
|
||||
|
||||
- `kind/bug` — especially destructive (data loss, crash-loop, OOM, eviction).
|
||||
- `area/security` / CVE.
|
||||
- Narrow chart fixes whose blast radius is contained (e.g. one operator's resources).
|
||||
- Patch-line-policy-matching dependency bumps (e.g. cilium 1.X.Y → 1.X.(Y+1) on a `release-A.B` shipping cilium 1.X — never a minor bump, only a patch within the line the branch already ships).
|
||||
|
||||
**Skip rule:**
|
||||
|
||||
- `kind/feature` — even with `lgtm`. A patch release is not a delivery vehicle for features.
|
||||
- CI-only changes (`area/ci`, test infrastructure).
|
||||
- Docs-only changes (those land via `update-website-docs` automatically).
|
||||
- Large multi-package dependency churn (Dependabot Go-deps bumps).
|
||||
- PRs whose conventional-commit type is `!` (breaking) or that mix `feat` with `fix`.
|
||||
|
||||
**Borderline cases:**
|
||||
|
||||
- Large refactors with `lgtm` AND an obvious bug payload — prefer extracting the bug fix into a narrower commit and backporting only that.
|
||||
- PRs labeled both `kind/bug` and `kind/feature` — split them.
|
||||
|
||||
### Backporting code that touches monotonic counters
|
||||
|
||||
Migrations are the canonical case. `packages/core/platform/images/migrations/run-migrations.sh` is a linear counter — it runs `seq $CURRENT_VERSION $((TARGET_VERSION - 1))` and stops if `CURRENT >= TARGET`. This creates a forward-upgrade trap. Walking through it with `release-1.2` as the maintenance branch:
|
||||
|
||||
- `main` has `targetVersion=10` and migrations 1–9 on disk.
|
||||
- The `release-1.2` branch was cut when `targetVersion=5` (migrations 1–4 on disk).
|
||||
- You backport a new migration to `release-1.2` at slot 5, and naively bump `targetVersion` 5 → 6 to enable it.
|
||||
- Customer running 1.2.x runs the new migration, stamps `cozystack-version=6` in-cluster.
|
||||
- On upgrade to 1.3 (main's `targetVersion=10`), the runner walks `seq 6 9` and runs main's migrations 6–9 — **main's own migration at slot 5 never runs.** The backport claimed that slot on `release-1.2`; main's slot 5 holds an unrelated migration that gets silently skipped.
|
||||
|
||||
Mitigations, in order of preference:
|
||||
|
||||
1. **Don't bump `targetVersion` in the backport.** Just ship the migration file. Prior precedent in the repo: an earlier maintenance branch's ACME backport did exactly this — the migration shipped but ran on no clusters until a later commit raised the counter. Pair this with relaxing the chart-side `{{ fail }}` guard on the maintenance branch (chart-side coalesce) so the new value works without the migration having executed.
|
||||
2. **If you must bump,** add an idempotent duplicate of the now-skipped main migration at a higher slot. Bounded cost (one extra file per affected migration) and survives the skip window.
|
||||
3. **Audit cross-branch counter state before deciding.** Compare migration numbering across `main`, `release-X.Y`, and `release-X.(Y-1)`. If a backport would create a skip window on any forward-upgrade path, choose option 1.
|
||||
|
||||
This rule generalizes to any monotonic-counter state: schema versions, feature-flag generations, anything that gates "have I run X yet?" with a single integer.
|
||||
|
||||
## Changelogs
|
||||
|
||||
### Where the canonical process lives
|
||||
|
||||
[`agents/changelog.md`](./agents/changelog.md) is the source of truth. Read it end-to-end before generating; do not infer the process from past commits or memory. The CI runs it under Copilot via the reusable `changelog-generate.yaml` — called by `promote-rc.yaml::changelog` at promote time and by the optional `changelog-rc.yaml` at rc time — and, as a backstop, `tags.yaml::generate-changelog`; locally the same prompt is replayable by following the doc directly.
|
||||
|
||||
### Common changelog failure modes
|
||||
|
||||
These are mistakes that have shipped in real changelogs. Verify against each before merging the changelog PR.
|
||||
|
||||
**1. Commits from outside the release range.** Caused by running `git log <prev>..HEAD` from `main` while generating a patch changelog — `HEAD` on `main` contains everything that landed since the tag plus backports merged to `release-X.Y` after the tag was cut. Use `git log <prev>..<new_tag>` explicitly. The doc was updated to require this after v1.3.1's changelog shipped with 14 entries for what was actually a 1-commit release.
|
||||
|
||||
**2. Original + backport listed as two separate entries.** Each backport must coalesce with its original into a single entry of the form `(in #orig, backport #bp)`. To verify after generation:
|
||||
|
||||
```bash
|
||||
# Every entry that mentions "backport" should also reference the original PR number
|
||||
grep -E 'backport #[0-9]+' docs/changelogs/v<new>.md | grep -v '#[0-9]*, backport'
|
||||
```
|
||||
|
||||
Output should be empty.
|
||||
|
||||
**3. Wrong PR author.** The squash-merge commit's author is whoever clicked "Merge" — not the person who wrote the code. **Always** resolve via `gh pr view <N> --json author --jq .author.login`, never `git log --format=%an`. This bites hardest for website-repo entries where the same merger handles most PRs.
|
||||
|
||||
**4. Superseding patch, stale changelog.** Tags are write-once, so a critical fix after `vX.Y.Z` ships as the next patch `vX.Y.(Z+1)` rather than moving the tag. If a changelog PR already exists for a tag you are superseding, the changelog work transfers to the new patch. Compare `git log <prev>..<new>` against what the existing `changelog-vX.Y.Z` branch already documents; only add/remove the deltas. If the only new commits are CI-internal or a revert of a feature that never reached a stable tag, no changelog edit is needed.
|
||||
|
||||
**5. PR numbers swapped inside prose.** The entry-format validator checks bullet entries but ignores Feature Highlights paragraphs and Upgrade Notes. Both have caused wrong PR references in shipped changelogs. Verify every `#NNNN` in prose with `gh pr view <N>`.
|
||||
|
||||
**6. Hallucinated entries.** Every entry must correspond to a commit in `git log <prev>..<new>`. Verify by extracting PR numbers from the file and grep-searching the commit range:
|
||||
|
||||
```bash
|
||||
grep -oE '#[0-9]+' docs/changelogs/v<new>.md | sort -u | while read pr; do
|
||||
n=${pr#\#}
|
||||
if ! git log <prev>..<new> --grep="#${n}" --oneline | grep -q .; then
|
||||
echo "PR $pr in changelog but not in commit range"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
**7. Bot accounts as human contributors.** `cozystack-ci[bot]`, `github-actions`, `dependabot`, `renovate`, and any `app/*` login must NOT appear in the `## Contributors` list. They legitimately appear in **per-entry attribution** (e.g. a Renovate-authored PR) — but not in the human roll-call.
|
||||
|
||||
**8. Title duplicated as description.** `* **fix(foo): X**: fix(foo): X (...)` means you wrote the conventional-commit subject twice and never wrote a user-facing description. The brief and the detail must say different things — the detail explains what the change means for users.
|
||||
|
||||
### Changelog pre-publish verification
|
||||
|
||||
Run these checks before merging the changelog PR. Catches problems in roughly the order they occur:
|
||||
|
||||
```bash
|
||||
PREV=v1.2.0; NEW=v1.2.1
|
||||
|
||||
# A. Every chart bump is mentioned somewhere in the changelog
|
||||
git diff --name-only $PREV..$NEW -- 'packages/*/charts/*/Chart.yaml' 'packages/*/*/charts/*/Chart.yaml' | while read f; do
|
||||
old=$(git show "$PREV:$f" 2>/dev/null | yq '.version // ""')
|
||||
new=$(git show "$NEW:$f" 2>/dev/null | yq '.version // ""')
|
||||
if [ "$old" != "$new" ]; then
|
||||
component=$(basename "$(dirname "$f")")
|
||||
if ! grep -qi "$component" docs/changelogs/$NEW.md; then
|
||||
echo "MISSING: $component $old -> $new ($f)"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# B. Out-of-Chart.yaml pins (Talos, tenant K8s, vendored images)
|
||||
for path in packages/core/talos/images/talos/profiles/installer.yaml packages/core/installer/values.yaml packages/apps/kubernetes/values.yaml; do
|
||||
git diff $PREV..$NEW -- "$path" | head -40
|
||||
done
|
||||
git diff --name-only $PREV..$NEW -- 'images/*/Dockerfile'
|
||||
|
||||
# C. Every cited #NNNN exists and is a merged PR (not an issue)
|
||||
grep -oE '#[0-9]+' docs/changelogs/$NEW.md | sort -u | sed 's/#//' | xargs -P 20 -I{} sh -c '
|
||||
state=$(gh api /repos/cozystack/cozystack/issues/{} --jq ".pull_request.merged_at // \"ISSUE\"" 2>/dev/null)
|
||||
echo "#{} $state"
|
||||
' | grep -E 'ISSUE|null|^#[0-9]+ $'
|
||||
|
||||
# D. URL liveness
|
||||
grep -oE 'https://[^)" ]+' docs/changelogs/$NEW.md | sort -u | xargs -P 10 -I{} sh -c '
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" {})
|
||||
if [ "$code" != "200" ]; then echo "$code {}"; fi
|
||||
'
|
||||
|
||||
# E. Website PR authors are PR authors, not commit authors
|
||||
grep -oE 'cozystack/website#[0-9]+' docs/changelogs/$NEW.md | sort -u | sed 's|cozystack/website#||' | while read n; do
|
||||
pr_author=$(gh pr view $n --repo cozystack/website --json author --jq .author.login)
|
||||
echo "website#$n -> @$pr_author"
|
||||
done
|
||||
|
||||
# F. Working tree clean except for the changelog
|
||||
git status --porcelain | grep -v "docs/changelogs/$NEW.md"
|
||||
# Should output nothing.
|
||||
```
|
||||
|
||||
Ship criterion: every chart bump surfaced, no issue-cited-as-PR, no 404 URLs, every website entry attributed via `gh pr view` (not commit author), `git status` shows only the changelog.
|
||||
|
||||
### Cozystack-specific URL quirks
|
||||
|
||||
`cozystack.io` paths are versioned: `/docs/operations/foo/` and `/docs/next/foo/` both 404. Only `/docs/vX.Y/foo/` and the unversioned canonical `/docs/foo/` serve content. AI-generated changelogs invent the unversioned form. Verify URLs with the `curl` loop above.
|
||||
|
||||
### Platform-component coverage
|
||||
|
||||
For a minor release (`vX.Y.0`), changelog entries should surface what an upstream chart bump actually brought in — features, breaking changes, security fixes. PR commit messages usually don't capture this; the upstream `CHANGELOG.md` or GitHub Release notes do. For each bumped `Chart.yaml`:
|
||||
|
||||
1. Read the `sources:` field — typically a GitHub URL.
|
||||
2. Fetch release notes strictly inside `old < x <= new` via `gh release view <tag> --repo <owner>/<repo> --json body,name`.
|
||||
3. Summarize in 2–5 user-impact bullets per component (new features, breaking changes, CVEs, important deprecations). Don't reproduce upstream notes verbatim; link to them.
|
||||
4. If a PR-level entry already mentions the bump, enrich that entry with bullets — don't duplicate the entry.
|
||||
5. If 3+ bumps have no PR-level mention, introduce a `## Platform Components` section.
|
||||
|
||||
If you can't find upstream notes for a bumped component, list it with version-only info and no bullets. Hallucinated bullets are worse than missing detail.
|
||||
|
||||
## Release-time fires — what's gone wrong recently
|
||||
|
||||
These are real regressions that escaped to users. The pattern in each: code passed CI, shipped in a stable release, broke customers. Read them as a working list of pre-release verification gaps.
|
||||
|
||||
### v1.3.0 — cert-manager ingressClassName regression (fix #2562)
|
||||
|
||||
**Bug:** A commit migrated ACME HTTP-01 to use the modern `acme.cert-manager.io/http01-ingress-ingressclassname` annotation. The matching ingress-shim code that reads that annotation was added by cert-manager upstream PR #8244 and was **never backported** to the `release-1.19` branch — but cozystack 1.3.0 shipped cert-manager v1.19.3. The annotation was silently dropped at runtime and every solver Ingress fell back to the ClusterIssuer's default class `tenant-root`. Let's Encrypt validation broke for every tenant whose `publishing.ingressName != tenant-root`.
|
||||
|
||||
**Blast radius:** v1.3.0, v1.3.1, and v1.1.7 (the migration was cherry-picked to `release-1.1`).
|
||||
|
||||
**Why CI missed it:** the migration commit was self-contained and `helm template` rendered cleanly. There was no E2E that creates a tenant with a non-default ingress class and watches a real Let's Encrypt cert issue. Verification had been done from the docs side ("the API has been available since cert-manager 1.12"), not against the shipped cert-manager source.
|
||||
|
||||
**Would have caught it:** non-root-tenant E2E with LE-staging cert issuance, asserting the solver Ingress lands on the tenant's class.
|
||||
|
||||
### v1.4.0 — kubevirt-instancetypes upgrade failure (fix #2612)
|
||||
|
||||
**Bug:** `packages/system/kubevirt-instancetypes/Makefile` contained `sed -i '/persistent: true/d' templates/preferences.yaml`, which stripped the only child of `preferredTPM`, leaving `preferredTPM:` (null) in six Windows preferences. KubeVirt v1.6.x silently accepted it. The v1.6.3 → v1.8.2 operator bump (2026-04-27) hardened the CRD's OpenAPI schema and started rejecting null for an object-typed field. `helm upgrade` then failed on every cluster with `kubevirt-instancetypes` enabled.
|
||||
|
||||
**Why CI missed it:** the kubevirt-operator bump and the inert `preferences.yaml` were two weeks apart in commit history. CI only tests fresh installs against the new CRDs; it doesn't run `helm upgrade` from prior versions. Schema regression on a vendored asset that hadn't been touched in 18 months.
|
||||
|
||||
**Would have caught it:** `helm template <new chart> | kubectl apply --dry-run=server -f -` against the freshly-installed CRDs of every bumped operator. Or an upgrade-from-N-1 E2E lane.
|
||||
|
||||
### v1.4.0 — Flux 2.7 → 2.8 readiness deadlock (#2602)
|
||||
|
||||
**Bug:** helm-controller v1.5.0 (shipped in Flux 2.8) changes the default wait strategy to kstatus polling, which polls **every applied resource including child HelmReleases**. The `packages/apps/kubernetes` umbrella has 19 child HRs that each `dependsOn: parent` (with a `lookup` guard so the dependency only activates after the parent exists). Pre-2.8: parent went Ready as soon as Helm returned. Post-2.8: parent waits for children, children wait for parent. Deadlock.
|
||||
|
||||
**Why CI missed it:** the cycle only manifests on the second reconcile (parent exists, so the `lookup` activates the child's `dependsOn`). Fresh-install E2E dodges this — the parent-already-exists branch isn't taken on the first apply.
|
||||
|
||||
**Would have caught it:** apply-mutate-reapply E2E (any chart bump in the kubernetes umbrella triggers a second reconcile). Plus an explicit PR-template requirement: for any flux/helm-controller bump, paste the upstream breaking-changes section into the PR body.
|
||||
|
||||
### Patterns these share
|
||||
|
||||
1. **Fresh-install CI is insufficient.** All three regressions passed fresh-install lanes.
|
||||
2. **Upstream breaking-defaults bumps are the common shape.** kubevirt CRD hardening, helm-controller default wait strategy, cert-manager API surface drift. Pre-merge reading of upstream changelogs would have caught two of three.
|
||||
3. **One-engineer-bitten-once is the only safety net for live verification.** The cert-manager bump PR included an explicit live-cluster verification step — because the same engineer had been bitten the prior week. There is no checklist requiring it.
|
||||
|
||||
## Pre-release verification checklist
|
||||
|
||||
For RCs and final releases, run this before merging the release PR. Each item is here because something has shipped without it and broken users.
|
||||
|
||||
- [ ] **The published rc passed mandatory full E2E.** Confirm the promote PR says `✅ RC full e2e was verified`; the normal promote PR itself runs no E2E. Add `full-e2e` manually only when a repeat is required, and treat a ⚠️ bypass line as an explicit release-risk decision.
|
||||
- [ ] **Upgrade from previous patch** (`vX.Y.(Z-1)`) on a real cluster. `helm upgrade` succeeds, all HelmReleases reach Ready, no CRD schema rejections.
|
||||
- [ ] **Upgrade from previous minor** (`v(X-1).Y.0`) for RC and minor releases. Same assertions.
|
||||
- [ ] **At least one tenant with `publishing.ingressName != tenant-root`** and a real Let's Encrypt-staging cert issuance.
|
||||
- [ ] **Apply-mutate-reapply** on the kubernetes umbrella HelmRelease (catches second-reconcile bugs).
|
||||
- [ ] **For every CRD-source bump** (kubevirt, cert-manager, flux, cilium, kamaji): `helm template | kubectl apply --dry-run=server` against the freshly-installed CRDs.
|
||||
- [ ] **For every flux / helm-controller / kubevirt-operator / cert-manager bump:** paste the upstream breaking-changes section of the bump's CHANGELOG into the PR body before merge.
|
||||
- [ ] **After cutting the tag,** watch `pull-requests-release.yaml::Finalize Release`. If it fails on `Draft release for v... not found`, undraft the release manually and file the workflow regression — this has been a known recurring failure since the `Publish draft release` step lost its explicit `github-token` in refactor `66a756b6`.
|
||||
- [ ] **The promote PR carries `docs/changelogs/vX.Y.Z.md` and it was verified** per [Changelog pre-publish verification](#changelog-pre-publish-verification). Finalize publishes this file as the release body, so verifying it before merge is verifying the release notes.
|
||||
- [ ] **The `cozystack/website` docs PR stays parked until publish.** Promotion opens it early; merging it before the GitHub release for `vX.Y.Z` is live 404s the new version's docs and API links. Merge it right after finalize publishes the release (regular-release step 6, patch step 7).
|
||||
|
||||
## CI failures release engineers commonly hit
|
||||
|
||||
| Symptom | Class | Diagnosis | Block release? |
|
||||
|---------|-------|-----------|----------------|
|
||||
| `Draft release for vX.Y.Z not found` on every release-* merge | Workflow regression — `Publish draft release` lost its `github-token` in a past refactor; default `GITHUB_TOKEN` cannot list drafts | At `pull-requests-release.yaml:163`. Undraft manually, file the fix | No |
|
||||
| Changelog Copilot job returns 402 | Token quota — `COPILOT_GITHUB_TOKEN` premium-request quota empty | Both changelog entry points hit it: the optional rc-time `changelog-rc` dispatch and the promote-time `changelog` job (both call the reusable `changelog-generate.yaml`), plus the `tags.yaml` backstop | No — `continue-on-error: true` on the AI step, generate changelog by hand following [`agents/changelog.md`](./agents/changelog.md) |
|
||||
| `Prepare Release` skipped and the run red, `Build arm64 images` failed | The rc's arm64 build failed | Read the `build-arm64` log; `build-arm64-nightly.yaml` usually failed the same way first. Fix on the line, cut the next rc | Yes — an rc is not published amd64 only |
|
||||
| `Stitch multi-arch indexes and republish` fails listing refs | `verify-multiarch.sh` found refs that are not amd64+arm64 indexes, or that it could not check | Each listed ref names its reason. Single-platform manifest or index without one architecture: make the image multi-arch, or allowlist its repository with a reason if it cannot be. Manifest unreadable: correct the ref or digest; the allowlist does not excuse it. Rate-limited or unavailable after 3 tries: re-run once the throttle clears | Yes |
|
||||
| E2E `kubernetes-test` fails with `CSINode does not contain driver csi.kubevirt.io` | Test ordering bug | Tenant's `kubernetes-${test_name}-csi` HelmRelease still installing when NFS PVC is created. Fix: wait for `csinode/<node>` to advertise the driver before creating the PVC | Yes — flaky and masks real CSI bugs |
|
||||
| `tenant-root` HR in perpetual upgrade-rollback loop after OIDC patch | Real bug at the intersection of #2602's readiness change and the cozystack reconcile cadence | Look for `rate: Wait(n=1) would exceed context deadline` in operator logs; multiple tenant child HRs stuck `InProgress` | Yes |
|
||||
| `cozy-dashboard` ImagePullBackOff from `cozystack-ui:latest` | Registry flake | Known noisy on some OCI mirrors; not currently in the prepull set | No — retry |
|
||||
| PR CI: `docker push` 409 on `:latest` for the same image across two PRs | Concurrent manifest race on a shared floating tag | The `:latest` tag was previously written by every `make image` regardless of PR. Resolved by the image-tag refactor (`IMAGE_TAG=pr-<N>-<sha>` in PR builds, `IMAGE_TAG=<ref_name>` only at release time) | No — `gh run rerun --failed` |
|
||||
|
||||
**Heuristic:** workflow-token and Copilot-quota issues are out-of-band — release the tag, fix the workflow in a follow-up PR. Test ordering bugs and tenant-root reconcile storms are real and should block.
|
||||
|
||||
If you find yourself doing the same manual fixup on two consecutive releases (e.g. "undraft the release"), open a workflow-regression issue. Workflow bugs with known manual workarounds rot silently for months.
|
||||
|
||||
## Tag immutability
|
||||
|
||||
Published tags are **write-once** — once a `vX.Y.Z` or rc tag is pushed it is never moved or deleted. (Nightlies are not git tags at all — they are GHCR OCI tags, see [Nightly builds](#nightly-builds).) Moving a tag silently poisons the Go module proxy / pkg.go.dev cache for `api/apps/v1alpha1/vX.Y.Z` and drifts SBOM/provenance toolchains, so the release flow is built so a move is impossible by construction:
|
||||
|
||||
| File | Tag / branch handling |
|
||||
|------|------------------------|
|
||||
| [`pull-requests-release.yaml`](../.github/workflows/pull-requests-release.yaml) | Verifies the pinned packages candidate before creating a stable name. Creates `vX.Y.Z` at the PR merge commit **write-once** (create if absent, no-op if unchanged, fail loudly if it would move — the merge commit is new, so there is nothing to move). Cuts the `api/apps/v1alpha1/<vTAG>` Go-submodule tag write-once at the same commit, **stable only** (never rc/beta/alpha). The `release-X.Y` maintenance branch is fast-forward-only. Retags the verified packages artifact and rc container digests to the stable image tag (also write-once at the image level). |
|
||||
| [`tags.yaml`](../.github/workflows/tags.yaml) | The `release-X.Y.Z-rc.N` staging branch is a mutable staging ref (compare-before-force: skipped when unchanged, force+log only when it genuinely moves). Fails fast on a stable `vX.Y.Z` tag with no pre-existing draft — stable tags come from `promote-rc.yaml`, never a hand push. |
|
||||
| [`promote-rc.yaml`](../.github/workflows/promote-rc.yaml) | Mutates no stable tags. Before staging, verifies green full E2E from the latest rc tag-push run or a tag-correlated manual published-tag run (unless loudly bypassed). Commits and publishes the rewritten packages tree under a run-specific temporary `promotion-*` tag, adds a second commit pinning its digest on the `release-X.Y.Z` branch, generates the changelog there, and opens the promote PR with `release` but no automatic `full-e2e`; the PR gate verifies the candidate and finalize repeats that verification before the stable tag and image retag. The staging branch is force-pushed on re-dispatch, but an existing `docs/changelogs/vX.Y.Z.md` is preserved across the rebuild. |
|
||||
|
||||
This is the immutable-tag + rc-promotion model from [#2677](https://github.com/cozystack/cozystack/issues/2677): stable `vX.Y.Z` is created only by promoting an existing rc, rc tags are write-once, and `api/apps/v1alpha1/vX.Y.Z` is created only on a stable release. The old nightly `auto-release.yaml` (which delete-recreated auto-bumped patch tags) has been removed. rc and stable tags accrete permanently; the only churning artifacts are GHCR nightlies and abandoned temporary promotion candidates, which [`retention.yaml`](../.github/workflows/retention.yaml) prunes (see [Nightly builds](#nightly-builds)) — GHCR storage is the cost vector to watch.
|
||||
|
||||
### Repo-side enforcement (tag-protection rulesets)
|
||||
|
||||
The guarantees above are enforced in the workflows. Repo admins back them with two GitHub **tag rulesets** targeting `v*`:
|
||||
|
||||
- **Immutability** — *Restrict updates* + *Restrict deletions*, **no bypass**. No actor (not even the CI app) can move or delete a shipped `v*` tag. Safe to enable at any time: nothing in the release flow ever moves or deletes a `v*` tag, it only creates new ones.
|
||||
- **Creation control** — *Restrict creations* with the **CI app as the sole bypass actor**, plus *Limit branches/tags updated in a single push* = 1. This turns "don't hand-push tags" from convention into hard enforcement: every `v*` tag — pre-release **and** stable — can then be created only by CI. Enable this **only after** [`cut-prerelease.yaml`](../.github/workflows/cut-prerelease.yaml) is in use, since that workflow is what makes CI the sole creator of pre-release tags; enabling it earlier would lock maintainers out of cutting rc/alpha/beta. (The Go-module tag `api/apps/v1alpha1/*` gets an equivalent pair of rulesets.)
|
||||
|
||||
## Splitting a release-blocking bundle PR
|
||||
|
||||
Sometimes the work that has to land before a release is a 40-commit grab bag (CI stabilization, dependency bumps, chart fixes). Splitting it makes it reviewable and survivable. The strategy that has worked:
|
||||
|
||||
1. **Verify what's already on `main`.** Use `git merge-base --is-ancestor` and subject grep to drop commits that have already landed.
|
||||
2. **Find existing open PRs that cover the same commits.** Force-push to the existing branch instead of opening a duplicate — preserves review threads.
|
||||
3. **Identify the "bottleneck" PR.** The one whose changes make previously-silent misconfigurations into hard errors (typically an operator-version bump). Everything else can fan out in parallel.
|
||||
4. **Squash iterative same-author bug-on-bug fixes** inside each split. Preserve authorship via `--author` on the squashed commit.
|
||||
5. **Mark the keystone PR as draft** and link in the body to the dependency PRs. Reviewers know it intentionally fails CI until the chain lands.
|
||||
6. **Beware of concurrent registry pushes.** When fanning out 5+ PR CI runs simultaneously, multiple `make image` jobs can collide on shared floating tags. The `IMAGE_TAG=pr-<N>-<sha>` refactor solved this; if you see `Conflicted with another upload of the same manifest`, `gh run rerun --failed`.
|
||||
|
||||
## Cleanup after release
|
||||
|
||||
- The `release-X.Y.Z` branch is deleted by GitHub when its PR merges. The `release-X.Y` maintenance branch is created by `cut-prerelease.yaml` at the rc freeze (create-only, never force-moved) and fast-forwarded afterwards by `pull-requests-release.yaml::Ensure maintenance branch`.
|
||||
- The draft release is published by the same workflow.
|
||||
- The release body is set by finalize from `docs/changelogs/vX.Y.Z.md` at publication. Afterwards, `update-releasenotes.yaml` re-syncs that file into the GitHub Release body on any push to `main` that touches it — so edits to a published changelog land on the release page the next time `main` moves.
|
||||
- Local cleanup: remove your worktree (`git worktree remove`) and prune merged release branches.
|
||||
## Pages
|
||||
|
||||
- [Introduction](./release/introduction.md): the three release types and the tag push that drives the pipeline.
|
||||
- [Release Candidates](./release/release-candidates.md): what an rc is, and how cutting the first one freezes `release-X.Y`.
|
||||
- [Regular Releases](./release/regular-releases.md): promoting an rc to `vX.Y.0`, step by step.
|
||||
- [Patch Releases](./release/patch-releases.md): the same flow for `vX.Y.Z` on an existing `release-X.Y` line.
|
||||
- [What CI does during the release process](./release/ci-pipeline.md): which workflow does what, phase by phase, including multi-arch builds and the rc E2E gate.
|
||||
- [Stable tags come from rc promotion](./release/stable-tags.md): why a stable tag only ever comes from promoting an rc.
|
||||
- [Release signatures and SBOM](./release/signatures-and-sbom.md): the source SBOM, checksums and cosign signature on each release, the SBOM each release image carries, and how to verify and read them.
|
||||
- [Nightly builds](./release/nightly-builds.md): how a nightly mirrors `main` to GHCR, and how to install one.
|
||||
- [Backports](./release/backports.md): the policy that decides what goes into a patch release, and how a contributor gets a fix there.
|
||||
- [Backport mechanics](./release/backport-mechanics.md): cherry-pick triage before a patch, and the monotonic-counter trap.
|
||||
- [Backport tasks](./release/backport-tasks/README.md): resolving a batch conflict, deferring the original a failed batch CI comes from, and checking a pull request title, with the check each has to pass.
|
||||
- [Changelogs](./release/changelogs.md): changelog failure modes and the pre-publish checks.
|
||||
- [Release-time fires](./release/release-time-fires.md): regressions that escaped to users, and the verification gaps they expose.
|
||||
- [Pre-release verification checklist](./release/pre-release-checklist.md): what to verify before merging a release PR.
|
||||
- [CI failures release engineers commonly hit](./release/ci-failures.md): known failures and whether each one blocks a release.
|
||||
- [Tag immutability](./release/tag-immutability.md): write-once tags and the rulesets that enforce them.
|
||||
- [Splitting a release-blocking bundle PR](./release/splitting-bundle-prs.md): how to break up a grab-bag PR that blocks a release.
|
||||
- [Cleanup after release](./release/cleanup.md): what is left to clean up once a release is out.
|
||||
|
||||
## See also
|
||||
|
||||
- [`agents/changelog.md`](./agents/changelog.md) — canonical changelog generation process.
|
||||
- [`agents/contributing.md`](./agents/contributing.md) — commit/PR conventions, backport label semantics.
|
||||
- [`agents/contributing.md`](./agents/contributing.md) — commit/PR conventions.
|
||||
- [`agents/releasing.md`](./agents/releasing.md) — pointer file for AI agents handling release tasks.
|
||||
- [`.github/workflows/cut-prerelease.yaml`](../.github/workflows/cut-prerelease.yaml) — the sole entry point for cutting a pre-release tag as the CI app.
|
||||
- [`.github/workflows/tags.yaml`](../.github/workflows/tags.yaml) — tag-push pipeline (rc build + mandatory `rc-e2e`; stable-tag changelog/website backstops).
|
||||
@@ -714,5 +53,6 @@ Sometimes the work that has to land before a release is a 40-commit grab bag (CI
|
||||
- [`agents/image-refs.md`](./agents/image-refs.md) — where image references live, what their tags mean, and the invariants promotion and mirroring rely on.
|
||||
- [`hack/lib/image-refs.sh`](../hack/lib/image-refs.sh) — the single enumeration of ref-bearing files, shared by the promote, retag and mirror tooling.
|
||||
- [`.github/workflows/retention.yaml`](../.github/workflows/retention.yaml) — GHCR nightly and abandoned promotion-candidate pruning.
|
||||
- [`.github/workflows/backport.yaml`](../.github/workflows/backport.yaml) — automatic cherry-pick bot.
|
||||
- [`cmd/backport-batch`](../cmd/backport-batch/README.md) — the backport batch that applies the [backport policy](./release/backports.md).
|
||||
- [`.github/workflows/backport-batch.yaml`](../.github/workflows/backport-batch.yaml) — runs the backport batch on Monday and Thursday for each line it runs for: the newest released lines, and a line in its rc window.
|
||||
- [`.github/workflows/update-releasenotes.yaml`](../.github/workflows/update-releasenotes.yaml) — sync changelog → GitHub Release body.
|
||||
@@ -0,0 +1,37 @@
|
||||
## Backport mechanics
|
||||
|
||||
Under the [backport policy](./backports.md) fixes reach a release line through the batch, and nobody labels pull requests for backport. This page covers the mechanics around it: cherry-pick triage before a patch, and the trap that keeps migrations out of patch releases.
|
||||
|
||||
### Cherry-pick triage before a patch
|
||||
|
||||
A patch release includes bugfixes for code that shipped in the corresponding minor `vX.Y.0`. What goes in is decided by the rules in [Backports](./backports.md), not per pull request, and each batch pull request says what it picked, what it deferred and what it skipped under which rule. What is left to check before a cut:
|
||||
|
||||
```bash
|
||||
# Exits 1 and prints the URLs while a pull request the rules take is not on
|
||||
# the line, including one only on the open batch PR, which the cut would not
|
||||
# carry; see cmd/backport-batch/README.md.
|
||||
go build ./cmd/backport-batch
|
||||
./backport-batch check --line release-X.Y --cut
|
||||
```
|
||||
|
||||
[`backport-batch check`](../../cmd/backport-batch/README.md) applies the rules to every pull request merged into `main` since the line branched, and looks for each one they take on the line and then on the open batch pull request: by picks whose `-x` lines name its merge commit or every commit of its own, by a pick that would change nothing, and, on commits from before the policy's batch-only day, by what their text says, such as `Backport of #N`. With `--cut` a change found only on the open batch pull request fails the check too, since a cut tags the line and would not carry it: merge the batch pull request first. A conflict the batch deferred, or an original deferred to the next batch, prints as a warning without failing the check, also once that batch merged or closed. An original that waits on the open batch for a maintainer fails the cut. A change that reached the line by hand counts like any other, by what reached the line.
|
||||
|
||||
### Backporting code that touches monotonic counters
|
||||
|
||||
The batch never takes a change like this: rule 4 in [Backports](./backports.md) holds migrations, `targetVersion`, sizeable API changes and CRDs for the next minor. What follows is why, and what a maintainer has to do when a migration must reach a release line anyway.
|
||||
|
||||
Migrations are the canonical case. `packages/core/platform/images/migrations/run-migrations.sh` is a linear counter — it runs `seq $CURRENT_VERSION $((TARGET_VERSION - 1))` and stops if `CURRENT >= TARGET`. This creates a forward-upgrade trap. Walking through it with `release-1.2` as the maintenance branch:
|
||||
|
||||
- `main` has `targetVersion=10` and migrations 1–9 on disk.
|
||||
- The `release-1.2` branch was cut when `targetVersion=5` (migrations 1–4 on disk).
|
||||
- You backport a new migration to `release-1.2` at slot 5, and naively bump `targetVersion` 5 → 6 to enable it.
|
||||
- Customer running 1.2.x runs the new migration, stamps `cozystack-version=6` in-cluster.
|
||||
- On upgrade to 1.3 (main's `targetVersion=10`), the runner walks `seq 6 9` and runs main's migrations 6–9 — **main's own migration at slot 5 never runs.** The backport claimed that slot on `release-1.2`; main's slot 5 holds an unrelated migration that gets silently skipped.
|
||||
|
||||
Mitigations, in order of preference:
|
||||
|
||||
1. **Don't bump `targetVersion` in the backport.** Just ship the migration file. Prior precedent in the repo: an earlier maintenance branch's ACME backport did exactly this — the migration shipped but ran on no clusters until a later commit raised the counter. Pair this with relaxing the chart-side `{{ fail }}` guard on the maintenance branch (chart-side coalesce) so the new value works without the migration having executed.
|
||||
2. **If you must bump,** add an idempotent duplicate of the now-skipped main migration at a higher slot. Bounded cost (one extra file per affected migration) and survives the skip window.
|
||||
3. **Audit cross-branch counter state before deciding.** Compare migration numbering across `main`, `release-X.Y`, and `release-X.(Y-1)`. If a backport would create a skip window on any forward-upgrade path, choose option 1.
|
||||
|
||||
This rule generalizes to any monotonic-counter state: schema versions, feature-flag generations, anything that gates "have I run X yet?" with a single integer.
|
||||
@@ -0,0 +1,29 @@
|
||||
## Backport tasks
|
||||
|
||||
The [backport batch](../backports.md) is built by [`backport-batch`](../../../cmd/backport-batch/README.md), and two steps around it need a maintainer's judgement. Each is a task with fixed inputs, one allowed output and a check in code that the result passes before anything is pushed. Both wait on the batch pull request: the batch's comment lists the deferred conflicts, and a failed batch CI shows on its checks.
|
||||
|
||||
| Task | Input | Output | Check |
|
||||
|---|---|---|---|
|
||||
| [Resolve a conflict](./resolve-conflict.md) | a deferred conflict on the batch pull request | the original's commits picked under one merge on the batch branch, or one pick for a squash | `backport-batch fold`, and `backport-batch verify` in CI |
|
||||
| [Attribute a failed batch CI](./failed-ci.md) | the failing jobs of the batch pull request | the culprit deferred to the next batch | `backport-batch run --defer` |
|
||||
|
||||
### Rules for every task
|
||||
|
||||
- Push nothing but the branch of the open batch pull request, `backport-batch/<line>-<yyyymmdd>` in this repository, and only what the task's check accepted. Never push to `main` or a `release-*` branch, and never open a second batch pull request: the scheduled run continues the open one.
|
||||
- Change no file the original pull request does not change. A conflict that cannot be resolved inside those files is not resolved: the original stays deferred.
|
||||
- Never rewrite the batch branch by hand. A plain push to it is a fast-forward; only `backport-batch run --defer --push` replaces it, leased on the head it rebuilt.
|
||||
- Leave the batch pull request's title and body to the tool: it rewrites both on every run, and a hand edit is lost or, inside the `backport-batch` block, makes `run`, `check`, `fold` and `notify` refuse the body, and inside the `backport-batch-report` block makes `run --push` and `backport-batch report` refuse it. Leave the tool's report comments as they are too: each keeps one classification of the line, and an edited one makes `backport-batch report` refuse it.
|
||||
- When the task cannot be done within these rules, push nothing and leave a comment on the batch pull request saying why, so the next maintainer to look finds it. Nothing is lost by waiting: a red batch does not merge, and a deferred original goes to the next batch.
|
||||
|
||||
### Getting the tool
|
||||
|
||||
Build it from `main`, in a clean worktree of this repository whose `origin` is this repository and with `gh` authenticated as someone who can push to the batch branch:
|
||||
|
||||
```bash
|
||||
git fetch origin
|
||||
git worktree add --detach ../backport-task origin/main
|
||||
cd ../backport-task
|
||||
go build -o /tmp/backport-batch ./cmd/backport-batch
|
||||
```
|
||||
|
||||
Run it from that worktree. Every command but `notify` and `report` fetches `origin` first, and `run`, `check`, `fold` and `lines` read the policy from `origin/main`.
|
||||
@@ -0,0 +1,41 @@
|
||||
## Attribute a failed batch CI
|
||||
|
||||
CI on the batch pull request runs the line's build and tests against every original the batch carries at once. When a check fails, this task finds the original the failure comes from and takes it out of the batch, so the rest can merge; the original goes to the next batch with the reason recorded. Read the [rules for every task](./README.md#rules-for-every-task) first.
|
||||
|
||||
### Inputs
|
||||
|
||||
- The open batch pull request for the line and its failing checks: `gh pr checks <batch>`, and the log of each failing job, `gh run view <run id> --log-failed`.
|
||||
- The originals the batch carries, one unit each on its first-parent line: `git log --first-parent --reverse --format='%h %s' origin/<line>..origin/backport-batch/<line>-<yyyymmdd>`. A merged pull request's unit is a merge whose message is the original's merge message, naming its number, and whose `Backported-from:` trailer names its merge commit on `main`; its commits are `git log --reverse --format='%h %an %s' <unit>^1..<unit>^2`. A squash-merged one is one commit whose subject ends in its number. The files a unit changes are `git diff --name-only <unit>^1 <unit>`.
|
||||
- The line's own CI on its head, `gh run list --branch <line>`, to tell a failure the batch brought from one the line already had.
|
||||
|
||||
### Output
|
||||
|
||||
The batch rebuilt without the culprits, by one `backport-batch run --defer` with `--push`, which records each culprit and the reason in the batch pull request. Nothing else: no commit by hand, no revert, no edit to the batch branch.
|
||||
|
||||
### Steps
|
||||
|
||||
1. Read the failing jobs and name what fails: the job, the package or test, and the first error.
|
||||
2. When the same job fails the same way on the line's head, or the failure is the runner's (a timeout pulling an image, a lost runner, a registry error), the batch did not cause it: defer nothing, and comment on the batch pull request with the job and the line's run that shows the same failure.
|
||||
3. Otherwise pin it on originals by their files: the originals that change the failing package, chart or test, or what it builds from. Take the fewest that explain the failure; when no original's files reach the failure, or every original's do, the culprit cannot be pinned from files alone: read the failure further before deferring anything.
|
||||
4. Write the reason as one line naming the job and the failure, for example `e2e-apps: postgres never becomes Ready`.
|
||||
|
||||
### Check before pushing
|
||||
|
||||
```bash
|
||||
/tmp/backport-batch run --line <line> --defer N[,N] --reason "<job>: <failure>"
|
||||
```
|
||||
|
||||
It rebuilds the branch locally without those originals and prints the new body, exiting 0. It exits 2 and names the original when dropping one leaves a later commit that no longer applies; add that one to `--defer` only when its files reach the failure too, otherwise comment on the batch pull request naming it and stop. Then publish the same command with `--push`:
|
||||
|
||||
```bash
|
||||
/tmp/backport-batch run --line <line> --defer N[,N] --reason "<job>: <failure>" --push
|
||||
```
|
||||
|
||||
It replaces the batch branch only if nobody pushed onto it since the rebuild read it, and updates the pull request body. CI runs again on the new head; if it fails again on another original, repeat from step 1.
|
||||
|
||||
When the culprits are everything the batch carries, there is no branch left to push: the body printed without `--push` says the batch is closing, and `--push` records the deferrals in the batch pull request, closes it and deletes its branch, still only if nobody pushed onto it since. The next batch tries them again. If it stops part way, the next `run --push` on the line finishes the close, and says so.
|
||||
|
||||
### Never
|
||||
|
||||
- Defer an original the failure does not reach, to make the batch green by taking out more than the culprit.
|
||||
- Push the rebuilt branch any other way, or push to it by hand.
|
||||
@@ -0,0 +1,66 @@
|
||||
## Resolve a conflict
|
||||
|
||||
The batch carries a pull request merged into `main` as its own commits, each picked with `git cherry-pick -x`, under a merge commit that names the original; a squash-merged one as one pick. When any of an original's commits conflicts, nothing of it stays on the batch, and the batch pull request lists it under **Deferred conflicts** with the commit that failed and its files, and in its `backport-batch` block. This task ports one such original onto the batch by hand, in the same shape. Read the [rules for every task](./README.md#rules-for-every-task) first.
|
||||
|
||||
### Inputs
|
||||
|
||||
- The open batch pull request for the line, its branch `backport-batch/<line>-<yyyymmdd>` and its head commit, as GitHub shows them: `gh pr view <batch> --json headRefName,headRefOid`. The resolution lands on that batch, so without an open batch pull request there is nothing to resolve onto: wait for the next batch, which lists the conflict again, or port the change as a pull request of its own against the line, as [fixes that exist only on a release line](../backports.md#fixes-that-exist-only-on-a-release-line) describes, and drop the original from the line.
|
||||
- The original's number `N` from the **Deferred conflicts** list, and its merge commit on `main`: `gh pr view N --json mergeCommit --jq .mergeCommit.oid`. It has two parents when the pull request was merged, one when it was squashed.
|
||||
- For a merged pull request, its own commits, oldest first, without the merges that brought `main` into its branch: `git rev-list --reverse --topo-order --no-merges <merge commit>^1..<merge commit>^2`.
|
||||
- The files the original changes, `git diff --name-only <merge commit>^1 <merge commit>`, which are the only files the resolution may touch.
|
||||
|
||||
### Output
|
||||
|
||||
For a merged pull request, one merge commit on the batch head: its first parent is the head of the batch pull request, and its second parent ends a straight line of picks of the pull request's own commits, in their order, each made with `git cherry-pick -x` so it keeps its author, message and sign-offs and ends in the `(cherry picked from commit …)` line git appends. Its message is the original's merge message with a `Backported-from: <merge commit>` trailer of its own. For a squash-merged pull request, one pick of its commit. Either way only files the original changes, and nothing else: no extra commit, no fix-up of an earlier pick.
|
||||
|
||||
### Steps
|
||||
|
||||
```bash
|
||||
git fetch origin
|
||||
git switch --detach <batch head>
|
||||
base=$(git rev-parse HEAD)
|
||||
```
|
||||
|
||||
Pick the original's commits, oldest first; for a squash-merged pull request the one commit to pick is its merge commit on `main`:
|
||||
|
||||
```bash
|
||||
git -c core.hooksPath=/dev/null cherry-pick -x --signoff <commit>
|
||||
```
|
||||
|
||||
When a pick conflicts, resolve each conflicted file so that it does on the line what that commit does on `main`, keeping the line's own code wherever the commit does not change it, and go on with the next commit:
|
||||
|
||||
```bash
|
||||
git add <resolved files>
|
||||
git -c core.hooksPath=/dev/null cherry-pick --continue --no-edit
|
||||
```
|
||||
|
||||
When the original builds on something the line does not have, a helper, a value or another pull request, the resolution would have to bring that along, which this task does not allow: abort with `git cherry-pick --abort`, go back with `git switch --detach "$base"`, and comment on the batch pull request which change the original depends on; it stays deferred until that change is on the line. When that change will never reach the line, drop the original there instead, with `backport-batch run --line <line> --drop N --reason "<builds on …>" --push`, so later batches stop trying it. A commit whose change the line already has stops the pick as empty; skip it with `git cherry-pick --skip`.
|
||||
|
||||
For a merged pull request, join the picks to the batch head with the merge commit:
|
||||
|
||||
```bash
|
||||
tip=$(git rev-parse HEAD)
|
||||
git switch --detach "$base"
|
||||
{ git log -1 --format=%B <merge commit>; printf '\nBackported-from: %s\n' <merge commit>; } > /tmp/batch-merge-message
|
||||
git -c core.hooksPath=/dev/null merge --no-ff -F /tmp/batch-merge-message "$tip"
|
||||
```
|
||||
|
||||
### Check before pushing
|
||||
|
||||
```bash
|
||||
/tmp/backport-batch fold --line <line> --original N --commit "$(git rev-parse HEAD)"
|
||||
```
|
||||
|
||||
It prints `ok` and exits 0, or prints why not and exits 1. Push only on `ok`, as a fast-forward of the batch branch:
|
||||
|
||||
```bash
|
||||
git push origin HEAD:refs/heads/backport-batch/<line>-<yyyymmdd>
|
||||
```
|
||||
|
||||
Resolve one original at a time and push each before starting the next: the next fold has to sit on the new head. CI on the batch pull request runs again on the push, `backport-batch verify` among it, and the next run of the batch lists the original as on the branch, with the picks the resolution changed marked as such.
|
||||
|
||||
### Never
|
||||
|
||||
- Push a commit `fold` did not print `ok` for, or push anywhere but the batch branch.
|
||||
- Squash the original's commits into one pick, reorder them, or pick a commit of another pull request into the merge.
|
||||
- Edit a file outside the original's files to make the pick build; that is the dependency case above.
|
||||
@@ -0,0 +1,124 @@
|
||||
## Backports
|
||||
|
||||
Backports are automatic. Everything that can go into a patch release goes in, and nobody decides it pull request by pull request: nobody requests a backport, and contributors never open backport pull requests. The one exception is a maintainer override, below. A batch job takes the fixes merged into `main` and cherry-picks them onto each `release-X.Y` line it runs for, following the rules on this page.
|
||||
|
||||
### Getting a fix into a patch release
|
||||
|
||||
Merge it into `main` with a `fix` title, for example `fix(postgres): keep the replica count on resize`. Nothing else is needed: if the change meets the rules below, the next batch picks it up for every [line the batch runs for](#which-lines-the-batch-runs-for).
|
||||
|
||||
The batch takes a pull request's type from its title alone, so a fix that ships inside a `feat` or `refactor` pull request does not reach a patch release. Put the fix in a pull request of its own.
|
||||
|
||||
### What the batch takes
|
||||
|
||||
The batch takes a pull request from `main` into release line X.Y when all of these hold:
|
||||
|
||||
1. It merged into `main` after `release-X.Y` branched, or after the line started under this policy if that is later (see [Where each line starts](#where-each-line-starts)), and the line does not carry it yet.
|
||||
2. Its title type is `fix` or `security`, the type of a fix that comes through a security advisory. CI and e2e fixes count, because they keep the line's own CI working. The title is read once any leading `[Backport …]` prefixes are dropped, and a title that does not read as a Conventional Commits title is kept out.
|
||||
3. It is not breaking: no `!` after the type or scope, and no `BREAKING CHANGE:` or `BREAKING-CHANGE:` footer at the start of any line of the description its merge into `main` recorded. A footer inside an HTML comment, where the pull request template quotes it, does not count.
|
||||
4. It touches no platform migrations, anything under `packages/core/platform/images/migrations/migrations/`, and no `targetVersion` line of `packages/core/platform/values.yaml`, adds, removes or edits no CustomResourceDefinition document (a CRD in a test fixture under `tests/` or `testdata/` does not count, and neither does an edit to another document in the same file), and makes no change to the API that the [API review gate](../../.github/workflows/api-review-gate.yaml) calls sizeable: a new or removed API group or resource, or a schema change that can reject what the old schema accepted. Those wait for the next minor; a reworded description or a new optional field under `api/` does not. [Backporting code that touches monotonic counters](./backport-mechanics.md#backporting-code-that-touches-monotonic-counters) shows what a migration on a release line breaks.
|
||||
5. Each of its commits picks cleanly with `git cherry-pick -x`, or its conflict is resolved during the batch, and the batch's build and tests pass. A pick that fails moves to the next batch, and the batch pull request lists it with the files that conflicted.
|
||||
6. On an older line, it is already on the next newer line the batch runs for. Lines are processed newest first, so an older line takes only what the newer one already carries.
|
||||
|
||||
The batch reads the pull requests off `main`'s first-parent history, one per merge commit or squash commit that names its pull request. A commit there that names none, a direct push for instance, and a merge of more than two parents are never taken; the batch pull request counts them.
|
||||
|
||||
### What the batch leaves out on one line
|
||||
|
||||
A pull request the rules take for every line can still be wrong for one of them, for instance because it builds on code that line does not have. A maintainer drops it from that line with `./backport-batch run --line <line> --drop <number> --reason <why> --push`: the batch takes it off, if it carries it, and no later batch for that line picks it again. The drop is kept in the [line's record](#the-lines-record-and-reports), so it needs an open batch pull request to record it on; before a line's first batch pull request exists there is nothing to record it on, and the command refuses. `--undrop <number>` withdraws it, and the next batch tries the pull request again. A pull request the line already carries cannot be dropped. To land a hand-made port instead, open it as a pull request of its own, as [fixes that exist only on a release line](#fixes-that-exist-only-on-a-release-line) describes, and drop the original.
|
||||
|
||||
### What waits for the next minor
|
||||
|
||||
`feat`, `chore` and `docs` pull requests, and dependency bumps (scope `deps`), ship with the next minor, not in a patch release; only a maintainer's `backport/include`, below, takes one. If a class of fixes should also wait for a minor, it gets a new rule on this page; a single pull request that the rules get wrong is what the overrides below are for.
|
||||
|
||||
### Maintainer overrides
|
||||
|
||||
A maintainer overrides the rules for one pull request with a label on the merged pull request on `main`:
|
||||
|
||||
- `backport/include` takes it although rules 2 to 4 keep it out, a `feat` for instance, or although it merged before its line started under this policy. It still has to pick cleanly or have its conflict resolved, pass CI on the batch, and, on an older line, already be on the newer one, and a line that already carries it does not take it again.
|
||||
- `backport/skip` keeps it out although the rules take it, and `backport-batch check` stops asking for it. An open batch that picked it before the label came keeps it until `backport-batch run --defer` takes it out; the batch pull request says so meanwhile.
|
||||
|
||||
A label decides for every line. To keep a pull request out of one line only, drop it there with `backport-batch run --drop`, as [what the batch leaves out on one line](#what-the-batch-leaves-out-on-one-line) describes; `backport/include` does not lift a drop.
|
||||
|
||||
`backport/skip` wins when a pull request carries both. Each run reads the labels afresh, and the batch pull request lists every override and what became of it.
|
||||
|
||||
### Which lines the batch runs for
|
||||
|
||||
The batch runs for the two newest lines that have shipped a stable release, and for every line branched after the newest of them that has not shipped one yet. A line has shipped once its first stable `vX.Y.Z` tag exists, which only the merge of its promotion creates ([stable tags come from rc promotion](./stable-tags.md)); an rc, alpha or beta tag does not count. How many released lines it keeps is `released-lines` in the block below.
|
||||
|
||||
With 1.6 and 1.7 released, cutting `v1.8.0-rc.1` creates `release-1.8` ([release candidates](./release-candidates.md)), and from the next run the batch runs for 1.6, 1.7 and 1.8. Once `v1.8.0` ships it runs for 1.7 and 1.8, and 1.6 takes no more batches. Rule 6 holds across all of them, newest first: while 1.8 is in its rc window, a fix reaches 1.7 only after the 1.8 batch that carries it has merged, and 1.6 only after 1.7's.
|
||||
|
||||
During its rc window a line is frozen: every later rc is cut from `release-X.Y` itself, and `main` is open for the next minor. The freeze closes the line to `main`, not to the batch, which is how fixes reach it: a batch pull request against a frozen line is built, checked and merged like any other, and the next rc carries what it merged once `check --cut` passes (see [Running the batch](#running-the-batch)). A line that drops out keeps its open batch pull request as it is, and `backport-batch` no longer runs or checks that line; a maintainer merges the pull request or closes it.
|
||||
|
||||
### Where each line starts
|
||||
|
||||
The policy applies in full from `release-1.7` on: a line released before it took effect keeps the patch content its users chose it with, so it takes only fixes merged after the policy started, and an older one only with `backport/include`. The batch reads the block below from this page, on `main`: a line it lists starts on that day, and any other line starts where it branched from `main`. The block also gives the `batch-only` day and `released-lines`.
|
||||
|
||||
From the `batch-only` day in the block, a change from `main` reaches a line only through the batch, and only the batch's picks show that a line carries it: a pick whose `(cherry picked from commit …)` line, the one git appends, names the original's merge commit or every commit of its own. A picked commit keeps its message and a batch merge carries the original's, so text in them, a `Backport of #N`, a `Backported-from:` trailer or an `-x` line the commit's author wrote, counts only on commits from before that day, where it records the hand backports and earlier batches.
|
||||
|
||||
```backport-policy
|
||||
# release-X.Y: YYYY-MM-DD, the day the line started under this policy
|
||||
release-1.6: 2026-10-26
|
||||
# batch-only: YYYY-MM-DD, the day from which only the batch's picks show what a line carries
|
||||
batch-only: 2026-10-26
|
||||
# released-lines: N, how many of the newest lines with a stable release the batch runs for
|
||||
released-lines: 2
|
||||
```
|
||||
|
||||
### Fixes that exist only on a release line
|
||||
|
||||
Some fixes go to a release line as a pull request of their own. Such pull requests are occasional, and maintainers open them: for a fix with no counterpart on `main`, for one that on `main` came as part of another pull request, or for one written for `main` rather than for the line. When the counterpart on `main` is a pull request of its own that the batch would take, the batch skips it as already on the line once picking it would change nothing there; otherwise a maintainer labels it `backport/skip`.
|
||||
|
||||
A pull request opened against a `release-*` branch is closed automatically, with a comment that links to this page, when its author is a user account, not a bot, that GitHub names neither the repository's owner nor a member nor a collaborator, and whose access to the repository GitHub reports as read or none. Anyone with triage access or more keeps their pull request open. When GitHub's answer on the author's access cannot be read, the check closes nothing and fails, naming the read that failed. The check runs only when a pull request is opened, so a maintainer keeps a contributor's release-only fix by reopening it. [`hack/close-manual-backport.sh`](../../hack/close-manual-backport.sh) does this from [`pr-labeler.yaml`](../../.github/workflows/pr-labeler.yaml); run it with `--pr <number> --dry-run` to see what it would decide for a given pull request when it is opened.
|
||||
|
||||
### Cadence
|
||||
|
||||
The batch runs for each of its lines twice a week, on Monday and Thursday: often enough that a fix reaches its line within days, rarely enough that nobody spends time on it every day, and early enough in the week that each run leaves working days to cut an rc, run its e2e and promote it before the weekend. Patch releases ship weekly.
|
||||
|
||||
### Running the batch
|
||||
|
||||
[`backport-batch`](../../cmd/backport-batch/README.md) builds the batch in a clean worktree, newest line first. The scheduled run below does this; by hand it is:
|
||||
|
||||
```bash
|
||||
go build ./cmd/backport-batch
|
||||
./backport-batch run --line release-1.7 --continue --push
|
||||
./backport-batch run --line release-1.6 --continue --push
|
||||
```
|
||||
|
||||
A run by hand trusts the open batch pull request the scheduled run opened as `cozystack-ci[bot]` and continues it; `BACKPORT_BATCH_BOT` names another bot, and set empty trusts none.
|
||||
|
||||
`./backport-batch lines` prints the lines, newest first. `run`, `check`, `fold` and `lines` read the policy block from this page on `main`, so until the page with its block is on `main` they need `--policy <ref>` naming a ref that has it, and the scheduled run, which passes none, fails.
|
||||
|
||||
`run` picks what the rules take onto `backport-batch/<line>-<yyyymmdd>` and opens a draft pull request against the line. A pull request merged into `main` with a merge commit goes onto the batch as its own commits, oldest first, each picked with `git cherry-pick -x --signoff` so it keeps its author, its message and its sign-offs and gains the sign-off of whoever runs the batch; commits that merged `main` into the pull request's branch are left out. A merge commit joins them to the batch: it carries the original's merge message from `main` and ends in a `Backported-from: <merge commit>` trailer. A pull request squash-merged into `main` goes on as one pick. When any commit of an original conflicts, nothing of that original stays on the batch. The batch pull request is merged into the line with a merge commit, so all of this reaches the line as it is. Its body lists the originals with how many commits each brought and which of them differ from the commit they came from, the conflicts it deferred with the commit that failed and its files, and what it skipped under which rule, and ends with the [line's record and the run's report](#the-lines-record-and-reports); `--push` prints the pull request's URL. While that pull request is open, the next run continues it, `--continue` reading its head, rather than opening another. CI on the batch pull request runs the line's build and tests, and a maintainer merges it. Since rule 6 reads the newer line itself, an older line takes a fix in the batch after the newer line's batch with that fix has merged.
|
||||
|
||||
A deferred conflict waits for the next batch. To land it sooner, pick the original's commits onto the batch head with `git cherry-pick -x`, resolving each conflict, and join them to the batch head with a merge commit that names the original; an original squash-merged into `main` is one pick of its commit there, with no merge. Check that merge or that pick with `./backport-batch fold --line <line> --original <number> --commit <sha>`, and push it onto the batch branch, as [resolving a conflict](./backport-tasks/resolve-conflict.md) describes. CI runs `backport-batch verify` on every push to a batch pull request, which holds every original on it to the same shape. When CI on the batch pull request fails, the original it comes from is taken out with `./backport-batch run --line <line> --defer <number> --reason <why> --push` and goes to the next batch, as [attributing a failed batch CI](./backport-tasks/failed-ci.md) describes; the rest of the batch merges without it. When it is all the batch carries, `--push` records why in the batch pull request, closes it and deletes its branch, and the next batch tries it again. `--drop` and `--undrop` change the record of the open batch pull request in the same way, and need one.
|
||||
|
||||
Before a patch release or an rc is cut from a line, `./backport-batch check --line <line> --cut` has to exit 0. A cut tags the line, so the check fails while a pull request the rules take is not on the line, including one that is only on the open batch pull request: merge the batch pull request first. It prints the URLs, and prints what the batch deferred as warnings, since deferring is a maintainer's call; so is a pull request labelled `backport/skip` that the open batch still carries. An original that waits on the batch for a maintainer fails the cut, also when it is all the batch has left. A cut that passes says so. Without `--cut`, as the scheduled run uses it, a change on the open batch pull request counts as carried.
|
||||
|
||||
### The line's record and reports
|
||||
|
||||
What the batch decided about a line, the conflicts it deferred, the originals taken out with `--defer` and those waiting for a maintainer, is kept in a `backport-batch` block in the body of the line's newest batch pull request, its record. `backport-batch` reads the record whether that pull request is open, merged or closed, and writes it only while the pull request is open: a merged or closed batch pull request is never edited or commented on again. So a conflict the batch deferred is still a warning in `check` after the batch merged, until a later batch tries it again. A run that picks nothing while no batch is open writes nothing, and says so. A record `backport-batch` cannot read, or one written by someone it does not trust, stops it rather than let it fall back to an older one; how it decides is in [its README](../../cmd/backport-batch/README.md#the-lines-record-and-reports).
|
||||
|
||||
Each run also reports where every pull request merged into `main` since the line branched stands, and what it read to decide. The body carries the latest report under the record, and a comment the tool posts on the open batch pull request keeps each one, so an earlier classification is still there after a later run replaced the body, and after the batch merged. Each drop in a report keeps its reason and the merge commit it is bound to. `./backport-batch report --pr <batch>` prints the report the body carries, and `--id <id>` an earlier one; it writes nothing and never fetches, and reads GitHub and, for a report that keeps drops, the clone's history at the heads the report was made from. A run that picks nothing while no batch is open has nowhere to keep its report: the next batch's report has the line's state.
|
||||
|
||||
### Changes to workflows
|
||||
|
||||
The scheduled run's token may not push a commit that changes `.github/workflows/`, so the scheduled run leaves such an original out (`run --no-workflow-files`). The batch pull request lists it under "Waiting for a maintainer", with the workflow files it changes, and so does the batch's comment. It is no conflict. The policy still takes it: `check` warns about it while it waits, and `check --cut` fails on it. When nothing else is picked there is no batch pull request to list it on, and `check` reports it as missing.
|
||||
|
||||
A maintainer brings it over with their own token, one that may push workflow changes (the Workflows permission, or the classic `workflow` scope), by continuing the open batch without the flag:
|
||||
|
||||
```bash
|
||||
./backport-batch run --line <line> --continue --push
|
||||
```
|
||||
|
||||
That picks it onto the batch in the batch's shape, after what the batch carries, and the body stops listing it; later scheduled runs continue the batch on top. To cut without it instead, take it out with `./backport-batch run --line <line> --defer <number> --reason <why> --push`.
|
||||
|
||||
### The scheduled run
|
||||
|
||||
[`backport-batch.yaml`](../../.github/workflows/backport-batch.yaml) runs [`hack/backport-batch-scheduled.sh`](../../hack/backport-batch-scheduled.sh) on Monday and Thursday. For each line the batch runs for, newest first, it continues the open batch pull request or opens one, runs `check`, and then `backport-batch notify`, which reports what waits on the batch. A batch open for three days or more, so one the previous run already saw, gets a comment on its pull request saying so and a line in the run's summary. The comment is one per pull request, updated in place on every run. It also lists the deferred conflicts, which a maintainer resolves as [resolving a conflict](./backport-tasks/resolve-conflict.md) describes, and the originals that wait because they change workflows. A failed batch CI waits for a maintainer too, as [attributing a failed batch CI](./backport-tasks/failed-ci.md) describes; the comment does not list it, and it shows on the batch pull request's checks. `hack/backport-batch-scheduled.sh --dry-run` does the same in the current worktree and changes nothing on GitHub. Its `check` sees the line without the batch the dry run just built, so it can exit 1 where the real run's check passes.
|
||||
|
||||
The batch runs as the cozystack-ci GitHub App, which the release workflows already use, and needs no secret of its own. The job mints a token from `COZYSTACK_CI_APP_ID` and `COZYSTACK_CI_PRIVATE_KEY` for this repository alone, with Contents and Pull requests write and nothing more, and fails when either secret is missing. Its pushes start CI on the batch pull request, which a push with the job's own `GITHUB_TOKEN` would not. Its commits carry the App's bot, `cozystack-ci[bot]`, whose batch pull request and comment `backport-batch` trusts: `BACKPORT_BATCH_BOT`, which the job sets from the token, names it, and a run without the variable trusts `cozystack-ci[bot]` too. The App's installation has to grant this repository Contents and Pull requests read and write, which an organization admin sees among the organization's installed GitHub Apps. It has no Workflows permission, so a change to `.github/workflows/` waits for a maintainer, as [changes to workflows](#changes-to-workflows) describes.
|
||||
|
||||
It also reads one repository setting, which a repository admin sets up: `BACKPORT_ALERT_TEAM`, a variable, the team or person the comment mentions, as `org/team` or a login. Unset, the comment mentions nobody: the batch pull request is a draft the App opened, so no one is notified of what waits unless they watch the repository.
|
||||
|
||||
Optionally, once the `Backport batch shape` check runs from `main`, a ruleset makes it required on the release lines. It targets `release-*` and excludes `release-*.*.*`: `release-*` alone also matches the `release-X.Y.Z-rc.N` and `release-X.Y.Z` staging branches that [`tags.yaml`](../../.github/workflows/tags.yaml) and [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) push, which have no batch on them, and a required check would refuse those pushes. "Do not require status checks on creation" stays on, since [`cut-prerelease.yaml`](../../.github/workflows/cut-prerelease.yaml) creates `release-X.Y` with a push at its first rc.
|
||||
|
||||
To run the scheduled script for real by hand, export your own token as `BACKPORT_BATCH_TOKEN`: without `--dry-run` it uses that variable and no other credential. With a token that may push workflow changes, leave `--no-workflow-files` out and the run brings over what waits.
|
||||
@@ -0,0 +1,107 @@
|
||||
## Changelogs
|
||||
|
||||
### Where the canonical process lives
|
||||
|
||||
[`agents/changelog.md`](../agents/changelog.md) is the source of truth. Read it end-to-end before generating; do not infer the process from past commits or memory. The CI runs it under Copilot via the reusable `changelog-generate.yaml` — called by `promote-rc.yaml::changelog` at promote time and by the optional `changelog-rc.yaml` at rc time — and, as a backstop, `tags.yaml::generate-changelog`; locally the same prompt is replayable by following the doc directly.
|
||||
|
||||
### Common changelog failure modes
|
||||
|
||||
These are mistakes that have shipped in real changelogs. Verify against each before merging the changelog PR.
|
||||
|
||||
**1. Commits from outside the release range.** Caused by running `git log <prev>..HEAD` from `main` while generating a patch changelog — `HEAD` on `main` contains everything that landed since the tag plus backports merged to `release-X.Y` after the tag was cut. Use `git log <prev>..<new_tag>` explicitly. The doc was updated to require this after v1.3.1's changelog shipped with 14 entries for what was actually a 1-commit release.
|
||||
|
||||
**2. Original + backport listed as two separate entries.** Each backport must coalesce with its original into a single entry of the form `(in #orig, backport #bp)`. To verify after generation:
|
||||
|
||||
```bash
|
||||
# Every entry that mentions "backport" should also reference the original PR number
|
||||
grep -E 'backport #[0-9]+' docs/changelogs/v<new>.md | grep -v '#[0-9]*, backport'
|
||||
```
|
||||
|
||||
Output should be empty.
|
||||
|
||||
**3. Wrong PR author.** The squash-merge commit's author is whoever clicked "Merge" — not the person who wrote the code. **Always** resolve via `gh pr view <N> --json author --jq .author.login`, never `git log --format=%an`. This bites hardest for website-repo entries where the same merger handles most PRs.
|
||||
|
||||
**4. Superseding patch, stale changelog.** Tags are write-once, so a critical fix after `vX.Y.Z` ships as the next patch `vX.Y.(Z+1)` rather than moving the tag. If a changelog PR already exists for a tag you are superseding, the changelog work transfers to the new patch. Compare `git log <prev>..<new>` against what the existing `changelog-vX.Y.Z` branch already documents; only add/remove the deltas. If the only new commits are CI-internal or a revert of a feature that never reached a stable tag, no changelog edit is needed.
|
||||
|
||||
**5. PR numbers swapped inside prose.** The entry-format validator checks bullet entries but ignores Feature Highlights paragraphs and Upgrade Notes. Both have caused wrong PR references in shipped changelogs. Verify every `#NNNN` in prose with `gh pr view <N>`.
|
||||
|
||||
**6. Hallucinated entries.** Every entry must correspond to a commit in `git log <prev>..<new>`. Verify by extracting PR numbers from the file and grep-searching the commit range:
|
||||
|
||||
```bash
|
||||
grep -oE '#[0-9]+' docs/changelogs/v<new>.md | sort -u | while read pr; do
|
||||
n=${pr#\#}
|
||||
if ! git log <prev>..<new> --grep="#${n}" --oneline | grep -q .; then
|
||||
echo "PR $pr in changelog but not in commit range"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
**7. Bot accounts as human contributors.** `cozystack-ci[bot]`, `github-actions`, `dependabot`, `renovate`, and any `app/*` login must NOT appear in the `## Contributors` list. They legitimately appear in **per-entry attribution** (e.g. a Renovate-authored PR) — but not in the human roll-call.
|
||||
|
||||
**8. Title duplicated as description.** `* **fix(foo): X**: fix(foo): X (...)` means you wrote the conventional-commit subject twice and never wrote a user-facing description. The brief and the detail must say different things — the detail explains what the change means for users.
|
||||
|
||||
### Changelog pre-publish verification
|
||||
|
||||
Run these checks before merging the changelog PR. Catches problems in roughly the order they occur:
|
||||
|
||||
```bash
|
||||
PREV=v1.2.0; NEW=v1.2.1
|
||||
|
||||
# A. Every chart bump is mentioned somewhere in the changelog
|
||||
git diff --name-only $PREV..$NEW -- 'packages/*/charts/*/Chart.yaml' 'packages/*/*/charts/*/Chart.yaml' | while read f; do
|
||||
old=$(git show "$PREV:$f" 2>/dev/null | yq '.version // ""')
|
||||
new=$(git show "$NEW:$f" 2>/dev/null | yq '.version // ""')
|
||||
if [ "$old" != "$new" ]; then
|
||||
component=$(basename "$(dirname "$f")")
|
||||
if ! grep -qi "$component" docs/changelogs/$NEW.md; then
|
||||
echo "MISSING: $component $old -> $new ($f)"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# B. Out-of-Chart.yaml pins (Talos, tenant K8s, vendored images)
|
||||
for path in packages/core/talos/images/talos/profiles/installer.yaml packages/core/installer/values.yaml packages/apps/kubernetes/values.yaml; do
|
||||
git diff $PREV..$NEW -- "$path" | head -40
|
||||
done
|
||||
git diff --name-only $PREV..$NEW -- 'images/*/Dockerfile'
|
||||
|
||||
# C. Every cited #NNNN exists and is a merged PR (not an issue)
|
||||
grep -oE '#[0-9]+' docs/changelogs/$NEW.md | sort -u | sed 's/#//' | xargs -P 20 -I{} sh -c '
|
||||
state=$(gh api /repos/cozystack/cozystack/issues/{} --jq ".pull_request.merged_at // \"ISSUE\"" 2>/dev/null)
|
||||
echo "#{} $state"
|
||||
' | grep -E 'ISSUE|null|^#[0-9]+ $'
|
||||
|
||||
# D. URL liveness
|
||||
grep -oE 'https://[^)" ]+' docs/changelogs/$NEW.md | sort -u | xargs -P 10 -I{} sh -c '
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" {})
|
||||
if [ "$code" != "200" ]; then echo "$code {}"; fi
|
||||
'
|
||||
|
||||
# E. Website PR authors are PR authors, not commit authors
|
||||
grep -oE 'cozystack/website#[0-9]+' docs/changelogs/$NEW.md | sort -u | sed 's|cozystack/website#||' | while read n; do
|
||||
pr_author=$(gh pr view $n --repo cozystack/website --json author --jq .author.login)
|
||||
echo "website#$n -> @$pr_author"
|
||||
done
|
||||
|
||||
# F. Working tree clean except for the changelog
|
||||
git status --porcelain | grep -v "docs/changelogs/$NEW.md"
|
||||
# Should output nothing.
|
||||
```
|
||||
|
||||
Ship criterion: every chart bump surfaced, no issue-cited-as-PR, no 404 URLs, every website entry attributed via `gh pr view` (not commit author), `git status` shows only the changelog.
|
||||
|
||||
### Cozystack-specific URL quirks
|
||||
|
||||
`cozystack.io` paths are versioned: `/docs/operations/foo/` and `/docs/next/foo/` both 404. Only `/docs/vX.Y/foo/` and the unversioned canonical `/docs/foo/` serve content. AI-generated changelogs invent the unversioned form. Verify URLs with the `curl` loop above.
|
||||
|
||||
### Platform-component coverage
|
||||
|
||||
For a minor release (`vX.Y.0`), changelog entries should surface what an upstream chart bump actually brought in — features, breaking changes, security fixes. PR commit messages usually don't capture this; the upstream `CHANGELOG.md` or GitHub Release notes do. For each bumped `Chart.yaml`:
|
||||
|
||||
1. Read the `sources:` field — typically a GitHub URL.
|
||||
2. Fetch release notes strictly inside `old < x <= new` via `gh release view <tag> --repo <owner>/<repo> --json body,name`.
|
||||
3. Summarize in 2–5 user-impact bullets per component (new features, breaking changes, CVEs, important deprecations). Don't reproduce upstream notes verbatim; link to them.
|
||||
4. If a PR-level entry already mentions the bump, enrich that entry with bullets — don't duplicate the entry.
|
||||
5. If 3+ bumps have no PR-level mention, introduce a `## Platform Components` section.
|
||||
|
||||
If you can't find upstream notes for a bumped component, list it with version-only info and no bullets. Hallucinated bullets are worse than missing detail.
|
||||
@@ -0,0 +1,16 @@
|
||||
## CI failures release engineers commonly hit
|
||||
|
||||
| Symptom | Class | Diagnosis | Block release? |
|
||||
|---------|-------|-----------|----------------|
|
||||
| `Draft release for vX.Y.Z not found` on every release-* merge | Workflow regression — `Publish draft release` lost its `github-token` in a past refactor; default `GITHUB_TOKEN` cannot list drafts | At `pull-requests-release.yaml:163`. Undraft manually, file the fix | No |
|
||||
| Changelog Copilot job returns 402 | Token quota — `COPILOT_GITHUB_TOKEN` premium-request quota empty | Both changelog entry points hit it: the optional rc-time `changelog-rc` dispatch and the promote-time `changelog` job (both call the reusable `changelog-generate.yaml`), plus the `tags.yaml` backstop | No — `continue-on-error: true` on the AI step, generate changelog by hand following [`agents/changelog.md`](../agents/changelog.md) |
|
||||
| `Prepare Release` skipped and the run red, `Build arm64 images` failed | The rc's arm64 build failed | Read the `build-arm64` log; `build-arm64-nightly.yaml` usually failed the same way first. Fix on the line, cut the next rc | Yes — an rc is not published amd64 only |
|
||||
| `Stitch multi-arch indexes and republish` fails listing refs | `verify-multiarch.sh` found refs that are not amd64+arm64 indexes, or that it could not check | Each listed ref names its reason. Single-platform manifest or index without one architecture: make the image multi-arch, or allowlist its repository with a reason if it cannot be. Manifest unreadable: correct the ref or digest; the allowlist does not excuse it. Rate-limited or unavailable after 3 tries: re-run once the throttle clears | Yes |
|
||||
| E2E `kubernetes-test` fails with `CSINode does not contain driver csi.kubevirt.io` | Test ordering bug | Tenant's `kubernetes-${test_name}-csi` HelmRelease still installing when NFS PVC is created. Fix: wait for `csinode/<node>` to advertise the driver before creating the PVC | Yes — flaky and masks real CSI bugs |
|
||||
| `tenant-root` HR in perpetual upgrade-rollback loop after OIDC patch | Real bug at the intersection of #2602's readiness change and the cozystack reconcile cadence | Look for `rate: Wait(n=1) would exceed context deadline` in operator logs; multiple tenant child HRs stuck `InProgress` | Yes |
|
||||
| `cozy-dashboard` ImagePullBackOff from `cozystack-ui:latest` | Registry flake | Known noisy on some OCI mirrors; not currently in the prepull set | No — retry |
|
||||
| PR CI: `docker push` 409 on `:latest` for the same image across two PRs | Concurrent manifest race on a shared floating tag | The `:latest` tag was previously written by every `make image` regardless of PR. Resolved by the image-tag refactor (`IMAGE_TAG=pr-<N>-<sha>` in PR builds, `IMAGE_TAG=<ref_name>` only at release time) | No — `gh run rerun --failed` |
|
||||
|
||||
**Heuristic:** workflow-token and Copilot-quota issues are out-of-band — release the tag, fix the workflow in a follow-up PR. Test ordering bugs and tenant-root reconcile storms are real and should block.
|
||||
|
||||
If you find yourself doing the same manual fixup on two consecutive releases (e.g. "undraft the release"), open a workflow-regression issue. Workflow bugs with known manual workarounds rot silently for months.
|
||||
@@ -0,0 +1,107 @@
|
||||
## What CI does during the release process
|
||||
|
||||
The numbered process of a [regular release](./regular-releases.md#regular-releases) is implemented by seven workflows, including the reusable/manual [`e2e-tag.yaml`](../../.github/workflows/e2e-tag.yaml), plus the reusable [`changelog-generate.yaml`](../../.github/workflows/changelog-generate.yaml) that two of them share to produce the changelog. Knowing which job does what makes the failure modes much easier to diagnose.
|
||||
|
||||
1. [`cut-prerelease.yaml`](../../.github/workflows/cut-prerelease.yaml) — manual dispatch (from `main` or `release-X.Y`): the sole entry point for creating a pre-release tag. Validates the `-alpha`/`-beta`/`-rc` tag, then pushes it (write-once, at the branch tip) as the CI app so `tags.yaml` fires. Cutting the first `vX.Y.0-rc.N` also creates `release-X.Y`, freezing the line; afterwards a `vX.Y.*` dispatch from `main` is refused. Stable tags are never created here.
|
||||
2. [`tags.yaml`](../../.github/workflows/tags.yaml) — fires on any pre-release tag push: `build-amd64` and `build-arm64` build the pre-release side by side, then `prepare-release` publishes it and pushes its digest-pinned staging branch, then — **only for `-rc.N` tags** — the mandatory `rc-e2e` job calls `e2e-tag.yaml` in the same run (`-alpha.N` / `-beta.N` skip it and are validated on demand with `e2e-tag.yaml`'s manual button). `generate-changelog` and `update-website-docs` also live here but are **backstops only** — both are normally produced at promotion time (see 4); on a stable tag push each self-skips or no-ops when the promote flow already did the work. These stable-tag backstops only fire because finalize pushes the stable tag with the CI app token (a `GITHUB_TOKEN` push triggers no run — see 5).
|
||||
3. [`e2e-tag.yaml`](../../.github/workflows/e2e-tag.yaml) — reusable from `tags.yaml::rc-e2e` and manually dispatchable for a published tag. It checks that the tag's release carries its nocloud disk, resolves the digest-pinned tree and runs the full install, OpenAPI, and Chainsaw suite on Talos containers. A tag cut before the container lane existed has to be dispatched from its own ref. Its dispatch run title is the constant workflow name, so the promote gate correlates a manual run through the successful tag-bearing E2E job name rather than the title.
|
||||
4. [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) — manual dispatch, from `main` only (the signature over a stable release's assets names that ref, see [Release signatures and SBOM](./signatures-and-sbom.md#release-signatures-and-sbom)). Five jobs: `parse` validates the rc tag and requires green full-E2E evidence before any staging; `promote` rewrites and commits the rc tags, publishes that packages tree as a run-specific temporary reproducible candidate, commits its digest pin in the `release-X.Y.Z` tree, and drafts the stable release; `changelog` calls the reusable `changelog-generate.yaml` **in parallel** to produce `docs/changelogs/vX.Y.Z.md` (regenerating it, or copying an rc-time changelog already committed to the `release-X.Y.Z-rc.N` staging branch by `changelog-rc.yaml` — see 7); `website-docs` (needs `parse`, `promote`) opens the `cozystack/website` docs PR from the staging branch via `FETCH_REF` (see [Phase 3](#phase-3--update-website-docs-now-the-backstop)); `open-pr` joins them, commits the changelog onto the staging branch, and opens the `release-X.Y.Z` promote PR with the `release` label but no automatic `full-e2e`. So the changelog and website docs are reviewed as part of the promotion, the already-checked rc suite is not repeated by default, and the PR body carries ✅ verified or ⚠️ bypassed plus ✅/⚠️ changelog and website-docs lines. Both `changelog` and `website-docs` are non-blocking. No stable registry name is created at dispatch.
|
||||
5. [`.github/workflows/pull-requests.yaml`](../../.github/workflows/pull-requests.yaml) verifies a promotion candidate against the prospective merge tree and original rc artifact before the required `E2E Tests` status can turn green. [`pull-requests-release.yaml`](../../.github/workflows/pull-requests-release.yaml) fires after merge and repeats that verification before any stable tag, then cuts the write-once stable + Go-module tags, publishes the release **with the merged changelog as its body**, retags the candidate and rc images to stable by digest (`:latest` gated on newest-stable), and publishes the stable chart.
|
||||
6. [`update-releasenotes.yaml`](../../.github/workflows/update-releasenotes.yaml) — fires on pushes to `main` that touch `docs/changelogs/v*.md`; syncs that content into the corresponding GitHub Release body. No longer the primary path (finalize sets the body directly, which is race-free and works for maintenance lines too) — this now covers later edits to a published changelog and manual re-syncs via `workflow_dispatch`.
|
||||
7. [`changelog-rc.yaml`](../../.github/workflows/changelog-rc.yaml) — manual dispatch, **optional**: generate the changelog at rc time instead of waiting for promotion, while there is still time to review it. It calls the same reusable [`changelog-generate.yaml`](../../.github/workflows/changelog-generate.yaml) and commits the result to the `release-X.Y.Z-rc.N` staging branch as `docs/changelogs/vX.Y.Z.md`; the promote-time `changelog` job then copies that file instead of running the AI again. Skipping this workflow changes nothing — promote generates the changelog itself, exactly as before.
|
||||
|
||||
### Phase 1 — `build-amd64`, `build-arm64` and `prepare-release` (hard gate)
|
||||
|
||||
On an rc tag push: builds images for both architectures, commits digest pins, creates the draft release, and pushes the `release-X.Y.Z-rc.N` staging branch. `build-amd64` holds the release-exists no-op and the rejection of a hand-pushed stable tag, and builds the amd64 images and the release assets (its `make build` also pushes the packages artifact and the installer chart, pinned on amd64 digests); `prepare-release` stitches, gates, republishes those two from the repinned tree, and publishes the release. (The stable promote PR is opened later by `promote-rc.yaml`, not here. When the stable tag itself is pushed by finalize, a draft already exists, so this job is a no-op — see [Phase 5](#phase-5--pull-requests-releaseyaml-finalize).)
|
||||
|
||||
The commit (`Prepare release vX.Y.Z`, authored by `cozystack-ci[bot]`) is **digest pins and image tags only**:
|
||||
|
||||
- `packages/core/{installer,platform,testing}/values.yaml`
|
||||
- `packages/system/*/values.yaml` (cozystack-api, dashboard, kamaji, linstor, kubevirt-csi-node, etc.)
|
||||
- `packages/apps/kubernetes/images/{kubevirt-csi-driver,ubuntu-container-disk-*}.tag`
|
||||
- `packages/system/dashboard/templates/configmap.yaml` (the `$tenantText` value)
|
||||
|
||||
Things that look surprising in the diff but are normal:
|
||||
|
||||
- **Zero-diff packages**: when buildx fully caches and upstream base images haven't moved, the digest is byte-identical to the previous release and won't appear. Verify the build step actually ran — don't assume "no diff = nothing built."
|
||||
- **Sudden churn on `ubuntu-container-disk-*` tags**: `cloud-images.ubuntu.com/noble/current/` is a moving target; these often rebuild even without code changes.
|
||||
- **Switched registry**: if a self-built image moved from `docker.io` to `ghcr.io`, the registry portion of the digest changes — that's a deliberate move, not a regression.
|
||||
- **`targetVersion` is NOT touched by the release PR.** Platform migration `targetVersion` is bumped earlier in a feature commit (e.g. `[platform] Bump migration targetVersion to 39 for migration 38`). The release PR only re-pins the `platform-migrations` image digest.
|
||||
|
||||
#### Multi-arch images
|
||||
|
||||
Every pre-release is published for amd64 and arm64. `tags.yaml::build-arm64` builds the images on a native arm64 runner and pushes each as `<tag>-arm64`, while `build-amd64` builds amd64, the Talos installer and matchbox for both arches, and the assets on x86 at the same time. `build-amd64` hands its stamped tree, the log of the tags it pushed and the assets to `prepare-release` as workflow artifacts; a missing one fails the upload or the download rather than publishing a partial release. `prepare-release` waits for both builds and runs only if both succeeded: when either fails, the rc is not prepared at all, so a release is never published amd64 only. It applies the stamped tree and runs [`hack/stitch-multiarch.sh`](../../hack/stitch-multiarch.sh), which joins each image with its arm64 twin into one index, moves onto it every tag the build pushed for that image (the rc tag and, under `PUBLISH_VERSIONED=1`, its component version, which the build records in `PUSHED_TAGS_LOG`), and repins the tree on the index digests, then [`hack/verify-multiarch.sh`](../../hack/verify-multiarch.sh), then republishes the packages artifact and the installer chart from the repinned tree. A stable tag skips `build-arm64` and builds nothing: promotion copies the rc's indexes with every platform, so the stable release is exactly as multi-arch as its rc. Because the two builds run in parallel, an rc that is not prepared (its arm64 build failed, or `prepare-release` failed before the stitch) still leaves behind what the amd64 build pushed: amd64-only images under `:<rc>` and under the component-version tags, and the `cozystack-packages:<rc>` artifact and the installer chart, both pinned on those amd64 digests. A component tag an earlier rc had moved to an index then points at an amd64-only image again, until a run that succeeds stitches it, a re-run of the same run included.
|
||||
|
||||
`verify-multiarch.sh` is a hard gate. It resolves every ref the tree pins, third-party images included, and fails naming each one that is not an index carrying both `linux/amd64` and `linux/arm64`, or that it cannot resolve at all. The fix is to make the image multi-arch (build it for both, or pin a multi-arch digest of the same version); an image that cannot exist for arm64 goes in [`hack/multiarch-allowlist`](../../hack/multiarch-allowlist) by repository, with the reason on the same line.
|
||||
|
||||
That gate sees only what the tree pins by digest. An image a vendored chart renders from its own defaults, or one an operator launches at runtime, is invisible to it. `rc-e2e` closes most of that gap: before tearing the sandbox down, [`e2e-tag.yaml`](../../.github/workflows/e2e-tag.yaml) reads every image the three Talos nodes pulled from their image stores (`hack/collect-images.sh`), each by the digest the pull resolved to, and runs `verify-multiarch.sh --refs-file` on that list with the same allowlist. A pre-release fails on any single-arch image, on any ref it cannot read or that is still rate-limited after the retries, and on a list the collection could not produce; a stable tag's run only warns, since it ships its rc's images. A tag whose tree predates the audit (it has no `hack/verify-multiarch.sh`) only warns and passes, so an rc cut before the audit existed can still be re-validated. The list and the audit output are in the run's `image-list-<tag>` artifact. Neither check covers a package that e2e never installs and whose images the tree does not pin by digest: an arm64 break there shows up only on an arm64 cluster.
|
||||
|
||||
[`build-arm64-nightly.yaml`](../../.github/workflows/build-arm64-nightly.yaml) builds `main` for arm64 every day, so an arm64 build break shows up there before the next rc. Its first step prints `verify-multiarch.sh --report` for `main`'s committed tree: `main` pins amd64 images, so its first-party refs are all listed and are expected to be, since the rc's stitch turns them into indexes. A third-party ref that an in-tree rebuild stamps over at build time, as the Harbor components and shadowbox are, is listed too and goes away on the rc. The lines that matter are the third-party refs no build replaces and refs that cannot be resolved; each of those fails the next rc until it is fixed or allowlisted. matchbox is listed too and does not matter: `build-amd64` rebuilds it as a two-arch index. The stitch names the images it skips in its own summary: an image with no `<tag>-arm64` twin, one whose digest is also pinned in a file the rewrite does not reach, and a pair that is not one amd64 and one arm64 image. matchbox (already an index) and the e2e sandbox (never built for arm64, and allowlisted) are skipped on every rc and pass the gate; any other skipped image fails it.
|
||||
|
||||
### RC E2E — mandatory check before promotion
|
||||
|
||||
For an rc, `prepare-release` must finish before `tags.yaml::rc-e2e` calls the published-tag lane, so the full suite tests the digest-pinned staging tree that promotion will reuse, and the release it runs against already carries its disk. `promote-rc.yaml::parse` then checks the latest `tags.yaml` push run whose `head_branch` is the rc tag and accepts it only when the reusable E2E job succeeded. GitHub exposes called-workflow job names as `E2E Release Candidate / E2E vX.Y.Z-rc.N (full suite)`, which the gate matches by the exact tag-bearing suffix.
|
||||
|
||||
The alternate evidence path scans retained successful `workflow_dispatch` runs of `e2e-tag.yaml` and accepts one whose direct E2E job is named `E2E vX.Y.Z-rc.N (full suite)`. The workflow's run title is always `E2E Release Tag` and therefore cannot correlate the input tag; artifact names also carry the tag but expire independently, so the job name is the strongest available correlator. If Actions retention or manual deletion removes the run/job metadata, or that job is renamed without updating the matcher, the gate fails closed and tells the maintainer to run the E2E Release Tag button again. `skip_e2e_gate=true` is an emergency override, not alternate evidence: it writes a loud warning and a ⚠️ PR-body line.
|
||||
|
||||
### Phase 2 — the changelog (generated at rc time or promotion, not here)
|
||||
|
||||
The changelog for `vX.Y.Z` is produced by [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml)'s `changelog` job (a thin call to the reusable [`changelog-generate.yaml`](../../.github/workflows/changelog-generate.yaml)) and committed onto the `release-X.Y.Z` staging branch by `open-pr`, so it rides the promote PR and is on the base branch the moment the release publishes. Finalize then uses that file verbatim as the GitHub Release body. Generating it here — on the stable tag push — would be too late by construction: that tag is created by finalize, which has already published the release. It can also be produced earlier, at rc time, via the optional [`changelog-rc.yaml`](../../.github/workflows/changelog-rc.yaml) button — which runs the same reusable core and commits the file to the `release-X.Y.Z-rc.N` staging branch, from which the promote-time `changelog` job copies it (validated) instead of regenerating.
|
||||
|
||||
The `changelog` job runs **in parallel** with `promote`, detached at the rc tag (the "promotion" configuration in §2 of [`agents/changelog.md`](../agents/changelog.md)), using a separate **read-only** GitHub App token for the AI step so the model cannot mutate the repo even with `--allow-all-tools`. Every write — the commit onto the staging branch and the PR — happens in `open-pr` under a write-scoped token.
|
||||
|
||||
Generation is deliberately **non-blocking**: a Copilot outage or exhausted token quota must never block a release. Only the AI step carries `continue-on-error`, and `open-pr` is gated on `promote` alone (`if: !cancelled() && needs.promote.result == 'success'`) so the promote PR opens regardless. When the changelog is missing the PR body says so explicitly — that ⚠️ line is the signal to act on.
|
||||
|
||||
Recovery, in order of preference:
|
||||
|
||||
1. Commit `docs/changelogs/vX.Y.Z.md` to the `release-X.Y.Z` branch by hand (follow [`agents/changelog.md`](../agents/changelog.md)). `promote` preserves it across a re-dispatch, and `open-pr` will not overwrite it with generated output.
|
||||
2. Re-dispatch `promote-rc.yaml` — generation runs again; anything already on the branch wins.
|
||||
3. Merge without one and let `tags.yaml::generate-changelog` (the backstop) open a changelog PR against `main` after the fact. The release is published with the draft's body until that merges.
|
||||
|
||||
Known failure modes:
|
||||
|
||||
- Copilot quota exhausted (`COPILOT_GITHUB_TOKEN` 402). Refill or rotate, then re-dispatch.
|
||||
- AI step timeout (hard 30-min cap).
|
||||
- Output truncated or whitespace-only — caught by the `Verify changelog` step (`hack/validate-changelog.sh`), which accepts either header convention (`# Cozystack vX.Y.Z` for a minor or `# vX.Y.Z (<date>)` for a patch), and requires the leading release-link comment pointing at `releases/tag/vX.Y.Z`, a `compare/...vX.Y.Z` link ending in this version, and at least one `## ` section — deliberately with no line-count floor, since a complete short patch changelog (v1.5.1 ships in 19 lines) must not be rejected as a fragment. It downgrades to the missing-changelog path rather than shipping a fragment as release notes.
|
||||
|
||||
**`tags.yaml::generate-changelog` is the backstop.** It self-skips when the changelog is already on `main`. It earns its keep in two cases: generation failed during promotion, or the promote PR targeted `release-X.Y` so the file never reached `main`. Since the rc freeze the second case is **every** release, minor and patch alike — the promote PR always targets the release line (see [Regular Releases](./regular-releases.md#regular-releases)), which makes this backstop the only route by which a published changelog reaches `main`. In that case it **ports the reviewed file from the tag commit verbatim** rather than regenerating — a second AI pass there would spend quota and then overwrite already-published, already-reviewed release notes.
|
||||
|
||||
### Phase 3 — `update-website-docs` (now the backstop)
|
||||
|
||||
The `cozystack/website` "update managed apps reference" PR is normally opened **earlier, at promote time**, by [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml)'s `website-docs` job — not here. That job fetches the README content from the `release-X.Y.Z` staging branch `promote` just pushed (via the website Makefile's `FETCH_REF` variable), so the docs are ready to review alongside the promote PR while the stable tag is still unborn. It refreshes the trunk version pins from the staging ref before promoting `next/` → `vX.Y/`, so a new minor's docs inherit fresh Talos/cozystack pins instead of a stale `next.yaml`. It is **non-blocking**: if it fails (or trips its `FETCH_REF`-support guard against a website checkout that predates that Makefile variable), the promote PR opens with a ⚠️ website-docs line and this tag-time job backfills the same branch.
|
||||
|
||||
**Merge-timing contract:** the website PR must NOT be merged before the release is published (finalize). Merging it early flips the site's latest-version pointer to a version whose versioned docs and `openapi.json` are not on GitHub yet, so every version-pin and API link for the new version 404s. Both the website PR body and the promote PR body say so; park the website PR until the GitHub release for `vX.Y.Z` is live, then merge.
|
||||
|
||||
This tag-time `update-website-docs` job stays as the **backstop**. It runs with `if: !cancelled() && needs.prepare-release.result == 'success' && !contains(github.ref_name, '-')` (surviving a failed/skipped changelog phase, stopping on cancel, and skipping every pre-release tag), decides whether to promote `next/` → `vX.Y/`, runs `make update-all`, stages `content hugo.yaml data/versions`, and force-pushes the same `update-docs-vX.Y.Z` branch. On the normal path — where `promote-rc.yaml::website-docs` already did the work — it no-ops into "No changes to commit". At tag time the stable tag exists, so its `BRANCH=RELEASE_TAG` fetch resolves without needing `FETCH_REF`. For it to fire at all, finalize must push the stable tag with the **CI app token** (see [Phase 5](#phase-5--pull-requests-releaseyaml-finalize)) — a tag pushed with the default `GITHUB_TOKEN` triggers no workflow run.
|
||||
|
||||
If anyone changes the website Makefile to write somewhere else (e.g. `static/`, `i18n/`), the `git add` list must grow in **both** jobs or the PR silently drops files.
|
||||
|
||||
### Phase 4 — review and merge the promote PR
|
||||
|
||||
Reviewer checklist for the `chore(release): promote vX.Y.Z-rc.N -> vX.Y.Z` PR:
|
||||
|
||||
- [ ] The PR body says `✅ RC full e2e was verified`; if it says the gate was bypassed, obtain explicit maintainer acceptance or add `full-e2e` and wait for that opt-in run.
|
||||
- [ ] `Verify release packages candidate` is green. It pulls the candidate and original rc artifact by digest, compares the candidate to GitHub's prospective merge tree, and feeds the required `E2E Tests` status even when full E2E is not repeated. Do not edit `packages/` on this PR after generation; if the guard reports drift, re-dispatch promotion to create and pin a new run-specific candidate. Note what a green here does *not* cover: GitHub fires no `pull_request` event when the base branch advances, so a `packages/` change merged to the base afterwards is never re-verified on the PR. Finalize repeats the check against the real merge commit and catches that case before any stable name exists — but it catches it after the promote PR has already merged, so a `packages/` change landing on the base while a promote PR is open is worth holding back.
|
||||
- [ ] The PR body's **website-docs line** is present: if ✅, the `cozystack/website` docs PR is open and **parked** — do NOT merge it until `vX.Y.Z` is published (see [Phase 3](#phase-3--update-website-docs-now-the-backstop)); if ⚠️, the tag-time backstop will backfill it after finalize.
|
||||
- [ ] Diff is digest pins + image tags only — nothing else.
|
||||
- [ ] No accidental config drift (a value file you don't recognize).
|
||||
- [ ] If any chart bumped, check the new digest pulls — `crane manifest ghcr.io/cozystack/cozystack/<name>@<digest>` should return.
|
||||
- [ ] Build artifacts on the draft release page exist and match the expected count.
|
||||
- [ ] The PR includes `docs/changelogs/vX.Y.Z.md`, it has content, and the entries match the commit range (see [Changelog pre-publish verification](./changelogs.md#changelog-pre-publish-verification)). If the PR body carries the ⚠️ "No changelog in this PR" line, resolve that before merging — finalize publishes this file as the release body, unless the PR also carries `docs/releasenotes/vX.Y.Z.md` (see [Phase 6](#phase-6--update-releasenotesyaml-sync-github-release-body)).
|
||||
|
||||
### Phase 5 — `pull-requests-release.yaml` (finalize)
|
||||
|
||||
Fires on merge of a PR that is merged, carries the `release` label, and is authored by `cozystack-ci[bot]` (the last guard closes the "name a branch `release-X.Y.Z`, get it labelled, merge it" hole). Head branch must match `release-X.Y.Z[-suffix]`. This is where **every consumer-visible or write-once side effect of a promotion happens** — after the rc's mandatory E2E was checked at dispatch and the PR merged, making promotion transactional at stable names. Steps:
|
||||
|
||||
1. **Verify the packages candidate before creating any stable tag.** [`hack/verify-promoted-packages.sh`](../../hack/verify-promoted-packages.sh) pulls the candidate by the digest committed in installer values, proves it matches the merged `packages/` tree after normalizing only the impossible self-reference, rejects surviving rc image tags, then follows the candidate's embedded pre-push `platformSourceRef` back to the original rc artifact and compares normalized container `repo@digest` sets. Any package edit after candidate publication or any changed container digest aborts finalize while all stable names are still absent.
|
||||
2. **Create the tag at the merge commit** (write-once). The merge commit of `Prepare release vX.Y.Z` did not exist before the PR opened, so there is nothing to move — the tag is created here for the first time. A pre-existing tag at a different commit fails the step loudly rather than being force-moved (see [Tag immutability](./tag-immutability.md#tag-immutability)). The checkout uses `persist-credentials: false` so this push authenticates as the **CI app** (via `git remote set-url`), not the default `GITHUB_TOKEN` — a `GITHUB_TOKEN` push creates no workflow run, so a persisted-credential push would silently skip `tags.yaml` entirely and its stable-tag backstops (`generate-changelog`, `update-website-docs`) would never fire.
|
||||
3. **Cut the `api/apps/v1alpha1/vX.Y.Z` Go-module tag** (write-once, stable only) at the same commit, so Go consumers of `api/apps/v1alpha1` get the release. Moved here from `tags.yaml`, whose `prepare-release` body is skipped for a promoted stable (the draft already exists).
|
||||
4. **Ensure the maintenance branch `release-X.Y` exists** at the tag commit. Since the rc freeze creates it, this is normally a no-op fast-forward — the merge commit is already the branch tip. Still created if missing; updated fast-forward-only — a non-fast-forward update warns and is left for a maintainer rather than being force-updated.
|
||||
5. **Publish the draft release**, with `docs/releasenotes/vX.Y.Z.md`, or without it `docs/changelogs/vX.Y.Z.md`, from the merge commit as its body (absent or whitespace-only files warn and leave the draft's body in place, rather than publishing blank notes, and a body over 100000 bytes is published as a link, see [Phase 6](#phase-6--update-releasenotesyaml-sync-github-release-body)). `make_latest` is computed against published-non-prerelease tags: prereleases stay `false`; tags older than the current max stay `false` (and the current max is force-restored to `latest` if necessary, so an older patch tag cut after a newer minor won't downgrade `latest`).
|
||||
6. **Retag the verified packages candidate and rc container images to stable** by digest (`hack/promote-retag.sh`, no container rebuild) and **publish the stable `cozy-installer` chart**. `:latest` (on both the images and the chart) moves only when this release's `make_latest` was `true` — the same decision as step 5, so the release's `latest` and the images' `:latest` never disagree. The chart is packaged with `platformVersion` stamped into its default values so the documented `helm --install --version X.Y.Z` path reports the stable version.
|
||||
|
||||
### Phase 6 — `update-releasenotes.yaml` (sync GitHub Release body)
|
||||
|
||||
Fires on pushes to `main` that touch `docs/changelogs/v*.md`. Reads each `docs/changelogs/vX.Y.Z.md` and PATCHes the matching GitHub Release's `body` if it differs. Since Phase 5 now sets the body directly from the merge commit, this is no longer on the critical path — it covers later **edits** to a published changelog, the backstop PR landing a changelog that reached `main` after the fact, and manual re-syncs via `workflow_dispatch`. It only watches `main`, which is why finalize (not this job) is what populates a release promoted onto `release-X.Y` — since the freeze, that is every release rather than only a patch.
|
||||
|
||||
The release body is `docs/releasenotes/vX.Y.Z.md` when it exists and is not blank, otherwise the changelog, which stays the full record either way, and a push to `main` touching either file fires this sync. Finalize sets the body after the stable tags exist, where a body GitHub refuses would strand the release, so at publish time a body over 100000 bytes is replaced by a one-line link to `docs/changelogs/vX.Y.Z.md` on the tag, with a warning in the run's log. This sync never writes that link: it warns and leaves such a release's body as it is. The `tags.yaml` backstop ports only the changelog to `main`, so release notes have to be added to `main` as well, otherwise the sync falls back to the changelog and replaces the release notes with it whenever the changelog fits.
|
||||
@@ -0,0 +1,6 @@
|
||||
## Cleanup after release
|
||||
|
||||
- The `release-X.Y.Z` branch is deleted by GitHub when its PR merges. The `release-X.Y` maintenance branch is created by `cut-prerelease.yaml` at the rc freeze (create-only, never force-moved) and fast-forwarded afterwards by `pull-requests-release.yaml::Ensure maintenance branch`.
|
||||
- The draft release is published by the same workflow.
|
||||
- The release body is set by finalize from `docs/changelogs/vX.Y.Z.md` at publication. Afterwards, `update-releasenotes.yaml` re-syncs that file into the GitHub Release body on any push to `main` that touches it — so edits to a published changelog land on the release page the next time `main` moves.
|
||||
- Local cleanup: remove your worktree (`git worktree remove`) and prune merged release branches.
|
||||
@@ -0,0 +1,11 @@
|
||||
## Introduction
|
||||
|
||||
Cozystack uses a staged release process to ensure stability and flexibility during development.
|
||||
|
||||
There are three types of releases:
|
||||
|
||||
- **Release Candidates (RC)** – Preview versions (e.g., `v1.2.0-rc.1`) used for final testing and validation.
|
||||
- **Regular Releases** – Final versions (e.g., `v1.2.0`) that are feature-complete and thoroughly tested.
|
||||
- **Patch Releases** – Bugfix-only updates (e.g., `v1.2.1`) made after a stable release, based on a dedicated release branch.
|
||||
|
||||
All three are matched by `tags.yaml`'s regex `^v\d+\.\d+\.\d+(-(alpha|beta|rc)\.\d+)?$`. The tag push is the trigger for the whole pipeline. **Pre-release tags (`-alpha` / `-beta` / `-rc`) are cut with the [`Cut Pre-release Tag`](../../.github/workflows/cut-prerelease.yaml) workflow** (manual dispatch from `main` or `release-X.Y`) — it pushes the tag as the CI app at the dispatch branch's tip, so GitHub fills `base_ref` (`tags.yaml`'s `Get base branch` step refuses tags pushed without a base) and the push triggers `tags.yaml` (a tag pushed with the default `GITHUB_TOKEN` would not). Stable `vX.Y.Z` tags are never pushed by hand at all — they are cut write-once by the promote flow at a PR merge commit. Only `-rc.N` pre-releases feed the promotion flow ([`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) accepts `vX.Y.Z-rc.N` alone); `-alpha.N` / `-beta.N` tags are built and published for download and validated on demand with the [`e2e-tag.yaml`](../../.github/workflows/e2e-tag.yaml) button, but are never promoted to a stable release.
|
||||
@@ -0,0 +1,21 @@
|
||||
## Nightly builds
|
||||
|
||||
A nightly is an **installable copy of `main` on GHCR — not a rebuild and not a release**. [`build-main.yaml`](../../.github/workflows/build-main.yaml) already builds every push to `main` into the CI registry (OCIR); the nightly promotes that build to the public release registry (GHCR) and proves it installs.
|
||||
|
||||
[`nightly.yaml`](../../.github/workflows/nightly.yaml) runs daily (gated by the `NIGHTLY_ENABLED` repo variable) as three jobs; build-disk and e2e both start once mirror has finished:
|
||||
|
||||
1. **mirror** — resolve the exact commit `cozystack-packages:main` was built from, then [`hack/nightly-mirror.sh`](../../hack/nightly-mirror.sh) copies every cozystack-owned component image OCIR→GHCR **by digest** (bit-for-bit, no rebuild) and re-publishes the rewritten `cozystack-packages` artifact and the `cozy-installer` chart to GHCR, tagged `0.0.0-nightly.<YYYYMMDD>.<run-id>.<attempt>` plus a floating `nightly`.
|
||||
2. **build-disk** — assemble the Talos `nocloud` disk from the upstream siderolabs imager (the profile references only `ghcr.io/siderolabs/*`, so this needs no cozystack image and no rebuild) and publish it as a GHCR OCI artifact (`cozystack-nocloud`) so a nightly is installable on real hardware. The nightly e2e does not boot it.
|
||||
3. **e2e** — stage the published GHCR closure (rewritten tree + the `cozy-installer` chart pinned to the GHCR packages artifact) and run the **full** app suite on Talos containers, which do not boot the disk. A nightly has no diff, so Test Impact Analysis does not apply.
|
||||
|
||||
No GitHub release and no `api/apps/v1alpha1/*` Go-module tag are created for a nightly. [`retention.yaml`](../../.github/workflows/retention.yaml) prunes old GHCR nightly versions (keeps the newest 14 per package the current nightly publishes, named from the published `cozystack-packages:nightly` tree by [`hack/retention-packages.sh`](../../hack/retention-packages.sh), except packages linked to another repository, which are skipped with a warning; the floating `nightly`, release tags and untagged versions are never touched) and abandoned packages candidates. A candidate version is eligible only when all of these hold: it carries at least one tag and **every** tag on that manifest matches the strict `promotion-vX.Y.Z-from-vX.Y.Z-rc.N-run-<run-id>-<attempt>` form, so a manifest that has been retagged stable is excluded; it was last updated more than 30 days ago; its digest is not pinned by `packages/core/installer/values.yaml` at the head of an **open** pull request that is same-repository, authored by `cozystack-ci[bot]`, on a `release-X.Y.Z` branch and carrying the `release` label; and its digest is not pinned by that same file at the tip of `main` or of any `release-X.Y` branch — which is what keeps a candidate safe between the promote PR merging and finalize retagging it. Any such PR head whose pin is not exactly one `digest=sha256:<64 hex>` stops the candidate sweep for that run and fails the job, without affecting the nightly sweep.
|
||||
|
||||
Install a nightly:
|
||||
|
||||
```bash
|
||||
helm upgrade --install cozystack \
|
||||
oci://ghcr.io/cozystack/cozystack/cozy-installer --version 0.0.0-nightly.20260626.15992304129.1
|
||||
# ...or --version nightly for the latest
|
||||
```
|
||||
|
||||
The matching Talos node image is `ghcr.io/cozystack/cozystack/cozystack-nocloud:0.0.0-nightly.20260626.15992304129.1` (pull with `oras`).
|
||||
@@ -0,0 +1,82 @@
|
||||
## Patch Releases
|
||||
|
||||
A patch release now follows exactly the same shape as a [regular release](./regular-releases.md#regular-releases) — by the time `vX.Y.0` ships, everything is already happening on `release-X.Y`. The only difference is where the line starts:
|
||||
|
||||
* `release-X.Y` already exists, created by the freeze at `vX.Y.0-rc.1`, so there is no branch to create.
|
||||
* Patch commits are cherry-picked onto it, and the rc is cut by dispatching **from `release-X.Y`** — the same dispatch used for `rc.2` of the original release.
|
||||
|
||||
|
||||
Let's assume that we've released `v1.2.0` and that development is ongoing. We have introduced a couple of new features and some fixes to features that we have released in `v1.2.0`.
|
||||
|
||||
Once problems were found and fixed, a patch release is due.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "Release v1.2.0" tag: "v1.2.0"
|
||||
checkout main
|
||||
commit id: "feature 4"
|
||||
commit id: "patch 1"
|
||||
commit id: "feature 5"
|
||||
commit id: "patch 2"
|
||||
```
|
||||
|
||||
|
||||
1. Fixes reach the existing `release-1.2` branch from `main` through the backport batch, which cherry-picks what the [backport policy](./backports.md) admits and nothing else.
|
||||
|
||||
The batch runs twice a week, so a fix merged into `main` reaches the line with the next batch that can pick it cleanly, without anyone asking for it.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "Release v1.2.0" tag: "v1.2.0"
|
||||
checkout main
|
||||
commit id: "feature 4"
|
||||
commit id: "patch 1"
|
||||
commit id: "feature 5"
|
||||
commit id: "patch 2"
|
||||
checkout release-1.2
|
||||
cherry-pick id: "patch 1"
|
||||
cherry-pick id: "patch 2"
|
||||
```
|
||||
|
||||
Merge the open batch pull request first. Before the cut, `./backport-batch check --line release-1.2 --cut` (build it with `go build ./cmd/backport-batch`) has to exit 0: it exits 1 while a fix the rules take is not on the line, including one that is only on the open batch pull request, which the cut would not carry, and prints the URLs; it prints what the batch deferred as warnings. See [Cherry-pick triage before a patch](./backport-mechanics.md#cherry-pick-triage-before-a-patch).
|
||||
|
||||
2. The maintainer cuts a release candidate (`v1.2.1-rc.N`) via the [`Cut Pre-release Tag`](../../.github/workflows/cut-prerelease.yaml) workflow (manual dispatch **from `release-1.2`**), which tags that branch's `HEAD`. CI builds and publishes the rc, pushes the `release-1.2.1-rc.N` staging branch, and runs the mandatory full `rc-e2e` job against the published tag.
|
||||
3. Watch `rc-e2e` and wait for it to pass, then complete the other release checks. A successful manual `e2e-tag.yaml` run for `v1.2.1-rc.N` is also valid evidence if the latest `tags.yaml` run does not carry a green `rc-e2e`. **Optional:** press the `Generate RC Changelog` button ([`changelog-rc.yaml`](../../.github/workflows/changelog-rc.yaml)) to generate `docs/changelogs/v1.2.1.md` at rc time and bank it on the rc staging branch; promotion then reuses it instead of regenerating.
|
||||
4. Once the checks are green, run [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) (manual dispatch **from `main`**, as for every promotion, not from `release-1.2`) with that rc tag. It runs the same five jobs as a regular promotion: the dispatch first gates on the latest tag-push `rc-e2e` or a successful tag-correlated manual E2E run (`skip_e2e_gate` overrides it), verifies that `release-1.2` already carries the candidate-aware pipeline, then commits and publishes the rewritten packages tree under a run-specific temporary `promotion-*` tag, adds a second commit pinning the candidate on the `release-1.2.1` branch, drafts the `v1.2.1` release with the restamped assets, generates or picks up `docs/changelogs/v1.2.1.md` and commits it onto that branch, opens the parked `cozystack/website` docs PR (updating the existing `v1.2/` directory in place, since a patch does not promote `next/`), and opens the promote PR into `release-1.2` with only the `release` label. The promote PR body carries the same ✅/⚠️ E2E, changelog, and website-docs lines and runs no E2E by default; add `full-e2e` manually to force a repeat. As with a regular release every stable name is deferred to merge; retention protects the temporary candidate while this PR is open and, after it merges, while `release-1.2` still pins it — until finalize retags it stable.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "Release v1.2.0" tag: "v1.2.0"
|
||||
checkout main
|
||||
commit id: "feature 4"
|
||||
commit id: "patch 1"
|
||||
commit id: "feature 5"
|
||||
commit id: "patch 2"
|
||||
checkout release-1.2
|
||||
cherry-pick id: "patch 1"
|
||||
cherry-pick id: "patch 2" tag: "v1.2.1-rc.1"
|
||||
branch release-1.2.1
|
||||
commit id: "Prepare release v1.2.1"
|
||||
checkout release-1.2
|
||||
merge release-1.2.1 id: "Pull request"
|
||||
```
|
||||
|
||||
Finally, when release is confirmed, the release sequence goes on.
|
||||
|
||||
5. Maintainer reviews the PR against the [Phase 4 checklist](./ci-pipeline.md#phase-4--review-and-merge-the-promote-pr), confirms the E2E-verification and changelog lines and notes the parked website-docs line, and merges it — **do not squash-merge**. GitHub removes the merged branch `release-1.2.1` (auto-delete-on-merge is enabled).
|
||||
6. CI workflow triggers on merge (all consumer-visible or write-once side effects, after the rc E2E gate and PR review):
|
||||
1. Verifies the pinned packages candidate against the merged tree and the original rc artifact's container digests.
|
||||
2. Creates the tag `v1.2.1` at the newly created merge commit — write-once. The tag is published here for the first time, never moved.
|
||||
3. Cuts the write-once `api/apps/v1alpha1/v1.2.1` Go-module tag at the same commit.
|
||||
4. Publishes the release page (`draft` → `latest`; `latest` moves only if `v1.2.1` is the newest published stable).
|
||||
5. Retags the verified packages candidate and rc container images by digest to `v1.2.1` and publishes the stable `cozy-installer` chart — no container rebuild.
|
||||
7. Once the release is published, merge the parked `cozystack/website` docs PR, then announce the release to the community.
|
||||
@@ -0,0 +1,14 @@
|
||||
## Pre-release verification checklist
|
||||
|
||||
For RCs and final releases, run this before merging the release PR. Each item is here because something has shipped without it and broken users.
|
||||
|
||||
- [ ] **The published rc passed mandatory full E2E.** Confirm the promote PR says `✅ RC full e2e was verified`; the normal promote PR itself runs no E2E. Add `full-e2e` manually only when a repeat is required, and treat a ⚠️ bypass line as an explicit release-risk decision.
|
||||
- [ ] **Upgrade from previous patch** (`vX.Y.(Z-1)`) on a real cluster. `helm upgrade` succeeds, all HelmReleases reach Ready, no CRD schema rejections.
|
||||
- [ ] **Upgrade from previous minor** (`v(X-1).Y.0`) for RC and minor releases. Same assertions.
|
||||
- [ ] **At least one tenant with `publishing.ingressName != tenant-root`** and a real Let's Encrypt-staging cert issuance.
|
||||
- [ ] **Apply-mutate-reapply** on the kubernetes umbrella HelmRelease (catches second-reconcile bugs).
|
||||
- [ ] **For every CRD-source bump** (kubevirt, cert-manager, flux, cilium, kamaji): `helm template | kubectl apply --dry-run=server` against the freshly-installed CRDs.
|
||||
- [ ] **For every flux / helm-controller / kubevirt-operator / cert-manager bump:** paste the upstream breaking-changes section of the bump's CHANGELOG into the PR body before merge.
|
||||
- [ ] **After cutting the tag,** watch `pull-requests-release.yaml::Finalize Release`. If it fails on `Draft release for v... not found`, undraft the release manually and file the workflow regression — this has been a known recurring failure since the `Publish draft release` step lost its explicit `github-token` in refactor `66a756b6`.
|
||||
- [ ] **The promote PR carries `docs/changelogs/vX.Y.Z.md` and it was verified** per [Changelog pre-publish verification](./changelogs.md#changelog-pre-publish-verification). Finalize publishes this file as the release body, so verifying it before merge is verifying the release notes.
|
||||
- [ ] **The `cozystack/website` docs PR stays parked until publish.** Promotion opens it early; merging it before the GitHub release for `vX.Y.Z` is live 404s the new version's docs and API links. Merge it right after finalize publishes the release (regular-release step 6, patch step 7).
|
||||
@@ -0,0 +1,78 @@
|
||||
## Regular Releases
|
||||
|
||||
A regular release `vX.Y.0` is **promoted from a release-candidate that already passed e2e** — never tagged by hand and never rebuilt. The bytes shipped as `vX.Y.0` are the exact `vX.Y.0-rc.N` images, retagged by digest. We'll use `v1.2.0` as an example.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature"
|
||||
commit id: "feature 2"
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
```
|
||||
|
||||
A regular release sequence runs as follows:
|
||||
|
||||
1. Cut the first release candidate (`v1.2.0-rc.1`) from the last good commit on `main`: run the [`Cut Pre-release Tag`](../../.github/workflows/cut-prerelease.yaml) workflow (manual dispatch **from `main`**) with the tag. It tags `main`'s HEAD as the CI app, then **creates `release-1.2` at that commit, freezing the line**; `tags.yaml` then builds the rc images, publishes the rc prerelease, pushes the digest-vendored `release-1.2.0-rc.N` staging branch, and runs its mandatory `rc-e2e` job by calling [`e2e-tag.yaml`](../../.github/workflows/e2e-tag.yaml) against that published tag.
|
||||
2. Watch the rc's mandatory full E2E and wait for it to pass, then complete the other checks in [Pre-release verification checklist](./pre-release-checklist.md#pre-release-verification-checklist). If the latest `tags.yaml` run cannot provide a green `rc-e2e`, a successful manual dispatch of [`e2e-tag.yaml`](../../.github/workflows/e2e-tag.yaml) for the same tag is equivalent gate evidence. `main` is already open for the next minor, so anything landing there is **not** in this release until it reaches `release-1.2`: a fix merged into `main` gets there through the [backport batch](./backports.md#which-lines-the-batch-runs-for), which runs for `release-1.2` from the freeze on, and a fix with no counterpart on `main` through a maintainer's pull request against `release-1.2`. A later rc is cut by dispatching the workflow **from `release-1.2`**, and only once `./backport-batch check --line release-1.2 --cut` exits 0: merge the open batch pull request first, so that the rc carries every fix the policy takes. **Optional:** once the notes are worth reviewing, press the `Generate RC Changelog` button ([`changelog-rc.yaml`](../../.github/workflows/changelog-rc.yaml)) to generate `docs/changelogs/v1.2.0.md` now and bank it on the `release-1.2.0-rc.N` staging branch — so there is time to review and hand-edit it before promotion. Skipping the button changes nothing; promotion generates the changelog itself.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature"
|
||||
commit id: "feature 2"
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout main
|
||||
commit id: "feature 4 (next minor)"
|
||||
checkout release-1.2
|
||||
commit id: "backported fix" tag: "v1.2.0-rc.2"
|
||||
```
|
||||
|
||||
3. Once the checks are green, run the [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) workflow (manual dispatch **from `main`**, the only branch it is dispatched from, whichever line the rc belongs to) with that rc tag. Its `parse` job verifies E2E before any staging: it accepts a successful `rc-e2e` job in the latest `tags.yaml` push run for the tag, or any retained successful manual `e2e-tag.yaml` run whose expanded E2E job name targets the tag. The emergency `skip_e2e_gate` input bypasses this check, logs a loud warning, and marks the promote PR with ⚠️. Before its first registry write, promotion also proves that the selected `main` or `release-X.Y` target already contains the candidate-aware PR gate, finalizer and trusted verifier. A maintenance branch that lacks them gets them before its next patch is promoted, through the batch with `backport/include` on the pull request that brought them to `main`, or through a maintainer's pull request against the line. Promotion **creates no stable name before merge**: dispatch does write to the registry — it publishes one `promotion-*` packages candidate, and that tag is real and publicly listable — but it creates no `vX.Y.Z` tag, moves no `latest`, and publishes no stable installer, so nothing a consumer resolves changes until the PR merges. It:
|
||||
1. Rewrites the rc version substring to `1.2.0` and commits that exact tree, serializes its `packages/` directory under a run-specific temporary OCI tag, then adds a second commit pinning the candidate's immutable digest in the installer before pushing `release-1.2.0`. Container digests remain exactly the rc's; the two commits give the OCI revision an honest source SHA without pretending the artifact can contain its own digest.
|
||||
2. Drafts the `v1.2.0` release and uploads the (restamped) assets.
|
||||
3. Generates `docs/changelogs/v1.2.0.md` (in a parallel job, detached at the rc tag) — or, if you pressed the rc-time `Generate RC Changelog` button, copies the file already banked on the `release-1.2.0-rc.N` staging branch instead of regenerating — and commits it onto the `release-1.2.0` branch, so the changelog is reviewed as part of the promotion and finalize can publish the release with it already in place. Non-blocking: if generation fails the PR still opens, carrying a ⚠️ line that says the changelog is missing.
|
||||
4. Opens the `cozystack/website` "update managed apps reference" PR (`update-docs-v1.2.0` branch), generating the docs from the `release-1.2.0` staging branch via `FETCH_REF`. Non-blocking, and **do not merge that PR until `v1.2.0` is published** — merging early breaks the live site's version routing (see [Phase 3](./ci-pipeline.md#phase-3--update-website-docs-now-the-backstop)).
|
||||
5. Opens the `chore(release): promote v1.2.0-rc.N -> v1.2.0` PR into `release-1.2`, labelled `release` only. The PR body records whether E2E was verified or bypassed, plus ✅/⚠️ changelog and website-docs lines. E2E does not run on this promote PR by default because the immutable rc already passed the mandatory full suite; a maintainer may add `full-e2e` to run it again. The stable retag itself is deferred to the merge (step 5), so an abandoned promotion leaves no stable-named images and cannot wedge a re-promotion; retention protects the temporary candidate for as long as this release PR remains open, then may remove it after 30 days.
|
||||
|
||||
The PR targets the maintenance branch, not `main`: `promote-rc.yaml` prefers `release-X.Y` whenever it exists, and since the rc freeze it always does. The digest-vendored `Prepare release` commit therefore stays on the release line instead of landing on `main`.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout release-1.2
|
||||
commit id: "backported fix" tag: "v1.2.0-rc.2"
|
||||
branch release-1.2.0
|
||||
checkout release-1.2.0
|
||||
commit id: "Prepare release v1.2.0"
|
||||
commit id: "Pin promoted packages artifact"
|
||||
checkout release-1.2
|
||||
merge release-1.2.0 id: "Pull Request"
|
||||
```
|
||||
|
||||
4. Maintainer reviews the PR and the draft release against the [Phase 4 checklist](./ci-pipeline.md#phase-4--review-and-merge-the-promote-pr): confirm the E2E-verification line is ✅ (or explicitly accept its ⚠️ bypass), the changelog line is ✅, and note the website-docs line — the `cozystack/website` docs PR that promotion parked, which stays unmerged for now. E2E does not run on this promote PR by default. Then merge — **do not squash-merge**, the stable tag must attach to a real merge commit. GitHub removes the merged branch `release-1.2.0` (the repo has auto-delete-on-merge enabled).
|
||||
5. CI workflow triggers on merge (this is where every consumer-visible or write-once side effect happens, after the rc E2E gate and PR review):
|
||||
1. Pulls the pinned packages candidate by digest and verifies that it matches the merged `packages/` tree apart from the artifact's impossible self-reference, carries no rc image tags, and preserves the original rc artifact's container `repo@digest` set.
|
||||
2. Creates the tag `v1.2.0` at the newly created merge commit — write-once. The tag is published here for the first time, never moved.
|
||||
3. Cuts the write-once `api/apps/v1alpha1/v1.2.0` Go-module tag at the same commit.
|
||||
4. Ensures the `release-1.2` maintenance branch exists at the tag commit. Since the rc freeze created it, this is normally a no-op fast-forward: the merge commit is already its tip.
|
||||
5. Publishes the release page (`draft` → `latest`), using `docs/changelogs/v1.2.0.md` from the merge commit as the release body.
|
||||
6. Retags the verified packages candidate and rc container images by digest to `v1.2.0` (and `:latest` only when `v1.2.0` is the newest published stable), and publishes the stable `cozy-installer` chart — no container rebuild.
|
||||
6. Once the release is published, merge the parked `cozystack/website` docs PR (opened back at promote time, step 3.iv) so the live site's version routing flips to `v1.2.0` — its versioned docs and `openapi.json` are on GitHub now. The maintainer can then announce the release to the community.
|
||||
|
||||
```mermaid
|
||||
gitGraph
|
||||
commit id: "feature 2"
|
||||
commit id: "feature 3" tag: "v1.2.0-rc.1"
|
||||
branch release-1.2
|
||||
checkout main
|
||||
commit id: "feature 4 (next minor)"
|
||||
checkout release-1.2
|
||||
commit id: "backported fix" tag: "v1.2.0-rc.2"
|
||||
branch release-1.2.0
|
||||
checkout release-1.2.0
|
||||
commit id: "Prepare release v1.2.0"
|
||||
checkout release-1.2
|
||||
merge release-1.2.0 id: "Release v1.2.0" tag: "v1.2.0"
|
||||
```
|
||||
|
||||
Note that `main` never receives the release commit. It forked away at the freeze and carries the next minor's work.
|
||||
@@ -0,0 +1,18 @@
|
||||
## Release Candidates
|
||||
|
||||
Release candidates are Cozystack versions that introduce new features and are published before a stable release. Their purpose is to help validate stability before finalizing a new feature release.
|
||||
|
||||
Release candidates are given numbers `vX.Y.0-rc.N`, for example, `v1.2.0-rc.1`. The first one is cut from `main`, once all major features for the upcoming release have merged and the release enters its testing phase.
|
||||
|
||||
**Cutting the first rc freezes the line.** [`cut-prerelease.yaml`](../../.github/workflows/cut-prerelease.yaml) creates `release-X.Y` at the tagged commit, and from that point the release's content is closed:
|
||||
|
||||
- Every later cut for that line — `rc.2`, `rc.3`, an `alpha`/`beta`, or a patch-line rc — must be dispatched from `release-X.Y`. A dispatch from `main` is refused.
|
||||
- Fixes reach the release only through the backport batch onto `release-X.Y`, or a maintainer's pull request against it (see [Backports](./backports.md)). The batch runs for the line from its next scheduled run on, [while it has not shipped](./backports.md#which-lines-the-batch-runs-for): it takes the fixes merged into `main` since the freeze, and an older line takes each of them only once this line's batch that carries it has merged.
|
||||
- A later rc is cut only once `backport-batch check --line release-X.Y --cut` exits 0, so that it carries every fix the batch takes: the open batch pull request merges first.
|
||||
- `main` reopens immediately for the next minor, so feature work never has to wait for the release to ship.
|
||||
|
||||
This is what makes an rc mean something. Previously `release-X.Y` was created only when the stable release merged, so `rc.2` was cut from `main`'s tip and silently absorbed everything that had landed since `rc.1` — the shipped release could contain code that no rc had ever validated.
|
||||
|
||||
**Un-freezing is manual and deliberate.** If an `rc.1` turns out to be unusable and the line has to be re-cut from `main`, delete `release-X.Y` by hand first; the refuse step blocks every other route back to a `main` dispatch, which is the point of it. Nothing automates the deletion, and nothing should — throwing away a frozen line is a decision worth making explicitly.
|
||||
|
||||
Each RC contributes to a cumulative set of release notes that will be finalized when `vX.Y.0` is released. After testing, if no critical issues remain, the last good rc is **promoted** to the regular release `vX.Y.0` — the exact rc images retagged by digest, never rebuilt and never tagged from a later commit (see [Regular Releases](./regular-releases.md#regular-releases)). Promotion cuts the write-once `vX.Y.0` tag at the promote PR's merge commit, fast-forwards the `release-X.Y` maintenance branch the freeze already created, and opens the way for patch releases.
|
||||
@@ -0,0 +1,35 @@
|
||||
## Release-time fires — what's gone wrong recently
|
||||
|
||||
These are real regressions that escaped to users. The pattern in each: code passed CI, shipped in a stable release, broke customers. Read them as a working list of pre-release verification gaps.
|
||||
|
||||
### v1.3.0 — cert-manager ingressClassName regression (fix #2562)
|
||||
|
||||
**Bug:** A commit migrated ACME HTTP-01 to use the modern `acme.cert-manager.io/http01-ingress-ingressclassname` annotation. The matching ingress-shim code that reads that annotation was added by cert-manager upstream PR #8244 and was **never backported** to the `release-1.19` branch — but cozystack 1.3.0 shipped cert-manager v1.19.3. The annotation was silently dropped at runtime and every solver Ingress fell back to the ClusterIssuer's default class `tenant-root`. Let's Encrypt validation broke for every tenant whose `publishing.ingressName != tenant-root`.
|
||||
|
||||
**Blast radius:** v1.3.0, v1.3.1, and v1.1.7 (the migration was cherry-picked to `release-1.1`).
|
||||
|
||||
**Why CI missed it:** the migration commit was self-contained and `helm template` rendered cleanly. There was no E2E that creates a tenant with a non-default ingress class and watches a real Let's Encrypt cert issue. Verification had been done from the docs side ("the API has been available since cert-manager 1.12"), not against the shipped cert-manager source.
|
||||
|
||||
**Would have caught it:** non-root-tenant E2E with LE-staging cert issuance, asserting the solver Ingress lands on the tenant's class.
|
||||
|
||||
### v1.4.0 — kubevirt-instancetypes upgrade failure (fix #2612)
|
||||
|
||||
**Bug:** `packages/system/kubevirt-instancetypes/Makefile` contained `sed -i '/persistent: true/d' templates/preferences.yaml`, which stripped the only child of `preferredTPM`, leaving `preferredTPM:` (null) in six Windows preferences. KubeVirt v1.6.x silently accepted it. The v1.6.3 → v1.8.2 operator bump (2026-04-27) hardened the CRD's OpenAPI schema and started rejecting null for an object-typed field. `helm upgrade` then failed on every cluster with `kubevirt-instancetypes` enabled.
|
||||
|
||||
**Why CI missed it:** the kubevirt-operator bump and the inert `preferences.yaml` were two weeks apart in commit history. CI only tests fresh installs against the new CRDs; it doesn't run `helm upgrade` from prior versions. Schema regression on a vendored asset that hadn't been touched in 18 months.
|
||||
|
||||
**Would have caught it:** `helm template <new chart> | kubectl apply --dry-run=server -f -` against the freshly-installed CRDs of every bumped operator. Or an upgrade-from-N-1 E2E lane.
|
||||
|
||||
### v1.4.0 — Flux 2.7 → 2.8 readiness deadlock (#2602)
|
||||
|
||||
**Bug:** helm-controller v1.5.0 (shipped in Flux 2.8) changes the default wait strategy to kstatus polling, which polls **every applied resource including child HelmReleases**. The `packages/apps/kubernetes` umbrella has 19 child HRs that each `dependsOn: parent` (with a `lookup` guard so the dependency only activates after the parent exists). Pre-2.8: parent went Ready as soon as Helm returned. Post-2.8: parent waits for children, children wait for parent. Deadlock.
|
||||
|
||||
**Why CI missed it:** the cycle only manifests on the second reconcile (parent exists, so the `lookup` activates the child's `dependsOn`). Fresh-install E2E dodges this — the parent-already-exists branch isn't taken on the first apply.
|
||||
|
||||
**Would have caught it:** apply-mutate-reapply E2E (any chart bump in the kubernetes umbrella triggers a second reconcile). Plus an explicit PR-template requirement: for any flux/helm-controller bump, paste the upstream breaking-changes section into the PR body.
|
||||
|
||||
### Patterns these share
|
||||
|
||||
1. **Fresh-install CI is insufficient.** All three regressions passed fresh-install lanes.
|
||||
2. **Upstream breaking-defaults bumps are the common shape.** kubevirt CRD hardening, helm-controller default wait strategy, cert-manager API surface drift. Pre-merge reading of upstream changelogs would have caught two of three.
|
||||
3. **One-engineer-bitten-once is the only safety net for live verification.** The cert-manager bump PR included an explicit live-cluster verification step — because the same engineer had been bitten the prior week. There is no checklist requiring it.
|
||||
@@ -0,0 +1,29 @@
|
||||
## Release signatures and SBOM
|
||||
|
||||
A release cut from a tree that has [`hack/sign-release-assets.sh`](../../hack/sign-release-assets.sh) carries three files beside its other assets, which [`hack/upload-assets.sh`](../../hack/upload-assets.sh) has that script write before it uploads anything. Releases cut before it have none of them, and neither does a maintenance line until the script is backported to it (see below).
|
||||
|
||||
- `cozystack-source-sbom.spdx.json` is an SPDX SBOM, made with syft, of the committed source tree (the Go modules, npm lockfiles, GitHub Actions references and other manifests syft reads in it) and of the `cozypkg` binaries in the `cozypkg-*.tar.gz` assets. It does not describe what runs in the cluster: the container images carry their own SBOMs, see [SBOM of the container images](#sbom-of-the-container-images), and the Talos installer image, ISO, disk, kernel and initramfs images have none. Its name says what it covers, so that `cozystack-sbom` stays free for an SBOM of the release as a whole.
|
||||
- `cozystack-checksums.txt` holds the SHA-256 of every other asset and of the SBOM.
|
||||
- `cozystack-checksums.txt.sigstore.json` is a Sigstore bundle with a keyless cosign signature over the checksums. Its certificate names the workflow that published the release: `tags.yaml` for a pre-release, `promote-rc.yaml` for a stable one.
|
||||
|
||||
One signature covers every asset through the checksums. To check a download, verify the checksums, then the file against them, with cosign v3.1.3 or later (v2.6.5 or later on the v2 line):
|
||||
|
||||
```sh
|
||||
cosign verify-blob cozystack-checksums.txt \
|
||||
--bundle cozystack-checksums.txt.sigstore.json \
|
||||
--certificate-identity-regexp '^https://github\.com/cozystack/cozystack/\.github/workflows/(tags\.yaml@refs/tags/v|promote-rc\.yaml@refs/heads/main$)' \
|
||||
--certificate-oidc-issuer https://token.actions.githubusercontent.com &&
|
||||
sha256sum --check --ignore-missing cozystack-checksums.txt
|
||||
```
|
||||
|
||||
Older cosign accepts a legacy-format bundle that carries a bare public key in place of a certificate and skips the identity and issuer checks for it ([GHSA-fx35-mq7g-6g98](https://github.com/sigstore/cosign/security/advisories/GHSA-fx35-mq7g-6g98)), so a bundle swapped in next to the download, by a mirror or a proxy, passes there. The bundles this repository writes are in the new format, but cosign picks the format from the file, and `--new-bundle-format` does not stop that on the v3 line (v3.1.2 still accepts such a bundle with it), so the cosign version is what protects the check.
|
||||
|
||||
The regexp pins the ref each workflow runs from: `tags.yaml` on a `v*` tag push, and `promote-rc.yaml` dispatched from `main`, the only branch it is dispatched from. A passing check proves that the checksums were signed by one of those runs in `cozystack/cozystack`. A `promote-rc.yaml` dispatched from another branch fails this check in its own upload step, with no asset uploaded, and an edited `promote-rc.yaml` has to be merged into `main` before it can sign. Until the `v*` [tag rulesets](./tag-immutability.md#repo-side-enforcement-tag-protection-rulesets) restrict tag creation to the CI app, though, anyone with write access can push a `v*` tag whose tree carries an edited `tags.yaml`, and its signature passes.
|
||||
|
||||
The script runs the same `cosign verify-blob` before it returns, so a signature this check would reject fails the job before the first upload, and the release is not published with assets but without a signature. Both jobs that call `make upload_assets`, `tags.yaml::prepare-release` and `promote-rc.yaml::promote`, hold `id-token: write` for the signing identity and install syft and cosign v3.1.3, the version users are asked for above. They install syft from its release tarball, checked against a SHA-256 committed in the workflow, and not with `anchore/sbom-action/download-syft`, which runs an install script fetched at run time in a job that holds the release credentials; [`hack/ci-tool-installs-contract.bats`](../../hack/ci-tool-installs-contract.bats) keeps it that way. A stable release gets its own three files rather than its rc's: promotion restamps several assets, so the checksums are recomputed and signed again.
|
||||
|
||||
`promote-rc.yaml` runs the `hack/upload-assets.sh` of the rc tree, not the dispatch ref's. A stable release is therefore signed only when its rc was tagged from a tree that has `hack/sign-release-assets.sh`, and a maintenance line starts carrying signatures once that script is backported to its `release-X.Y` branch.
|
||||
|
||||
### SBOM of the container images
|
||||
|
||||
The container images a release pins by digest each carry an SPDX SBOM of the packages in them, which BuildKit attaches at build time as an attestation (`SBOM=1` in `tags.yaml`, scanner and flags in [`hack/common-envs.mk`](../../hack/common-envs.mk)). The attestation is an entry of the image's index, so the digest the release pins names both, and promotion copies it with the image. Read it with `docker buildx imagetools inspect <image>@<digest> --format '{{ json .SBOM }}'`, which prints one document per platform. The attestations are not signed. A release cut before `SBOM=1` was set has none, and neither has the Talos installer image, which the imager builds and `skopeo` pushes, nor the third-party images `packages/` vendors by digest.
|
||||
@@ -0,0 +1,10 @@
|
||||
## Splitting a release-blocking bundle PR
|
||||
|
||||
Sometimes the work that has to land before a release is a 40-commit grab bag (CI stabilization, dependency bumps, chart fixes). Splitting it makes it reviewable and survivable. The strategy that has worked:
|
||||
|
||||
1. **Verify what's already on `main`.** Use `git merge-base --is-ancestor` and subject grep to drop commits that have already landed.
|
||||
2. **Find existing open PRs that cover the same commits.** Force-push to the existing branch instead of opening a duplicate — preserves review threads.
|
||||
3. **Identify the "bottleneck" PR.** The one whose changes make previously-silent misconfigurations into hard errors (typically an operator-version bump). Everything else can fan out in parallel.
|
||||
4. **Squash iterative same-author bug-on-bug fixes** inside each split. Preserve authorship via `--author` on the squashed commit.
|
||||
5. **Mark the keystone PR as draft** and link in the body to the dependency PRs. Reviewers know it intentionally fails CI until the chain lands.
|
||||
6. **Beware of concurrent registry pushes.** When fanning out 5+ PR CI runs simultaneously, multiple `make image` jobs can collide on shared floating tags. The `IMAGE_TAG=pr-<N>-<sha>` refactor solved this; if you see `Conflicted with another upload of the same manifest`, `gh run rerun --failed`.
|
||||
@@ -0,0 +1,7 @@
|
||||
## Stable tags come from rc promotion
|
||||
|
||||
A stable `vX.Y.Z` is created only by **promoting an existing release-candidate** that already passed e2e — never by rebuilding and never by a cron.
|
||||
|
||||
- [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) is triggered manually once an rc has gone green. It rewrites the rc version substring in the vendored image tags (the container digests stay the rc's), commits and publishes the resulting declarative packages tree under a unique run-specific `promotion-*` tag, commits that digest pin, and opens the `release-X.Y.Z` staging PR. The PR gate verifies the candidate against the prospective merge and original rc artifact; finalize repeats that check before creating the write-once stable tag at the merge commit, then retags the verified packages artifact and rc's already-built, e2e-passed container digests to the stable tag **by digest** (no container rebuild — see [`hack/promote-retag.sh`](../../hack/promote-retag.sh)) and publishes the release. An abandoned promotion leaves no stable-named bytes; its temporary candidate is retention-managed.
|
||||
|
||||
So a commit on a supported `release-X.Y` line ships when a maintainer promotes the next rc for that line — not automatically within 24h. Pushing a stable `vX.Y.Z` tag by hand is **not** a supported path: `tags.yaml` fails fast on a stable tag that has no pre-existing draft ("stable tags come from promote-rc.yaml"), and even so the finalize step would refuse to move a pre-existing tag. Pre-releases (`vX.Y.Z-rc.N`, `-alpha.N`, `-beta.N`) are cut with the [`Cut Pre-release Tag`](../../.github/workflows/cut-prerelease.yaml) workflow, which pushes the tag as the CI app; [`tags.yaml`](../../.github/workflows/tags.yaml) then fires on that push, builds it, and publishes the pre-release. Cutting via the workflow (rather than a manual `git push`) is what lets repo admins lock `v*` tag creation to the CI app — see the tag-protection note below.
|
||||
@@ -0,0 +1,18 @@
|
||||
## Tag immutability
|
||||
|
||||
Published tags are **write-once** — once a `vX.Y.Z` or rc tag is pushed it is never moved or deleted. (Nightlies are not git tags at all — they are GHCR OCI tags, see [Nightly builds](./nightly-builds.md#nightly-builds).) Moving a tag silently poisons the Go module proxy / pkg.go.dev cache for `api/apps/v1alpha1/vX.Y.Z` and drifts SBOM/provenance toolchains, so the release flow is built so a move is impossible by construction:
|
||||
|
||||
| File | Tag / branch handling |
|
||||
|------|------------------------|
|
||||
| [`pull-requests-release.yaml`](../../.github/workflows/pull-requests-release.yaml) | Verifies the pinned packages candidate before creating a stable name. Creates `vX.Y.Z` at the PR merge commit **write-once** (create if absent, no-op if unchanged, fail loudly if it would move — the merge commit is new, so there is nothing to move). Cuts the `api/apps/v1alpha1/<vTAG>` Go-submodule tag write-once at the same commit, **stable only** (never rc/beta/alpha). The `release-X.Y` maintenance branch is fast-forward-only. Retags the verified packages artifact and rc container digests to the stable image tag (also write-once at the image level). |
|
||||
| [`tags.yaml`](../../.github/workflows/tags.yaml) | The `release-X.Y.Z-rc.N` staging branch is a mutable staging ref (compare-before-force: skipped when unchanged, force+log only when it genuinely moves). Fails fast on a stable `vX.Y.Z` tag with no pre-existing draft — stable tags come from `promote-rc.yaml`, never a hand push. |
|
||||
| [`promote-rc.yaml`](../../.github/workflows/promote-rc.yaml) | Mutates no stable tags. Before staging, verifies green full E2E from the latest rc tag-push run or a tag-correlated manual published-tag run (unless loudly bypassed). Commits and publishes the rewritten packages tree under a run-specific temporary `promotion-*` tag, adds a second commit pinning its digest on the `release-X.Y.Z` branch, generates the changelog there, and opens the promote PR with `release` but no automatic `full-e2e`; the PR gate verifies the candidate and finalize repeats that verification before the stable tag and image retag. The staging branch is force-pushed on re-dispatch, but an existing `docs/changelogs/vX.Y.Z.md` is preserved across the rebuild. |
|
||||
|
||||
This is the immutable-tag + rc-promotion model from [#2677](https://github.com/cozystack/cozystack/issues/2677): stable `vX.Y.Z` is created only by promoting an existing rc, rc tags are write-once, and `api/apps/v1alpha1/vX.Y.Z` is created only on a stable release. The old nightly `auto-release.yaml` (which delete-recreated auto-bumped patch tags) has been removed. rc and stable tags accrete permanently; the only churning artifacts are GHCR nightlies and abandoned temporary promotion candidates, which [`retention.yaml`](../../.github/workflows/retention.yaml) prunes (see [Nightly builds](./nightly-builds.md#nightly-builds)) — GHCR storage is the cost vector to watch.
|
||||
|
||||
### Repo-side enforcement (tag-protection rulesets)
|
||||
|
||||
The guarantees above are enforced in the workflows. Repo admins back them with two GitHub **tag rulesets** targeting `v*`:
|
||||
|
||||
- **Immutability** — *Restrict updates* + *Restrict deletions*, **no bypass**. No actor (not even the CI app) can move or delete a shipped `v*` tag. Safe to enable at any time: nothing in the release flow ever moves or deletes a `v*` tag, it only creates new ones.
|
||||
- **Creation control** — *Restrict creations* with the **CI app as the sole bypass actor**, plus *Limit branches/tags updated in a single push* = 1. This turns "don't hand-push tags" from convention into hard enforcement: every `v*` tag — pre-release **and** stable — can then be created only by CI. Enable this **only after** [`cut-prerelease.yaml`](../../.github/workflows/cut-prerelease.yaml) is in use, since that workflow is what makes CI the sole creator of pre-release tags; enabling it earlier would lock maintainers out of cutting rc/alpha/beta. (The Go-module tag `api/apps/v1alpha1/*` gets an equivalent pair of rulesets.)
|
||||
@@ -40,11 +40,7 @@ One person may hold more than one role on a small incident.
|
||||
4. **Private fix development.** The fix is developed under a **GitHub Security
|
||||
Advisory** (private fork) so that code and discussion stay confidential until
|
||||
release.
|
||||
5. **Coordinated release.** The fix is shipped through the normal patch-release
|
||||
and backport process. Which supported lines receive the backport is decided
|
||||
per fix, weighing severity against backport risk — see the *Supported
|
||||
Versions* table in `SECURITY.md`, which also records that no documented rule
|
||||
governs that choice.
|
||||
5. **Coordinated release.** The fix is shipped through the normal patch-release and backport process: the backport batch carries it onto the supported lines, the two latest minor releases, newest first, by the rules in [`docs/release/backports.md`](../release/backports.md), which a maintainer can override for a single fix. See the *Supported Versions* table in `SECURITY.md`.
|
||||
6. **Disclosure.** A GHSA — and, when applicable, a CVE — is published. Public
|
||||
disclosure follows the outbound channels: GitHub Releases and release notes,
|
||||
the changelog, and the GHSA itself. The reporter is credited unless anonymity
|
||||
|
||||
@@ -71,9 +71,6 @@ The script:
|
||||
had already replayed. The Pod log names each topic before its replay and
|
||||
again once it landed; empty or delete every topic it shows as started, not
|
||||
only the one the refusal names, then re-run.
|
||||
- The tarball in object storage is not managed by the Cozystack `Backup`
|
||||
lifecycle: deleting the `Backup` CR removes the reference but leaves the S3
|
||||
object in place. Tenants who need to reclaim space should delete the object
|
||||
(or set a bucket lifecycle policy).
|
||||
- The tarball in object storage is not managed by the Cozystack `Backup` lifecycle: deleting the `Backup` CR removes the reference but leaves the S3 object in place. Tenants who need to reclaim space should delete the object, or set a lifecycle rule on the bucket. A Simple or MultiZone Cozystack SeaweedFS applies such rules once a day only while its `lifecycle.enabled` is on, which is not the default; a Client instance applies none itself, so the SeaweedFS it connects to must have `lifecycle.enabled` on. Another object store applies them only if it implements bucket lifecycle.
|
||||
- Consumer-group offsets, topic configs, ACLs and users are not part of the
|
||||
backup; consumers start from the restored topic's fresh offsets.
|
||||
@@ -55,9 +55,6 @@ The script:
|
||||
|
||||
- Both flows rely on the strategy template branching on `.Mode`. See
|
||||
`01-create-strategy.sh`.
|
||||
- The tarball in object storage is not managed by the Cozystack `Backup`
|
||||
lifecycle: deleting the `Backup` CR removes the reference but leaves the S3
|
||||
object in place. Tenants who need to reclaim space should delete the object
|
||||
(or set a bucket lifecycle policy).
|
||||
- The tarball in object storage is not managed by the Cozystack `Backup` lifecycle: deleting the `Backup` CR removes the reference but leaves the S3 object in place. Tenants who need to reclaim space should delete the object, or set a lifecycle rule on the bucket. A Simple or MultiZone Cozystack SeaweedFS applies such rules once a day only while its `lifecycle.enabled` is on, which is not the default; a Client instance applies none itself, so the SeaweedFS it connects to must have `lifecycle.enabled` on. Another object store applies them only if it implements bucket lifecycle.
|
||||
- `nats stream restore` recreates the stream's configuration from the snapshot;
|
||||
consumers are restored as captured at backup time.
|
||||
@@ -16,6 +16,7 @@ require (
|
||||
github.com/go-logr/logr v1.4.4
|
||||
github.com/go-logr/zapr v1.3.0
|
||||
github.com/go-task/slim-sprig/v3 v3.0.0
|
||||
github.com/google/cel-go v0.29.2
|
||||
github.com/minio/minio-go/v7 v7.0.98
|
||||
github.com/onsi/ginkgo/v2 v2.33.0
|
||||
github.com/onsi/gomega v1.43.1
|
||||
@@ -83,7 +84,6 @@ require (
|
||||
github.com/go-openapi/testify/enable/yaml/v2 v2.8.0 // indirect
|
||||
github.com/go-openapi/testify/v2 v2.8.0 // indirect
|
||||
github.com/golang/protobuf v1.5.4 // indirect
|
||||
github.com/google/cel-go v0.29.2 // indirect
|
||||
github.com/google/gnostic-models v0.7.1 // indirect
|
||||
github.com/google/go-cmp v0.7.0 // indirect
|
||||
github.com/google/pprof v0.0.0-20260906184651-6331bc6350fe // indirect
|
||||
|
||||
@@ -0,0 +1,186 @@
|
||||
#!/usr/bin/env bats
|
||||
# Tests for hack/backport-batch-scheduled.sh, the scheduled backport batch, and
|
||||
# for the workflow that runs it, .github/workflows/backport-batch.yaml.
|
||||
#
|
||||
# The backport-batch binary is stubbed through BACKPORT_BATCH: every call is
|
||||
# recorded with the tokens gh would see, `lines` answers with $STUB_LINES, and
|
||||
# a subcommand fails for the line named in $STUB_FAIL_<subcommand>.
|
||||
#
|
||||
# Run with: bats hack/backport-batch-scheduled.bats
|
||||
|
||||
load test_helper
|
||||
|
||||
SCRIPT="${BATS_TEST_DIRNAME}/backport-batch-scheduled.sh"
|
||||
WF="${BATS_TEST_DIRNAME}/../.github/workflows/backport-batch.yaml"
|
||||
|
||||
setup() {
|
||||
strict_setup
|
||||
export TMPDIR="$BATS_TEST_TMPDIR"
|
||||
export CALLS="${BATS_TEST_TMPDIR}/calls"
|
||||
export BACKPORT_BATCH="${BATS_TEST_TMPDIR}/backport-batch"
|
||||
cat > "$BACKPORT_BATCH" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
printf 'GH_TOKEN=%s GITHUB_TOKEN=%s %s\n' "${GH_TOKEN-unset}" "${GITHUB_TOKEN-unset}" "$*" >> "$CALLS"
|
||||
line=${3:-}
|
||||
case "$1" in
|
||||
lines) printf '%s\n' $STUB_LINES ;;
|
||||
run) [ "${STUB_FAIL_run:-}" != "$line" ] || exit 2 ;;
|
||||
check) [ "${STUB_FAIL_check:-}" != "$line" ] || exit 1 ;;
|
||||
notify)
|
||||
[ "${STUB_FAIL_notify:-}" != "$line" ] || exit 2
|
||||
echo "$line: something waits"
|
||||
;;
|
||||
esac
|
||||
EOF
|
||||
chmod +x "$BACKPORT_BATCH"
|
||||
export STUB_LINES="release-1.7 release-1.6"
|
||||
unset STUB_FAIL_run STUB_FAIL_check STUB_FAIL_notify
|
||||
unset BACKPORT_ALERT_TEAM GITHUB_STEP_SUMMARY GH_TOKEN
|
||||
export GITHUB_TOKEN=job-token
|
||||
export BACKPORT_BATCH_TOKEN=batch-token
|
||||
: > "$CALLS"
|
||||
}
|
||||
|
||||
# The calls recorded, without the token fields.
|
||||
calls() {
|
||||
sed 's/^GH_TOKEN=[^ ]* GITHUB_TOKEN=[^ ]* //' "$CALLS"
|
||||
}
|
||||
|
||||
@test "runs, checks and reports each line, newest first, pushing with the batch's token alone" {
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(calls)" = "lines
|
||||
run --line release-1.7 --continue --push
|
||||
check --line release-1.7
|
||||
notify --line release-1.7 --push
|
||||
run --line release-1.6 --continue --push
|
||||
check --line release-1.6
|
||||
notify --line release-1.6 --push" ]
|
||||
# Every call sees the batch's token, and the job's token is gone.
|
||||
[ "$(grep -c '^GH_TOKEN=batch-token GITHUB_TOKEN=unset ' "$CALLS")" -eq 7 ]
|
||||
}
|
||||
|
||||
@test "a dry run pushes, opens and comments nothing, and needs no token" {
|
||||
unset BACKPORT_BATCH_TOKEN
|
||||
run "$SCRIPT" --dry-run
|
||||
[ "$status" -eq 0 ]
|
||||
if grep -q -- '--push' "$CALLS"; then
|
||||
echo "a dry run passed --push:"; cat "$CALLS"
|
||||
false
|
||||
fi
|
||||
[ "$(calls | grep -c '^run --line release-.* --continue$')" -eq 2 ]
|
||||
[ "$(calls | grep -c '^notify --line release-.*[0-9]$')" -eq 2 ]
|
||||
}
|
||||
|
||||
@test "--no-workflow-files reaches run alone, and an unknown argument stops the script" {
|
||||
run "$SCRIPT" --no-workflow-files
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(calls | grep -c '^run --line release-.* --continue --no-workflow-files --push$')" -eq 2 ]
|
||||
if calls | grep -v '^run ' | grep -q -- '--no-workflow-files'; then
|
||||
echo "the flag reached another command:"; calls
|
||||
false
|
||||
fi
|
||||
|
||||
: > "$CALLS"
|
||||
run "$SCRIPT" --dry-run --no-workflow-files
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(calls | grep -c '^run --line release-.* --continue --no-workflow-files$')" -eq 2 ]
|
||||
|
||||
: > "$CALLS"
|
||||
run "$SCRIPT" --no-workflows
|
||||
[ "$status" -eq 2 ]
|
||||
[ ! -s "$CALLS" ]
|
||||
}
|
||||
|
||||
@test "without BACKPORT_BATCH_TOKEN it stops before anything, even with the job's token" {
|
||||
unset BACKPORT_BATCH_TOKEN
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 2 ]
|
||||
[[ "$output" == *"BACKPORT_BATCH_TOKEN is empty"* ]]
|
||||
[ ! -s "$CALLS" ]
|
||||
}
|
||||
|
||||
@test "outside a worktree of the repository it stops before anything" {
|
||||
export GIT_CEILING_DIRECTORIES="$BATS_TEST_TMPDIR"
|
||||
mkdir "$BATS_TEST_TMPDIR/elsewhere"
|
||||
cd "$BATS_TEST_TMPDIR/elsewhere"
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 2 ]
|
||||
[[ "$output" == *"not in a worktree of the repository"* ]]
|
||||
[ ! -s "$CALLS" ]
|
||||
}
|
||||
|
||||
@test "the team is mentioned only when BACKPORT_ALERT_TEAM is set" {
|
||||
export BACKPORT_ALERT_TEAM=cozystack/maintainers
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(calls | grep -c '^notify --line release-.* --mention cozystack/maintainers --push$')" -eq 2 ]
|
||||
|
||||
export BACKPORT_ALERT_TEAM=
|
||||
: > "$CALLS"
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
if grep -q -- '--mention' "$CALLS"; then
|
||||
echo "an empty team was mentioned:"; cat "$CALLS"
|
||||
false
|
||||
fi
|
||||
}
|
||||
|
||||
@test "a line that fails goes into the summary, the other line still runs, and the job fails" {
|
||||
export GITHUB_STEP_SUMMARY="${BATS_TEST_TMPDIR}/summary"
|
||||
export STUB_FAIL_run=release-1.7 STUB_FAIL_check=release-1.6
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
[ "$(calls | grep -c '^run --line release-1.6 ')" -eq 1 ]
|
||||
grep -qx -- '- release-1.7: the run failed with exit 2' "$GITHUB_STEP_SUMMARY"
|
||||
grep -qx -- '- release-1.6: the check exited 1; see its output above' "$GITHUB_STEP_SUMMARY"
|
||||
grep -qx -- '- release-1.7: something waits' "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Each failure fails the job on its own.
|
||||
export STUB_FAIL_run=release-1.7 STUB_FAIL_check=
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
export STUB_FAIL_run= STUB_FAIL_check=release-1.6
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
export STUB_FAIL_check= STUB_FAIL_notify=release-1.6
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 1 ]
|
||||
export STUB_FAIL_notify=
|
||||
run "$SCRIPT"
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "the workflow runs Monday and Thursday as the cozystack-ci App and never with the job's token" {
|
||||
[ "$(yq -r '.on.schedule[0].cron' "$WF")" = "23 6 * * 1,4" ]
|
||||
[ "$(yq -r '.on.schedule | length' "$WF")" -eq 1 ]
|
||||
[ "$(yq -o=json -I=0 '.permissions' "$WF")" = "{}" ]
|
||||
[ "$(yq -o=json -I=0 '.jobs.batch.permissions' "$WF")" = '{"contents":"read"}' ]
|
||||
[ "$(yq -r '.jobs.batch.steps[] | select(.uses == "actions/checkout*") | .with."persist-credentials"' "$WF")" = "false" ]
|
||||
run_step='.jobs.batch.steps[] | select(.name == "Run the batch")'
|
||||
[ "$(yq -r "$run_step | .run" "$WF")" = "hack/backport-batch-scheduled.sh --no-workflow-files" ]
|
||||
[ "$(yq -r "$run_step | .env.BACKPORT_BATCH_TOKEN" "$WF")" = '${{ steps.app-token.outputs.token }}' ]
|
||||
[ "$(yq -r "$run_step | .env.BACKPORT_BATCH_BOT" "$WF")" = '${{ steps.app-token.outputs.app-slug }}[bot]' ]
|
||||
|
||||
# The App token covers this repository alone, with Contents and Pull
|
||||
# requests and nothing else: no workflows, no packages.
|
||||
mint='.jobs.batch.steps[] | select(.id == "app-token")'
|
||||
[ "$(yq -r "$mint | .uses" "$WF")" = "actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547" ]
|
||||
[ "$(yq -r "$mint | .with.owner" "$WF")" = "cozystack" ]
|
||||
[ "$(yq -r "$mint | .with.repositories" "$WF")" = "cozystack" ]
|
||||
[ "$(yq -r "$mint | .with | to_entries | map(select(.key | test(\"^permission-\"))) | map(.key + \"=\" + .value) | join(\",\")" "$WF")" = "permission-contents=write,permission-pull-requests=write" ]
|
||||
# A token minted after the checkout lives an hour; the job ends before.
|
||||
[ "$(yq -r '.jobs.batch."timeout-minutes"' "$WF")" -le 45 ]
|
||||
|
||||
# The commits carry the App's bot, read from the token, not a person.
|
||||
identity='.jobs.batch.steps[] | select(.name == "Use the App'"'"'s bot as git identity and credential")'
|
||||
[ "$(yq -r "$identity | .env.APP_SLUG" "$WF")" = '${{ steps.app-token.outputs.app-slug }}' ]
|
||||
yq -r "$identity | .run" "$WF" | grep -qF 'gh api "/users/${APP_SLUG}[bot]" --jq .id'
|
||||
yq -r "$identity | .run" "$WF" | grep -qF 'user.email "${id}+${APP_SLUG}[bot]@users.noreply.github.com"'
|
||||
|
||||
# No step reaches for the job's own token or a person's batch token.
|
||||
if grep -v '^[[:space:]]*#' "$WF" | grep -Eq 'secrets\.GITHUB_TOKEN|github\.token|secrets\.BACKPORT_BATCH_TOKEN|gh api user'; then
|
||||
echo "the workflow reads the job's own token or a person's"
|
||||
false
|
||||
fi
|
||||
}
|
||||
Executable
+122
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env bash
|
||||
# The scheduled backport batch (docs/release/backports.md): for each line the
|
||||
# batch runs for, newest first, continue the open batch pull request or open
|
||||
# one, check the line, and report what waits on the batch. This is what
|
||||
# .github/workflows/backport-batch.yaml runs; a maintainer can run it by hand
|
||||
# from a clean worktree of the repository.
|
||||
#
|
||||
# Usage: backport-batch-scheduled.sh [--dry-run] [--no-workflow-files]
|
||||
#
|
||||
# --dry-run builds and checks each batch in this worktree and prints what it
|
||||
# would report, but pushes nothing, opens or edits no pull request and writes
|
||||
# no comment. Its check sees the line without the batch it just built, so it
|
||||
# can exit 1 where the real run's check passes. Without --dry-run the batch's
|
||||
# token has to be in BACKPORT_BATCH_TOKEN, and it is the only token gh sees:
|
||||
# the job's own GITHUB_TOKEN is never a fallback, because a push made with it
|
||||
# starts no CI on the batch pull request. The workflow passes the cozystack-ci
|
||||
# App's token; a maintainer running this by hand passes their own.
|
||||
#
|
||||
# --no-workflow-files goes to run: an original that changes .github/workflows/
|
||||
# is left on the batch pull request for a maintainer instead of picked, for a
|
||||
# token that may not push such a commit, as the workflow's is.
|
||||
#
|
||||
# Environment:
|
||||
# BACKPORT_BATCH the backport-batch binary (default: built from this tree)
|
||||
# BACKPORT_BATCH_TOKEN the token the batch pushes, opens and comments with
|
||||
# BACKPORT_BATCH_BOT the bot the scheduled batch writes as, <app>[bot],
|
||||
# whose batch pull request and comment are trusted;
|
||||
# cozystack-ci[bot] when unset
|
||||
# BACKPORT_ALERT_TEAM the team or person the batch's comment mentions,
|
||||
# org/team or a login; nobody is mentioned when it is
|
||||
# unset
|
||||
# GITHUB_STEP_SUMMARY when set, each finding is appended to it as a line
|
||||
#
|
||||
# Exits 0 when every line ran and its check passed, 1 when a line's run,
|
||||
# check or report failed, and 2 on a usage or setup error.
|
||||
set -euo pipefail
|
||||
|
||||
die() {
|
||||
echo "backport-batch-scheduled: $*" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
usage="usage: backport-batch-scheduled.sh [--dry-run] [--no-workflow-files]"
|
||||
dry_run=false
|
||||
# An empty array expands under set -u only as ${a[@]+"${a[@]}"} on bash before
|
||||
# 4.4, which is what macOS ships.
|
||||
workflows=()
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--dry-run) dry_run=true ;;
|
||||
--no-workflow-files) workflows=(--no-workflow-files) ;;
|
||||
*) die "$usage" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
push=()
|
||||
if ! $dry_run; then
|
||||
[ -n "${BACKPORT_BATCH_TOKEN:-}" ] ||
|
||||
die "BACKPORT_BATCH_TOKEN is empty; the workflow sets it to the cozystack-ci App's token, a run by hand to your own (docs/release/backports.md#the-scheduled-run). The job's GITHUB_TOKEN is not used: its pushes start no CI."
|
||||
export GH_TOKEN="$BACKPORT_BATCH_TOKEN"
|
||||
unset GITHUB_TOKEN
|
||||
push=(--push)
|
||||
fi
|
||||
|
||||
mention=()
|
||||
if [ -n "${BACKPORT_ALERT_TEAM:-}" ]; then
|
||||
mention=(--mention "$BACKPORT_ALERT_TEAM")
|
||||
fi
|
||||
|
||||
top=$(git rev-parse --show-toplevel) || die "not in a worktree of the repository"
|
||||
cd "$top"
|
||||
out=$(mktemp -d)
|
||||
bin=${BACKPORT_BATCH:-}
|
||||
if [ -z "$bin" ]; then
|
||||
# Built before the first run switches this tree to a batch branch, whose
|
||||
# line may not have the tool at all.
|
||||
bin="$out/backport-batch"
|
||||
go build -o "$bin" ./cmd/backport-batch
|
||||
fi
|
||||
|
||||
report() {
|
||||
echo "$1"
|
||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
||||
printf -- '- %s\n' "$1" >> "$GITHUB_STEP_SUMMARY"
|
||||
fi
|
||||
}
|
||||
|
||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
||||
printf '## Backport batch\n\n' >> "$GITHUB_STEP_SUMMARY"
|
||||
fi
|
||||
lines=$("$bin" lines) || die "cannot list the release lines"
|
||||
[ -n "$lines" ] || die "no release line to run the batch for"
|
||||
|
||||
status=0
|
||||
for line in $lines; do
|
||||
echo "::group::$line"
|
||||
rc=0
|
||||
"$bin" run --line "$line" --continue ${workflows[@]+"${workflows[@]}"} ${push[@]+"${push[@]}"} > "$out/$line.md" || rc=$?
|
||||
if [ "$rc" -ne 0 ]; then
|
||||
report "$line: the run failed with exit $rc"
|
||||
status=1
|
||||
elif $dry_run; then
|
||||
report "$line: the batch is built in this tree, its body in $out/$line.md"
|
||||
fi
|
||||
rc=0
|
||||
"$bin" check --line "$line" || rc=$?
|
||||
if [ "$rc" -ne 0 ]; then
|
||||
report "$line: the check exited $rc; see its output above"
|
||||
status=1
|
||||
fi
|
||||
rc=0
|
||||
notes=$("$bin" notify --line "$line" ${mention[@]+"${mention[@]}"} ${push[@]+"${push[@]}"}) || rc=$?
|
||||
while IFS= read -r note; do
|
||||
[ -z "$note" ] || report "$note"
|
||||
done <<< "$notes"
|
||||
if [ "$rc" -ne 0 ]; then
|
||||
report "$line: reporting what waits on the batch failed with exit $rc"
|
||||
status=1
|
||||
fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
exit "$status"
|
||||
@@ -0,0 +1,59 @@
|
||||
#!/usr/bin/env bats
|
||||
# Contract for the backport-batch-verify job of .github/workflows/pr-labeler.yaml,
|
||||
# which runs `backport-batch verify` on every push to a batch pull request.
|
||||
#
|
||||
# The job runs under pull_request_target, so it holds a token on a pull
|
||||
# request's events and must never run code from the pull request. These tests
|
||||
# pin what keeps it that way: the tool is built from main, the head is only
|
||||
# fetched, the token reads contents and nothing more, and no expression
|
||||
# reaches a shell.
|
||||
#
|
||||
# Run with: bats hack/backport-batch-verify-job.bats
|
||||
|
||||
load test_helper
|
||||
|
||||
WF="${BATS_TEST_DIRNAME}/../.github/workflows/pr-labeler.yaml"
|
||||
JOB='.jobs.backport-batch-verify'
|
||||
|
||||
@test "the job runs on pull_request_target, so main's copy reaches release lines" {
|
||||
[ "$(yq -r '.on | keys | .[]' "$WF")" = "pull_request_target" ]
|
||||
[ "$(yq -r "$JOB.name" "$WF")" = "Backport batch shape" ]
|
||||
}
|
||||
|
||||
@test "the job runs only for a batch branch of this repository into a release line" {
|
||||
cond="$(yq -r "$JOB.if" "$WF")"
|
||||
[[ "$cond" == *"startsWith(github.event.pull_request.base.ref, 'release-')"* ]]
|
||||
[[ "$cond" == *"startsWith(github.event.pull_request.head.ref, 'backport-batch/')"* ]]
|
||||
[[ "$cond" == *"github.event.pull_request.head.repo.full_name == github.repository"* ]]
|
||||
[ "$(grep -o '&&' <<<"$cond" | wc -l)" -eq 2 ]
|
||||
}
|
||||
|
||||
@test "the token reads contents and nothing else" {
|
||||
[ "$(yq -o=json -I=0 "$JOB.permissions" "$WF")" = '{"contents":"read"}' ]
|
||||
}
|
||||
|
||||
@test "main is checked out, without credentials, and the head only fetched" {
|
||||
[ "$(yq -r "$JOB.steps[0].uses" "$WF")" = "actions/checkout@11d5960a326750d5838078e36cf38b85af677262" ]
|
||||
[ "$(yq -r "$JOB.steps[0].with.ref" "$WF")" = "main" ]
|
||||
[ "$(yq -r "$JOB.steps[0].with.persist-credentials" "$WF")" = "false" ]
|
||||
[ "$(yq -r "[$JOB.steps[] | select(.uses == \"actions/checkout*\")] | length" "$WF")" -eq 1 ]
|
||||
[ "$(yq -r "$JOB.steps[] | select(.name == \"Build backport-batch from main\") | .run" "$WF")" = 'go build -o "$RUNNER_TEMP/backport-batch" ./cmd/backport-batch' ]
|
||||
[ "$(yq -r "$JOB.steps[] | select(.name == \"Fetch the batch head as data\") | .run" "$WF")" = 'git fetch --no-tags --quiet origin "$HEAD_SHA"' ]
|
||||
}
|
||||
|
||||
@test "the tool is built before the head is fetched, and verify gets it through env" {
|
||||
names="$(yq -r "$JOB.steps[].name" "$WF")"
|
||||
[ "$(grep -n 'Build backport-batch from main' <<<"$names" | cut -d: -f1)" -lt "$(grep -n 'Fetch the batch head as data' <<<"$names" | cut -d: -f1)" ]
|
||||
run_line="$(yq -r "$JOB.steps[] | select(.name == \"Verify every original on the batch\") | .run" "$WF")"
|
||||
[ "$run_line" = '"$RUNNER_TEMP/backport-batch" verify --line "$BASE_REF" --head "$HEAD_SHA"' ]
|
||||
}
|
||||
|
||||
@test "no expression reaches a shell in the job" {
|
||||
runs="$(yq -r "$JOB.steps[] | select(has(\"run\")) | .run" "$WF")"
|
||||
[ -n "$runs" ]
|
||||
if grep -q '\${{' <<<"$runs"; then
|
||||
echo "a run: line carries an expression:"
|
||||
echo "$runs"
|
||||
false
|
||||
fi
|
||||
}
|
||||
Loaded 100 of 289 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user