fix(release): withhold or refuse a unit that does only part of its original on the line

A clean cherry-pick says nothing about whether the original landed
whole: git deletes a file the line never had without a conflict and
leaves a file the line lacks untouched. #5088 reached the 1.6 batch
that way, carrying only its test while the two deletions that are the
fix had no effect.

Every M and D path of an original is now judged at its counterpart on
the line, followed through renames on main, on the line, or both. A
modified path missing before and after, or left unchanged when the line
holds neither the original's blob nor a clean git merge-file result, is
omitted; a deleted path absent before is omitted. Content merge-file
refuses, mode changes and skipped rename detection become diagnostics
and never count as omissions; I/O failures stop the command.

The same guard runs in four places: the picker, which resets a partial
pick and parks the original; a check-time pass over every unit on the
open batch at its own first parent, whatever its label, which fails
check --cut and verify; the same pass over every unpicked original's
clean net merge, which parks a partial one with no record needed; and
fold. contained accepts the renamed path, so a fix of a renamed file
passes verify and fold. The unit mapping also closes the found() gap: a
merged original whose first pick came out empty is on the batch, not
missing.

The record moves to version 3 with parked entries and partial units,
and reports to format 2 with typed parked entries; format 1 reports
still read.

Assisted-By: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This commit is contained in:
Myasnikov Daniil committed 2026-11-10 10:58:32 +00:00
1 parent 2c1cb2f41c
commit c617a99453
21 files changed
+1856 -57

No files matched your search

+6 -2
View File
@@ -37,6 +37,8 @@ The pull request body goes to stdout, built from the line's own `.github/PULL_RE
`--no-workflow-files` is for a token that may not push a change to `.github/workflows/`, which the scheduled run's App token may not. An original that changes anything there is not picked: the body lists it under "Waiting for a maintainer" with those files, and the `backport-batch` block records it with `"workflows": true`, so `check`, `notify` and a run that continues the batch know it waits. It is never a conflict. A run without the flag, a maintainer's with their own token, picks it like any other original, onto the open batch when it continues one.
A clean pick can still carry only part of its original: git deletes a file the line never had without a word, and leaves alone a file the line lacks. So every path the original modifies or deletes, `M` or `D` in `git diff --name-status --no-renames M^1 M`, is judged at its counterpart on the line: the path itself, or the one a rename `git diff -M` finds between the line's fork point and `M^1` on `main`, between the fork point and the line, or both in turn. A modified path counts as left without effect when its counterpart exists neither before nor after the pick; when the pick leaves an existing file as it was, the file counts as there only when its blob is the original's, binary files included, or when `git merge-file` of the original's preimage, the line's file and the original's postimage merges cleanly back into the line's file. A deleted path counts as left without effect when its counterpart is absent before a pick that changes something, a deletion the line already made being no change of that pick, or still there after it, whatever the pick changed in it; when the original adds that path itself, as when the line made the same rename, it is judged as a modified file against what the original makes of that path. Picks that change nothing carry the original only when the line already has what it does to every file it modifies or deletes, so a change made where its branch merged `main`, which no pick brings, is still missing. What cannot be judged, a mode or type change, content `merge-file` refuses, or a rename git skipped looking for, is shown with the original and never counts against it. An original whose net merge onto the line, `git merge-tree --merge-base=M^1`, leaves something without effect is not tried, and one whose picks do is taken back off the branch: either is parked as partial, listed under "Parked" with those paths, and its release note stays out of the body. A conflict is never called partial. `backport/include` lifts no park.
`--defer N[,N] --reason TEXT` takes originals out of the open batch pull request, usually the ones a failing check was pinned on. It builds the batch branch again off `origin/<line>` without their units: while the batch still sits on the line's head, the units before the first one it drops stay as they are, and every later one, a pushed resolution included, is picked again in its own shape, a merge's picks one by one and then the merge with its message. When dropping one leaves a later commit that no longer applies, the run stops and names that commit's original, to defer as well. It adds no new original. It also takes out an original that got `backport/skip` after the batch picked it, which a run never removes on its own and the body lists as still on the branch. The body records each deferred original with the reason in the `backport-batch` block; a run that continues this batch keeps them out, and the next batch tries them again. A batch that carries a merge commit other than an original's, as GitHub's update-branch button makes, is refused, since picking such a merge again would lose what it resolved. When it takes out everything the batch carries, there is nothing left to push, and `--push` closes the batch pull request instead, as [the line's record](#the-lines-record-and-reports) describes.
`--drop N[,N] --reason TEXT` drops originals from the line: where `--defer` holds one out of this batch, a drop keeps it out of every later batch of the line too, until `--undrop N[,N]` withdraws it. A drop is recorded in the [line's record](#the-lines-record-and-reports), so both need an open batch pull request that carries one, and exit 2 without it; once a batch merges, its drops are carried into the next batch's record. A dropped original the open batch carries is taken off it as `--defer` would, and the rebuild is pushed the same way; any other drop or undrop changes only the record, so the batch stays as it is even when the line moved since. An undropped original is missing until the next run tries it. A drop names the original's merge commit on `main` as well as its number, and a record whose drop does not match what that merge commit merged stops every command with exit 2. An original the line already carries cannot be dropped, and neither can a number that merged into `main` before the line branched. The order is the rules', then the drop: what `backport/skip`, rules 2 to 4 or the line itself decide comes first, with the drop kept on record beside it, and `backport/include` does not lift a drop. The body lists dropped originals under their own heading among the skipped ones, `check` prints a note for each, and `fold` refuses one.
@@ -51,13 +53,15 @@ Exits 1 and prints the URL of every pull request the rules take that is neither
An original with `backport/skip` that the open batch picked before the label came prints a warning in either mode, naming the batch and `run --defer`, which takes it out; it fails neither.
A unit on the open batch that leaves part of its original without effect, judged as [`run`](#run) judges a pick, at the unit's own first parent and whatever its original's rule, label or drop, prints a warning naming what it leaves out, and fails `--cut` ahead of any skip, drop or stale warning: take it out with `run --defer`. An original that is parked as partial prints as a note, `#N is parked: only part of it applies`, and fails neither.
### fold
Checks a hand-made resolution of a deferred conflict before it is pushed onto the batch, in the shape the run gives an original. For a merged pull request `--commit` is a merge whose first parent is the head of the open batch pull request and whose message names the original in exactly one `Backported-from:` trailer; its second parent ends a straight line from that head of picks of the pull request's own commits, in their order, each ending in the `(cherry picked from commit …)` line of the commit it came from, none claiming another commit of the same original, a commit of another pull request never among them. For a squash-merged one it is a single commit on that head whose `(cherry picked from commit …)` line names the original's commit on `main`. Either has to change something, and only files the original changes. The [conflict task](../../docs/release/backport-tasks/resolve-conflict.md) gives the commands. fold prints `ok` and exits 0, or prints why not and exits 1. It never commits or pushes; push one checked resolution at a time, since the next one has to sit on the new head.
Checks a hand-made resolution of a deferred conflict before it is pushed onto the batch, in the shape the run gives an original. For a merged pull request `--commit` is a merge whose first parent is the head of the open batch pull request and whose message names the original in exactly one `Backported-from:` trailer; its second parent ends a straight line from that head of picks of the pull request's own commits, in their order, each ending in the `(cherry picked from commit …)` line of the commit it came from, none claiming another commit of the same original, a commit of another pull request never among them. For a squash-merged one it is a single commit on that head whose `(cherry picked from commit …)` line names the original's commit on `main`. Either has to change something, and only files the original changes or the paths a rename on `main` or on the line gave them, and has to leave nothing the original modifies or deletes without effect, judged as [`run`](#run) judges a pick: a resolution that ports what the line has and leaves out what it lacks is refused, since such a port goes to the line as a pull request of its own. The [conflict task](../../docs/release/backport-tasks/resolve-conflict.md) gives the commands. fold prints `ok` and exits 0, or prints why not and exits 1. It never commits or pushes; push one checked resolution at a time, since the next one has to sit on the new head.
### verify
Holds every unit of a batch to the shape fold checks for one, the clean picks as well as resolutions: it walks the batch's first-parent line from `origin/<line>` to `--head`, or to the head of the open batch pull request, and requires each unit to be one original, merged into `main` after the line branched, in that shape, and no original twice. A commit that carries no original fails it, and so does a merge of the line into the batch. It prints one line per unit, and exits 0 when all pass, 1 when one does not, and 2 when it cannot tell. With `--head` it reads git alone, nothing from GitHub. fold runs only where someone resolved a conflict and chose to run it; [`pr-labeler.yaml`](../../.github/workflows/pr-labeler.yaml) runs verify on every push to a batch pull request, from a build of `main`, with the batch head fetched as data.
Holds every unit of a batch to the shape fold checks for one, the clean picks as well as resolutions: it walks the batch's first-parent line from `origin/<line>` to `--head`, or to the head of the open batch pull request, and requires each unit to be one original, merged into `main` after the line branched, in that shape, and no original twice. A commit that carries no original fails it, and so does a merge of the line into the batch, and a unit that leaves part of its original without effect, judged as [`run`](#run) judges a pick. It prints one line per unit, and exits 0 when all pass, 1 when one does not, and 2 when it cannot tell. With `--head` it reads git alone, nothing from GitHub. fold runs only where someone resolved a conflict and chose to run it; [`pr-labeler.yaml`](../../.github/workflows/pr-labeler.yaml) runs verify on every push to a batch pull request, from a build of `main`, with the batch head fetched as data.
### notify
+47 -4
View File
@@ -198,8 +198,14 @@ func (a *app) run(args []string) int {
}
}
for _, v := range b.qualifying() {
on := false
if carried != nil {
if on, err = b.onBranch(carried, v); err != nil {
return a.fail(err)
}
}
switch {
case carried != nil && found(carried, v):
case on:
r.onBranch = append(r.onBranch, v)
continue
case held[v.o.number].Reason != "":
@@ -210,6 +216,17 @@ func (a *app) run(args []string) int {
r.skipped[ruleOnLine] = append(r.skipped[ruleOnLine], v)
continue
}
if v.partial != nil {
// Judged at the line's head; what the batch picked since may
// supply what it lacked there.
if err := a.partialAtHead(b, v); err != nil {
return a.fail(err)
}
if v.partial != nil {
r.parked = append(r.parked, v)
continue
}
}
if *noWorkflows {
files, err := workflowFiles(v.commits)
if err != nil {
@@ -220,11 +237,19 @@ func (a *app) run(args []string) int {
continue
}
}
p, err := pickOriginal(v)
p, err := pickOriginal(v, func(base, head string) ([]string, error) {
omitted, diag, err := b.omittedEffects(v, base, head)
v.addDiag(diag...)
return omitted, err
})
if err != nil {
return a.fail(err)
}
switch p.status {
case "partial":
v.partial = p.files
v.park = &parkedEntry{Original: v.o.number, Merge: v.o.merge, Kind: parkPartial, Files: sortedClean(p.files)}
r.parked = append(r.parked, v)
case "picked":
r.onBranch = append(r.onBranch, v)
case "conflict":
@@ -246,10 +271,23 @@ func (a *app) run(args []string) int {
if !descends {
return a.fail(fmt.Errorf("%.12s does not descend from --onto %.12s", head, *onto))
}
if err := b.markPartial(r, carried.head); err != nil {
return a.fail(err)
}
}
return a.finish(b, r, branch, date, *push, false)
}
// partialAtHead judges v again where the run has got to, when earlier picks
// moved the branch past the line's head.
func (a *app) partialAtHead(b *batch, v *verdict) error {
head, err := resolve("HEAD")
if err != nil || head == b.lineHead {
return err
}
return b.partialAt(v, head)
}
// keptOut lists the originals with backport/skip, or dropped, that the batch
// still carries: a run never takes a commit off the batch, --defer does.
func keptOut(b *batch, carried *history) ([]*verdict, error) {
@@ -261,7 +299,11 @@ func keptOut(b *batch, carried *history) ([]*verdict, error) {
if err := b.load(v); err != nil {
return nil, err
}
if found(carried, v) {
on, err := b.onBranch(carried, v)
if err != nil {
return nil, err
}
if on {
out = append(out, v)
}
}
@@ -365,7 +407,8 @@ func (r *report) lineRecord(doc reportDoc) lineRecord {
dropped = []droppedEntry{}
}
slices.SortFunc(dropped, func(a, b droppedEntry) int { return a.Original - b.Original })
return lineRecord{Version: 2, Line: doc.Line, Built: &built, Report: doc.ID, Closing: r.closing, Deferred: slices.Clone(doc.Primary.Deferred), Dropped: dropped, Stale: slices.Clone(doc.Overlays.Stale)}
return lineRecord{Version: recordVersion, Line: doc.Line, Built: &built, Report: doc.ID, Closing: r.closing, Deferred: slices.Clone(doc.Primary.Deferred), Dropped: dropped, Stale: slices.Clone(doc.Overlays.Stale),
Parked: slices.Clone(doc.Primary.Parked), Partial: slices.Clone(doc.Overlays.Partial)}
}
// branchDate is the date a batch branch was opened under.
+52 -2
View File
@@ -238,8 +238,10 @@ type report struct {
overrides []*verdict
stale []*verdict
start time.Time
// shapes says, for an original on the branch, what it brought.
shapes map[*verdict]string
// shapes says, for an original on the branch, what it brought, and
// partialFiles what a partial unit leaves without effect.
shapes map[*verdict]string
partialFiles map[*verdict][]string
// missing are originals the policy takes that the run neither tried nor
// found anywhere.
missing []*verdict
@@ -253,6 +255,11 @@ type report struct {
closing *closing
// dropped are the maintainers' drops the record keeps.
dropped []droppedEntry
// parked are what the batch leaves out because it cannot land whole,
// each with its park, and partial the originals on the branch whose unit
// does only part of what they do on main.
parked []*verdict
partial []*verdict
}
// ruleName is how output names what kept an original out: the policy's
@@ -346,6 +353,18 @@ func (r *report) what(details map[int]bool) string {
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), strings.Join(files, ", "))
}
}
if len(r.partial) > 0 {
fmt.Fprintf(&b, "\n### Still on this branch: only part of it applies\n\nEach of these is on this branch but leaves files of its original as they were on %s, so it carries only part of what it does on `main`. Take each out with `backport-batch run --defer` before the cut; what applies of it goes as a pull request of its own.\n\n", codeSpan(r.line.branch()))
for _, v := range r.partial {
fmt.Fprintf(&b, "- %s %s: leaves %s without effect\n", prLink(v.o.number), titleSpan(v.o.title), fileSpans(r.partialFiles[v]))
}
}
if len(r.parked) > 0 {
fmt.Fprintf(&b, "\n### Parked\n\nThese cannot land on %s whole, so the batch does not pick them: each would leave part of it without effect there, or builds on a change that never reaches the line. A maintainer ports what applies as a pull request of its own, or drops the original with `backport-batch run --drop`; `backport/include` does not lift a park.\n\n", codeSpan(r.line.branch()))
for _, v := range r.parked {
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(v.o.number), titleSpan(v.o.title), parkReason(*v.park))
}
}
if len(r.overrides) > 0 {
b.WriteString("\n### Maintainer overrides\n\n")
for _, v := range r.overrides {
@@ -384,6 +403,8 @@ func (r *report) what(details map[int]bool) string {
// outcome is what became of an original a label overrode.
func (r *report) outcome(v *verdict) string {
switch {
case slices.Contains(r.partial, v):
return "still on this branch, but only part of it applies; take it out with `backport-batch run --defer`"
case slices.Contains(r.stale, v) && v.rule == ruleDropped:
return "dropped, but still on this branch; take it out with `backport-batch run --defer`"
case slices.Contains(r.stale, v):
@@ -398,12 +419,41 @@ func (r *report) outcome(v *verdict) string {
return "deferred to the next batch"
case slices.ContainsFunc(r.waiting, func(p pick) bool { return p.v == v }):
return "waiting for a maintainer: it changes " + codeSpan(workflowsDir)
case slices.Contains(r.parked, v):
return "parked: " + parkReason(*v.park)
case v.rule != 0:
return "skipped by " + ruleName(v.rule) + ", " + titleSpan(v.reason)
}
return "not picked"
}
// fileSpans lists files as code spans, the first few of them.
func fileSpans(files []string) string {
var out []string
for _, f := range files[:min(len(files), filesListed)] {
out = append(out, codeSpan(f))
}
if more := len(files) - len(out); more > 0 {
out = append(out, fmt.Sprintf("%d more", more))
}
return strings.Join(out, ", ")
}
// parkReason says why a park leaves its original out, for the body.
func parkReason(p parkedEntry) string {
if p.Kind == parkPartial {
return "only part of it applies; it would leave " + fileSpans(p.Files) + " without effect"
}
var needs []string
for _, n := range p.Needs {
needs = append(needs, prLink(n))
}
for _, c := range p.Commits {
needs = append(needs, codeSpan(c[:12]))
}
return "builds on " + strings.Join(needs, ", ") + ", which never reaches this line"
}
func (r *report) notes() string {
var notes []string
for _, v := range r.onBranch {
+26 -3
View File
@@ -20,6 +20,7 @@ import (
"fmt"
"slices"
"strconv"
"strings"
)
// check says whether every original the policy takes is on the line, on the
@@ -163,6 +164,10 @@ func (b *batch) dispositions(cut bool) (map[*verdict]disposition, error) {
named := v.o.url() + " " + terminalSafe(v.o.title)
e, isDeferred := deferred[v.o.number]
switch {
case b.partialUnits[v] != nil && cut:
out[v] = disposition{outcomeUnmerged, fmt.Sprintf("#%d is on batch #%d, but only part of it applies to %s: it leaves %s without effect; take it out with backport-batch run --defer before the cut: %s", v.o.number, b.pr.Number, b.line.branch(), quotedFiles(b.partialUnits[v]), named)}
case b.partialUnits[v] != nil:
out[v] = disposition{outcomeWarning, fmt.Sprintf("#%d is on batch #%d, but only part of it applies to %s: it leaves %s without effect; take it out with backport-batch run --defer: %s", v.o.number, b.pr.Number, b.line.branch(), quotedFiles(b.partialUnits[v]), named)}
case slices.Contains(stale, v):
out[v] = disposition{outcomeWarning, fmt.Sprintf("#%d is still on batch #%d; take it out with backport-batch run --defer: %s", v.o.number, b.pr.Number, named)}
case v.rule == ruleDropped:
@@ -173,14 +178,16 @@ func (b *batch) dispositions(cut bool) (map[*verdict]disposition, error) {
if cut {
out[v] = disposition{outcomeUnmerged, fmt.Sprintf("#%d is only on open batch PR #%d, merge it before the cut: %s", v.o.number, b.pr.Number, named)}
}
case isDeferred && e.Reason != "":
out[v] = disposition{outcomeWarning, fmt.Sprintf("deferred on %s to the next batch: %s (%s)", on, named, terminalSafe(e.Reason))}
case v.park != nil && v.park.Kind == parkPartial:
out[v] = disposition{outcomeNote, fmt.Sprintf("#%d is parked: only part of it applies to %s, which would leave %s without effect; port what applies as a pull request of its own: %s", v.o.number, b.line.branch(), quotedFiles(v.park.Files), named)}
case isDeferred && e.Workflows && !b.src.open:
out[v] = disposition{outcomeMissing, named}
case isDeferred && e.Workflows && cut:
out[v] = disposition{outcomeWaiting, fmt.Sprintf("#%d changes %s and waits on batch PR %s for a maintainer, bring it over or defer it before the cut: %s", v.o.number, workflowsDir, on, named)}
case isDeferred && e.Workflows:
out[v] = disposition{outcomeWarning, fmt.Sprintf("waiting on %s for a maintainer, since it changes %s: %s", on, workflowsDir, named)}
case isDeferred && e.Reason != "":
out[v] = disposition{outcomeWarning, fmt.Sprintf("deferred on %s to the next batch: %s (%s)", on, named, terminalSafe(e.Reason))}
case isDeferred:
out[v] = disposition{outcomeWarning, fmt.Sprintf("deferred conflict on %s: %s", on, named)}
default:
@@ -190,6 +197,18 @@ func (b *batch) dispositions(cut bool) (map[*verdict]disposition, error) {
return out, nil
}
// quotedFiles names files for a terminal, each quoted, the first few of them.
func quotedFiles(files []string) string {
var out []string
for _, f := range files[:min(len(files), filesListed)] {
out = append(out, strconv.Quote(f))
}
if more := len(files) - len(out); more > 0 {
out = append(out, fmt.Sprintf("%d more", more))
}
return strings.Join(out, ", ")
}
// terminalSafe keeps a title from carrying escape sequences into a terminal.
func terminalSafe(s string) string { return capped(oneLine(clean(s, false)), titleLimit) }
@@ -289,5 +308,9 @@ func refuseFold(b *batch, n int, commit string) (string, error) {
case carried != v:
return fmt.Sprintf("%.12s carries #%d, not #%d", commit, carried.o.number, n), nil
}
return "", nil
omitted, _, err := b.omittedEffects(v, u.parents[0], u.commit)
if err != nil || len(omitted) == 0 {
return "", err
}
return leftOut(v, b.line, omitted), nil
}
+11 -2
View File
@@ -78,7 +78,7 @@ func (a *app) rebuild(b *batch, numbers []int, reason string, push bool) int {
switch v := b.find(n); {
case v == nil:
return a.fail(fmt.Errorf("#%d did not merge into main after %s branched", n, b.line.branch()))
case v.rule != 0 && !slices.Contains(stale, v):
case v.rule != 0 && !slices.Contains(stale, v) && b.partialUnits[v] == nil:
return a.fail(fmt.Errorf("#%d is not one the batch takes: %s, %s", n, ruleName(v.rule), terminalSafe(v.reason)))
}
hold[n] = reason
@@ -140,11 +140,17 @@ func (a *app) reclassify(b *batch, picks *history, hold map[int]string, drops []
}
for _, v := range b.qualifying() {
e := recorded[v.o.number]
on, err := b.onBranch(picks, v)
if err != nil {
return a.fail(err)
}
switch {
case found(picks, v):
case on:
r.onBranch = append(r.onBranch, v)
case hold[v.o.number] != "":
r.held = append(r.held, pick{v: v, status: "held", reason: hold[v.o.number]})
case v.park != nil:
r.parked = append(r.parked, v)
case e.Workflows:
r.waiting = append(r.waiting, pick{v: v, status: "waiting", files: e.Files, more: e.More})
case len(e.Files) > 0:
@@ -155,6 +161,9 @@ func (a *app) reclassify(b *batch, picks *history, hold map[int]string, drops []
r.missing = append(r.missing, v)
}
}
if err := b.markPartial(r, picks.head); err != nil {
return a.fail(err)
}
return a.finish(b, r, b.pr.Head.Ref, date, push, true)
}
+555
View File
@@ -0,0 +1,555 @@
/*
Copyright 2026 The Cozystack Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
)
// A clean pick can still carry only part of an original: git deletes a file
// the line never had without a word, and leaves alone a file the line lacks.
// What follows tells such a unit from a whole one, by what it leaves without
// effect of the files the original modifies or deletes.
// treeEntry is one record of git ls-tree.
type treeEntry struct{ mode, kind, oid string }
// regular says whether git can merge the entry as text: a file, not a
// symlink, a submodule or a tree.
func (e treeEntry) regular() bool {
return e.kind == "blob" && (e.mode == "100644" || e.mode == "100755")
}
// treeEntries reads the entries paths have at treeish, from full ls-tree
// records: a path that is a tree there, or absent, has none.
func treeEntries(treeish string, paths []string) (map[string]treeEntry, error) {
out := map[string]treeEntry{}
want := map[string]bool{}
for _, p := range paths {
want[p] = true
}
const chunk = 500
for len(paths) > 0 {
n := min(chunk, len(paths))
args := append([]string{"--literal-pathspecs", "ls-tree", "-r", "-z", "--full-tree", "--end-of-options", treeish, "--"}, paths[:n]...)
raw, err := git(args...)
if err != nil {
return nil, err
}
for _, rec := range splitNUL(raw) {
meta, path, ok := strings.Cut(rec, "\t")
f := strings.Fields(meta)
if !ok || len(f) != 3 || checkSHA(f[2]) != nil {
return nil, fmt.Errorf("git ls-tree %s: unreadable record %q", treeish, terminalSafe(rec))
}
if want[path] {
out[path] = treeEntry{mode: f[0], kind: f[1], oid: f[2]}
}
}
paths = paths[n:]
}
return out, nil
}
// renameSet is what git diff -M found renamed between two commits, old path
// to new, and whether it looked at every candidate.
type renameSet struct {
moved map[string]string
complete bool
}
// renames reads the renames between from and to, once per pair. git says on
// stderr when it skipped inexact detection for too many files; complete is
// then false, and no path may be called absent for want of a rename.
func (b *batch) renames(from, to string) (renameSet, error) {
key := from + ".." + to
if s, ok := b.renameCache[key]; ok {
return s, nil
}
// -l0 lifts git's own limit, which the picker's merge does not share:
// a rename the pick follows has to be one the checks of its shape see.
stdout, stderr, code, err := gitStatus("diff", "--no-ext-diff", "--no-textconv", "-M", "-l"+strconv.Itoa(b.renameLimit), "--name-status", "-z", "--diff-filter=R", "--end-of-options", from, to)
switch {
case err != nil:
return renameSet{}, err
case code != 0:
return renameSet{}, fmt.Errorf("git diff -M %s %s: exit %d: %s", from, to, code, strings.TrimSpace(stderr))
}
f := splitNUL(stdout)
if len(f)%3 != 0 {
return renameSet{}, fmt.Errorf("unreadable renames between %s and %s", from, to)
}
s := renameSet{moved: map[string]string{}, complete: !strings.Contains(stderr, "detection was skipped")}
for i := 0; i < len(f); i += 3 {
if !strings.HasPrefix(f[i], "R") {
return renameSet{}, fmt.Errorf("unreadable renames between %s and %s: %q", from, to, terminalSafe(f[i]))
}
s.moved[f[i+1]] = f[i+2]
}
if b.renameCache == nil {
b.renameCache = map[string]renameSet{}
}
b.renameCache[key] = s
return s, nil
}
// effectPaths are the paths v modifies or deletes; what it adds has nothing
// to correspond to.
func effectPaths(v *verdict) []string {
var out []string
for _, c := range v.changes {
if c.status == 'M' || c.status == 'D' {
out = append(out, c.path)
}
}
return out
}
// correspondence maps a path of the original (at M^1) to the path holding the
// same file on target, through a main-side rename base..M^1, a line-side
// rename base..target, or both composed through the base path. A path with
// no counterpart maps to itself, and one git skipped rename detection for is
// left out and named in unknown: it is not known to be absent.
func (b *batch) correspondence(v *verdict, target string) (map[string]string, []string, error) {
paths := effectPaths(v)
have, err := treeEntries(target, paths)
if err != nil {
return nil, nil, err
}
corr := map[string]string{}
var missing []string
for _, p := range paths {
if _, ok := have[p]; ok {
corr[p] = p
} else {
missing = append(missing, p)
}
}
if len(missing) == 0 {
return corr, nil, nil
}
onMain, err := b.renames(b.base, v.o.merge+"^1")
if err != nil {
return nil, nil, err
}
onLine, err := b.renames(b.base, target)
if err != nil {
return nil, nil, err
}
fromBase := map[string]string{}
for old, now := range onMain.moved {
fromBase[now] = old
}
candidates := map[string]string{}
var names []string
for _, p := range missing {
q := p
if old, ok := fromBase[p]; ok {
q = old
}
r := q
if now, ok := onLine.moved[q]; ok {
r = now
}
if r != p {
candidates[p] = r
names = append(names, r)
}
}
found, err := treeEntries(target, names)
if err != nil {
return nil, nil, err
}
var unknown []string
for _, p := range missing {
if r, ok := candidates[p]; ok {
if _, ok := found[r]; ok {
corr[p] = r
continue
}
}
if !onMain.complete || !onLine.complete {
unknown = append(unknown, p)
continue
}
corr[p] = p
}
return corr, unknown, nil
}
// omittedEffects lists the paths v modifies or deletes that the change from
// before to after leaves without effect, each judged at its corresponding
// path c(p) on before. A modified path is omitted when c(p) exists on neither
// side, applied when the change touches it, and when the change leaves an
// existing file alone, satisfied only if the line already has what the
// original makes of it. A deleted path is omitted when c(p) is absent before:
// a deletion the line already did is no change this unit makes. diag says
// what could not be judged, which is never counted as omitted.
func (b *batch) omittedEffects(v *verdict, before, after string) ([]string, []string, error) {
if err := b.load(v); err != nil {
return nil, nil, err
}
corr, unknown, err := b.correspondence(v, before)
if err != nil {
return nil, nil, err
}
var diag []string
for _, p := range unknown {
diag = append(diag, "git skipped rename detection, so where "+strconv.Quote(p)+" is on the line is not known")
}
added := map[string]bool{}
for _, c := range v.changes {
if c.status == 'A' {
added[c.path] = true
}
}
// A deleted path whose counterpart the original adds itself is a rename
// the line already made the same way: it is judged as a modified file,
// against what the original makes of that path.
var at, own, moved []string
movedTo := map[string]string{}
for _, c := range v.changes {
cp, ok := corr[c.path]
if !ok {
continue
}
at = append(at, cp)
switch {
case c.status == 'M':
own = append(own, c.path)
case c.status == 'D' && cp != c.path && added[cp]:
own, moved = append(own, c.path), append(moved, cp)
movedTo[c.path] = cp
}
}
bef, err := treeEntries(before, at)
if err != nil {
return nil, nil, err
}
aft, err := treeEntries(after, at)
if err != nil {
return nil, nil, err
}
pre, err := treeEntries(v.o.merge+"^1", own)
if err != nil {
return nil, nil, err
}
post, err := treeEntries(v.o.merge, append(slices.Clone(own), moved...))
if err != nil {
return nil, nil, err
}
// A change that makes none is judged by its trees, whatever names them:
// a commit and the tree a net merge wrote can be the same change.
same := before == after
if !same {
bt, err := resolveTree(before)
if err != nil {
return nil, nil, err
}
at, err := resolveTree(after)
if err != nil {
return nil, nil, err
}
same = bt == at
}
var omitted []string
// modified judges a path whose counterpart cp should end up as the
// original's postimage p1 of it.
modified := func(path, cp string, p0, p1 treeEntry) error {
be, inBefore := bef[cp]
ae, inAfter := aft[cp]
switch {
case !inBefore && !inAfter:
omitted = append(omitted, path)
return nil
case inBefore != inAfter || be != ae:
return nil
case be.oid == p1.oid && (p0.mode == p1.mode || be.mode == p1.mode):
return nil
case !be.regular() || !p0.regular() || !p1.regular() || p0.mode != p1.mode:
diag = append(diag, "the mode or type of "+strconv.Quote(path)+" changes, which only a person can judge")
return nil
}
applied, unsupported, err := b.alreadyApplied(p0.oid, be.oid, p1.oid)
switch {
case err != nil:
return fmt.Errorf("judging %s of #%d: %w", strconv.Quote(path), v.o.number, err)
case unsupported:
diag = append(diag, "git cannot merge "+strconv.Quote(path)+" as text, so whether the line has its change is not known")
case !applied:
omitted = append(omitted, path)
}
return nil
}
for _, c := range v.changes {
cp, ok := corr[c.path]
if !ok {
continue
}
switch {
case c.status == 'D' && movedTo[c.path] != "":
if err := modified(c.path, cp, pre[c.path], post[cp]); err != nil {
return nil, nil, err
}
case c.status == 'D':
// Absent before: the line already made the deletion, which is
// no change of a unit that makes others, and is all of one
// that makes none. Still there after: the deletion did not
// happen, whatever the unit changed in the file; keeping what
// the line made of it is a port of its own.
_, inBefore := bef[cp]
_, inAfter := aft[cp]
if !inBefore && !same || inAfter {
omitted = append(omitted, c.path)
}
case c.status == 'M':
if err := modified(c.path, cp, pre[c.path], post[c.path]); err != nil {
return nil, nil, err
}
}
}
for i := range diag {
diag[i] = oneLineText(diag[i])
}
return omitted, diag, nil
}
// alreadyApplied says whether cur, a file the unit leaves alone, already
// holds the original's change from pre to post: it is post itself, or git
// merge-file of the three merges cleanly back into cur. unsupported is
// content merge-file refuses, such as differing binary files, which says
// nothing either way.
func (b *batch) alreadyApplied(pre, cur, post string) (applied, unsupported bool, err error) {
if cur == post {
return true, false, nil
}
b.mergeFiles++
blobs, err := readObjects([]string{pre, cur, post}, "blob")
if err != nil {
return false, false, err
}
dir, err := os.MkdirTemp("", "backport-batch-merge-file-*")
if err != nil {
return false, false, err
}
defer os.RemoveAll(dir)
names := map[string]string{}
for _, f := range []struct{ name, oid string }{{"cur", cur}, {"pre", pre}, {"post", post}} {
names[f.name] = filepath.Join(dir, f.name)
if err := os.WriteFile(names[f.name], blobs[f.oid], 0o600); err != nil {
return false, false, err
}
}
stdout, stderr, code, err := gitStatus("merge-file", "-p", names["cur"], names["pre"], names["post"])
switch {
case err != nil:
return false, false, err
case code == 0:
return stdout == string(blobs[cur]), false, nil
case code > 0 && code < 128:
// The number of conflicts.
return false, false, nil
case strings.Contains(stderr, "Cannot merge binary files"):
return false, true, nil
}
return false, false, fmt.Errorf("git merge-file: exit %d: %s", code, strings.TrimSpace(stderr))
}
// unitsOn maps each original the batch head carries a unit of to that unit,
// by its Backported-from trailer or appended -x line: found needs every
// commit of a merged original, and misses one whose first pick came out
// empty because the line already had that change.
func (b *batch) unitsOn(head string) (map[*verdict]string, error) {
if m, ok := b.unitCache[head]; ok {
return m, nil
}
units, err := batchUnits(head, b.lineHead)
if err != nil {
return nil, err
}
m := map[*verdict]string{}
for _, u := range units {
if v := b.unitOriginal(u); v != nil && m[v] == "" {
m[v] = u.commit
}
}
if b.unitCache == nil {
b.unitCache = map[string]map[*verdict]string{}
}
b.unitCache[head] = m
return m, nil
}
// onBranch says whether the batch head h carries v.
func (b *batch) onBranch(h *history, v *verdict) (bool, error) {
if found(h, v) {
return true, nil
}
m, err := b.unitsOn(h.head)
return m[v] != "", err
}
// partialStates finds what applies only in part on the line. Every unit on
// the open batch is checked against its own first parent, whatever the rules
// now say of its original: one that leaves an effect out fails the cut until
// it is taken out. Every original the policy takes and nothing carries is
// checked against the tree its clean net merge onto the line writes, so a
// partial one is found with no record and no adder; a conflict is never
// called partial.
func (b *batch) partialStates() error {
b.partialUnits = map[*verdict][]string{}
if b.pr != nil {
var err error
if b.partialUnits, err = b.unitsPartial(b.prHead); err != nil {
return err
}
on, err := b.unitsOn(b.prHead)
if err != nil {
return err
}
for v, commit := range on {
if v.rule == 0 && v.onBatch == "" {
v.onBatch = commit
}
}
}
for _, v := range b.qualifying() {
if v.onBatch != "" {
continue
}
if err := b.partialAt(v, b.lineHead); err != nil {
return err
}
}
return nil
}
// unitsPartial maps each original whose unit between the line's head and
// head leaves part of it without effect to what it leaves.
func (b *batch) unitsPartial(head string) (map[*verdict][]string, error) {
units, err := batchUnits(head, b.lineHead)
if err != nil {
return nil, err
}
out := map[*verdict][]string{}
for _, u := range units {
v := b.unitOriginal(u)
if v == nil || len(u.parents) == 0 {
continue
}
omitted, diag, err := b.omittedEffects(v, u.parents[0], u.commit)
if err != nil {
return nil, err
}
v.addDiag(diag...)
if len(omitted) > 0 {
out[v] = append(out[v], omitted...)
}
}
return out, nil
}
// markPartial puts every original on the branch at head whose unit there
// does only part of it among the partial units, and so among the stale
// ones: still on the batch, and to be taken out before the cut.
func (b *batch) markPartial(r *report, head string) error {
units := b.partialUnits
if b.pr == nil || head != b.prHead {
var err error
if units, err = b.unitsPartial(head); err != nil {
return err
}
}
r.partialFiles = map[*verdict][]string{}
for _, v := range b.verdicts {
files := units[v]
if len(files) == 0 {
continue
}
r.partial = append(r.partial, v)
r.partialFiles[v] = files
if !slices.Contains(r.stale, v) {
r.stale = append(r.stale, v)
}
}
return nil
}
// partialAt judges v's clean net merge onto head, and parks it, or clears
// the park, by what that merge leaves without effect.
func (b *batch) partialAt(v *verdict, head string) error {
tree, err := b.netTree(head, v.o)
if err != nil {
return err
}
v.partial = nil
if v.park != nil && v.park.Kind == parkPartial {
v.park = nil
}
if tree == "" {
return nil
}
omitted, diag, err := b.omittedEffects(v, head, tree)
if err != nil {
return err
}
v.addDiag(diag...)
if len(omitted) > 0 {
v.partial = omitted
v.park = &parkedEntry{Original: v.o.number, Merge: v.o.merge, Kind: parkPartial, Files: sortedClean(omitted)}
}
return nil
}
// netTree is the tree a clean net merge of o onto head writes, "" for a
// conflict; the line's head reuses what judging the originals merged.
func (b *batch) netTree(head string, o *original) (string, error) {
if head == b.lineHead && b.lineHist != nil {
return b.lineHist.netTree(o)
}
tree, _, err := noopPick(head, o)
return tree, err
}
// addDiag keeps what could not be judged of v, each line once.
func (v *verdict) addDiag(lines ...string) {
for _, ln := range lines {
if ln != "" && !slices.Contains(v.diag, ln) {
v.diag = append(v.diag, ln)
}
}
}
func sortedClean(files []string) []string {
out := cleanFiles(files)
slices.Sort(out)
return slices.Compact(out)
}
// leftOut is the refusal for a unit that leaves paths of v without effect.
func leftOut(v *verdict, l line, omitted []string) string {
quoted := make([]string, len(omitted))
for i, p := range omitted {
quoted[i] = strconv.Quote(p)
}
return fmt.Sprintf("#%d leaves %s without effect on %s; port only what the line has as a pull request of its own", v.o.number, strings.Join(quoted, ", "), l.branch())
}
+350
View File
@@ -0,0 +1,350 @@
/*
Copyright 2026 The Cozystack Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"os"
"path/filepath"
"slices"
"strings"
"testing"
)
const appFile = "one\ntwo\nthree\nfour\nfive\n"
// renameWorld is the sign-off's rename history: the line keeps app/old.txt,
// main renames it to app/new.txt with #3, a feat, and #5 fixes app/new.txt.
func renameWorld(t *testing.T) (*fixture, string) {
t.Helper()
f := newFixture(t)
f.squash(2, "chore: an app file", "", map[string]string{"app/old.txt": appFile})
f.lines()
f.pr(3, "feat(app): rename the app file", "", map[string]string{"app/old.txt": "", "app/new.txt": appFile})
mc5 := f.pr(5, "fix(app): fix the app file", "", map[string]string{"app/new.txt": strings.Replace(appFile, "three", "THREE", 1)})
return f, mc5
}
// emptyBatch pushes the line's head as the batch branch and opens a batch
// pull request on it with body, for a fold to land on.
func emptyBatch(f *fixture, gh *fakeGH, body string) *foldWorld {
f.t.Helper()
f.git("fetch", "-q", "origin")
f.git("push", "-q", "origin", "origin/release-1.1:refs/heads/"+batchBranch1)
f.git("fetch", "-q", "origin")
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, body)}
return &foldWorld{f: f, gh: gh, head: f.git("rev-parse", "origin/"+batchBranch1)}
}
// A fix of a file main renamed picks onto the file's old name on the line,
// and every check of the batch's shape takes the old name as the same file:
// verify, a hand fold, and a rebuild by --defer.
func TestRenamePickThroughFoldAndVerify(t *testing.T) {
f, mc5 := renameWorld(t)
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
gh := &fakeGH{}
head := openBatch(t, f, gh)
if got := f.subjects("origin/release-1.1", head); !slices.Equal(got, []string{"fix(app): fix the app file (#5)", "fix(app): six (#6)"}) {
t.Fatalf("the batch carries %q, want #5 and #6", got)
}
if got := f.git("show", head+":app/old.txt"); !strings.Contains(got, "THREE") {
t.Fatalf("#5 did not land on app/old.txt: %q", got)
}
if code, out, stderr := f.exec(gh, "verify", "--line", "release-1.1", "--head", head); code != 0 {
t.Errorf("verify = %d\n%s\n%s", code, out, stderr)
}
t.Setenv("GIT_COMMITTER_DATE", "2026-10-27T06:00:00Z")
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "6", "--reason", "fails the e2e", "--push"); code != 0 {
t.Fatalf("run --defer 6 --push = %d\n%s", code, stderr)
}
if code, out, stderr := f.exec(gh, "verify", "--line", "release-1.1", "--head", f.remoteHead(batchBranch1)); code != 0 {
t.Errorf("verify after the rebuild = %d\n%s\n%s", code, out, stderr)
}
f, mc5 = renameWorld(t)
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/old.txt": strings.Replace(appFile, "three", "THREE", 1)})
fold := w.merge(w.head, pick, "fix(app): fix the app file (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 0 || out != "ok\n" {
t.Errorf("fold of #5 onto app/old.txt = %d, %q\n%s", code, out, stderr)
}
}
// shaped reads the line from the fixture's remote as verify does.
func shaped(t *testing.T, f *fixture) *batch {
t.Helper()
f.git("fetch", "-q", "origin")
b, err := shapeOf(line11)
if err != nil {
t.Fatal(err)
}
return b
}
// commitOn writes files onto parent and returns the commit, leaving every
// branch where it was.
func commitOn(f *fixture, parent string, files map[string]string) string {
f.t.Helper()
f.git("switch", "-q", "--detach", parent)
f.apply(files)
f.git("commit", "-q", "--allow-empty", "-m", "a unit")
sha := f.git("rev-parse", "HEAD")
f.git("switch", "-q", "main")
return sha
}
// A path of the original is followed to where the line keeps the same file:
// through a rename on main, on the line, or both, composed through the path
// at the base. Where git skipped rename detection the path is unknown, never
// absent.
func TestCorrespondence(t *testing.T) {
corr := func(t *testing.T, f *fixture, n int, limit ...int) (map[string]string, []string) {
t.Helper()
b := shaped(t, f)
if len(limit) > 0 {
b.renameLimit = limit[0]
}
v := b.find(n)
if err := b.load(v); err != nil {
t.Fatal(err)
}
c, unknown, err := b.correspondence(v, b.lineHead)
if err != nil {
t.Fatal(err)
}
return c, unknown
}
t.Run("renamed on main", func(t *testing.T) {
f, _ := renameWorld(t)
if c, unknown := corr(t, f, 5); c["app/new.txt"] != "app/old.txt" || len(unknown) != 0 {
t.Errorf("correspondence = %v, %v; want app/new.txt on app/old.txt", c, unknown)
}
})
t.Run("renamed on the line", func(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: an app file", "", map[string]string{"app/a.txt": appFile})
f.lines()
f.onLine("release-1.1", "fix(app): the line renames it", map[string]string{"app/a.txt": "", "app/line.txt": appFile})
f.pr(5, "fix(app): fix it", "", map[string]string{"app/a.txt": strings.Replace(appFile, "two", "TWO", 1)})
if c, unknown := corr(t, f, 5); c["app/a.txt"] != "app/line.txt" || len(unknown) != 0 {
t.Errorf("correspondence = %v, %v; want app/a.txt on app/line.txt", c, unknown)
}
})
t.Run("renamed on both", func(t *testing.T) {
f, _ := renameWorld(t)
f.onLine("release-1.1", "fix(app): the line renames it its own way", map[string]string{"app/old.txt": "", "app/release.txt": appFile})
if c, unknown := corr(t, f, 5); c["app/new.txt"] != "app/release.txt" || len(unknown) != 0 {
t.Errorf("correspondence = %v, %v; want app/new.txt on app/release.txt", c, unknown)
}
})
t.Run("not renamed, absent", func(t *testing.T) {
f := newFixture(t)
f.lines()
f.pr(3, "feat(app): x", "", map[string]string{"app/x.txt": appFile})
f.pr(5, "fix(app): fix x", "", map[string]string{"app/x.txt": strings.Replace(appFile, "two", "TWO", 1)})
if c, unknown := corr(t, f, 5); c["app/x.txt"] != "app/x.txt" || len(unknown) != 0 {
t.Errorf("correspondence = %v, %v; want app/x.txt on itself", c, unknown)
}
})
t.Run("rename detection skipped", func(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: two app files", "", map[string]string{"app/a.txt": appFile, "app/b.txt": appFile + "six\n"})
f.lines()
f.onLine("release-1.1", "fix(app): the line renames both, changed", map[string]string{"app/a.txt": "", "app/b.txt": "", "app/x.txt": appFile + "x\n", "app/y.txt": appFile + "six\ny\n"})
f.pr(5, "fix(app): fix a", "", map[string]string{"app/a.txt": strings.Replace(appFile, "two", "TWO", 1)})
// Past the limit, git says it skipped looking; a configured limit
// of its own does not decide for the tool.
f.git("config", "diff.renameLimit", "1")
if c, unknown := corr(t, f, 5, 0); c["app/a.txt"] != "app/x.txt" || len(unknown) != 0 {
t.Errorf("correspondence under diff.renameLimit=1 = %v, %v; want app/a.txt on app/x.txt", c, unknown)
}
if c, unknown := corr(t, f, 5, 1); !slices.Equal(unknown, []string{"app/a.txt"}) || c["app/a.txt"] != "" {
t.Errorf("correspondence past the limit = %v, %v; want app/a.txt unknown", c, unknown)
}
})
}
// What a change leaves without effect: a modified file the line lacks on both
// sides of it, a deleted file the line never had. A mode change that only a
// person can judge is a diagnostic, not an omission.
func TestOmittedEffects(t *testing.T) {
omitted := func(t *testing.T, f *fixture, n int, files map[string]string) ([]string, []string) {
t.Helper()
b := shaped(t, f)
after := commitOn(f, b.lineHead, files)
om, diag, err := b.omittedEffects(b.find(n), b.lineHead, after)
if err != nil {
t.Fatal(err)
}
return om, diag
}
t.Run("a modified file missing before and after", func(t *testing.T) {
f := newFixture(t)
f.lines()
f.pr(3, "feat(app): x", "", map[string]string{"app/x.txt": appFile})
f.pr(5, "fix(app): fix x and five", "", map[string]string{"app/x.txt": "fixed\n", "app/five.txt": "5\n"})
if om, diag := omitted(t, f, 5, map[string]string{"app/five.txt": "5\n"}); !slices.Equal(om, []string{"app/x.txt"}) || len(diag) != 0 {
t.Errorf("omitted = %q, %q; want app/x.txt", om, diag)
}
})
t.Run("a deleted file absent before", func(t *testing.T) {
f, _ := policyWorld(t)
if om, _ := omitted(t, f, 5, map[string]string{"chart/tests/values_test.yaml": "suite: values\n"}); !slices.Equal(om, []string{"chart/templates/policy.yaml", "chart/tests/policy_test.yaml"}) {
t.Errorf("omitted = %q; want both policy files", om)
}
})
t.Run("a mode change", func(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a script", "", map[string]string{"app/run.sh": "#!/bin/sh\n", "app/b.txt": "old\n"})
f.lines()
f.git("switch", "-q", "-c", "pr-5")
if err := os.Chmod(filepath.Join(f.dir, "app/run.sh"), 0o755); err != nil {
t.Fatal(err)
}
f.write("app/b.txt", "fixed\n")
f.git("add", "-A")
f.git("commit", "-q", "-m", "run it")
f.git("switch", "-q", "main")
f.git("merge", "-q", "--no-ff", "-m", "fix(app): make the script runnable (#5)", "pr-5")
f.git("push", "-q", "origin", "main")
om, diag := omitted(t, f, 5, map[string]string{"app/b.txt": "fixed\n"})
if len(om) != 0 || len(diag) != 1 || !strings.Contains(diag[0], `the mode or type of "app/run.sh" changes`) {
t.Errorf("omitted = %q, %q; want a diagnostic for app/run.sh and nothing omitted", om, diag)
}
})
}
// fixedWorld is the sign-off's two-file case: the base has a and b as old,
// #5 fixes both, and the line already has a as onLine says, if anything.
func fixedWorld(t *testing.T, a, old, onLine, fixed string) (*fixture, string) {
t.Helper()
f := newFixture(t)
f.squash(2, "chore: a and b", "", map[string]string{a: old, "app/b.txt": "old\n"})
f.lines()
if onLine != "" {
f.onLine("release-1.1", "fix(app): a, on the line", map[string]string{a: onLine})
}
mc5 := f.pr(5, "fix(app): a and b", "", map[string]string{a: fixed, "app/b.txt": "fixed\n"})
return f, mc5
}
// The line can already have what an original does to a file: then a pick
// that leaves the file alone carries the original whole. The check-time
// pass, the picker, fold and verify all take it, binary files included,
// without asking git merge-file what blob equality settles. A different fix
// on the line, or a deletion of what the line never had, is still left out.
func TestAlreadyAppliedFileIsNotOmitted(t *testing.T) {
const textPost = "one\nfixed\nthree\nfour\nfive\n"
accepted := func(t *testing.T, a, old, onLine, fixed string) {
t.Helper()
f, _ := fixedWorld(t, a, old, onLine, fixed)
if code, stdout, stderr := f.exec(nil, "check", "--line", "release-1.1"); code != 1 || !strings.Contains(stdout, "/pull/5 ") || strings.Contains(stderr, "parked") {
t.Errorf("check = %d, want #5 missing and not parked:\n%s\n%s", code, stdout, stderr)
}
gh := &fakeGH{}
head := openBatch(t, f, gh)
if got := f.subjects("origin/release-1.1", head); !slices.Equal(got, []string{"fix(app): a and b (#5)"}) {
t.Errorf("picked %q, want #5", got)
}
if code, out, stderr := f.exec(gh, "verify", "--line", "release-1.1", "--head", head); code != 0 {
t.Errorf("verify = %d\n%s\n%s", code, out, stderr)
}
if code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1", "--cut"); code != 1 || strings.Contains(stdout, "only part of it applies") {
t.Errorf("check --cut = %d, want #5 only on the batch, whole:\n%s\n%s", code, stdout, stderr)
}
f, mc5 := fixedWorld(t, a, old, onLine, fixed)
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/b.txt": "fixed\n"})
fold := w.merge(w.head, pick, "fix(app): a and b (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 0 {
t.Errorf("fold = %d, %q\n%s", code, out, stderr)
}
}
t.Run("the line has the original's file", func(t *testing.T) {
accepted(t, "app/a.txt", "old\n", "fixed\n", "fixed\n")
})
t.Run("the line has the fix beside lines of its own", func(t *testing.T) {
accepted(t, "app/a.txt", strings.Replace(textPost, "fixed", "old", 1), textPost+"line only\n", textPost)
})
t.Run("the line has the original's binary file", func(t *testing.T) {
accepted(t, "app/a.bin", "old\x00payload", "fixed\x00payload", "fixed\x00payload")
f, _ := fixedWorld(t, "app/a.bin", "old\x00payload", "fixed\x00payload", "fixed\x00payload")
b := shaped(t, f)
om, diag, err := b.omittedEffects(b.find(5), b.lineHead, commitOn(f, b.lineHead, map[string]string{"app/b.txt": "fixed\n"}))
if err != nil || len(om)+len(diag) != 0 || b.mergeFiles != 0 {
t.Errorf("omitted = %q, %q, %v, merge-file ran %d times; want nothing, and merge-file never asked", om, diag, err, b.mergeFiles)
}
})
t.Run("an earlier unit of the batch brought the file", func(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a and b", "", map[string]string{"app/a.txt": "old\n", "app/b.txt": "old\n"})
f.lines()
f.pr(4, "fix(app): a", "", map[string]string{"app/a.txt": "fixed\n"})
f.squash(40, "chore: put a back for now", "", map[string]string{"app/a.txt": "old\n"})
f.pr(5, "fix(app): a and b", "", map[string]string{"app/a.txt": "fixed\n", "app/b.txt": "fixed\n"})
gh := &fakeGH{}
head := openBatch(t, f, gh)
if got := f.subjects("origin/release-1.1", head); !slices.Equal(got, []string{"fix(app): a (#4)", "fix(app): a and b (#5)"}) {
t.Errorf("picked %q, want #4 and #5", got)
}
if code, out, stderr := f.exec(gh, "verify", "--line", "release-1.1", "--head", head); code != 0 {
t.Errorf("verify = %d\n%s\n%s", code, out, stderr)
}
})
t.Run("the line has a different fix", func(t *testing.T) {
f, mc5 := fixedWorld(t, "app/a.txt", "old\n", "other\n", "fixed\n")
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/b.txt": "fixed\n"})
fold := w.merge(w.head, pick, "fix(app): a and b (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 1 || !strings.Contains(out, `leaves "app/a.txt" without effect`) {
t.Errorf("fold = %d, %q, want it refused for app/a.txt\n%s", code, out, stderr)
}
})
t.Run("the line has a different binary file", func(t *testing.T) {
f, _ := fixedWorld(t, "app/a.bin", "old\x00payload", "other\x00payload", "fixed\x00payload")
b := shaped(t, f)
om, diag, err := b.omittedEffects(b.find(5), b.lineHead, commitOn(f, b.lineHead, map[string]string{"app/b.txt": "fixed\n"}))
if err != nil || len(om) != 0 || len(diag) != 1 || !strings.Contains(diag[0], `cannot merge "app/a.bin" as text`) {
t.Errorf("omitted = %q, %q, %v; want a diagnostic for app/a.bin, nothing omitted", om, diag, err)
}
})
t.Run("a deletion of what the line never had", func(t *testing.T) {
f, mc5 := policyWorld(t)
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"chart/tests/values_test.yaml": "suite: values\n"})
fold := w.merge(w.head, pick, "fix(chart): remove the policy a rebase brought back (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 1 || !strings.Contains(out, "without effect") {
t.Errorf("fold = %d, %q, want it refused\n%s", code, out, stderr)
}
})
}
// A change that makes none, whatever names its two sides, leaves a deletion
// the line already made as it is: the original is there, not partial.
func TestDeletionAlreadyMadeByNoChange(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a policy", "", map[string]string{"app/policy.yaml": "allow: all\n"})
f.lines()
f.onLine("release-1.1", "fix(app): the line drops the policy", map[string]string{"app/policy.yaml": ""})
f.pr(5, "fix(app): drop the policy", "", map[string]string{"app/policy.yaml": ""})
b := shaped(t, f)
tree := f.git("rev-parse", b.lineHead+"^{tree}")
for _, after := range []string{b.lineHead, tree} {
if om, _, err := b.omittedEffects(b.find(5), b.lineHead, after); err != nil || len(om) != 0 {
t.Errorf("omitted against %.12s = %q, %v; want nothing", after, om, err)
}
}
}
+56
View File
@@ -348,3 +348,59 @@ func TestFoldRefusesADroppedOriginal(t *testing.T) {
t.Errorf("fold = %d, %q, %s; want a refusal naming the drop", code, stdout, stderr)
}
}
// A resolution that ports what the line has of #5 and leaves alone a file
// the line lacks is no fold: what #5 does to that file would be lost, and a
// line-only port is a pull request of its own.
func TestFoldRefusesAnOmittedEffect(t *testing.T) {
f := newFixture(t)
f.lines()
f.pr(3, "feat(app): a feature the line never gets", "", map[string]string{"app/feature2.txt": "v1\n"})
mc5 := f.pr(5, "fix(app): five", "", map[string]string{"app/feature2.txt": "v2\n", "app/five.txt": "5\n"})
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/five.txt": "5\n"})
fold := w.merge(w.head, pick, "fix(app): five (#5)\n\n"+batchTrailer+": "+mc5)
code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold)
if code != 1 || out != "refused: #5 leaves \"app/feature2.txt\" without effect on release-1.1; port only what the line has as a pull request of its own\n" {
t.Errorf("fold = %d, %q, want it refused for app/feature2.txt\n%s", code, out, stderr)
}
}
// A resolution that keeps a file the original deletes leaves the deletion
// without effect.
func TestFoldRefusesAKeptDeletion(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: x", "", map[string]string{"app/x.txt": "x\n"})
f.lines()
mc5 := f.pr(5, "fix(app): five, and x goes", "", map[string]string{"app/x.txt": "", "app/five.txt": "5\n"})
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/five.txt": "5\n"})
fold := w.merge(w.head, pick, "fix(app): five, and x goes (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 1 || !strings.Contains(out, `leaves "app/x.txt" without effect`) {
t.Errorf("fold that keeps app/x.txt = %d, %q\n%s", code, out, stderr)
}
}
// A fold that keeps a file the original deletes leaves the deletion without
// effect, whatever it changes in the file: a port that keeps what the line
// made of it, or the file with a line added, is a pull request of its own.
func TestFoldRefusesAChangedKeptDeletion(t *testing.T) {
for name, kept := range map[string]string{
"the line's own content": "line only\n",
"the file, one line more": "feature\nline only\n# reviewed\n",
} {
t.Run(name, func(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a feature", "", map[string]string{"app/x.txt": "feature\n"})
f.lines()
f.onLine("release-1.1", "fix(app): a line-only setting beside the feature", map[string]string{"app/x.txt": "feature\nline only\n"})
mc5 := f.pr(5, "fix(app): drop the feature, and five", "", map[string]string{"app/x.txt": "", "app/five.txt": "5\n"})
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/x.txt": kept, "app/five.txt": "5\n"})
fold := w.merge(w.head, pick, "fix(app): drop the feature, and five (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 1 || !strings.Contains(out, `leaves "app/x.txt" without effect`) {
t.Errorf("fold = %d, %q, want it refused\n%s", code, out, stderr)
}
})
}
}
+34 -11
View File
@@ -205,6 +205,9 @@ type history struct {
picks []pickRef
batches map[string]string
backports map[int]string
// trees caches what a net merge of an original onto head writes, by its
// merge commit: "" for a conflict.
trees map[string]string
}
func readHistory(head, base string, textUntil time.Time) (*history, error) {
@@ -225,7 +228,7 @@ func readHistory(head, base string, textUntil time.Time) (*history, error) {
// readPicks reads only the messages of head's commits outside base, which is
// all a batch branch is asked about; textUntil is the batch-only day.
func readPicks(head, base string, textUntil time.Time) (*history, error) {
h := &history{head: head, reachable: map[string]bool{}, appended: map[string]string{}, batches: map[string]string{}, backports: map[int]string{}}
h := &history{head: head, reachable: map[string]bool{}, appended: map[string]string{}, batches: map[string]string{}, backports: map[int]string{}, trees: map[string]string{}}
oids, err := revList("--end-of-options", head, "^"+base)
if err != nil {
return nil, err
@@ -332,34 +335,54 @@ func (h *history) carries(o *original, commits []string) (string, error) {
if c, ok := h.backports[o.number]; ok {
return `named by "Backport of" in ` + c[:12], nil
}
noop, err := noopPick(h.head, o)
tree, err := h.netTree(o)
if err != nil {
return "", err
}
if noop {
headTree, err := resolveTree(h.head)
if err != nil {
return "", err
}
if tree == headTree {
return "a pick would change nothing", nil
}
return "", nil
}
// noopPick reports whether cherry-picking the original onto head would leave
// head's tree as it is: the content is there, whatever names it.
func noopPick(head string, o *original) (bool, error) {
// netTree is noopPick's tree for o onto the branch's head, read once.
func (h *history) netTree(o *original) (string, error) {
if tree, ok := h.trees[o.merge]; ok {
return tree, nil
}
tree, _, err := noopPick(h.head, o)
if err == nil {
h.trees[o.merge] = tree
}
return tree, err
}
// noopPick merges the original onto head as one change and returns the tree
// that writes for a clean merge, "" for a conflict, and whether the tree is
// head's own: the content is there, whatever names it.
func noopPick(head string, o *original) (string, bool, error) {
stdout, stderr, code, err := gitStatus("merge-tree", "--write-tree", "--no-messages", "--merge-base="+o.merge+"^1", "--end-of-options", head, o.merge)
switch {
case err != nil:
return false, err
return "", false, err
case code == 1:
return false, nil
return "", false, nil
case code != 0:
return false, fmt.Errorf("git merge-tree for #%d: exit %d: %s", o.number, code, strings.TrimSpace(stderr))
return "", false, fmt.Errorf("git merge-tree for #%d: exit %d: %s", o.number, code, strings.TrimSpace(stderr))
}
tree, _, _ := strings.Cut(stdout, "\n")
if err := checkSHA(tree); err != nil {
return "", false, fmt.Errorf("git merge-tree for #%d: %w", o.number, err)
}
headTree, err := resolveTree(head)
if err != nil {
return false, err
return "", false, err
}
return tree == headTree, nil
return tree, tree == headTree, nil
}
func resolveTree(commit string) (string, error) {
+20
View File
@@ -211,6 +211,13 @@ type verdict struct {
onBatch string
// label is the maintainer override the original carries, if any.
label string
// partial lists what its clean net merge onto the line would leave
// without effect, and park why the batch leaves it out, if it does.
partial []string
park *parkedEntry
// diag is what could not be judged about it, shown whatever became of
// it.
diag []string
}
// batch is everything read about a line before anything is written.
@@ -242,6 +249,16 @@ type batch struct {
// textUntil is the batch-only day: what a commit's text says counts as
// evidence only on commits from before it.
textUntil time.Time
// partialUnits are the originals whose unit on the open batch leaves
// what it lists without effect.
partialUnits map[*verdict][]string
// What one command read once and asks again: renames by commit pair,
// units by batch head, and how often git merge-file ran.
renameCache map[string]renameSet
unitCache map[string]map[*verdict]string
mergeFiles int
// renameLimit is the -l rename detection runs with; 0, git's widest.
renameLimit int
}
// fetch moves to the top of the working tree first: every path the batch
@@ -355,6 +372,9 @@ func (a *app) prepare(l line) (*batch, error) {
}
b.verdicts = append(b.verdicts, v)
}
if err := b.partialStates(); err != nil {
return nil, err
}
return b, nil
}
+331
View File
@@ -0,0 +1,331 @@
/*
Copyright 2026 The Cozystack Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"slices"
"strings"
"testing"
)
var line11, _ = parseLine("release-1.1")
// handBatch pushes a batch carrying merges in the batch's shape, as a run
// made one before the partial guard, and opens its pull request with a
// record that defers nothing.
func handBatch(f *fixture, gh *fakeGH, merges ...string) string {
f.t.Helper()
f.git("fetch", "-q", "origin")
f.git("switch", "-q", "--detach", "origin/release-1.1")
for _, mc := range merges {
if parents := strings.Fields(f.git("rev-list", "--parents", "-n", "1", mc)); len(parents) == 2 {
f.git("cherry-pick", "-x", mc)
continue
}
base := f.git("rev-parse", "HEAD")
for _, c := range strings.Fields(f.git("rev-list", "--reverse", "--no-merges", mc+"^1.."+mc+"^2")) {
f.git("cherry-pick", "-x", c)
}
merge := f.joined(base, "HEAD", f.git("log", "-1", "--format=%B", mc)+"\n\n"+batchTrailer+": "+mc)
f.git("update-ref", "HEAD", merge)
}
f.git("push", "-q", "origin", "HEAD:refs/heads/"+batchBranch1)
f.git("switch", "-q", "main")
gh.prs = []pullRequest{f.batchPR(90, "release-1.1", batchBranch1, "batch\n\n"+deferredBlock(line11, nil, nil, nil)+"\n")}
return f.remoteHead(batchBranch1)
}
// policyWorld is #5088's shape: #3, a feat the line never gets, adds a
// policy and its test; #5 deletes both again and adds a test of its own.
func policyWorld(t *testing.T) (*fixture, string) {
t.Helper()
f := newFixture(t)
f.lines()
f.pr(3, "feat(chart): a network policy", "", map[string]string{"chart/templates/policy.yaml": "kind: NetworkPolicy\n", "chart/tests/policy_test.yaml": "suite: policy\n"})
mc5 := f.pr(5, "fix(chart): remove the policy a rebase brought back", "```release-note\nfix(chart): the stray network policy is gone.\n```\n",
map[string]string{"chart/templates/policy.yaml": "", "chart/tests/policy_test.yaml": "", "chart/tests/values_test.yaml": "suite: values\n"})
return f, mc5
}
// A clean pick that would delete files the line never had, and so carry only
// #5's test, is withheld: #5 is parked as partial, with the paths it would
// leave without effect, and its release note stays out of the body.
func TestPartialPickIsWithheld(t *testing.T) {
f, mc5 := policyWorld(t)
f.pr(6, "fix(app): six", "```release-note\nfix(app): six works.\n```\n", map[string]string{"app/six.txt": "6\n"})
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
if code != 0 {
t.Fatalf("run = %d\n%s", code, stderr)
}
if got := f.subjects("origin/release-1.1", "HEAD"); !slices.Equal(got, []string{"fix(app): six (#6)"}) {
t.Errorf("picked %q, want only #6", got)
}
want := `"parked":[{"original":5,"merge":"` + mc5 + `","kind":"partial","files":["chart/templates/policy.yaml","chart/tests/policy_test.yaml"]}]`
if !strings.Contains(stdout, want) {
t.Errorf("the record does not park #5 as partial (%s):\n%s", want, stdout)
}
if !strings.Contains(stdout, "### Parked") || !strings.Contains(stdout, "`chart/templates/policy.yaml`") {
t.Errorf("the body does not say #5 is parked and why:\n%s", stdout)
}
if strings.Contains(stdout, "stray network policy") || !strings.Contains(stdout, "six works") {
t.Errorf("the release notes are not #6's alone:\n%s", stdout)
}
}
// A file the line removed itself, which no original added after the line
// branched, still leaves #5's deletion without effect: check finds it with no
// record and parks it, and run picks nothing.
func TestPartialWithoutAnAdder(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: add a policy", "", map[string]string{"app/policy.yaml": "allow: all\n"})
f.lines()
f.onLine("release-1.1", "fix(app): the line drops the policy", map[string]string{"app/policy.yaml": ""})
f.pr(5, "fix(app): drop the policy", "", map[string]string{"app/policy.yaml": "", "app/policy_test.txt": "gone\n"})
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
if code != 0 {
t.Fatalf("run = %d\n%s", code, stderr)
}
if got := f.subjects("origin/release-1.1", "HEAD"); len(got) != 0 {
t.Errorf("picked %q, want nothing", got)
}
if !strings.Contains(stdout, "### Parked") {
t.Errorf("the body does not park #5:\n%s", stdout)
}
f.git("switch", "-q", "main")
code, stdout, stderr = f.exec(nil, "check", "--line", "release-1.1")
if code != 0 || !strings.Contains(stderr, "note: #5 is parked: only part of it applies to release-1.1") {
t.Errorf("check = %d, want 0 with a parked-partial note:\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
}
// The picker judges what its picks did, not what the original's merge did:
// #5's first commit deletes a file the line no longer has and comes out
// empty, its second adds a test, and the unit left would be the test alone.
func TestPartialByAnEmptyOwnCommit(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: add a policy", "", map[string]string{"app/policy.yaml": "allow: all\n"})
f.lines()
f.onLine("release-1.1", "fix(app): the line drops the policy", map[string]string{"app/policy.yaml": ""})
f.prCommits(5, "fix(app): drop the policy and test it", "```release-note\nfix(app): no more policy.\n```\n",
change{files: map[string]string{"app/policy.yaml": ""}},
change{files: map[string]string{"app/policy_test.txt": "gone\n"}})
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
if code != 0 {
t.Fatalf("run = %d\n%s", code, stderr)
}
if got := f.subjects("origin/release-1.1", "HEAD"); len(got) != 0 {
t.Errorf("picked %q, want #5 withheld", got)
}
if strings.Contains(stdout, "no more policy") || !strings.Contains(stdout, "### Parked") {
t.Errorf("#5's note is in the body, or #5 is not parked:\n%s", stdout)
}
}
// A batch built before the guard can already carry a partial unit. It fails
// the cut and verify whatever its label says, and --defer takes it off,
// closing a batch it was the last unit of.
func TestExistingPartialUnitFailsTheCut(t *testing.T) {
f, mc5 := policyWorld(t)
gh := &fakeGH{}
head := handBatch(f, gh, mc5)
code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1", "--cut")
if code != 1 || !strings.Contains(stdout, "#5 is on batch #90, but only part of it applies to release-1.1") {
t.Errorf("check --cut = %d, want 1 naming the partial unit:\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
code, stdout, stderr = f.exec(gh, "verify", "--line", "release-1.1", "--head", head)
if code != 1 || !strings.Contains(stdout, `#5 leaves "chart/templates/policy.yaml", "chart/tests/policy_test.yaml" without effect on release-1.1`) {
t.Errorf("verify = %d, want 1 naming what #5 leaves out:\n%s\n%s", code, stdout, stderr)
}
gh.labels = map[string][]int{labelSkip: {5}}
code, stdout, stderr = f.exec(gh, "check", "--line", "release-1.1", "--cut")
if code != 1 || !strings.Contains(stdout, "#5 is on batch #90, but only part of it applies to release-1.1") {
t.Errorf("check --cut with backport/skip = %d, want the partial unit to still fail it:\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
code, _, stderr = f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "only part of it applies", "--push")
if code != 0 || prState(gh.find(90)) != "closed" || f.remoteHead(batchBranch1) != "" {
t.Errorf("run --defer 5 --push = %d, #90 %s, branch at %q; want the batch closed and its branch gone:\n%s", code, prState(gh.find(90)), f.remoteHead(batchBranch1), stderr)
}
}
// An original whose first pick came out empty, because the line already had
// that change, is on the batch all the same: check does not call it missing,
// a continuation keeps it picked, and so does a rebuild without another
// original.
func TestOnBatchAfterAnEmptyFirstPick(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a and b", "", map[string]string{"app/a.txt": "old\n", "app/b.txt": "old\n"})
f.lines()
f.onLine("release-1.1", "fix(app): a, on the line first", map[string]string{"app/a.txt": "fixed\n"})
f.prCommits(5, "fix(app): a and b", "",
change{files: map[string]string{"app/a.txt": "fixed\n"}},
change{files: map[string]string{"app/b.txt": "fixed\n"}})
f.pr(6, "fix(app): six", "", map[string]string{"app/six.txt": "6\n"})
gh := &fakeGH{}
head := openBatch(t, f, gh)
if got := f.subjects("origin/release-1.1", head); !slices.Equal(got, []string{"fix(app): a and b (#5)", "fix(app): six (#6)"}) {
t.Fatalf("the batch carries %q, want #5 and #6", got)
}
code, stdout, stderr := f.exec(gh, "check", "--line", "release-1.1")
if code != 0 {
t.Errorf("check = %d, want 0 with #5 on the batch:\nstdout:\n%s\nstderr:\n%s", code, stdout, stderr)
}
code, stdout, _ = f.exec(gh, "check", "--line", "release-1.1", "--cut")
if code != 1 || !strings.Contains(stdout, "#5 is only on open batch PR #90") {
t.Errorf("check --cut = %d, want #5 only on the batch:\n%s", code, stdout)
}
code, stdout, stderr = f.exec(gh, "run", "--line", "release-1.1", "--continue")
if code != 0 || !strings.Contains(stdout, `"picked":[5,6]`) {
t.Errorf("run --continue = %d, want #5 and #6 picked:\n%s\n%s", code, stdout, stderr)
}
f.git("switch", "-q", "main")
code, stdout, stderr = f.exec(gh, "run", "--line", "release-1.1", "--defer", "6", "--reason", "fails the e2e")
if code != 0 || !strings.Contains(stdout, `"picked":[5]`) || !strings.Contains(stdout, `"missing":[]`) {
t.Errorf("run --defer 6 = %d, want #5 still picked:\n%s\n%s", code, stdout, stderr)
}
}
// The picker judges what its picks did where the run has got to: #4
// deletes the file #5 also deletes, so after #4 #5 would bring only its
// test, which the net merge at the line's head cannot see.
func TestPartialByAnEarlierPick(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a policy", "", map[string]string{"app/policy.yaml": "allow: all\n"})
f.lines()
f.pr(4, "fix(app): drop the policy", "", map[string]string{"app/policy.yaml": "", "app/four.txt": "4\n"})
f.pr(40, "feat(app): the policy again", "", map[string]string{"app/policy.yaml": "allow: all\n"})
f.pr(5, "fix(app): drop the policy and test it", "", map[string]string{"app/policy.yaml": "", "app/policy_test.txt": "gone\n"})
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
if code != 0 || !strings.Contains(stdout, `"picked":[4]`) || !strings.Contains(stdout, `"parked":[{"original":5,`) {
t.Errorf("run = %d, want #4 picked and #5 parked as partial:\n%s\n%s", code, stdout, stderr)
}
}
// What an earlier pick brought clears a partial park the line's head
// gave: the line removed the policy, #4 brings it back, and #5's deletion
// then has something to delete.
func TestPartialClearedByAnEarlierPick(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a policy", "", map[string]string{"app/policy.yaml": "allow: all\n"})
f.lines()
f.onLine("release-1.1", "fix(app): the line drops the policy", map[string]string{"app/policy.yaml": ""})
f.pr(3, "feat(app): no policy", "", map[string]string{"app/policy.yaml": ""})
f.pr(4, "fix(app): the policy, back", "", map[string]string{"app/policy.yaml": "allow: some\n"})
f.pr(5, "fix(app): drop the policy and test it", "", map[string]string{"app/policy.yaml": "", "app/policy_test.txt": "gone\n"})
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1")
if code != 0 || !strings.Contains(stdout, `"picked":[4,5]`) {
t.Errorf("run = %d, want #4 and #5 picked:\n%s\n%s", code, stdout, stderr)
}
}
// A file the line renamed the way the original does is the original's own
// rename, not a deletion left without effect.
func TestSameRenameOnBothSidesIsWhole(t *testing.T) {
const long = "l01\nl02\nl03\nl04\nl05\nl06\nl07\nl08\nl09\nl10\nl11\nl12\n"
f := newFixture(t)
f.squash(2, "chore: a file", "", map[string]string{"app/p.txt": long})
f.lines()
f.onLine("release-1.1", "fix(app): the line renames p to q", map[string]string{"app/p.txt": "", "app/q.txt": long})
f.pr(5, "fix(app): rename p to q, and fix l10", "", map[string]string{"app/p.txt": "", "app/q.txt": strings.Replace(long, "l10", "L10", 1)})
if code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1"); code != 0 || !strings.Contains(stdout, `"picked":[5]`) {
t.Errorf("run = %d, want #5 picked whole:\n%s\n%s", code, stdout, stderr)
}
}
// The line's own rename stands for the original's only with the original's
// change in it: a fold that leaves the renamed file as it was is refused.
func TestSameRenameStillNeedsItsFix(t *testing.T) {
const long = "l01\nl02\nl03\nl04\nl05\nl06\nl07\nl08\nl09\nl10\nl11\nl12\n"
f := newFixture(t)
f.squash(2, "chore: a file", "", map[string]string{"app/p.txt": long})
f.lines()
f.onLine("release-1.1", "fix(app): the line renames p to q", map[string]string{"app/p.txt": "", "app/q.txt": long})
mc5 := f.pr(5, "fix(app): rename p to q, fix l10, and five", "", map[string]string{"app/p.txt": "", "app/q.txt": strings.Replace(long, "l10", "L10", 1), "app/five.txt": "5\n"})
w := emptyBatch(f, &fakeGH{}, "")
pick := w.pick(w.head, f.git("rev-parse", mc5+"^2"), map[string]string{"app/five.txt": "5\n"})
fold := w.merge(w.head, pick, "fix(app): rename p to q, fix l10, and five (#5)\n\n"+batchTrailer+": "+mc5)
if code, out, stderr := f.exec(w.gh, "fold", "--line", "release-1.1", "--original", "5", "--commit", fold); code != 1 || !strings.Contains(out, `leaves "app/p.txt" without effect`) {
t.Errorf("fold without the fix in q = %d, %q\n%s", code, out, stderr)
}
}
// A pick that comes out empty is judged too: #5's own commit did half its
// fix, which the line has, and a merge of main into its branch did the
// rest, which no pick carries; #5 is not on the line.
func TestEmptyPicksStillJudged(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: b", "", map[string]string{"app/b.txt": "old\n"})
f.lines()
f.onLine("release-1.1", "fix(app): b, half of it", map[string]string{"app/b.txt": "middle\n"})
f.git("switch", "-q", "-c", "pr-5", "main")
f.apply(map[string]string{"app/b.txt": "middle\n"})
f.git("commit", "-q", "-m", "b, half of it")
f.git("switch", "-q", "main")
f.apply(map[string]string{"app/main.txt": "moved\n"})
f.git("commit", "-q", "-m", "chore: move main along")
f.git("switch", "-q", "pr-5")
f.git("merge", "-q", "--no-ff", "--no-commit", "main")
f.apply(map[string]string{"app/b.txt": "fixed\n"})
f.git("commit", "-q", "-m", "Merge branch 'main' into pr-5")
f.git("switch", "-q", "main")
f.git("merge", "-q", "--no-ff", "-m", "fix(app): b (#5)", "pr-5")
f.git("push", "-q", "origin", "main")
if code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1"); code != 0 || strings.Contains(stdout, `"on-line":[5]`) || !strings.Contains(stdout, `"parked":[{"original":5,`) {
t.Errorf("run = %d, want #5 parked as partial, not on the line:\n%s\n%s", code, stdout, stderr)
}
}
// --defer takes off a unit that does only part of its original whatever
// rule now keeps that original out, as check tells it to.
func TestDeferTakesOffAPartialExcludedUnit(t *testing.T) {
f := newFixture(t)
f.lines()
f.pr(3, "feat(chart): a network policy", "", map[string]string{"chart/templates/policy.yaml": "kind: NetworkPolicy\n"})
mc5 := f.pr(5, "feat(chart): remove the policy, and test values", "", map[string]string{"chart/templates/policy.yaml": "", "chart/tests/values_test.yaml": "suite: values\n"})
gh := &fakeGH{}
handBatch(f, gh, mc5)
if code, stdout, _ := f.exec(gh, "check", "--line", "release-1.1", "--cut"); code != 1 || !strings.Contains(stdout, "only part of it applies") {
t.Fatalf("check --cut = %d, want the partial unit to fail it:\n%s", code, stdout)
}
if code, _, stderr := f.exec(gh, "run", "--line", "release-1.1", "--defer", "5", "--reason", "only part of it applies", "--push"); code != 0 || prState(gh.find(90)) != "closed" {
t.Errorf("run --defer 5 --push = %d, #90 %s; want the unit taken off and the batch closed:\n%s", code, prState(gh.find(90)), stderr)
}
}
// An empty pick leaves a deletion undone that only a merge of main into the
// original's branch made: the original is not on the line.
func TestEmptyPickLeavesADeletionUndone(t *testing.T) {
f := newFixture(t)
f.squash(2, "chore: a and b", "", map[string]string{"app/a.txt": "old feature\n", "app/b.txt": "old\n"})
f.lines()
f.onLine("release-1.1", "fix(app): b, already", map[string]string{"app/b.txt": "fixed\n"})
f.git("switch", "-q", "-c", "pr-5", "main")
f.apply(map[string]string{"app/b.txt": "fixed\n"})
f.git("commit", "-q", "-m", "b")
f.git("switch", "-q", "main")
f.apply(map[string]string{"app/main.txt": "moved\n"})
f.git("commit", "-q", "-m", "chore: move main along")
f.git("switch", "-q", "pr-5")
f.git("merge", "-q", "--no-ff", "--no-commit", "main")
f.git("rm", "-q", "app/a.txt")
f.git("commit", "-q", "-m", "Merge branch 'main' into pr-5")
f.git("switch", "-q", "main")
f.git("merge", "-q", "--no-ff", "-m", "fix(app): b, and a goes (#5)", "pr-5")
f.git("push", "-q", "origin", "main")
if code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1"); code != 0 || strings.Contains(stdout, `"on-line":[5]`) || !strings.Contains(stdout, `"parked":[{"original":5,`) {
t.Errorf("run = %d, want #5 parked as partial, not on the line:\n%s\n%s", code, stdout, stderr)
}
}
+102 -2
View File
@@ -34,7 +34,8 @@ import (
// carries a record of its own.
// lineRecord is the one line of JSON in the block. Version 1, the block
// before it had a version, holds only line and deferred.
// before it had a version, holds only line and deferred; version 3 adds what
// is parked and which units on the batch do only part of their original.
type lineRecord struct {
Version int `json:"version"`
Line string `json:"line"`
@@ -44,6 +45,80 @@ type lineRecord struct {
Deferred []deferredEntry `json:"deferred"`
Dropped []droppedEntry `json:"dropped"`
Stale []int `json:"stale"`
Parked []parkedEntry `json:"parked,omitempty"`
Partial []int `json:"partial,omitempty"`
}
// recordVersion is the version a run writes.
const recordVersion = 3
// lineRecordV2 is version 2, which has none of version 3's keys.
type lineRecordV2 struct {
Version int `json:"version"`
Line string `json:"line"`
Built *builtFrom `json:"built,omitempty"`
Report string `json:"report,omitempty"`
Closing *closing `json:"closing"`
Deferred []deferredEntry `json:"deferred"`
Dropped []droppedEntry `json:"dropped"`
Stale []int `json:"stale"`
}
// The kinds of park: an original that builds on what never reaches the
// line, and one whose clean pick would leave part of it without effect.
const (
parkDependency = "dependency"
parkPartial = "partial"
)
// parkedEntry is an original the batch leaves out because it cannot land on
// the line whole: what it needs that never comes, or the paths it would
// leave without effect. announced says notify has named it on the open
// batch, for exactly this entry.
type parkedEntry struct {
Original int `json:"original"`
Merge string `json:"merge"`
Kind string `json:"kind"`
Needs []int `json:"needs,omitempty"`
Commits []string `json:"commits,omitempty"`
Files []string `json:"files,omitempty"`
Announced bool `json:"announced,omitempty"`
}
// validate holds a parked entry to what a run writes. Its errors read after
// "the block" or "report <id>".
func (p parkedEntry) validate() error {
sortedUnique := func(n int, less func(i int) bool) bool {
for i := 1; i < n; i++ {
if !less(i) {
return false
}
}
return true
}
switch {
case checkSHA(p.Merge) != nil:
return fmt.Errorf("parks #%d by merge %q", p.Original, p.Merge)
case !sortedUnique(len(p.Needs), func(i int) bool { return p.Needs[i-1] < p.Needs[i] }) || slices.ContainsFunc(p.Needs, func(n int) bool { return n <= 0 || n == p.Original }):
return fmt.Errorf("parks #%d for originals %v, which are not what a run writes", p.Original, p.Needs)
case !sortedUnique(len(p.Commits), func(i int) bool { return p.Commits[i-1] < p.Commits[i] }) || slices.ContainsFunc(p.Commits, func(c string) bool { return checkSHA(c) != nil }):
return fmt.Errorf("parks #%d for commits %q, which are not what a run writes", p.Original, p.Commits)
case !sortedUnique(len(p.Files), func(i int) bool { return p.Files[i-1] < p.Files[i] }) || !slices.Equal(p.Files, cleanFiles(p.Files)):
return fmt.Errorf("parks #%d for files %q, which are not what a run writes", p.Original, p.Files)
}
switch p.Kind {
case parkPartial:
if len(p.Files) == 0 || len(p.Needs)+len(p.Commits) > 0 {
return fmt.Errorf("parks #%d as partial with no files it leaves without effect, or with what it needs", p.Original)
}
case parkDependency:
if len(p.Needs)+len(p.Commits) == 0 || len(p.Files) > 0 {
return fmt.Errorf("parks #%d for a dependency it does not name, or with files", p.Original)
}
default:
return fmt.Errorf("parks #%d as %q, which is no kind of park", p.Original, p.Kind)
}
return nil
}
// builtFrom is what a classification was made of; batch is the head it
@@ -173,9 +248,13 @@ func parseRecord(body string, l line) (*lineRecord, error) {
err = decodeStrict(blocks[0], &v1)
rec = lineRecord{Version: 1, Line: v1.Line, Deferred: v1.Deferred}
case 2:
var v2 lineRecordV2
err = decodeStrict(blocks[0], &v2)
rec = lineRecordWithout(v2)
case 3:
err = decodeStrict(blocks[0], &rec)
default:
return nil, fmt.Errorf("the %s block is version %d, which this backport-batch does not read; it reads 1 and 2", deferredFence, version)
return nil, fmt.Errorf("the %s block is version %d, which this backport-batch does not read; it reads 1 to %d", deferredFence, version, recordVersion)
}
if err != nil {
return nil, fmt.Errorf("reading the %s block: %w", deferredFence, err)
@@ -244,6 +323,14 @@ func (rec *lineRecord) validate(l line) error {
return fmt.Errorf("drops #%d for %q, which is no reason a run writes", d.Original, d.Reason)
}
}
for _, p := range rec.Parked {
if err := once(p.Original); err != nil {
return err
}
if err := p.validate(); err != nil {
return err
}
}
staleSeen := map[int]bool{}
for _, n := range rec.Stale {
if n <= 0 || staleSeen[n] {
@@ -251,6 +338,14 @@ func (rec *lineRecord) validate(l line) error {
}
staleSeen[n] = true
}
partialSeen := map[int]bool{}
for _, n := range rec.Partial {
// A unit that does only part of its original is still on the batch.
if !staleSeen[n] || partialSeen[n] {
return fmt.Errorf("lists #%d as a partial unit more than once, or not as still on the batch", n)
}
partialSeen[n] = true
}
if rec.Version < 2 {
return nil
}
@@ -297,6 +392,11 @@ func (b *builtFrom) validate() error {
var runURLRE = regexp.MustCompile(`^https://[^/\s]+/[^/\s]+/[^/\s]+/actions/runs/[0-9]+(/attempts/[0-9]+)?$`)
// lineRecordWithout is a version 2 record as the newest version holds it.
func lineRecordWithout(v2 lineRecordV2) lineRecord {
return lineRecord{Version: v2.Version, Line: v2.Line, Built: v2.Built, Report: v2.Report, Closing: v2.Closing, Deferred: v2.Deferred, Dropped: v2.Dropped, Stale: v2.Stale}
}
// recordBlock is the block that holds rec. One line of JSON: encoding/json
// escapes newlines and <, > and &, so nothing in a file name or a reason can
// end the fence or open HTML.
+42 -1
View File
@@ -18,6 +18,7 @@ package main
import (
"fmt"
"slices"
"strings"
"testing"
)
@@ -272,7 +273,7 @@ func TestRecordNamesItsInputs(t *testing.T) {
want := builtFrom{Main: f.git("rev-parse", "origin/main"), Line: f.git("rev-parse", "origin/release-1.0"), Newer: f.git("rev-parse", "origin/release-1.1"),
Policy: f.git("rev-parse", "origin/main"), Batch: f.remoteHead(branch), Tool: f.tool, Run: "https://github.com/cozystack/cozystack/actions/runs/123/attempts/2",
At: "2026-10-26T06:00:00Z", Options: runOptions{NoWorkflowFiles: true, Mode: modeNew}}
if rec.Version != 2 || *rec.Built != want || bodyReportOf(t, stdout, l).doc.Built != want || rec.Report != bodyReportOf(t, stdout, l).doc.ID {
if rec.Version != recordVersion || *rec.Built != want || bodyReportOf(t, stdout, l).doc.Built != want || rec.Report != bodyReportOf(t, stdout, l).doc.ID {
t.Errorf("record = %+v, built %+v; want built %+v and the report it names", rec, rec.Built, want)
}
@@ -366,3 +367,43 @@ func TestPRStateNamesOnlyKnownStates(t *testing.T) {
}
}
}
// v3 is v2 as version 3, with extra after the stale list.
func v3(extra string) string {
return strings.Replace(v2(extra), `"version":2`, `"version":3`, 1)
}
// Version 3 adds what is parked and which units do only part of their
// original, each held to what a run writes, and version 2 has neither.
func TestRecordVersion3(t *testing.T) {
l, _ := parseLine("release-1.1")
block := func(json string) string { return "text\n\n" + deferredFence + "\n" + json + "\n```\n" }
sha, other := strings.Repeat("a", 40), strings.Repeat("c", 40)
parked := `,"parked":[{"original":10,"merge":"` + sha + `","kind":"partial","files":["a","b"]},{"original":11,"merge":"` + sha + `","kind":"dependency","needs":[3,4],"commits":["` + other + `"],"announced":true}],"partial":[9]`
rec, err := parseRecord(block(v3(parked)), l)
if err != nil || rec.Version != 3 || len(rec.Parked) != 2 || !rec.Parked[1].Announced || !slices.Equal(rec.Partial, []int{9}) {
t.Fatalf("parseRecord of a whole v3 record = %+v, %v", rec, err)
}
for name, json := range map[string]string{
"version 2 with a park": v2(parked),
"version 4": strings.Replace(v3(""), `"version":3`, `"version":4`, 1),
"a park of a deferred original": v3(strings.Replace(parked, `"original":10`, `"original":5`, 1)),
"a park of a dropped original": v3(strings.Replace(parked, `"original":10`, `"original":8`, 1)),
"a partial park with no files": v3(strings.Replace(parked, `"files":["a","b"]`, `"files":[]`, 1)),
"a partial park that needs": v3(strings.Replace(parked, `"files":["a","b"]`, `"files":["a"],"needs":[3]`, 1)),
"unsorted files": v3(strings.Replace(parked, `"files":["a","b"]`, `"files":["b","a"]`, 1)),
"a file that drives a terminal": v3(strings.Replace(parked, `"files":["a","b"]`, `"files":["a\u001b[2J"]`, 1)),
"a dependency park with no need": v3(strings.Replace(strings.Replace(parked, `"needs":[3,4],`, ``, 1), `"commits":["`+other+`"],`, ``, 1)),
"unsorted needs": v3(strings.Replace(parked, `"needs":[3,4]`, `"needs":[4,3]`, 1)),
"a park that needs itself": v3(strings.Replace(parked, `"needs":[3,4]`, `"needs":[3,11]`, 1)),
"a short commit": v3(strings.Replace(parked, `"commits":["`+other+`"]`, `"commits":["abc"]`, 1)),
"a short merge": v3(strings.Replace(parked, `"merge":"`+sha+`","kind":"partial"`, `"merge":"abc","kind":"partial"`, 1)),
"an unknown kind": v3(strings.Replace(parked, `"kind":"partial"`, `"kind":"later"`, 1)),
"a partial unit not on the batch": v3(strings.Replace(parked, `"partial":[9]`, `"partial":[10]`, 1)),
"a partial unit twice": v3(strings.Replace(parked, `"partial":[9]`, `"partial":[9,9]`, 1)),
} {
if _, err := parseRecord(block(json), l); err == nil {
t.Errorf("%s: parseRecord accepted it", name)
}
}
}
+114 -15
View File
@@ -23,6 +23,7 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"os"
"regexp"
"runtime/debug"
@@ -43,7 +44,9 @@ import (
const (
reportFence = "```backport-batch-report"
reportMarker = "<!-- backport-batch:report "
reportFormat = 1
// reportFormat is the format a run writes. Format 2 adds parked
// originals and partial units; format 1, which has neither, is read too.
reportFormat = 2
// reportLimit keeps a report comment well inside GitHub's 65536.
reportLimit = 60000
)
@@ -67,11 +70,11 @@ type reportOverrides struct {
// reportPrimary puts every original in exactly one place.
type reportPrimary struct {
Picked []int `json:"picked"`
Missing []int `json:"missing"`
Deferred []deferredEntry `json:"deferred"`
Parked []json.RawMessage `json:"parked"`
Skipped reportSkipped `json:"skipped"`
Picked []int `json:"picked"`
Missing []int `json:"missing"`
Deferred []deferredEntry `json:"deferred"`
Parked []parkedEntry `json:"parked"`
Skipped reportSkipped `json:"skipped"`
}
type reportSkipped struct {
@@ -86,12 +89,15 @@ type reportSkipped struct {
}
// reportOverlays mark originals that are also something else: stale ones
// are excluded but still on the batch, and each drop is the record's entry,
// with the merge commit it is bound to and its reason.
// are excluded, or applied only in part, but still on the batch, partial
// ones those whose unit there leaves part of them without effect, and each
// drop is the record's entry, with the merge commit it is bound to and its
// reason.
type reportOverlays struct {
Dropped []droppedEntry `json:"dropped"`
Stale []int `json:"stale"`
Changed []int `json:"changed"`
Partial []int `json:"partial,omitempty"`
}
func (s *reportSkipped) list(rule int) *[]int {
@@ -155,6 +161,18 @@ func deferredEntries(conflicts, held, waiting []pick) []deferredEntry {
return out
}
// parkedEntries is what the record and the report say about what a run
// parked; whether notify named it is the record's alone.
func parkedEntries(vs []*verdict) []parkedEntry {
out := []parkedEntry{}
for _, v := range vs {
p := *v.park
p.Announced = false
out = append(out, p)
}
return out
}
// buildReport makes the report of a run whose built is set. Every original
// main merged since the line branched has to land in exactly one primary
// category; one that does not is an error rather than a report that
@@ -170,7 +188,7 @@ func buildReport(b *batch, r *report) (reportDoc, error) {
doc.Overrides.Skip = append(doc.Overrides.Skip, v.o.number)
}
}
doc.Primary = reportPrimary{Picked: numbers(r.onBranch), Missing: numbers(r.missing), Deferred: deferredEntries(r.deferred, r.held, r.waiting), Parked: []json.RawMessage{}}
doc.Primary = reportPrimary{Picked: numbers(r.onBranch), Missing: numbers(r.missing), Deferred: deferredEntries(r.deferred, r.held, r.waiting), Parked: parkedEntries(r.parked)}
for _, l := range doc.Primary.Skipped.lists() {
*l.list = []int{}
}
@@ -182,6 +200,14 @@ func buildReport(b *batch, r *report) (reportDoc, error) {
*list = append(*list, numbers(vs)...)
}
doc.Overlays = reportOverlays{Dropped: []droppedEntry{}, Stale: numbers(r.stale), Changed: []int{}}
if len(r.partial) > 0 {
doc.Overlays.Partial = numbers(r.partial)
}
for _, v := range b.verdicts {
if len(v.diag) > 0 {
doc.Diag[strconv.Itoa(v.o.number)] = slices.Clone(v.diag)
}
}
doc.Overlays.Dropped = append(doc.Overlays.Dropped, r.dropped...)
doc.canonical()
where, err := doc.categories()
@@ -203,7 +229,7 @@ func buildReport(b *batch, r *report) (reportDoc, error) {
// canonical sorts every list, so equal classifications are equal bytes.
func (d *reportDoc) canonical() {
lists := []*[]int{&d.Primary.Picked, &d.Primary.Missing, &d.Overrides.Include, &d.Overrides.Skip, &d.Overlays.Stale, &d.Overlays.Changed}
lists := []*[]int{&d.Primary.Picked, &d.Primary.Missing, &d.Overrides.Include, &d.Overrides.Skip, &d.Overlays.Stale, &d.Overlays.Changed, &d.Overlays.Partial}
for _, l := range d.Primary.Skipped.lists() {
lists = append(lists, l.list)
}
@@ -211,6 +237,7 @@ func (d *reportDoc) canonical() {
slices.Sort(*l)
}
slices.SortFunc(d.Primary.Deferred, func(a, b deferredEntry) int { return cmp.Compare(a.Original, b.Original) })
slices.SortFunc(d.Primary.Parked, func(a, b parkedEntry) int { return cmp.Compare(a.Original, b.Original) })
slices.SortFunc(d.Overlays.Dropped, func(a, b droppedEntry) int { return cmp.Compare(a.Original, b.Original) })
slices.Sort(d.Other)
}
@@ -244,6 +271,11 @@ func (d *reportDoc) categories() (map[int]string, error) {
return nil, err
}
}
for _, e := range d.Primary.Parked {
if err := put(e.Original, "parked"); err != nil {
return nil, err
}
}
for _, l := range d.Primary.Skipped.lists() {
for _, n := range *l.list {
if err := put(n, "skipped."+l.name); err != nil {
@@ -285,7 +317,7 @@ func parseReport(raw string, l line) (reportDoc, error) {
if err := decodeStrict(raw, &doc); err != nil {
return doc, fmt.Errorf("reading the report: %w", err)
}
if doc.Format != reportFormat {
if doc.Format != 1 && doc.Format != reportFormat {
return doc, fmt.Errorf("the report is format %d, which this backport-batch does not read", doc.Format)
}
if !reportIDRE.MatchString(doc.ID) || reportID(doc) != doc.ID {
@@ -333,7 +365,20 @@ func (d *reportDoc) validate(l line) error {
}
}
}
for _, list := range [][]int{d.Overrides.Include, d.Overrides.Skip, d.Overlays.Stale, d.Overlays.Changed} {
for _, p := range d.Primary.Parked {
if p.Original <= 0 {
return fmt.Errorf("names original %d", p.Original)
}
if err := p.validate(); err != nil {
return err
}
// Whether notify named it is the record's to say, not the
// classification's.
if p.Announced {
return fmt.Errorf("has #%d announced", p.Original)
}
}
for _, list := range [][]int{d.Overrides.Include, d.Overrides.Skip, d.Overlays.Stale, d.Overlays.Changed, d.Overlays.Partial} {
for _, n := range list {
if n <= 0 {
return fmt.Errorf("names original %d", n)
@@ -351,9 +396,40 @@ func (d *reportDoc) links() error {
if err != nil {
return err
}
// PR B gives parked originals a shape; until then none is written.
if len(d.Primary.Parked) > 0 {
return errors.New("parks originals, which this backport-batch does not do")
// Format 1 was written before anything was parked.
if d.Format == 1 && len(d.Primary.Parked)+len(d.Overlays.Partial) > 0 {
return errors.New("parks originals, which format 1 does not")
}
// What a dependency park needs never reaches the line: it is excluded
// by a rule, a label, a drop or a park of its own, or a commit on main
// that merged no pull request.
other := map[string]bool{}
for _, sha := range d.Other {
other[sha] = true
}
for _, p := range d.Primary.Parked {
for _, n := range p.Needs {
switch c := where[n]; c {
case "parked", "skipped.before-start", "skipped.type", "skipped.breaking", "skipped.touches", "skipped.skip-label", "skipped.dropped":
default:
return fmt.Errorf("parks #%d for #%d, which it has %s", p.Original, n, or(c, "nowhere"))
}
}
for _, c := range p.Commits {
if !other[c] {
return fmt.Errorf("parks #%d for %.12s, which it does not count among main's commits that merged no pull request", p.Original, c)
}
}
}
// A partial unit is still on the batch.
stale := map[int]bool{}
for _, n := range d.Overlays.Stale {
stale[n] = true
}
for _, n := range d.Overlays.Partial {
if !stale[n] {
return fmt.Errorf("marks #%d partial on the batch, but not still on it", n)
}
}
drops := map[int]bool{}
for _, e := range d.Overlays.Dropped {
@@ -668,6 +744,23 @@ func (a *app) printReport(pr *pullRequest, s storedReport, inBody *storedReport,
}
lines = append(lines, line{e.Original, text})
}
for _, e := range d.Primary.Parked {
text := "parked: "
switch {
case e.Kind == parkPartial:
text += "only part of it applies, it would leave " + strings.Join(e.Files, ", ") + " without effect"
default:
var needs []string
for _, n := range e.Needs {
needs = append(needs, "#"+strconv.Itoa(n))
}
for _, c := range e.Commits {
needs = append(needs, c[:12])
}
text += "builds on " + strings.Join(needs, ", ") + ", which never reaches the line"
}
lines = append(lines, line{e.Original, text})
}
for _, l := range d.Primary.Skipped.lists() {
for _, n := range *l.list {
lines = append(lines, line{n, "skipped." + l.name})
@@ -683,10 +776,16 @@ func (a *app) printReport(pr *pullRequest, s storedReport, inBody *storedReport,
}
}
overlay("stale, still on the batch", d.Overlays.Stale)
overlay("only part of it applies, still on the batch", d.Overlays.Partial)
for _, e := range d.Overlays.Dropped {
fmt.Fprintf(a.stdout, "dropped: #%d, merged as %.12s, on #%d: %s\n", e.Original, terminalSafe(e.Merge), e.On, terminalSafe(e.Reason))
}
overlay("changed in the pick", d.Overlays.Changed)
for _, n := range slices.Sorted(maps.Keys(d.Diag)) {
for _, ln := range d.Diag[n] {
fmt.Fprintf(a.stdout, "#%s: %s\n", terminalSafe(n), terminalSafe(ln))
}
}
if len(d.Other) > 0 {
other := make([]string, len(d.Other))
for i, sha := range d.Other {
+52 -2
View File
@@ -17,7 +17,6 @@ limitations under the License.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
@@ -602,7 +601,42 @@ func TestReportRefusesWhatItCannotVouchFor(t *testing.T) {
d.Primary.Deferred = []deferredEntry{{Original: 15, Files: []string{".github/workflows/ci.yaml"}, Workflows: true}}
},
"a close that still picks": func(d *reportDoc) { d.Built.Options.Mode = modeClose },
"a parked original": func(d *reportDoc) { d.Primary.Parked = []json.RawMessage{json.RawMessage(`{"original":16}`)} },
"a park with no merge": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Kind: parkPartial, Files: []string{"a"}}}
},
"a parked original it picks": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 5, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"a"}}}
},
"a park of an unknown kind": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: "later", Files: []string{"a"}}}
},
"a partial park that needs an original": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"a"}, Needs: []int{5}}}
},
"a partial park of unsorted files": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"b", "a"}}}
},
"a partial park of a file that drives a terminal": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"a\x1b[2J"}}}
},
"a park announced in the report": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"a"}, Announced: true}}
},
"a park for an original it picks": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkDependency, Needs: []int{5}}}
},
"a park that needs itself": func(d *reportDoc) {
d.Primary.Skipped.Type = []int{17}
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkDependency, Needs: []int{16, 17}}}
},
"a park for a commit that merged a pull request": func(d *reportDoc) {
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkDependency, Commits: []string{strings.Repeat("c", 40)}}}
},
"a partial unit not on the batch": func(d *reportDoc) { d.Overlays.Partial = []int{5} },
"format 1 with a park": func(d *reportDoc) {
d.Format = 1
d.Primary.Parked = []parkedEntry{{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"a"}}}
},
} {
s := forged(change)
c, _ := reportComment(storedReport{doc: forged(func(*reportDoc) {}).doc, raw: s.raw})
@@ -620,6 +654,22 @@ func TestReportRefusesWhatItCannotVouchFor(t *testing.T) {
if code, stdout, stderr := reportRun(&fakeGH{prs: []pullRequest{notifyPR("20261026", "text\n\n"+reportBlock(forged(func(*reportDoc) {}).raw)+"\n")}}, "--pr", "90"); code != 0 || !strings.Contains(stdout, "#5 picked") {
t.Errorf("report of a sound one = %d:\n%s%s", code, stdout, stderr)
}
// What a run wrote before anything was parked reads as it was written.
format1 := forged(func(d *reportDoc) { d.Format, d.Primary.Parked = 1, []parkedEntry{} })
if code, stdout, stderr := reportRun(&fakeGH{prs: []pullRequest{notifyPR("20261026", "text\n\n"+reportBlock(format1.raw)+"\n")}}, "--pr", "90"); code != 0 || !strings.Contains(stdout, "#5 picked") {
t.Errorf("report of a sound format 1 one = %d:\n%s%s", code, stdout, stderr)
}
parked := forged(func(d *reportDoc) {
d.Primary.Skipped.Type = []int{17}
d.Other = append(d.Other, strings.Repeat("c", 40))
d.Primary.Parked = []parkedEntry{
{Original: 16, Merge: strings.Repeat("b", 40), Kind: parkPartial, Files: []string{"chart/templates/policy.yaml"}},
{Original: 18, Merge: strings.Repeat("b", 40), Kind: parkDependency, Needs: []int{16, 17}, Commits: []string{strings.Repeat("c", 40)}},
}
})
if code, stdout, stderr := reportRun(&fakeGH{prs: []pullRequest{notifyPR("20261026", "text\n\n"+reportBlock(parked.raw)+"\n")}}, "--pr", "90"); code != 0 || !strings.Contains(stdout, "#16 parked: only part of it applies") || !strings.Contains(stdout, "#18 parked: builds on #16, #17, cccccccccccc") {
t.Errorf("report of sound parks = %d:\n%s%s", code, stdout, stderr)
}
}
// What report prints stays one line per field and carries nothing a
+40 -6
View File
@@ -17,6 +17,7 @@ limitations under the License.
package main
import (
"errors"
"fmt"
"os"
"slices"
@@ -33,10 +34,11 @@ import (
// pickOriginal picks one original onto HEAD in that shape. A conflict on any
// of its commits takes the branch back to where it was before the original,
// so nothing of it is left half applied. Hooks are off so that each commit
// carries its message and -x line as git writes them; signing is left to the
// user's configuration.
func pickOriginal(v *verdict) (pick, error) {
// so nothing of it is left half applied, and so does a clean pick validate
// finds leaving something of the original without effect: partial, with
// those paths. Hooks are off so that each commit carries its message and -x
// line as git writes them; signing is left to the user's configuration.
func pickOriginal(v *verdict, validate func(base, head string) ([]string, error)) (pick, error) {
p := pick{v: v}
base, err := resolve("HEAD")
if err != nil {
@@ -59,10 +61,34 @@ func pickOriginal(v *verdict) (pick, error) {
switch {
case err != nil:
return p, err
case head == base && validate != nil:
// Picks that change nothing carry the original only when the line
// already has its changes: a change made where its branch merged
// main is in no pick.
omitted, err := validate(base, head)
if err != nil {
return p, err
}
if len(omitted) > 0 {
p.status, p.files = "partial", omitted
return p, nil
}
p.status = "empty"
return p, nil
case head == base:
p.status = "empty"
return p, nil
case v.o.parents < 2:
}
if validate != nil {
omitted, err := validate(base, head)
if err == nil && len(omitted) > 0 {
p.status, p.files = "partial", omitted
}
if err != nil || len(omitted) > 0 {
return p, errors.Join(err, resetTo(base))
}
}
if v.o.parents < 2 {
p.status, p.commit = "picked", head
return p, nil
}
@@ -301,7 +327,8 @@ func checkMerge(b *batch, u unit) (string, *verdict, error) {
}
// contained says why the change from base to tip is not inside the files v
// changes, or "" when it is.
// changes, or the paths a rename on main or on the line gave them at base,
// or "" when it is.
func contained(b *batch, v *verdict, base, tip string) (string, error) {
if err := b.load(v); err != nil {
return "", err
@@ -313,7 +340,14 @@ func contained(b *batch, v *verdict, base, tip string) (string, error) {
if len(changes) == 0 {
return fmt.Sprintf("%.12s changes nothing", tip), nil
}
corr, _, err := b.correspondence(v, base)
if err != nil {
return "", err
}
allowed := paths(v.changes)
for _, c := range corr {
allowed = append(allowed, c)
}
var extra []string
for _, p := range paths(changes) {
if !slices.Contains(allowed, p) {
+9
View File
@@ -91,6 +91,15 @@ func (a *app) verify(args []string) int {
if why == "" && seen[v] != "" {
why = fmt.Sprintf("carries #%d again, which %.12s already carries", v.o.number, seen[v])
}
if why == "" {
omitted, _, err := b.omittedEffects(v, u.parents[0], u.commit)
if err != nil {
return a.fail(err)
}
if len(omitted) > 0 {
why = leftOut(v, l, omitted)
}
}
if why != "" {
refused = append(refused, why)
fmt.Fprintf(a.stdout, "refused %.12s: %s\n", u.commit, why)
+1 -1
View File
@@ -14,7 +14,7 @@ go build ./cmd/backport-batch
./backport-batch check --line release-X.Y --cut
```
[`backport-batch check`](../../cmd/backport-batch/README.md) applies the rules to every pull request merged into `main` since the line branched, and looks for each one they take on the line and then on the open batch pull request: by picks whose `-x` lines name its merge commit or every commit of its own, by a pick that would change nothing, and, on commits from before the policy's batch-only day, by what their text says, such as `Backport of #N`. With `--cut` a change found only on the open batch pull request fails the check too, since a cut tags the line and would not carry it: merge the batch pull request first. A conflict the batch deferred, or an original deferred to the next batch, prints as a warning without failing the check, also once that batch merged or closed. An original that waits on the open batch for a maintainer fails the cut. A change that reached the line by hand counts like any other, by what reached the line.
[`backport-batch check`](../../cmd/backport-batch/README.md) applies the rules to every pull request merged into `main` since the line branched, and looks for each one they take on the line and then on the open batch pull request: by picks whose `-x` lines name its merge commit or every commit of its own, by a pick that would change nothing, and, on commits from before the policy's batch-only day, by what their text says, such as `Backport of #N`. With `--cut` a change found only on the open batch pull request fails the check too, since a cut tags the line and would not carry it: merge the batch pull request first. A conflict the batch deferred, or an original deferred to the next batch, prints as a warning without failing the check, also once that batch merged or closed. An original that waits on the open batch for a maintainer fails the cut, and so does a unit on the open batch that carries only part of its original. A change that reached the line by hand counts like any other, by what reached the line.
### Backporting code that touches monotonic counters
+1 -1
View File
@@ -10,7 +10,7 @@ The [backport batch](../backports.md) is built by [`backport-batch`](../../../cm
### Rules for every task
- Push nothing but the branch of the open batch pull request, `backport-batch/<line>-<yyyymmdd>` in this repository, and only what the task's check accepted. Never push to `main` or a `release-*` branch, and never open a second batch pull request: the scheduled run continues the open one.
- Change no file the original pull request does not change. A conflict that cannot be resolved inside those files is not resolved: the original stays deferred.
- Change no file the original pull request does not change, or the path a rename on `main` or on the line gave it. A conflict that cannot be resolved inside those files is not resolved: the original stays deferred.
- Never rewrite the batch branch by hand. A plain push to it is a fast-forward; only `backport-batch run --defer --push` replaces it, leased on the head it rebuilt.
- Leave the batch pull request's title and body to the tool: it rewrites both on every run, and a hand edit is lost or, inside the `backport-batch` block, makes `run`, `check`, `fold` and `notify` refuse the body, and inside the `backport-batch-report` block makes `run --push` and `backport-batch report` refuse it. Leave the tool's report comments as they are too: each keeps one classification of the line, and an edited one makes `backport-batch report` refuse it.
- When the task cannot be done within these rules, push nothing and leave a comment on the batch pull request saying why, so the next maintainer to look finds it. Nothing is lost by waiting: a red batch does not merge, and a deferred original goes to the next batch.
@@ -34,7 +34,7 @@ git add <resolved files>
git -c core.hooksPath=/dev/null cherry-pick --continue --no-edit
```
When the original builds on something the line does not have, a helper, a value or another pull request, the resolution would have to bring that along, which this task does not allow: abort with `git cherry-pick --abort`, go back with `git switch --detach "$base"`, and comment on the batch pull request which change the original depends on; it stays deferred until that change is on the line. When that change will never reach the line, drop the original there instead, with `backport-batch run --line <line> --drop N --reason "<builds on …>" --push`, so later batches stop trying it. A commit whose change the line already has stops the pick as empty; skip it with `git cherry-pick --skip`.
When the original builds on something the line does not have, a helper, a value or another pull request, the resolution would have to bring that along, which this task does not allow: abort with `git cherry-pick --abort`, go back with `git switch --detach "$base"`, and comment on the batch pull request which change the original depends on; it stays deferred until that change is on the line. When that change will never reach the line, drop the original there instead, with `backport-batch run --line <line> --drop N --reason "<builds on …>" --push`, so later batches stop trying it. A commit whose change the line already has stops the pick as empty; skip it with `git cherry-pick --skip`. A resolution that ports only what the line has and leaves a file of the original as it was, one the line does not have for instance, is refused by `fold`: such a port goes to the line as a pull request of its own, as [fixes that exist only on a release line](../backports.md#fixes-that-exist-only-on-a-release-line) describes, and the original is dropped there.
For a merged pull request, join the picks to the batch head with the merge commit:
+6 -4
View File
@@ -25,6 +25,8 @@ The batch reads the pull requests off `main`'s first-parent history, one per mer
A pull request the rules take for every line can still be wrong for one of them, for instance because it builds on code that line does not have. A maintainer drops it from that line with `./backport-batch run --line <line> --drop <number> --reason <why> --push`: the batch takes it off, if it carries it, and no later batch for that line picks it again. The drop is kept in the [line's record](#the-lines-record-and-reports), so it needs an open batch pull request to record it on; before a line's first batch pull request exists there is nothing to record it on, and the command refuses. `--undrop <number>` withdraws it, and the next batch tries the pull request again. A pull request the line already carries cannot be dropped. To land a hand-made port instead, open it as a pull request of its own, as [fixes that exist only on a release line](#fixes-that-exist-only-on-a-release-line) describes, and drop the original.
The batch also leaves a pull request out of a line on its own when it cannot land there whole. A pick that would leave part of what the pull request does on `main` without effect on the line, a change to a file the line does not have, or the deletion of a file the line never had or already removed, is not made: the pull request is parked, the batch pull request lists it under "Parked" with those files, and `check` prints it as a note without failing. A file the line already has as the pull request makes it counts as there, and so does a file `main` or the line renamed, under its other name. A unit already on the open batch that does only part of its original fails `check --cut` and `backport-batch verify`, whatever its label says, until `backport-batch run --defer` takes it out. What applies of a parked pull request goes to the line as a pull request of its own, as [fixes that exist only on a release line](#fixes-that-exist-only-on-a-release-line) describes.
### What waits for the next minor
`feat`, `chore` and `docs` pull requests, and dependency bumps (scope `deps`), ship with the next minor, not in a patch release; only a maintainer's `backport/include`, below, takes one. If a class of fixes should also wait for a minor, it gets a new rule on this page; a single pull request that the rules get wrong is what the overrides below are for.
@@ -33,7 +35,7 @@ A pull request the rules take for every line can still be wrong for one of them,
A maintainer overrides the rules for one pull request with a label on the merged pull request on `main`:
- `backport/include` takes it although rules 2 to 4 keep it out, a `feat` for instance, or although it merged before its line started under this policy. It still has to pick cleanly or have its conflict resolved, pass CI on the batch, and, on an older line, already be on the newer one, and a line that already carries it does not take it again.
- `backport/include` takes it although rules 2 to 4 keep it out, a `feat` for instance, or although it merged before its line started under this policy. It still has to pick cleanly or have its conflict resolved, pass CI on the batch, and, on an older line, already be on the newer one, and a line that already carries it does not take it again. It lifts no park, as [what the batch leaves out on one line](#what-the-batch-leaves-out-on-one-line) describes.
- `backport/skip` keeps it out although the rules take it, and `backport-batch check` stops asking for it. An open batch that picked it before the label came keeps it until `backport-batch run --defer` takes it out; the batch pull request says so meanwhile.
A label decides for every line. To keep a pull request out of one line only, drop it there with `backport-batch run --drop`, as [what the batch leaves out on one line](#what-the-batch-leaves-out-on-one-line) describes; `backport/include` does not lift a drop.
@@ -65,7 +67,7 @@ released-lines: 2
### Fixes that exist only on a release line
Some fixes go to a release line as a pull request of their own. Such pull requests are occasional, and maintainers open them: for a fix with no counterpart on `main`, for one that on `main` came as part of another pull request, or for one written for `main` rather than for the line. When the counterpart on `main` is a pull request of its own that the batch would take, the batch skips it as already on the line once picking it would change nothing there; otherwise a maintainer labels it `backport/skip`.
Some fixes go to a release line as a pull request of their own. Such pull requests are occasional, and maintainers open them: for a fix with no counterpart on `main`, for one that on `main` came as part of another pull request, or for one written for `main` rather than for the line. When the counterpart on `main` is a pull request of its own that the batch would take, the batch skips it as already on the line once picking it would change nothing there; otherwise a maintainer labels it `backport/skip`. A port of only what a line has of a pull request from `main`, one that leaves out what the line lacks, is such a pull request as well: the batch carries no unit that does part of its original, and `backport-batch fold` refuses one.
A pull request opened against a `release-*` branch is closed automatically, with a comment that links to this page, when its author is a user account, not a bot, that GitHub names neither the repository's owner nor a member nor a collaborator, and whose access to the repository GitHub reports as read or none. Anyone with triage access or more keeps their pull request open. When GitHub's answer on the author's access cannot be read, the check closes nothing and fails, naming the read that failed. The check runs only when a pull request is opened, so a maintainer keeps a contributor's release-only fix by reopening it. [`hack/close-manual-backport.sh`](../../hack/close-manual-backport.sh) does this from [`pr-labeler.yaml`](../../.github/workflows/pr-labeler.yaml); run it with `--pr <number> --dry-run` to see what it would decide for a given pull request when it is opened.
@@ -87,11 +89,11 @@ A run by hand trusts the open batch pull request the scheduled run opened as `co
`./backport-batch lines` prints the lines, newest first. `run`, `check`, `fold` and `lines` read the policy block from this page on `main`, so until the page with its block is on `main` they need `--policy <ref>` naming a ref that has it, and the scheduled run, which passes none, fails.
`run` picks what the rules take onto `backport-batch/<line>-<yyyymmdd>` and opens a draft pull request against the line. A pull request merged into `main` with a merge commit goes onto the batch as its own commits, oldest first, each picked with `git cherry-pick -x --signoff` so it keeps its author, its message and its sign-offs and gains the sign-off of whoever runs the batch; commits that merged `main` into the pull request's branch are left out. A merge commit joins them to the batch: it carries the original's merge message from `main` and ends in a `Backported-from: <merge commit>` trailer. A pull request squash-merged into `main` goes on as one pick. When any commit of an original conflicts, nothing of that original stays on the batch. The batch pull request is merged into the line with a merge commit, so all of this reaches the line as it is. Its body lists the originals with how many commits each brought and which of them differ from the commit they came from, the conflicts it deferred with the commit that failed and its files, and what it skipped under which rule, and ends with the [line's record and the run's report](#the-lines-record-and-reports); `--push` prints the pull request's URL. While that pull request is open, the next run continues it, `--continue` reading its head, rather than opening another. CI on the batch pull request runs the line's build and tests, and a maintainer merges it. Since rule 6 reads the newer line itself, an older line takes a fix in the batch after the newer line's batch with that fix has merged.
`run` picks what the rules take onto `backport-batch/<line>-<yyyymmdd>` and opens a draft pull request against the line. A pull request merged into `main` with a merge commit goes onto the batch as its own commits, oldest first, each picked with `git cherry-pick -x --signoff` so it keeps its author, its message and its sign-offs and gains the sign-off of whoever runs the batch; commits that merged `main` into the pull request's branch are left out. A merge commit joins them to the batch: it carries the original's merge message from `main` and ends in a `Backported-from: <merge commit>` trailer. A pull request squash-merged into `main` goes on as one pick. When any commit of an original conflicts, nothing of that original stays on the batch. The batch pull request is merged into the line with a merge commit, so all of this reaches the line as it is. Its body lists the originals with how many commits each brought and which of them differ from the commit they came from, the conflicts it deferred with the commit that failed and its files, what it parked and why, and what it skipped under which rule, and ends with the [line's record and the run's report](#the-lines-record-and-reports); `--push` prints the pull request's URL. While that pull request is open, the next run continues it, `--continue` reading its head, rather than opening another. CI on the batch pull request runs the line's build and tests, and a maintainer merges it. Since rule 6 reads the newer line itself, an older line takes a fix in the batch after the newer line's batch with that fix has merged.
A deferred conflict waits for the next batch. To land it sooner, pick the original's commits onto the batch head with `git cherry-pick -x`, resolving each conflict, and join them to the batch head with a merge commit that names the original; an original squash-merged into `main` is one pick of its commit there, with no merge. Check that merge or that pick with `./backport-batch fold --line <line> --original <number> --commit <sha>`, and push it onto the batch branch, as [resolving a conflict](./backport-tasks/resolve-conflict.md) describes. CI runs `backport-batch verify` on every push to a batch pull request, which holds every original on it to the same shape. When CI on the batch pull request fails, the original it comes from is taken out with `./backport-batch run --line <line> --defer <number> --reason <why> --push` and goes to the next batch, as [attributing a failed batch CI](./backport-tasks/failed-ci.md) describes; the rest of the batch merges without it. When it is all the batch carries, `--push` records why in the batch pull request, closes it and deletes its branch, and the next batch tries it again. `--drop` and `--undrop` change the record of the open batch pull request in the same way, and need one.
Before a patch release or an rc is cut from a line, `./backport-batch check --line <line> --cut` has to exit 0. A cut tags the line, so the check fails while a pull request the rules take is not on the line, including one that is only on the open batch pull request: merge the batch pull request first. It prints the URLs, and prints what the batch deferred as warnings, since deferring is a maintainer's call; so is a pull request labelled `backport/skip` that the open batch still carries. An original that waits on the batch for a maintainer fails the cut, also when it is all the batch has left. A cut that passes says so. Without `--cut`, as the scheduled run uses it, a change on the open batch pull request counts as carried.
Before a patch release or an rc is cut from a line, `./backport-batch check --line <line> --cut` has to exit 0. A cut tags the line, so the check fails while a pull request the rules take is not on the line, including one that is only on the open batch pull request: merge the batch pull request first. It prints the URLs, and prints what the batch deferred as warnings, since deferring is a maintainer's call; so is a pull request labelled `backport/skip` that the open batch still carries. An original that waits on the batch for a maintainer fails the cut, also when it is all the batch has left, and so does a unit on the batch that carries only part of its original, whatever its label. A cut that passes says so. Without `--cut`, as the scheduled run uses it, a change on the open batch pull request counts as carried.
### The line's record and reports