mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 12:13:55 +09:00
feat(release): name what an original builds on that its line lacks
On 1.6, seven of the twelve originals rule 6 holds back build on code the line never got: #5071 changes files #3940 added on main, #4974 reads a variable only #4386 defines. Each conflicts on every run, and nothing tells a maintainer which missing pull request the conflict comes from, which every human backport so far had to work out by hand. For each original the policy takes and nothing carries, run now reads what it builds on from git alone. A modified or deleted file the line lacks is named by the first-parent commit on main that added it (V1). A removed or changed line is blamed at M^1, mapped to the first-parent commit that brought the blamed commit, and kept only while its text, edges trimmed, is absent from the line's file (V2c), so a hand backport under another SHA clears it. Generated files and the keys update-crd.sh writes into an ApplicationDefinition are not blamed. Paths go to git literally, blame gets a validated full SHA and no --end-of-options, which git refuses there, and every call has a two-minute deadline. Caps, deadlines, binary files, symlinks, submodules, unmapped commits and skipped rename detection become diagnostics shown with the original on every outcome and in the report; malformed git output stops the run. This commit only annotates; nothing is parked yet. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
This commit is contained in:
9 files changed
+1058
-11
No files matched your search
@@ -109,6 +109,15 @@ Only the line's own commits count, the ones after the point where it branched fr
|
||||
|
||||
Titles and bodies are read from the merge commits on `main`, which hold them as a maintainer merged them; the live pull request stays editable by its author.
|
||||
|
||||
## What an original builds on
|
||||
|
||||
An original the policy takes can build on code its line does not have: another pull request merged into `main` after the line branched that the line never got, or a commit pushed to `main` directly. For each original the policy takes that neither the line nor the open batch carries, `run` reads what it builds on from git alone, in two ways:
|
||||
|
||||
- A file it modifies or deletes whose counterpart, as [`run`](#run) follows renames, the line does not have is named by the first-parent commit on `main` that added it since the line branched: `git --literal-pathspecs log --first-parent --diff-merges=first-parent --no-renames --diff-filter=A -1 <fork point>..M^1 -- <path>`. A file nothing added since the fork point is shown with the original, since no other pull request brought it.
|
||||
- A line it removes or changes in a file the line has is given by `git blame --porcelain -w`, at `M^1` named by its full commit, to a commit, which counts through the first-parent commit on `main` that brought it: each commit of `main`'s first-parent line since the fork point brings itself and every commit of the range reachable from its other parents that no earlier one brought. A commit from before the fork point, which the line has, and the original's own count for nothing. The commit counts only while one of its lines, with its edges trimmed and nothing else, is absent from the line's version of the file, so a hand backport under another commit clears it. A file the line already has as the original makes it, the same blob or a clean `git merge-file` back into the line's file, builds on nothing.
|
||||
|
||||
Lines a tool wrote are not blamed: a generated file whole, `zz_generated*`, one with a `Code generated … DO NOT EDIT` marker in its first 100 lines, `go.sum` and the package-manager lock files, and in an ApplicationDefinition under `packages/system/*-rd/cozyrds/` the keys `hack/update-crd.sh` writes, found by their indentation. Controller-gen CRDs and `values.schema.json` are blamed like any other file. Each `git log`, `git diff` and `git blame` the detector and the rename detection run has two minutes, and rename detection that runs out of time counts as skipped, and an original has at most 256 files, missing or blamed, 20 000 blamed lines and 8 MiB per file read for it. Evidence it cannot read, past one of those bounds, a binary file, a symlink, a submodule, a commit blame names that did not come through `main`, a path that is not UTF-8, or a rename git skipped looking for, is shown with the original, whatever becomes of it, under its line in the body and in the report's `diag`. Output of git the tool cannot read whole stops it with exit 2, and is never taken for an absent dependency. A dependency carried only by lines an original adds is not seen. `run` prints how long reading the line took on stderr.
|
||||
|
||||
## The line's record and reports
|
||||
|
||||
The line's record is the `backport-batch` block of its newest batch pull request: the newest pull request against the line from a `backport-batch/<line>-<yyyymmdd>` branch of this repository, opened by an author the batch trusts as above, whether it is open, merged or closed. `run`, `check`, `fold` and `notify` find it in GitHub's list of the line's pull requests, newest first, reading at most ten pages of 100. A batch pull request a person opened without the block is passed over. A block whose author the batch does not trust, a bot's batch without its block, a block that cannot be read whole, a closed record newer than the open batch, or ten pages with no record stop the command with exit 2, naming the pull request, rather than fall back to an older record.
|
||||
|
||||
@@ -138,6 +138,7 @@ func (a *app) run(args []string) int {
|
||||
if err := fetch(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
started := time.Now()
|
||||
b, err := a.prepare(l)
|
||||
if err != nil {
|
||||
return a.fail(err)
|
||||
@@ -145,6 +146,12 @@ func (a *app) run(args []string) int {
|
||||
if goOn, code := a.resumeClose(b, *push, deferred != nil || drops != nil); !goOn {
|
||||
return code
|
||||
}
|
||||
if err := b.analyze(); err != nil {
|
||||
return a.fail(err)
|
||||
}
|
||||
// The scheduled job has half an hour for every line; this says where it
|
||||
// goes.
|
||||
fmt.Fprintf(a.stderr, "prepared %s in %s, %s of it judging partial units and what originals build on\n", l.branch(), time.Since(started).Round(time.Millisecond), b.detecting.Round(time.Millisecond))
|
||||
if drops != nil {
|
||||
return a.redrop(b, drops, reason, *undropArg != "", *push)
|
||||
}
|
||||
|
||||
@@ -315,7 +315,7 @@ func (r *report) what(details map[int]bool) string {
|
||||
if s, ok := r.shapes[v]; ok {
|
||||
shape = ", " + s.String()
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s %s%s\n", prLink(v.o.number), titleSpan(v.o.title), shape)
|
||||
fmt.Fprintf(&b, "- %s %s%s\n%s", prLink(v.o.number), titleSpan(v.o.title), shape, diagItems(v))
|
||||
}
|
||||
if len(r.deferred) > 0 {
|
||||
fmt.Fprintf(&b, "\n### Deferred conflicts\n\nThese do not pick cleanly onto this branch. Each names the first of its commits that conflicted and the files it conflicted in, and nothing of it is on the branch. Each moves to the next batch, unless a resolution checked with `backport-batch fold` is pushed here first, as [resolving a conflict](%sresolve-conflict.md) describes.\n\n", tasksURL)
|
||||
@@ -331,13 +331,13 @@ func (r *report) what(details map[int]bool) string {
|
||||
if p.failed != "" {
|
||||
at = " at " + codeSpan(p.failed[:12])
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s %s%s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), at, strings.Join(files, ", "))
|
||||
fmt.Fprintf(&b, "- %s %s%s: %s\n%s", prLink(p.v.o.number), titleSpan(p.v.o.title), at, strings.Join(files, ", "), diagItems(p.v))
|
||||
}
|
||||
}
|
||||
if len(r.held) > 0 {
|
||||
b.WriteString("\n### Deferred to the next batch\n\nTaken out of this batch with `backport-batch run --defer`, each for the reason given. This pull request keeps them out; the next batch picks them again.\n\n")
|
||||
for _, p := range r.held {
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), titleSpan(p.reason))
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n%s", prLink(p.v.o.number), titleSpan(p.v.o.title), titleSpan(p.reason), diagItems(p.v))
|
||||
}
|
||||
}
|
||||
if len(r.waiting) > 0 {
|
||||
@@ -350,7 +350,7 @@ func (r *report) what(details map[int]bool) string {
|
||||
if more := len(p.files) - len(files) + p.more; more > 0 {
|
||||
files = append(files, fmt.Sprintf("%d more", more))
|
||||
}
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(p.v.o.number), titleSpan(p.v.o.title), strings.Join(files, ", "))
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n%s", prLink(p.v.o.number), titleSpan(p.v.o.title), strings.Join(files, ", "), diagItems(p.v))
|
||||
}
|
||||
}
|
||||
if changed := r.changed(); len(changed) > 0 {
|
||||
@@ -378,7 +378,7 @@ func (r *report) what(details map[int]bool) string {
|
||||
if len(r.parked) > 0 {
|
||||
fmt.Fprintf(&b, "\n### Parked\n\nThese cannot land on %s whole, so the batch does not pick them: each would leave part of it without effect there, or builds on a change that never reaches the line. A maintainer ports what applies as a pull request of its own, or drops the original with `backport-batch run --drop`; `backport/include` does not lift a park.\n\n", codeSpan(r.line.branch()))
|
||||
for _, v := range r.parked {
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n", prLink(v.o.number), titleSpan(v.o.title), parkReason(*v.park))
|
||||
fmt.Fprintf(&b, "- %s %s: %s\n%s", prLink(v.o.number), titleSpan(v.o.title), parkReason(*v.park), diagItems(v))
|
||||
}
|
||||
}
|
||||
if len(r.overrides) > 0 {
|
||||
@@ -443,6 +443,16 @@ func (r *report) outcome(v *verdict) string {
|
||||
return "not picked"
|
||||
}
|
||||
|
||||
// diagItems lists, under an original's line, what could not be judged about
|
||||
// it or what it was seen to build on, each as a code span.
|
||||
func diagItems(v *verdict) string {
|
||||
var b strings.Builder
|
||||
for _, d := range v.diag {
|
||||
fmt.Fprintf(&b, " - %s\n", titleSpan(d))
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// fileSpans lists files as code spans, the first few of them.
|
||||
func fileSpans(files []string) string {
|
||||
var out []string
|
||||
|
||||
@@ -0,0 +1,653 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"path"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// What an original builds on that its line lacks is read from git alone, in
|
||||
// two ways: a file it modifies or deletes that the line does not have, named
|
||||
// by the first-parent commit on main that added it (V1), and a line it
|
||||
// removes or changes that git blame gives to a commit the line lacks, kept
|
||||
// only while that line's text is absent from the line's file (V2c). Either
|
||||
// names the first-parent commit that brought the evidence to main: the merge
|
||||
// of another original, or a commit that merged no pull request.
|
||||
|
||||
// Bounds per original, past which the evidence is incomplete rather than
|
||||
// read at any cost.
|
||||
const (
|
||||
maxDepFiles = 256
|
||||
maxBlamedLines = 20000
|
||||
maxDepBlob = 8 << 20
|
||||
gitDeadline = 2 * time.Minute
|
||||
)
|
||||
|
||||
// dependency is a first-parent commit on main an original builds on, with
|
||||
// the original's paths the evidence is in.
|
||||
type dependency struct {
|
||||
merge string
|
||||
// v is the original merge merged, nil for a commit that merged none.
|
||||
v *verdict
|
||||
paths []string
|
||||
// added says some of the evidence is a file the line lacks (V1).
|
||||
added bool
|
||||
}
|
||||
|
||||
// errDeadline is a git call that ran out of time: incomplete evidence, not
|
||||
// a failure.
|
||||
var errDeadline = errors.New("git took longer than " + gitDeadline.String())
|
||||
|
||||
// gitTimed runs git under the deadline the detector gives each call.
|
||||
func gitTimed(args ...string) (string, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), gitDeadline)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "git", args...)
|
||||
cmd.Env = append(cmd.Environ(), commandEnv...)
|
||||
var out, errOut bytes.Buffer
|
||||
cmd.Stdout, cmd.Stderr = &out, &errOut
|
||||
err := cmd.Run()
|
||||
switch {
|
||||
case ctx.Err() != nil:
|
||||
return "", errDeadline
|
||||
case err != nil:
|
||||
return "", fmt.Errorf("git %s: %w: %s", quotedArgs(args), err, terminalSafe(errOut.String()))
|
||||
}
|
||||
return out.String(), nil
|
||||
}
|
||||
|
||||
// quotedArgs names a git command's arguments for an error, each quoted: a
|
||||
// file name can hold a newline, and the error reaches a terminal and a CI
|
||||
// log, which would take the rest for a line of their own.
|
||||
func quotedArgs(args []string) string {
|
||||
out := make([]string, len(args))
|
||||
for i, a := range args {
|
||||
out[i] = strconv.Quote(a)
|
||||
}
|
||||
return strings.Join(out, " ")
|
||||
}
|
||||
|
||||
// gitStatusTimed is gitStatus under the detector's deadline: a non-zero
|
||||
// exit is an answer, and running out of time is errDeadline.
|
||||
func gitStatusTimed(args ...string) (stdout, stderr string, code int, err error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), gitDeadline)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "git", args...)
|
||||
cmd.Env = append(cmd.Environ(), commandEnv...)
|
||||
var out, errOut bytes.Buffer
|
||||
cmd.Stdout, cmd.Stderr = &out, &errOut
|
||||
err = cmd.Run()
|
||||
var ee *exec.ExitError
|
||||
switch {
|
||||
case ctx.Err() != nil:
|
||||
return "", "", -1, errDeadline
|
||||
case errors.As(err, &ee):
|
||||
return out.String(), errOut.String(), ee.ExitCode(), nil
|
||||
case err != nil:
|
||||
return "", "", -1, fmt.Errorf("git %s: %w", quotedArgs(args), err)
|
||||
}
|
||||
return out.String(), errOut.String(), 0, nil
|
||||
}
|
||||
|
||||
// broughtBy maps every commit in base..mainHead to the first-parent commit
|
||||
// that brought it to main: each first-parent commit claims itself, then
|
||||
// every commit of the range reachable from its other parents that no
|
||||
// earlier one claimed. A commit outside the map is an ancestor of base.
|
||||
func broughtBy(base, mainHead string) (map[string]string, error) {
|
||||
first, err := revList("--first-parent", "--reverse", "--end-of-options", base+".."+mainHead)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out, err := git("rev-list", "--parents", "--topo-order", "--end-of-options", base+".."+mainHead)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
parents := map[string][]string{}
|
||||
for ln := range strings.Lines(out) {
|
||||
f := strings.Fields(ln)
|
||||
if len(f) == 0 {
|
||||
continue
|
||||
}
|
||||
for _, sha := range f {
|
||||
if checkSHA(sha) != nil {
|
||||
return nil, fmt.Errorf("rev-list: %q is not a commit", sha)
|
||||
}
|
||||
}
|
||||
parents[f[0]] = f[1:]
|
||||
}
|
||||
by := map[string]string{}
|
||||
for _, fp := range first {
|
||||
if _, ok := parents[fp]; !ok {
|
||||
return nil, fmt.Errorf("rev-list: %s is on main's first-parent line but not in its range", fp)
|
||||
}
|
||||
by[fp] = fp
|
||||
stack := slices.Clone(parents[fp][min(1, len(parents[fp])):])
|
||||
for len(stack) > 0 {
|
||||
c := stack[len(stack)-1]
|
||||
stack = stack[:len(stack)-1]
|
||||
if _, inRange := parents[c]; !inRange || by[c] != "" {
|
||||
continue
|
||||
}
|
||||
by[c] = fp
|
||||
stack = append(stack, parents[c]...)
|
||||
}
|
||||
}
|
||||
return by, nil
|
||||
}
|
||||
|
||||
// hunk is an old range of a diff, a,+b in git blame's terms.
|
||||
type hunk struct{ start, count int }
|
||||
|
||||
var hunkRE = regexp.MustCompile(`^@@ -(\d+)(?:,(\d+))? \+\d+(?:,\d+)? @@`)
|
||||
|
||||
// oldRanges reads the removed or changed line ranges of a -U0 diff of one
|
||||
// file, and whether git called it binary. A header it cannot read is an
|
||||
// error, never evidence of absence.
|
||||
func oldRanges(diff string) ([]hunk, bool, error) {
|
||||
var out []hunk
|
||||
for ln := range strings.Lines(diff) {
|
||||
ln = strings.TrimSuffix(ln, "\n")
|
||||
switch {
|
||||
case strings.HasPrefix(ln, "Binary files ") && strings.HasSuffix(ln, " differ"):
|
||||
return nil, true, nil
|
||||
case strings.HasPrefix(ln, "@@"):
|
||||
m := hunkRE.FindStringSubmatch(ln)
|
||||
if m == nil {
|
||||
return nil, false, fmt.Errorf("unreadable hunk header %q", terminalSafe(ln))
|
||||
}
|
||||
start, _ := strconv.Atoi(m[1])
|
||||
count := 1
|
||||
if m[2] != "" {
|
||||
count, _ = strconv.Atoi(m[2])
|
||||
}
|
||||
if count > 0 {
|
||||
out = append(out, hunk{start, count})
|
||||
}
|
||||
}
|
||||
}
|
||||
return out, false, nil
|
||||
}
|
||||
|
||||
// blameArgs blames ranges of file at rev, a full commit, which git takes for
|
||||
// a revision whatever follows: blame refuses --end-of-options.
|
||||
func blameArgs(rev, file string, ranges []hunk) ([]string, error) {
|
||||
if err := checkSHA(rev); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
args := []string{"blame", "--porcelain", "-w"}
|
||||
for _, r := range ranges {
|
||||
args = append(args, "-L", fmt.Sprintf("%d,+%d", r.start, r.count))
|
||||
}
|
||||
return append(args, rev, "--", file), nil
|
||||
}
|
||||
|
||||
// blamedLine is one line git blame gives to a commit.
|
||||
type blamedLine struct{ commit, text string }
|
||||
|
||||
// parseBlame reads git blame --porcelain: a header per line, the commit's
|
||||
// fields the first time it appears, and the line itself after a tab.
|
||||
// Anything else is an error, never evidence of absence.
|
||||
func parseBlame(out string) ([]blamedLine, error) {
|
||||
var lines []blamedLine
|
||||
commit := ""
|
||||
for ln := range strings.Lines(out) {
|
||||
ln = strings.TrimSuffix(ln, "\n")
|
||||
switch {
|
||||
case strings.HasPrefix(ln, "\t"):
|
||||
if commit == "" {
|
||||
return nil, errors.New("git blame: a line with no header")
|
||||
}
|
||||
lines = append(lines, blamedLine{commit: commit, text: ln[1:]})
|
||||
commit = ""
|
||||
case commit == "":
|
||||
f := strings.Fields(ln)
|
||||
if len(f) < 3 || len(f) > 4 || checkSHA(f[0]) != nil {
|
||||
return nil, fmt.Errorf("git blame: unreadable header %q", terminalSafe(ln))
|
||||
}
|
||||
for _, n := range f[1:] {
|
||||
if v, err := strconv.Atoi(n); err != nil || v <= 0 {
|
||||
return nil, fmt.Errorf("git blame: unreadable header %q", terminalSafe(ln))
|
||||
}
|
||||
}
|
||||
commit = f[0]
|
||||
}
|
||||
}
|
||||
if commit != "" {
|
||||
return nil, errors.New("git blame: a header with no line")
|
||||
}
|
||||
return lines, nil
|
||||
}
|
||||
|
||||
var generatedMarkerRE = regexp.MustCompile(`Code generated .* DO NOT EDIT`)
|
||||
|
||||
// generatedFiles are written whole by a tool, whatever they hold.
|
||||
var generatedFiles = []string{"go.sum", "package-lock.json", "yarn.lock", "pnpm-lock.yaml"}
|
||||
|
||||
// cozyRDGenerated are the keys hack/update-crd.sh writes into an
|
||||
// ApplicationDefinition, the rest of which people write.
|
||||
var cozyRDGenerated = []string{
|
||||
"apiVersion", "kind", "metadata.name", "spec.application.openAPISchema", "spec.release.prefix",
|
||||
"spec.release.chartRef.kind", "spec.release.chartRef.name", "spec.release.chartRef.namespace",
|
||||
"spec.dashboard.description", "spec.dashboard.icon", "spec.dashboard.keysOrder",
|
||||
}
|
||||
|
||||
var yamlKeyRE = regexp.MustCompile(`^( *)([A-Za-z0-9_.-]+):(?:\s|$)`)
|
||||
|
||||
// generatedLines says which lines of a file at a commit a tool wrote: all of
|
||||
// them for a generated file, the keys hack/update-crd.sh sets for an
|
||||
// ApplicationDefinition, found by indentation, and none otherwise. ok is
|
||||
// false for an ApplicationDefinition the scan cannot read, whose lines are
|
||||
// then not blamed at all.
|
||||
func generatedLines(file string, blob []byte) (skip map[int]bool, whole, ok bool) {
|
||||
base := path.Base(file)
|
||||
if strings.HasPrefix(base, "zz_generated") || slices.Contains(generatedFiles, base) {
|
||||
return nil, true, true
|
||||
}
|
||||
lines := strings.Split(string(blob), "\n")
|
||||
for _, ln := range lines[:min(len(lines), 100)] {
|
||||
if generatedMarkerRE.MatchString(ln) {
|
||||
return nil, true, true
|
||||
}
|
||||
}
|
||||
if rd, _ := path.Match(cozyRDGlob, file); !rd {
|
||||
return nil, false, true
|
||||
}
|
||||
skip = map[int]bool{}
|
||||
type key struct {
|
||||
indent int
|
||||
name string
|
||||
}
|
||||
var stack []key
|
||||
genIndent := -1
|
||||
for i, ln := range lines {
|
||||
n := i + 1
|
||||
trimmed := strings.TrimLeft(ln, " ")
|
||||
indent := len(ln) - len(trimmed)
|
||||
switch {
|
||||
case strings.HasPrefix(trimmed, "\t") || strings.HasPrefix(ln, "---") && i > 0:
|
||||
return nil, false, false
|
||||
case trimmed == "" || strings.HasPrefix(trimmed, "#"):
|
||||
if genIndent >= 0 {
|
||||
skip[n] = true
|
||||
}
|
||||
continue
|
||||
case genIndent >= 0 && indent > genIndent:
|
||||
skip[n] = true
|
||||
continue
|
||||
}
|
||||
genIndent = -1
|
||||
m := yamlKeyRE.FindStringSubmatch(ln)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
for len(stack) > 0 && stack[len(stack)-1].indent >= indent {
|
||||
stack = stack[:len(stack)-1]
|
||||
}
|
||||
stack = append(stack, key{indent, m[2]})
|
||||
var names []string
|
||||
for _, k := range stack {
|
||||
names = append(names, k.name)
|
||||
}
|
||||
if slices.Contains(cozyRDGenerated, strings.Join(names, ".")) {
|
||||
skip[n] = true
|
||||
genIndent = indent
|
||||
}
|
||||
}
|
||||
return skip, false, true
|
||||
}
|
||||
|
||||
// trimmedLines is the set of a blob's lines with their edges trimmed, which
|
||||
// is all a reformat of indentation or trailing space changes.
|
||||
func trimmedLines(blob []byte) map[string]bool {
|
||||
set := map[string]bool{}
|
||||
for ln := range strings.Lines(string(blob)) {
|
||||
set[strings.TrimSpace(ln)] = true
|
||||
}
|
||||
return set
|
||||
}
|
||||
|
||||
// shown is a path as a diagnostic names it.
|
||||
func shown(p string) string { return strconv.Quote(p) }
|
||||
|
||||
// incomplete is a diagnostic about evidence the detector could not read.
|
||||
func incomplete(format string, args ...any) string {
|
||||
return oneLineText("incomplete evidence: " + fmt.Sprintf(format, args...))
|
||||
}
|
||||
|
||||
// dependencies reads what v builds on that target lacks, with what it could
|
||||
// not read. V1 looks at the paths whose counterpart target lacks, V2c at
|
||||
// the ones it has.
|
||||
func (b *batch) dependencies(v *verdict, target string) ([]dependency, []string, error) {
|
||||
if err := b.load(v); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if b.brought == nil {
|
||||
var err error
|
||||
if b.brought, err = broughtBy(b.base, b.mainHead); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
}
|
||||
corr, unknown, err := b.correspondence(v, target)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
var diag []string
|
||||
for _, p := range unknown {
|
||||
diag = append(diag, incomplete("git skipped rename detection for %s", shown(p)))
|
||||
}
|
||||
var at []string
|
||||
for _, c := range corr {
|
||||
at = append(at, c)
|
||||
}
|
||||
have, err := treeEntries(target, at)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
parent, err := resolve(v.o.merge + "^1")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
byMerge := map[string]*dependency{}
|
||||
add := func(merge, p string, added bool) {
|
||||
d := byMerge[merge]
|
||||
if d == nil {
|
||||
d = &dependency{merge: merge, v: b.findMerge(merge)}
|
||||
byMerge[merge] = d
|
||||
}
|
||||
if !slices.Contains(d.paths, p) {
|
||||
d.paths = append(d.paths, p)
|
||||
}
|
||||
d.added = d.added || added
|
||||
}
|
||||
files, blamed := 0, 0
|
||||
for _, c := range v.changes {
|
||||
cp, ok := corr[c.path]
|
||||
if !ok || c.status != 'M' && c.status != 'D' {
|
||||
continue
|
||||
}
|
||||
if !utf8.ValidString(c.path) {
|
||||
diag = append(diag, incomplete("a path that is not UTF-8, %s", shown(c.path)))
|
||||
continue
|
||||
}
|
||||
if files++; files > maxDepFiles {
|
||||
diag = append(diag, incomplete("more than %d files to read", maxDepFiles))
|
||||
break
|
||||
}
|
||||
cur, present := have[cp]
|
||||
if !present {
|
||||
out, err := gitTimed("--literal-pathspecs", "log", "--first-parent", "--diff-merges=first-parent", "--no-renames", "--diff-filter=A", "--format=%H", "--no-patch", "-1", "--end-of-options", b.base+".."+parent, "--", c.path)
|
||||
switch {
|
||||
case errors.Is(err, errDeadline):
|
||||
diag = append(diag, incomplete("%s, finding what added %s", err, shown(c.path)))
|
||||
continue
|
||||
case err != nil:
|
||||
return nil, nil, err
|
||||
}
|
||||
adder := strings.TrimSpace(out)
|
||||
if adder == "" {
|
||||
diag = append(diag, oneLineText(fmt.Sprintf("%s is not on the line, and nothing merged into main since it branched added it", shown(c.path))))
|
||||
continue
|
||||
}
|
||||
if checkSHA(adder) != nil {
|
||||
return nil, nil, fmt.Errorf("git log for %s: %q is not a commit", shown(c.path), terminalSafe(adder))
|
||||
}
|
||||
add(adder, c.path, true)
|
||||
continue
|
||||
}
|
||||
found, more, err := b.blamed(v, parent, c.path, cur, target, maxBlamedLines-blamed)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
blamed += more.lines
|
||||
diag = append(diag, more.diag...)
|
||||
for merge := range found {
|
||||
add(merge, c.path, false)
|
||||
}
|
||||
if more.capped {
|
||||
break
|
||||
}
|
||||
}
|
||||
var deps []dependency
|
||||
for _, d := range byMerge {
|
||||
slices.Sort(d.paths)
|
||||
deps = append(deps, *d)
|
||||
}
|
||||
slices.SortFunc(deps, func(x, y dependency) int { return strings.Compare(x.merge, y.merge) })
|
||||
return deps, diag, nil
|
||||
}
|
||||
|
||||
// blameResult is what blaming one file read: how many lines, what it could
|
||||
// not read, and whether the original's line budget ran out.
|
||||
type blameResult struct {
|
||||
lines int
|
||||
diag []string
|
||||
capped bool
|
||||
}
|
||||
|
||||
// blamed gives each line of file that v removes or changes, at parent, to
|
||||
// the first-parent commit that brought it, and keeps the commits one of
|
||||
// whose lines target's counterpart of the file, cur, lacks.
|
||||
func (b *batch) blamed(v *verdict, parent, file string, cur treeEntry, target string, budget int) (map[string]bool, blameResult, error) {
|
||||
var res blameResult
|
||||
entries, err := treeEntries(parent, []string{file})
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
pre := entries[file]
|
||||
switch {
|
||||
case pre.mode == "120000" || cur.mode == "120000":
|
||||
res.diag = append(res.diag, incomplete("%s is a symlink", shown(file)))
|
||||
return nil, res, nil
|
||||
case pre.kind == "commit" || cur.kind == "commit":
|
||||
res.diag = append(res.diag, incomplete("%s is a submodule", shown(file)))
|
||||
return nil, res, nil
|
||||
case !pre.regular() || !cur.regular():
|
||||
res.diag = append(res.diag, incomplete("%s is no file git can blame", shown(file)))
|
||||
return nil, res, nil
|
||||
}
|
||||
// A file the line already has as the original makes it builds on
|
||||
// nothing the line lacks, whoever wrote the lines it replaced.
|
||||
if v.o.merge != "" && cur.oid != pre.oid {
|
||||
post, err := treeEntries(v.o.merge, []string{file})
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
if p1, ok := post[file]; ok && p1.regular() {
|
||||
if cur.oid == p1.oid {
|
||||
return nil, res, nil
|
||||
}
|
||||
applied, _, err := b.alreadyApplied(pre.oid, cur.oid, p1.oid)
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
if applied {
|
||||
return nil, res, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, oid := range []string{pre.oid, cur.oid} {
|
||||
out, err := git("cat-file", "-s", oid)
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
size, err := strconv.Atoi(strings.TrimSpace(out))
|
||||
if err != nil {
|
||||
return nil, res, fmt.Errorf("git cat-file -s %s: %q is no size", oid, terminalSafe(out))
|
||||
}
|
||||
if size > maxDepBlob {
|
||||
res.diag = append(res.diag, incomplete("%s is larger than %d bytes", shown(file), maxDepBlob))
|
||||
return nil, res, nil
|
||||
}
|
||||
}
|
||||
diff, err := gitTimed("--literal-pathspecs", "diff", "--no-ext-diff", "--no-textconv", "--no-renames", "-U0", "--end-of-options", parent, v.o.merge, "--", file)
|
||||
if errors.Is(err, errDeadline) {
|
||||
res.diag = append(res.diag, incomplete("%s, diffing %s", err, shown(file)))
|
||||
return nil, res, nil
|
||||
} else if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
ranges, binary, err := oldRanges(diff)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, res, fmt.Errorf("the diff of %s in #%d: %w", shown(file), v.o.number, err)
|
||||
case binary:
|
||||
res.diag = append(res.diag, incomplete("%s is binary", shown(file)))
|
||||
return nil, res, nil
|
||||
case len(ranges) == 0:
|
||||
return nil, res, nil
|
||||
}
|
||||
blobs, err := readObjects([]string{pre.oid, cur.oid}, "blob")
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
skip, whole, ok := generatedLines(file, blobs[pre.oid])
|
||||
switch {
|
||||
case whole:
|
||||
return nil, res, nil
|
||||
case !ok:
|
||||
res.diag = append(res.diag, incomplete("%s, whose generated keys could not be told apart", shown(file)))
|
||||
return nil, res, nil
|
||||
}
|
||||
var kept []hunk
|
||||
for _, r := range ranges {
|
||||
for n := r.start; n < r.start+r.count; n++ {
|
||||
if skip[n] {
|
||||
continue
|
||||
}
|
||||
if l := len(kept) - 1; l >= 0 && kept[l].start+kept[l].count == n {
|
||||
kept[l].count++
|
||||
} else {
|
||||
kept = append(kept, hunk{n, 1})
|
||||
}
|
||||
res.lines++
|
||||
}
|
||||
}
|
||||
if res.lines > budget {
|
||||
res.diag = append(res.diag, incomplete("more than %d lines to blame", maxBlamedLines))
|
||||
res.capped = true
|
||||
return nil, res, nil
|
||||
}
|
||||
if len(kept) == 0 {
|
||||
return nil, res, nil
|
||||
}
|
||||
args, err := blameArgs(parent, file, kept)
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
out, err := gitTimed(args...)
|
||||
if errors.Is(err, errDeadline) {
|
||||
res.diag = append(res.diag, incomplete("%s, blaming %s", err, shown(file)))
|
||||
return nil, res, nil
|
||||
} else if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
lines, err := parseBlame(out)
|
||||
if err != nil {
|
||||
return nil, res, fmt.Errorf("blaming %s for #%d: %w", shown(file), v.o.number, err)
|
||||
}
|
||||
onTarget := trimmedLines(blobs[cur.oid])
|
||||
found := map[string]bool{}
|
||||
unmapped := false
|
||||
for _, ln := range lines {
|
||||
merge, inRange := b.brought[ln.commit]
|
||||
switch {
|
||||
case !inRange:
|
||||
// An ancestor of the line's fork point, which the line has, or
|
||||
// a commit main does not have at all.
|
||||
before, err := b.beforeBase(ln.commit)
|
||||
if err != nil {
|
||||
return nil, res, err
|
||||
}
|
||||
unmapped = unmapped || !before
|
||||
case merge == v.o.merge:
|
||||
case !onTarget[strings.TrimSpace(ln.text)]:
|
||||
found[merge] = true
|
||||
}
|
||||
}
|
||||
if unmapped {
|
||||
res.diag = append(res.diag, incomplete("git blame gives lines of %s to a commit that did not come through main", shown(file)))
|
||||
}
|
||||
return found, res, nil
|
||||
}
|
||||
|
||||
// timed adds what has passed since start to the time the detector took.
|
||||
func (b *batch) timed(start time.Time) { b.detecting += time.Since(start) }
|
||||
|
||||
// beforeBase says whether commit is an ancestor of the line's fork point,
|
||||
// once per commit.
|
||||
func (b *batch) beforeBase(commit string) (bool, error) {
|
||||
if ok, seen := b.ancestors[commit]; seen {
|
||||
return ok, nil
|
||||
}
|
||||
ok, err := isAncestor(commit, b.base)
|
||||
if err == nil {
|
||||
if b.ancestors == nil {
|
||||
b.ancestors = map[string]bool{}
|
||||
}
|
||||
b.ancestors[commit] = ok
|
||||
}
|
||||
return ok, err
|
||||
}
|
||||
|
||||
// depNote is how a dependency is shown with its original.
|
||||
func (d dependency) note(l line) string {
|
||||
who := fmt.Sprintf("%.12s, a commit on main that merged no pull request", d.merge)
|
||||
if d.v != nil {
|
||||
who = fmt.Sprintf("#%d", d.v.o.number)
|
||||
}
|
||||
quoted := make([]string, len(d.paths))
|
||||
for i, p := range d.paths {
|
||||
quoted[i] = shown(p)
|
||||
}
|
||||
how := "in " + strings.Join(quoted, ", ")
|
||||
if d.added {
|
||||
how = "which added " + strings.Join(quoted, ", ")
|
||||
}
|
||||
return oneLineText(fmt.Sprintf("builds on %s, %s, which %s lacks", who, how, l.branch()))
|
||||
}
|
||||
|
||||
// analyze reads what every original the policy takes and nothing carries
|
||||
// builds on that the line lacks, and keeps it with the original.
|
||||
func (b *batch) analyze() error {
|
||||
defer b.timed(time.Now())
|
||||
for _, v := range b.qualifying() {
|
||||
if v.onBatch != "" {
|
||||
continue
|
||||
}
|
||||
deps, diag, err := b.dependencies(v, b.lineHead)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v.deps = deps
|
||||
v.addDiag(diag...)
|
||||
for _, d := range deps {
|
||||
v.addDiag(d.note(b.line))
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,329 @@
|
||||
/*
|
||||
Copyright 2026 The Cozystack Authors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// depsOf reads what original n builds on that the line lacks.
|
||||
func depsOf(t testing.TB, f *fixture, n int) ([]dependency, []string) {
|
||||
t.Helper()
|
||||
f.git("fetch", "-q", "origin")
|
||||
b, err := shapeOf(line11)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
deps, diag, err := b.dependencies(b.find(n), b.lineHead)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return deps, diag
|
||||
}
|
||||
|
||||
// depNames lists dependencies as #N or a short commit, with their paths.
|
||||
func depNames(deps []dependency) []string {
|
||||
var out []string
|
||||
for _, d := range deps {
|
||||
who := d.merge[:12]
|
||||
if d.v != nil {
|
||||
who = fmt.Sprintf("#%d", d.v.o.number)
|
||||
}
|
||||
out = append(out, fmt.Sprintf("%s %v %v", who, d.paths, d.added))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Every commit main gained since the line branched is brought by one commit
|
||||
// of its first-parent line: a pull request's own commits by its merge, a
|
||||
// direct push by itself, and a commit from before the fork point by none.
|
||||
func TestBroughtBy(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
base := f.git("rev-parse", "origin/release-1.1")
|
||||
mc3 := f.prCommits(3, "feat(app): three", "",
|
||||
change{files: map[string]string{"app/3.txt": "3\n"}},
|
||||
change{mergeMain: true},
|
||||
change{files: map[string]string{"app/3.txt": "3b\n"}})
|
||||
f.write("app/push.txt", "pushed\n")
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "chore: a direct push")
|
||||
f.git("push", "-q", "origin", "main")
|
||||
push := f.git("rev-parse", "HEAD")
|
||||
mc5 := f.pr(5, "fix(app): five", "", map[string]string{"app/five.txt": "5\n"})
|
||||
by, err := broughtBy(base, mc5)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
moved := f.git("rev-parse", mc3+"^1")
|
||||
for _, c := range strings.Fields(f.git("rev-list", mc3+"^1.."+mc3+"^2")) {
|
||||
if c == moved {
|
||||
continue
|
||||
}
|
||||
if by[c] != mc3 {
|
||||
t.Errorf("%.12s, a commit of #3's branch, is brought by %.12s, want #3's merge", c, by[c])
|
||||
}
|
||||
}
|
||||
for c, want := range map[string]string{mc3: mc3, moved: moved, push: push, mc5: mc5, f.git("rev-parse", mc5+"^2"): mc5} {
|
||||
if by[c] != want {
|
||||
t.Errorf("%.12s is brought by %.12s, want %.12s", c, by[c], want)
|
||||
}
|
||||
}
|
||||
if _, ok := by[base]; ok {
|
||||
t.Errorf("the fork point %.12s is in the map", base)
|
||||
}
|
||||
}
|
||||
|
||||
// V1: a file the original changes that the line does not have names the
|
||||
// first-parent commit on main that added it, a merge or a direct push.
|
||||
func TestDependenciesV1(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(3, "feat(app): x", "", map[string]string{"app/x.txt": appFile})
|
||||
f.write("app/y.txt", appFile)
|
||||
f.git("add", "-A")
|
||||
f.git("commit", "-q", "-m", "chore: y, pushed")
|
||||
f.git("push", "-q", "origin", "main")
|
||||
push := f.git("rev-parse", "HEAD")
|
||||
f.pr(5, "fix(app): fix x and y", "", map[string]string{"app/x.txt": "fixed\n", "app/y.txt": "fixed\n"})
|
||||
deps, diag := depsOf(t, f, 5)
|
||||
want := []string{push[:12] + " [app/y.txt] true", "#3 [app/x.txt] true"}
|
||||
slices.Sort(want)
|
||||
got := depNames(deps)
|
||||
slices.Sort(got)
|
||||
if !slices.Equal(got, want) || len(diag) != 0 {
|
||||
t.Errorf("dependencies = %q, %q; want %q", got, diag, want)
|
||||
}
|
||||
}
|
||||
|
||||
// v2World gives the line app/a.txt as the base had it, and main a feat that
|
||||
// changes its second line and a fix of that same line.
|
||||
func v2World(t testing.TB, onLine string) *fixture {
|
||||
f := newFixture(t)
|
||||
f.squash(2, "chore: a", "", map[string]string{"app/a.txt": appFile})
|
||||
f.lines()
|
||||
if onLine != "" {
|
||||
f.onLine("release-1.1", "fix(app): a, by hand", map[string]string{"app/a.txt": onLine})
|
||||
}
|
||||
f.pr(3, "feat(app): a loud two", "", map[string]string{"app/a.txt": strings.Replace(appFile, "two", "TWO", 1)})
|
||||
f.pr(5, "fix(app): an excited two", "", map[string]string{"app/a.txt": strings.Replace(appFile, "two", "TWO!", 1)})
|
||||
return f
|
||||
}
|
||||
|
||||
// V2c: a line the original changes that blame gives to another original is
|
||||
// a dependency only while the line's file lacks that line's text.
|
||||
func TestDependenciesV2c(t *testing.T) {
|
||||
if deps, diag := depsOf(t, v2World(t, ""), 5); !slices.Equal(depNames(deps), []string{"#3 [app/a.txt] false"}) || len(diag) != 0 {
|
||||
t.Errorf("dependencies = %q, %q; want #3 through app/a.txt", depNames(deps), diag)
|
||||
}
|
||||
if deps, _ := depsOf(t, v2World(t, strings.Replace(appFile, "two", "TWO", 1)), 5); len(deps) != 0 {
|
||||
t.Errorf("dependencies with #3's line on the line by hand = %q; want none", depNames(deps))
|
||||
}
|
||||
}
|
||||
|
||||
// The content check trims a line's edges and nothing else: a reindented line
|
||||
// is there, a line with a space inside it is not.
|
||||
func TestDependenciesTrimOnlyEdges(t *testing.T) {
|
||||
if deps, _ := depsOf(t, v2World(t, strings.Replace(appFile, "two", " TWO\t", 1)), 5); len(deps) != 0 {
|
||||
t.Errorf("dependencies with #3's line reindented = %q; want none", depNames(deps))
|
||||
}
|
||||
if deps, _ := depsOf(t, v2World(t, strings.Replace(appFile, "two", "T WO", 1)), 5); !slices.Equal(depNames(deps), []string{"#3 [app/a.txt] false"}) {
|
||||
t.Errorf("dependencies with a space inside #3's line = %q; want #3", depNames(deps))
|
||||
}
|
||||
}
|
||||
|
||||
// Lines a tool wrote are not blamed: a generated file whole, and the keys
|
||||
// hack/update-crd.sh writes into an ApplicationDefinition, while the keys a
|
||||
// person writes there are.
|
||||
func TestDependenciesSkipGeneratedRanges(t *testing.T) {
|
||||
const rd = "packages/system/app-rd/cozyrds/app.yaml"
|
||||
rdFile := func(schema, category string) string {
|
||||
return "apiVersion: cozystack.io/v1alpha1\nkind: ApplicationDefinition\nmetadata:\n name: app\nspec:\n application:\n kind: App\n openAPISchema: |-\n {\"v\":" + schema + "}\n dashboard:\n category: " + category + "\n keysOrder: [[\"spec\", \"" + schema + "\"]]\n"
|
||||
}
|
||||
world := func(t *testing.T, before, feat, fix map[string]string) *fixture {
|
||||
f := newFixture(t)
|
||||
f.squash(2, "chore: generated files", "", before)
|
||||
f.lines()
|
||||
f.pr(3, "feat(api): regenerate", "", feat)
|
||||
f.pr(5, "fix(api): regenerate again", "", fix)
|
||||
return f
|
||||
}
|
||||
gen := "// Code generated by controller-gen. DO NOT EDIT.\n\npackage v1\n\nvar a = 1\n"
|
||||
f := world(t,
|
||||
map[string]string{"api/zz_generated.deepcopy.go": "package v1\n\nvar a = 1\n", "api/types_gen.go": gen, rd: rdFile("1", "A")},
|
||||
map[string]string{"api/zz_generated.deepcopy.go": "package v1\n\nvar a = 2\n", "api/types_gen.go": strings.Replace(gen, "= 1", "= 2", 1), rd: rdFile("2", "A")},
|
||||
map[string]string{"api/zz_generated.deepcopy.go": "package v1\n\nvar a = 3\n", "api/types_gen.go": strings.Replace(gen, "= 1", "= 3", 1), rd: rdFile("3", "A")})
|
||||
if deps, diag := depsOf(t, f, 5); len(deps) != 0 || len(diag) != 0 {
|
||||
t.Errorf("dependencies through generated lines = %q, %q; want none", depNames(deps), diag)
|
||||
}
|
||||
f = world(t, map[string]string{rd: rdFile("1", "A")}, map[string]string{rd: rdFile("2", "B")}, map[string]string{rd: rdFile("3", "C")})
|
||||
if deps, _ := depsOf(t, f, 5); !slices.Equal(depNames(deps), []string{"#3 [" + rd + "] false"}) {
|
||||
t.Errorf("dependencies through a key a person writes = %q; want #3", depNames(deps))
|
||||
}
|
||||
}
|
||||
|
||||
// Evidence the detector cannot read, a binary file or a symlink, is a
|
||||
// diagnostic and never a dependency.
|
||||
func TestDependenciesIncompleteNeverParks(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.squash(2, "chore: an image", "", map[string]string{"app/a.bin": "a\x00one"})
|
||||
f.lines()
|
||||
f.pr(3, "feat(app): a new image", "", map[string]string{"app/a.bin": "a\x00two"})
|
||||
f.pr(5, "fix(app): a fixed image", "", map[string]string{"app/a.bin": "a\x00three"})
|
||||
if deps, diag := depsOf(t, f, 5); len(deps) != 0 || !slices.Equal(diag, []string{`incomplete evidence: "app/a.bin" is binary`}) {
|
||||
t.Errorf("dependencies = %q, %q; want none and a diagnostic", depNames(deps), diag)
|
||||
}
|
||||
}
|
||||
|
||||
// Output of git the detector cannot read whole is an error, never taken
|
||||
// for an absent dependency.
|
||||
func TestDependenciesMalformedOutputIsAnError(t *testing.T) {
|
||||
sha := strings.Repeat("a", 40)
|
||||
for name, out := range map[string]string{
|
||||
"a short header": sha + " 1\n\tline\n",
|
||||
"a header of no commit": "aaaa 1 1 1\n\tline\n",
|
||||
"a header of no number": sha + " x 1 1\n\tline\n",
|
||||
"a line with no header": "\tline\n",
|
||||
"a header with no line": sha + " 1 1 1\nauthor x\n",
|
||||
} {
|
||||
if _, err := parseBlame(out); err == nil {
|
||||
t.Errorf("%s: parseBlame accepted %q", name, out)
|
||||
}
|
||||
}
|
||||
if lines, err := parseBlame(sha + " 2 2 1\nauthor x\nfilename a\n\ttwo\n" + sha + " 4 4\n\tfour\n"); err != nil || len(lines) != 2 || lines[1].text != "four" {
|
||||
t.Errorf("parseBlame of sound output = %+v, %v", lines, err)
|
||||
}
|
||||
if _, _, err := oldRanges("@@ -x +1 @@\n"); err == nil {
|
||||
t.Error("oldRanges accepted an unreadable hunk header")
|
||||
}
|
||||
}
|
||||
|
||||
// Paths go to git as they are: a name a pathspec would read as a pattern
|
||||
// means only itself.
|
||||
func TestDependenciesLiteralPaths(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
f.pr(3, "feat(app): a bracketed file", "", map[string]string{"app/[x].txt": appFile})
|
||||
f.pr(4, "feat(app): x", "", map[string]string{"app/x.txt": appFile})
|
||||
f.pr(5, "fix(app): fix the bracketed file", "", map[string]string{"app/[x].txt": "fixed\n"})
|
||||
if deps, _ := depsOf(t, f, 5); !slices.Equal(depNames(deps), []string{"#3 [app/[x].txt] true"}) {
|
||||
t.Errorf("dependencies = %q; want #3 alone", depNames(deps))
|
||||
}
|
||||
}
|
||||
|
||||
// A file with no final newline blames its last line like any other.
|
||||
func TestDependenciesNoFinalNewline(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.squash(2, "chore: a", "", map[string]string{"app/a.txt": "one\ntwo"})
|
||||
f.lines()
|
||||
f.pr(3, "feat(app): a loud two", "", map[string]string{"app/a.txt": "one\nTWO"})
|
||||
f.pr(5, "fix(app): an excited two", "", map[string]string{"app/a.txt": "one\nTWO!"})
|
||||
if deps, diag := depsOf(t, f, 5); !slices.Equal(depNames(deps), []string{"#3 [app/a.txt] false"}) || len(diag) != 0 {
|
||||
t.Errorf("dependencies = %q, %q; want #3", depNames(deps), diag)
|
||||
}
|
||||
}
|
||||
|
||||
// Blame takes the revision as a full commit name, which git cannot read as
|
||||
// an option, and refuses --end-of-options, so the tool passes none.
|
||||
func TestBlameInvocation(t *testing.T) {
|
||||
sha := strings.Repeat("a", 40)
|
||||
args, err := blameArgs(sha, "-x", []hunk{{2, 3}, {9, 1}})
|
||||
if want := []string{"blame", "--porcelain", "-w", "-L", "2,+3", "-L", "9,+1", sha, "--", "-x"}; err != nil || !slices.Equal(args, want) {
|
||||
t.Errorf("blameArgs = %q, %v; want %q", args, err, want)
|
||||
}
|
||||
for _, rev := range []string{"HEAD", "--output=x", strings.Repeat("a", 39)} {
|
||||
if _, err := blameArgs(rev, "a", nil); err == nil {
|
||||
t.Errorf("blameArgs took %q for a revision", rev)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// What the detector could not read shows with the original whatever became
|
||||
// of it: picked, waiting for a maintainer, or a deferred conflict, in the
|
||||
// body and in the report.
|
||||
func TestDiagnosticsShowOnEveryOutcome(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.squash(2, "chore: images", "", map[string]string{"app/p.bin": "p\x00one", "app/w.bin": "w\x00one", "app/c.bin": "c\x00one", "app/c.txt": "old\n"})
|
||||
f.lines()
|
||||
f.onLine("release-1.1", "fix(app): c, on the line", map[string]string{"app/c.txt": "line\n"})
|
||||
f.pr(5, "fix(app): picked", "", map[string]string{"app/p.bin": "p\x00two"})
|
||||
f.pr(6, "fix(ci): waiting", "", map[string]string{"app/w.bin": "w\x00two", ".github/workflows/six.yaml": "on: push\n"})
|
||||
f.pr(7, "fix(app): conflicting", "", map[string]string{"app/c.bin": "c\x00two", "app/c.txt": "seven\n"})
|
||||
code, stdout, stderr := f.exec(nil, "run", "--line", "release-1.1", "--no-workflow-files")
|
||||
if code != 0 {
|
||||
t.Fatalf("run = %d\n%s", code, stderr)
|
||||
}
|
||||
for _, s := range []struct{ section, file string }{{"[#5]", "p.bin"}, {"### Deferred conflicts", "c.bin"}, {"### Waiting for a maintainer", "w.bin"}} {
|
||||
_, after, ok := strings.Cut(stdout, s.section)
|
||||
if want := " - `incomplete evidence: \"app/" + s.file + "\" is binary`"; !ok || !strings.Contains(after, want) {
|
||||
t.Errorf("the body does not show %q after %s:\n%s", want, s.section, stdout)
|
||||
}
|
||||
}
|
||||
for n, file := range map[int]string{5: "p.bin", 6: "w.bin", 7: "c.bin"} {
|
||||
if want := fmt.Sprintf(`"%d":["incomplete evidence: \"app/%s\" is binary"]`, n, file); !strings.Contains(stdout, want) {
|
||||
t.Errorf("the report lacks %s:\n%s", want, stdout)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// BenchmarkAnalyze reads what forty originals build on, over a history
|
||||
// where half of them build on a feat the line lacks.
|
||||
func BenchmarkAnalyze(b *testing.B) {
|
||||
f := newFixture(b)
|
||||
f.squash(2, "chore: files", "", map[string]string{"app/a.txt": appFile})
|
||||
f.lines()
|
||||
for n := 10; n < 50; n++ {
|
||||
file := fmt.Sprintf("app/f%d.txt", n)
|
||||
if n%2 == 0 {
|
||||
f.pr(n, "feat(app): "+file, "", map[string]string{file: appFile})
|
||||
continue
|
||||
}
|
||||
prev := fmt.Sprintf("app/f%d.txt", n-1)
|
||||
f.pr(n, "fix(app): "+prev, "", map[string]string{prev: strings.Replace(appFile, "two", fmt.Sprint(n), 1), "app/a.txt": strings.Replace(appFile, "three", fmt.Sprint(n), 1)})
|
||||
}
|
||||
f.git("fetch", "-q", "origin")
|
||||
b.ResetTimer()
|
||||
for range b.N {
|
||||
bt, err := shapeOf(line11)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
if err := bt.analyze(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Files the line lacks count toward the files an original may have read
|
||||
// for it, like the ones blamed.
|
||||
func TestDependenciesCapMissingFiles(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.lines()
|
||||
added, fixed := map[string]string{}, map[string]string{}
|
||||
for i := range maxDepFiles + 1 {
|
||||
added[fmt.Sprintf("app/f%03d.txt", i)] = "v1\n"
|
||||
fixed[fmt.Sprintf("app/f%03d.txt", i)] = "v2\n"
|
||||
}
|
||||
f.pr(3, "feat(app): many files", "", added)
|
||||
f.pr(5, "fix(app): all of them", "", fixed)
|
||||
if _, diag := depsOf(t, f, 5); !slices.ContainsFunc(diag, func(d string) bool { return strings.Contains(d, fmt.Sprintf("more than %d files", maxDepFiles)) }) {
|
||||
t.Errorf("diag = %q, want the file cap", diag)
|
||||
}
|
||||
}
|
||||
@@ -17,12 +17,14 @@ limitations under the License.
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A clean pick can still carry only part of an original: git deletes a file
|
||||
@@ -51,9 +53,14 @@ func treeEntries(treeish string, paths []string) (map[string]treeEntry, error) {
|
||||
for len(paths) > 0 {
|
||||
n := min(chunk, len(paths))
|
||||
args := append([]string{"--literal-pathspecs", "ls-tree", "-r", "-z", "--full-tree", "--end-of-options", treeish, "--"}, paths[:n]...)
|
||||
raw, err := git(args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
raw, stderr, code, err := gitStatus(args...)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, fmt.Errorf("git ls-tree %s did not run", terminalSafe(treeish))
|
||||
case code != 0:
|
||||
// The paths stay out of the error: they are file names someone
|
||||
// chose, and the error reaches a terminal.
|
||||
return nil, fmt.Errorf("git ls-tree %s: exit %d: %s", terminalSafe(treeish), code, terminalSafe(stderr))
|
||||
}
|
||||
for _, rec := range splitNUL(raw) {
|
||||
meta, path, ok := strings.Cut(rec, "\t")
|
||||
@@ -87,8 +94,17 @@ func (b *batch) renames(from, to string) (renameSet, error) {
|
||||
}
|
||||
// -l0 lifts git's own limit, which the picker's merge does not share:
|
||||
// a rename the pick follows has to be one the checks of its shape see.
|
||||
stdout, stderr, code, err := gitStatus("diff", "--no-ext-diff", "--no-textconv", "-M", "-l"+strconv.Itoa(b.renameLimit), "--name-status", "-z", "--diff-filter=R", "--end-of-options", from, to)
|
||||
stdout, stderr, code, err := gitStatusTimed("diff", "--no-ext-diff", "--no-textconv", "-M", "-l"+strconv.Itoa(b.renameLimit), "--name-status", "-z", "--diff-filter=R", "--end-of-options", from, to)
|
||||
switch {
|
||||
case errors.Is(err, errDeadline):
|
||||
// Out of time is rename detection skipped: no path is called
|
||||
// absent for want of it.
|
||||
s := renameSet{moved: map[string]string{}}
|
||||
if b.renameCache == nil {
|
||||
b.renameCache = map[string]renameSet{}
|
||||
}
|
||||
b.renameCache[key] = s
|
||||
return s, nil
|
||||
case err != nil:
|
||||
return renameSet{}, err
|
||||
case code != 0:
|
||||
@@ -416,6 +432,7 @@ func (b *batch) onBranch(h *history, v *verdict) (bool, error) {
|
||||
// partial one is found with no record and no adder; a conflict is never
|
||||
// called partial.
|
||||
func (b *batch) partialStates() error {
|
||||
defer b.timed(time.Now())
|
||||
b.partialUnits = map[*verdict][]string{}
|
||||
if b.pr != nil {
|
||||
var err error
|
||||
|
||||
@@ -227,6 +227,19 @@ func TestOmittedEffects(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
// What git says when it fails names file names as text, never as lines a
|
||||
// terminal or a CI log would take for its own.
|
||||
func TestGitErrorsQuoteFileNames(t *testing.T) {
|
||||
newFixture(t)
|
||||
const evil = "x\n::warning::forged"
|
||||
if _, err := gitTimed("log", "--no-such-option", "--", evil); err == nil || strings.Contains(err.Error(), "\n::warning") {
|
||||
t.Errorf("gitTimed error = %v, want the path quoted", err)
|
||||
}
|
||||
if _, err := treeEntries("no-such-revision", []string{evil}); err == nil || strings.Contains(err.Error(), "\n::warning") {
|
||||
t.Errorf("treeEntries error = %v, want the path quoted", err)
|
||||
}
|
||||
}
|
||||
|
||||
// fixedWorld is the sign-off's two-file case: the base has a and b as old,
|
||||
// #5 fixes both, and the line already has a as onLine says, if anything.
|
||||
func fixedWorld(t *testing.T, a, old, onLine, fixed string) (*fixture, string) {
|
||||
|
||||
@@ -33,7 +33,7 @@ import (
|
||||
// main and release lines shaped the way GitHub leaves them: every pull request
|
||||
// lands as a merge commit whose message is its title, its number and its body.
|
||||
type fixture struct {
|
||||
t *testing.T
|
||||
t testing.TB
|
||||
dir string
|
||||
remote string
|
||||
now time.Time
|
||||
@@ -41,7 +41,7 @@ type fixture struct {
|
||||
tool string
|
||||
}
|
||||
|
||||
func newFixture(t *testing.T) *fixture {
|
||||
func newFixture(t testing.TB) *fixture {
|
||||
t.Helper()
|
||||
// A hook or an outer repository can export these into the test process.
|
||||
for _, key := range []string{"GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE", "GIT_COMMON_DIR", "GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES"} {
|
||||
|
||||
@@ -215,6 +215,8 @@ type verdict struct {
|
||||
// without effect, and park why the batch leaves it out, if it does.
|
||||
partial []string
|
||||
park *parkedEntry
|
||||
// deps is what it builds on that the line lacks.
|
||||
deps []dependency
|
||||
// diag is what could not be judged about it, shown whatever became of
|
||||
// it.
|
||||
diag []string
|
||||
@@ -259,6 +261,13 @@ type batch struct {
|
||||
mergeFiles int
|
||||
// renameLimit is the -l rename detection runs with; 0, git's widest.
|
||||
renameLimit int
|
||||
// brought maps main's commits since the fork point to the first-parent
|
||||
// commit that brought each, and ancestors says which commits blame
|
||||
// named are older than the fork point.
|
||||
brought map[string]string
|
||||
ancestors map[string]bool
|
||||
// detecting is how long judging partial units and dependencies took.
|
||||
detecting time.Duration
|
||||
}
|
||||
|
||||
// fetch moves to the top of the working tree first: every path the batch
|
||||
|
||||
Reference in new issue
Block a user