Files
cozystack/hack/testdata/needs-semantics.yaml
T
Myasnikov Daniil dd308644c8 ci(release): execute the release-changelog logic in tests, not just grep it
The existing suite asserts on workflow YAML. That catches a rename or a
deleted `if:` and nothing else: it cannot tell whether the shell inside a
`run:` block does what the step name claims, and it stayed green while the
logic underneath was wrong — twice, in the two preceding rounds.

Add hack/release-changelog-behaviour.bats, which runs things for real.

The GitHub Actions semantics the whole non-blocking design rests on are
now verified by executing them. hack/testdata/needs-semantics.yaml mirrors
the promote / changelog / open-pr shapes with a fatal (non-AI) step in the
changelog job, and `act` runs it. Result: open-pr DOES run when a job it
needs has failed, `needs.changelog.result` is `failure`, and — the part
that could not be read off the YAML — a failed job's outputs come back
EMPTY, while a successful job's propagate. That empty-output behaviour is
precisely why open-pr re-establishes changelog presence from disk instead
of trusting needs.changelog.outputs. Deleting the `if:` from the fixture
makes open-pr get skipped, so the gate is demonstrably load-bearing.

The git logic moves into hack/changelog-preserve.sh and is exercised
against real bare remotes: a branch that exists, one that does not, one
with no changelog, one with an empty changelog, and the full
preserve/checkout -B/restore/force-push cycle promote actually performs.
The last of these directly reproduces the regression the preserve step
exists to prevent. An empty changelog is deliberately NOT preserved — it
would suppress generation and then publish as blank notes.

finalize's release-body selection runs under node across all three inputs
(good, whitespace-only, absent) and the resulting decision table is
compared as a whole.

Replace the shell overlay with a sparse second actions/checkout of the
dispatch ref at .release-tooling. The overlay had a bootstrap problem I
introduced: the script that fetches missing tooling was itself missing
from the rc tree, so it could never have worked for exactly the trees that
need it. actions/checkout comes from the runner, not the repo, so it has
no such circularity. .release-tooling is gitignored because promote runs
`git add -A`, and the contract suite pins that.

All 21 tests mutation-checked. Breaking changelog-preserve.sh, making it
return an empty file, removing the fixture's `if:`, dropping the tooling
checkout, and renaming its path each fail the suite and pass on restore.

Not covered, and stated in the suite header rather than implied: the AI
generation step, the real GitHub API calls, and whether the sparse
checkout resolves on GitHub's runners.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-07-24 11:34:17 +05:00

37 lines
1.3 KiB
YAML

# Fixture for hack/release-changelog-contract.bats: proves, by RUNNING it, the
# GitHub Actions semantics promote-rc.yaml depends on — that a job needing a
# FAILED job still runs when its `if:` gates only on a different, successful one.
# Mirrors the promote / changelog / open-pr shapes exactly.
name: needs-semantics-fixture
on: [push]
jobs:
promote:
runs-on: ubuntu-latest
outputs:
stable_version: ${{ steps.p.outputs.stable_version }}
steps:
- id: p
run: echo "stable_version=1.6.0" >> "$GITHUB_OUTPUT"
changelog:
runs-on: ubuntu-latest
outputs:
present: ${{ steps.v.outputs.present }}
steps:
# Stands in for `npm i -g @github/copilot` failing: a fatal step in the
# changelog job that is NOT the continue-on-error AI step.
- run: exit 1
- id: v
run: echo "present=true" >> "$GITHUB_OUTPUT"
open-pr:
runs-on: ubuntu-latest
needs: [promote, changelog]
if: ${{ !cancelled() && needs.promote.result == 'success' }}
steps:
- run: |
echo "OPEN_PR_RAN=yes"
echo "changelog_result=${{ needs.changelog.result }}"
echo "changelog_present=[${{ needs.changelog.outputs.present }}]"
echo "stable_version=[${{ needs.promote.outputs.stable_version }}]"