admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-11-08 21:14:56 +09:00
Code Issues Packages Projects Releases Wiki Activity
Files
575b41e1cd84ea438cb2ea35f4c66355f69d2210
cozystack/.github/workflows/pull-requests.yaml
T
Aleksei Sviridkin 954be15341 fix(e2e): bound every read in the report collector and the step that runs it (#3596) 
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does

A single hung read in `hack/cozyreport.sh` did not truncate
`cozyreport.tgz`, it lost the artifact outright. The collector writes
its tarball on its last line, the `Collect report` step carried no
`timeout-minutes` of its own, and `Upload cozyreport.tgz` is the step
directly after it — so a read that never returned spent whatever was
left of the job budget, the job was cancelled, and the upload never ran.
The runs where the report is the only evidence left are the runs that
produced none. `|| true` guards against a non-zero exit, not against a
read that does not return.

Two halves, independent of each other.

**The step gets a ceiling of its own**, 30 minutes, in all three
workflows that run it. The number is measured against the step rather
than divided out of the job: pulling the `Collect report` timings out of
recent e2e runs gives 3 to 42 seconds, on failed jobs as much as on
green ones, and 30 minutes leaves five sixths of the job budget for the
steps that upload what this one produced. It carries `continue-on-error:
true` beside it, because the step's `|| true` is its standing statement
that collecting diagnostics is never fatal and a step killed at its
ceiling never reaches that `|| true` — without the tolerance, a passing
run whose collection ran long would be published as failed, and every
later `failure()` in the job would start meaning "the tests failed or
the report was slow". That tolerance also renders the overrun green, so
a following step turns the collect step's `outcome` into a `::warning::`
annotation: a bound that fires is recorded, and the outer bound is not
exempt from the rule the inner ones follow.

**Every read in the collector is bounded**, against four sections
before, and gates and host commands are reads too. An `if kubectl get
deploy …` that decides whether a module runs costs more unbounded than
anything it guards, because the module below it never starts; `df -h`
blocks indefinitely on a stale mount and takes the tarball with it by
exactly the mechanism above. A gate cut off short is recorded, since
skipping the module leaves the same empty tree as a cluster that never
ran the component, while a gate answering "not installed" is recorded
nowhere — that is the cluster answering correctly, and a marker firing
on every install without KubeVirt is one a reader learns to skip.

The six LINSTOR reads run a command inside a pod and go through a reader
of their own. They are the only reads whose duration a workload sets
rather than the apiserver, and `kubectl exec` returns the remote
command's exit status — a 124 there is either the collector's deadline
or a command that itself exited 124, so the existing classifier would
have named this script's own timeout as the cause of something that may
have happened inside a pod. The new note states what the status leaves
open, and sits beside the output rather than inside it, since two of the
six stream a gzipped tarball. A cut-off bundle is kept rather than
dropped, and the note says so: the emptiness check at those call sites
asks whether the archive lists a first member, which a truncated gzip
does.

What this does not do is bound the report. The pod, VM, VMI and services
walks carry a count cap and a time budget; every other per-object walk
carries neither, so their cost is the cluster's to set, and a failed
install with a hundred not-Ready HelmReleases spends four hundred
bounded reads in one of them. A per-read ceiling removes the read that
never returns, not the thousand reads that each return slowly. The file
header says so where it previously implied the opposite, and the step
ceiling is what covers the rest.

Two frozen lists in the tests are gone, replaced by one invariant
derived from the code: no read below the library guard runs without a
ceiling. Both lists were already wrong. The header named four bounded
sections while the file had ten, and the per-section counts of unbounded
reads only ever matched a redirect into `$VAR/`, which no gate, selector
or command substitution has — so they described the scan rather than the
file.

The guard that held the old exec assumption was satisfied by leaving
every exec unbounded, which is the state the file was in. It now also
requires each exec to be bounded, and requires that reader not to borrow
the classification written for reads whose 124 is unambiguous. Matching
on the verb rather than on the string `kubectl exec` is part of it: half
of these calls spell it `kubectl -n cozy-linstor exec`, which that
string never saw.

Two properties worth knowing before this lands. `kubectl get events -A
--sort-by=.lastTimestamp` sorts client-side, so kubectl buffers the
whole event list and writes nothing until it finishes — under a flat
per-read ceiling a cut-off there leaves a file holding only the marker,
not a short tail. That read had no ceiling at all before, where the same
slowness could take the whole tarball, so this is a narrower failure
rather than a new one. And `kubectl logs -n cozy-kamaji
deployment/kamaji` gains `--tail=2000`, matching every other controller
log in the file: it was the only unbounded-in-size log, which under a
wall-clock ceiling is the one most likely to be cut at its *end*, where
a control-plane bootstrap failure sits. Longer kamaji history than 2000
lines is no longer collected.

Two accepted imperfections in how the change is split across commits,
named rather than left to be found. `docs/agents/e2e-testing.md` keeps
this material in a single shared paragraph that all three commits touch,
so in the middle commit one sentence about the module gates lands a
commit ahead of the gates themselves. And two corrections to the exec
reader — it now keeps the remote command's whole message in the listing
rather than quoting the line `kubectl exec` always appends last, and its
cut-off note names both of the two things that can happen to a partial
bundle instead of asserting one — sit in the last commit rather than in
the one that introduced that reader. Both are true of the branch tip,
where every statement is correct and appears once; the cost of moving
them is another rebase across the one shared paragraph, which is not
worth what it buys.

Refs #3568.


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Downstream repositories

<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.

Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.

IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:

1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note
fix(e2e): every read in the diagnostic report collector now carries a wall-clock ceiling — module gates, the reads that run a command inside a LINSTOR pod, and the host commands included — and a ceiling that fires is recorded in the artifact rather than only in the job log. The workflow step that runs the collector carries a ceiling of its own, so a read that does not return costs one file and a warning annotation instead of consuming the job budget and losing cozyreport.tgz entirely.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added bounded diagnostic collection across Kubernetes, Talos, LINSTOR,
host, controller, and command outputs.
- Reports preserve partial results and identify timeouts, failures,
missing output, and truncated artifacts.
- Added warnings when report collection fails or exceeds its time limit.

- **Bug Fixes**
- Corrected resource targets, selectors, deployment extraction, and
command matching.
- Improved handling for empty selections, unavailable resources, and
missing timeout tools.

- **Documentation**
- Expanded guidance for diagnostic markers, collection limits, timeouts,
and incomplete output.

- **Tests**
- Added coverage for bounded reads, failure reporting, markers,
selectors, logs, and subprocess mocking.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-07 17:47:51 +03:00

888 lines
42 KiB
YAML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
name: Pull Request
env:
# TODO: unhardcode this
REGISTRY: iad.ocir.io/idyksih5sir9/cozystack
on:
pull_request:
# `labeled` makes the promote PR's manual full-e2e opt-in actionable after
# the bot opens it. The plan/resolve guards below discard every other label
# event before it can launch build or E2E work.
types: [opened, synchronize, reopened, labeled]
# Cancel in‑flight runs for the same PR when a new push arrives.
concurrency:
group: pr-${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
# Top-level token defaults to read-only. A job-level block replaces this one
# rather than adding to it, so each job restates every scope it needs.
permissions:
contents: read
jobs:
# Compute the docs-only gate and the per-package build matrix from the PR
# diff. hack/build-matrix.sh parses the root Makefile `build:` target (the
# single source of truth) and emits only the units whose package dir changed,
# escalating to the full set on a shared-dependency change. talos + installer
# are handled by dedicated jobs (see build-matrix.sh for why).
plan:
name: Plan build
runs-on: ubuntu-latest
if: github.event.action != 'labeled' || github.event.label.name == 'full-e2e'
permissions:
contents: read
outputs:
code: ${{ steps.p.outputs.code }}
trigger_map: ${{ steps.p.outputs.trigger_map }}
matrix: ${{ steps.p.outputs.matrix }}
any: ${{ steps.p.outputs.any }}
touched: ${{ steps.p.outputs.touched }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
- name: Compute changed files and build matrix
id: p
env:
BASE_REF: ${{ github.base_ref }}
run: |
git diff --name-only "origin/${BASE_REF}...HEAD" > /tmp/changed.txt
echo "Changed files:"; sed 's/^/ /' /tmp/changed.txt
if grep -qvE '^docs/' /tmp/changed.txt; then
echo "code=true" >> "$GITHUB_OUTPUT"
else
echo "code=false" >> "$GITHUB_OUTPUT"
fi
# docs/agents/contributing.md carries the downstream trigger map, which
# hack/downstream-trigger-map.bats asserts against: every path the map
# cites still exists, and its repository list still matches the PR
# template. Editing the map alone is a docs-only PR, which would skip the
# unit tests and exempt the map from its own guard. This flag lets the
# unit-test job opt back in, and only that job: it deliberately does not
# feed `code`, so a docs PR still builds nothing.
#
# Detect the map by its old path too: a rename is reported as the new path
# alone, so a PR that moves the map would skip the very test that pins the
# literal below to it, and break for whoever pushes next.
git diff --name-only --no-renames "origin/${BASE_REF}...HEAD" > /tmp/changed_norenames.txt
if grep -qxF 'docs/agents/contributing.md' /tmp/changed_norenames.txt; then
echo "trigger_map=true" >> "$GITHUB_OUTPUT"
else
echo "trigger_map=false" >> "$GITHUB_OUTPUT"
fi
matrix="$(hack/build-matrix.sh /tmp/changed.txt)"
echo "Build matrix: $matrix"
echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
# Package dirs the PR edited (packages/<area>/<name>). Their committed
# refs are the PR's intent and must survive the current-main overlay —
# otherwise an upstream image bump in a non-build-unit package would be
# silently reverted in e2e/the installer artifact (see
# hack/overlay-main-images.sh). Whitespace-separated, no trailing slash.
touched="$(sed -nE 's#^(packages/[^/]+/[^/]+)/.*#\1#p' /tmp/changed.txt | sort -u | tr '\n' ' ')"
echo "Touched packages: $touched"
echo "touched=$touched" >> "$GITHUB_OUTPUT"
if [ "$matrix" = "[]" ]; then
echo "any=false" >> "$GITHUB_OUTPUT"
else
echo "any=true" >> "$GITHUB_OUTPUT"
fi
# Helm-unittest + Go controller tests. Decoupled from the image build and from
# e2e so they give fast, independent signal; they gate the PR as their own
# required check rather than blocking e2e from starting.
checks:
name: Unit & controller tests
runs-on: oracle-vm-4cpu-16gb-x86-64
timeout-minutes: 30
permissions:
contents: read
needs: ["plan"]
# trigger_map opts a docs-only PR back in when it edits the downstream trigger
# map, so hack/downstream-trigger-map.bats still guards it. No other job reads
# that flag: a docs PR runs the unit tests and nothing heavier.
if: |
(needs.plan.outputs.code == 'true' || needs.plan.outputs.trigger_map == 'true')
&& !contains(github.event.pull_request.labels.*.name, 'release')
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
# Ephemeral runners don't carry the helm-unittest plugin the persistent
# self-hosted runner accreted. Install if absent (idempotent).
- name: Set up test toolchain
run: |
helm plugin list 2>/dev/null | grep -q unittest \
|| helm plugin install https://github.com/helm-unittest/helm-unittest
- name: Run unit tests
run: make unit-tests
- name: Run controller Go tests
run: make test-controllers
# One image (package) per job, fanned out across the ephemeral pool. Each job
# gets its own buildx (docker-container driver) so the shared-buildkit
# contention that wedged the old single Build job (#2937) is severed by
# construction. Each job patches only its own package's values.yaml/.tag files
# and uploads that diff as a patch fragment; `finalize` reassembles them.
build:
name: Build ${{ matrix.pkg }}
needs: ["plan"]
if: |
needs.plan.outputs.code == 'true'
&& needs.plan.outputs.any == 'true'
&& !contains(github.event.pull_request.labels.*.name, 'release')
runs-on: oracle-vm-4cpu-16gb-x86-64
timeout-minutes: 30
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
pkg: ${{ fromJSON(needs.plan.outputs.matrix) }}
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
fetch-tags: true
# installer is built in finalize, but some package image targets shell out
# to flux; install it if absent (idempotent, cheap).
- name: Set up build toolchain
run: |
if ! command -v flux >/dev/null; then
install_script="$(mktemp)"
curl -fsSL https://fluxcd.io/install.sh -o "$install_script"
sudo bash "$install_script"
rm -f "$install_script"
fi
- name: Set up Docker config
run: |
if [ -d ~/.docker ]; then
cp -r ~/.docker "${{ runner.temp }}/.docker"
fi
- name: Login to GitHub Container Registry
if: ${{ !github.event.pull_request.head.repo.fork }}
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: ${{ secrets.OCIR_USER }}
password: ${{ secrets.OCIR_TOKEN }}
registry: iad.ocir.io
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
# docker-container driver: isolated buildkit per job (no shared-buildkit
# contention) and a prerequisite for the mode=max cache reads (#2938).
# Its buildkitd also routes docker.io base-image pulls through
# mirror.gcr.io (hack/buildkitd.toml): the Build step authenticates only
# to the push registry (OCIR), so base images (golang/ubuntu/nginx/...)
# would otherwise be pulled from Docker Hub anonymously and intermittently
# trip its anonymous-pull rate limit; the mirror sidesteps that and falls
# back to Docker Hub on a miss/outage. Create the builder under the SAME
# DOCKER_CONFIG the Build step reads (runner.temp/.docker, holding the
# OCIR auth) so buildx forwards push/cache auth to the container builder.
- name: Set up Buildx (docker-container driver, docker.io mirror)
id: buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
with:
driver: docker-container
buildkitd-config: hack/buildkitd.toml
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
- name: Build ${{ matrix.pkg }}
run: make -C "$PKG" image
env:
PKG: ${{ matrix.pkg }}
DOCKER_CONFIG: ${{ runner.temp }}/.docker
# Build on the docker-container builder created above so base-image
# pulls go through the docker.io mirror (BUILDX_ARGS adds
# --builder=$(BUILDER)).
BUILDER: ${{ steps.buildx.outputs.name }}
# Unique per (PR, commit) so concurrent PR builds never push the same
# tag. PR images are consumed by digest; :latest belongs to releases.
IMAGE_TAG: pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
PUBLISH_VERSIONED: '0'
PUBLISH_FLOATING: '0'
- name: Capture digest patch fragment
id: frag
env:
PKG: ${{ matrix.pkg }}
run: |
safe="$(echo "$PKG" | tr '/' '-')"
mkdir -p _out/fragments
# --binary so vendored binary provider bundles (components*.gz),
# which some package image targets regenerate, survive reassembly
# in finalize (a plain "Binary files differ" stub is unappliable).
git diff --binary HEAD > "_out/fragments/${safe}.patch"
echo "safe=${safe}" >> "$GITHUB_OUTPUT"
echo "Fragment for ${PKG}:"; git --no-pager diff --stat HEAD
- name: Upload patch fragment
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: pr-patch-fragment-${{ steps.frag.outputs.safe }}
path: _out/fragments/${{ steps.frag.outputs.safe }}.patch
if-no-files-found: error
# Talos is its own leg: the nocloud disk and the installer tarball are heavy
# and shared through _out/assets, so the talos image and the disk are built in
# one job. Always runs on non-docs PRs — e2e needs the disk regardless of the
# per-package matrix scope.
build-talos:
name: Build Talos
needs: ["plan"]
if: |
needs.plan.outputs.code == 'true'
&& !contains(github.event.pull_request.labels.*.name, 'release')
runs-on: oracle-vm-4cpu-16gb-x86-64
timeout-minutes: 40
permissions:
contents: read
packages: write
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
fetch-tags: true
- name: Set up Docker config
run: |
if [ -d ~/.docker ]; then
cp -r ~/.docker "${{ runner.temp }}/.docker"
fi
- name: Login to GitHub Container Registry
if: ${{ !github.event.pull_request.head.repo.fork }}
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: ${{ secrets.OCIR_USER }}
password: ${{ secrets.OCIR_TOKEN }}
registry: iad.ocir.io
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
# buildkitd-config routes docker.io base-image pulls through mirror.gcr.io
# (hack/buildkitd.toml) so the talos image build dodges Docker Hub's
# anonymous-pull rate limit; see the build job for the full rationale.
- name: Set up Buildx (docker-container driver, docker.io mirror)
id: buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
with:
driver: docker-container
buildkitd-config: hack/buildkitd.toml
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
- name: Build Talos image (matchbox + installer tarball)
run: make -C packages/core/talos image
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
BUILDER: ${{ steps.buildx.outputs.name }}
IMAGE_TAG: pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
PUBLISH_VERSIONED: '0'
PUBLISH_FLOATING: '0'
- name: Build Talos nocloud disk
run: make -C packages/core/talos talos-nocloud
- name: Capture Talos patch fragment
run: |
mkdir -p _out/fragments
git diff --binary HEAD > _out/fragments/talos.patch
echo "Talos fragment:"; git --no-pager diff --stat HEAD
- name: Upload Talos patch fragment
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: pr-patch-fragment-talos
path: _out/fragments/talos.patch
if-no-files-found: error
- name: Upload Talos image
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: talos-image
path: _out/assets/nocloud-amd64.raw.xz
if-no-files-found: error
# Reassemble the unified pr.patch from every fragment, then build the installer
# LAST: `make -C packages/core/installer image` runs `flux push artifact
# --path=packages`, which bundles the WHOLE digest-patched tree into the OCI
# artifact the operator pulls — so every other package's digest edit must
# already be applied here. e2e consumes the resulting pr.patch unchanged.
finalize:
name: Finalize (merge patches + installer)
# Depends on `checks` too: gating finalize on unit/controller tests cascades
# to e2e (which needs finalize), so a red unit test skips the expensive
# install+e2e path instead of wasting ~1h on it. Restores the gating the old
# in-build-job tests gave, without re-coupling the cheap parallel builds.
needs: ["plan", "checks", "build", "build-talos"]
if: |
!cancelled()
&& needs.plan.outputs.code == 'true'
&& !contains(github.event.pull_request.labels.*.name, 'release')
&& needs.checks.result == 'success'
&& (needs.build.result == 'success' || needs.build.result == 'skipped')
&& needs.build-talos.result == 'success'
runs-on: oracle-vm-4cpu-16gb-x86-64
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
fetch-tags: true
- name: Set up build toolchain
run: |
if ! command -v flux >/dev/null; then
install_script="$(mktemp)"
curl -fsSL https://fluxcd.io/install.sh -o "$install_script"
sudo bash "$install_script"
rm -f "$install_script"
fi
- name: Download patch fragments
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
pattern: pr-patch-fragment-*
merge-multiple: true
path: _out/fragments
- name: Apply digest fragments
run: |
applied=0
# Each fragment patches only its own package's files; --3way lets two
# fragments that touch adjacent lines of a shared file (e.g. a package
# Makefile that writes a sibling package's values.yaml) merge cleanly
# instead of the second apply aborting the whole finalize.
for f in _out/fragments/*.patch; do
[ -s "$f" ] || continue
echo "Applying $f"
git apply --3way "$f"
applied=$((applied + 1))
done
echo "Applied $applied fragment(s)"
git --no-pager diff --stat HEAD
- name: Set up Docker config
run: |
if [ -d ~/.docker ]; then
cp -r ~/.docker "${{ runner.temp }}/.docker"
fi
- name: Login to GitHub Container Registry
if: ${{ !github.event.pull_request.head.repo.fork }}
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: ${{ secrets.OCIR_USER }}
password: ${{ secrets.OCIR_TOKEN }}
registry: iad.ocir.io
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
# Overlay current-main image refs onto every package this PR did NOT
# rebuild. Without it, unbuilt packages fall back to the last-release refs
# committed in the tree (e.g. v1.5.0) even when main is hundreds of commits
# ahead — so e2e and the installer's packages artifact would ship stale
# images for everything outside this PR's build matrix. Source of truth is
# the cozystack-packages:main OCI artifact build-main.yaml already
# publishes (the whole current-main packages tree, refs digest-pinned to
# main's images). Runs after login (creds available) and before the
# installer build below so the flux-pushed packages tree — and the
# pr.patch e2e applies — already reflect the overlay. A missing artifact
# (build-main not yet green) is a no-op: packages keep their committed
# refs (prior behaviour). See hack/overlay-main-images.sh.
#
# Read the artifact for the PR's OWN base branch, not always main. Overlaying
# main's images onto a release-line PR mixes generations: main's binaries
# against that line's charts. #3437 (a one-line change on release-1.6) failed
# install deterministically that way, because main's cozystack-controller
# served an aggregated OpenAPI release-1.6's charts could not validate
# against (`unknown model in reference: …v1alpha1.OptionSpec`) — a broken
# lane reading as a broken PR. build-release.yaml publishes
# cozystack-packages:<line> for every maintained release-<major>.<minor>
# branch, so each base branch has its own artifact to read.
#
# A missing artifact stays a no-op (packages keep their committed refs), which
# also covers a line whose first build-release run has not landed yet.
- name: Pull base-branch packages tree
run: |
rm -rf _out/mainpkgs
mkdir -p _out/mainpkgs
if ! flux pull artifact "oci://${REGISTRY}/cozystack-packages:${BASE_REF}" \
--output _out/mainpkgs; then
rm -rf _out/mainpkgs
# Degrading to committed refs is safe, but SAY so: on a release line a
# missing artifact means build-release.yaml has not published this line
# yet (first build pending, or its branch filter does not match this
# branch), and the PR is then testing the line's last release instead of
# its tip. Silent would look identical to a working overlay.
if [ "${BASE_REF}" = "main" ]; then
echo "cozystack-packages:${BASE_REF} pull failed — keeping committed refs"
else
echo "::warning title=No packages artifact for ${BASE_REF}::Keeping committed refs, so unbuilt packages use this line's last release rather than its tip. Check that build-release.yaml has run for ${BASE_REF}."
fi
fi
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
# Env-passed, not interpolated into the script: base_ref is repo-controlled
# here (a branch name in this repo), but the workflow-injection hygiene
# applies uniformly.
BASE_REF: ${{ github.base_ref }}
- name: Overlay base-branch refs for unbuilt packages
# Best-effort: this only improves WHICH images e2e and the installer use,
# so an unexpected failure must degrade to the committed refs (prior
# behaviour), never block the PR. overlay-main-images.sh already handles
# the expected paths (missing artifact, drift, absent unit) internally —
# including the empty directory the step above leaves when a line has no
# artifact yet.
run: |
hack/overlay-main-images.sh _out/mainpkgs "$BUILD_MATRIX" "$PR_TOUCHED" \
|| echo "::warning::overlay-main-images failed; unbuilt packages keep committed refs"
env:
# The set of packages THIS PR rebuilt; their pr-<N>-<sha> refs are
# authoritative, so overlay-main-images.sh skips them. Computed by the
# plan job from the Makefile (a JSON array of package dirs), not from
# untrusted PR input — passed via env per workflow-injection hygiene.
BUILD_MATRIX: ${{ needs.plan.outputs.matrix }}
# Packages THIS PR edited (changed-file dirs from the plan job's git
# diff). Their committed refs are the PR's intent — an upstream image
# bump in a non-build-unit package would otherwise be silently reverted
# by the overlay — so overlay-main-images.sh preserves them. Also
# env-passed, derived from the diff, not from untrusted PR input.
PR_TOUCHED: ${{ needs.plan.outputs.touched }}
# buildkitd-config routes docker.io base-image pulls through mirror.gcr.io
# (hack/buildkitd.toml) so the installer build dodges Docker Hub's
# anonymous-pull rate limit; see the build job for the full rationale.
- name: Set up Buildx (docker-container driver, docker.io mirror)
id: buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
with:
driver: docker-container
buildkitd-config: hack/buildkitd.toml
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
# Builds the operator image AND flux-pushes the now-fully-patched packages
# tree, then patches the installer's platformSourceRef to that artifact.
- name: Build installer (operator + packages artifact)
run: make -C packages/core/installer image
env:
DOCKER_CONFIG: ${{ runner.temp }}/.docker
BUILDER: ${{ steps.buildx.outputs.name }}
IMAGE_TAG: pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
PUBLISH_VERSIONED: '0'
PUBLISH_FLOATING: '0'
- name: Assemble pr.patch
run: |
mkdir -p _out/assets
# --binary so regenerated provider bundles (components*.gz) stay in
# the patch e2e applies; mirrors main's pr.patch binary fix (#e566992).
git diff --binary HEAD > _out/assets/pr.patch
echo "Final pr.patch:"; git --no-pager diff --stat HEAD
- name: Upload git diff patch
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: pr-patch
path: _out/assets/pr.patch
if-no-files-found: error
resolve_assets:
name: "Resolve assets"
runs-on: ubuntu-latest
if: |
contains(github.event.pull_request.labels.*.name, 'release')
&& (github.event.action != 'labeled' || github.event.label.name == 'full-e2e')
outputs:
disk_id: ${{ steps.fetch_assets.outputs.disk_id }}
permissions:
contents: read
steps:
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1
with:
app-id: ${{ secrets.COZYSTACK_CI_APP_ID }}
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
owner: cozystack
repositories: ${{ github.event.repository.name }}
permission-contents: read
- name: Checkout code
if: contains(github.event.pull_request.labels.*.name, 'release')
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
fetch-tags: true
- name: Extract tag from PR branch (release PR)
if: contains(github.event.pull_request.labels.*.name, 'release')
id: get_tag
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const branch = context.payload.pull_request.head.ref;
const m = branch.match(/^release-(\d+\.\d+\.\d+(?:[-\w\.]+)?)$/);
if (!m) {
core.setFailed(`❌ Branch '${branch}' does not match 'release-X.Y.Z[-suffix]'`);
return;
}
core.setOutput('tag', `v${m[1]}`);
- name: Find draft release & asset IDs (release PR)
if: contains(github.event.pull_request.labels.*.name, 'release')
id: fetch_assets
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
github-token: ${{ steps.app-token.outputs.token }}
script: |
const tag = '${{ steps.get_tag.outputs.tag }}';
const releases = await github.rest.repos.listReleases({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 100
});
const draft = releases.data.find(r => r.tag_name === tag && r.draft);
if (!draft) {
core.setFailed(`Draft release '${tag}' not found`);
return;
}
const find = (n) => draft.assets.find(a => a.name === n)?.id;
const diskId = find('nocloud-amd64.raw.xz');
if (!diskId) {
core.setFailed('Required assets missing in draft release');
return;
}
core.setOutput('disk_id', diskId);
e2e:
name: "E2E Tests"
# The sandbox boots three QEMU guests (hack/e2e-prepare-cluster.bats), each
# with `-smp 8 -m 24576`, so the guests alone demand 24 vCPU and 72 GiB.
# On the 24-vCPU runner that is a 1:1 CPU commit with zero host headroom for
# QEMU's own emulation/IO threads, vhost-net, the host iptables MASQUERADE
# every guest image pull traverses, containerd, the kernel and the runner
# agent. During "Install Cozystack" (all platform operators + image pulls
# across three nodes at once) the guest vCPUs contend with those host threads
# and lose cycles to CPU steal, so in-guest kubelet/apiserver readiness and
# liveness probes miss their deadlines ("context deadline exceeded"). A
# HelmRelease then flips InProgress -> Failed and its dependents cascade to
# DependencyNotReady — on a different component each run, the signature of
# resource thrash rather than a product bug. The 32-vCPU / 128 GiB class
# keeps the guests' 24 vCPU / 72 GiB unchanged (no slower install) while
# leaving 8 vCPU (25%) and 56 GiB of host headroom, which removes the
# oversubscription that starves the probes.
runs-on: oracle-vm-32cpu-128gb-x86-64
timeout-minutes: 180
permissions:
contents: read
packages: read
checks: write
needs: ["finalize", "build-talos", "resolve_assets"]
# Regular PRs still enter through finalize. A release-labelled promote PR
# enters through resolve_assets only when a maintainer explicitly adds
# full-e2e; without that label the immutable rc's already-green suite is not
# repeated. The labeled trigger above makes that post-open opt-in start a run.
if: |
!cancelled()
&& (
needs.finalize.result == 'success'
|| (
needs.resolve_assets.result == 'success'
&& contains(github.event.pull_request.labels.*.name, 'release')
&& contains(github.event.pull_request.labels.*.name, 'full-e2e')
)
)
steps:
- name: Generate GitHub App token
if: contains(github.event.pull_request.labels.*.name, 'release')
id: app-token
uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1
with:
app-id: ${{ secrets.COZYSTACK_CI_APP_ID }}
private-key: ${{ secrets.COZYSTACK_CI_PRIVATE_KEY }}
owner: cozystack
repositories: ${{ github.event.repository.name }}
permission-contents: read
# ▸ Checkout and prepare the codebase
# fetch-depth: 0 is required by the "Select E2E tests" step so that
# `git diff origin/${BASE_REF}...HEAD` can resolve the base ref.
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
# ▸ Regular PR path – download the Talos disk from build-talos and pr.patch from finalize
- name: "Download Talos image (regular PR)"
if: "!contains(github.event.pull_request.labels.*.name, 'release')"
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: talos-image
path: _out/assets
- name: Download PR patch
if: "!contains(github.event.pull_request.labels.*.name, 'release')"
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: pr-patch
path: _out/assets
- name: Apply patch
if: "!contains(github.event.pull_request.labels.*.name, 'release')"
run: |
git apply _out/assets/pr.patch
# ▸ Release PR path – fetch artefacts from the corresponding draft release
- name: Download assets from draft release (release PR)
if: contains(github.event.pull_request.labels.*.name, 'release')
run: |
mkdir -p _out/assets
curl -sSL -H "Authorization: token ${APP_TOKEN}" -H "Accept: application/octet-stream" \
-o _out/assets/nocloud-amd64.raw.xz \
"https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/assets/${{ needs.resolve_assets.outputs.disk_id }}"
env:
APP_TOKEN: ${{ steps.app-token.outputs.token }}
- name: Set sandbox ID
run: echo "SANDBOX_NAME=cozy-e2e-sandbox-$(echo "${GITHUB_REPOSITORY}:${GITHUB_WORKFLOW}:${GITHUB_REF}" | sha256sum | cut -c1-10)" >> $GITHUB_ENV
# ▸ Prepare environment
- name: Prepare environment
run: |
attempt=0
until make SANDBOX_NAME=$SANDBOX_NAME prepare-env; do
attempt=$((attempt + 1))
if [ $attempt -ge 3 ]; then
echo "Attempt $attempt failed, exiting..."
exit 1
fi
echo "Attempt $attempt failed, retrying..."
done
echo "Prepare environment completed after $((attempt + 1)) attempts"
# ▸ Install Cozystack
- name: Install Cozystack into sandbox
run: |
if ! make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME install-cozystack; then
echo "❌ Install failed (no retry — see diagnostics below)"
echo "::group::Diagnostics: HelmRelease status"
docker exec "$SANDBOX_NAME" sh -c 'kubectl get hr -A -o wide 2>&1 | tail -100' || true
echo "::endgroup::"
echo "::group::Diagnostics: not-Ready HelmReleases (describe)"
docker exec "$SANDBOX_NAME" sh -c 'kubectl get hr -A --no-headers 2>/dev/null | awk "\$4 != \"True\" {print \$1\" \"\$2}"' \
| while read -r ns name; do
echo "--- $ns/$name ---"
docker exec "$SANDBOX_NAME" sh -c "kubectl describe hr '$name' -n '$ns' 2>&1 | tail -50" || true
done
echo "::endgroup::"
echo "::group::Diagnostics: recent events"
docker exec "$SANDBOX_NAME" sh -c 'kubectl get events -A --sort-by=.lastTimestamp 2>&1 | tail -50' || true
echo "::endgroup::"
exit 1
fi
# Test Impact Analysis (TIA): walk the PackageSource graph for the PR
# diff and select only the Chainsaw suites affected. Runs by default; opt
# out via the `full-e2e` label to run the full suite.
# `skip=true` means nothing to test (docs-only PR).
- name: Select E2E tests
id: select
if: ${{ !contains(github.event.pull_request.labels.*.name, 'full-e2e') }}
env:
BASE_REF: ${{ github.base_ref }}
run: |
git diff --name-only "origin/${BASE_REF}...HEAD" > /tmp/changed.txt
apps=$(./hack/select-e2e.sh /tmp/changed.txt)
echo "Selected apps: ${apps:-<none>}"
echo "apps=${apps}" >> $GITHUB_OUTPUT
if [ -z "${apps}" ]; then
echo "skip=true" >> $GITHUB_OUTPUT
else
echo "skip=false" >> $GITHUB_OUTPUT
fi
- name: Run OpenAPI tests
run: |
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME test-openapi
# ▸ Run E2E tests (Chainsaw, scoped by Test Impact Analysis)
#
# The "Select E2E tests" step above picks the suites impacted by the PR
# diff. With the `full-e2e` label that step is skipped, SELECTED_APPS is
# empty, and we fall back to the whole suite; an empty selection on a
# docs-only PR skips this step entirely (the skip != 'true' guard).
#
# Chainsaw discovers each named suite under hack/e2e-chainsaw/ and polls
# every assertion to its own timeout, so the old per-app 3-retry loop is
# gone — a failure is reported as a structured assertion diff, with
# cluster-state diagnostics dumped below on a non-zero exit.
- name: Run E2E tests
id: e2e_tests
if: ${{ steps.select.outputs.skip != 'true' }}
env:
SELECTED_APPS: ${{ steps.select.outputs.apps }}
FULL_E2E: ${{ contains(github.event.pull_request.labels.*.name, 'full-e2e') }}
run: |
if [ "$FULL_E2E" = "true" ] || [ -z "$SELECTED_APPS" ]; then
suites="" # empty => test-chainsaw runs the whole suite
echo "Running full chainsaw suite"
else
suites="$SELECTED_APPS"
echo "Running selected chainsaw suites: $suites"
fi
if ! make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME test-chainsaw CHAINSAW_SUITES="$suites"; then
echo "❌ Chainsaw E2E failed (see the assertion diffs above and diagnostics below)"
echo "::group::Diagnostics: HelmRelease status"
docker exec "$SANDBOX_NAME" sh -c 'kubectl get hr -A -o wide 2>&1 | tail -50' || true
echo "::endgroup::"
echo "::group::Diagnostics: recent events"
docker exec "$SANDBOX_NAME" sh -c 'kubectl get events -A --sort-by=.lastTimestamp 2>&1 | tail -30' || true
echo "::endgroup::"
# COSI state at the moment of failure — the CRDs ship no printer
# columns, so pull the readiness fields explicitly (cozyreport
# captures the same at end-of-job, but bucket states move between
# a test failing and the report being taken).
echo "::group::Diagnostics: COSI bucket state"
docker exec "$SANDBOX_NAME" sh -c '
kubectl get bucketclaims.objectstorage.k8s.io -A -o custom-columns="NAMESPACE:.metadata.namespace,NAME:.metadata.name,READY:.status.bucketReady,BUCKET:.status.bucketName" 2>&1
kubectl get buckets.objectstorage.k8s.io -o custom-columns="NAME:.metadata.name,READY:.status.bucketReady,ID:.status.bucketID,CLAIMNS:.spec.bucketClaim.namespace" 2>&1
kubectl get bucketaccesses.objectstorage.k8s.io -A -o custom-columns="NAMESPACE:.metadata.namespace,NAME:.metadata.name,GRANTED:.status.accessGranted,CLAIM:.spec.bucketClaimName" 2>&1
' || true
echo "::endgroup::"
exit 1
fi
echo "✅ All selected E2E tests passed"
- name: Collect chainsaw report
if: always()
run: |
mkdir -p _out
docker cp "$SANDBOX_NAME":/workspace/hack/e2e-chainsaw/chainsaw-report.xml \
_out/chainsaw-report.xml || true
- name: Upload chainsaw report
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: chainsaw-report
path: _out/chainsaw-report.xml
if-no-files-found: ignore
# ▸ Collect debug information (always runs)
#
# A ceiling of its own, not the job's 180 minutes. `Upload
# cozyreport.tgz` is the next step and the collector writes its tarball
# at the very end of its run, so a read that never returns spends what
# is left of the job budget, the job is cancelled, and the upload never
# runs -- the artifact is lost rather than truncated, on exactly the run
# where it is the only evidence left. `|| true` guards against a
# non-zero exit, not against a read that does not return.
#
# 30 minutes is measured against the step: it takes 3-42s across sampled
# runs, on failed e2e jobs as well as green ones. It RESERVES nothing --
# `timeout-minutes` caps a step, it does not carve budget out of the job,
# so a suite that has already burned 175 of the 180 leaves this step five
# minutes and the upload is cancelled with everything else. What the cap
# buys is that a hung read cannot be the reason.
# `continue-on-error` because the ceiling must not change what the job
# REPORTS. `|| true` on the make call is this step's standing statement
# that collecting diagnostics is never fatal, and a step killed at its
# ceiling never reaches that `|| true` -- the runner ends the process. So
# without this, a passing E2E run whose collection ran long is published
# as a failed one, and every later `failure()` in this job starts meaning
# "the tests failed OR the report was slow".
- name: Collect report
id: collect_report
if: always()
timeout-minutes: 30
continue-on-error: true
run: |
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME collect-report || true
# `continue-on-error` keeps the overrun out of the job's result, which also
# keeps it out of everyone's view: a step that ended at its ceiling renders
# green and the only trace is in the raw log. This branch's own standard is
# that a bound which fires is recorded, and that has to hold for the outer
# bound too.
#
# The wording says what `outcome` establishes and no more: a step can end
# non-zero without any ceiling firing, so naming the timeout as the cause
# would be the false-mechanism claim the collector refuses to make for a 137.
- name: Report collection overran its ceiling
if: always() && steps.collect_report.outcome == 'failure'
run: |
echo "::warning title=cozyreport::Collect report ended non-zero -- its timeout-minutes fired, or the step failed before the collector ran; cozyreport.tgz is missing or partial for this run"
- name: Upload cozyreport.tgz
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: cozyreport
path: _out/cozyreport.tgz
- name: Collect images list
if: always()
run: |
make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME collect-images || true
- name: Upload image list
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: image-list
path: _out/images.txt
# ▸ Open an SSH breakpoint to the failing sandbox so maintainers can attach,
# inspect Talos/Cozystack state and resume with `breakpoint resume`.
#
# Opens when the E2E job fails AND any of: the `debug` label, the
# `release` label (release PRs are bot-created, so their presence
# implies maintainer authorship), or the PR author is a maintainer
# (author_association is OWNER/MEMBER/COLLABORATOR). The maintainer-
# authorship path needs no label and does NOT move the jobs onto
# self-hosted runners — only the `debug` label flips `runs-on`, so a
# maintainer PR opens the breakpoint on the default oracle-vm runner the
# job already uses and simply holds that ephemeral runner. authorized-users
# is kept as a second defense layer controlling who may actually SSH in.
#
# Uses cozystack/breakpoint-action (fork of namespacelabs/breakpoint-action)
# pinned by SHA. The fork adds: pause-idle mode (initial grace period for
# the first SSH connection, idle-aware exit afterwards), endpoint output
# and ::notice:: annotation, and a dedicated Check Run "Breakpoint Open"
# that carries the SSH endpoint in output.summary while the breakpoint is
# paused (conclusion=failure → standard ✗ in `gh pr checks`; updated to
# conclusion=success when the breakpoint exits).
#
# Configured via the BREAKPOINT_ENDPOINT repository variable — in fork
# PRs repository variables are not exposed, so the step is skipped and
# forks cannot reach the self-hosted rendezvous server.
- name: Breakpoint on E2E failure
if: |
failure() &&
vars.BREAKPOINT_ENDPOINT != '' &&
(contains(github.event.pull_request.labels.*.name, 'debug')
|| contains(github.event.pull_request.labels.*.name, 'release')
|| contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association))
# cozystack/breakpoint-action v2-cozy.1
# mode: pause-idle defaults: grace-period=20m, idle-timeout=10m
uses: cozystack/breakpoint-action@a6f3a6f87be398ad63b6577351e3398e53f578e4
with:
mode: pause-idle
endpoint: ${{ vars.BREAKPOINT_ENDPOINT }}
authorized-users: androndo, Arsolitt, IvanHunters, kvaps, lexfrei, lllamnyp, mattia-eleuteri, matthieu-robin, myasnikovdaniil, sircthulhu, tym83
check-run-name: "Breakpoint Open"
github-token: ${{ github.token }}
check-run-summary-template: |
## 🔴 SSH breakpoint open — paused for debug
```
{endpoint}
```
Enter the e2e sandbox after SSH:
```
docker exec -ti $(docker ps --filter name=cozy-e2e-sandbox -q | head -1) bash
export KUBECONFIG=/workspace/kubeconfig
```
Resume from inside: `breakpoint resume`. Otherwise the breakpoint
exits 10 minutes after the last SSH session disconnects.
# ▸ Tear down environment (always runs)
- name: Tear down sandbox
if: always()
run: make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME delete || true
Reference in new issue View Git Blame Copy Permalink