fix(ci): add live-registry audit for unpublished leaf pins (#4303) (#4304)

* fix(ci): add live-registry audit for unpublished leaf pins (#4303)

audit-wrapper-dep-ranges.mjs and audit-leaf-package-publish.mjs only check
that a non-bundled @claude-flow/* leaf's CLI pin agrees with the workspace's
own version. Neither asks npm whether that version was ever published.

#4279 bumped @claude-flow/memory to 3.0.6 in the workspace and pinned the CLI
to match; both existing audits were green since workspace and pin agreed with
each other, but the standalone `npm publish` for memory was never run. 3.57.2
shipped depending on a version of memory that did not exist, breaking every
fresh install with ETARGET.

Add audit-leaf-versions-published.mjs, which queries the live npm registry
for each non-bundled leaf and fails if the version the CLI's pin resolves to
was never published. Wired into the plugin-package-audit CI job (same gate as
the two existing leaf audits) and documented as a required pre-tag step in
CLAUDE.md's publishing rules.

Co-Authored-By: RuFlo <ruv@ruv.net>

* fix: use encodeURIComponent instead of a first-occurrence .replace for the registry URL

CodeQL flagged name.replace('@', '%40') as incomplete string escaping —
String.replace with a string pattern only touches the first match. It
happened to be harmless for npm scope names (exactly one leading @), but
it's the wrong tool for escaping a URL path segment. encodeURIComponent
also correctly escapes the scope separator (%2F), verified against the
live registry (200 OK for @claude-flow/memory).

Co-Authored-By: RuFlo <ruv@ruv.net>

* fix(ci): baseline the new audit test file like its plain-script sibling

scripts/__tests__/audit-leaf-versions-published.test.mjs is a bare Node
script (console.log + node:assert/strict), the same shape as the
already-baselined scripts/__tests__/audit-supply-chain.test.mjs — it has
no vitest-discoverable describe/it block, so the root vitest ratchet
reports "No test suite found in file" and fails CI's Test Suite job.

This is not a real regression: the file was never runnable by vitest in
the first place (same category as its sibling, which IS correctly listed
here already). The test itself passes fine when run directly via
`node scripts/__tests__/audit-leaf-versions-published.test.mjs` (6/6, see
the PR description).

Co-Authored-By: RuFlo <ruv@ruv.net>

---------

Co-authored-by: r <r@x>

.github/workflows/v3-ci.yml

@@ -1839,6 +1839,19 @@ jobs:
         shell: bash
         run: node scripts/audit-leaf-package-publish.mjs
 
+      - name: Run leaf-versions-published audit (#4303)
+        # audit-wrapper-dep-ranges.mjs and audit-leaf-package-publish.mjs above
+        # only check that the CLI's pin and the leaf's own workspace version
+        # agree with EACH OTHER. #4303 slipped through both: #4279 bumped
+        # @claude-flow/memory to 3.0.6 in the workspace and pinned the CLI to
+        # match, so both audits were green — but nobody ran the standalone
+        # `npm publish` for memory, so @claude-flow/cli@3.57.2 shipped
+        # depending on a version that didn't exist, and every fresh install
+        # failed with ETARGET. This queries the live npm registry so a pin
+        # bump that outruns its own publish fails CI instead of shipping.
+        shell: bash
+        run: node scripts/audit-leaf-versions-published.mjs
+
       - name: Run better-sqlite3 override audit (#2219)
         # agentdb pins better-sqlite3 as an OPTIONAL dep at ^11.8.1, which has
         # no Node 24/25/26 prebuild → the optional native build fails silently

CLAUDE.md

@@ -1026,6 +1026,17 @@ memory_search_unified({ query: "authentication security", limit: 5 })
   It must use `semver.satisfies`, not a string compare: a pin may be a range,
   and a literal match would then report a false DRIFT. It also fails loudly
   at zero leaves checked — a guard that inspects nothing must not report clean.
+- **The drift check above only proves the CLI's pin and the workspace version agree with
+  EACH OTHER — it says nothing about whether that version is actually on npm.** #4303
+  (2026-10-20) shipped `@claude-flow/cli@3.57.2` depending on `@claude-flow/memory@3.0.6`,
+  which had never been published — #4279 bumped the workspace version and the CLI's pin
+  together, so the drift check above was clean, but nobody ran the standalone `npm
+  publish` for `memory`. Every fresh install of that line failed with `ETARGET`. Run
+  `node scripts/audit-leaf-versions-published.mjs` before tagging — it queries the live
+  npm registry for every non-bundled leaf pin and fails if the version it resolves to was
+  never published. This is now also a required CI check (`plugin-package-audit` job in
+  `v3-ci.yml`), so a PR that bumps a pin ahead of its own leaf's publish fails on the PR,
+  not on a consumer's install.
 - MUST update ALL dist-tags for ALL THREE packages after publishing (latest + alpha + v3alpha all point to the same version)
 - Publish order: `@claude-flow/cli` first, then `claude-flow` (umbrella), then `ruflo` (alias umbrella)
 - MUST run verification for ALL THREE before telling user publishing is complete

scripts/__tests__/audit-leaf-versions-published.test.mjs

@@ -0,0 +1,95 @@
+#!/usr/bin/env node
+/**
+ * Unit tests for scripts/audit-leaf-versions-published.mjs's pure decision
+ * logic (evaluateLeaf, collectLeaves). These run against fixture data, not
+ * the live npm registry, so they're deterministic and need no network.
+ *
+ * The regression this guards against is ruvnet/ruflo#4303: a leaf's pin and
+ * its own workspace version agreed with each other, but neither was ever
+ * actually published — so the guard must fail even when "internal
+ * consistency" checks (audit-wrapper-dep-ranges, audit-leaf-package-publish)
+ * would pass.
+ *
+ * Run via: node scripts/__tests__/audit-leaf-versions-published.test.mjs
+ */
+
+import assert from 'node:assert/strict';
+import { evaluateLeaf, collectLeaves } from '../audit-leaf-versions-published.mjs';
+
+console.log('test: exact pin that is NOT published is a violation (the #4303 shape)');
+{
+  const leaf = { name: '@claude-flow/memory', range: '3.0.6', workspaceVersion: '3.0.6' };
+  const result = evaluateLeaf(leaf, ['3.0.4', '3.0.5']); // 3.0.6 never published
+  assert.equal(result.ok, false);
+  assert.match(result.violation, /pins the exact version "3\.0\.6"/);
+  assert.match(result.violation, /NOT on the npm registry/);
+  console.log('  pass');
+}
+
+console.log('test: exact pin that IS published is ok');
+{
+  const leaf = { name: '@claude-flow/memory', range: '3.0.6', workspaceVersion: '3.0.6' };
+  const result = evaluateLeaf(leaf, ['3.0.4', '3.0.5', '3.0.6']);
+  assert.equal(result.ok, true);
+  assert.equal(result.resolvedVersion, '3.0.6');
+  assert.equal(result.violation, null);
+  console.log('  pass');
+}
+
+console.log('test: range with a satisfying published version is ok');
+{
+  const leaf = { name: '@claude-flow/cli-core', range: '^3.7.0', workspaceVersion: '3.7.5' };
+  const result = evaluateLeaf(leaf, ['3.7.0', '3.7.4', '3.7.5']);
+  assert.equal(result.ok, true);
+  assert.equal(result.resolvedVersion, '3.7.5');
+  console.log('  pass');
+}
+
+console.log('test: range with NO satisfying published version is a violation');
+{
+  const leaf = { name: '@claude-flow/cli-core', range: '^3.8.0', workspaceVersion: '3.8.0' };
+  const result = evaluateLeaf(leaf, ['3.7.0', '3.7.4', '3.7.5']); // nothing in the 3.8.x line published
+  assert.equal(result.ok, false);
+  assert.match(result.violation, /no published version .* satisfies/);
+  console.log('  pass');
+}
+
+console.log('test: package entirely missing from the registry is a violation');
+{
+  const leaf = { name: '@claude-flow/does-not-exist', range: '1.0.0', workspaceVersion: '1.0.0' };
+  const result = evaluateLeaf(leaf, null); // fetchPublishedVersions returns null on 404
+  assert.equal(result.ok, false);
+  assert.match(result.violation, /does not exist on the npm registry at all/);
+  console.log('  pass');
+}
+
+console.log('test: collectLeaves skips bundled deps and deps with no workspace package.json');
+{
+  const cliPkg = {
+    dependencies: {
+      '@claude-flow/memory': '3.0.6',
+      '@claude-flow/mcp': '3.1.1', // bundled — must be skipped
+      'not-a-claude-flow-dep': '1.0.0', // wrong prefix — must be skipped
+    },
+    optionalDependencies: {
+      '@claude-flow/neural': '^3.0.0',
+      '@claude-flow/no-workspace-copy': '1.0.0', // no local package.json — must be skipped
+    },
+  };
+  const bundled = new Set(['@claude-flow/mcp']);
+  const workspace = {
+    '@claude-flow/memory': { version: '3.0.6' },
+    '@claude-flow/neural': { version: '3.0.1' },
+  };
+  const leaves = collectLeaves(cliPkg, bundled, (name) => workspace[name] ?? null);
+
+  assert.equal(leaves.length, 2);
+  const names = leaves.map((l) => l.name).sort();
+  assert.deepEqual(names, ['@claude-flow/memory', '@claude-flow/neural']);
+  const memory = leaves.find((l) => l.name === '@claude-flow/memory');
+  assert.equal(memory.range, '3.0.6');
+  assert.equal(memory.workspaceVersion, '3.0.6');
+  console.log('  pass');
+}
+
+console.log('\nall audit-leaf-versions-published tests passed');

scripts/audit-leaf-versions-published.mjs

@@ -0,0 +1,213 @@
+#!/usr/bin/env node
+/**
+ * Live-registry guard for ruvnet/ruflo#4303.
+ *
+ * audit-wrapper-dep-ranges.mjs and audit-leaf-package-publish.mjs both check
+ * INTERNAL consistency: does the CLI's pinned range for a non-bundled
+ * @claude-flow/* leaf cover that leaf's own workspace package.json version?
+ * Neither ever asks npm whether the version they agree on was actually
+ * published.
+ *
+ * #4279 bumped @claude-flow/memory to 3.0.6 in the workspace and pinned the
+ * CLI to 3.0.6 to match — both audits above passed, because workspace and
+ * pin agreed with each other. Nobody ran `npm publish` for the memory leaf.
+ * @claude-flow/cli@3.57.2 shipped depending on a version of memory that did
+ * not exist on the registry, so every fresh `npm install` of that line
+ * failed with ETARGET (#4303).
+ *
+ * This asserts, against the live npm registry, that the exact version (or,
+ * for a range, at least one satisfying version) each non-bundled leaf
+ * resolves to is actually published.
+ *
+ * Usage: node scripts/audit-leaf-versions-published.mjs
+ * Exit codes: 0 ok / 1 violations, a registry query failure, or the guard
+ * inspected nothing. A query failure is NOT treated as a pass — "couldn't
+ * prove it's safe" must block the same as "proved it's broken".
+ */
+
+import { readFileSync, existsSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+import { dirname, join, resolve } from 'node:path';
+import https from 'node:https';
+import semver from 'semver';
+
+import { INTERNAL_RUNTIME_PACKAGES } from './stage-internal-runtime-bundles.mjs';
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+const REPO_ROOT = resolve(__dirname, '..');
+const CLI_PKG = 'v3/@claude-flow/cli/package.json';
+
+function readJson(rel) {
+  const p = join(REPO_ROOT, rel);
+  return existsSync(p) ? JSON.parse(readFileSync(p, 'utf8')) : null;
+}
+
+export function fetchJson(url, { timeoutMs = 15000 } = {}) {
+  return new Promise((resolvePromise, reject) => {
+    const req = https.get(url, { headers: { accept: 'application/json' } }, (res) => {
+      let data = '';
+      res.on('data', (c) => { data += c; });
+      res.on('end', () => {
+        if (res.statusCode === 404) return resolvePromise({ notFound: true });
+        if (res.statusCode && res.statusCode >= 400) {
+          return reject(new Error(`HTTP ${res.statusCode} for ${url}`));
+        }
+        try {
+          resolvePromise(JSON.parse(data));
+        } catch (err) {
+          reject(new Error(`unparseable response from ${url}: ${err.message}`));
+        }
+      });
+    });
+    req.on('error', reject);
+    req.setTimeout(timeoutMs, () => req.destroy(new Error(`timeout fetching ${url}`)));
+  });
+}
+
+export async function fetchPublishedVersions(name, fetchImpl = fetchJson) {
+  // encodeURIComponent, not a string .replace('@', ...): replace() only
+  // touches the FIRST match, which happens to be harmless for a leading
+  // scope char but is the wrong tool for escaping a URL path segment.
+  const url = `https://registry.npmjs.org/${encodeURIComponent(name)}`;
+  const body = await fetchImpl(url);
+  if (body.notFound) return null;
+  return Object.keys(body.versions ?? {});
+}
+
+/**
+ * Pure decision logic, kept separate from the network call so it can be unit
+ * tested with fixture data instead of hitting the real registry.
+ *
+ * @param {{name: string, range: string}} leaf
+ * @param {string[]|null} publishedVersions - null means the package itself
+ *   was not found on the registry at all.
+ * @returns {{ok: boolean, resolvedVersion: string|null, violation: string|null, note: string|null}}
+ */
+export function evaluateLeaf(leaf, publishedVersions) {
+  if (publishedVersions === null) {
+    return {
+      ok: false,
+      resolvedVersion: null,
+      violation: `${leaf.name}: package does not exist on the npm registry at all.`,
+      note: null,
+    };
+  }
+
+  const exact = /^\d+\.\d+\.\d+/.test(leaf.range) ? leaf.range : null;
+
+  if (exact) {
+    const ok = publishedVersions.includes(exact);
+    const latest = publishedVersions.slice().sort(semver.compare).at(-1) ?? 'none';
+    return {
+      ok,
+      resolvedVersion: ok ? exact : null,
+      violation: ok
+        ? null
+        : `${leaf.name}: the CLI pins the exact version "${exact}", but that version is NOT on the ` +
+          `npm registry (latest published: ${latest}). A fresh \`npm install\` of @claude-flow/cli ` +
+          `will fail with ETARGET (ruvnet/ruflo#4303). Publish ${leaf.name}@${exact} before releasing, ` +
+          `or pin back to a version that is already published.`,
+      note: null,
+    };
+  }
+
+  const satisfying = publishedVersions.filter((v) =>
+    semver.satisfies(v, leaf.range, { includePrerelease: true }),
+  );
+  const ok = satisfying.length > 0;
+  const resolvedVersion = ok ? satisfying.slice().sort(semver.compare).at(-1) : null;
+  return {
+    ok,
+    resolvedVersion,
+    violation: ok
+      ? null
+      : `${leaf.name}: no published version on the npm registry satisfies the CLI's range ` +
+        `"${leaf.range}". A fresh \`npm install\` will fail to resolve this dependency.`,
+    note: null,
+  };
+}
+
+export function collectLeaves(cliPkg, bundled, readWorkspacePkg) {
+  const declared = { ...(cliPkg.dependencies ?? {}), ...(cliPkg.optionalDependencies ?? {}) };
+  const leaves = [];
+  for (const [name, range] of Object.entries(declared)) {
+    if (!name.startsWith('@claude-flow/')) continue;
+    if (bundled.has(name)) continue;
+    const pkg = readWorkspacePkg(name);
+    if (!pkg) continue; // not a workspace package — nothing in this repo to ship
+    leaves.push({ name, range, workspaceVersion: pkg.version });
+  }
+  return leaves;
+}
+
+async function main() {
+  const bundled = new Set(INTERNAL_RUNTIME_PACKAGES.map((p) => p.name));
+  if (bundled.size === 0) {
+    console.error('✗ INTERNAL_RUNTIME_PACKAGES is empty — cannot classify deps. Guard aborted.');
+    process.exit(1);
+  }
+
+  const cliPkg = readJson(CLI_PKG);
+  if (!cliPkg) {
+    console.error(`✗ ${CLI_PKG} not found — guard aborted.`);
+    process.exit(1);
+  }
+
+  const leaves = collectLeaves(cliPkg, bundled, (name) => readJson(`v3/${name}/package.json`));
+
+  console.log('leaf-versions-published audit — live npm registry check');
+  console.log(`  bundled (ship with the train, skipped): ${[...bundled].join(', ')}`);
+
+  if (leaves.length === 0) {
+    console.error('✗ inspected 0 non-bundled leaves — the classifier is broken, not the repo.');
+    console.error('  Refusing to report a clean result. Check CLI deps and INTERNAL_RUNTIME_PACKAGES.');
+    process.exit(1);
+  }
+
+  const violations = [];
+  let checked = 0;
+
+  for (const leaf of leaves) {
+    let versions;
+    try {
+      versions = await fetchPublishedVersions(leaf.name);
+    } catch (err) {
+      violations.push(
+        `${leaf.name}: could not query the npm registry (${err.message}). Treating this as ` +
+        `unverified, not as a pass — a release must not go out on an unproven leaf.`,
+      );
+      continue;
+    }
+    checked++;
+    const result = evaluateLeaf(leaf, versions);
+    console.log(
+      `  ${leaf.name.padEnd(36)} cli pins ${leaf.range.padEnd(22)} resolves to: ` +
+      `${result.resolvedVersion ?? 'NONE'} ${result.ok ? '✓' : '✗'}`,
+    );
+    if (!result.ok) violations.push(result.violation);
+
+    if (versions && !versions.includes(leaf.workspaceVersion)) {
+      console.log(
+        `  NOTE: ${leaf.name} workspace version ${leaf.workspaceVersion} is not yet published ` +
+        `(this is fine mid-development; it only matters at release time).`,
+      );
+    }
+  }
+
+  console.log(`  checked ${checked}/${leaves.length} non-bundled leaf package(s) against the live registry`);
+
+  if (violations.length === 0) {
+    console.log('  ok: every non-bundled leaf dependency the CLI declares is actually published');
+    process.exit(0);
+  }
+
+  console.error('\nviolations:');
+  for (const v of violations) console.error(`  ✗ ${v}`);
+  console.error(`\n${violations.length} violation(s) — releasing/installing now would reproduce ruvnet/ruflo#4303.`);
+  process.exit(1);
+}
+
+// Only run as a script, not when imported by the test file.
+if (import.meta.url === `file://${process.argv[1]}`) {
+  main();
+}

scripts/ci-test-baseline.txt

@@ -46,6 +46,7 @@ ruflo/src/ruvocal/src/lib/utils/tree/isMessageId.spec.ts
 ruflo/src/ruvocal/src/lib/utils/tree/treeHelpers.spec.ts
 ruflo/src/ruvocal/src/lib/wasm/tests/wasm-capabilities.test.ts
 ruflo/src/ruvocal/src/routes/login/callback/updateUser.spec.ts
+scripts/__tests__/audit-leaf-versions-published.test.mjs
 scripts/__tests__/audit-supply-chain.test.mjs
 scripts/__tests__/stage-internal-runtime-bundles.test.mjs
 tests/context-persistence-hook.test.mjs