Files
ruflo/plugins/ruflo-workflows
rUv e0d0af1cef feat(gaia): pre-submission exploit audit + signed attestation (ADR-167) (#2543)
* feat(gaia): pre-submission exploit audit + signed attestation (ADR-167)

Motivated by UC Berkeley RDI's April-2026 finding that all 8 major agent
benchmarks were gamed to ~98-100% without solving a single task (GAIA leaked
~98% of answers via public answer DBs + normalization collisions; o3/Claude-3.7
monkey-patch graders in 30%+ of runs per METR).

Signing proves transport-integrity; this audit proves earning-integrity. The
audit report is deterministic ($0, offline, pure-core) and is registered as an
ADR-103 witness fix marker so its sha256 + "clean": true marker are signed INTO
manifest.md.json.

- ADR-167 (v3/docs/adr): threat model, check catalogue mapped to RDI/METR
  vectors, manifest-attestation design, honest harness-gap limitations.
- gaia-audit.mjs: 7 pure checks (answer-leakage, no-work, oracle-leakage,
  grader-isolation, normalization-collision, voting-disclosure, split-integrity);
  CLI exit 0 clean / 1 CRITICAL / 2 usage. Schema-blocked checks return skip
  with a harness_gap note, never a false pass.
- Tests: 27 passing; clean + dirty fixtures triggering every vector.
- Wired --audit into /gaia validate; pre-sign gate into /gaia submit
  (refuses CRITICAL fail unless --allow-dirty) + gaia-submission skill.
- Bump ruflo-workflows 0.4.0 → 0.5.0.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01V84YJJK7RBF9TuUo4eVSW7

* docs(adr): ADR-168 — GAIA harness evidence recording (implements ADR-167 §7 forward contract)

Co-Authored-By: RuFlo <ruv@ruv.net>

* docs(adr): ADR-169 — benchmark reporting integrity standard (strict-EM headlines, view-labeled scaling, disclosed gaps)

Codifies the discipline the FRAMES ablation self-audit measured against the
Berkeley RDI vectors. Completes the integrity family: ADR-167 audits the
submission, ADR-168 records the evidence, ADR-169 governs how numbers are
reported.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(gaia): port static source-scan checks (AUD-8/9/10) into gaia-audit

Ports the unique, non-overlapping checks from the reverted #2547 duplicate
gaia-integrity.mjs into gaia-audit.mjs as a new static source-scan family,
matching ADR-167's check-registry / status / harness_gap design:

- AUD-8 answer-key-reads (CRITICAL, fail-closed): scans runner sources +
  produced artifacts for answer/gold/solution/ground-truth-shaped paths
  referenced outside the sanctioned dataset dir(s).
- AUD-9 dynamic-eval (CRITICAL, fail-closed): static scan of the gaia-bench
  runner sources for eval()/new Function()/exec-family calls with a non-literal
  first arg. Preserves the #2547 false-positive refinements (multi-line aware,
  RegExp.prototype.exec excluded, fixed-string commands + comments ignored) —
  passes clean against ruflo's own multi-line `gcloud secrets … execSync`.
- AUD-10 judge-injection (WARN): scans the agent's PRODUCED answer/outputs for
  judge-directed prompt-injection markers. Distinct from AUD-3 oracle-leakage
  (input-side gold-answer scan), so both run.

Unlike AUD-1/3/4 these need no trajectory instrumentation — they enforce today
and skip honestly (never false-pass) when the source population is absent
(standalone/ejected run or --skip-source-scan). The #2547 provenance stamp and
--allow-integrity-override are NOT ported: #2543's signed attestation and the
`/gaia submit --allow-dirty` gate already own provenance + override, so the new
checks flow through the same attestation.clean/strict_clean path.

CLI adds --repo-root, --dataset-dir (repeatable), --skip-source-scan.
Fixtures: planted evil runner (eval + interpolated exec), planted answer-key
read, judge-injection results. Tests: 44 green (17 new). Docs: ADR-167 §3/§7
check table, gaia-validate.md, gaia-submission SKILL.md.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3

* fix(ci): advance ruflo-workflows smoke version assertion to 0.5.0

The AUD-8/9/10 port bumped plugin.json to 0.5.0 but the plugin's own
smoke step 1 still asserted 0.4.0 — the single failing step behind both
smoke-all and the meta-smoke leg of smoke-without-metaharness on #2543.

Co-Authored-By: RuFlo <ruv@ruv.net>
2026-07-03 16:46:31 -04:00
..

ruflo-workflows

Workflow automation across two complementary surfaces:

  1. MCP workflow_* tools — declarative, persisted workflow definitions with a full state-machine lifecycle (create → run ↔ pause → complete/cancel). Best for long-lived, resumable, human-gated pipelines.
  2. Native Claude Code Workflow JS — imperative orchestration scripts (.claude/workflows/*.js) that fan subagents out deterministically via agent / parallel / pipeline / phase. Best for comprehensive fan-out: review, audit, migration, research.

Neither subsumes the other — see Choosing a surface.

Install

/plugin marketplace add ruvnet/ruflo
/plugin install ruflo-workflows@ruflo

Features

  • MCP workflow definitions: multi-step processes with conditions, parallel steps, and templates
  • Lifecycle management: execute, pause, resume, cancel running workflows
  • Approval gates: manual pause points for human review
  • Native orchestration: author .claude/workflows/*.js fan-out/pipeline scripts and run them with the Workflow tool

Commands

  • /workflow -- List MCP workflows + templates and native .claude/workflows/*.js scripts

Skills

  • workflow-create -- Author MCP workflow templates or native .claude/workflows/*.js orchestration scripts
  • workflow-run -- Execute/manage MCP workflows or invoke + resume native workflows

Compatibility

  • CLI: pinned to @claude-flow/cli v3.6 major+minor.
  • Verification: bash plugins/ruflo-workflows/scripts/smoke.sh is the contract.

MCP surface (10 tools)

All defined at v3/@claude-flow/cli/src/mcp-tools/workflow-tools.ts:

Tool Purpose
workflow_create Create a new workflow definition
workflow_run Run a workflow with inputs
workflow_execute Execute a one-shot workflow without persistence
workflow_status Inspect a running workflow
workflow_list List workflows
workflow_pause Pause a running workflow
workflow_resume Resume a paused workflow
workflow_cancel Cancel a workflow
workflow_delete Delete a workflow definition
workflow_template Manage workflow templates

Lifecycle state machine

created ──run──→ running ──pause──→ paused ──resume──→ running
                    │                  │
                    │                  └──cancel──→ cancelled
                    │
                    ├──complete──→ completed
                    └──cancel────→ cancelled
State Allowed transitions
created running (via workflow_run), cancelled (via workflow_cancel)
running paused (via workflow_pause), completed (auto), cancelled (via workflow_cancel)
paused running (via workflow_resume), cancelled (via workflow_cancel)
completed terminal
cancelled terminal

workflow_execute is the stateless path — fire-and-forget, no persisted state machine.

Native Workflow Orchestration (Claude Code Workflow tool)

The native surface runs a JavaScript orchestration script that fans subagents out deterministically. Scripts live in .claude/workflows/*.js and each begins with a pure-literal export const meta block — the meta.name makes it an invocable named workflow.

export const meta = {
  name: 'plugin-contract-audit',
  description: 'Run every plugin smoke contract, diagnose failures, report',
  phases: [{ title: 'Sweep' }, { title: 'Diagnose' }, { title: 'Report' }],
}
// body runs inside an async wrapper — top-level await + return are legal
const sweep = await agent('run all plugins/*/scripts/smoke.sh ...', { schema: SWEEP_SCHEMA })
const failures = (sweep?.results || []).filter((r) => r.failed > 0)
const diagnoses = await parallel(failures.map((f) => () =>
  agent(`diagnose ${f.plugin}`, { schema: DIAGNOSIS_SCHEMA })))
return { audited: sweep.results.length, failures, diagnoses }

The four-hook API

Hook Purpose
agent(prompt, opts) Spawn one subagent; with opts.schema returns validated structured output
parallel(thunks) Run thunks concurrently with a barrier (await all)
pipeline(items, ...stages) Run each item through all stages independently — no barrier
phase(title) / log(msg) Progress grouping and narration

meta MUST be a pure literal (no variables, calls, or interpolation). Use parallel only when you genuinely need every result together; otherwise prefer pipeline.

Invocation

Workflow({ name: 'plugin-contract-audit' })            // run a named .claude/workflows/*.js
Workflow({ scriptPath: '.claude/workflows/foo.js' })   // run a script by path
Workflow({ name: 'plugin-contract-audit', args: 'ruflo-agentdb' })  // pass args (the script's `args` global)
Workflow({ scriptPath, resumeFromRunId: 'wf_…' })      // resume — unchanged agent() calls return cached

The repo ships a reference workflow at .claude/workflows/plugin-contract-audit.js and a worked example at .claude/workflows/intelligence-system-hardening.js.

Choosing a surface

If you need… Use
A persisted definition that pauses for human approval and resumes across sessions MCP workflow_*
A stateful lifecycle (created → running ↔ paused → completed/cancelled) the engine schedules MCP workflow_*
Deterministic fan-out of many subagents (review N dimensions, audit N plugins, migrate N files) Native Workflow JS
Structured, schema-validated results aggregated in code Native Workflow JS
A one-shot stateless run MCP workflow_execute or native Workflow

Namespace coordination

This plugin owns the workflows-state AgentDB namespace (kebab-case, follows the convention from ruflo-agentdb ADR-0001 §"Namespace convention"). Reserved namespaces (pattern, claude-memories, default) MUST NOT be shadowed.

workflows-state indexes workflow definitions, current state, run history, and template metadata. Accessed via memory_* (namespace-routed).

Verification

bash plugins/ruflo-workflows/scripts/smoke.sh
# Expected: "15 passed, 0 failed"

Architecture Decisions

  • ruflo-agentdb — namespace convention owner
  • ruflo-loop-workers — sibling automation surface (loops are recurring; workflows are stateful pipelines)
  • ruflo-sparc — SPARC phase transitions can be modeled as workflows