feat(funnel): honor opt-outs in the messages fetch, wire opt-in telemetry, fix clone tracker

Operational telemetry (ADR-309, legitimate interest):
- `hooks refresh-funnel` (SessionStart) checked only the RUFLO_FUNNEL env
  kill switch. `ruflo funnel disable`, project config, enterprise policy and
  a declined disclosure all still called funnel.ruv.io every session. It now
  resolves the full funnel precedence and makes no request when disabled.
- The fetch's User-Agent adds the CLI release and OS/CPU family only:
  `ruflo-funnel/messages ruflo/<v> (<platform>; <arch>)`, sanitized to token
  characters. The live endpoint accepts it.
- docs/privacy/funnel-legitimate-interest-assessment.md is the assessment
  ADR-309 requires. ADR-309 links it, and ADR-310's acceptance rule now names
  the one documented default request.

Opt-in telemetry (ADR-302/305):
- `ruflo funnel telemetry on|off|status` is the only way to grant the
  telemetry consent domain. Nothing granted it before, so the event queue
  never left the machine.
- With consent, refresh-funnel flushes queued events once per session.
- `funnel disable` and `telemetry off` revoke consent and delete the ID and
  the local queue.

Clone tracker:
- A failed traffic fetch was recorded as 0 clones (snapshots #15-#18 read 0
  while GitHub reported ~258k in 14 days). The script now exits 1 and writes
  nothing.
- The workflow reads TRAFFIC_TOKEN, falling back to GITHUB_TOKEN, because
  the traffic API needs push access.

Co-Authored-By: RuFlo <ruv@ruv.net>
This commit is contained in:
randRuFlo committed 2026-11-06 11:05:25 -05:00
1 parent 776385ab7c
commit 53f95e36e6
10 files changed
+304 -10

No files matched your search

+3 -1
View File
@@ -54,7 +54,9 @@ jobs:
- name: Run clone tracker
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# The traffic API needs push access, which the Actions GITHUB_TOKEN
# lacks. TRAFFIC_TOKEN is a fine-grained PAT with Administration: read.
GH_TOKEN: ${{ secrets.TRAFFIC_TOKEN || secrets.GITHUB_TOKEN }}
run: node scripts/track-clones.mjs
- name: Show resulting artifacts
@@ -0,0 +1,59 @@
# Legitimate Interest Assessment: funnel messages fetch
- **Status:** Accepted, 2026-11-06
- **Scope:** the one default-on network request ruflo makes: `GET https://funnel.ruv.io/v1/messages`
- **Basis:** ADR-309 "operational telemetry: legitimate-interest basis, with a documented, minimized Legitimate Interest Assessment checked into the repo"
- **Out of scope:** opt-in event telemetry (`POST /v1/events`, the pseudonymous funnel ID). That runs on explicit consent (ADR-302 `telemetry` domain, `ruflo funnel telemetry on`), not on legitimate interest.
## 1. What is processed
| Data | Source | Why |
|---|---|---|
| Source IP address | the HTTPS connection itself | unavoidable for any request; used server-side only to count distinct clients per day |
| User-Agent `ruflo-funnel/messages ruflo/<version> (<platform>; <arch>)` | the CLI | release adoption and OS support decisions |
| Request time | the server | daily aggregation |
Nothing else is sent. The request carries no ID, path, hostname, username, repository, prompt, command line, or account data. The CLI sanitizes the User-Agent fields to token characters (`funnelUserAgent()` in `v3/@claude-flow/cli/src/funnel/message-transport.ts`).
**When it happens:** at most once per Claude Code session, from the SessionStart hook (`hooks refresh-funnel`), and at most once per 6 hours per machine because of the local cache.
## 2. Purpose test
ruflo is free, open-source software with no accounts. This request is the only way the maintainers can tell:
- how many installations are active day to day and month to month;
- which releases are still in use, so fixes and security advisories can be targeted;
- which operating systems and CPU families to support and test.
It also delivers the statusline tips the user sees, so the request serves the user as well as the maintainers.
## 3. Necessity test
- **npm download counts are not a substitute.** CI pipelines and repeated `npx` downloads inflate them, and they cannot show active use or release adoption.
- **Asking everyone for consent first would remove the signal.** Without the default request there is no measure of active use at all.
- **The data is the minimum that answers the purpose.** There is no persistent identifier. Distinct clients are counted by IP, with IPv6 grouped per /64, which already undercounts behind NAT. That is an accepted cost of not identifying anyone.
## 4. Balancing test
- **Reasonable expectation:** a developer tool fetching tips and checking release information is common practice. The first statusline message discloses the funnel (ADR-301) and how to turn it off.
- **Impact on the individual:** low. An IP address is personal data under GDPR, so it is treated as such:
- raw request logs keep the IP for **30 days** (Cloud Logging default, and the BigQuery raw table's partition expiry);
- the long-lived dataset `ruflo_funnel.daily_active` holds **counts only, never IPs**.
- **Control:** every opt-out stops the request. This is enforced in `hooks refresh-funnel` and covered by `__tests__/funnel-telemetry-optout.test.ts`. The opt-outs are:
- `ruflo funnel disable` (user-wide);
- `funnel.enabled: false` in `claude-flow.config.json` (per project);
- `RUFLO_FUNNEL=0` or `RUFLO_FUNNEL_MESSAGES=0` (environment);
- an enterprise policy file;
- declining the disclosure.
- **No profiling:** request data is never joined with Cognitum accounts or with opt-in telemetry events (ADR-309 "no cross-product behavioral profiling").
**Conclusion:** the interest is legitimate, the processing is necessary and minimal, and the user's interests do not override it. This is conditional on the controls above holding, and they are covered by tests.
## 5. Review
Re-assess this document if any of the following happens:
- a field is added to the request;
- raw-log retention is extended beyond 30 days;
- request data is joined with any other dataset;
- the request starts carrying an identifier.
+12 -2
View File
@@ -66,12 +66,14 @@ function fetchClones(repo) {
// array form prevents accidental shell metacharacter expansion (CWE-78).
const out = execFileSync('gh', ['api', `repos/${repo}/traffic/clones`], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
stdio: ['ignore', 'pipe', 'pipe'],
});
const j = JSON.parse(out);
return { count: j.count ?? 0, uniques: j.uniques ?? 0, days: j.clones ?? [] };
} catch (err) {
console.error(` [warn] failed to fetch clones for ${repo}: ${err.message?.slice(0, 80)}`);
// The traffic API needs push access; Actions' GITHUB_TOKEN gets 403 here.
const why = String(err.stderr || err.message || err).trim().split('\n').slice(-2).join(' ').slice(0, 200);
console.error(` [error] failed to fetch clones for ${repo}: ${why}`);
return { count: 0, uniques: 0, days: [], error: 'fetch-failed' };
}
}
@@ -151,6 +153,14 @@ async function main() {
vector.push(c.count, c.uniques);
console.error(` ${repo.padEnd(24)} count=${String(c.count).padStart(8)} uniques=${String(c.uniques).padStart(6)}`);
}
// A failed fetch used to be recorded as 0 clones, which poisoned the ledger
// (snapshots #15-#18 all read 0 while GitHub reported ~258k). Refuse to
// write a snapshot instead.
const failed = REPOS.filter((r) => clones[r].error);
if (failed.length) {
console.error(`clone tracker: ${failed.length}/${REPOS.length} repo(s) failed (${failed.join(', ')}). Set the TRAFFIC_TOKEN secret (fine-grained PAT, Administration: read on these repos). No snapshot written.`);
process.exit(1);
}
console.error('--- fetching npm download totals (last year, headline pkgs) ---');
const npm = {};
@@ -0,0 +1,158 @@
/**
* ADR-309 operational telemetry + ADR-302/305 opt-in telemetry.
*
* - `ruflo funnel disable` (and every other opt-out source) stops the
* SessionStart `hooks refresh-funnel` network call, not just the promo row.
* - The messages fetch identifies only the release and OS/CPU family.
* - `ruflo funnel telemetry on|off` is the only way to grant the telemetry
* consent domain; off (and `funnel disable`) revoke it and delete the ID.
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
const requestSpy = vi.hoisted(() => vi.fn());
vi.mock('https', async (importOriginal) => {
const actual = await importOriginal<typeof import('https')>();
return { ...actual, request: requestSpy, default: { ...actual, request: requestSpy } };
});
import { hooksCommand } from '../src/commands/hooks.js';
import { funnelCommand } from '../src/commands/funnel.js';
import { funnelUserAgent, getFunnelId, hasConsent } from '../src/funnel/index.js';
import type { CommandContext } from '../src/types.js';
const sub = (cmd: typeof hooksCommand, name: string) => {
const found = cmd.subcommands?.find((c) => c.name === name);
if (!found?.action) throw new Error(`missing subcommand ${name}`);
return found.action;
};
const ctx = (args: string[] = [], flags: Record<string, unknown> = {}): CommandContext =>
({ args, flags: { quiet: true, ...flags }, cwd: process.cwd(), interactive: false }) as CommandContext;
let stateDir: string;
let savedEnv: NodeJS.ProcessEnv;
beforeEach(() => {
stateDir = fs.mkdtempSync(path.join(os.tmpdir(), 'funnel-telemetry-'));
savedEnv = { ...process.env };
process.env.RUFLO_STATE_DIR = stateDir;
for (const k of ['RUFLO_FUNNEL', 'RUFLO_FUNNEL_MESSAGES', 'RUFLO_ENTERPRISE_POLICY', 'CI', 'GITHUB_ACTIONS']) delete process.env[k];
requestSpy.mockReset();
// Any request that does get made fails fast (no real network in tests).
requestSpy.mockImplementation((_opts: unknown, _cb: unknown) => {
const handlers: Record<string, (e?: unknown) => void> = {};
const req = {
on: (ev: string, fn: (e?: unknown) => void) => { handlers[ev] = fn; return req; },
setTimeout: () => req,
write: () => true,
end: () => { queueMicrotask(() => handlers.error?.(new Error('offline'))); },
destroy: () => undefined,
};
return req;
});
});
afterEach(() => {
process.env = savedEnv;
fs.rmSync(stateDir, { recursive: true, force: true });
});
describe('refresh-funnel honors every opt-out (ADR-309)', () => {
it('makes the messages request when the funnel is on', async () => {
await sub(hooksCommand, 'refresh-funnel')(ctx());
expect(requestSpy).toHaveBeenCalledTimes(1);
const opts = requestSpy.mock.calls[0][0] as { hostname: string; headers: Record<string, string> };
expect(opts.hostname).toBe('funnel.ruv.io');
expect(opts.headers['User-Agent']).toMatch(/^ruflo-funnel\/messages ruflo\/\S+ \(\w+; \w+\)$/);
});
it('makes no request after `ruflo funnel disable`', async () => {
await sub(funnelCommand, 'disable')(ctx());
const res = await sub(hooksCommand, 'refresh-funnel')(ctx());
expect(requestSpy).not.toHaveBeenCalled();
expect(res.data).toMatchObject({ refreshed: false, skipped: 'funnel-disabled', decidedBy: 'user-config' });
});
it('makes no request when RUFLO_FUNNEL=0', async () => {
process.env.RUFLO_FUNNEL = '0';
await sub(hooksCommand, 'refresh-funnel')(ctx());
expect(requestSpy).not.toHaveBeenCalled();
});
it('makes no request when the project config disables the funnel', async () => {
const proj = fs.mkdtempSync(path.join(os.tmpdir(), 'funnel-proj-'));
const cwd = process.cwd();
try {
fs.mkdirSync(path.join(proj, '.claude-flow'), { recursive: true });
fs.writeFileSync(path.join(proj, '.claude-flow', 'config.json'), JSON.stringify({ funnel: { enabled: false } }));
fs.writeFileSync(path.join(proj, 'claude-flow.config.json'), JSON.stringify({ funnel: { enabled: false } }));
process.chdir(proj);
const res = await sub(hooksCommand, 'refresh-funnel')(ctx());
expect(requestSpy).not.toHaveBeenCalled();
expect(res.data).toMatchObject({ skipped: 'funnel-disabled', decidedBy: 'project-config' });
} finally {
process.chdir(cwd);
fs.rmSync(proj, { recursive: true, force: true });
}
});
});
describe('funnelUserAgent — release + OS family only', () => {
it('carries version, platform and arch and nothing else', () => {
expect(funnelUserAgent('3.72.3', 'linux', 'x64')).toBe('ruflo-funnel/messages ruflo/3.72.3 (linux; x64)');
});
it('strips anything that is not a token character', () => {
const ua = funnelUserAgent('1.0.0\r\nX-Evil: 1', 'darwin (beta)', 'arm64;');
expect(ua).toBe('ruflo-funnel/messages ruflo/1.0.0X-Evil1 (darwinbeta; arm64)');
expect(ua).not.toMatch(/[\r\n]/);
});
});
describe('funnel telemetry on|off (ADR-302 telemetry domain)', () => {
it('is off by default and creates no ID', () => {
expect(hasConsent('telemetry')).toBe(false);
expect(getFunnelId()).toBeNull();
});
it('on grants consent and creates the pseudonymous ID; off revokes and deletes it', async () => {
const on = await sub(funnelCommand, 'telemetry')(ctx(['on']));
expect(on.success).toBe(true);
expect(hasConsent('telemetry')).toBe(true);
expect(getFunnelId()).toMatch(/^[0-9a-f-]{36}$/);
await sub(funnelCommand, 'telemetry')(ctx(['off']));
expect(hasConsent('telemetry')).toBe(false);
expect(getFunnelId()).toBeNull();
expect(fs.existsSync(path.join(stateDir, 'funnel-id.json'))).toBe(false);
});
it('funnel disable also revokes telemetry consent', async () => {
await sub(funnelCommand, 'telemetry')(ctx(['on']));
await sub(funnelCommand, 'disable')(ctx());
expect(hasConsent('telemetry')).toBe(false);
});
it('refresh-funnel flushes queued events only with consent', async () => {
await sub(hooksCommand, 'refresh-funnel')(ctx());
expect(requestSpy).toHaveBeenCalledTimes(1); // messages only
requestSpy.mockClear();
await sub(funnelCommand, 'telemetry')(ctx(['on']));
fs.writeFileSync(
path.join(stateDir, 'funnel-events.jsonl'),
JSON.stringify({ event: 'promo_impression', surface: 'statusline', release: '3.72.3', timestampBucket: '2026-11-06' }) + '\n',
);
const res = await sub(hooksCommand, 'refresh-funnel')(ctx());
const hosts = requestSpy.mock.calls.map((c) => `${(c[0] as { method: string }).method} ${(c[0] as { path: string }).path}`);
expect(hosts).toEqual(['GET /v1/messages', 'POST /v1/events']);
expect((res.data as { events?: unknown }).events).toBeDefined();
});
it('rejects an unknown option', async () => {
const res = await sub(funnelCommand, 'telemetry')(ctx(['maybe']));
expect(res.success).toBe(false);
});
});
+37 -2
View File
@@ -11,6 +11,7 @@
* funnel earnings show accrued and paid revenue-share balance
* funnel unenroll revoke local enrollment (funnel itself stays enabled)
* funnel id print the pseudonymous funnel ID, if one exists
* funnel telemetry opt in or out of anonymous usage telemetry (on | off | status)
*/
import { execFile } from 'node:child_process';
@@ -83,12 +84,46 @@ const disableSub: Command = {
action: async (): Promise<CommandResult> => {
setUserConfigEnabled(false);
recordDisclosureDeclined();
// "All surfaces" includes opt-in event telemetry: revoke it too.
if (hasConsent('telemetry')) recordConsent('telemetry', false, 'cli-funnel-disable');
deleteFunnelData(); // opt-out deletes the pseudonymous ID + local event queue
output.printSuccess('Funnel disabled. All promotional surfaces are off; local funnel data deleted.');
output.printSuccess('Funnel disabled. All promotional surfaces are off, no funnel.ruv.io requests are made, and local funnel data is deleted.');
return { success: true };
},
};
// ADR-302 consent domain `telemetry` (ADR-305/309): explicit opt-in to the
// anonymous funnel event stream. Enrollment never toggles it; only this
// command does. Off by default.
const telemetrySub: Command = {
name: 'telemetry',
description: 'Opt in or out of anonymous usage telemetry: telemetry on | off | status',
action: async (ctx: CommandContext): Promise<CommandResult> => {
const verb = String(ctx.args[0] ?? 'status').toLowerCase();
if (verb === 'on') {
recordConsent('telemetry', true, 'cli-funnel-telemetry');
const id = getFunnelId();
output.printSuccess('Telemetry on. Once per session ruflo sends anonymous funnel events (daily-bucketed, no paths, prompts, code or account data) with a random ID that rotates every 90 days.');
output.writeln('Turn it off any time with `ruflo funnel telemetry off`; that also deletes the ID and the local event queue.');
return { success: true, data: { telemetry: true, id } };
}
if (verb === 'off') {
recordConsent('telemetry', false, 'cli-funnel-telemetry');
deleteFunnelData();
output.printSuccess('Telemetry off. The pseudonymous ID and the local event queue are deleted.');
return { success: true, data: { telemetry: false } };
}
if (verb !== 'status') {
output.printError(`Unknown telemetry option "${verb}". Use: on | off | status`);
return { success: false, exitCode: 1 };
}
const on = hasConsent('telemetry');
output.writeln(`Telemetry: ${on ? 'on (opted in)' : 'off'}`);
output.writeln('Without opting in, ruflo makes one anonymous request per session to fetch tips (release + OS only); `ruflo funnel disable` stops that too.');
return { success: true, data: { telemetry: on } };
},
};
const enableSub: Command = {
name: 'enable',
description: 'Re-enable funnel surfaces at the user tier (env/enterprise disables still win)',
@@ -328,7 +363,7 @@ const idSub: Command = {
export const funnelCommand: Command = {
name: 'funnel',
description: 'Control the Cognitum lifecycle funnel surfaces (tips, enrollment, notices)',
subcommands: [statusSub, disableSub, enableSub, acceptSub, openSub, enrollSub, earningsSub, unenrollSub, idSub],
subcommands: [statusSub, disableSub, enableSub, acceptSub, openSub, enrollSub, earningsSub, unenrollSub, idSub, telemetrySub],
examples: [
{ command: 'ruflo funnel status', description: 'Effective state + deciding source' },
{ command: 'ruflo funnel accept', description: 'Skip the 24h grace so rotation starts now' },
+13 -2
View File
@@ -5452,8 +5452,19 @@ const refreshFunnelCommand: Command = {
],
action: async (ctx: CommandContext): Promise<CommandResult> => {
try {
const { refreshRemoteMessages } = await import('../funnel/index.js');
const result = await refreshRemoteMessages();
const { refreshRemoteMessages, resolveFunnelEnabled, hasConsent, flushEvents } = await import('../funnel/index.js');
// `ruflo funnel disable` (and every other opt-out source) must stop the
// network call, not only the promo row (ADR-309 operational telemetry).
const decision = resolveFunnelEnabled();
const result: Record<string, unknown> = decision.enabled
? await refreshRemoteMessages()
: { refreshed: false, skipped: 'funnel-disabled', decidedBy: decision.decidedBy };
// Opt-in event telemetry (ADR-305/309): only with an explicit, current
// telemetry consent receipt; flushEvents re-checks consent itself.
if (decision.enabled && hasConsent('telemetry')) {
const { rufloPackageVersion } = await import('../mcp-protocol.js');
result.events = await flushEvents({ release: rufloPackageVersion() });
}
if (!ctx.flags.quiet) {
output.writeln(JSON.stringify(result));
}
+2
View File
@@ -39,8 +39,10 @@ export {
export {
DEFAULT_MESSAGES_ENDPOINT,
getRemoteMessages,
funnelUserAgent,
refreshRemoteMessages,
} from './message-transport.js';
export { flushEvents } from './event-transport.js';
export {
PROMO_REPEAT_CAP_MS,
PROMO_SLOT_MODULO,
@@ -37,8 +37,25 @@ import { URL as NodeUrl } from 'url';
import type { FunnelMessage } from './types.js';
import { isValidMessage } from './messages.js';
import { readStateJson, writeStateJson } from './state.js';
import { rufloPackageVersion } from '../mcp-protocol.js';
const CACHE_FILE = 'funnel-messages-cache.json';
/**
* User-Agent for the messages fetch — ADR-309 operational telemetry (see
* docs/privacy/funnel-legitimate-interest-assessment.md). Only the CLI
* release and the coarse OS/CPU family, so the server can count releases
* and platforms. No identifier, path, hostname, or account: the
* `ruflo-funnel/messages` prefix stays first so existing log filters match.
*/
export function funnelUserAgent(
version: string = rufloPackageVersion(),
platform: string = process.platform,
arch: string = process.arch,
): string {
const clean = (v: string) => v.replace(/[^A-Za-z0-9._-]/g, '').slice(0, 32) || 'unknown';
return `ruflo-funnel/messages ruflo/${clean(version)} (${clean(platform)}; ${clean(arch)})`;
}
const CACHE_TTL_MS = 6 * 60 * 60 * 1000; // 6h — matches server Cache-Control
const CACHE_MAX_BYTES = 128 * 1024;
const CACHE_MAX_MESSAGES = 200;
@@ -95,7 +112,7 @@ function httpsGet(url: string): Promise<FetchResult> {
path: target.pathname + target.search,
method: 'GET',
headers: {
'User-Agent': 'ruflo-funnel/messages',
'User-Agent': funnelUserAgent(),
'Accept': 'application/json',
},
},
@@ -21,7 +21,7 @@ The funnel turns an open-source developer tool into an acquisition channel for a
## Privacy classification and lawful basis
- The pseudonymous funnel ID (ADR-305) plus server-side raw events constitute **personal data** under GDPR and are treated as such under CCPA/CPRA. "Anonymous" is not claimed anywhere identifiers exist.
- **Operational telemetry** (crash/health, disable rates): **legitimate-interest** basis, with a documented, minimized Legitimate Interest Assessment checked into the repo.
- **Operational telemetry** (crash/health, disable rates): **legitimate-interest** basis, with a documented, minimized Legitimate Interest Assessment checked into the repo. The one default-on request, the `GET /v1/messages` tips fetch, is assessed in [docs/privacy/funnel-legitimate-interest-assessment.md](../../../docs/privacy/funnel-legitimate-interest-assessment.md) (2026-11-06). It carries only release and OS/CPU family, every opt-out stops it, raw IPs are kept for 30 days, and aggregates are counts only.
- **Marketing attribution** (signup/conversion events): **explicit opt-in only** — rides the telemetry consent domain (ADR-302), never inferred from product usage.
- **No cross-product behavioral profiling** without a separate, explicit consent — a Cognitum account must not silently enrich funnel events, and vice versa.
- **Prohibited in any funnel event, permanently:** raw prompts, command lines, file paths, repository names, model inputs/outputs, hostnames, usernames.
@@ -80,6 +80,6 @@ Disclosure acceptance, disable rate, signup initiation, account completion, prox
Take an existing **offline** installation, upgrade it, and run the CLI in TTY, non-TTY, CI, screen-reader/reduced-motion, and enterprise-disabled configurations. Prove that:
1. No promotional content appears before the ADR-301 disclosure.
2. No network request occurs without opt-in (verified by the zero-unexpected-network-calls gate under packet capture).
2. No network request occurs without opt-in, except the ADR-309 operational `GET /v1/messages` fetch ([Legitimate Interest Assessment](../../../docs/privacy/funnel-legitimate-interest-assessment.md)). Every opt-out source above disables that fetch too (verified by the zero-unexpected-network-calls gate under packet capture, and by `__tests__/funnel-telemetry-optout.test.ts`).
3. `RUFLO_FUNNEL=0` suppresses every funnel surface (ADR-301, 302, 303).
4. All core ruflo commands remain behaviorally identical — exit codes, operational output, and latency within gate thresholds — across every configuration above.