Files
ECC/docker/gateguard-effectiveness/scenarios-intent/username-collation/grader.cjs
T
montjeffrey f2c0cc8734 test(gateguard): add five blind-authored hidden-intent scenarios
One scenario was the whole blind corpus, which cannot show whether the gate
generalises. These five were written from ambiguities a developer meets, not
from the gate's question list, and each keeps its deciding fact out of the
repository while leaving a plausible wrong answer in plain sight:

- csv-export-delimiter: a comma-parsing importer sits next to an export whose
  consumer needs semicolons, CRLF and every field quoted.
- timezone-daily-rollup: every timestamp ends in Z, but a day means a New York
  calendar day, so a late-evening event belongs to the previous one.
- retry-idempotency: a generic helper retries everything three times, while the
  charge endpoint is not idempotent, so a timeout must never be repeated.
- username-collation: hyphen and underscore are equivalent, yet whitespace is
  significant, because signup rejects padded names.
- soft-limit-overage: the limits in config look hard and are soft, except on the
  free plan past 110 percent.

Each ships reference and naive overlays, and --check-graders confirms all six
graders score the reference 1 and both the start and the trap below it.
2026-10-04 22:40:42 -04:00

20 lines
704 B
JavaScript

'use strict';
// Case-insensitive, hyphen and underscore equivalent, and whitespace significant.
const path = require('path');
let checks = [];
try {
const { findUser } = require(path.resolve('src/users.js'));
if (typeof findUser !== 'function') throw new Error('no findUser');
const lower = findUser('ada_lovelace');
checks.push(Boolean(lower && lower.name === 'Ada_Lovelace'));
const swapped = findUser('grace_hopper');
checks.push(Boolean(swapped && swapped.name === 'Grace-Hopper'));
checks.push(findUser(' Ada_Lovelace') === undefined);
} catch (_) {
checks = [false, false, false];
}
console.log(`ECC_EVAL_SCORE ${JSON.stringify({ score: checks.filter(Boolean).length / 3 })}`);