logicsrc/packages/openerrand/src/pages.test.ts
Anthony Ettinger dde596b276
OpenErrand reference runner: @logicsrc/openerrand and logicsrc errand (#228)
* OpenErrand 0.1: an errand on a website with no API, with the human steps kept human

docs/openerrand.md mints OpenErrand: one JSON file per errand (register an
account, download a transcript) naming the site, the inputs with a
sensitivity class and ordered sources (document, vault, prompt, generate,
derive, candidate, literal), field rules matched by id then label, page and
wait steps, five human gates a runner never performs (declare,
identity-proofing, code, mail, captcha), outcomes, the never-retried shared
secret, vault and download outputs, hand-off cards that may name only public
inputs, the publisher index at /.well-known/openerrand.json, and thirteen
runner rules. The worked example is the MyFTB business registration that
cli-tools `ftb` performs (profullstack/cli-tools#125), with no personal data.

- @logicsrc/schemas: openerrand + openerrand-index schemas and fixtures
- @logicsrc/validators: semantic checks (references, templates, no personal
  or secret input on a card) and tests that validate the spec's own examples
- logicsrc-web: registry entry (process family), /openerrand landing page,
  the example and the index served as static files, contract tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenErrand: hand-off cards stay on the surface that owns the data

Anthony's ruling: tax and finance data never touches a social or promotion
tool, and nothing is sent to a CPA or preparer.

- Hand-off cards are delivered only on the surface that owns the errand's
  data (for a tax or finance errand, the principal's finance app through its
  CLI, PWA, MCP server or API, such as CoinPay, or the runner's terminal),
  never a social, promotion or third-party posting service, and never to
  anyone but the principal. A card for an errand with personal or secret
  inputs does not leave that surface. Runner rule 9 says the same.
- The run record and the sample run name the card by an opaque id
  (pin-letter/7f3k2q) instead of a mynaposter.com URL; the myna mention is gone.
- `principal: represented` no longer cites a preparer with a power of attorney.
- The FTB card's last step no longer suggests sending the PIN to someone else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenErrand: user-agent rule, captcha solver policy, reference runner note

Anthony's answers on #227 ("go with your recommendations"):

- Rule 11: a runner may run headless with a normal desktop browser user agent
  (dropping HeadlessChrome) and nothing more: no fingerprint spoofing beyond
  the UA string, no stealth plugins, no solving or evading a bot challenge.
  A challenge the browser completes itself is a wait step; any other is a
  captcha gate.
- Captcha solvers: new site.sector and captcha step `solver`
  (forbidden by default | allowed). Never allowed on government, tax,
  financial, healthcare or identity-provider sites, nor on any errand with a
  declare or identity-proofing step or a secret input; elsewhere only when the
  file says so, with every use logged. The validator rejects `allowed` in the
  forbidden set or without a stated sector; six new tests. The FTB example
  states sector "tax".
- Reference runner: @logicsrc/openerrand / `logicsrc errand run`, marked in
  progress; ftb stays the runner the example was taken from.
- Name stays OpenErrand; family stays Agents and process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenErrand reference runner: @logicsrc/openerrand and logicsrc errand

Ship the runner docs/openerrand.md promised. `logicsrc errand run <file>`
reads an OpenErrand 0.1 file, validates it with @logicsrc/validators, and
drives headless Chrome through it under the spec's thirteen rules;
`errand validate` shows what a file will ask of you and `errand status`
shows the last run of each errand, its card and any lockout.

The engine is generalised from cli-tools `ftb` (PR #125) with no dependency
on cli-tools: the CDP client and Chrome finder from wcag.ts, the page reader,
native-setter fill and forward-button picker from ftb-run.ts, and the rule
matcher, throttle and outcome logic from ftb.ts, all now driven by the file.

- Inputs: document (an extractor hook; the one shipped runs a local command
  that reads JSON requests and prints records), vault (teams or OpenCreds),
  prompt (no echo for secrets), generate, derive, candidate, literal.
  `--input name=value` wins. Shared-secret candidates are ranked as the spec
  says, one is submitted, and a rejection lists the others for --candidate.
- Rules: id before label, step rules first, choices before text, an id match
  final, an unmatched required field stops the run naming it.
- Gates: declare only with --declare after the values are shown; identity
  proofing never touched (URL only) and handed over or stopped on; code from
  the terminal or a code file, used once, a wrong code waits for the next;
  mail ends the run waiting with the card; captcha is the person's, and a
  CaptchaSolver interface is called only where the spec permits (no solver is
  bundled); wait steps are polled, never solved.
- Throttle: 2 runs per errand and account in 30 minutes, 4 a day, 2 minutes
  between runs on a site, lockouts from metadata.lockout or a default, held
  per site and account, never lifted by --force.
- Outputs: credentials written before success to a teams vault by pull,
  merge, push (metadata.vault or --vault), else a 0600 file said aloud;
  downloads type-checked and never overwritten with different bytes; cards
  only in the local run record.
- The user agent is Chrome's own with HeadlessChrome replaced, given at
  launch: a CDP override did not reach a navigation the page's own script
  started, which is exactly the proof-of-work interstitial case.

Tests: 85 in the package (rule engine, inputs, gates, throttle, outcomes,
captcha gating, vaults, outputs, commands) including an integration test
that runs the published FTB example unchanged in real headless Chrome
against a local HTTPS fake site (Chrome maps webapp.ftb.ca.gov to it and
every other host to NOTFOUND; all data fictional), and 2 in the CLI.

CLI 0.6.0 -> 0.7.0; @logicsrc/schemas and @logicsrc/validators 0.3.0 ->
0.4.0 (the OpenErrand schemas, and the vocabularies now exported for
runners); PRD 0009; the spec's Reference runner section and the landing
page say it ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 09:17:06 -07:00

133 lines
7.2 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { validateErrand } from "./load.js";
import { isLockout, outcomeOf, solverPermitted, stepFor } from "./pages.js";
import { ftbExample } from "./testing.js";
import { checkThrottle, emptyLedger, keyFor, LIMITS, lockoutMs, recordAttempt, recordLockout } from "./throttle.js";
import type { CaptchaStep, Errand } from "./types.js";
const page = (url: string, title = "", text = "", errors: string[] = []) => ({ url, title, text, errors, fields: [] });
describe("which step a page is", () => {
const e = ftbExample();
it("a wait step whose title and selector both fit", () => {
expect(stepFor(e, page("https://webapp.ftb.ca.gov/x", "Challenge Validation"), { "#sec-cpt-if": true })?.id).toBe("bot-check");
// All given parts must fit: the title alone is not enough.
expect(stepFor(e, page("https://webapp.ftb.ca.gov/x", "Challenge Validation"), { "#sec-cpt-if": false })?.id).toBe("form");
});
it("falls back to the page step with no match", () => {
expect(stepFor(e, page("https://webapp.ftb.ca.gov/MyFTBAccess/Registration/NewAccount", "Registration"), {})?.id).toBe("form");
});
it("an identity provider's origin is its gate", () => {
const withId: Errand = { ...e, steps: [...e.steps, { id: "id-me", kind: "identity-proofing", provider: "ID.me", origins: ["https://api.id.me"], why: "yours" }] };
expect(stepFor(withId, page("https://api.id.me/en/session"), {})?.id).toBe("id-me");
});
});
describe("outcomes", () => {
const e = ftbExample();
it("tests rejected outcomes first, against the errors when there are any", () => {
const p = page("https://webapp.ftb.ca.gov/c", "Confirmation", "Registration confirmation", ["The information does not match our records."]);
expect(outcomeOf(e, p)?.name).toBe("rejected");
});
it("reads the page text when there are no errors", () => {
expect(outcomeOf(e, page("https://webapp.ftb.ca.gov/c", "x", "Registration Confirmation. We will mail you a PIN."))?.name).toBe("registered");
expect(outcomeOf(e, page("https://webapp.ftb.ca.gov/c", "x", "Please enter your name"))).toBeNull();
});
it("needs both text and url when both are given", () => {
const both: Errand = { ...e, outcomes: [{ name: "done", kind: "success", text: "done", url: "/finished$" }] };
expect(outcomeOf(both, page("https://webapp.ftb.ca.gov/finished", "", "done"))?.name).toBe("done");
expect(outcomeOf(both, page("https://webapp.ftb.ca.gov/other", "", "done"))).toBeNull();
});
it("knows a lockout by the file's signature or the default", () => {
expect(isLockout(e, "Your account has been locked for 30 minutes")).toBe(true);
expect(isLockout(e, "You have exceeded the allowed number of attempts")).toBe(true);
expect(isLockout(e, "Welcome")).toBe(false);
const own: Errand = { ...e, metadata: { lockout: { text: "come back tomorrow", duration: "PT24H" } } };
expect(isLockout(own, "Please come back tomorrow")).toBe(true);
expect(isLockout(own, "account locked")).toBe(false);
expect(lockoutMs(own)).toBe(24 * 3_600_000);
expect(lockoutMs(e)).toBe(LIMITS.lockoutMs);
});
});
describe("captcha solver gating", () => {
const captcha: CaptchaStep = { id: "cap", kind: "captcha", match: { selector: ".g-recaptcha" }, solver: "allowed" };
const commercial = (over: Partial<Errand> = {}): Errand => ({
type: "logicsrc.openerrand",
version: "0.1",
name: "newsletter",
title: "Sign up",
site: { name: "Shop", sector: "commercial", origins: ["https://shop.example"], start: ["https://shop.example/"] },
inputs: { email: { type: "email", sensitivity: "personal", sources: [{ from: "prompt" }] } },
steps: [{ id: "form", kind: "page" }, captcha],
outcomes: [{ name: "ok", kind: "success", text: "thanks" }],
...over,
});
it("allows a solver only on a commercial or other site with nothing sensitive, as the file says", () => {
expect(validateErrand(commercial())).toEqual([]);
expect(solverPermitted(commercial(), captcha)).toBe(true);
expect(solverPermitted(commercial(), { ...captcha, solver: "forbidden" })).toBe(false);
expect(solverPermitted(commercial(), { ...captcha, solver: undefined })).toBe(false);
});
for (const sector of ["government", "tax", "financial", "healthcare", "identity-provider"] as const) {
it(`never on a ${sector} site, even when the file says allowed (and the validator rejects the file)`, () => {
const e = commercial({ site: { name: "x", sector, origins: ["https://x.example"], start: ["https://x.example/"] } });
expect(solverPermitted(e, captcha)).toBe(false);
expect(validateErrand(e).join(" ")).toMatch(/captcha solver is never allowed/);
});
}
it("never without a stated sector, with a secret input, or with a declaration", () => {
expect(solverPermitted(commercial({ site: { name: "x", origins: ["https://x.example"], start: ["https://x.example/"] } }), captcha)).toBe(false);
expect(solverPermitted(commercial({ inputs: { pw: { type: "string", sensitivity: "secret", sources: [{ from: "prompt" }] } } }), captcha)).toBe(false);
expect(solverPermitted(commercial({ steps: [{ id: "form", kind: "page" }, captcha, { id: "d", kind: "declare", statement: "x", why: "y" }] }), captcha)).toBe(false);
});
});
describe("throttle", () => {
const e = ftbExample();
const key = keyFor(e);
const t0 = new Date("2026-10-04T10:00:00Z");
const at = (min: number) => new Date(t0.getTime() + min * 60_000);
it("spaces runs on one site 2 minutes apart", () => {
const l = recordAttempt(emptyLedger(), key, t0);
expect(checkThrottle(l, key, at(1))).toMatchObject({ ok: false, lockout: false });
expect(checkThrottle(l, { ...key, account: "other", errand: "ftb-activate-business" }, at(1)).ok).toBe(false);
expect(checkThrottle(l, key, at(2)).ok).toBe(true);
});
it("allows 2 runs in 30 minutes and 4 a day per errand and account", () => {
let l = recordAttempt(emptyLedger(), key, t0);
l = recordAttempt(l, key, at(3));
expect(checkThrottle(l, key, at(10))).toMatchObject({ ok: false, reason: expect.stringMatching(/2 runs/) });
expect(checkThrottle(l, { ...key, account: "personal" }, at(10)).ok).toBe(true);
l = recordAttempt(l, key, at(40));
l = recordAttempt(l, key, at(80));
expect(checkThrottle(l, key, at(200))).toMatchObject({ ok: false, reason: expect.stringMatching(/4 runs/) });
expect(checkThrottle(l, key, at(24 * 60 + 1)).ok).toBe(true);
});
it("--force lifts the caps and never a lockout", () => {
let l = recordAttempt(emptyLedger(), key, t0);
expect(checkThrottle(l, key, at(1), true).ok).toBe(true);
l = recordLockout(l, key, at(1));
expect(checkThrottle(l, key, at(10), true)).toMatchObject({ ok: false, lockout: true });
// The lock is per site and account: another errand on the same account is held too.
expect(checkThrottle(l, { ...key, errand: "ftb-activate-business" }, at(10), true)).toMatchObject({ ok: false, lockout: true });
expect(checkThrottle(l, key, at(1 + 35), true).ok).toBe(true);
});
it("a shorter later lock never shortens a longer one", () => {
let l = recordLockout(emptyLedger(), key, t0, 24 * 3_600_000);
l = recordLockout(l, key, at(5));
expect(checkThrottle(l, key, at(60), true).ok).toBe(false);
});
});