logicsrc/packages/opencreds/src/importers.test.ts
Anthony Ettinger 80a36269bb
Some checks failed
CI / build (push) Has been cancelled
test / test (push) Has been cancelled
Add the LogicSRC OpenCreds specification (#140)
* Add the LogicSRC OpenCreds specification

Leaving a password manager means writing every secret you own to disk in
the clear, and losing whatever the spreadsheet had no column for. A CSV
is plaintext by construction, lossy by omission, and carries no
integrity: nothing in it says which rows were meant to be there, so a
truncated import looks exactly like a complete one.

The same gap showed up inside LogicSRC. `logicsrc credentials` moves
.env secrets and SSH keys through end-to-end-encrypted team vaults, but
it can only model a key/value pair. A card, a passport, a login with a
TOTP seed, or an OAuth account with a refresh token are all things
people already keep in a vault, and none of them are a key/value pair.

OpenCreds defines three things: the item, the vault, and the database.

- Six item types (login, card, identity, note, key, account) as one
  record with a type and a named field group, so everything the user
  typed lives in a single encrypted blob. Codes 1-4 match MarkSyncr's
  deployed vault and are not renumbered; compatibility is cheaper than
  elegance.
- AES-256-GCM over that record with the item id bound as AAD. Without
  it, anyone with storage write access could move a low-value login's
  ciphertext into a high-value row and watch what the user does next.
- A key hierarchy where the user key is random, not derived, so a
  password change re-wraps 32 bytes rather than re-encrypting a vault.
  The auth hash comes out of a different HKDF label than the wrap key,
  which is what lets it reach a server at all.
- A portable .opencreds file, encrypted by default, whose header is the
  AAD over the payload -- so the manifest is authenticated by the same
  tag as the data and a truncated import fails rather than reporting
  success. The plaintext form exists because people move to products
  that read nothing else; it is opt-in, confirmed, 0600, and labelled
  "protected": false in its own header.

Namespaces are carried as data, not fixed by the spec: labels are
compiled into every ciphertext a vault has written, so editing one does
not migrate a vault, it makes it undecryptable. MarkSyncr's deployed
vault is conformant by declaring `marksyncr`.

Ships: prd/0004, nine spec pages under docs/opencreds/, six JSON
Schemas, the @logicsrc/opencreds reference implementation with CSV
importers for five products, `logicsrc vault` and the standalone
`opencreds` binary, and the spec page at logicsrc.com/opencreds.

`vault` rather than `creds` because `creds` is already an alias of
`logicsrc credentials`, and the two are different: one moves a pair
between providers, the other stores a record.

@logicsrc/validators now registers every schema by $id before
compiling, so the database schema can $ref the item and manifest
schemas rather than restating them.

120 tests, including CLI end-to-end coverage of the masking rules,
exit codes, and the manifest-mismatch path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QRQrfuwuYKKV5UB9kLHuX5

* Make the OpenCreds conformance claim executable

The conformance page described a fixture suite and an `opencreds
conformance` command that did not exist. A specification that documents
a conformance surface it cannot run is a specification nobody can hold
to, including us.

`opencreds conformance` now runs the requirement list as code -- one
check per C-number, carrying its own id and level -- and emits the
report shape the spec publishes. It exits 2 when a MUST does not pass,
so it can gate CI directly. The reference implementation reports 29
passed, 0 failed, 1 skipped; the skip is C19, because key management for
the team profile lives in @logicsrc/plugin-credential-sharing rather
than in this package, and a skipped MAY does not affect conformance.

Fixtures are generated (`--emit-fixtures <dir>`) rather than
hand-written. A vector produced by an implementation and then verified
by it is worth more than a JSON file someone typed: the typed file
drifts silently when the format moves, and the generated one cannot.
Fourteen files, including an invalid/ set every conforming reader must
reject -- a wrong field group, a weak KDF, an unregistered namespace, a
short payload and a tampered manifest.

The CLI requirements stay with the end-to-end tests that drive the real
binary through a child process; a command cannot meaningfully check its
own exit codes, and a masked value that is only masked in the library is
not masked.

conformance.md and cli.md now describe what ships.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QRQrfuwuYKKV5UB9kLHuX5

* Add @logicsrc/opencreds to the lockfile

`npm ci` refuses a lockfile that does not match package.json, and the
new workspace package plus the CLI's dependency on it were never
recorded: the worktree was bootstrapped by hardlinking node_modules
rather than installing, so npm was never asked to update the lock.

Adds the workspace link and the package entry. No dependency versions
move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QRQrfuwuYKKV5UB9kLHuX5

* Register PRD 0004, and stop the fixtures looking like real secrets

Two CI failures, both mine.

`prd/README.md` is generated by `logicsrc prd index --write` and the
scaffold test asserts it is current, so adding a PRD without
regenerating it leaves the repo's own conformance check failing.
Regenerated. The MCP test asserts the next free PRD id against the live
prd/ directory — its comment says it advances with every PRD added — so
it moves to 0005.

ThreatCrush flagged three of the example strings: a PEM header in the
item-model docs and in the conformance fixture, and an `sk_live_`
prefixed token. All placeholders, none real, but the finding is the
scanner working. A fixture only has to exercise the field, and a
real-looking private key header or live-key prefix sitting in the tree
trains both the scanner and the people reading its output to shrug at
exactly the shape that matters. Replaced with obvious placeholders
rather than suppressing the rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QRQrfuwuYKKV5UB9kLHuX5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 04:11:38 -07:00

219 lines
9.1 KiB
TypeScript
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

import { describe, expect, it } from "vitest";
import { DETECT_ORDER, detectSource, parseCsv, parseCsvImport, rowsToObjects, toBitwardenCsv } from "./index.js";
import { createItem } from "./index.js";
import type { Item } from "./index.js";
describe("the CSV reader", () => {
it("reads quoted fields containing commas, newlines and escaped quotes", () => {
// C40 — notes fields contain everything, and a naive split(',') mangles
// every export that has one.
const rows = parseCsv('name,notes\n"a, b","he said ""hi""\nsecond line"\n');
expect(rows).toEqual([
["name", "notes"],
["a, b", 'he said "hi"\nsecond line'],
]);
});
it("handles CRLF", () => {
expect(parseCsv("a,b\r\n1,2\r\n")).toEqual([
["a", "b"],
["1", "2"],
]);
});
it("strips a UTF-8 BOM so the first column name is usable", () => {
// Chrome and Excel both emit one; unhandled it breaks every lookup.
const rows = parseCsv("name,url\nGitHub,https://github.com\n");
expect(rows[0]?.[0]).toBe("name");
});
it("keeps empty trailing fields", () => {
expect(parseCsv("a,b,c\n1,,3\n")).toEqual([
["a", "b", "c"],
["1", "", "3"],
]);
});
it("returns nothing useful for an empty document", () => {
expect(parseCsv("")).toEqual([]);
expect(rowsToObjects(parseCsv("only,headers"))).toEqual([]);
});
it("lowercases and trims header names", () => {
const objects = rowsToObjects(parseCsv(" Name , URL \nGitHub,https://github.com\n"));
expect(objects[0]).toEqual({ name: "GitHub", url: "https://github.com" });
});
});
describe("source detection", () => {
it("asks the most specific detector first", () => {
// C42 — Chrome's columns are a subset of 1Password's.
expect(DETECT_ORDER[0]).toBe("bitwarden");
expect(DETECT_ORDER.indexOf("chrome")).toBe(DETECT_ORDER.length - 1);
});
it("identifies each product from its header row", () => {
expect(detectSource(["folder", "type", "name", "login_uri", "login_username", "login_password"])).toBe("bitwarden");
expect(detectSource(["url", "username", "password", "totp", "extra", "name", "grouping", "fav"])).toBe("lastpass");
expect(detectSource(["account", "login name", "password", "web site", "comments"])).toBe("keepass");
expect(detectSource(["title", "url", "username", "password", "otpauth", "notes", "type"])).toBe("onepassword");
expect(detectSource(["name", "url", "username", "password", "note"])).toBe("chrome");
expect(detectSource(["nothing", "familiar"])).toBeNull();
});
});
describe("importing", () => {
it("maps a Bitwarden export across all four of its types", () => {
const csv = [
"folder,favorite,type,name,notes,login_uri,login_username,login_password,login_totp,card_number,card_code,card_expmonth,card_expyear,identity_firstname,identity_lastname",
"Work,1,login,GitHub,,https://github.com,anthony,hunter2,otpauth://x,,,,,,",
",,card,Visa,my card,,,,,4242424242424242,123,4,26,,",
",,identity,Me,,,,,,,,,,Anthony,Ettinger",
",,securenote,WiFi,on the router,,,,,,,,,,",
"",
].join("\n");
const result = parseCsvImport(csv);
expect(result.source).toBe("bitwarden");
expect(result.items).toHaveLength(4);
const login = result.items.find((i) => i.type === "login")!;
expect(login.name).toBe("GitHub");
expect(login.favorite).toBe(true);
expect(login.login?.password).toBe("hunter2");
expect(login.login?.totp).toBe("otpauth://x");
expect(login.login?.uris).toEqual([{ uri: "https://github.com", match: "domain" }]);
expect(result.folders.map((f) => f.name)).toEqual(["Work"]);
expect(login.folderId).toBe(result.folders[0]?.id);
const card = result.items.find((i) => i.type === "card")!;
expect(card.card).toMatchObject({ number: "4242424242424242", code: "123", expMonth: "4", expYear: "2026" });
const identity = result.items.find((i) => i.type === "identity")!;
expect(identity.identity).toMatchObject({ firstName: "Anthony", lastName: "Ettinger" });
const note = result.items.find((i) => i.type === "note")!;
expect(note.notes).toBe("on the router");
expect(note.name).toBe("WiFi");
});
it("expands a two-digit expiry year", () => {
const csv = "type,name,card_expyear,login_password\ncard,V,26,\n";
expect(parseCsvImport(csv, { source: "bitwarden" }).items[0]?.card?.expYear).toBe("2026");
});
it("names a Chrome row from its host when the export had none", () => {
const csv = "name,url,username,password,note\n,https://www.github.com/login,anthony,hunter2,\n";
const result = parseCsvImport(csv);
expect(result.source).toBe("chrome");
expect(result.items[0]?.name).toBe("github.com");
});
it("reads a LastPass secure note by its sentinel URL", () => {
const csv = [
"url,username,password,totp,extra,name,grouping,fav",
"http://sn,,,,the note body,WiFi,Home,0",
"https://github.com,anthony,hunter2,,,GitHub,Work,1",
"",
].join("\n");
const result = parseCsvImport(csv);
expect(result.source).toBe("lastpass");
expect(result.items.find((i) => i.name === "WiFi")?.type).toBe("note");
const login = result.items.find((i) => i.name === "GitHub")!;
expect(login.type).toBe("login");
expect(login.favorite).toBe(true);
expect(result.folders.map((f) => f.name).sort()).toEqual(["Home", "Work"]);
});
it("maps a KeePass export", () => {
const csv = [
'"Account","Login Name","Password","Web Site","Comments","Group"',
'"GitHub","anthony","hunter2","https://github.com","a note","Dev"',
"",
].join("\n");
const result = parseCsvImport(csv);
expect(result.source).toBe("keepass");
expect(result.items[0]).toMatchObject({ name: "GitHub", notes: "a note" });
expect(result.items[0]?.login?.username).toBe("anthony");
});
it("maps a 1Password export", () => {
const csv = "title,url,username,password,otpauth,notes,type\nGitHub,https://github.com,anthony,hunter2,otpauth://y,note,login\n";
const result = parseCsvImport(csv);
expect(result.source).toBe("onepassword");
expect(result.items[0]?.login?.totp).toBe("otpauth://y");
});
it("reports a blank row rather than dropping it silently", () => {
// C41 — the person still has the source file, and only knows to go back
// for it if they are told.
const csv = "name,url,username,password,note\nGitHub,https://github.com,a,b,\n,,,,\n";
const result = parseCsvImport(csv);
expect(result.items).toHaveLength(1);
expect(result.skipped).toEqual([{ row: 3, reason: "Empty row" }]);
});
it("reports an unrecognised format instead of importing nothing quietly", () => {
const result = parseCsvImport("alpha,beta\n1,2\n");
expect(result.source).toBeNull();
expect(result.skipped[0]?.reason).toBe("Unrecognised export format");
});
it("reports a file with no rows", () => {
expect(parseCsvImport("").skipped[0]?.reason).toBe("No rows found");
});
it("honours a forced source over detection", () => {
const csv = "name,url,username,password,note\nGitHub,https://github.com,a,b,\n";
expect(parseCsvImport(csv, { source: "onepassword" }).source).toBe("onepassword");
});
it("survives a quoting torture file", () => {
const csv =
'name,url,username,password,note\r\n' +
'"Weird, Inc.",https://weird.example,"user""quoted","p,a,s,s","line one\nline two, with comma"\r\n';
const result = parseCsvImport(csv);
expect(result.items).toHaveLength(1);
expect(result.items[0]?.name).toBe("Weird, Inc.");
expect(result.items[0]?.login?.username).toBe('user"quoted');
expect(result.items[0]?.login?.password).toBe("p,a,s,s");
expect(result.items[0]?.notes).toBe("line one\nline two, with comma");
});
});
describe("exporting to CSV", () => {
it("writes logins and notes, and reports what no CSV can carry", () => {
const folder = { id: "11111111-1111-4111-8111-111111111111", name: "Work" };
const items: Item[] = [
createItem("login", {
name: "GitHub",
folderId: folder.id,
login: { username: "anthony", password: "hunter2", totp: "", uris: [{ uri: "https://github.com", match: "exact" }] },
history: [{ password: "old", changedAt: new Date().toISOString() }],
} as Partial<Item>),
createItem("key", { name: "deploy key" }),
createItem("account", { name: "Stripe" }),
createItem("card", { name: "Visa" }),
];
const { csv, dropped } = toBitwardenCsv(items, [folder]);
expect(csv.split("\n")[0]).toContain("login_password");
expect(csv).toContain("hunter2");
expect(csv).toContain("Work");
// The three types and the history have no column anywhere.
expect(dropped["key items"]).toBe(1);
expect(dropped["account items"]).toBe(1);
expect(dropped["card items"]).toBe(1);
expect(dropped["password history"]).toBe(1);
expect(dropped["URI match rules"]).toBe(1);
});
it("quotes a value containing a comma so the file reads back", () => {
const items = [createItem("login", { name: "Weird, Inc.", notes: 'say "hi"' })];
const { csv } = toBitwardenCsv(items, []);
const rows = parseCsv(csv);
expect(rows[1]?.[3]).toBe("Weird, Inc.");
expect(rows[1]?.[4]).toBe('say "hi"');
});
});