test(web): fail the build when marketing copy drifts from shipped providers (#126)
Some checks are pending
CI / build (push) Waiting to run
test / test (push) Waiting to run

The last three PRs all fixed the same class of bug. /credential-sharing
and README.md are hand-written copy; the providers they advertise are a
real registry in @logicsrc/plugin-credential-sharing. Nothing connected
the two, so the `team` provider shipped on 2026-07-13 and three weeks
later both surfaces still described a five-provider tool with no mention
of teams. The docs were right the whole time -- only the pages people
actually land on had gone stale, which is worse, because it reads as
"the product cannot do this" rather than as a documentation gap.

Assert it instead. For every provider in the registry, the Credential
Sharing section and the README must say something that counts as
advertising it. The registry's own `name` cannot be the proof -- `env`
is "Local .env file" and `team` is "LogicSRC Team Vault", neither of
which is how the copy reads -- so each provider declares its own
pattern, and a provider with no declaration fails too. That way adding
a provider forces a deliberate answer about the customer-facing copy.

Verified against the bug it is meant to catch: reverting the team copy
reproduces "These providers ship but /credential-sharing never mentions
them: team", and reverting the README line reproduces the same for
sh1pt and team.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Anthony Ettinger 2026-08-03 07:18:13 -07:00 committed by GitHub
parent bfb536c894
commit 4c88155f08
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 87 additions and 0 deletions

View file

@ -0,0 +1,85 @@
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
import { describe, expect, it } from "vitest";
import { credentialProviders } from "@logicsrc/plugin-credential-sharing";
import { renderPageMarkup } from "../src/lib/page-markup";
/**
* Guards the marketing page against the product.
*
* /credential-sharing is hand-written copy in page-markup.ts, while the
* providers it advertises are a real registry in the plugin. Nothing connected
* the two: the `team` provider shipped on 2026-07-13 and three weeks later the
* page still described a five-provider tool with no mention of teams, which is
* long enough for a reader to conclude the capability did not exist. The docs
* were correct the whole time -- only the surfaces people actually land on had
* drifted. These tests turn that drift into a failing build.
*/
// A shipped provider id -> proof that the customer-facing copy mentions it.
// The registry's own `name` is not usable as the proof: `env` is "Local .env
// file" and `team` is "LogicSRC Team Vault", neither of which is how the copy
// reads. So each provider declares what "advertised" looks like for it, and
// the first test below makes adding a provider without an entry a failure.
const MARKETING_PROOF: Record<string, RegExp> = {
env: /\.env/,
doppler: /Doppler/,
railway: /Railway/,
"github-secrets": /GitHub Secrets/,
sh1pt: /sh1pt/,
team: /[Tt]eam vault/
};
const REPO_ROOT = resolve(process.cwd(), "../..");
/** Just the Credential Sharing band, so a stray match elsewhere cannot pass. */
function credentialSection(): string {
const markup = renderPageMarkup();
const start = markup.indexOf('<section id="credential-sharing"');
expect(start, "the credential-sharing section should exist").toBeGreaterThan(-1);
const end = markup.indexOf("</section>", start);
return markup.slice(start, end);
}
describe("marketing copy tracks the shipped credential providers", () => {
it("every shipped provider declares what advertising it looks like", () => {
const missing = credentialProviders
.filter((provider) => !MARKETING_PROOF[provider.id])
.map((provider) => provider.id);
expect(
missing,
`Add these provider ids to MARKETING_PROOF, then make sure the marketing page and README actually say so: ${missing.join(", ")}`
).toEqual([]);
});
it("the credential sharing section names every shipped provider", () => {
const section = credentialSection();
const unadvertised = credentialProviders
.filter((provider) => {
const proof = MARKETING_PROOF[provider.id];
return proof ? !proof.test(section) : false;
})
.map((provider) => provider.id);
expect(
unadvertised,
`These providers ship but /credential-sharing never mentions them: ${unadvertised.join(", ")}. Update renderPageMarkup in src/lib/page-markup.ts.`
).toEqual([]);
});
it("the README names every shipped provider", () => {
const readme = readFileSync(resolve(REPO_ROOT, "README.md"), "utf8");
const unadvertised = credentialProviders
.filter((provider) => {
const proof = MARKETING_PROOF[provider.id];
return proof ? !proof.test(readme) : false;
})
.map((provider) => provider.id);
expect(
unadvertised,
`These providers ship but README.md never mentions them: ${unadvertised.join(", ")}.`
).toEqual([]);
});
});