logicsrc/apps/logicsrc-web
Anthony Ettinger 93af4770ac
Charge AI training crawlers for access (x402 gateway) (#141)
* Charge AI training crawlers for access (@profullstack/x402-gateway)

Training crawlers (GPTBot, ClaudeBot, CCBot, meta-externalagent, Bytespider,
Applebot-Extended) get 402 Payment Required with an x402 offer, or the sales
page at /crawl, and a paid pass opens the site for a day. People, search
engines and retrieval crawlers pass through untouched. robots.txt is now
generated from the same lists.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2

* Type the middleware as returning Response | NextResponse

The crawl gateway answers with a plain Fetch Response.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2

* Contract test awaits the now-async proxy

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 14:49:39 -07:00
..
contract Charge AI training crawlers for access (x402 gateway) (#141) 2026-09-05 14:49:39 -07:00
e2e fix(web): the Top-Level Pages cards led nowhere, and /privacy was not a page (#114) 2026-07-30 14:35:51 -07:00
public fix(install): build before replacing the working install (#118) 2026-07-30 20:18:46 -07:00
src Charge AI training crawlers for access (x402 gateway) (#141) 2026-09-05 14:49:39 -07:00
.gitignore Migrate logicsrc-web from Vite SPA to Next.js 16 App Router 2026-06-07 03:41:48 +00:00
next.config.ts feat(web): serve the CLI login flow from the apex (#106) 2026-07-30 10:25:16 -07:00
package.json Charge AI training crawlers for access (x402 gateway) (#141) 2026-09-05 14:49:39 -07:00
playwright.config.ts Migrate logicsrc-web from Vite SPA to Next.js 16 App Router 2026-06-07 03:41:48 +00:00
tsconfig.json Migrate logicsrc-web from Vite SPA to Next.js 16 App Router 2026-06-07 03:41:48 +00:00
vitest.config.ts Migrate logicsrc-web from Vite SPA to Next.js 16 App Router 2026-06-07 03:41:48 +00:00