bbs.profullstack/logicsrc
1
0
Fork 0
mirror of https://github.com/profullstack/logicsrc.git synced 2026-09-10 19:26:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Charge AI training crawlers for access (x402 gateway) (#141)

Browse source 
* Charge AI training crawlers for access (@profullstack/x402-gateway)

Training crawlers (GPTBot, ClaudeBot, CCBot, meta-externalagent, Bytespider,
Applebot-Extended) get 402 Payment Required with an x402 offer, or the sales
page at /crawl, and a paid pass opens the site for a day. People, search
engines and retrieval crawlers pass through untouched. robots.txt is now
generated from the same lists.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2

* Type the middleware as returning Response | NextResponse

The crawl gateway answers with a plain Fetch Response.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2

* Contract test awaits the now-async proxy

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YafYxayh7Gqe5MWNNQMev2

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Anthony Ettinger 2026-09-05 14:49:39 -07:00 • committed by GitHub
parent 80a36269bb
commit 93af4770ac
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 65 additions and 35 deletions
Download patch file Download diff file Expand all files Collapse all files

10
package-lock.json generated
View file

@ -141,6 +141,7 @@
"@logicsrc/openprd": "file:../../packages/openprd",
"@profullstack/autoblog": "github:profullstack/autoblog#75e54af",
"@profullstack/stack": "^0.1.3",
"@profullstack/x402-gateway": "^0.1.0",
"@supabase/supabase-js": "^2.105.4",
"marked": "^18.0.5",
"next": "16.2.6",
@ -2618,6 +2619,15 @@
}
}
},
"node_modules/@profullstack/x402-gateway": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/@profullstack/x402-gateway/-/x402-gateway-0.1.0.tgz",
"integrity": "sha512-B7tWvWk/bIEoqyec6UoyRF1pO7X/+b+wFRv2ZFIClqskmEpyxoA559ZgdTvnxqAIvuDeE9v56nVpYRQ+lmOZQQ==",
"license": "MIT",
"engines": {
"node": ">=20.11"
}
},
"node_modules/@rolldown/binding-android-arm64": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.3.tgz",