Merge branch 'feat/files-sftp' into feat/notify-creds

# Conflicts:
#	cmd/agentbbs/main.go
This commit is contained in:
Anthony Ettinger 2026-06-23 10:46:00 +00:00
commit 95d1afb59a
45 changed files with 2924 additions and 389 deletions

View file

@ -210,6 +210,10 @@ directly over SFTP with the member's existing SSH login key.
member reaches their private files with the same key they log in with
(`sftp files@bbs.profullstack.com`). `scp`/`rsync -e ssh` work over the same
endpoint.
- **Eligibility:** members-only, **free for every member** (free and paid alike,
like IRC and News — not a Premium-gated perk). Non-members are refused at the
handshake; guests don't see the in-BBS browser. Operators can revoke an
individual account's SFTP access without affecting its BBS login.
- **In-BBS view:** a TUI file browser for the user's own workspace and the
shared area (list, rename, delete, up/download path, view usage vs. quota).
- **Operator TUI:** an admin management surface for the SFTP server — list

View file

@ -17,6 +17,20 @@ The username (`files`) is conventional and ignored — **identity is your SSH
key** (one key = one account, like the rest of the BBS). `scp`/`rsync` work
because they tunnel over the same SSH transport.
## Who can use it
**Members only — but free for every member.** Like IRC and News, file storage is
gated on membership, *not* on the paid Founding Lifetime plan:
- **Non-members can't connect.** A key that isn't a registered account is
refused at the SFTP handshake (`this key isn't a member — register first`), and
guests don't see the in-hub Files browser.
- **Every verified member can connect, run, and join** — free and paid alike.
There is no Premium gate anywhere in the Files path; the plan only affects
unrelated perks (custom email, domains, Tor).
- Operators can revoke an individual account's SFTP access (abuse response)
without touching its BBS login — see the management TUI below.
## Two areas
When you connect you see a virtual root with two directories:

View file

@ -1,16 +1,22 @@
# Mail — self-hosted Mailu at `mail.profullstack.com`
# Mail — self-hosted Mailu
AgentBBS gives **Founding Lifetime (paid) members** a real mailbox at
`<name>@mail.profullstack.com`, reached two ways:
AgentBBS gives **every verified member** (free and paid alike) a real mailbox at
`<name>@bbs.profullstack.com`, reached two ways:
- **Webmail**`https://mail.profullstack.com` (Roundcube), the only
member-facing mail surface.
- **Webmail**`https://mail.profullstack.com` (Roundcube).
- **AgentMail** — the in-BBS client (`internal/mailbox`): the `Mail` hub entry
or `ssh mail@bbs.profullstack.com` (a TUI for humans, a JSON bot mode for
agents). It connects to this stack.
Two distinct names are involved — don't conflate them:
| | value | role |
|---|---|---|
| **Address domain** | `bbs.profullstack.com` | the `@`-part of member addresses (`AGENTBBS_MAIL_ADDR_DOMAIN`) |
| **Mail server host** | `mail.profullstack.com` | where IMAP/SMTP/webmail actually run (`AGENTBBS_MAIL_DOMAIN`) |
The apex `profullstack.com` is **reserved for corporate mail** and is not served
here — member mail lives only on the `mail.` subdomain.
here.
## Architecture
@ -19,97 +25,139 @@ Mailu (Postfix + Dovecot + Roundcube + rspamd) runs as a Docker Compose stack:
- Mailu owns the **mail ports** on the host: `25, 465, 587, 993, 995`.
- Mailu's HTTP front is bound to **loopback** (`127.0.0.1:8080`); **Caddy**
reverse-proxies `https://mail.profullstack.com` to it (webmail + admin).
reverse-proxies `https://mail.profullstack.com` to it (webmail + admin + API).
- **TLS:** `TLS_FLAVOR=mail` — Mailu does *not* run its own ACME (Caddy is the
only ACME client). Caddy obtains the `mail.profullstack.com` cert from its site
block; [`deploy/mailu/refresh-certs.sh`](../deploy/mailu/refresh-certs.sh)
copies it into Mailu and reloads it on renewal — the same pattern as the
Ergo/IRC and NNTP cert refreshers.
only ACME client). Caddy obtains the `mail.profullstack.com` cert; the cert
refresher copies it into Mailu and reloads on renewal.
- The **agentbbs gateway** reads/sends on behalf of members: IMAP via a Dovecot
**master user** (one secret opens any mailbox), SMTP via the co-located relay
on `127.0.0.1:25`. Members therefore never manage an IMAP/SMTP password.
- **Provisioning** is automatic: when a member verifies their email at `join@`
(or opens `Mail`), agentbbs ensures `<name>@bbs.profullstack.com` exists via
Mailu's **admin REST API** (`internal/mailu`, token = `API_TOKEN`). The manual
`deploy/mailu/provision-mailbox.sh` is only for the gateway master user and
backfills.
```
┌─────────── Caddy (:443) ───────────┐
webmail → │ mail.profullstack.com → 127.0.0.1:8080 (Mailu front, HTTP)
webmail → │ mail.profullstack.com → 127.0.0.1:8080 (Mailu front: webmail/admin/API)
└───────────────┬─────────────────────┘
│ copies LE cert (refresh-certs.sh)
clients → Mailu front (:25 :465 :587 :993 :995) ──→ Postfix / Dovecot / rspamd
agentbbs ──IMAP 993 (master user)──┘ ──SMTP 127.0.0.1:25 (local relay)──▶
agentbbs ──admin API (token) http://127.0.0.1:8080/api/v1──▶ (auto-provision)
```
## DNS
`mail.profullstack.com` and `smtp.profullstack.com` A records are added. Also set:
Mail is delivered to the **address domain** (`bbs.profullstack.com`), so its MX
must point at the **server host** (`mail.profullstack.com`):
| Type | Host | Value |
|---|---|---|
| A | `mail.profullstack.com` | host IP |
| A | `smtp.profullstack.com` | host IP |
| MX | `mail.profullstack.com` | `10 mail.profullstack.com.` |
| TXT (SPF) | `mail.profullstack.com` | `v=spf1 mx -all` |
| TXT (DMARC) | `_dmarc.mail.profullstack.com` | `v=DMARC1; p=quarantine; rua=mailto:postmaster@mail.profullstack.com` |
| TXT (DKIM) | `dkim._domainkey.mail.profullstack.com` | from `flask mailu config-export` after first boot |
| MX | `bbs.profullstack.com` | `10 mail.profullstack.com.` |
| TXT (SPF) | `bbs.profullstack.com` | `v=spf1 mx -all` |
| TXT (DMARC) | `_dmarc.bbs.profullstack.com` | `v=DMARC1; p=quarantine; rua=mailto:postmaster@bbs.profullstack.com` |
| TXT (DKIM) | `dkim._domainkey.bbs.profullstack.com` | from `flask mailu config-export` after first boot |
| PTR | host IP | `mail.profullstack.com` (set at your VPS provider) |
> **Port 25 / deliverability:** many cloud providers block outbound `:25` by
> default — request an unblock, set the PTR/rDNS, and warm the IP, or relay
> outbound through a smarthost. Inbound MX and the gateway's local submission
> work regardless.
> **Port 25 / deliverability:** many cloud providers (incl. DigitalOcean) block
> outbound `:25` by default — request an unblock, set the PTR/rDNS, and warm the
> IP, or relay outbound through a smarthost. Inbound MX and the gateway's local
> submission work regardless.
## Install
```bash
cd /opt/agentbbs/deploy/mailu
cp mailu.env.example mailu.env # fill SECRET_KEY, INITIAL_ADMIN_PW, etc.
cp mailu.env.example mailu.env # fill SECRET_KEY, INITIAL_ADMIN_PW, API_TOKEN, DOMAIN=bbs.profullstack.com, HOSTNAMES=mail.profullstack.com
docker compose up -d
# seed the gateway master user + (optionally) backfill member mailboxes:
# add the address domain + the gateway master user:
docker compose exec admin flask mailu domain bbs.profullstack.com
AGENTBBS_MAIL_MASTER_USER=gateway ./provision-mailbox.sh --master "$(openssl rand -hex 16)"
```
Add the Caddy site (setup.sh writes this when `MAIL=1`):
```
mail.profullstack.com {
encode zstd gzip
reverse_proxy 127.0.0.1:8080
}
```
Then install the cert refresher on a timer (setup.sh does this too):
```bash
install -m 0755 deploy/mailu/refresh-certs.sh /usr/local/bin/agentbbs-mailu-certs
# systemd timer runs it every ~12h; first run swaps in the real cert once Caddy issues it.
```
setup.sh writes the Caddy `mail.profullstack.com` site and the cert-refresh
timer when `MAIL=1`, and brings the stack up once `mailu.env` exists.
## agentbbs gateway env
Set these on the agentbbs service so the `Mail` hub entry / `ssh mail@` work:
Set these on the agentbbs service (setup.sh §9e upserts the non-secret ones):
| Var | Value |
|---|---|
| `AGENTBBS_MAIL_ADDR_DOMAIN` | `bbs.profullstack.com` |
| `AGENTBBS_MAIL_DOMAIN` | `mail.profullstack.com` |
| `AGENTBBS_MAIL_IMAP_ADDR` | `mail.profullstack.com:993` |
| `AGENTBBS_MAIL_IMAP_ADDR` | `127.0.0.1:14143` (Dovecot direct, loopback) |
| `AGENTBBS_MAIL_IMAP_PLAINTEXT` | `1` (the loopback path is plaintext) |
| `AGENTBBS_MAIL_SMTP_ADDR` | `127.0.0.1:25` |
| `AGENTBBS_MAIL_ADMIN_URL` | `http://127.0.0.1:8080` |
| `AGENTBBS_MAIL_API_TOKEN` | the Mailu `API_TOKEN` (secret) |
| `AGENTBBS_MAIL_MASTER_USER` | `gateway` |
| `AGENTBBS_MAIL_MASTER_PASS` | the master password set above |
| `AGENTBBS_MAIL_MASTER_PASS` | the master password set above (secret) |
| `AGENTBBS_WEBMAIL_URL` | `https://mail.profullstack.com` (default = mail host) |
Without `AGENTBBS_MAIL_API_TOKEN` auto-provisioning is skipped (the address is
still shown); without `AGENTBBS_MAIL_MASTER_PASS` the gateway can't open
mailboxes.
### Why the gateway talks to Dovecot directly (plaintext loopback)
Mailu's **front** (nginx mail proxy) pre-authenticates every IMAP/SMTP login
against Mailu's user DB before proxying to Dovecot — and it rejects the Dovecot
master-user login form `<addr>*gateway`. So the gateway must reach **Dovecot
directly**, bypassing the front. The `imap` container has no TLS cert (only the
front does), so the bypass is plaintext over loopback — safe because the
connection (and the master password) never leave the host. Wiring:
- Publish Dovecot's IMAP on loopback (docker-compose.override.yml):
`imap.ports: ["127.0.0.1:14143:143"]`.
- The Dovecot master user is defined in `data/overrides/dovecot/dovecot.conf`
(Mailu includes exactly that filename — *not* `*.conf`):
```
auth_master_user_separator = *
passdb { driver = passwd-file; master = yes; args = /overrides/master-users }
```
with `data/overrides/dovecot/master-users` holding `gateway:{SHA512-CRYPT}$6$…`
(the hash of `AGENTBBS_MAIL_MASTER_PASS`). The file must be **world-readable
(644)** — Dovecot reads it as a non-root user, and 640 root:root yields a
`temp_fail`. Do **not** add `result_success = continue` (that would also
require the target user's own password); the target mailbox comes from userdb.
- Point the gateway at it: `AGENTBBS_MAIL_IMAP_ADDR=127.0.0.1:14143` +
`AGENTBBS_MAIL_IMAP_PLAINTEXT=1`.
## Sending mail from the BBS (verify codes + notifications)
The join@ verification code and signup notifications use `internal/mail` (the
`AGENTBBS_SMTP_*` knobs), separate from the per-member mailbox client. Point
them at the local Mailu relay so codes actually send:
```
AGENTBBS_SMTP_HOST=127.0.0.1
AGENTBBS_SMTP_PORT=25
AGENTBBS_SMTP_FROM=bbs@bbs.profullstack.com
# user/pass omitted: the co-located relay accepts local submission unauthenticated
```
## Provisioning member mailboxes
A mailbox must exist before the gateway can open it. Provision when a member
becomes paid (or backfill):
Provisioning is automatic at `join@` verification. To create or backfill by hand:
```bash
deploy/mailu/provision-mailbox.sh alice # creates alice@mail.profullstack.com
deploy/mailu/provision-mailbox.sh alice # creates alice@bbs.profullstack.com
```
The Dovecot **master user** (`gateway`) then authenticates as any member with
the login form `alice*gateway` + the master password — which is exactly what
(Set `MAIL_DOMAIN=bbs.profullstack.com` for the script, since the address domain
differs from the server host.)
The Dovecot **master user** (`gateway`) authenticates as any member with the
login form `alice*gateway` + the master password — exactly what
`internal/mailbox`'s IMAP adapter sends. See
[`deploy/mailu/README.md`](../deploy/mailu/README.md) for the master-user
override and operational details.
[`deploy/mailu/README.md`](../deploy/mailu/README.md) for details.
## Webmail only for members