# ipaudio: Audio on ipfile Swarms Version: **0.1** (draft) Status: draft. Extends the OpenSwarm core ([spec.md](./spec.md)) and rides on [ipfile](./ipfile.md), [ippay](./ippay.md) and [ipdb](./ipdb.md). Nothing below is implemented. ## 1. Scope `ipaudio` defines the records that make an `ipfile` playable as music or speech: 1. A **track**: renditions (each one an `ipfile`), a seek index, gapless trim values, contributors and royalty split. 2. A **release**: an ordered set of tracks with artwork and release metadata. 3. **Streaming rules**: how a player maps time to pieces, buffers, switches bitrate and plays two tracks without a gap. 4. A **gateway bridge** for players that only speak HTTP. Everything about bytes, keys, payment and discovery is `ipfile` and `ippay`. This document adds no wire messages. ### Non-goals A player. Loudness normalisation. A recommendation system. DRM beyond `ipfile` key delivery. Radio (a continuous stream is `iplive`). ## 2. Terminology | Term | Meaning | | --- | --- | | **rendition** | One encoding of a track, stored as one `ipfile`. | | **anchor** | A `[timeMs, byteOffset]` pair at which decoding can start. | | **seek index** | The list of anchors for a rendition. | | **gapless trim** | Encoder delay and end padding in samples, so tracks join without silence. | ## 3. Data model ### 3.1 Track ```json { "openswarm": "0.1", "type": "ipaudio.track", "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", "title": "Interview with Ada", "artist": "Ada", "artists": [{ "name": "Ada", "role": "primary" }], "durationMs": 2941200, "isrc": null, "language": "en", "artwork": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", "renditions": [ { "id": "opus-96", "codec": "opus", "container": "ogg", "bitrateKbps": 96, "sampleRate": 48000, "channels": 2, "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d", "size": 35294400, "gapless": { "encoderDelay": 312, "endPadding": 1216 }, "index": [[0, 0], [2000, 24012], [4000, 48160], [6000, 72301]] }, { "id": "flac", "codec": "flac", "container": "flac", "bitrateKbps": 0, "sampleRate": 48000, "channels": 2, "file": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be", "size": 734003200, "gapless": { "encoderDelay": 0, "endPadding": 0 }, "index": [[0, 8192], [2000, 507904], [4000, 1007616], [6000, 1503232]] } ], "royalties": [ { "label": "Ada", "network": "eip155:8453", "address": "0xCC3b072391AE7A8d10cF00DdC5F61DB2cA5541E5", "bps": 8000 }, { "label": "Producer", "network": "eip155:8453", "address": "0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf", "bps": 2000 } ], "createdAt": "2026-09-05T18:05:00.000Z", "sigs": [ { "alg": "ed25519", "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" } ] } ``` (`index` is truncated to four anchors here. A real one has one anchor per two seconds of audio.) | Field | Rule | | --- | --- | | `publisher` | Signs the record. MUST equal the `publisher` of every referenced `ipfile` manifest. | | `durationMs` | Of the plaintext audio. | | `artwork` | File key of an `ipfile` holding a JPEG or PNG. MAY be null. | | `renditions[].codec` | One of `opus`, `aac`, `flac`, `mp3`. | | `renditions[].container` | `ogg` for Opus, `mp4` for AAC (fragmented, CMAF), `flac`, `mp3`. | | `renditions[].bitrateKbps` | Target bitrate; `0` for lossless. | | `renditions[].file` | The `ipfile` manifest's file key. | | `renditions[].size` | MUST equal that manifest's `size`. | | `renditions[].gapless` | Samples to trim at the start and end, as the encoder reported. `0` for FLAC. | | `renditions[].index` | Anchors, ascending by time, first at `[0, o]` where `o` is the byte offset of the first decodable unit after headers. | | `royalties` | How the publisher's share of every voucher is split (§5). | A track MUST have at least one rendition. Every rendition of a track MUST have anchors at the same set of `timeMs` values (§4.3). ### 3.2 Release ```json { "openswarm": "0.1", "type": "ipaudio.release", "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", "title": "Conversations, Volume 1", "artist": "Ada", "kind": "podcast", "releasedAt": "2026-09-05", "upc": null, "artwork": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94", "tracks": [ { "position": 1, "subject": "sha256:d6c3f8285b7871d6a400cba14408288a9acde679f12e1e7dc276f29ca7c493ff", "feed": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e/default" } ], "description": "Long-form interviews.", "createdAt": "2026-09-05T18:06:00.000Z", "sigs": [ { "alg": "ed25519", "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e", "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w==" } ] } ``` `kind` is one of `album`, `ep`, `single`, `podcast`, `audiobook`, `compilation`. `tracks[].subject` is the record id of the track's first revision, which is also its `ipdb` key (`track:`), so a track can be revised without breaking the release. ### 3.3 Where records live Both records are `ipdb` entries in the publisher's feed under `track:` and `release:`. The renditions are `ipfile` manifests under `file:` in the same feed. A client that finds a release has everything it needs in one feed. ## 4. Streaming ### 4.1 Time to pieces To play from time `t` on rendition `r`: ``` anchor = the last index entry with timeMs <= t offset = anchor.byteOffset piece = floor(offset / manifest.pieceLength) ``` Request pieces from `piece` onward in order. Decrypt each as it completes (`ipfile` §4.1), verify against the plaintext piece layer, and feed the decoder from `offset`. The first decodable unit is guaranteed at the anchor because the index was built from container boundaries (Ogg page, MP4 fragment, MP3 frame, FLAC frame). ### 4.2 Buffering A player SHOULD hold the pieces covering the next 30 seconds and request the next piece when the buffer drops below 15 seconds. It SHOULD send vouchers on the `ipfile` schedule regardless of playback state; a paused player that has received bytes still owes for them. With 256 KiB pieces and a 96 kbps rendition a piece is about 21 seconds of audio. That is why `ipaudio` recommends 256 KiB: one piece is one buffer step. ### 4.3 Adaptive bitrate Because every rendition shares the same anchor times, a switch is: finish the current anchor interval on rendition A, start the next interval on rendition B from its anchor at the same `timeMs`. The publisher guarantees alignment by encoding every rendition with the same segment duration (2000 ms recommended). A player picks the highest `bitrateKbps` whose piece download rate over the last 10 seconds exceeds 1.5 times the bitrate. Switching renditions means joining a second swarm. The pass is per file, so a pass scoped by `publishers` rather than `files` is what a player wants (`ippay` §3.1). ### 4.4 Gapless When track A ends and track B begins, the player trims `endPadding` samples from A's decoded tail and `encoderDelay` samples from B's decoded head, and concatenates the sample streams. It MUST have B's first piece decrypted before A's last anchor interval starts, which means it holds a grant for B already. A release player therefore requests the grant for track `n + 1` when track `n` starts. ## 5. Royalties An `ipfile` manifest names one `pricing.payTo` for the publisher's share. `ipaudio` splits that share further with `royalties`: a list of recipients and basis points summing to 10000. A hub applies the split when it can see the track record for a file; the `ipfile` manifest's `meta.track` field carries the track's record id so the hub can look it up in `ipdb`. Where the hub cannot find the track, the whole publisher share goes to `payTo` and a `com.logicsrc.openswarm.ippay.split.unresolved.v1` event is emitted. A contributor is paid per byte served of any rendition of the track. That is the whole royalty model: no plays counted, no per-stream rate, no minimum. It is a consequence of `ippay`, not a separate system. ## 6. Publishing procedure `ip audio publish ` where the input names source files: 1. For each rendition requested (`--renditions opus-96,flac`), transcode with ffmpeg or submit an `ipaudio.transcode` job to c0mpute (`c0mpute.md` §3). Record `encoderDelay` and `endPadding` from the encoder. 2. Build the seek index by walking container boundaries at 2000 ms steps. 3. `ip file add` each rendition with `--piece-length 262144`, `--meta.track `; collect file keys. 4. Write the track record, sign, `ip db put track:`. Revise each rendition manifest's `meta.track` to the track id (metadata-only revision). 5. Optionally write or update a release. ## 7. Gateway bridge A gateway that is also a keeper (holds the content key under a delegating grant) MAY serve decrypted audio to pass holders: ``` GET /audio// Authorization: Bearer Range: bytes=0- ``` It returns `206` with the plaintext bytes, `Content-Type` from the container, `Accept-Ranges: bytes`, and `X-OpenSwarm-Credit` per `ippay` §7. This is what lets a podcast app, a car, or bittorrented.com's existing `` (which plays a plain URL) consume an `ipaudio` track. The gateway is the payee for those bytes and is paid by voucher like any seeder; the client is the pass holder's HTTP library, which sends vouchers in `X-OpenSwarm-Voucher`. A gateway MAY also render an RSS 2.0 feed for a `podcast` release with `` URLs of the form above, so a release is subscribable in any podcast client that can send a bearer header, and previewable in any client at all if the publisher sets `perGib` and `keyUsd` to zero. ## 8. Recommended encodings | Rendition id | Codec | Container | Settings | | --- | --- | --- | --- | | `opus-64` | Opus | Ogg | 64 kbps VBR, 48 kHz, speech | | `opus-96` | Opus | Ogg | 96 kbps VBR, 48 kHz | | `opus-160` | Opus | Ogg | 160 kbps VBR, 48 kHz, music | | `aac-128` | AAC-LC | fragmented MP4 | 128 kbps, for Safari and CarPlay | | `flac` | FLAC | FLAC | Lossless, source sample rate | Ogg Opus and FLAC carry their own seek structure; the index is still REQUIRED so a player never has to read the container to find a piece. ## 9. Security and privacy Nothing beyond `ipfile`. Two notes: a rendition list reveals bitrates and sizes to anyone who has the track record, and a seek index reveals the container's page structure; neither reveals audio. A publisher who does not want the catalogue public uses a paid `ipdb` segment (`ipdb` §9). ## 10. Events | Type | Emitted by | | --- | --- | | `com.logicsrc.openswarm.ipaudio.track.published.v1` | publisher | | `com.logicsrc.openswarm.ipaudio.release.published.v1` | publisher | Play counts are not events in this family. A client MAY report them to a publisher's webhook as `com.logicsrc.openswarm.ipaudio.play.v1` with `{ "track", "rendition", "ms" }`; nothing in the payment loop depends on it. ## 11. Implementations | Piece | Exists | Where | What is new | | --- | --- | --- | --- | | Plain-URL audio player, range streaming, HLS fallback | Yes | `media-streamer` ``, `/api/stream`, `/api/stream/hls` | The gateway bridge is what it would point at | | Podcast catalogue and RSS | Yes | `p0dcasters.com`, `media-streamer` podcasts lib | Rendering RSS from a release record | | Per-rendition transcode on the network | Yes (video) | `c0mpute` `ffmpeg.transcode` | An audio preset with gapless values in the result | | Track and release records, seek index, gapless join, royalty split | No | | All of it | ## 12. Conformance An `ipaudio` player maps time to pieces per §4.1, switches only at shared anchors per §4.3, trims per §4.4, and holds a valid pass for every rendition it fetches. A publisher emits records per §3 with aligned anchors. A gateway implementing §7 applies `ippay` §7 headers. ## 13. Version history | Version | Date | Change | | --- | --- | --- | | 0.1 | 2026-09-05 | Initial draft. |