logicsrc/docs/openswarm/ipvideo.md

ipvideo: Video on Demand on ipfile Swarms

Version: 0.1 (draft) Status: draft. Extends the OpenSwarm core (spec.md) and rides on ipfile, ippay and ipdb. Nothing below is implemented.

1. Scope

ipvideo defines the records that make a set of ipfile swarms playable as a film, an episode or a clip:

1. A title: renditions (CMAF fragmented MP4, one ipfile each), a segment index, subtitles, thumbnails, chapters, artwork.
2. Playback rules: segment-aligned adaptive bitrate over pieces, MSE in the browser.
3. A gateway bridge that renders HLS for players that speak nothing else.
4. Transcode on c0mpute: how the renditions get made.

Bytes, keys, payment and discovery are ipfile and ippay. No new wire messages.

Non-goals

Live (that is iplive). Interactive video. Ad insertion (c0mpute's DAI DIPs may reference a title; this document does not). DRM in the device sense.

2. Terminology

Term	Meaning
rendition	One encoding ladder step, stored as one ipfile in CMAF form.
segment	A fragmented MP4 moof + mdat pair, starting with a keyframe.
init segment	The ftyp + moov bytes a decoder needs before any segment.
segment index	The list of { t, d, o, l } for every segment of a rendition.
ladder	The set of renditions of a title, aligned on segment boundaries.

3. Data model

3.1 Title

{
  "openswarm": "0.1",
  "type": "ipvideo.title",
  "publisher": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e",
  "title": "The Interview",
  "kind": "movie",
  "series": null,
  "season": null,
  "episode": null,
  "year": 2026,
  "durationMs": 5400000,
  "language": "en",
  "artwork": {
    "poster": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94",
    "backdrop": null
  },
  "renditions": [
    {
      "id": "1080p",
      "container": "cmaf",
      "codecs": "avc1.640028,mp4a.40.2",
      "width": 1920,
      "height": 1080,
      "frameRate": 24,
      "bitrateKbps": 5000,
      "file": "ed25519:0d87e09c7fea3ad6ba6c2f3e027ea47f5b245452899910948470906704c5295d",
      "size": 3375000000,
      "init": { "o": 0, "l": 1524 },
      "segmentsFile": "ed25519:00bc9c01ca00938016e276fe5b5748cfe13fd02bc53cc651535682c8dd5c83be"
    },
    {
      "id": "480p",
      "container": "cmaf",
      "codecs": "avc1.64001e,mp4a.40.2",
      "width": 854,
      "height": 480,
      "frameRate": 24,
      "bitrateKbps": 1400,
      "file": "ed25519:d2d05fcad07ecbee6ff87c95159ec9969fce6c0fb9f41ff78c2c07ebe8a06c94",
      "size": 945000000,
      "init": { "o": 0, "l": 1498 },
      "segments": [
        { "t": 0, "d": 4000, "o": 1498, "l": 702113 },
        { "t": 4000, "d": 4000, "o": 703611, "l": 698402 }
      ]
    }
  ],
  "audio": [
    { "id": "en", "language": "en", "label": "English", "channels": 2, "default": true }
  ],
  "subtitles": [
    { "id": "en-cc", "language": "en", "kind": "captions", "format": "vtt", "file": "ed25519:c9f17d4eaf2b28122b46d111cef6697d2c3f708a19d4628bce5c0304481b355b" }
  ],
  "thumbnails": {
    "file": "ed25519:4b74eb43677e4d03af5fb0856333f9aa21d9a5a3bbf944b13aa7eef379c7a342",
    "vtt": "ed25519:ba8cd1a97b865a05cdc5840c63b086dd463624a831874c1ba0d7c4ce6f806f7a",
    "intervalMs": 10000
  },
  "chapters": [
    { "t": 0, "title": "Opening" },
    { "t": 1830000, "title": "The question" }
  ],
  "royalties": [],
  "createdAt": "2026-09-05T18:20:00.000Z",
  "sigs": [
    {
      "alg": "ed25519",
      "key": "ed25519:5d292428e8a68946e5225136c8b10e8f33ab78a45e1663d0730996dd2b63d59e",
      "sig": "RGRH7pDpXYcagULqj+8z05pLH99Lc6yLmfdbYgtGeRpjX5l0NcT3QSq9AWBtLw+5+JJiKoZ3s4ty74q/zBSE8w=="
    }
  ]
}

(The 480p segment list is truncated to two entries; a real one has one entry per segment.)

Field	Rule
kind	movie, episode, clip, trailer. series, season, episode are set for episode.
renditions[].container	cmaf in this version: fragmented MP4 per ISO 23000-19, one segment per moof.
renditions[].codecs	RFC 6381 string as MSE isTypeSupported wants it.
renditions[].file	The ipfile manifest's file key. size MUST match.
renditions[].init	Byte range of the init segment within the plaintext.
renditions[].segments	Inline index, or segmentsFile: file key of a free ipfile (keys.public set) whose plaintext is the JSON array. Inline when under 64 KiB, file otherwise.
segments[].t, d	Presentation start and duration in ms. o, l: byte offset and length in the plaintext.
audio	Audio tracks muxed in the renditions. Alternate-language audio is a rendition with codecs of audio only and video: false.
subtitles[].format	vtt in this version. Each is a free ipfile unless the publisher prices it.
thumbnails	A sprite sheet ipfile and a WebVTT file mapping times to sprite regions, as hls.js and video.js expect.
royalties	As ipaudio §5. Empty means the whole publisher share goes to payTo.

Every rendition MUST share the same t list. Every segment MUST start with a keyframe. Segment duration SHOULD be 4000 ms, matching c0mpute's live-stream DIP (-hls_time 4) so a recorded live stream and a VOD title use one ladder.

3.2 Where records live

ipdb under title:<id> in the publisher's feed. Renditions, subtitles and thumbnails are ipfile manifests in the same feed with meta.title = <title record id>.

4. Playback

4.1 Segment to pieces

To play segment s of rendition r:

first piece = floor(s.o / pieceLength)
last piece  = floor((s.o + s.l - 1) / pieceLength)

Request those pieces; decrypt; verify against the plaintext piece layer; slice [s.o, s.o + s.l); append to the MSE SourceBuffer after the init segment. With 1 MiB pieces and a 5 Mbps rendition a 4 s segment is about 2.5 MiB, so two to three pieces.

4.2 Adaptive bitrate

Switch at segment boundaries only. The next segment is fetched from the rendition whose bitrateKbps is the highest below 0.8 times the measured piece throughput over the last 10 seconds, with a two-segment hysteresis before stepping up. Because t lists are shared, the MSE buffer stays contiguous across a switch; a new init segment is appended before the first segment of the new rendition, as in any CMAF player.

4.3 Buffering and priority

Hold 30 seconds ahead. Request the pieces for the next segment with sequential priority (what media-streamer's torrent.select does for range streaming today). Seeking to time t: find the segment with the largest t <= target, drop outstanding requests, start there. A player MUST keep sending vouchers for bytes received while seeking, including bytes it then discards; discarded bytes were still served.

4.4 Browser

A browser peer is a WebTorrent client with the ipfile extension, a WebCrypto AES-CTR decryptor, and MSE. This is bittorrented.com's player with three additions and no server in the path. Where MSE is unavailable (some iOS contexts), the gateway bridge in §5 applies.

5. Gateway bridge

A keeper gateway renders HLS from the title record:

GET /video/<title id>/master.m3u8
GET /video/<title id>/<rendition id>/index.m3u8
GET /video/<title id>/<rendition id>/data        (plaintext, Range)
Authorization: Bearer <base64url pass>

The media playlist uses #EXT-X-MAP for the init range and #EXT-X-BYTERANGE for every segment against the single data URL, so the gateway serves plaintext by range from decrypted pieces and never has to split files. Subtitles become #EXT-X-MEDIA entries pointing at /file/<file key>/data. Credit and voucher headers are as ippay §7; a player that cannot send vouchers (a bare <video> tag) is limited to the pass's first credit window unless the gateway is configured to trust the pass for the title's full size.

HLS's own encryption (EXT-X-KEY with AES-128-CBC) is not used. The gateway is a keeper; it decrypts. A publisher who does not want any gateway holding a key does not delegate to one, and browser MSE is the only path.

6. Transcode on c0mpute

ip video publish <source> --ladder default submits one c0mpute job per rendition. The existing ffmpeg.transcode workload takes TranscodeSpec { codec, bitrate_bps, width, height, keyframe_interval, hardware_pref, extra_ffmpeg_args }; ipvideo adds extra_ffmpeg_args that force CMAF output and a 4 s keyframe interval, and expects the result to include the segment index. The proposed ipvideo.transcode workload (c0mpute.md §3) is that: a wrapper whose TranscodeResult carries segments, init, codecs and gapless-irrelevant fields, and whose output is handed to ip file add on the worker so the rendition is seeded from where it was made.

Default ladder:

id	codecs	size	bitrate
1080p	avc1.640028,mp4a.40.2	1920x1080	5000 kbps
720p	avc1.64001f,mp4a.40.2	1280x720	2800 kbps
480p	avc1.64001e,mp4a.40.2	854x480	1400 kbps
360p	avc1.64001e,mp4a.40.2	640x360	800 kbps
1080p-av1	av01.0.08M.08,opus	1920x1080	3000 kbps, optional

7. Security and privacy

As ipfile. A segment index reveals segment sizes, which for a constant-quality encode leak scene complexity over time; a publisher who cares uses constant bitrate. Subtitle files are small free swarms and reveal the dialogue to anyone; price them if that matters.

8. Events

Type	Emitted by
com.logicsrc.openswarm.ipvideo.title.published.v1	publisher
com.logicsrc.openswarm.ipvideo.transcode.completed.v1	c0mpute worker

9. Implementations

Piece	Exists	Where	What is new
Browser WebTorrent player, MSE-compatible fMP4 output, HLS fallback, range seeking	Yes	media-streamer (media-player-modal.tsx, transcoding.ts, /api/stream/hls)	Piece decryption before MSE; the title record instead of vod_titles
Hardware ffmpeg transcode job with codec ladder	Yes	c0mpute (c0mpute-transcode, TranscodeSpec)	CMAF output flags and the segment index in the result
VOD catalogue with per-title price and pass window	Yes (central)	media-streamer vod_titles, vod_providers	The title record and ipfile pricing replace both tables
Thumbnail VTT, chapters, subtitle tracks	Partly	hls.js and video.js conventions	The record fields
Title record, segment index file, HLS bridge from records	No		All of it

10. Conformance

A player fetches by segment per §4.1, switches at boundaries per §4.2, and vouchers for discarded bytes per §4.3. A publisher emits aligned ladders with keyframe-aligned segments. A gateway implementing §5 renders byte-range HLS and applies ippay §7.

11. Version history

Version	Date	Change
0.1	2026-09-05	Initial draft.