logicsrc/docs/openontology-interoperability.md
Anthony Ettinger da5f6f8381
Some checks are pending
CI / build (push) Waiting to run
test / test (push) Waiting to run
feat(openontology): Phase 2 + Phase 3 — storage, REST/SSE, MCP, RDF/SHACL, adapters, TUI, explorer (#101)
Everything the two shipped PRD phases deferred, minus what is called out below.

Storage (Phase 2)
  @logicsrc/openontology gains a SQLite/Turso adapter. It hydrates the read
  model at open, serves reads synchronously — a query evaluator that awaits per
  triple pattern is unusable — and buffers mutations as SQL that flush() writes
  in one transaction. Versioned idempotent migrations; indexes over subject,
  predicate, entity-valued object, status, both time axes, aliases, and external
  ids; FTS5 for label/alias search. The append-only status log is replayed on
  open, so retractions, supersessions, and merge redirects survive a reopen.

REST + SSE + OpenAPI (Phase 2)
  16 paths under /api/ontologies in logicsrc-web, described at
  /api/ontologies/openapi and referencing the published JSON Schemas rather
  than restating them. No token is read-only; a curator token can apply; an
  agent token can propose and cannot apply. Idempotency-Key on mutations,
  revision ETags, 409 on a stale base revision, and an SSE stream that emits
  the same event objects as the JSON endpoint.

MCP (Phase 2)
  OpenOntology and OpenPRD surfaces on the standards server: spec/manifest/
  schema/queries and PRD spec/index as resources, 11 ontology tools and 6 PRD
  tools, 7 prompts. Read-only by default; OPENONTOLOGY_MCP_WRITABLE=1 buys
  proposals, never applies — the denial is the shared policy layer, not a
  second rule that could drift.

Interoperability (Phase 3)
  RDF/Turtle export and import of the reified profile, plus the plain triple
  for asserted relationships so a consumer wanting only the accepted graph gets
  one. SHACL for 5 of 7 constraint kinds; `unique` and `query` are reported as
  unmapped in both the return value and the generated Turtle, because a shape
  that quietly means something narrower is worse than no shape.

Source adapters (Phase 3)
  CSV, JSON, YAML, NDJSON, Markdown, generic JSON HTTP, and GitHub. All produce
  PROPOSED change-set operations with source, evidence selector, run id, and
  confidence attached; fetch is injected so ingestion is offline and testable.
  Each declares its capabilities, so "nothing was deleted upstream" is never
  confused with "this adapter cannot see deletions" — none of the seven can.

TUI + explorer
  Keyboard-first panels (types, entities, claims, sources, queries, change
  sets, validation, audit) as plain strings that survive SSH and 60 columns;
  status is a glyph and a word, never colour alone; the key bar wraps rather
  than truncating. Wired as `logicsrc ontology tui`. A read-only web explorer
  at /openontology/explore with entity and claim views showing status, both
  clocks, confidence, sources, evidence, and append-only history — plus an
  /openprd page for the companion standard.

Bugs found and fixed while testing
  - the API built a new engine per request, so `explain` could never find a
    resultId from a prior request; engines are now cached per role
  - the TUI status bar called engine.validateOntologyPackage(), appending a
    package.validated event on every repaint; it now uses the pure validator

Verification: 76 new tests (527 total across the monorepo, all passing); full
build green; the libSQL adapter is exercised against real files, the API
through its route handlers, and MCP over an in-memory transport.

Not included: PWA review/approval write flows (they need an auth story this
deployment does not have), OWL/RDFS mappings, SPARQL/Cypher/Datalog query
adapters, and Phase 4 governed actions. The compatibility matrix marks those
"planned", not "supported".

Refs: prd/0001-add-logicsrc-openontology-spec.md

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 05:10:33 -07:00

7.8 KiB

OpenOntology interoperability

How OpenOntology relates to JSON Schema, JSON-LD, RDF, SHACL, PROV-O, and external identifier systems — and, just as importantly, where the mappings stop. Companion to OpenOntology.

The guiding rule: report what a format cannot carry; never drop it silently.

Where each format sits

Format Role
JSON Schema 2020-12 Canonical, normative contract. The schemas are the standard.
Canonical JSON Deterministic bytes for hashing, signing, diffing, publishing.
YAML Human authoring convenience. Compiles to canonical JSON.
NDJSON Streaming format for large entity/claim/source files.
JSON-LD 1.1 Interoperability profile for the semantic-web world.
RDF / Turtle Planned export of the losslessly mappable subset.
SHACL Planned mapping for the constraint subset with equivalent semantics.
PROV-O Vocabulary reused for provenance where the semantics genuinely match.

OpenOntology does not require RDF, OWL, SPARQL, or a triple store. It maps to them so that consumers who need formal reasoning can get there.

JSON-LD

logicsrc ontology export --dir ./ethereum-ecosystem --format jsonld --out graph.jsonld

Entities become nodes. Claims are reified — each claim is its own node with subject, predicate, object, status, time, confidence, and provenance — because the provenance is the point. A bare triple cannot say "asserted by this agent, from this commit, valid since April, confidence 0.94."

Provenance terms alias PROV-O rather than inventing parallel vocabulary:

OpenOntology JSON-LD term
assertedAt prov:generatedAtTime
assertedBy prov:wasAttributedTo
sources prov:wasDerivedFrom
runId prov:wasGeneratedBy
confidence oo:confidence (xsd:double)
validTime.from / .to oo:validFrom / oo:validTo
status oo:status

Compact ids canonicalize to IRIs against the package namespace; the Namespace object binds the prefix, which is what makes the reverse direction unambiguous. The round trip JSON → JSON-LD → JSON preserves ids, types, subjects, predicates, objects, typed values, language tags, statuses, times, confidence, sources, evidence, and supersession links.

Lossy fields

The 0.1 JSON-LD profile does not carry: tags, license, visibility, retention, changeSet, model, firstParty, derivedFrom, retractionReason, and extensions. Export reports them per object:

warning: 3 object(s) have fields this format cannot carry:
  eth:claim:0042: tags, license

The same holds in the other direction: importing a foreign vocabulary that expresses semantics the core model lacks reports the gap instead of quietly discarding it.

External identifiers

An entity carries namespaced external ids without treating any external service as the identity authority:

externalIds:
  github: averyl
  wikidata: Q000000
  orcid: 0000-0000-0000-0000
  did: "did:example:abc"

Lookup works by exact id, canonical name, alias, or external id. findEntities returns ranked candidates with the evidence for each match — never a silent single answer.

sameAs is a reviewable claim, not an implicit merge. Two records only become one through an approved merge-entity operation, and the losing id survives as a redirect.

RDF and Turtle

logicsrc ontology export --dir ./ethereum-ecosystem --format turtle --out graph.ttl

Claims are reified, exactly as in JSON-LD, and an asserted relationship claim additionally emits the plain triple — so a consumer that only wants the current accepted graph gets one without unpacking provenance.

Import parses the profile this exporter produces rather than pretending to be a general Turtle parser. Anything it cannot interpret is listed in unsupported, never dropped silently.

SHACL

logicsrc ontology export --dir ./ethereum-ecosystem --format shacl --out shapes.ttl

Five constraint kinds map onto SHACL Core: required-predicate, cardinality, allowed-values, domain-range, and temporal-bounds. Severity carries across (errorsh:Violation, warningsh:Warning).

Two do not, and are reported as unmapped in the returned value and as comments in the generated Turtle:

  • unique — graph-wide uniqueness has no portable SHACL Core equivalent; it needs a sh:SPARQLConstraint.
  • query — an OpenOntology saved query is a triple-pattern AST, not SPARQL.

A shape that silently means something narrower than the constraint it came from is worse than no shape, so those stay unmapped until the mapping is real.

OWL/RDFS

Still planned. An optional mapping for consumers needing formal reasoning. OpenOntology itself infers nothing: transitivity, symmetry, and inverses apply only when the schema declares them and a query asks.

Compatibility matrix

Target 0.1 status Notes
JSON Schema 2020-12 supported Canonical contract; 16 object kinds
Canonical JSON + digest supported Deterministic across Node.js and Bun
YAML authoring supported Same digest as equivalent JSON
NDJSON supported Streaming entity/claim/source/evidence files
JSON-LD 1.1 export supported Reified claims, PROV-O aliases, lossy report
JSON-LD 1.1 import supported Round-trips the reference profile
PROV-O partial Provenance terms aliased in JSON-LD and Turtle; full mapping later
RDF / Turtle export supported Reified claims + plain triples for asserted relationships
RDF / Turtle import supported Round-trips the reference profile; reports what it cannot read
SHACL partial 5 of 7 constraint kinds; unique and query reported as unmapped
OWL / RDFS planned Optional, for external reasoners
SPARQL planned Query AST → SPARQL adapter
Cypher planned Query AST → Cypher adapter
Datalog planned Query AST → Datalog adapter
SQLite / Turso supported createLibsqlStore, versioned migrations, FTS5 entity search
REST + OpenAPI supported 16 paths, described at /api/ontologies/openapi
Server-Sent Events supported Same event objects as the JSON endpoint
MCP supported Resources, tools, and prompts; writes propose, never apply
Neo4j / vector DBs not required Optional adapters; never mandatory

Query portability

The triple-pattern AST is deliberately small so a second implementation is achievable. Adapters translate it to SQL, SPARQL, Cypher, or Datalog and must report their capabilities — an adapter that cannot do multi-hop traversal or asOf says so rather than returning a subtly wrong answer.

Aggregation, grouping, faceting, and path-finding are P1: useful, but not in the 0.1 evaluator, so that the portable core stays implementable.

Embeddings

Optional, rebuildable, provider-neutral derived data. They may improve discovery; they are never the sole representation of a fact and never authoritative. A package with its embeddings deleted loses nothing canonical.

Importing

Imports propose; they never apply:

logicsrc ontology import --file graph.jsonld --dir ./my-ecosystem
{ "entities": 63, "claims": 169, "proposedOperations": 12 }

The result is a set of operations for a change set. Source adapters must declare whether they can read public data, private data, incremental changes, and deletions, and what licence the source carries. An ingestion run is repeatable from its declared sources, mappings, parser version, and model configuration.

Deduplication runs on stable id, external id, exact alias, normalized URL, and reviewed similarity candidates — with the last of those always going to a human.