PersonX abandons ___ altogether \t xIntent \t to be selfish
PersonX abandons ___ altogether \t xEffect \t none
baseball bat \t ObjectUse \t hit a ball

Read ATOMIC-2020 — a commonsense knowledge graph of if-then inferences — and
translate it into a vaelii corpus.

Of everything this repo reads, ATOMIC is the one whose content is **already the shape
vaelii's assumption strengths were built for**.  Cyc marks a handful of assertions
`:default`; an OBO ontology and an OWL vocabulary are definitional throughout.  Every
one of ATOMIC's 1.3 million tuples is a defeasible generalization — *if PersonX
abandons the cat, PersonX probably wants to find a new home for it* — collected by
asking people what usually follows.  Nothing in it is a definition, so nothing in it
is written `:monotonic`, and a corpus where every sentence can be retracted by a later
one is a corpus a truth-maintenance system has something to do with.

## The format

ATOMIC-2020 ships as tab-separated triples, split into `train.tsv`, `dev.tsv` and
`test.tsv`:

    PersonX abandons ___ altogether \t xIntent \t to be selfish
    PersonX abandons ___ altogether \t xEffect \t none
    baseball bat \t ObjectUse \t hit a ball

The 23 relations fall into three families, and the families are this corpus's
contexts:

* **social** — what a person intends, needs, feels, wants, or becomes because of an
  event (`xIntent`, `xNeed`, `xAttr`, `xEffect`, `xReact`, `xWant`, `xReason`, and the
  `o*` versions for the *other* people in the event);
* **event** — how events sit next to each other (`isAfter`, `isBefore`, `Causes`,
  `HasSubEvent`, `HinderedBy`, `isFilledBy`);
* **physical** — what a thing is for, where it is, what it is made of (`ObjectUse`,
  `AtLocation`, `MadeUpOf`, `HasProperty`, `CapableOf`, `Desires`, `NotDesires`).

## Nodes are phrases, and stay phrases

An ATOMIC node is a fragment of English — `PersonX abandons ___ altogether`, `to be
selfish`, `baseball bat`.  It is not a term and has no definition anywhere, so each
one becomes an **individual** named from its own text, with the text itself kept
alongside as `(nodeText PersonXAbandonsAltogether "PersonX abandons ___ altogether")`.

The temptation is to make `baseball bat` a type and `PersonX abandons ___ altogether`
an event template, and it is worth saying why that is not done: ATOMIC never marks
which of its nodes are kinds, the same string appears on both sides of relations from
different families, and a type with no members and no supertype is a name with nothing
attached.  Inventing a type system the corpus does not have would make the output look
more structured than the source and be wrong in ways nothing could check.

Two rows with the same text are the **same** node, which is what makes the result a
graph rather than a list: `to be selfish` reached as PersonX's intent and reached
again as somebody's reaction is one individual with two relations to it.

A `none` tail — ATOMIC's own marker for "annotators said nothing follows" — is
dropped and counted.  It is a real annotation and it is not a fact.

Read CFASL — Cyc's binary serialization format — as Clojure data.

A CFASL stream is a sequence of self-describing objects: one **opcode** byte, then
whatever that opcode's payload is, recursively.  Nothing frames an object but its own
opcode, so a reader that mis-sizes one payload desynchronizes the whole rest of the
file rather than failing where the mistake is — which is why `vaelii.foreign.units`
checks its record count against the dump's own and treats a short read as an error.

**Where this comes from.**  Cycorp published CFASL's reference implementation under
the Apache License 2.0, so none of this was reverse-engineered:
`com.cyc.cycjava.cycl.cfasl` supplies the opcode table and every encoding below, and
`org.opencyc.api.CfaslInputStream` (the same license, per OpenCyc's `LEGAL.txt`)
supplies the four the SubL sources declare and leave undefined — float, bignum,
character and byte-vector.  `licenses/THIRD-PARTY.md` records the attribution and
which upstreams are *not* admissible here.

**The opcode table is the server's, not the API client's**, and they disagree in
exactly the range a dump uses: the client reads 36/37/38 as source / source-def /
axiom, while a KB dump writes deduction / kb-hl-support / clause-struc, and 50 is the
client's special-object against the server's common-symbol.  Reading unit files with
the client's table decodes the first few thousand objects and then quietly produces
nonsense.  The table below is `cfasl.lisp`'s and `cfasl-kb-methods.lisp`'s.

**Handles are not resolved here.**  A constant, NART, assertion or clause-struc is
written as its opcode plus an integer dump-id, and what that id means lives in another
file of the same dump.  So a caller passes `resolvers` — `{:constant f :nart f …}` —
and anything it does not name reads back as its own marker list, `(:nart 23227)`, the
same shape `vaelii.foreign.cycl` yields for a reference its own dump could not
resolve.  That indirection is upstream's too (`*cfasl-nart-handle-lookup-func*`).

The converter's command line, across every format this artifact reads.

Converting an ontology is a **one-off**, and it is the point of the whole repo: a
reader is finished the day its corpus has been converted once into the format vaelii
writes.  So the CLI is one verb per thing you do to a corpus, and the format only
appears where it has to.  `-main`'s docstring is the usage text a run prints; the
README has the worked examples.

**`load` takes no format.**  Every converter here writes the same corpus
(`vaelii.foreign.corpus`), so loading one is format-independent: all the format decides
is which `profiles` the `--profile` name is looked up in, and the corpus says which it
is in its own `meta.edn`.

Nothing here is required by the readers: this namespace depends on all five and none
of them depends on it, so the plugin seam still loads exactly the one namespace whose
format was asked for.

meta.edn                     format, source, counts, context load order
names.edn                    foreign term -> vaelii term, with its role
report.edn                   what converted, what dropped, and why
NOTICE                       whose knowledge this is, and on what terms
kb/Topology.txt              the genlContext wiring of every context
kb/<C>Context.txt            that context's `:default`-strength sentences
kb/<C>Context.monotonic.txt  its `:monotonic` ones

The corpus — what every reader in this repo converts **to**, and the one thing that
loads it.

A translated ontology is not a dump and not a jar resource: it is a directory of plain
vaelii sentence files, one s-expression per line, partitioned by context.

    meta.edn                     format, source, counts, context load order
    names.edn                    foreign term -> vaelii term, with its role
    report.edn                   what converted, what dropped, and why
    NOTICE                       whose knowledge this is, and on what terms
    kb/Topology.txt              the genlContext wiring of every context
    kb/<C>Context.txt            that context's `:default`-strength sentences
    kb/<C>Context.monotonic.txt  its `:monotonic` ones

**The `NOTICE` is not decoration.**  A translated ontology is a *reformulation* of
somebody else's knowledge, and every source this repo reads attaches attribution terms
that survive one — OpenCyc says so outright, extending its licence to "renamings and
other logically equivalent reformulations … in any formal language".  So the
obligation travels with the corpus and not with this code, which means it has to be
written into the directory: a corpus is data on a filesystem, and the notice is what
keeps it attributable once somebody has copied it somewhere else.

**One format, five converters.**  Cyc, RDF, WordNet, OBO and ATOMIC disagree about
everything except the shape of the answer, so the answer is where they are made to
agree: a corpus says nothing about where it came from beyond `meta.edn`'s `:format`
line, and `load-dir!` never asks.  That is worth more than tidiness — vaelii's own
catalog recognizes a corpus by `meta.edn` holding a `:context-order` and loads it
through one reader, so a directory any of these converters wrote is a directory the
catalog already knows how to open.

The files are read off the **filesystem**, not the classpath: a converted corpus is
data beside a KB directory, not shipped schema, and the interesting ones do not fit
in a jar.

## Why loading is layered

`load-dir!` does not read the files in the order they were written.  It reads the
whole corpus five times — term definitions, hierarchy, the rest of the schema, type
memberships, then everything else — and the reason is not speed:

* term definitions first is an **identity** argument.  `(reifiableFunction F)` and
  `(termOfUnit K E)` decide which constant a non-atomic term reifies to, and every
  later sentence mentioning that term is reified against them as they stand then.
  Arriving late, they mint a second constant for an expression that already has one —
  or miss the reification altogether.

* `genl` edges next is a **cost** argument.  An edge retires the memo of the cached
  transitive closure and everything else reads that closure back, so interleaved, each
  fact pays for a closure the next edge is about to retire.

* memberships before facts is a **correctness** argument.  `argIsa` is open-world
  about an argument with no type at all and closed about one that has any, so a fact
  checked before its arguments' memberships have arrived is refused on a partial
  answer — and the same corpus in a different order would keep a different set of
  sentences.  Loading every membership first is what makes "what types does this term
  have" complete before anything asks.

A refusal is **counted, not swallowed**: which of vaelii's checks a foreign ontology
trips is the most useful thing an import has to say about it.

Translate an OpenCyc KB — the CycL assertion dump read by `vaelii.foreign.cycl`
— into vaelii KB files, and load them.

Cyc and vaelii represent the same idea (a contextualized common-sense KB) with
different spellings, so the translation is mostly a **renaming** plus a small
table of sentence rewrites.  Three things make it more than a rename:

* **A role decides a term's spelling.**  vaelii's naming invariants encode a
  term's role in its symbol — a type is `snake_case`, a predicate `camelCase`, an
  individual `CapitalCamelCase`, a context ends in `Context` — while Cyc writes
  every constant `CapitalCamelCase` (predicates aside) and states the role as an
  assertion.  So the role must be **recovered first**: pass 1 reads the whole dump
  and classifies every constant, pass 2 translates with that classification in
  hand.  A constant's position is the evidence — arg 2 of `isa` is a collection,
  the head of a formula is a predicate, the microtheory slot is a context — backed
  by Cyc's own `(isa X Collection)` / `(isa X Microtheory)` declarations where they
  exist.  Position is decisive because collection-hood is usually *inherited* in
  Cyc (a term is an instance of some metatype, not of `Collection` itself), so the
  explicit declarations alone would classify a fraction of the KB.

* **Cyc says with a predicate what vaelii says with a record field.**  `(isa P
  TransitiveBinaryPredicate)` is vaelii's `(transitive P)`; `(genls A B)` is
  `(genl a b)`; `(genlMt A B)` is `(genlContext AContext BContext)`; `(arg1Isa P
  C)` is `(argIsa p 1 c)`; a `:monotonic` assertion is a `{:strength :monotonic}`
  premise.  Everything without such a mapping stays an ordinary fact under its own
  (renamed) predicate — uninterpreted by the engine but stored, indexed and
  queryable, which is the honest outcome for vocabulary vaelii has no theory of.

* **Some of the dump is not knowledge.**  SubL code (`afterAdding` hooks), forms
  the dumper could not resolve (`(:nart 23227)`), and assertions whose
  microtheory is itself unresolved are dropped, each with a counted reason.  The
  report is part of the output: a conversion that silently dropped a tenth of the
  KB would look exactly like one that dropped nothing.

**Output.**  A `vaelii.foreign.corpus` directory — plain vaelii sentence files, one
s-expression per line, partitioned by context — with `report.edn` carrying the
per-predicate converted/dropped counts and `names.edn` the Cyc constant -> vaelii term
map.  Cyc's `:monotonic` / `:default` marking is the corpus's strength split.

Entry points: `convert!` (dump -> directory) and `load-dir!` (directory -> kb), both
also reachable from `vaelii.foreign.convert`'s command line.

(ke-assert '(#$genls #$Dog #$Mammal) #$BiologyMt :monotonic :forward)

Read a CycL assertion dump — the s-expression form of an OpenCyc KB — as data.

This is the reader for a corpus that arrives already extracted as text;
`vaelii.foreign.units` reads the distribution's own binary dump, and both yield the
same maps.  A CycL dump is a stream of `ke-assert` forms, one per assertion:

    (ke-assert '(#$genls #$Dog #$Mammal) #$BiologyMt :monotonic :forward)

so each carries a **formula**, the **microtheory** it holds in, its **strength**
(`:monotonic` / `:default` — Cyc's own monotonic-vs-default truth value, which is
vaelii's assumption strength under another name) and its **direction**
(`:forward` / `:backward` / `:code`).

Nothing here interprets CycL: this namespace is the *lexer*, and
`vaelii.foreign.cyc` is the translation.  Reading is separated because the
dump's syntax is Common Lisp, not EDN — `#$Constant` is not a reader tag, a
string may span lines, and a float prints as `1.0d0` — so `clojure.edn/read`
cannot be pointed at it.

**How a CycL token comes back.**  Every kind of token maps to ordinary Clojure
data, and the *namespace* of a symbol is what distinguishes the kinds that would
otherwise collide:

| CycL           | Clojure                | why                                    |
|----------------|------------------------|----------------------------------------|
| `#$Dog`        | `cyc/Dog`              | a KB constant — the only kind translated |
| `?X`           | `?X`                   | a variable, already vaelii's spelling  |
| `foo-bar`      | `subl/foo-bar`         | a bare SubL symbol — executable code, never KB content |
| `"text"`       | `"text"`               | |
| `:keyword`     | `:keyword`             | |
| `12`, `1.0d0`  | `12`, `1.0`            | |
| `(a b)`        | `(a b)`                | |

A form the dumper could not resolve reads back as its own marker list —
`(:nart 23227)`, `(:unresolved-assertion 88)` — and stays that way, for the
translation to reject.

The reader is a lazy seq over a `Reader`, so a 268 MB dump streams in constant
memory.

Compare two corpora — what one says and the other does not.

Every reader here writes the same corpus, so any two are comparable whatever produced
them, and that turns out to be worth more than a debugging aid.

**The case that motivated it.**  OpenCyc publishes its knowledge base twice: as the
binary CFASL dump the inference engine loads, and as an OWL export.  This repo reads
both, by two code paths that share nothing below `corpus/write!` — one is an opcode
table over a byte stream, the other an XML lexer feeding a description-logic
projection.  Converting the same KB through both and comparing is the only check
either reader has that is not a fixture somebody wrote to match what the code already
did.  If the taxonomies agree, two independent implementations agree; where they
differ, the difference is a *finding* — about a reader, or about the two releases.

Because they are two releases.  The OWL export and the CFASL dump ship on their own
schedules, so the diff is not expected to be empty and an empty one would be the
surprising result.  What the diff is for is telling the two kinds of difference apart:
a term missing on one side because the release genuinely lacks it, and a term missing
because a reader dropped it.

**What is compared, and why not everything.**  The *taxonomy* — `genl` edges and the
terms in them — plus a per-predicate sentence count.  Not the full sentence sets: a
corpus of two million sentences would need a real graph diff to compare completely,
and most of the difference would be lexical noise (labels, comments, identifiers) that
says nothing about whether the two agree.  The taxonomy is the part with a truth
value.

Names are the join key, and that is sound here for a specific reason: both readers
name from the same underlying Cyc term names, so `#$BadmintonAccessory` in the dump
and `Badminton_accessory` in the OWL both spell to `badminton_accessory`.  Comparing
corpora from unrelated sources would need a mapping and this offers none.

[Term]
id: GO:0000278
name: mitotic cell cycle
namespace: biological_process
def: "The cell cycle of a eukaryotic cell..." [GOC:mah]
is_a: GO:0007049 ! cell cycle
relationship: part_of GO:0051301

[Typedef]
id: part_of
is_transitive: true
inverse_of: has_part

Read an OBO-format ontology — the OBO Foundry's own stanza syntax — and translate it
into a vaelii corpus.

OBO is the format the life sciences standardized on: the Gene Ontology, ChEBI, Uberon,
the Disease Ontology and about two hundred others, all CC-BY, all built by curators
rather than generated.  A `.obo` file is a header followed by stanzas:

    [Term]
    id: GO:0000278
    name: mitotic cell cycle
    namespace: biological_process
    def: "The cell cycle of a eukaryotic cell..." [GOC:mah]
    is_a: GO:0007049 ! cell cycle
    relationship: part_of GO:0051301

    [Typedef]
    id: part_of
    is_transitive: true
    inverse_of: has_part

which is why it gets its own reader rather than going through the RDF one: the same
ontologies also publish an OWL serialization, and the OWL is *generated from* the OBO
by a mapping that turns every `relationship:` into an existential restriction — which
is precisely the OWL construct that has no Horn form.  Reading the OBO directly keeps
the relations vaelii can hold.

## What comes across

| OBO                                | vaelii                                        |
|------------------------------------|-----------------------------------------------|
| `[Term]`                           | a **type** (snake_case, named from `name:`)   |
| `[Typedef]`                        | a **predicate** (camelCase, named from `id:`) |
| `[Instance]`                       | an **individual**                             |
| `namespace:`                       | the **context** the stanza's sentences land in |
| `is_a: X`                          | `(genl this x)`                               |
| `relationship: R X`                | `(r this x)`                                  |
| `disjoint_from: X`                 | `(disjoint this x)`                           |
| `intersection_of:` (all plain)     | both `genl` edges **and** the sufficient-condition rule |
| `union_of: X`                      | `(genl x this)`, each member                  |
| `domain:` / `range:`               | `(argIsa p 1 c)` / `(argIsa p 2 c)`           |
| `is_transitive:` and its siblings  | `(transitive p)`, `(symmetric p)`, …          |
| `inverse_of: Q`                    | `(inverse p q)`                               |
| `holds_over_chain: A B`            | `(implies (and (a ?x ?y) (b ?y ?z)) (p ?x ?z))` |
| `transitive_over: R`               | `(implies (and (p ?x ?y) (r ?y ?z)) (p ?x ?z))` |
| `def:` / `comment:`                | `(comment t "…")`                             |
| `name:` / `synonym:` / `xref:`     | `(label t "…")`, `(synonym t "…")`, `(xref t "…")` |

## What `relationship:` means here, and what it does not

`relationship: part_of GO:0051301` is read as the **class-level fact** `(partOf
mitotic_cell_cycle cell_division)`.  That is what the OBO tooling means by it in
practice and what every browser shows, but it is *not* what the OWL mapping says: there
it is `MitoticCellCycle ⊑ ∃part_of.CellDivision`, an existential restriction on each
instance.  The two agree about the ontology's shape and disagree about what follows for
an individual, and only the first has a form vaelii can store.  Taking it is a
deliberate weakening, recorded here so nobody has to rediscover it from the output.

**An obsolete term is dropped**, with its stanza counted, unless `:obsolete? true`.
OBO never deletes: a retired term keeps its id, gains `is_obsolete: true`, and loses
its `is_a` edges, so importing one gives a term with no place in the hierarchy and a
name that collides with its replacement's.

ex:Bob ex:hasJob [ ex:employer ex:Acme ; ex:role ex:Engineer ; ex:since 2020 ] .
-> (hasJob Bob Acme Engineer 2020)   with  (arity hasJob 4)

ex:Bob ex:award [ ex:name "Prize" ; ex:year 2001 ] .
-> (award Bob BobAward) (name BobAward "Prize") (year BobAward 2001)

Translate an RDF graph — RDFS and OWL, read by `vaelii.foreign.turtle` — into a
vaelii corpus.

This is the reader with the widest reach in the repo and the one that gives up the
most, and both come from the same fact: **OWL is a description logic and vaelii is a
Horn rule engine.**  So the translation is not a rename with a table of exceptions the
way Cyc's is.  It is a *projection*: the fragment of OWL that has a Horn reading comes
across as rules and declarations, and the rest is dropped under a counted reason.

## What comes across

| RDF / OWL                                      | vaelii                              |
|------------------------------------------------|-------------------------------------|
| `C rdfs:subClassOf D`                          | `(genl c d)`                        |
| `X rdf:type C`                                 | `(c X)`                             |
| `P rdfs:subPropertyOf Q`                       | `(genl p q)`                        |
| `P rdfs:domain C` / `rdfs:range C`             | `(argIsa p 1 c)` / `(argIsa p 2 c)` |
| `C owl:disjointWith D`                         | `(disjoint c d)`                    |
| `owl:AllDisjointClasses`                       | `(disjoint …)`, pairwise            |
| `P owl:inverseOf Q`                            | `(inverse p q)`                     |
| `P a owl:TransitiveProperty` (and the rest)    | `(transitive p)`, `(symmetric p)`, … |
| `C owl:equivalentClass D`                      | `(genl c d)` **and** `(genl d c)`   |
| `P owl:propertyChainAxiom (Q R)`               | `(implies (and (q ?x ?y) (r ?y ?z)) (p ?x ?z))` |
| `C ⊑ ∀P.D`   (`owl:allValuesFrom`)             | `(implies (and (c ?x) (p ?x ?y)) (d ?y))` |
| `C ⊑ ∃P.{v}` (`owl:hasValue`)                  | `(implies (c ?x) (p ?x V))`         |
| `C ≡ ∃P.D`   (`owl:someValuesFrom`)            | `(implies (and (p ?x ?y) (d ?y)) (c ?x))` |
| `C ≡ D1 ⊓ D2` (`owl:intersectionOf`)           | the rule **and** both `genl` edges  |
| `O owl:imports O2`                             | `(genlContext OContext O2Context)`  |
| anything else                                  | `(p S O)`, a plain fact             |

## What does not, and why that is the interesting half

* **`C ⊑ ∃P.D`** — an existential in the *conclusion*.  "Every hand has a finger"
  obliges the reasoner to invent a finger, which is what a description logic does and
  a rule engine does not.  Counted as `:existential-superclass`, and it is the single
  largest drop on a real ontology.
* **`owl:unionOf` in the ⊑ direction, `owl:oneOf`, `owl:complementOf`** — a real
  disjunction, a real closed class, a real negation.  Each is a claim vaelii could
  hold only by weakening it into something else.
* **Cardinality** (`owl:minCardinality`, `maxCardinality`, `qualifiedCardinality`) —
  counting constraints have no Horn form at all.  `owl:FunctionalProperty` is the one
  exception and only behind `:functional?`, for the same reason it is behind that flag
  in the Cyc reader: vaelii's `functional` *merges* two values through the equality
  closure rather than refusing the second.

Reading the two directions of `owl:equivalentClass` separately is what buys most of
what is kept: `C ≡ ∃P.D` is worthless as a definition and perfectly good as the
sufficient-condition half of one.

## Blank nodes, and the n-ary relations they stand for

RDF triples are binary, so a relation with more than two arguments has to be written
as a **node with one edge per argument** — the W3C n-ary pattern, Wikidata's
statements, schema.org's Role, and every `[ :city "Austin" ; :zip "78701" ]` anybody
ever wrote.  vaelii predicates have no arity limit, so such a node can become one fact
of higher arity:

    ex:Bob ex:hasJob [ ex:employer ex:Acme ; ex:role ex:Engineer ; ex:since 2020 ] .
    -> (hasJob Bob Acme Engineer 2020)   with  (arity hasJob 4)

That is only sound when the shape is **uniform**, and `bnode-plan` is the pass that
decides.  Flattening positionally needs every node a predicate reaches to carry the
same qualifiers, exactly once each, with nameable fillers, reached from one place, and
the predicate to take no plain object anywhere else.  Miss any of those and the
arguments silently misalign — `(hasJob Ann Beta 1999)` with a date where a rule reads
a role.  vaelii is open-world about arity until something declares it, so nothing
would refuse the misaligned fact; it would simply never match.  Hence the `(arity p
n)` written beside the tuples: once declared, a later fact at the wrong arity is a
refusal rather than a sentence quietly matching nothing.

Every other blank node is **skolemized** — given a name built from the subject and
predicate that reach it (`BobAward`), with its own triples kept as ordinary facts:

    ex:Bob ex:award [ ex:name "Prize" ; ex:year 2001 ] .
    -> (award Bob BobAward) (name BobAward "Prize") (year BobAward 2001)

That loses nothing — it is what the n-ary pattern already means — and it is what a
node with an identity of its own wants regardless.  A **typed** node is always
skolemized: `[ a :Employment ; … ]` claims to be an instance of something, and
flattening would take away the term the membership hangs on.  So is a **shared** one,
since flattening would copy it rather than move it.

Only the interior of an OWL construct is left alone, because the axiom that owns it
has already said what it means.  `:n-ary? false` skolemizes everything, which is the
reading to take if a rule set was written against the joined form.

## Naming and contexts

A term is named from its IRI's **local part** — after the last `#` or `/` — and its
role decides the spelling, so `ex:Mammal` used as a class becomes the type `mammal`
and `ex:hasPart` used as a property becomes `hasPart`.  Two IRIs with the same local
part in different namespaces collide, which is normal and is resolved by suffix, in a
sorted pass, so a re-run of the same graph produces the same names.

A **named graph** is a context (that is what N-Quads' fourth term is for), and so is
an `owl:Ontology`.  A graph that declares neither lands in one context named for the
source.  `owl:imports` is a `genlContext` edge, which makes an ontology's import
closure the context hierarchy it always was.

<rdf:Description rdf:about="#Dog">            <!-- node:     the subject -->
  <rdfs:subClassOf rdf:resource="#Mammal"/>    <!-- property: one triple  -->
  <rdfs:label xml:lang="en">dog</rdfs:label>
</rdf:Description>

Read an RDF/XML document as the same triples `vaelii.foreign.turtle` yields.

RDF/XML is a **third syntax for the graph the other two already carry**, so this is a
second lexer and not a second reader: it emits `{:s :p :o}` maps with terms spelled
exactly as `turtle` spells them, and `vaelii.foreign.rdf`'s OWL projection is reached
without changing a line of it.  Nothing here knows what `rdfs:subClassOf` means.

**Why it has to exist.**  Turtle is what a person writes and N-Triples is what a dump
ships, but RDF/XML is what an *OWL ontology* is published as — it was the only
serialization OWL 1 defined, so the tooling that generates ontologies still emits it.
OpenCyc's OWL export, DOLCE, and the OWL product of every OBO Foundry ontology are all
RDF/XML.  A reader that stopped at Turtle could not open any of them.

**The syntax is striped**, alternating node and property elements:

```xml
<rdf:Description rdf:about="#Dog">            <!-- node:     the subject -->
  <rdfs:subClassOf rdf:resource="#Mammal"/>    <!-- property: one triple  -->
  <rdfs:label xml:lang="en">dog</rdfs:label>
</rdf:Description>
```

and nearly everything else in the grammar abbreviates that shape.  A node element's
tag is an `rdf:type` triple unless it is `rdf:Description`; an attribute that is not
syntax is a triple with a literal object; `rdf:parseType` switches the content between
three readings; `rdf:li` counts.  The awkward part of RDF/XML is not the striping but
how many ways it offers to write one triple, and this reader's job is that every one
of them lands as the same data.

**Parsed with the JDK's StAX**, pulled one top-level node element at a time, so a
252 MB ontology streams rather than building a DOM.  DTD support is on because RDF/XML
routinely declares internal entities for its namespaces (`<!ENTITY owl "http://…">`,
then `&owl;Class`) and a parser rejecting those could not read the files it exists for;
**external** entities are off, which is what keeps that safe.

**What is approximated.**  `rdf:parseType="Literal"` should hold the exclusive XML
canonicalization of its content.  This writes a faithful but not canonical
serialization — attribute order and namespace declarations may differ from C14N — so
an XMLLiteral compares equal to itself and not necessarily to another parser's.  No
ontology reasons over one, and `vaelii.foreign.rdf` keeps it as a string.

type         snake_case                 dog, physical_object
predicate    camelCase                  parentOf, argIsa
individual   CapitalCamelCase           Rover, Einstein
function     CapitalCamelCase           GroupFn
context      CapitalCamelCase + Context UniverseContext

Spell a foreign name as a vaelii term.

Every ontology this repo reads names its terms in its own alphabet — Cyc writes
`#$Agent-PartiallyTangible`, RDF an IRI, OBO `GO:0008150`, WordNet a lemma and a byte
offset, ATOMIC an English sentence — and vaelii names a term with a symbol whose
*shape encodes its role* (`vaelii.impl.naming`):

    type         snake_case                 dog, physical_object
    predicate    camelCase                  parentOf, argIsa
    individual   CapitalCamelCase           Rover, Einstein
    function     CapitalCamelCase           GroupFn
    context      CapitalCamelCase + Context UniverseContext

So a translation cannot rename term-by-term as it goes: it has to know a term's
**role** first, and every reader here earns that the same way — a classifying pass
over the source, then a naming pass with the classification in hand.  What differs
between them is only what counts as evidence, which is each reader's business; the
spelling and the collision handling are this namespace's, so five readers cannot
disagree about what a legal predicate name looks like.

Two properties matter more than they look:

* **Renaming is not injective, so collisions are the normal case.**  `GO:0008150`
  and `GO:0008151` may both be named `biological_process`, `owl:Class` and
  `rdfs:Class` both `class`.  `name-table` resolves them with the smallest numeric
  suffix that is free, in a **sorted** pass, so the same source always produces the
  same names whatever order it was read in — a corpus you can diff between runs.

* **A name becomes a file name.**  A context file is `<Context>.txt`, and a
  filesystem bounds a name at 255 bytes, while a computed Cyc microtheory or an
  ATOMIC event runs far longer.  `abbreviate` truncates and appends a hash of the
  whole, in letters rather than digits so `spell` cannot break the hash into a word
  of its own.

Read an RDF graph — N-Triples, N-Quads or Turtle — as data.

This is the *lexer*; `vaelii.foreign.rdf` is the translation, and the split is the
same one `cycl` and `cyc` make.  Nothing here knows what `rdfs:subClassOf` means.

**Three syntaxes, one reader.**  They are the same grammar at three levels of sugar,
so separating them would mean three parsers that disagree about IRIs:

| syntax     | what it adds                                                     |
|------------|------------------------------------------------------------------|
| N-Triples  | `<s> <p> <o> .` — one statement per line, no abbreviation of any kind |
| N-Quads    | a fourth term, the **graph** the statement belongs to            |
| Turtle     | `@prefix`, `@base`, `a`, `;`, `,`, `[…]`, `(…)`, numeric and boolean literals |

A bulk dump ships as N-Triples or N-Quads (Wikidata, YAGO, DBpedia) because a
line-per-statement file streams and shards; a hand-written ontology ships as Turtle
(schema.org, BFO, most of the OBO Foundry's OWL) because a person has to read it.  A
reader that could only take the first would be a reader you cannot point at an
ontology, and one that could only take the second would not scale to a real dump.

**How a term comes back.**  As in `cycl`, the *namespace* of a symbol is what tells
two kinds apart that would otherwise collide:

| RDF                        | Clojure                        |
|----------------------------|--------------------------------|
| `<http://example.org/Dog>` | `iri/http://example.org/Dog`   |
| `ex:Dog` (prefix expanded) | `iri/http://example.org/Dog`   |
| `_:b0`                     | `bnode/b0`                     |
| `"text"`                   | `"text"`                       |
| `42`, `1.5`, `true`        | `42`, `1.5`, `true`            |
| `"dog"@en`                 | `{:lex "dog" :lang "en"}`      |
| `"2020-01-01"^^xsd:date`   | `{:lex "2020-01-01" :datatype iri/…}` |

A literal is a **plain Clojure value** whenever it has nothing else to say — a
language tag or a datatype vaelii has no native reading of is what makes it a map
instead.  That keeps the common case cheap and the tagged case honest: a Wikidata
dump's labels differ only by `@lang`, and a reader that dropped the tag would make
fifty translations of one label indistinguishable.

**Blank nodes are expanded, not deferred.**  `[ :p :o ]` and `( a b c )` are
abbreviations for triples about a fresh blank node, so the reader emits those triples
and hands back the node — which is why one statement can yield many, and why the seq
is over *triples* rather than over lines.

The reader is a lazy seq over a `Reader`, so a multi-gigabyte dump streams in constant
memory.

constant-shell.text     "name" -> dump-id, in plain text: the whole name table
nart-hl-formula.cfasl   nart id -> its HL formula, whose head is a function
clause-struc.cfasl      clause-struc id -> a CNF, which is how a rule is stored
assertion.cfasl         id, formula-data, mt, flags, arguments, plist

Read an OpenCyc KB dump directory — `server/cyc/run/units/<n>/` — as assertions.

This is the other end of `vaelii.foreign.cycl`: the same
`{:formula :mt :strength :direction}` maps, taken straight from the binary dump
instead of from a text re-dump of it, so `vaelii.foreign.cyc` translates either
without knowing which it got.

**A dump is several files that only mean something together.**  Each holds a flat
sequence of records — a copyright string, then `dump-id` followed by that record's
fields — and every reference between them is an integer id resolved against another
file:

    constant-shell.text     "name" -> dump-id, in plain text: the whole name table
    nart-hl-formula.cfasl   nart id -> its HL formula, whose head is a function
    clause-struc.cfasl      clause-struc id -> a CNF, which is how a rule is stored
    assertion.cfasl         id, formula-data, mt, flags, arguments, plist

So the three small tables are read whole and the 780 MB assertion file streams
against them.  `constant-shell.text` being text is not a convenience this reader
invented — the dump writes the name table twice, once as CFASL and once as text, and
the text one is complete.

**A NART may name a NART defined later in the file.**  Resolving during the read would
render that forward reference as an unresolved marker and then propagate it into every
assertion mentioning the term, so the formulas are read with NART references left as
markers and expanded when an assertion is *emitted* — by which point every NART is in
the table.  `nart-depth-limit` and `nart-node-limit` are what a NART that refers back
to itself would otherwise do.

**What the counts are for.**  Nothing frames a CFASL object but its own opcode, so a
mis-sized payload desynchronizes the rest of the file instead of failing in place.
Each dump states its own record count in `<name>-count.text`, and a read that reaches
the end of the file is compared against it: a short read means the stream went out of
step, and it is an error rather than a smaller KB.  A read that stopped early has no
such comparison to make — the stated count describes the whole file — so `:limit`
samples a dump without the check and only without it.

Provenance of the format, and which upstream sources may be used on it:
`licenses/THIRD-PARTY.md`.

02084071 05 n 01 dog 0 019 @ 02083346 n 0000 ~ 02084732 n 0000 … | a member of the genus Canis

offset  lexfile  pos  w_cnt(hex)  word lex_id…  p_cnt  pointers…  | gloss

Read a WordNet database — Princeton's own `dict/` directory — and translate it into a
vaelii corpus.

WordNet is the odd one out here: it is a **lexical** database, organized around which
words mean the same thing, and only incidentally an ontology.  What makes it worth
importing anyway is that its central relation is subsumption.  A synset is a set of
synonymous words, `@` points from a synset to a more general one, and 82,000 noun
synsets wired that way are a usable upper taxonomy that nobody has to build.

## The format

`data.noun`, `data.verb`, `data.adj` and `data.adv` — the WNDB format, which has not
changed since 1993 and which Open English WordNet still publishes alongside its XML:

    02084071 05 n 01 dog 0 019 @ 02083346 n 0000 ~ 02084732 n 0000 … | a member of the genus Canis

    offset  lexfile  pos  w_cnt(hex)  word lex_id…  p_cnt  pointers…  | gloss

A pointer is `symbol offset pos source/target`.  This reader takes the data files
alone: they are self-contained, and the index files exist to answer "which senses does
this word have", which is a lookup rather than a fact.

## What comes across

| WordNet                        | vaelii                                             |
|--------------------------------|----------------------------------------------------|
| a synset                       | a **type** — `dog_n`, spelled from its first word  |
| `@`  hypernym                  | `(genl dog_n canine_n)`                            |
| `@i` instance hypernym         | the synset is an **individual**: `(composer_n Bach)` |
| `&`  similar to (satellite)    | `(genl damp_a wet_a)`                              |
| `*`  entailment                | a **defeasible rule** — see below                  |
| `%m` `%s` `%p` meronyms        | `(memberMeronym …)`, `(substanceMeronym …)`, `(partMeronym …)` |
| `!` `=` `>` `^` `$` `<` `\\` `+` | `(antonym …)`, `(attribute …)`, `(causes …)`, …    |
| `;c` `;r` `;u` domains         | `(topicDomain …)`, `(regionDomain …)`, `(usageDomain …)` |
| each word of a synset          | `(wordForm dog_n "dog")`                           |
| the gloss                      | `(comment dog_n "a member of the genus Canis")`    |

**`~` and the holonyms are not dropped for being uninteresting** — they are the exact
inverses of `@` and the meronyms, stored in both directions because WordNet is a
lookup structure.  Importing both would double the corpus to say each thing twice.

**Entailment becomes a rule.**  `snore *> sleep` says that snoring entails sleeping,
and a verb synset is a type of event, so this reads as `(implies (snore_v ?x)
(sleep_v ?x))` over events — the one relation in WordNet with a genuine inferential
reading, and it is why importing this into a rule engine is worth doing at all.  It
lands **defeasible** (`set/defaultRule`), because it is a generalization about usage
rather than a definition, and it fires on nothing WordNet itself contains: it is
vocabulary for somebody else's events.  `:entailment-rules? false` writes plain
`(entails a b)` facts instead.

## Naming and contexts

A synset is named from its first word plus its part of speech — `dog_n`, `run_v` —
which collides constantly, because that is what a word having several senses *is*.
Collisions take a numeric suffix in offset order, so `dog_n` and `dog_n_2` are stable
across runs but are **not** WordNet's own sense numbers, which live in the index files
this reader does not read.  `(wnOffset dog_n "n02084071")` is written for every
synset, and is the identifier to join on.

Each part of speech is its own context under `WordNetContext`, so a profile can load
the noun taxonomy without the other three.