Liking cljdoc? Tell your friends :D

hive-ingestor-rfc.addon

IAddon that teaches hive-ingestor about the RFC series.

The host owns the seams; this addon owns the corpus. On initialize! it registers its ISource factory in the host's source registry, so the series becomes reachable as ingest source source=rfc dir=... without the host ever naming an RFC. shutdown! retracts every registration it owns.

IAddon that teaches hive-ingestor about the RFC series.

The host owns the seams; this addon owns the corpus. On initialize! it
registers its ISource factory in the host's source registry, so the series
becomes reachable as `ingest source source=rfc dir=...` without the host
ever naming an RFC. shutdown! retracts every registration it owns.
raw docstring

hive-ingestor-rfc.index

The RFC registry: what a document cannot know about itself.

An RFC's masthead carries Obsoletes:/Updates: — edges pointing BACKWARD. Whether the document has since been obsoleted, and what its current status is, exist only in the registry (rfc-index.xml). A parser reading the .txt can build supersession edges but can never mark a document HISTORIC, so retrieval over the corpus answers present-tense questions confidently and wrongly.

This namespace turns the registry into facets the ingest pipeline attaches to both the document and its chunks.

The RFC registry: what a document cannot know about itself.

An RFC's masthead carries Obsoletes:/Updates: — edges pointing BACKWARD.
Whether the document has since been obsoleted, and what its current status
is, exist only in the registry (rfc-index.xml). A parser reading the .txt
can build supersession edges but can never mark a document HISTORIC, so
retrieval over the corpus answers present-tense questions confidently and
wrongly.

This namespace turns the registry into facets the ingest pipeline attaches
to both the document and its chunks.
raw docstring

hive-ingestor-rfc.mirror

ISource over a local rsync mirror of the RFC series.

Bulk access to the series is rsync, not HTTP: rsync -avz --delete rsync.rfc-editor.org::rfcs-text-only <dir>

Only the transport differs from the web source — the same rule chain, the same RfcTextParser and the same Document shape are reused through web-docs/body->document, and :document/source is the canonical rfc-editor URL, so document ids and cross-document KG edges agree with anything ingested over HTTP.

The series is ~10k files; a run is a WINDOW over it (:from, :to, :offset, :limit). Re-running a window is an upsert, not a duplicate, because chunk and document ids are derived from the source URL — that is the resumable story: crash, re-run the window, continue.

ISource over a local rsync mirror of the RFC series.

Bulk access to the series is rsync, not HTTP:
  rsync -avz --delete rsync.rfc-editor.org::rfcs-text-only <dir>

Only the transport differs from the web source — the same rule chain, the
same RfcTextParser and the same Document shape are reused through
web-docs/body->document, and :document/source is the canonical
rfc-editor URL, so document ids and cross-document KG edges agree with
anything ingested over HTTP.

The series is ~10k files; a run is a WINDOW over it (:from, :to, :offset,
:limit). Re-running a window is an upsert, not a duplicate, because chunk
and document ids are derived from the source URL — that is the resumable
story: crash, re-run the window, continue.
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close