One poll of one feed, and one pass over every due feed.
Effects arrive as functions so the pipeline runs the same against the network and the memory store as against test doubles:
:fetch (fn [url {:keys [etag last-modified timeout-ms max-bytes]}]) -> Result :file! (fn [entry]) -> Result {:id :duplicate?} :deliver! (fn [settings subscription feed items]) -> delivery, optional: replaces filing entry by entry (hive-ingestor delivery). A delivery is {:filed [item-key] :created :duplicates :failed :last-error} :save! (fn [state]) persists the whole state :now (fn []) -> epoch seconds
One poll of one feed, and one pass over every due feed.
Effects arrive as functions so the pipeline runs the same against the
network and the memory store as against test doubles:
:fetch (fn [url {:keys [etag last-modified timeout-ms max-bytes]}]) -> Result
:file! (fn [entry]) -> Result {:id :duplicate?}
:deliver! (fn [settings subscription feed items]) -> delivery, optional:
replaces filing entry by entry (hive-ingestor delivery).
A delivery is {:filed [item-key] :created :duplicates :failed :last-error}
:save! (fn [state]) persists the whole state
:now (fn []) -> epoch seconds(file-items! {:keys [file!]} settings subscription feed items)Deliver items by filing each as a memory entry through file!: the keys
of those filed or already present.
Deliver `items` by filing each as a memory entry through `file!`: the keys of those filed or already present.
(poll-due! {:keys [now save!] :as deps} settings state {:keys [force? feed-id]})Poll every subscription due at now (all of them when force?), saving the
state after each. Returns {:state state' :reports [..]}.
A feed that throws is reported and the pass goes on to the next one.
Poll every subscription due at now (all of them when `force?`), saving the
state after each. Returns `{:state state' :reports [..]}`.
A feed that throws is reported and the pass goes on to the next one.(poll-feed! {:keys [fetch now] :as deps} settings subscription feed-state)Poll subscription once, with its :feed/auth when it has one. Returns
{:state feed-state' :report report}.
An item's key joins the seen set only once it is in memory (created or
already there), so an item the store refused is offered again next poll. A
poll whose fetch or parse fails keeps the last successful poll time and
records the attempt, which feed-due? turns into a retry.
Poll `subscription` once, with its `:feed/auth` when it has one. Returns
`{:state feed-state' :report report}`.
An item's key joins the seen set only once it is in memory (created or
already there), so an item the store refused is offered again next poll. A
poll whose fetch or parse fails keeps the last successful poll time and
records the attempt, which `feed-due?` turns into a retry.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |