Bounded Datahike calls: every read runs on ONE bounded executor under a deadline, with one reopen-and-retry on a non-timeout failure.
Why not hive-weave.retry/with-recovery: up to hive-weave 0.4.2 it calls
safe-future-call without a :pool, so every read is a clojure.core/future
on the unbounded send-off pool (measured: 114 send-off threads, the largest
CPU bucket in the JVM). safe-future-call itself takes :pool since 0.4.x,
which is all this namespace needs. Once a hive-weave release whose
with-recovery forwards :pool is pinned in deps.edn, read-with-retry can
collapse onto it with the same options.
The executor is a port: the store is built with one (:read-pool on
create-store) and tests hand it a recording one.
Bounded Datahike calls: every read runs on ONE bounded executor under a deadline, with one reopen-and-retry on a non-timeout failure. Why not `hive-weave.retry/with-recovery`: up to hive-weave 0.4.2 it calls `safe-future-call` without a :pool, so every read is a `clojure.core/future` on the unbounded send-off pool (measured: 114 send-off threads, the largest CPU bucket in the JVM). `safe-future-call` itself takes :pool since 0.4.x, which is all this namespace needs. Once a hive-weave release whose `with-recovery` forwards :pool is pinned in deps.edn, `read-with-retry` can collapse onto it with the same options. The executor is a port: the store is built with one (`:read-pool` on `create-store`) and tests hand it a recording one.
The running read's cancel token, bound on the pool thread for the duration of
one attempt; nil outside a read. An IDeref whose value is nil while the read
may go on and a cancellation map once it must stop: its caller gave up
(interrupted or timed out), the pool thread was interrupted, or the attempt's
deadline passed. Datahike's d/q takes it as :cancel; loops the store runs
itself call check-cancel!.
The running read's cancel token, bound on the pool thread for the duration of one attempt; nil outside a read. An IDeref whose value is nil while the read may go on and a cancellation map once it must stop: its caller gave up (interrupted or timed out), the pool thread was interrupted, or the attempt's deadline passed. Datahike's d/q takes it as :cancel; loops the store runs itself call `check-cancel!`.
(check-cancel!)Throw if the running read has been cancelled. A no-op outside a read.
Throw if the running read has been cancelled. A no-op outside a read.
(default-read-pool)The process-wide read pool, created on first use. Process-wide on purpose: the bound has to hold across stores and callers. :abort so a saturated pool sheds a read as an error instead of running it on the caller's thread, where the deadline cannot reach it. Its threads are daemons; it is never shut down.
The process-wide read pool, created on first use. Process-wide on purpose: the bound has to hold across stores and callers. :abort so a saturated pool sheds a read as an error instead of running it on the caller's thread, where the deadline cannot reach it. Its threads are daemons; it is never shut down.
Concurrent Datahike reads the shared pool runs. Reads are CPU and index-cache bound, so more threads than this buy contention, not throughput.
Concurrent Datahike reads the shared pool runs. Reads are CPU and index-cache bound, so more threads than this buy contention, not throughput.
Reads queued behind the running ones before a read is shed as :weave/rejected.
Reads queued behind the running ones before a read is shed as :weave/rejected.
(read-pool-of store)The pool store reads on: its own :read-pool when it was built with one,
else the process-wide default.
The pool `store` reads on: its own :read-pool when it was built with one, else the process-wide default.
(read-with-retry {:keys [timeout-ms pool name recover!]} f)Run read thunk f as ONE task on pool under timeout-ms. On a retryable
failure call recover! once (on the caller's thread) and run f once more,
again as one task. Returns the value or throws throw-read-failed!.
A read the caller abandons (timeout, or the caller's thread interrupted, as a
future-cancel of the caller does) is cancelled on the pool as well; inside
f, *cancel* is that read's token (see with-cancel, check-cancel!).
Options: :timeout-ms (pos-int, required), :pool (ExecutorService, required), :name (label), :recover! (0-arg fn).
Run read thunk `f` as ONE task on `pool` under `timeout-ms`. On a retryable failure call `recover!` once (on the caller's thread) and run `f` once more, again as one task. Returns the value or throws `throw-read-failed!`. A read the caller abandons (timeout, or the caller's thread interrupted, as a future-cancel of the caller does) is cancelled on the pool as well; inside `f`, `*cancel*` is that read's token (see `with-cancel`, `check-cancel!`). Options: :timeout-ms (pos-int, required), :pool (ExecutorService, required), :name (label), :recover! (0-arg fn).
(with-cancel q)Datahike query q carrying the running read's cancel token as :cancel, so
d/q stops at its next check point once the read is cancelled. A query map
(one with :query) keeps every key, and a :cancel of its own still applies; any
other query form becomes {:query q}. Outside a read, q unchanged.
Datahike query `q` carrying the running read's cancel token as :cancel, so
d/q stops at its next check point once the read is cancelled. A query map
(one with :query) keeps every key, and a :cancel of its own still applies; any
other query form becomes {:query q}. Outside a read, `q` unchanged.(write-with-retry {:keys [timeout-ms name recover!]} f)Bounded Datahike write with auto-heal. Per-attempt budget timeout-ms;
recover via recover! on non-timeout failure; timeouts surface immediately.
Bounded Datahike write with auto-heal. Per-attempt budget `timeout-ms`; recover via `recover!` on non-timeout failure; timeouts surface immediately.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |