core.async go blocks whose failure is a VALUE.
safe-go / safe-go-loop return a channel yielding a hive-dsl Result:
{:ok v} {:error :weave/exception :name label :message .. :class ..} {:error :weave/timeout :name label :timeout-ms ms}
Park ops (<!, >!, alts!) work normally inside the body. There is no
safe-go-call counterpart to hive-weave.safe/safe-future-call: the body
must stay lexically inside a/go.
bound-go / bound-go-loop are DEPRECATED since 0.3.0.
core.async go blocks whose failure is a VALUE.
`safe-go` / `safe-go-loop` return a channel yielding a hive-dsl Result:
{:ok v}
{:error :weave/exception :name label :message .. :class ..}
{:error :weave/timeout :name label :timeout-ms ms}
Park ops (`<!`, `>!`, `alts!`) work normally inside the body. There is no
`safe-go-call` counterpart to `hive-weave.safe/safe-future-call`: the body
must stay lexically inside `a/go`.
`bound-go` / `bound-go-loop` are DEPRECATED since 0.3.0.Malli value objects for hive-weave.async. Drives the m/=> contracts on the pure constructors and the synthesized tests.
Malli value objects for hive-weave.async. Drives the m/=> contracts on the pure constructors and the synthesized tests.
Resource broker — combines a budget gate (cost-based admission) with a heap sentinel (JVM pressure signal). Consumers (hive-mcp, hive-knowledge, ...) depend on IResourceBroker; the default impl applies a policy table at submit-time:
:normal → pass through with caller's timeout :high → extend timeout (heap pressure: give GC headroom) :critical → reject-fast with :retry-after-ms (don't compound pressure)
Resource broker — combines a budget gate (cost-based admission) with a heap sentinel (JVM pressure signal). Consumers (hive-mcp, hive-knowledge, ...) depend on IResourceBroker; the default impl applies a policy table at submit-time: :normal → pass through with caller's timeout :high → extend timeout (heap pressure: give GC headroom) :critical → reject-fast with :retry-after-ms (don't compound pressure)
Unit-agnostic budget gate. 1 permit = 1 unit; caller picks the unit (bytes/MiB/slots/etc). Saturation policy: block up to :timeout-ms, then (r/err :budget/timeout ...).
Unit-agnostic budget gate. 1 permit = 1 unit; caller picks the unit (bytes/MiB/slots/etc). Saturation policy: block up to :timeout-ms, then (r/err :budget/timeout ...).
hive-weave — bounded, timed, safe execution primitives.
Weaves parallel/concurrent/async strands with built-in safety: every operation has a timeout, every parallel fan-out is bounded.
Modules: hive-weave.safe — safe deref, safe future (never hangs) hive-weave.gate — semaphore-bounded execution hive-weave.parallel — bounded-pmap, fork-join, fan-out hive-weave.pool — bounded ThreadPoolExecutor + safe await! hive-weave.timed — timed interceptors, timed handlers
Quick start: (require '[hive-weave.core :as weave])
;; Safe deref (replaces bare @) (weave/deref-safe my-promise 5000 fallback)
;; Bounded parallel map (replaces pmap) (weave/bounded-pmap {:concurrency 4 :timeout-ms 5000} fetch-preview ids)
;; Fork-join with budget (weave/fork-join {:budget-ms 15000} [:tags #(query-tags tags) {}] [:kg #(expand-kg ids) #{}])
;; Gate for resource protection (def db-gate (weave/gate {:permits 4 :timeout-ms 10000 :name "db"})) (weave/with-gate db-gate (query ...))
;; Timed interceptor (for hive-events) (weave/->timed-interceptor :id :my/enrichment :timeout-ms 15000 :after (fn [ctx] ...))
hive-weave — bounded, timed, safe execution primitives.
Weaves parallel/concurrent/async strands with built-in safety:
every operation has a timeout, every parallel fan-out is bounded.
Modules:
hive-weave.safe — safe deref, safe future (never hangs)
hive-weave.gate — semaphore-bounded execution
hive-weave.parallel — bounded-pmap, fork-join, fan-out
hive-weave.pool — bounded ThreadPoolExecutor + safe await!
hive-weave.timed — timed interceptors, timed handlers
Quick start:
(require '[hive-weave.core :as weave])
;; Safe deref (replaces bare @)
(weave/deref-safe my-promise 5000 fallback)
;; Bounded parallel map (replaces pmap)
(weave/bounded-pmap {:concurrency 4 :timeout-ms 5000}
fetch-preview ids)
;; Fork-join with budget
(weave/fork-join {:budget-ms 15000}
[:tags #(query-tags tags) {}]
[:kg #(expand-kg ids) #{}])
;; Gate for resource protection
(def db-gate (weave/gate {:permits 4 :timeout-ms 10000 :name "db"}))
(weave/with-gate db-gate (query ...))
;; Timed interceptor (for hive-events)
(weave/->timed-interceptor
:id :my/enrichment :timeout-ms 15000
:after (fn [ctx] ...))Concurrency gate — bounded-permit execution with timeout.
A gate wraps a java.util.concurrent.Semaphore with:
Three ways to use:
with-gate — bracket macro, throws on timeoutgate-run — function, returns Resultderef-gate — gated deref for promises/futures with timeoutAlso satisfies hive-weave.budget/IBudgetGate so consumers can depend on the unit-agnostic protocol regardless of whether the underlying gate is slot-permit-based (this ns) or byte-cost-based (hive-weave.budget).
Usage: (def db-read (gate {:permits 4 :timeout-ms 15000 :name "db-read"}))
(with-gate db-read (query-database ...)) (deref-gate db-read (chroma/query coll embedding))
Concurrency gate — bounded-permit execution with timeout.
A gate wraps a java.util.concurrent.Semaphore with:
- Timeout-aware permit acquisition
- Result integration (gate-run returns ok/err instead of throwing)
- Diagnostics (available permits, queue length)
Three ways to use:
1. `with-gate` — bracket macro, throws on timeout
2. `gate-run` — function, returns Result
3. `deref-gate` — gated deref for promises/futures with timeout
Also satisfies hive-weave.budget/IBudgetGate so consumers can depend on
the unit-agnostic protocol regardless of whether the underlying gate is
slot-permit-based (this ns) or byte-cost-based (hive-weave.budget).
Usage:
(def db-read (gate {:permits 4 :timeout-ms 15000 :name "db-read"}))
(with-gate db-read (query-database ...))
(deref-gate db-read (chroma/query coll embedding))VRAM-budget gate. Thin alias of hive-weave.budget with :unit :mib; preserves the historical :gpu/* error keys for existing callers.
VRAM-budget gate. Thin alias of hive-weave.budget with :unit :mib; preserves the historical :gpu/* error keys for existing callers.
Guarded execution — bounded futures/pool tasks with cleanup hooks.
Extends hive-weave.safe and hive-weave.pool with two missing
pieces for production runs:
:on-cancel — a 0-arg cleanup thunk invoked when the task is
killed by timeout. Runs on a separate cleanup pool so a hung
task body can't block its own cleanup. Capped by
:cleanup-timeout-ms. Used to release resources the task held
(datahike connections, LSP probes, scan-state atoms, file locks).
:alert! — an injectable 1-arg fn invoked with a structured event
map on timeout or exception. Keeps hive-weave decoupled from
hive-events — callers wire their own emitter (telemetry, hivemind
shout, dashboard, ...). When omitted, no alert side-effect runs.
Failure mode: fail loud — every guarded call returns a hive-dsl Result. Callers must explicitly opt out via their own try/catch if they want exceptions to surface raw.
Quick reference: (require '[hive-weave.guarded :as wg])
(wg/guarded-future-call {:timeout-ms 60000 :name "carto-scan-hive-knowledge" :on-cancel (fn [] (reset-scan-state! :hive-knowledge)) :alert! (fn [ev] (events/emit! [:weave/task-killed ev]))} (fn [] (run-scan-async! [:hive-knowledge])))
(wg/guarded-await! my-pool (fn [] (long-running-write)) {:timeout-ms 30000 :name "datahike-write" :on-cancel (fn [] (reopen-conn!)) :alert! telemetry-emit})
Guarded execution — bounded futures/pool tasks with cleanup hooks.
Extends `hive-weave.safe` and `hive-weave.pool` with two missing
pieces for production runs:
- `:on-cancel` — a 0-arg cleanup thunk invoked when the task is
killed by timeout. Runs on a *separate* cleanup pool so a hung
task body can't block its own cleanup. Capped by
`:cleanup-timeout-ms`. Used to release resources the task held
(datahike connections, LSP probes, scan-state atoms, file locks).
- `:alert!` — an injectable 1-arg fn invoked with a structured event
map on timeout *or* exception. Keeps `hive-weave` decoupled from
`hive-events` — callers wire their own emitter (telemetry, hivemind
shout, dashboard, ...). When omitted, no alert side-effect runs.
Failure mode: fail loud — every guarded call returns a hive-dsl
Result. Callers must explicitly opt out via their own try/catch if
they want exceptions to surface raw.
Quick reference:
(require '[hive-weave.guarded :as wg])
(wg/guarded-future-call
{:timeout-ms 60000
:name "carto-scan-hive-knowledge"
:on-cancel (fn [] (reset-scan-state! :hive-knowledge))
:alert! (fn [ev] (events/emit! [:weave/task-killed ev]))}
(fn [] (run-scan-async! [:hive-knowledge])))
(wg/guarded-await!
my-pool
(fn [] (long-running-write))
{:timeout-ms 30000
:name "datahike-write"
:on-cancel (fn [] (reopen-conn!))
:alert! telemetry-emit})JVM heap pressure sentinel. Samples Runtime memory periodically, derives a 3-state pressure level (:normal :high :critical) with hysteresis, and publishes both to atoms that observers can read or subscribe to.
Does NOT enforce admission policy — that's the broker's job. The sentinel is the signal; consumers decide what to do with it.
JVM heap pressure sentinel. Samples Runtime memory periodically, derives a 3-state pressure level (:normal :high :critical) with hysteresis, and publishes both to atoms that observers can read or subscribe to. Does NOT enforce admission policy — that's the broker's job. The sentinel is the signal; consumers decide what to do with it.
Bounded parallel execution — safe alternatives to pmap and raw futures.
bounded-pmap — pmap with concurrency limit + per-item timeoutfork-join — concurrent futures with collective timeout budgetfan-out — fire N tasks, collect results with timeoutUnlike pmap, these primitives:
Bounded parallel execution — safe alternatives to pmap and raw futures. - `bounded-pmap` — pmap with concurrency limit + per-item timeout - `fork-join` — concurrent futures with collective timeout budget - `fan-out` — fire N tasks, collect results with timeout Unlike `pmap`, these primitives: 1. Bound concurrency (no unbounded thread creation) 2. Have timeouts (no indefinite hangs) 3. Return fallback values on timeout (graceful degradation)
Bounded thread-pool primitives — factory + safe submit/await.
Extends hive-weave with a pool abstraction so downstream code does not reach into java.util.concurrent directly (DIP).
Responsibilities:
ThreadPoolExecutor with CallerRunsPolicy
backpressure and a named thread factory (for JVM diagnostics).submit! returning an opaque Future-like handle.await! — submit + block up to a timeout, returning a
fallback on timeout/error. Never hangs.pool-stats and shutdown! for lifecycle.Callers keep pool instances in their own registry (e.g. named
io/compute/event/memory pools) and hand them to await! when they
need bounded, isolated execution for a piece of work.
Quick start: (require '[hive-weave.pool :as wp])
(def db-pool (wp/make-pool {:name "db" :size 8}))
(wp/await! db-pool (fn [] (query-database ...)) {:timeout-ms 5000 :fallback ::db-timeout}) ;; => result or ::db-timeout
Bounded thread-pool primitives — factory + safe submit/await.
Extends hive-weave with a pool abstraction so downstream code does
not reach into java.util.concurrent directly (DIP).
Responsibilities:
- Construct a bounded `ThreadPoolExecutor` with CallerRunsPolicy
backpressure and a named thread factory (for JVM diagnostics).
- Expose `submit!` returning an opaque Future-like handle.
- Expose `await!` — submit + block up to a timeout, returning a
fallback on timeout/error. Never hangs.
- Re-export `pool-stats` and `shutdown!` for lifecycle.
Callers keep pool *instances* in their own registry (e.g. named
io/compute/event/memory pools) and hand them to `await!` when they
need bounded, isolated execution for a piece of work.
Quick start:
(require '[hive-weave.pool :as wp])
(def db-pool (wp/make-pool {:name "db" :size 8}))
(wp/await! db-pool
(fn [] (query-database ...))
{:timeout-ms 5000 :fallback ::db-timeout})
;; => result or ::db-timeoutBounded retry with pluggable recovery.
with-recovery runs a thunk under a safe-future-call timeout.
On non-timeout failure, calls recover! once and retries the thunk.
Timeouts surface immediately because retry only doubles latency
when the operation is alive but slow; on a dead resource it is the
recovery hook (reopen connection, recreate client, refresh cache)
that restores liveness, not another attempt at the same call.
Generalizes the read-with-retry / write-with-retry pattern from
hive-mcp.knowledge-graph.store.datahike so other stores
(Milvus/Qdrant/Chroma/NATS) can opt into the same auto-heal
contract without rewriting the classify-and-reopen loop.
Design rules (anchored in repo memory):
:on-failure returning a Result, but the default is throw.hive-weave.parallel or add an explicit :max-attempts knob
when a real caller demands it. Don't make this fn a swiss-army
knife.Bounded retry with pluggable recovery. `with-recovery` runs a thunk under a `safe-future-call` timeout. On non-timeout failure, calls `recover!` once and retries the thunk. Timeouts surface immediately because retry only doubles latency when the operation is alive but slow; on a dead resource it is the recovery hook (reopen connection, recreate client, refresh cache) that restores liveness, not another attempt at the same call. Generalizes the `read-with-retry` / `write-with-retry` pattern from `hive-mcp.knowledge-graph.store.datahike` so other stores (Milvus/Qdrant/Chroma/NATS) can opt into the same auto-heal contract without rewriting the classify-and-reopen loop. Design rules (anchored in repo memory): - **Fail loud** (decision 20260428174346-1a85b1ae): terminal failure throws — never silently substitute a default. Callers can opt out via `:on-failure` returning a Result, but the *default* is throw. - **No silent-drop** (principle 20260413204752-2d78b124): the failure shape is a hive-dsl Result, not a sentinel value, so callers that opt out can still distinguish 'no data' from 'call failed'. - **Single-retry by design**: needs more attempts? Compose with `hive-weave.parallel` or add an explicit `:max-attempts` knob when a real caller demands it. Don't make this fn a swiss-army knife.
Safe execution primitives — the antidote to bare @ and raw future.
Every bare @(future ...) or @(promise) is a potential hang.
This namespace provides bounded alternatives that always terminate:
deref-safe — deref with timeout + fallback (never hangs)deref-safe! — deref with timeout, throws on timeout (never hangs)safe-future — future with timeout + Result returnsafe-future! — future with timeout, throws on timeoutAll primitives return within their timeout budget. No exceptions.
Safe execution primitives — the antidote to bare @ and raw future. Every bare `@(future ...)` or `@(promise)` is a potential hang. This namespace provides bounded alternatives that always terminate: - `deref-safe` — deref with timeout + fallback (never hangs) - `deref-safe!` — deref with timeout, throws on timeout (never hangs) - `safe-future` — future with timeout + Result return - `safe-future!` — future with timeout, throws on timeout All primitives return within their timeout budget. No exceptions.
FIFO serializer — single-writer queue for resources that don't tolerate concurrent writers (konserve filestore, datahike writer, external services with strict request ordering).
gate (hive-weave.gate) is the right choice when the
caller needs the result back synchronously and you can afford
to block on the permit. Concurrent reads, bounded fan-out, etc.pool (hive-weave.pool) is the right choice when work
items are independent and ordering doesn't matter..ksv.new -> .ksv rename race; SQLite WAL on a single file; an HTTP API
with strict request ordering).Two closed ADTs make the message and outcome surface explicit:
SerializerMsg — what flows through the queue :msg/task { :key, :f, :promise } :msg/poison
SubmitOutcome — what submit! returns
:submit/ok { :promise }
:submit/timeout { :queue-size :timeout-ms }
:submit/closed
TaskOutcome — what the submission promise resolves to :task/ok { :value } :task/failed { :class :message }
Domain ADTs make the worker loop a adt-case exhaustive match —
no string-typing, no half-cases.
The serializer holds a single dedicated worker thread + a bounded
LinkedBlockingQueue. submit! enqueues; if the queue is full,
the submitter blocks up to :submit-timeout-ms waiting for
space (true backpressure — the caller can't outrun the worker).
When :coalesce-key-fn is provided (or a :key is passed to
submit!), every submission carries a key. Before enqueuing,
the queue is scanned for a pending task with the same key; if
found, the pending task is replaced by the new one. Coalescing
is opt-in.
serializer returns a record. close! shuts down the worker
gracefully (drains the queue then exits via a :msg/poison
message). Repeated close! calls are no-ops. After close,
submit! returns (submit-outcome :submit/closed).
FIFO serializer — single-writer queue for resources that don't
tolerate concurrent writers (konserve filestore, datahike writer,
external services with strict request ordering).
## When to reach for this
- **A `gate`** (`hive-weave.gate`) is the right choice when the
caller needs the result back synchronously and you can afford
to block on the permit. Concurrent reads, bounded fan-out, etc.
- **A `pool`** (`hive-weave.pool`) is the right choice when work
items are independent and ordering doesn't matter.
- **A serializer is the right choice when**:
1. Concurrent execution corrupts the resource (konserve `.ksv.new
-> .ksv` rename race; SQLite WAL on a single file; an HTTP API
with strict request ordering).
2. Callers don't need the result synchronously — they fire-and-
forget and check completion via a side channel (a promise the
submitter can deref later, or just log on failure).
3. Ordering matters (FIFO).
4. Repeated submissions for the same key can be coalesced (only
the latest matters — e.g. an upsert that's idempotent on a
unique key, where an in-flight submission about to be
overwritten can be dropped).
## Type model
Two closed ADTs make the message and outcome surface explicit:
SerializerMsg — what flows through the queue
:msg/task { :key, :f, :promise }
:msg/poison
SubmitOutcome — what `submit!` returns
:submit/ok { :promise }
:submit/timeout { :queue-size :timeout-ms }
:submit/closed
TaskOutcome — what the submission promise resolves to
:task/ok { :value }
:task/failed { :class :message }
Domain ADTs make the worker loop a `adt-case` exhaustive match —
no string-typing, no half-cases.
## Backpressure model
The serializer holds a single dedicated worker thread + a bounded
`LinkedBlockingQueue`. `submit!` enqueues; if the queue is full,
the submitter **blocks up to `:submit-timeout-ms`** waiting for
space (true backpressure — the caller can't outrun the worker).
## Coalescing
When `:coalesce-key-fn` is provided (or a `:key` is passed to
`submit!`), every submission carries a key. Before enqueuing,
the queue is scanned for a pending task with the same key; if
found, the pending task is **replaced** by the new one. Coalescing
is opt-in.
## Lifecycle
`serializer` returns a record. `close!` shuts down the worker
gracefully (drains the queue then exits via a `:msg/poison`
message). Repeated `close!` calls are no-ops. After close,
`submit!` returns `(submit-outcome :submit/closed)`.Timed wrappers for interceptors and handlers.
Wraps any function with a timeout budget so it can't hang the caller. Primary use: event system interceptors that may call external services.
wrap-timed — wrap any (fn [x] -> x) with timeout->timed-interceptor — interceptor with timeout on :before/:aftertimed-handler — handler with timeout + fallbackTimed wrappers for interceptors and handlers. Wraps any function with a timeout budget so it can't hang the caller. Primary use: event system interceptors that may call external services. - `wrap-timed` — wrap any (fn [x] -> x) with timeout - `->timed-interceptor` — interceptor with timeout on :before/:after - `timed-handler` — handler with timeout + fallback
Annotations-as-data for the :typed rung of hive-weave.async. Checked by the :typed alias; the checker itself is never a runtime dep.
Annotations-as-data for the :typed rung of hive-weave.async. Checked by the :typed alias; the checker itself is never a runtime dep.
No vars found in this namespace.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |