Liking cljdoc? Tell your friends :D

hive-weave.async

core.async go blocks whose failure is a VALUE.

safe-go / safe-go-loop return a channel yielding a hive-dsl Result:

{:ok v} {:error :weave/exception :name label :message .. :class ..} {:error :weave/timeout :name label :timeout-ms ms}

Park ops (<!, >!, alts!) work normally inside the body. There is no safe-go-call counterpart to hive-weave.safe/safe-future-call: the body must stay lexically inside a/go.

bound-go / bound-go-loop are DEPRECATED since 0.3.0.

core.async go blocks whose failure is a VALUE.

`safe-go` / `safe-go-loop` return a channel yielding a hive-dsl Result:

  {:ok v}
  {:error :weave/exception :name label :message .. :class ..}
  {:error :weave/timeout   :name label :timeout-ms ms}

Park ops (`<!`, `>!`, `alts!`) work normally inside the body. There is no
`safe-go-call` counterpart to `hive-weave.safe/safe-future-call`: the body
must stay lexically inside `a/go`.

`bound-go` / `bound-go-loop` are DEPRECATED since 0.3.0.
raw docstring

hive-weave.async.schema

Malli value objects for hive-weave.async. Drives the m/=> contracts on the pure constructors and the synthesized tests.

Malli value objects for hive-weave.async. Drives the m/=> contracts on the
pure constructors and the synthesized tests.
raw docstring

hive-weave.broker

Resource broker — combines a budget gate (cost-based admission) with a heap sentinel (JVM pressure signal). Consumers (hive-mcp, hive-knowledge, ...) depend on IResourceBroker; the default impl applies a policy table at submit-time:

:normal → pass through with caller's timeout :high → extend timeout (heap pressure: give GC headroom) :critical → reject-fast with :retry-after-ms (don't compound pressure)

Resource broker — combines a budget gate (cost-based admission) with a heap
sentinel (JVM pressure signal). Consumers (hive-mcp, hive-knowledge, ...)
depend on IResourceBroker; the default impl applies a policy table at
submit-time:

  :normal   → pass through with caller's timeout
  :high     → extend timeout (heap pressure: give GC headroom)
  :critical → reject-fast with :retry-after-ms (don't compound pressure)
raw docstring

hive-weave.budget

Unit-agnostic budget gate. 1 permit = 1 unit; caller picks the unit (bytes/MiB/slots/etc). Saturation policy: block up to :timeout-ms, then (r/err :budget/timeout ...).

Unit-agnostic budget gate. 1 permit = 1 unit; caller picks the unit
(bytes/MiB/slots/etc). Saturation policy: block up to :timeout-ms,
then (r/err :budget/timeout ...).
raw docstring

hive-weave.core

hive-weave — bounded, timed, safe execution primitives.

Weaves parallel/concurrent/async strands with built-in safety: every operation has a timeout, every parallel fan-out is bounded.

Modules: hive-weave.safe — safe deref, safe future (never hangs) hive-weave.gate — semaphore-bounded execution hive-weave.parallel — bounded-pmap, fork-join, fan-out hive-weave.pool — bounded ThreadPoolExecutor + safe await! hive-weave.timed — timed interceptors, timed handlers

Quick start: (require '[hive-weave.core :as weave])

;; Safe deref (replaces bare @) (weave/deref-safe my-promise 5000 fallback)

;; Bounded parallel map (replaces pmap) (weave/bounded-pmap {:concurrency 4 :timeout-ms 5000} fetch-preview ids)

;; Fork-join with budget (weave/fork-join {:budget-ms 15000} [:tags #(query-tags tags) {}] [:kg #(expand-kg ids) #{}])

;; Gate for resource protection (def db-gate (weave/gate {:permits 4 :timeout-ms 10000 :name "db"})) (weave/with-gate db-gate (query ...))

;; Timed interceptor (for hive-events) (weave/->timed-interceptor :id :my/enrichment :timeout-ms 15000 :after (fn [ctx] ...))

hive-weave — bounded, timed, safe execution primitives.

Weaves parallel/concurrent/async strands with built-in safety:
every operation has a timeout, every parallel fan-out is bounded.

Modules:
  hive-weave.safe      — safe deref, safe future (never hangs)
  hive-weave.gate      — semaphore-bounded execution
  hive-weave.parallel  — bounded-pmap, fork-join, fan-out
  hive-weave.pool      — bounded ThreadPoolExecutor + safe await!
  hive-weave.timed     — timed interceptors, timed handlers

Quick start:
  (require '[hive-weave.core :as weave])

  ;; Safe deref (replaces bare @)
  (weave/deref-safe my-promise 5000 fallback)

  ;; Bounded parallel map (replaces pmap)
  (weave/bounded-pmap {:concurrency 4 :timeout-ms 5000}
    fetch-preview ids)

  ;; Fork-join with budget
  (weave/fork-join {:budget-ms 15000}
    [:tags  #(query-tags tags)     {}]
    [:kg    #(expand-kg ids)       #{}])

  ;; Gate for resource protection
  (def db-gate (weave/gate {:permits 4 :timeout-ms 10000 :name "db"}))
  (weave/with-gate db-gate (query ...))

  ;; Timed interceptor (for hive-events)
  (weave/->timed-interceptor
    :id :my/enrichment :timeout-ms 15000
    :after (fn [ctx] ...))
raw docstring

hive-weave.gate

Concurrency gate — bounded-permit execution with timeout.

A gate wraps a java.util.concurrent.Semaphore with:

  • Timeout-aware permit acquisition
  • Result integration (gate-run returns ok/err instead of throwing)
  • Diagnostics (available permits, queue length)

Three ways to use:

  1. with-gate — bracket macro, throws on timeout
  2. gate-run — function, returns Result
  3. deref-gate — gated deref for promises/futures with timeout

Also satisfies hive-weave.budget/IBudgetGate so consumers can depend on the unit-agnostic protocol regardless of whether the underlying gate is slot-permit-based (this ns) or byte-cost-based (hive-weave.budget).

Usage: (def db-read (gate {:permits 4 :timeout-ms 15000 :name "db-read"}))

(with-gate db-read (query-database ...)) (deref-gate db-read (chroma/query coll embedding))

Concurrency gate — bounded-permit execution with timeout.

A gate wraps a java.util.concurrent.Semaphore with:
- Timeout-aware permit acquisition
- Result integration (gate-run returns ok/err instead of throwing)
- Diagnostics (available permits, queue length)

Three ways to use:

1. `with-gate` — bracket macro, throws on timeout
2. `gate-run`  — function, returns Result
3. `deref-gate` — gated deref for promises/futures with timeout

Also satisfies hive-weave.budget/IBudgetGate so consumers can depend on
the unit-agnostic protocol regardless of whether the underlying gate is
slot-permit-based (this ns) or byte-cost-based (hive-weave.budget).

Usage:
  (def db-read (gate {:permits 4 :timeout-ms 15000 :name "db-read"}))

  (with-gate db-read (query-database ...))
  (deref-gate db-read (chroma/query coll embedding))
raw docstring

hive-weave.gpu

VRAM-budget gate. Thin alias of hive-weave.budget with :unit :mib; preserves the historical :gpu/* error keys for existing callers.

VRAM-budget gate. Thin alias of hive-weave.budget with :unit :mib;
preserves the historical :gpu/* error keys for existing callers.
raw docstring

hive-weave.guarded

Guarded execution — bounded futures/pool tasks with cleanup hooks.

Extends hive-weave.safe and hive-weave.pool with two missing pieces for production runs:

  • :on-cancel — a 0-arg cleanup thunk invoked when the task is killed by timeout. Runs on a separate cleanup pool so a hung task body can't block its own cleanup. Capped by :cleanup-timeout-ms. Used to release resources the task held (datahike connections, LSP probes, scan-state atoms, file locks).

  • :alert! — an injectable 1-arg fn invoked with a structured event map on timeout or exception. Keeps hive-weave decoupled from hive-events — callers wire their own emitter (telemetry, hivemind shout, dashboard, ...). When omitted, no alert side-effect runs.

Failure mode: fail loud — every guarded call returns a hive-dsl Result. Callers must explicitly opt out via their own try/catch if they want exceptions to surface raw.

Quick reference: (require '[hive-weave.guarded :as wg])

(wg/guarded-future-call {:timeout-ms 60000 :name "carto-scan-hive-knowledge" :on-cancel (fn [] (reset-scan-state! :hive-knowledge)) :alert! (fn [ev] (events/emit! [:weave/task-killed ev]))} (fn [] (run-scan-async! [:hive-knowledge])))

(wg/guarded-await! my-pool (fn [] (long-running-write)) {:timeout-ms 30000 :name "datahike-write" :on-cancel (fn [] (reopen-conn!)) :alert! telemetry-emit})

Guarded execution — bounded futures/pool tasks with cleanup hooks.

Extends `hive-weave.safe` and `hive-weave.pool` with two missing
pieces for production runs:

- `:on-cancel` — a 0-arg cleanup thunk invoked when the task is
  killed by timeout. Runs on a *separate* cleanup pool so a hung
  task body can't block its own cleanup. Capped by
  `:cleanup-timeout-ms`. Used to release resources the task held
  (datahike connections, LSP probes, scan-state atoms, file locks).

- `:alert!` — an injectable 1-arg fn invoked with a structured event
  map on timeout *or* exception. Keeps `hive-weave` decoupled from
  `hive-events` — callers wire their own emitter (telemetry, hivemind
  shout, dashboard, ...). When omitted, no alert side-effect runs.

Failure mode: fail loud — every guarded call returns a hive-dsl
Result. Callers must explicitly opt out via their own try/catch if
they want exceptions to surface raw.

Quick reference:
  (require '[hive-weave.guarded :as wg])

  (wg/guarded-future-call
    {:timeout-ms 60000
     :name       "carto-scan-hive-knowledge"
     :on-cancel  (fn [] (reset-scan-state! :hive-knowledge))
     :alert!     (fn [ev] (events/emit! [:weave/task-killed ev]))}
    (fn [] (run-scan-async! [:hive-knowledge])))

  (wg/guarded-await!
    my-pool
    (fn [] (long-running-write))
    {:timeout-ms 30000
     :name       "datahike-write"
     :on-cancel  (fn [] (reopen-conn!))
     :alert!     telemetry-emit})
raw docstring

hive-weave.heap

JVM heap pressure sentinel. Samples Runtime memory periodically, derives a 3-state pressure level (:normal :high :critical) with hysteresis, and publishes both to atoms that observers can read or subscribe to.

Does NOT enforce admission policy — that's the broker's job. The sentinel is the signal; consumers decide what to do with it.

JVM heap pressure sentinel. Samples Runtime memory periodically, derives a
3-state pressure level (:normal :high :critical) with hysteresis, and
publishes both to atoms that observers can read or subscribe to.

Does NOT enforce admission policy — that's the broker's job. The sentinel
is the signal; consumers decide what to do with it.
raw docstring

hive-weave.parallel

Bounded parallel execution — safe alternatives to pmap and raw futures.

  • bounded-pmap — pmap with concurrency limit + per-item timeout
  • fork-join — concurrent futures with collective timeout budget
  • fan-out — fire N tasks, collect results with timeout

Unlike pmap, these primitives:

  1. Bound concurrency (no unbounded thread creation)
  2. Have timeouts (no indefinite hangs)
  3. Return fallback values on timeout (graceful degradation)
Bounded parallel execution — safe alternatives to pmap and raw futures.

- `bounded-pmap`  — pmap with concurrency limit + per-item timeout
- `fork-join`     — concurrent futures with collective timeout budget
- `fan-out`       — fire N tasks, collect results with timeout

Unlike `pmap`, these primitives:
1. Bound concurrency (no unbounded thread creation)
2. Have timeouts (no indefinite hangs)
3. Return fallback values on timeout (graceful degradation)
raw docstring

hive-weave.pool

Bounded thread-pool primitives — factory + safe submit/await.

Extends hive-weave with a pool abstraction so downstream code does not reach into java.util.concurrent directly (DIP).

Responsibilities:

  • Construct a bounded ThreadPoolExecutor with CallerRunsPolicy backpressure and a named thread factory (for JVM diagnostics).
  • Expose submit! returning an opaque Future-like handle.
  • Expose await! — submit + block up to a timeout, returning a fallback on timeout/error. Never hangs.
  • Re-export pool-stats and shutdown! for lifecycle.

Callers keep pool instances in their own registry (e.g. named io/compute/event/memory pools) and hand them to await! when they need bounded, isolated execution for a piece of work.

Quick start: (require '[hive-weave.pool :as wp])

(def db-pool (wp/make-pool {:name "db" :size 8}))

(wp/await! db-pool (fn [] (query-database ...)) {:timeout-ms 5000 :fallback ::db-timeout}) ;; => result or ::db-timeout

Bounded thread-pool primitives — factory + safe submit/await.

Extends hive-weave with a pool abstraction so downstream code does
not reach into java.util.concurrent directly (DIP).

Responsibilities:
- Construct a bounded `ThreadPoolExecutor` with CallerRunsPolicy
  backpressure and a named thread factory (for JVM diagnostics).
- Expose `submit!` returning an opaque Future-like handle.
- Expose `await!` — submit + block up to a timeout, returning a
  fallback on timeout/error. Never hangs.
- Re-export `pool-stats` and `shutdown!` for lifecycle.

Callers keep pool *instances* in their own registry (e.g. named
io/compute/event/memory pools) and hand them to `await!` when they
need bounded, isolated execution for a piece of work.

Quick start:
  (require '[hive-weave.pool :as wp])

  (def db-pool (wp/make-pool {:name "db" :size 8}))

  (wp/await! db-pool
             (fn [] (query-database ...))
             {:timeout-ms 5000 :fallback ::db-timeout})
  ;; => result or ::db-timeout
raw docstring

hive-weave.retry

Bounded retry with pluggable recovery.

with-recovery runs a thunk under a safe-future-call timeout. On non-timeout failure, calls recover! once and retries the thunk. Timeouts surface immediately because retry only doubles latency when the operation is alive but slow; on a dead resource it is the recovery hook (reopen connection, recreate client, refresh cache) that restores liveness, not another attempt at the same call.

Generalizes the read-with-retry / write-with-retry pattern from hive-mcp.knowledge-graph.store.datahike so other stores (Milvus/Qdrant/Chroma/NATS) can opt into the same auto-heal contract without rewriting the classify-and-reopen loop.

Design rules (anchored in repo memory):

  • Fail loud (decision 20260428174346-1a85b1ae): terminal failure throws — never silently substitute a default. Callers can opt out via :on-failure returning a Result, but the default is throw.
  • No silent-drop (principle 20260413204752-2d78b124): the failure shape is a hive-dsl Result, not a sentinel value, so callers that opt out can still distinguish 'no data' from 'call failed'.
  • Single-retry by design: needs more attempts? Compose with hive-weave.parallel or add an explicit :max-attempts knob when a real caller demands it. Don't make this fn a swiss-army knife.
Bounded retry with pluggable recovery.

`with-recovery` runs a thunk under a `safe-future-call` timeout.
On non-timeout failure, calls `recover!` once and retries the thunk.
Timeouts surface immediately because retry only doubles latency
when the operation is alive but slow; on a dead resource it is the
recovery hook (reopen connection, recreate client, refresh cache)
that restores liveness, not another attempt at the same call.

Generalizes the `read-with-retry` / `write-with-retry` pattern from
`hive-mcp.knowledge-graph.store.datahike` so other stores
(Milvus/Qdrant/Chroma/NATS) can opt into the same auto-heal
contract without rewriting the classify-and-reopen loop.

Design rules (anchored in repo memory):

- **Fail loud** (decision 20260428174346-1a85b1ae): terminal failure
  throws — never silently substitute a default. Callers can opt out
  via `:on-failure` returning a Result, but the *default* is throw.
- **No silent-drop** (principle 20260413204752-2d78b124): the
  failure shape is a hive-dsl Result, not a sentinel value, so
  callers that opt out can still distinguish 'no data' from
  'call failed'.
- **Single-retry by design**: needs more attempts? Compose with
  `hive-weave.parallel` or add an explicit `:max-attempts` knob
  when a real caller demands it. Don't make this fn a swiss-army
  knife.
raw docstring

hive-weave.safe

Safe execution primitives — the antidote to bare @ and raw future.

Every bare @(future ...) or @(promise) is a potential hang. This namespace provides bounded alternatives that always terminate:

  • deref-safe — deref with timeout + fallback (never hangs)
  • deref-safe! — deref with timeout, throws on timeout (never hangs)
  • safe-future — future with timeout + Result return
  • safe-future! — future with timeout, throws on timeout

All primitives return within their timeout budget. No exceptions.

Safe execution primitives — the antidote to bare @ and raw future.

Every bare `@(future ...)` or `@(promise)` is a potential hang.
This namespace provides bounded alternatives that always terminate:

- `deref-safe`  — deref with timeout + fallback (never hangs)
- `deref-safe!` — deref with timeout, throws on timeout (never hangs)
- `safe-future`  — future with timeout + Result return
- `safe-future!` — future with timeout, throws on timeout

All primitives return within their timeout budget. No exceptions.
raw docstring

hive-weave.serializer

FIFO serializer — single-writer queue for resources that don't tolerate concurrent writers (konserve filestore, datahike writer, external services with strict request ordering).

When to reach for this

  • A gate (hive-weave.gate) is the right choice when the caller needs the result back synchronously and you can afford to block on the permit. Concurrent reads, bounded fan-out, etc.
  • A pool (hive-weave.pool) is the right choice when work items are independent and ordering doesn't matter.
  • A serializer is the right choice when:
    1. Concurrent execution corrupts the resource (konserve .ksv.new -> .ksv rename race; SQLite WAL on a single file; an HTTP API with strict request ordering).
    2. Callers don't need the result synchronously — they fire-and- forget and check completion via a side channel (a promise the submitter can deref later, or just log on failure).
    3. Ordering matters (FIFO).
    4. Repeated submissions for the same key can be coalesced (only the latest matters — e.g. an upsert that's idempotent on a unique key, where an in-flight submission about to be overwritten can be dropped).

Type model

Two closed ADTs make the message and outcome surface explicit:

SerializerMsg — what flows through the queue :msg/task { :key, :f, :promise } :msg/poison

SubmitOutcome — what submit! returns :submit/ok { :promise } :submit/timeout { :queue-size :timeout-ms } :submit/closed

TaskOutcome — what the submission promise resolves to :task/ok { :value } :task/failed { :class :message }

Domain ADTs make the worker loop a adt-case exhaustive match — no string-typing, no half-cases.

Backpressure model

The serializer holds a single dedicated worker thread + a bounded LinkedBlockingQueue. submit! enqueues; if the queue is full, the submitter blocks up to :submit-timeout-ms waiting for space (true backpressure — the caller can't outrun the worker).

Coalescing

When :coalesce-key-fn is provided (or a :key is passed to submit!), every submission carries a key. Before enqueuing, the queue is scanned for a pending task with the same key; if found, the pending task is replaced by the new one. Coalescing is opt-in.

Lifecycle

serializer returns a record. close! shuts down the worker gracefully (drains the queue then exits via a :msg/poison message). Repeated close! calls are no-ops. After close, submit! returns (submit-outcome :submit/closed).

FIFO serializer — single-writer queue for resources that don't
tolerate concurrent writers (konserve filestore, datahike writer,
external services with strict request ordering).

## When to reach for this

- **A `gate`** (`hive-weave.gate`) is the right choice when the
  caller needs the result back synchronously and you can afford
  to block on the permit. Concurrent reads, bounded fan-out, etc.
- **A `pool`** (`hive-weave.pool`) is the right choice when work
  items are independent and ordering doesn't matter.
- **A serializer is the right choice when**:
  1. Concurrent execution corrupts the resource (konserve `.ksv.new
     -> .ksv` rename race; SQLite WAL on a single file; an HTTP API
     with strict request ordering).
  2. Callers don't need the result synchronously — they fire-and-
     forget and check completion via a side channel (a promise the
     submitter can deref later, or just log on failure).
  3. Ordering matters (FIFO).
  4. Repeated submissions for the same key can be coalesced (only
     the latest matters — e.g. an upsert that's idempotent on a
     unique key, where an in-flight submission about to be
     overwritten can be dropped).

## Type model

Two closed ADTs make the message and outcome surface explicit:

  SerializerMsg — what flows through the queue
    :msg/task         { :key, :f, :promise }
    :msg/poison

  SubmitOutcome — what `submit!` returns
    :submit/ok        { :promise }
    :submit/timeout   { :queue-size :timeout-ms }
    :submit/closed

  TaskOutcome   — what the submission promise resolves to
    :task/ok          { :value }
    :task/failed      { :class :message }

Domain ADTs make the worker loop a `adt-case` exhaustive match —
no string-typing, no half-cases.

## Backpressure model

The serializer holds a single dedicated worker thread + a bounded
`LinkedBlockingQueue`. `submit!` enqueues; if the queue is full,
the submitter **blocks up to `:submit-timeout-ms`** waiting for
space (true backpressure — the caller can't outrun the worker).

## Coalescing

When `:coalesce-key-fn` is provided (or a `:key` is passed to
`submit!`), every submission carries a key. Before enqueuing,
the queue is scanned for a pending task with the same key; if
found, the pending task is **replaced** by the new one. Coalescing
is opt-in.

## Lifecycle

`serializer` returns a record. `close!` shuts down the worker
gracefully (drains the queue then exits via a `:msg/poison`
message). Repeated `close!` calls are no-ops. After close,
`submit!` returns `(submit-outcome :submit/closed)`.
raw docstring

hive-weave.timed

Timed wrappers for interceptors and handlers.

Wraps any function with a timeout budget so it can't hang the caller. Primary use: event system interceptors that may call external services.

  • wrap-timed — wrap any (fn [x] -> x) with timeout
  • ->timed-interceptor — interceptor with timeout on :before/:after
  • timed-handler — handler with timeout + fallback
Timed wrappers for interceptors and handlers.

Wraps any function with a timeout budget so it can't hang the caller.
Primary use: event system interceptors that may call external services.

- `wrap-timed`            — wrap any (fn [x] -> x) with timeout
- `->timed-interceptor`   — interceptor with timeout on :before/:after
- `timed-handler`         — handler with timeout + fallback
raw docstring

hive-weave.typed-anns

Annotations-as-data for the :typed rung of hive-weave.async. Checked by the :typed alias; the checker itself is never a runtime dep.

Annotations-as-data for the :typed rung of hive-weave.async.
Checked by the :typed alias; the checker itself is never a runtime dep.
raw docstring

No vars found in this namespace.

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close