Liking cljdoc? Tell your friends :D

hive-milvus.batch

Batch processing: embed entries + upsert to Milvus + persist cursor.

Batch processing: embed entries + upsert to Milvus + persist cursor.
raw docstring

hive-milvus.circuit

Circuit breaker for Milvus protocol operations.

States: :closed (normal) | :open (fail-fast) | :half-open (probing).

Flow:

  • :closed - calls pass through. N consecutive connection failures (threshold) -> :open with opened-at=now.
  • :open - check returns {:success? false :error :circuit-open :retry-after ms :reconnecting? true} until cooldown-ms has elapsed since opened-at.
  • cooldown elapsed - next check transitions to :half-open and allows the call through as a probe. record-success! closes the breaker and notifies subscribers; record-failure! reopens it (resets opened-at=now).

Subscribers: callers can subscribe! a 0-arity fn invoked on every :closed transition (open->closed or half-open->closed). Used by the queue drain module to flush buffered writes when Milvus is healthy.

Thread-safety: single atom, swap! transitions; listener fns run on the caller's thread that closed the breaker (keep them fast or spawn futures inside).

Time discipline: routes wall-clock through hive-ttracking.clock/now-millis so tests can pin time deterministically — never call System/currentTimeMillis directly.

Circuit breaker for Milvus protocol operations.

States: :closed (normal) | :open (fail-fast) | :half-open (probing).

Flow:
- :closed - calls pass through. N consecutive connection failures
  (threshold) -> :open with opened-at=now.
- :open - `check` returns {:success? false :error :circuit-open
  :retry-after ms :reconnecting? true} until cooldown-ms has elapsed
  since opened-at.
- cooldown elapsed - next `check` transitions to :half-open and allows
  the call through as a probe. `record-success!` closes the breaker
  and notifies subscribers; `record-failure!` reopens it (resets
  opened-at=now).

Subscribers: callers can `subscribe!` a 0-arity fn invoked on every
:closed transition (open->closed or half-open->closed). Used by the
queue drain module to flush buffered writes when Milvus is healthy.

Thread-safety: single atom, swap! transitions; listener fns run on
the caller's thread that closed the breaker (keep them fast or spawn
futures inside).

Time discipline: routes wall-clock through `hive-ttracking.clock/now-millis`
so tests can pin time deterministically — never call
`System/currentTimeMillis` directly.
raw docstring

hive-milvus.collection.config

Typed config for the collection bounded context, resolved via hive-di defconfig. Reads [:milvus :collections] block from ~/.config/hive-mcp/config.edn.

Two fields:

  • :sunset — set of Milvus collection names that are RETIRED. Locator returns them in known-collections (so reads still fan out to legacy data) but excludes from active-collections (so writes never land there).
  • :base-collection-name — Chroma form of the legacy 768-d collection. Default "hive-mcp-memory" mirrors hive-milvus.collection.naming/legacy-base-collection.
Typed config for the collection bounded context, resolved via
hive-di `defconfig`. Reads `[:milvus :collections]` block from
`~/.config/hive-mcp/config.edn`.

Two fields:

- `:sunset` — set of Milvus collection names that are RETIRED.
              Locator returns them in `known-collections` (so reads
              still fan out to legacy data) but excludes from
              `active-collections` (so writes never land there).
- `:base-collection-name` — Chroma form of the legacy 768-d
              collection. Default `"hive-mcp-memory"` mirrors
              `hive-milvus.collection.naming/legacy-base-collection`.
raw docstring

hive-milvus.collection.ensure

L3 — ICollectionEnsure impl. Idempotent collection creation with dim-check.

Wraps the imperative hive-milvus.store.index/ensure-collection! in the Result railway and adds an in-process dim-registry that rejects re-ensures requesting a different dimension for the same collection name, surfacing a dim mismatch at ensure! time rather than at the first failing upsert.

L3 — `ICollectionEnsure` impl. Idempotent collection creation with
dim-check.

Wraps the imperative `hive-milvus.store.index/ensure-collection!`
in the Result railway and adds an in-process dim-registry that
rejects re-ensures requesting a different dimension for the same
collection name, surfacing a dim mismatch at `ensure!` time rather
than at the first failing upsert.
raw docstring

hive-milvus.collection.invariant

THE DIMENSION INVARIANT: the width of the vector the embedder produces for a collection MUST equal the width that collection holds.

When it does not, nothing crashes. Milvus rejects an insert with error 1804, which a write path can swallow; and a SEARCH with a wrong-width query vector either errors or — the dangerous case, when another model happens to share the width — returns confident neighbours from a space the query was never embedded into. That is noise wearing the costume of an answer, and it is the shape the 2026-07-12 outage wore.

hive-milvus.store.search.boundary/CollectionEmbedder and hive-milvus.embedder/embed-for-entry already REFUSE to serve on a mismatch (:embedder/dim-mismatch) — read path and write path. That is the per-call half of the invariant, and it is the half that keeps a wrong vector out of the index.

This namespace is the OTHER half: assert it ONCE, at startup, so a misconfiguration is discovered when the server boots rather than on the first query of the day, silently, per call, forever.

Pure core / effectful shell: check — pure, one collection. violations — pure, a set of already-read (collection, expected, actual). verify — the boundary: reads live embedder dims, calls the pure core.

UNKNOWN IS NOT OK. naming/dim-of returns nil for a collection whose width is not knowable from its name (anything that is not a memory collection). Those are reported under :unknown — never folded into :ok. A checker that reports 'fine' when it means 'I could not tell' is the exact failure this file exists to prevent.

THE DIMENSION INVARIANT: the width of the vector the embedder produces for a
collection MUST equal the width that collection holds.

When it does not, nothing crashes. Milvus rejects an insert with error 1804,
which a write path can swallow; and a SEARCH with a wrong-width query vector
either errors or — the dangerous case, when another model happens to share
the width — returns confident neighbours from a space the query was never
embedded into. That is noise wearing the costume of an answer, and it is the
shape the 2026-07-12 outage wore.

`hive-milvus.store.search.boundary/CollectionEmbedder` and
`hive-milvus.embedder/embed-for-entry` already REFUSE to serve on a mismatch
(`:embedder/dim-mismatch`) — read path and write path. That is the per-call
half of the invariant, and it is the half that keeps a wrong vector out of
the index.

This namespace is the OTHER half: assert it ONCE, at startup, so a
misconfiguration is discovered when the server boots rather than on the
first query of the day, silently, per call, forever.

Pure core / effectful shell:
  `check`      — pure, one collection.
  `violations` — pure, a set of already-read (collection, expected, actual).
  `verify`     — the boundary: reads live embedder dims, calls the pure core.

UNKNOWN IS NOT OK. `naming/dim-of` returns nil for a collection whose width
is not knowable from its name (anything that is not a memory collection).
Those are reported under `:unknown` — never folded into `:ok`. A checker
that reports 'fine' when it means 'I could not tell' is the exact failure
this file exists to prevent.
raw docstring

hive-milvus.collection.locator

L2 — default ICollectionLocator impl.

Thin wrapper composing the pure naming layer (naming/ref-for-dim) with the sunset-aware config (config/sunset?). Like the router, reads config lazily on each call so hot-flips propagate.

The locator is the single source of truth for the spec→CollectionRef mapping. The write pipeline must NEVER construct a CollectionRef from a raw type or string — only from a resolved ProviderSpec via this protocol. That invariant is what eliminates the bypass paths that fed the 1804 bug.

L2 — default `ICollectionLocator` impl.

Thin wrapper composing the pure naming layer (`naming/ref-for-dim`)
with the sunset-aware config (`config/sunset?`). Like the router,
reads config lazily on each call so hot-flips propagate.

The locator is the single source of truth for the
spec→`CollectionRef` mapping. The write pipeline must NEVER
construct a CollectionRef from a raw type or string — only from a
resolved `ProviderSpec` via this protocol. That invariant is what
eliminates the bypass paths that fed the 1804 bug.
raw docstring

hive-milvus.collection.naming

L1 pure — collection-name derivation.

Two pure functions:

  • chroma-name : dim → Chroma collection name. 768 stays on the legacy hive-mcp-memory; other dims get the -<dim>d suffix.
  • milvus-name : Chroma name → Milvus name (underscores not hyphens). Idempotent — a name already in Milvus form passes through. Reuses hive-milvus.collections/collection->milvus-name so the two definitions cannot drift.

Why this lives outside routing.clj: SRP. Routing is about resolving a ProviderSpec; naming is about projecting a dim to a string. The L2 locator composes both.

L1 pure — collection-name derivation.

Two pure functions:

- `chroma-name` : dim → Chroma collection name.
                  768 stays on the legacy `hive-mcp-memory`; other
                  dims get the `-<dim>d` suffix.
- `milvus-name` : Chroma name → Milvus name (underscores not
                  hyphens). Idempotent — a name already in Milvus
                  form passes through. Reuses
                  `hive-milvus.collections/collection->milvus-name`
                  so the two definitions cannot drift.

Why this lives outside `routing.clj`: SRP. Routing is about
resolving a `ProviderSpec`; naming is about projecting a dim to a
string. The L2 locator composes both.
raw docstring

hive-milvus.collection.protocol

L0 contract — collection bounded context.

Two narrow protocols (ISP):

  • ICollectionLocator — spec → CollectionRef. Pure lookup. Knows sunset state so writes can avoid retired collections while reads can fan out across both active + sunset (graceful drain).
  • ICollectionEnsure — create-if-missing on the backend with mandatory dim-check. Rejects an ensure! whose ref-dim disagrees with an existing collection's dim — the precise check that prevents the silent 1804 schema-mismatch.

Reload-safety: defonce-guarded.

L0 contract — collection bounded context.

Two narrow protocols (ISP):

- `ICollectionLocator` — spec → `CollectionRef`. Pure lookup. Knows
                         sunset state so writes can avoid retired
                         collections while reads can fan out across
                         both active + sunset (graceful drain).
- `ICollectionEnsure`  — create-if-missing on the backend with
                         mandatory dim-check. Rejects an `ensure!`
                         whose ref-dim disagrees with an existing
                         collection's dim — the precise check that
                         prevents the silent 1804 schema-mismatch.

Reload-safety: `defonce`-guarded.
raw docstring

hive-milvus.collections

Multi-collection helpers: filtering, naming.

Multi-collection helpers: filtering, naming.
raw docstring

hive-milvus.config

Typed config for Milvus addon — declared via hive-di defconfig.

Replaces ad-hoc env-var parsing previously scattered through addon.clj. Generates (via the macro):

MilvusConfig — ADT (:config/resolved | :unresolved | :invalid) MilvusConfig-fields — field registry (source of truth) MilvusConfig-schema — Malli closed map schema resolve-MilvusConfig — resolver fn (0/1/2 arity) returning Result

Resolution order (per field):

  1. Explicit overrides map (e.g. addon manifest config)
  2. Environment variable lookup (System/getenv by default)
  3. blank->nil normalization ("" → trigger default)
  4. Pre-typed default (skips coercion)
  5. hive-dsl.coerce on string env values

Optional fields (currently :token) resolve to nil when unset.

Typed config for Milvus addon — declared via hive-di defconfig.

Replaces ad-hoc env-var parsing previously scattered through addon.clj.
Generates (via the macro):

  MilvusConfig             — ADT (:config/resolved | :unresolved | :invalid)
  MilvusConfig-fields      — field registry (source of truth)
  MilvusConfig-schema      — Malli closed map schema
  resolve-MilvusConfig     — resolver fn (0/1/2 arity) returning Result

Resolution order (per field):
  1. Explicit overrides map (e.g. addon manifest config)
  2. Environment variable lookup (System/getenv by default)
  3. blank->nil normalization ("" → trigger default)
  4. Pre-typed default (skips coercion)
  5. hive-dsl.coerce on string env values

Optional fields (currently :token) resolve to nil when unset.
raw docstring

hive-milvus.cursor

Persistent resume state for batch migrations.

Persistent resume state for batch migrations.
raw docstring

hive-milvus.dimensions

Detect embedding dimension for Chroma collections.

Detect embedding dimension for Chroma collections.
raw docstring

hive-milvus.embed

Ollama embedding client.

Ollama embedding client.
raw docstring

hive-milvus.embed.port

Consumer-owned embedding port + injection slot.

IEmbedder is the boundary hive-milvus calls for every provider lookup, routing decision and remote embed. The host injects a concrete adapter via set-embedder! at boot; a no-op default keeps the core loadable and testable standalone (no hive-mcp on the path).

Protocol methods are --prefixed and dispatch on an injected instance; the public fns below deref the slot so call sites stay free of (current) threading. Method contracts:

-embed-entry entry+collection+content => (r/ok vec) | (r/err :embedder/no-provider {…}) | (r/err :embedder/embed-failed {…}) -embed-text collection+text => vec (may throw) -routing-for-type[+size] memory-type[+content] => {:collection-name str :dimension int :max-tokens int :provider-key kw} | nil -no-embed-type? memory-type => bool -collection-names => seq of chroma-style names -dimension-for-collection collection => int | nil -configured-collection-names => seq of chroma-style names -provider-available-for? collection => bool -collection-backed? collection => bool

Consumer-owned embedding port + injection slot.

`IEmbedder` is the boundary hive-milvus calls for every provider
lookup, routing decision and remote embed. The host injects a
concrete adapter via `set-embedder!` at boot; a no-op default keeps
the core loadable and testable standalone (no hive-mcp on the path).

Protocol methods are `-`-prefixed and dispatch on an injected
instance; the public fns below deref the slot so call sites stay
free of `(current)` threading. Method contracts:

  -embed-entry             entry+collection+content => (r/ok vec) |
                           (r/err :embedder/no-provider {…}) |
                           (r/err :embedder/embed-failed {…})
  -embed-text              collection+text => vec (may throw)
  -routing-for-type[+size] memory-type[+content] =>
                           {:collection-name str :dimension int
                            :max-tokens int :provider-key kw} | nil
  -no-embed-type?          memory-type => bool
  -collection-names        => seq of chroma-style names
  -dimension-for-collection collection => int | nil
  -configured-collection-names => seq of chroma-style names
  -provider-available-for? collection => bool
  -collection-backed?      collection => bool
raw docstring

hive-milvus.embedder

BOUNDARY: re-embed an entry's content via the injected embedding port. Record-shape conversion stays pure in hive-milvus.store.schema; the provider lookup + remote embed live behind hive-milvus.embed.port.

embed-for-entry policy (port-agnostic): blank content => (r/ok []) no-embed type => (r/ok zero-vector) sized to the type's routed dim otherwise => port/embed-entry, then dim-guarded.

Returns (r/ok vec) or one of :embedder/no-provider :embedder/embed-failed :embedder/dim-mismatch.

BOUNDARY: re-embed an entry's content via the injected embedding port.
Record-shape conversion stays pure in `hive-milvus.store.schema`; the
provider lookup + remote embed live behind `hive-milvus.embed.port`.

`embed-for-entry` policy (port-agnostic):
  blank content => (r/ok [])
  no-embed type => (r/ok zero-vector) sized to the type's routed dim
  otherwise     => `port/embed-entry`, then dim-guarded.

Returns (r/ok vec) or one of
  :embedder/no-provider  :embedder/embed-failed  :embedder/dim-mismatch.
raw docstring

hive-milvus.failure

Failure ADT + classifier + legacy-shape translator for hive-milvus.

Pure, no side effects: classifies Throwables into the closed MilvusFailure ADT and translates ADT values back to the legacy {:success? false :errors [...] :reconnecting? ...} map shape that protocol callers expect. Classification delegates to milvus-clj.client/classify-error (dispatch on the ::client/transport ex-data tag) and walks the full .getCause chain, not just the top throwable.

Failure ADT + classifier + legacy-shape translator for hive-milvus.

Pure, no side effects: classifies Throwables into the closed
`MilvusFailure` ADT and translates ADT values back to the legacy
`{:success? false :errors [...] :reconnecting? ...}` map shape that
protocol callers expect. Classification delegates to
`milvus-clj.client/classify-error` (dispatch on the
`::client/transport` ex-data tag) and walks the full `.getCause`
chain, not just the top throwable.
raw docstring

hive-milvus.migrate

Chroma SQLite → Milvus migration.

Railway-oriented pipeline: resolve-config → read-source → resolve-cursor → connect-target → migrate-batches!

Each step returns Result (ok/err). Cursor persists after each batch for safe interrupt/resume.

Chroma SQLite → Milvus migration.

Railway-oriented pipeline: resolve-config → read-source → resolve-cursor
→ connect-target → migrate-batches!

Each step returns Result (ok/err). Cursor persists after each batch
for safe interrupt/resume.
raw docstring

hive-milvus.queue

Global in-memory write queue + read fail-soft shim for Milvus outages.

Single JVM-wide singleton — shared across ALL projects and agents, not per-project. Purpose: when the circuit breaker opens because Milvus is unreachable, mutating protocol calls are enqueued here instead of failing; reads return a degraded response carrying actionable recovery tips. On the circuit's :closed transition, a background future drains the queue via hive-weave.parallel/bounded-pmap with concurrency 8 and a 15 s per-item timeout. Pre-drain the batch is coalesced by (op,id) keeping the latest mutation, so a burst of writes to the same entry collapses to one RPC. Any operation that still fails is pushed to the tail for the next pass; drain repeats until the queue is empty, the circuit reopens, or a pass makes zero progress.

Intentionally has no hard dependency on the circuit ns or store.clj — those wire it in (see enqueue!, drain!, degraded-response, make-store-dispatch). Keeping the boundary clean lets circuit, store, and queue evolve independently and lets tests exercise the queue without standing up Milvus.

Global in-memory write queue + read fail-soft shim for Milvus outages.

Single JVM-wide singleton — shared across ALL projects and agents, not
per-project. Purpose: when the circuit breaker opens because Milvus is
unreachable, mutating protocol calls are enqueued here instead of
failing; reads return a degraded response carrying actionable recovery
tips. On the circuit's :closed transition, a background future drains
the queue via `hive-weave.parallel/bounded-pmap` with concurrency 8
and a 15 s per-item timeout. Pre-drain the batch is coalesced by
(op,id) keeping the latest mutation, so a burst of writes to the same
entry collapses to one RPC. Any operation that still fails is pushed
to the tail for the next pass; drain repeats until the queue is empty,
the circuit reopens, or a pass makes zero progress.

Intentionally has no hard dependency on the circuit ns or store.clj —
those wire it in (see `enqueue!`, `drain!`, `degraded-response`,
`make-store-dispatch`). Keeping the boundary clean lets circuit,
store, and queue evolve independently and lets tests exercise the
queue without standing up Milvus.
raw docstring

hive-milvus.record

Pure: memory entry → Milvus insert record.

Pure: memory entry → Milvus insert record.
raw docstring

hive-milvus.relocate

Background re-embedding + relocation of memory entries from non-canonical Milvus collections (e.g. legacy 768-d hive_mcp_memory) into per-dim collections (hive_mcp_memory_1024d, hive_mcp_memory_4096d) driven by current type-routing config.

Idempotent + resumable. entries/relocate-entry! is itself a no-op once an entry is already in its canonical collection, so resume never duplicates.

Background re-embedding + relocation of memory entries from
non-canonical Milvus collections (e.g. legacy 768-d
`hive_mcp_memory`) into per-dim collections (`hive_mcp_memory_1024d`,
`hive_mcp_memory_4096d`) driven by current type-routing config.

Idempotent + resumable. `entries/relocate-entry!` is itself a no-op
once an entry is already in its canonical collection, so resume
never duplicates.
raw docstring

hive-milvus.relocate.boundary

BOUNDARY: thin Result-tracked wrappers around Milvus side effects used by the relocation pipeline. Each fn does ONE Milvus call, wrapped in r/try-effect* so failures stay railway-tracked instead of escaping as exceptions.

These replace the bare @(milvus/...) + bare try/catch patterns in hive-milvus.store.entries for the routing-aware path. Pre-existing entries.clj fns are not modified here (out of scope per the refactor plan).

BOUNDARY: thin Result-tracked wrappers around Milvus side effects
used by the relocation pipeline. Each fn does ONE Milvus call,
wrapped in `r/try-effect*` so failures stay railway-tracked
instead of escaping as exceptions.

These replace the bare `@(milvus/...)` + bare `try`/`catch`
patterns in `hive-milvus.store.entries` for the routing-aware
path. Pre-existing entries.clj fns are not modified here (out of
scope per the refactor plan).
raw docstring

hive-milvus.relocate.collectors.entry

COLLECT: locate an entry's current physical collection and read its record. Wraps the existing lookup/find-entry-collection + query/get-entry-by-id chain in Result envelopes so the pipeline sees them as data.

Imports the leaf hive-milvus.store.lookup ns rather than hive-milvus.store.entries to keep the relocate pipeline cycle-free (entries delegates routing-aware ops to the pipeline; the pipeline must NOT loop back into entries).

COLLECT: locate an entry's current physical collection and read its
record. Wraps the existing `lookup/find-entry-collection` +
`query/get-entry-by-id` chain in Result envelopes so the pipeline
sees them as data.

Imports the leaf `hive-milvus.store.lookup` ns rather than
`hive-milvus.store.entries` to keep the relocate pipeline cycle-free
(entries delegates routing-aware ops to the pipeline; the pipeline
must NOT loop back into entries).
raw docstring

hive-milvus.relocate.enumerate

Every id in a collection.

A drain can read the head repeatedly because moved rows leave the source. A COPY leaves them, so it needs the full id set up front — and Milvus caps a query at max-page rows and returns them in no particular order, so a single page is not the collection.

The id space is therefore split into disjoint prefix buckets, each fetched whole. A bucket that comes back full may have been truncated, so it is split again. Nothing is ever assumed about ordering, and a bucket that cannot be split further raises rather than returning a short answer.

Every id in a collection.

A drain can read the head repeatedly because moved rows leave the source.
A COPY leaves them, so it needs the full id set up front — and Milvus caps a
query at `max-page` rows and returns them in no particular order, so a single
page is not the collection.

The id space is therefore split into disjoint prefix buckets, each fetched
whole. A bucket that comes back full may have been truncated, so it is split
again. Nothing is ever assumed about ordering, and a bucket that cannot be
split further raises rather than returning a short answer.
raw docstring

hive-milvus.relocate.pipeline

PIPELINE: compose the collectors + promoters + boundaries into the single-entry relocate operation. All-Result; first :err short-circuits.

The flow:

collect-existing-entry ;; COLLECT → compute-target ;; PROMOTE (pure) → classify-relocation-need ;; PROMOTE (pure) → IF :no-op → return r/ok with :moved? false IF :move → ensure-target-collection ;; PROMOTE/BOUNDARY edge → build-target-record ;; PROMOTE (calls embedder) → milvus-write! ;; BOUNDARY → milvus-delete! ;; BOUNDARY → return r/ok with :moved? true

Replaces the body of hive-milvus.store.entries/relocate-entry! with a Result-tracked, layer-disciplined version.

PIPELINE: compose the collectors + promoters + boundaries into the
single-entry relocate operation. All-Result; first :err short-circuits.

The flow:

  collect-existing-entry           ;; COLLECT
    → compute-target               ;; PROMOTE (pure)
    → classify-relocation-need     ;; PROMOTE (pure)
    → IF :no-op → return r/ok with :moved? false
      IF :move  → ensure-target-collection  ;; PROMOTE/BOUNDARY edge
                → build-target-record       ;; PROMOTE (calls embedder)
                → milvus-write!              ;; BOUNDARY
                → milvus-delete!             ;; BOUNDARY
                → return r/ok with :moved? true

Replaces the body of `hive-milvus.store.entries/relocate-entry!`
with a Result-tracked, layer-disciplined version.
raw docstring

hive-milvus.relocate.plan

Pure decisions for a drain pass: which rows to ask for next, and what a finished round means for the loop. Plain data in, plain data out — no client, no clock, no state.

Pure decisions for a drain pass: which rows to ask for next, and what a
finished round means for the loop. Plain data in, plain data out — no
client, no clock, no state.
raw docstring

hive-milvus.relocate.promoters.record

PROMOTE: wrap the embedder BOUNDARY call + the pure schema record builder into a Result-tracked promoter. The single caller (the pipeline) gets one r/ok / r/err to thread through.

Why this is a promoter even though it touches the embedder: embed-for-entry returns Result; from the pipeline's POV the embedding is just data flowing through the bundle. The actual I/O (HTTP call to Ollama) is encapsulated in the embedder ns; here we only orchestrate.

PROMOTE: wrap the embedder BOUNDARY call + the pure schema record
builder into a Result-tracked promoter. The single caller (the
pipeline) gets one r/ok / r/err to thread through.

Why this is a promoter even though it touches the embedder:
`embed-for-entry` returns Result; from the pipeline's POV the
embedding is just data flowing through the bundle. The actual
I/O (HTTP call to Ollama) is encapsulated in the embedder ns;
here we only orchestrate.
raw docstring

hive-milvus.relocate.promoters.routing

PROMOTE: pure functions that compute relocation routing decisions from an entry. No I/O, no embedder calls, no Milvus reads — purely data → Result.

Used by hive-milvus.relocate.pipeline/relocate-one to decide whether an entry needs to move and where it should go before any BOUNDARY effect fires.

PROMOTE: pure functions that compute relocation routing decisions
from an entry. No I/O, no embedder calls, no Milvus reads — purely
data → Result.

Used by `hive-milvus.relocate.pipeline/relocate-one` to decide
whether an entry needs to move and where it should go before any
BOUNDARY effect fires.
raw docstring

hive-milvus.relocate.source

Where the next ids to relocate come from.

The drain reads the head of the source collection and excludes what it already handled. It relies on relocation removing the row from the source, so the head is always fresh work — it never assumes the store returns rows in any order.

Where the next ids to relocate come from.

The drain reads the head of the source collection and excludes what it
already handled. It relies on relocation removing the row from the source,
so the head is always fresh work — it never assumes the store returns rows
in any order.
raw docstring

hive-milvus.resilience.probe

Liveness probing — Boundary stage of the resilience CPPB pipeline.

Single responsibility: ask the underlying milvus client whether it can actually reach the server, with a tiny TTL cache so a burst of probes amortises to one RPC.

Transport-agnostic: dispatches through milvus-clj.client/ILivenessProbe (extended by every transport's defrecord). Adding a new transport requires zero changes here — that is the point of the protocol seam.

Why a separate ns from reconnect/retry: each of probe / reconnect / retry has its own rate of change. The probe surface is the most stable (one cached read); reconnect-loop algorithm and retry budget knobs change more often. Splitting them keeps each ns small and individually testable (mock the protocol; no need to mock a loop).

Time discipline: never call System/currentTimeMillis directly — go through hive-ttracking.clock/now-millis so tests can pin time.

Liveness probing — Boundary stage of the resilience CPPB pipeline.

Single responsibility: ask the underlying milvus client whether it can
actually reach the server, with a tiny TTL cache so a burst of probes
amortises to one RPC.

Transport-agnostic: dispatches through `milvus-clj.client/ILivenessProbe`
(extended by every transport's defrecord). Adding a new transport
requires zero changes here — that is the point of the protocol seam.

Why a separate ns from reconnect/retry: each of probe / reconnect /
retry has its own rate of change. The probe surface is the most stable
(one cached read); reconnect-loop algorithm and retry budget knobs
change more often. Splitting them keeps each ns small and individually
testable (mock the protocol; no need to mock a loop).

Time discipline: never call `System/currentTimeMillis` directly — go
through `hive-ttracking.clock/now-millis` so tests can pin time.
raw docstring

hive-milvus.resilience.reconnect

Reconnect-loop ownership — Boundary stage of the resilience CPPB pipeline.

Single responsibility: drive the singleton milvus client back to a reachable state after a transient failure. The loop's success criterion is an actual probe round-trip, NOT the post-connect! atom state — that distinction is the bug fix vs the prior design where connect! always 'succeeded' (it just sets the atom) and the loop exited even when the network was still down.

Decoupled from probe.clj (which owns the cache) and retry.clj (which owns the reactive pipeline) per SRP. This ns owns ONE atom (reconnect-state) and its loop future.

Time discipline: never call System/currentTimeMillis directly — go through hive-ttracking.clock/now-millis so tests can pin time.

Reconnect-loop ownership — Boundary stage of the resilience CPPB
pipeline.

Single responsibility: drive the singleton milvus client back to a
reachable state after a transient failure. The loop's success
criterion is an actual probe round-trip, NOT the post-`connect!`
atom state — that distinction is the bug fix vs the prior design
where `connect!` always 'succeeded' (it just sets the atom) and the
loop exited even when the network was still down.

Decoupled from probe.clj (which owns the cache) and retry.clj
(which owns the reactive pipeline) per SRP. This ns owns ONE atom
(`reconnect-state`) and its loop future.

Time discipline: never call `System/currentTimeMillis` directly — go
through `hive-ttracking.clock/now-millis` so tests can pin time.
raw docstring

hive-milvus.resilience.retry

Reactive try / classify / retry pipeline — Pipeline stage of the resilience CPPB pipeline.

Single responsibility: wrap one RPC body in a fault-tolerant pipeline that classifies failure, kicks the reconnect loop, awaits verified recovery, and retries exactly once. Public surface:

  • with-auto-reconnect — function form, takes a thunk
  • resilient — macro sugar that combines ensure-live! with with-auto-reconnect for use inside protocol method bodies

Composes Promote (failure/classify, circuit/check) over Boundary (probe, reconnect). Stays pure with respect to the singleton client — never mutates default-client directly; only orchestrates the Boundary nses that do.

Migration note: this ns replaces hive-milvus.store.health. The old ns kept circuit, probe, reconnect, classify, retry all in one file (5 responsibilities). This split means each concern can evolve independently.

Reactive try / classify / retry pipeline — Pipeline stage of the
resilience CPPB pipeline.

Single responsibility: wrap one RPC body in a fault-tolerant
pipeline that classifies failure, kicks the reconnect loop, awaits
verified recovery, and retries exactly once. Public surface:

  - `with-auto-reconnect` — function form, takes a thunk
  - `resilient` — macro sugar that combines `ensure-live!` with
    `with-auto-reconnect` for use inside protocol method bodies

Composes Promote (`failure/classify`, `circuit/check`) over Boundary
(`probe`, `reconnect`). Stays pure with respect to the singleton
client — never mutates `default-client` directly; only orchestrates
the Boundary nses that do.

Migration note: this ns replaces `hive-milvus.store.health`. The old
ns kept circuit, probe, reconnect, classify, retry all in one file
(5 responsibilities). This split means each concern can evolve
independently.
raw docstring

hive-milvus.resilience.scheduler

Proactive periodic liveness scheduler — Boundary stage of the resilience CPPB pipeline.

Single responsibility: own a timer that fires a health check on a fixed cadence, so a connection that dies while the store is IDLE (no traffic to trigger the reactive retry path or the on-demand ensure-live! gate) still heals. This closes the gap where auto-heal was 100% traffic-driven: after a network blip with no in-flight RPC, nothing re-probed and the dead client sat dead until the next operation happened to touch it.

Why this ns owns ONLY the timer (SRP + DRY):

  • The health decision — 'is it dead, and if so start a heal loop WITHOUT disturbing one already in flight' — already lives in retry/ensure-live! (reads cached liveness, kicks the reconnect loop only when-not it is already running). Re-implementing it here would duplicate the guard and risk calling reconnect/kick! mid-heal, which drops the fresh client on every tick. So the tick DELEGATES the policy and keeps only the scheduling (Boundary) concern.
  • Complements, does not replace: reactive retry (on failure) + preemptive ensure-live! (at op start) stay as-is; this adds the idle heartbeat that neither covered.

Lifecycle: started from store.lifecycle/connect! once the config-atom exists; stopped from store.lifecycle/disconnect! before teardown, so the timer can't kick a heal loop right after the operator closed the store.

Time discipline: never call System/currentTimeMillis directly — go through hive-ttracking.clock/now-millis (mirrors probe/reconnect).

Proactive periodic liveness scheduler — Boundary stage of the resilience
CPPB pipeline.

Single responsibility: own a timer that fires a health check on a fixed
cadence, so a connection that dies while the store is IDLE (no traffic to
trigger the reactive `retry` path or the on-demand `ensure-live!` gate)
still heals. This closes the gap where auto-heal was 100% traffic-driven:
after a network blip with no in-flight RPC, nothing re-probed and the dead
client sat dead until the next operation happened to touch it.

Why this ns owns ONLY the timer (SRP + DRY):
- The health *decision* — 'is it dead, and if so start a heal loop WITHOUT
  disturbing one already in flight' — already lives in `retry/ensure-live!`
  (reads cached liveness, kicks the reconnect loop only `when-not` it is
  already running). Re-implementing it here would duplicate the guard and
  risk calling `reconnect/kick!` mid-heal, which drops the fresh client on
  every tick. So the tick DELEGATES the policy and keeps only the
  scheduling (Boundary) concern.
- Complements, does not replace: reactive retry (on failure) + preemptive
  ensure-live! (at op start) stay as-is; this adds the idle heartbeat that
  neither covered.

Lifecycle: started from `store.lifecycle/connect!` once the config-atom
exists; stopped from `store.lifecycle/disconnect!` before teardown, so the
timer can't kick a heal loop right after the operator closed the store.

Time discipline: never call `System/currentTimeMillis` directly — go
through `hive-ttracking.clock/now-millis` (mirrors probe/reconnect).
raw docstring

hive-milvus.sqlite

Chroma SQLite reader — pure JDBC access, no domain logic.

Chroma SQLite reader — pure JDBC access, no domain logic.
raw docstring

hive-milvus.storage.protocol

L0 contract — storage bounded context.

Two narrow protocols (ISP):

  • IVectorWrite — upsert one or more entries to a CollectionRef.
  • IVectorRead — search across one or more CollectionRefs with fan-out semantics (read may union results from active + sunset collections during drain).

Both protocols depend only on ICollectionLocator (via the ref) — they have zero coupling to embedder or router internals. The write pipeline (in storage/write_*.clj) is responsible for resolving spec → ref BEFORE calling upsert!; this layer never derives a collection from a raw type.

Reload-safety: defonce-guarded.

L0 contract — storage bounded context.

Two narrow protocols (ISP):

- `IVectorWrite` — upsert one or more entries to a `CollectionRef`.
- `IVectorRead`  — search across one or more `CollectionRef`s with
                  fan-out semantics (read may union results from
                  active + sunset collections during drain).

Both protocols depend only on `ICollectionLocator` (via the ref) —
they have zero coupling to embedder or router internals. The write
pipeline (in `storage/write_*.clj`) is responsible for resolving
spec → ref BEFORE calling `upsert!`; this layer never derives a
collection from a raw type.

Reload-safety: `defonce`-guarded.
raw docstring

hive-milvus.store

Milvus implementation of IMemoryStore protocol.

Standalone addon project. Wraps milvus-clj.api into protocol methods.

Milvus requires explicit embeddings on insert and search; those are produced through the injected hive-milvus.embed.port boundary.

DDD: Repository pattern — MilvusMemoryStore is the Milvus aggregate adapter.

This namespace is the façade over four cohesive submodules: hive-milvus.store.schema — entry<->record, filter expressions hive-milvus.store.index — collection loading + scalar indexes hive-milvus.store.query — single-entry read / read-modify-write hive-milvus.resilience.retry — reactive retry pipeline + resilient hive-milvus.resilience.probe — cached liveness query hive-milvus.resilience.reconnect — background heal loop, probe-verified

The old hive-milvus.store.health was a 5-responsibility monolith (circuit, probe, reconnect-loop, retry, classify); split into the three resilience.* nses per SRP. store.health now exists as a deprecation shim that re-exports the new locations so external callers don't break.

The defrecord MilvusMemoryStore + create-store live here because the protocol methods can't be split across files.

Milvus implementation of IMemoryStore protocol.

Standalone addon project. Wraps milvus-clj.api into protocol methods.

Milvus requires explicit embeddings on insert and search; those are
produced through the injected `hive-milvus.embed.port` boundary.

DDD: Repository pattern — MilvusMemoryStore is the Milvus aggregate adapter.

This namespace is the façade over four cohesive submodules:
  hive-milvus.store.schema       — entry<->record, filter expressions
  hive-milvus.store.index        — collection loading + scalar indexes
  hive-milvus.store.query        — single-entry read / read-modify-write
  hive-milvus.resilience.retry   — reactive retry pipeline + `resilient`
  hive-milvus.resilience.probe   — cached liveness query
  hive-milvus.resilience.reconnect — background heal loop, probe-verified

The old `hive-milvus.store.health` was a 5-responsibility monolith
(circuit, probe, reconnect-loop, retry, classify); split into the
three `resilience.*` nses per SRP. `store.health` now exists as a
deprecation shim that re-exports the new locations so external
callers don't break.

The `defrecord MilvusMemoryStore` + `create-store` live here because
the protocol methods can't be split across files.
raw docstring

hive-milvus.store.analytics

Analytics protocol helpers for MilvusMemoryStore.

Analytics protocol helpers for MilvusMemoryStore.
raw docstring

hive-milvus.store.batch

Batch protocol helpers for MilvusMemoryStore.

Batch protocol helpers for MilvusMemoryStore.
raw docstring

hive-milvus.store.deref

Bounded deref for milvus-clj RPC futures. Replaces bare @(milvus/...); on timeout throws a :milvus/timeout-tagged ex-info.

Bounded deref for milvus-clj RPC futures. Replaces bare `@(milvus/...)`;
on timeout throws a `:milvus/timeout`-tagged ex-info.
raw docstring

hive-milvus.store.health

DEPRECATED — kept only as a re-export shim for the public surface that pre-resilience-split consumers (store/batch.clj, store/entries.clj, store/lifecycle.clj, store/analytics.clj, store/staleness.clj) already :refer from this ns.

New code MUST require directly from one of:

hive-milvus.resilience.probe — cached liveness query hive-milvus.resilience.reconnect — heal-loop + verified reconnect hive-milvus.resilience.retry — reactive retry + resilient macro

Why split: SRP. The old monolith owned 5 concerns (circuit, probe, loop, retry, classify) in one 290-line file; each had its own rate of change and test surface. The split makes adding a transport (e.g. future WebSocket) require zero changes to the resilience layer — only an extend-type for milvus-clj.client/ILivenessProbe.

The bug fix that motivated the split: the old loop's success criterion was post-connect! atom state, not an actual probe round-trip. After a network flip the atom would be set fresh while the underlying transport was still unreachable, so the loop exited prematurely and operators had to restart the JVM. The new reconnect/try-reconnect-and-verify! requires both connect! AND a successful probe! to declare recovery.

This shim re-exports the public surface so old consumers compile unchanged. Will be deleted in a future PR after consumers migrate.

DEPRECATED — kept only as a re-export shim for the public surface that
pre-resilience-split consumers (`store/batch.clj`, `store/entries.clj`,
`store/lifecycle.clj`, `store/analytics.clj`, `store/staleness.clj`)
already `:refer` from this ns.

New code MUST require directly from one of:

  hive-milvus.resilience.probe      — cached liveness query
  hive-milvus.resilience.reconnect  — heal-loop + verified reconnect
  hive-milvus.resilience.retry      — reactive retry + `resilient` macro

Why split: SRP. The old monolith owned 5 concerns (circuit, probe,
loop, retry, classify) in one 290-line file; each had its own rate
of change and test surface. The split makes adding a transport (e.g.
future WebSocket) require zero changes to the resilience layer —
only an `extend-type` for `milvus-clj.client/ILivenessProbe`.

The bug fix that motivated the split: the old loop's success
criterion was post-`connect!` atom state, not an actual probe
round-trip. After a network flip the atom would be set fresh while
the underlying transport was still unreachable, so the loop exited
prematurely and operators had to restart the JVM. The new
`reconnect/try-reconnect-and-verify!` requires both `connect!` AND
a successful `probe!` to declare recovery.

This shim re-exports the public surface so old consumers compile
unchanged. Will be deleted in a future PR after consumers migrate.
raw docstring

hive-milvus.store.index

Collection loading and scalar-index management for MilvusMemoryStore.

Owns two process-wide memoization atoms:

  • loaded-collections: collections this process has load-collection'd.
  • indexed-collections: collections we've probed + installed INVERTED scalar indexes on.

Both are defonce so hot-reload doesn't desync with milvus-clj's own singleton client state.

Collection loading and scalar-index management for MilvusMemoryStore.

Owns two process-wide memoization atoms:
- `loaded-collections`: collections this process has `load-collection`'d.
- `indexed-collections`: collections we've probed + installed INVERTED
  scalar indexes on.

Both are `defonce` so hot-reload doesn't desync with milvus-clj's own
singleton client state.
raw docstring

hive-milvus.store.lifecycle

Connection lifecycle helpers for MilvusMemoryStore.

Connection lifecycle helpers for MilvusMemoryStore.
raw docstring

hive-milvus.store.lookup

Leaf ns for cross-collection id lookup. Extracted out of hive-milvus.store.entries so the relocate/ pipeline can call into it without creating a cycle (entries → pipeline → collectors → entries). No upward dependencies — only routing + query.

Leaf ns for cross-collection id lookup. Extracted out of
`hive-milvus.store.entries` so the `relocate/` pipeline can call
into it without creating a cycle (entries → pipeline → collectors
→ entries). No upward dependencies — only `routing` + `query`.
raw docstring

hive-milvus.store.query

Single-entry read + read-merge-upsert helpers.

These are the primitives every protocol method composes with when it needs to fetch a row by id or mutate it through a read-modify-write cycle. They don't own any state — the collection name + id are passed through, and the resilient-retry wrapper is applied by the caller.

Single-entry read + read-merge-upsert helpers.

These are the primitives every protocol method composes with when it
needs to fetch a row by id or mutate it through a read-modify-write
cycle. They don't own any state — the collection name + id are passed
through, and the resilient-retry wrapper is applied by the caller.
raw docstring

hive-milvus.store.routing

Per-type collection routing for Milvus.

Bridges the injected embedding port's type→provider routing (port/routing-for-type) to Milvus collection naming (underscores, no hyphens). Each provider dimension lives in its own Milvus collection so dual-run migrations (e.g. nomic 768-d → qwen3-embedding:0.6b 1024-d) coexist without schema conflict — Milvus collections have immutable dimension at create time.

Collection-name conventions:

  • Default 768-d collection: hive_mcp_memory (legacy name preserved)
  • Per-dim collections: hive_mcp_memory_<dim>d

Reads without a type opt fan out across all known collections; writes are routed to the entry's type's collection and auto-create on first use via index/ensure-collection!.

Per-type collection routing for Milvus.

Bridges the injected embedding port's type→provider routing
(`port/routing-for-type`) to Milvus collection naming (underscores,
no hyphens). Each provider dimension lives in its own Milvus collection
so dual-run migrations (e.g. nomic 768-d → qwen3-embedding:0.6b 1024-d)
coexist without schema conflict — Milvus collections have immutable
dimension at create time.

Collection-name conventions:

- Default 768-d collection: `hive_mcp_memory` (legacy name preserved)
- Per-dim collections:      `hive_mcp_memory_<dim>d`

Reads without a type opt fan out across all known collections; writes
are routed to the entry's type's collection and auto-create on first
use via `index/ensure-collection!`.
raw docstring

hive-milvus.store.schema

Entry <-> Milvus record conversion and filter expression builders.

Pure functions only — no side effects, no Milvus RPCs. Extracted from hive-milvus.store so the schema/translation layer can be understood (and tested) independently of the connection + resilience machinery.

Entry <-> Milvus record conversion and filter expression builders.

Pure functions only — no side effects, no Milvus RPCs. Extracted from
hive-milvus.store so the schema/translation layer can be understood
(and tested) independently of the connection + resilience machinery.
raw docstring

hive-milvus.store.search.boundary

The effects a search performs: embedding the query, and asking a collection for its nearest rows. Both are ports; both return Result.

The effects a search performs: embedding the query, and asking a collection
for its nearest rows. Both are ports; both return Result.
raw docstring

hive-milvus.store.search.fusion

Merge hits from different embedding spaces. Pure.

Merge hits from different embedding spaces. Pure.
raw docstring

hive-milvus.store.search.pipeline

Compose a semantic search from its ports: resolve targets, embed the query into each, search each, fuse the rankings.

Compose a semantic search from its ports: resolve targets, embed the query
into each, search each, fuse the rankings.
raw docstring

hive-milvus.store.search.target

Which collections a search runs against, paired with the embedding space each is written in.

Which collections a search runs against, paired with the embedding space each
is written in.
raw docstring

hive-milvus.store.staleness

Staleness protocol helpers for MilvusMemoryStore.

Staleness protocol helpers for MilvusMemoryStore.
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close