Collection loading and scalar-index management for MilvusMemoryStore.
Owns two process-wide memoization atoms:
loaded-collections: collections this process has load-collection'd.indexed-collections: collections we've probed + installed INVERTED
scalar indexes on.Both are defonce so hot-reload doesn't desync with milvus-clj's own
singleton client state.
Collection loading and scalar-index management for MilvusMemoryStore. Owns two process-wide memoization atoms: - `loaded-collections`: collections this process has `load-collection`'d. - `indexed-collections`: collections we've probed + installed INVERTED scalar indexes on. Both are `defonce` so hot-reload doesn't desync with milvus-clj's own singleton client state.
(ensure-collection! collection-name dimension)Ensure the memory collection exists and is loaded into memory.
After Milvus restart, collections exist but aren't loaded — queries
return empty unless we explicitly load. The per-process memo
(loaded-collections) is consulted FIRST: a memo hit issues no RPC at
all, so a write path calling this per entry costs nothing after the
first call. Only a miss probes has-collection, then creates or loads.
invalidate-loaded-collection! drops the memo when reconnect detects
a fresh server or a dropped collection.
Ensure the memory collection exists and is loaded into memory. After Milvus restart, collections exist but aren't loaded — queries return empty unless we explicitly load. The per-process memo (`loaded-collections`) is consulted FIRST: a memo hit issues no RPC at all, so a write path calling this per entry costs nothing after the first call. Only a miss probes has-collection, then creates or loads. `invalidate-loaded-collection!` drops the memo when reconnect detects a fresh server or a dropped collection.
(ensure-scalar-indexes! collection-name)Idempotently create INVERTED scalar indexes on filter fields used by
catchup (type, project_id). Milvus createIndex on an already-indexed
field returns a non-OK code; we catch and keep going — the failure mode
for 'already exists' is indistinguishable from other errors via the
current milvus-clj surface, so we log at debug + carry on.
Metric-type is a required param on CreateIndexParam even for scalar indexes; Milvus server ignores it when the index type is INVERTED.
Idempotently create INVERTED scalar indexes on filter fields used by catchup (type, project_id). Milvus `createIndex` on an already-indexed field returns a non-OK code; we catch and keep going — the failure mode for 'already exists' is indistinguishable from other errors via the current milvus-clj surface, so we log at debug + carry on. Metric-type is a required param on CreateIndexParam even for scalar indexes; Milvus server ignores it when the index type is INVERTED.
Memoization guard so we only probe/install scalar indexes once per process per collection. defonce so hot-reload doesn't reset the guard while loaded-collections (also defonce) still holds the collection — that desync caused ensure-scalar-indexes! to be skipped on reconnect.
Memoization guard so we only probe/install scalar indexes once per process per collection. defonce so hot-reload doesn't reset the guard while loaded-collections (also defonce) still holds the collection — that desync caused ensure-scalar-indexes! to be skipped on reconnect.
Fields that back hot catchup filters. Without INVERTED scalar indexes on these, Milvus cold-path queries fall back to full collection scans (~50s per query on a few-thousand-row collection). Order matters for logging only — each index is created independently and idempotently.
Fields that back hot catchup filters. Without INVERTED scalar indexes on these, Milvus cold-path queries fall back to full collection scans (~50s per query on a few-thousand-row collection). Order matters for logging only — each index is created independently and idempotently.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |