PIPELINE: compose the collectors + promoters + boundaries into the single-entry relocate operation. All-Result; first :err short-circuits.
The flow:
collect-existing-entry ;; COLLECT → compute-target ;; PROMOTE (pure) → classify-relocation-need ;; PROMOTE (pure) → IF :no-op → return r/ok with :moved? false IF :move → ensure-target-collection ;; PROMOTE/BOUNDARY edge → build-target-record ;; PROMOTE (calls embedder) → milvus-write! ;; BOUNDARY → milvus-delete! ;; BOUNDARY → return r/ok with :moved? true
Replaces the body of hive-milvus.store.entries/relocate-entry!
with a Result-tracked, layer-disciplined version.
PIPELINE: compose the collectors + promoters + boundaries into the
single-entry relocate operation. All-Result; first :err short-circuits.
The flow:
collect-existing-entry ;; COLLECT
→ compute-target ;; PROMOTE (pure)
→ classify-relocation-need ;; PROMOTE (pure)
→ IF :no-op → return r/ok with :moved? false
IF :move → ensure-target-collection ;; PROMOTE/BOUNDARY edge
→ build-target-record ;; PROMOTE (calls embedder)
→ milvus-write! ;; BOUNDARY
→ milvus-delete! ;; BOUNDARY
→ return r/ok with :moved? true
Replaces the body of `hive-milvus.store.entries/relocate-entry!`
with a Result-tracked, layer-disciplined version.(commit-page writer disposition prepared)Write a page of prepared entries with ONE upsert per target collection.
prepared is a seq of [id Result] pairs, each Result as prepare-one
answers it (an err or a no-op passes straight through). Ready bundles are
grouped by :target-coll and written through writer (IRecordWriter);
disposition then runs for each bundle whose write landed, so a move
deletes a source row only after its target write.
Returns [[id Result<place-one answer>] ...] in the order given.
Write a page of prepared entries with ONE upsert per target collection. `prepared` is a seq of [id Result] pairs, each Result as `prepare-one` answers it (an err or a no-op passes straight through). Ready bundles are grouped by :target-coll and written through `writer` (IRecordWriter); `disposition` then runs for each bundle whose write landed, so a move deletes a source row only after its target write. Returns [[id Result<place-one answer>] ...] in the order given.
(copy-one config-atom id)Copy the entry at id into its canonical collection, LEAVING the source row
in place. Non-destructive: the old collection remains a complete backup.
Copy the entry at `id` into its canonical collection, LEAVING the source row in place. Non-destructive: the old collection remains a complete backup.
(delete-source)Move: the source row is removed once the target write has landed.
Move: the source row is removed once the target write has landed.
(-write-records this coll records)Upsert records into coll in ONE call. Result<count>.
Upsert `records` into `coll` in ONE call. Result<count>.
(-after-write this bundle)What becomes of the source row once the target write has landed.
What becomes of the source row once the target write has landed.
(-describe-disposition this)(keep-source)Copy: the source row is left exactly where it is. The old collection stays a readable backup, at the cost of holding the entry twice.
Copy: the source row is left exactly where it is. The old collection stays a readable backup, at the cost of holding the entry twice.
(merge-updates existing updates id)Pure: the entry existing with updates merged over it, keeping id as
its identity. An :id inside updates is ignored, so an update can neither
rename the entry nor redirect the write onto another row. nil updates
leaves existing unchanged apart from the pinned :id.
Pure: the entry `existing` with `updates` merged over it, keeping `id` as its identity. An :id inside `updates` is ignored, so an update can neither rename the entry nor redirect the write onto another row. nil `updates` leaves `existing` unchanged apart from the pinned :id.
(milvus-writer)The production record writer: one Milvus upsert per call.
The production record writer: one Milvus upsert per call.
(place-one config-atom id disposition)(place-one config-atom id disposition overrides)Put the entry at id into its canonical collection, then let disposition
decide what happens to the source row. overrides (optional) is merged
into the collected entry before it is re-embedded: {:embed-text s} lets
a caller whose stored :content is not embeddable (a store decorator that
encrypts at rest) supply the text; :embed-text never reaches the record.
Returns: (r/ok {:placed? true :source-kept? bool :from src :to target :id id}) (r/ok {:placed? false :from src :to target :id id}) on no-op (r/err :collector/not-found | :routing/no-target | :embedder/embed-failed | :boundary/milvus-write-failed | ...)
Idempotent: the target write is an upsert, and an entry already in its canonical collection is a no-op.
Composed of prepare-one (collect .. build-target-record), the single
milvus-write!, and finish-one (disposition). commit-page reuses the
same two halves with one batched write per target collection.
Put the entry at `id` into its canonical collection, then let `disposition`
decide what happens to the source row. `overrides` (optional) is merged
into the collected entry before it is re-embedded: `{:embed-text s}` lets
a caller whose stored `:content` is not embeddable (a store decorator that
encrypts at rest) supply the text; `:embed-text` never reaches the record.
Returns:
(r/ok {:placed? true :source-kept? bool :from src :to target :id id})
(r/ok {:placed? false :from src :to target :id id}) on no-op
(r/err :collector/not-found | :routing/no-target |
:embedder/embed-failed | :boundary/milvus-write-failed | ...)
Idempotent: the target write is an upsert, and an entry already in its
canonical collection is a no-op.
Composed of `prepare-one` (collect .. build-target-record), the single
`milvus-write!`, and `finish-one` (disposition). `commit-page` reuses the
same two halves with one batched write per target collection.(prepare-one config-atom id)(prepare-one config-atom id overrides)Everything place-one does BEFORE the target write: collect, route,
classify and, for a move, ensure the target and build the record.
overrides as in place-one.
Returns: (r/ok {:prepare/status :ready :bundle b}) b carries :target-coll :record (r/ok {:prepare/status :no-op :result m}) m is place-one's no-op answer (r/err ...) any collect/route/embed failure
Everything `place-one` does BEFORE the target write: collect, route,
classify and, for a move, ensure the target and build the record.
`overrides` as in `place-one`.
Returns:
(r/ok {:prepare/status :ready :bundle b}) b carries :target-coll :record
(r/ok {:prepare/status :no-op :result m}) m is place-one's no-op answer
(r/err ...) any collect/route/embed failure(relocate-one config-atom id)(relocate-one config-atom id overrides)Relocate the entry at id to its canonical collection, REMOVING it from the
source. See place-one (and its overrides); :moved? mirrors :placed?
for legacy callers.
Relocate the entry at `id` to its canonical collection, REMOVING it from the source. See `place-one` (and its `overrides`); `:moved?` mirrors `:placed?` for legacy callers.
(relocate-update config-atom id updates)Update-with-relocation: merge updates into the entry, recompute
its target collection from the merged shape, and either upsert in
place (target == src) or move via relocate-one (target ≠ src).
Replaces the body of entries/update-entry! for the routing-aware
path. Returns r/ok merged-entry on success or r/err on failure.
id is the entry's identity, not an updatable field: an :id inside
updates is overridden, so it can neither redirect the write nor make
the source delete hit another row.
An update that stays in place and leaves the text to embed unchanged
keeps the stored vector (boundary/keep-stored-vector) rather than
calling the embedder again.
Note: relocate-one only relocates the entry as it CURRENTLY is.
When updates change the :type and the new :type routes to a
different collection, we do NOT use relocate-one — we explicitly
write the merged entry to the new collection, then delete from src.
This subtle difference is why relocate-update is its own fn
rather than a wrapper over relocate-one.
Update-with-relocation: merge `updates` into the entry, recompute its target collection from the merged shape, and either upsert in place (target == src) or move via `relocate-one` (target ≠ src). Replaces the body of `entries/update-entry!` for the routing-aware path. Returns r/ok merged-entry on success or r/err on failure. `id` is the entry's identity, not an updatable field: an :id inside `updates` is overridden, so it can neither redirect the write nor make the source delete hit another row. An update that stays in place and leaves the text to embed unchanged keeps the stored vector (`boundary/keep-stored-vector`) rather than calling the embedder again. Note: `relocate-one` only relocates the entry as it CURRENTLY is. When updates change the :type and the new :type routes to a different collection, we do NOT use relocate-one — we explicitly write the merged entry to the new collection, then delete from src. This subtle difference is why `relocate-update` is its own fn rather than a wrapper over `relocate-one`.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |