us.whitford.fulcro.rad.database-adapters.git.embeddings — us.whitford/fulcro-rad-git 0.2.0-RC1

In-process embedding index — format-compatible with git-embed CLI.

Provides:

• Index serialization / deserialization (byte-identical to git-embed Rust format)
• Git-backed persistence via refs/embed/v1/index blob
• Vector math: cosine similarity, truncate-and-normalize, centroid
• Search, similar, score, score-file, cohesion
• ONNX inference: ensure-model!, load-model, embed-document, embed-query, embed-documents (batched), embed-text (chunked)
• update! — walk tree, embed un-indexed blobs, save index
• gc-index — remove embeddings for SHAs no longer in HEAD tree

The binary format is: [version: i32 BE] [model: java-utf] [dims: i32 BE] [count: i32 BE] for each entry: [sha: java-utf] [vector: f32 × dims, each BE IEEE 754]

'java-utf' = DataOutputStream.writeUTF — 2-byte BE length + MUTF-8 bytes. This matches the Rust serialization used by git-embed exactly.

available? clj

(available?)

True if in-process embeddings can be used (ONNX runtime + model files present).

blob-sha->paths clj

(blob-sha->paths repo)

Build {sha → [path ...]} map from a tree walk of the HEAD tree.

centroid clj

(centroid vectors dims)

Compute centroid (mean + L2-normalize) of a collection of float-arrays. Returns nil for empty input.

chunk-threshold clj

clear-index clj

(clear-index repo)

Delete the embedding index ref. Returns true if deleted.

close-model! clj

(close-model! {:keys [session inference-tok counting-tok]})

Close the ONNX session and tokenizers.

cohesion clj

(cohesion repo index space-paths {:keys [dims] :or {dims default-dims}})

Score each file in the space against the space's own centroid. Returns {:files [{:path :score :blob-sha}] :mean float :size int}

cosine-similarity clj

(cosine-similarity a b dims)

Cosine similarity between two float-arrays, truncated to dims dimensions. Returns 0.0 for zero vectors.

default-dims clj

default-model clj

default-top clj

deserialize-index clj

(deserialize-index data)

Deserialize bytes to index map. Reads git-embed Rust serialization.

embed-document clj

(embed-document model content)

Embed document content with search_document: prefix.

embed-documents clj

(embed-documents {:keys [counting-tok batch-config] :as model} texts)

Embed multiple documents with batched inference. Returns vector of float-arrays in same order as input texts. Matches Rust EmbedModel::embed_documents().

embed-query clj

(embed-query model query)

Embed query text with search_query: prefix.

embed-ref clj

embed-text clj

(embed-text {:keys [counting-tok] :as model} text prefix)

Embed text with prefix, handling chunking for texts exceeding chunk-threshold. Returns float-array of model-dims. Matches Rust EmbedModel::embed_text().

empty-index clj

(empty-index)

Return an empty embedding index.

ensure-model! clj

(ensure-model!)

Download model files to model-dir if not already present. Uses same location as git-embed so both tools share files. Returns the model-dir File. Throws on download failure.

gc-index clj

(gc-index repo index)

Remove embeddings for blob SHAs no longer present in the HEAD tree. Returns {:removed int :index updated-index}.

load-config clj

(load-config repo-root)

Load .gitembed from repo-root. Returns parsed config map. Returns {:spaces {}} if .gitembed does not exist.

load-index clj

(load-index repo)

Load embedding index from refs/embed/v1/index via JGit. Returns empty-index if ref doesn't exist. Graceful on corrupt data.

load-model clj

(load-model)

Load the ONNX model and tokenizers from model-dir. Calls ensure-model! if model files are absent. Returns an EmbedModel record. Caller owns the lifecycle — call close-model! when done.

max-seq-length clj

model-dims clj

model-dir clj

(model-dir)

Returns the model directory: ~/.git-embed/models/nomic-embed-text-v1.5/ Same location as git-embed CLI — both tools share model files.

model-id clj

model-name clj

model-ready? clj

(model-ready?)

True if model files exist at ~/.git-embed/models/nomic-embed-text-v1.5/

onnx-available? clj

(onnx-available?)

True if ONNX Runtime is on the classpath.

parse-config clj

(parse-config content)

Parse .gitembed INI content into {:spaces {name {:name str :paths [str]}}}. Format matches git-embed's config.rs exactly:

[space "nucleus"] path = agentus/nucleus/registry/

[space "architecture"] path = mementum/knowledge/architecture/ path = mementum/knowledge/designs/

Empty config returned for blank/comment-only input. Comments: lines starting with # or ;. Blank lines ignored. Unquoted space names are accepted.

prefix-document clj

prefix-query clj

resolve-space-paths clj

(resolve-space-paths config space-name path-prefix)

Resolve path prefixes for a named space or explicit path list.

• space-name: keyword or string name in config — returns that space's :paths
• path-prefix: explicit string prefix — wraps in a single-element vector
• both nil: returns nil (caller handles as 'no restriction')

config is the result of load-config or parse-config.

save-index clj

(save-index repo index)

Serialize index and write to refs/embed/v1/index via JGit.

score clj

(score repo
       index
       space-paths
       target-paths
       {:keys [dims] :or {dims default-dims}})

Score files at target-paths against a concept space defined by space-paths. Space files are excluded from targets.

opts — {:dims 768} Returns {:results [{:path :score :blob-sha}] :space-size int :mean float}

score-file clj

(score-file repo
            index
            space-paths
            target-file
            {:keys [dims] :or {dims default-dims}})

Score a single file against a concept space.

opts — {:dims 768}

search clj

(search repo
        index
        query-vec
        {:keys [dims top path-prefix] :or {dims default-dims top default-top}})

Search index for vectors most similar to query-vec. Returns up to top results sorted by descending cosine similarity.

query-vec — seq or float-array of floats opts — {:dims 768 :top 10 :path-prefix nil}

serialize-index clj

(serialize-index {:keys [version model dims embeddings]})

Serialize index map to bytes. Byte-identical to git-embed Rust serialization.

similar clj

(similar repo
         index
         file-path
         {:keys [dims top path-prefix] :or {dims default-dims top default-top}})

Find files similar to file-path. Returns up to top results.

opts — {:dims 768 :top 10 :path-prefix nil}

text-extensions clj

text-file? clj

(text-file? path)

Returns true if path has a text file extension (matches git-embed TEXT_EXTENSIONS).

truncate-and-normalize clj

(truncate-and-normalize v dims)

Truncate float-array to dims dimensions and L2-normalize. Returns a new float-array.

update! clj

(update! repo model index)

Walk HEAD tree, find un-indexed text blobs, embed them, save index. Requires a loaded EmbedModel. Returns count of newly indexed blobs.