nd-db.core — com.luposlip/nd-db 0.4.0

combinr clj

(combinr)

(combinr _)

(combinr acc more)

create-index clj

(create-index filename idx-id)

Builds up an index of Entity IDs as keys (IDs extracted with id-fn), and as value a vector with 2 values:

• the start index in the text file to start read EDN for the input doc
• the length in bytes input doc

Default lines processed per batch per core is set to 8, which is pretty low. This is due to potential memory issues. You can tweak this with environment variable (bash):

export NDDB_LINES_PER_CORE=250

db clj

(db {:keys [id-fn id-name id-type doc-type filename]
     :or {id-type :string}
     :as params})

Creates a database var which can be used to perform queries

db? clj

(db? candidate)

get-id-fn clj

(get-id-fn {:keys [id-name id-type source-type]})

index-fns clj

In the form {"filename1" {"9298cvoa" {index-w-first-few-ids-as-str-=-9298cvoa} "filename2" {"ao3oijf8" {index-w-first-few-ids-as-str-=-ao3oijf8}}}

index-id clj

(index-id {:keys [id-fn filename id-name id-type] :as params})

This function generates a pseudo unique index ID for the combination of the ID function and the filename.

infer-doctype clj

(infer-doctype filename)

parse-doc clj

(parse-doc db doc-str)

q clj multimethod

Queries a single or multiple docs from the database by a single or multiple IDs matching those from the .nd* database by id-fn.

• returns EDN for the matching document.

reducr clj

(reducr id-fn)