datalevin.core — datalevin 0.6.21

User facing API for Datalevin library features

add clj

(add ent attr value)

Add an attribute value to an entity of a Datalog database

add-doc clj

(add-doc engine doc-ref doc-text)

Add a document to the search engine, doc-ref can be arbitrary Clojure data that uniquely refers to the document in the system. doc-text is the content of the document as a string. The search engine does not store the original text, and assumes that caller can retrieve them by doc-ref. This function is for online update of search engine index. Search index is updated with the new text if the doc-ref already exists. For index creation of bulk data, use search-index-writer.

clear clj

(clear conn)

Close the Datalog database, then clear all data, including schema.

clear-dbi clj

(clear-dbi db dbi-name)

Clear data in the DBI (i.e sub-database) of the key-value store, but leave it open

clear-docs clj

(clear-docs engine)

Remove all documents from the search engine index. It is useful because rebuilding search index may be faster than updating some documents.

close clj

(close conn)

Close the connection to a Datalog db

close-db clj

(close-db db)

Close the Datalog database

close-kv clj

(close-kv db)

Close this key-value store

closed-kv? clj

(closed-kv? db)

Return true if this key-value store is closed

closed? clj

(closed? conn)

Return true when the underlying Datalog DB is closed or when conn is nil or contains nil

commit clj

(commit writer)

Commit writes to search index, must be called after writing all documents.

conn-from-datoms clj

(conn-from-datoms datoms)

(conn-from-datoms datoms dir)

(conn-from-datoms datoms dir schema)

(conn-from-datoms datoms dir schema opts)

Create a mutable reference to a Datalog database with the given datoms added to it. dir could be a local directory path or a dtlv connection URI string.

opts map has keys:

• :validate-data?, a boolean, instructing the system to validate data type during transaction. Default is false.

• :auto-entity-time?, a boolean indicating whether to maintain :db/created-at and :db/updated-at values for each entity. Default is false.

• :search-opts, an option map that will be passed to the built-in full-text search engine

• :kv-opts, an option map that will be passed to the underlying kV store

conn-from-db clj

(conn-from-db db)

Creates a mutable reference to a given Datalog database. See create-conn.

conn? clj

(conn? conn)

Returns true if this is an open connection to a local Datalog db, false otherwise.

copy clj

(copy db dest)

(copy db dest compact?)

Copy a Datalog or key-value database to a destination directory path, optionally compact while copying, default not compact.

create-conn clj

(create-conn)

(create-conn dir)

(create-conn dir schema)

(create-conn dir schema opts)

Creates a mutable reference (a “connection”) to a Datalog database at the given location and opens the database. Creates the database if it doesn't exist yet. Update the schema if one is given. Return the connection. dir could be a local directory path or a dtlv connection URI string.

opts map may have keys:

• :validate-data?, a boolean, instructing the system to validate data type during transaction. Default is false.

• :auto-entity-time?, a boolean indicating whether to maintain :db/created-at and :db/updated-at values for each entity. Default is false.

• :search-opts, an option map that will be passed to the built-in full-text search engine

• :kv-opts, an option map that will be passed to the underlying kV store

Please note that the connection should be managed like a stateful resource. Application should hold on to the same connection rather than opening multiple connections to the same database in the same process.

Connections are lightweight in-memory structures (~atoms). See also transact!, db, close, get-conn, and open-kv.

To access underlying DB, deref: @conn.

Usage:

       (create-conn)

       (create-conn "/tmp/test-create-conn")

       (create-conn "/tmp/test-create-conn" {:likes {:db/cardinality :db.cardinality/many}})

       (create-conn "dtlv://datalevin:[e-mail]/mydb" {} {:auto-entity-time? true})

datom clj

(datom e a v)

(datom e a v tx)

(datom e a v tx added)

Low-level fn to create raw datoms in a Datalog db.

Optionally with transaction id (number) and added flag (true for addition, false for retraction).

See also init-db.

datom-a clj

(datom-a d)

Return the attribute of a datom

datom-e clj

(datom-e d)

Return the entity id of a datom

datom-v clj

(datom-v d)

Return the value of a datom

datom? clj

(datom? x)

Returns true if the given value is a datom, false otherwise.

datoms clj

(datoms db index)

(datoms db index c1)

(datoms db index c1 c2)

(datoms db index c1 c2 c3)

(datoms db index c1 c2 c3 c4)

Index lookup in Datalog db. Returns a sequence of datoms (lazy iterator over actual DB index) which components (e, a, v) match passed arguments.

Datoms are sorted in index sort order. Possible index values are: :eav, :ave, or :vea (only available for :db.type/ref datoms).

Usage:

; find all datoms for entity id == 1 (any attrs and values)
; sort by attribute, then value
(datoms db :eav 1)
; => (#datalevin/Datom [1 :friends 2]
;     #datalevin/Datom [1 :likes "fries"]
;     #datalevin/Datom [1 :likes "pizza"]
;     #datalevin/Datom [1 :name "Ivan"])

; find all datoms for entity id == 1 and attribute == :likes (any values)
; sorted by value
(datoms db :eav 1 :likes)
; => (#datalevin/Datom [1 :likes "fries"]
;     #datalevin/Datom [1 :likes "pizza"])

; find all datoms for entity id == 1, attribute == :likes and value == "pizza"
(datoms db :eav 1 :likes "pizza")
; => (#datalevin/Datom [1 :likes "pizza"])

; find all datoms that have attribute == `:likes` and value == `"pizza"` (any entity id)
(datoms db :ave :likes "pizza")
; => (#datalevin/Datom [1 :likes "pizza"]
;     #datalevin/Datom [2 :likes "pizza"])

; find all datoms sorted by entity id, then attribute, then value
(datoms db :eav) ; => (...)

Useful patterns:

; get all values of :db.cardinality/many attribute
(->> (datoms db :eav eid attr) (map :v))

; lookup entity ids by attribute value
(->> (datoms db :ave attr value) (map :e))

; find N entities with lowest attr value (e.g. 10 earliest posts)
(->> (datoms db :ave attr) (take N))

; find N entities with highest attr value (e.g. 10 latest posts)
(->> (datoms db :ave attr) (reverse) (take N))

Gotchas:

• Index lookup is usually more efficient than doing a query with a single clause.
• Resulting iterator is calculated in constant time and small constant memory overhead.

db clj

(db conn)

Returns the underlying Datalog database object from a connection. Note that Datalevin does not have "db as a value" feature, the returned object is NOT a database value, but a reference to the database object.

Exists for Datomic API compatibility.

db? clj

(db? x)

Returns true if the given value is a Datalog database. Has the side effect of updating the cache of the db to the most recent. Return false otherwise.

dir clj

(dir db)

Return the path or URI string of the key-value store

doc-count clj

(doc-count engine)

Return the number of documents in the search index

doc-indexed? clj

(doc-indexed? engine doc-ref)

Test if a doc-ref is already in the search index

doc-refs clj

(doc-refs engine)

Return a seq of doc-ref in the search index

drop-dbi clj

(drop-dbi db dbi-name)

Clear data in the DBI (i.e. sub-database) of the key-value store, then delete it

empty-db clj

(empty-db)

(empty-db dir)

(empty-db dir schema)

(empty-db dir schema opts)

Open a Datalog database at the given location. dir could be a local directory path or a dtlv connection URI string. Creates an empty database there if it does not exist yet. Update the schema if one is given. Return reference to the database.

opts map has keys:

• :validate-data?, a boolean, instructing the system to validate data type during transaction. Default is false.

• :auto-entity-time?, a boolean indicating whether to maintain :db/created-at and :db/updated-at values for each entity. Default is false.

• :search-opts, an option map that will be passed to the built-in full-text search engine

• :kv-opts, an option map that will be passed to the underlying kV store

Usage:

        (empty-db)

        (empty-db "/tmp/test-empty-db")

        (empty-db "/tmp/test-empty-db" {:likes {:db/cardinality :db.cardinality/many}})

        (empty-db "dtlv://datalevin:[e-mail]/mydb" {} {:auto-entity-time? true :search-engine {:analyzer blank-space-analyzer}})

entid clj

(entid db eid)

Given lookup ref [unique-attr value], returns numberic entity id.

db is a Datalog database.

If entity does not exist, returns nil.

For numeric eid returns eid itself (does not check for entity existence in that case).

entity clj

(entity db eid)

Retrieves an entity by its id from a Datalog database. Entities are lazy map-like structures to navigate Datalevin database content.

db is a Datalog database.

For eid pass entity id or lookup attr:

(entity db 1)
(entity db [:unique-attr :value])

If entity does not exist, nil is returned:

(entity db 100500) ; => nil

Creating an entity by id is very cheap, almost no-op, as attr access is on-demand:

(entity db 1) ; => {:db/id 1}

Entity attributes can be lazily accessed through key lookups:

(:attr (entity db 1)) ; => :value
(get (entity db 1) :attr) ; => :value

Cardinality many attributes are returned sequences:

(:attrs (entity db 1)) ; => [:v1 :v2 :v3]

Reference attributes are returned as another entities:

(:ref (entity db 1)) ; => {:db/id 2}
(:ns/ref (entity db 1)) ; => {:db/id 2}

References can be walked backwards by prepending _ to name part of an attribute:

(:_ref (entity db 2)) ; => [{:db/id 1}]
(:ns/_ref (entity db 2)) ; => [{:db/id 1}]

Reverse reference lookup returns sequence of entities unless attribute is marked as :db/component:

(:_component-ref (entity db 2)) ; => {:db/id 1}

Entity gotchas:

• Entities print as map, but are not exactly maps (they have compatible get interface though).
• Entities retain reference to the database.
• Creating an entity by id is very cheap, almost no-op (attributes are looked up on demand).
• Comparing entities just compares their ids. Be careful when comparing entities taken from differenct dbs or from different versions of the same db.
• Accessed entity attributes are cached on entity itself (except backward references).
• When printing, only cached attributes (the ones you have accessed before) are printed. See touch.

entity-db clj

(entity-db entity)

Returns a Datalog db that entity was created from.

entries clj

(entries db dbi-name)

Get the number of data entries in a DBI (i.e. sub-db) of the key-value store

get-conn clj

(get-conn dir)

(get-conn dir schema)

(get-conn dir schema opts)

Obtain an open connection to a Datalog database. dir could be a local directory path or a dtlv connection URI string. Create the database if it does not exist. Reuse the same connection if a connection to the same database already exists. Open the database if it is closed. Return the connection.

See also create-conn and with-conn

get-first clj

(get-first db dbi-name k-range)

(get-first db dbi-name k-range k-type)

(get-first db dbi-name k-range k-type v-type)

(get-first db dbi-name k-range k-type v-type ignore-key?)

Return the first kv pair in the specified key range in the key-value store;

k-range is a vector [range-type k1 k2], range-type can be one of :all, :at-least, :at-most, :closed, :closed-open, :greater-than, :less-than, :open, :open-closed, plus backward variants that put a -back suffix to each of the above, e.g. :all-back;

k-type and v-type are data types of k and v, respectively. The allowed data types are described in read-buffer.

Only the value will be returned if ignore-key? is true; If value is to be ignored, put :ignore as v-type

If both key and value are ignored, return true if found an entry, otherwise return nil.

Examples:

      (get-first lmdb "c" [:all] :long :long)
      ;;==> [0 1]

      ;; ignore value
      (get-first lmdb "c" [:all-back] :long :ignore)
      ;;==> [999 nil]

      ;; ignore key
      (get-first lmdb "a" [:greater-than 9] :long :data true)
      ;;==> {:some :data}

      ;; ignore both, this is like testing if the range is empty
      (get-first lmdb "a" [:greater-than 5] :long :ignore true)
      ;;==> true

get-range clj

(get-range db dbi-name k-range)

(get-range db dbi-name k-range k-type)

(get-range db dbi-name k-range k-type v-type)

(get-range db dbi-name k-range k-type v-type ignore-key?)

Return a seq of kv pairs in the specified key range in the key-value store;

k-range is a vector [range-type k1 k2], range-type can be one of :all, :at-least, :at-most, :closed, :closed-open, :greater-than, :less-than, :open, :open-closed, plus backward variants that put a -back suffix to each of the above, e.g. :all-back;

k-type and v-type are data types of k and v, respectively. The allowed data types are described in read-buffer.

Only the value will be returned if ignore-key? is true, default is false;

If value is to be ignored, put :ignore as v-type

Examples:

      (get-range lmdb "c" [:at-least 9] :long :long)
      ;;==> [[10 11] [11 15] [13 14]]

      ;; ignore value
      (get-range lmdb "c" [:all-back] :long :ignore)
      ;;==> [[999 nil] [998 nil]]

      ;; ignore keys, only return values
      (get-range lmdb "a" [:closed 9 11] :long :long true)
      ;;==> [10 11 12]

      ;; out of range
      (get-range lmdb "c" [:greater-than 1500] :long :ignore)
      ;;==> [] 

get-some clj

(get-some db dbi-name pred k-range)

(get-some db dbi-name pred k-range k-type)

(get-some db dbi-name pred k-range k-type v-type)

(get-some db dbi-name pred k-range k-type v-type ignore-key?)

Return the first kv pair that has logical true value of (pred x) in the key-value store, where pred is a function, x is an IKV fetched from the store, with both key and value fields being a ByteBuffer.

pred can use read-buffer to read the content.

To access store on a server, [[interpret.inter-fn]] should be used to define the pred.

k-range is a vector [range-type k1 k2], range-type can be one of :all, :at-least, :at-most, :closed, :closed-open, :greater-than, :less-than, :open, :open-closed, plus backward variants that put a -back suffix to each of the above, e.g. :all-back;

k-type and v-type are data types of k and v, respectively. The allowed data types are described in read-buffer.

Only the value will be returned if ignore-key? is true; If value is to be ignored, put :ignore as v-type

Examples:

      (require '[datalevin.interpret :as i])

      (def pred (i/inter-fn [kv]
                 (let [^long lk (read-buffer (k kv) :long)]
                  (> lk 15)))

      (get-some lmdb "c" pred [:less-than 20] :long :long)
      ;;==> [16 2]

      ;; ignore key
      (get-some lmdb "c" pred [:greater-than 9] :long :data true)
      ;;==> 16 

get-value clj

(get-value db dbi-name k)

(get-value db dbi-name k k-type)

(get-value db dbi-name k k-type v-type)

(get-value db dbi-name k k-type v-type ignore-key?)

Get kv pair of the specified key k in the key-value store.

k-type and v-type are data types of k and v, respectively. The allowed data types are described in read-buffer.

If ignore-key? is true (default true), only return the value, otherwise return [k v], where v is the value

Examples:

  (get-value lmdb "a" 1)
  ;;==> 2

  ;; specify data types
  (get-value lmdb "a" :annunaki/enki :attr :data)
  ;;==> true

  ;; return key value pair
  (get-value lmdb "a" 1 :data :data false)
  ;;==> [1 2]

  ;; key doesn't exist
  (get-value lmdb "a" 2)
  ;;==> nil 

hexify-string clj

(hexify-string s)

Turn a string into a hexified string

index-range clj

(index-range db attr start end)

Returns part of :avet index between [_ attr start] and [_ attr end] in AVET sort order.

Same properties as datoms.

Usage:

(index-range db :likes "a" "zzzzzzzzz")
; => (#datalevin/Datom [2 :likes "candy"]
;     #datalevin/Datom [1 :likes "fries"]
;     #datalevin/Datom [2 :likes "pie"]
;     #datalevin/Datom [1 :likes "pizza"]
;     #datalevin/Datom [2 :likes "pizza"])

(index-range db :likes "egg" "pineapple")
; => (#datalevin/Datom [1 :likes "fries"]
;     #datalevin/Datom [2 :likes "pie"])

Useful patterns:

; find all entities with age in a specific range (inclusive)
(->> (index-range db :age 18 60) (map :e))

init-db clj

(init-db datoms)

(init-db datoms dir)

(init-db datoms dir schema)

(init-db datoms dir schema opts)

Low-level function for creating a Datalog database quickly from a trusted sequence of datoms, useful for bulk data loading. dir could be a local directory path or a dtlv connection URI string. Does no validation on inputs, so datoms must be well-formed and match schema.

opts map has keys:

• :validate-data?, a boolean, instructing the system to validate data type during transaction. Default is false.

• :auto-entity-time?, a boolean indicating whether to maintain :db/created-at and :db/updated-at values for each entity. Default is false.

• :search-opts, an option map that will be passed to the built-in full-text search engine

• :kv-opts, an option map that will be passed to the underlying kV store

      See also [[datom]], [[new-search-engine]].
  

k clj

(k kv)

Key of a key value pair

list-dbis clj

(list-dbis db)

List the names of the sub-databases in the key-value store

listen! clj

(listen! conn callback)

(listen! conn key callback)

Listen for changes on the given connection to a Datalog db. Whenever a transaction is applied to the database via transact!, the callback is called with the transaction report. key is any opaque unique value.

Idempotent. Calling listen! with the same key twice will override old callback with the new value.

Returns the key under which this listener is registered. See also unlisten!.

new-search-engine clj

(new-search-engine lmdb)

(new-search-engine lmdb opts)

Create a search engine. The search index is stored in the passed-in key-value database opened by open-kv.

opts is an option map that may contain these keys:

• :domain is an identifier string, indicates the domain of this search engine. This way, multiple independent search engines can reside in the same key-value database, each with its own domain identifier.

• :analyzer is a function that takes a text string and return a seq of [term, position, offset], where term is a word, position is the sequence number of the term in the document, and offset is the character offset of the term in the document. E.g. for a blank space analyzer and the document "The quick brown fox jumps over the lazy dog", ["quick" 1 4] would be the second entry of the resulting seq.

• :query-analyzer is a similar function that overrides the analyzer at query time (and not indexing time). Mostly useful for autocomplete search in conjunction with the datalevin.search-utils/prefix-token-filter.

See datalevin.search-utils for some functions to customize search.

open-dbi clj

(open-dbi db dbi-name)

(open-dbi db dbi-name opts)

Open a named DBI (i.e. sub-db) in the key-value store. opts is an option map that may have the following keys:

• :validate-data?, a boolean, instructing the system to validate data type during transaction. Default is false.

• :key-size is the max size of the key in bytes, cannot be greater than 511, default is 511.

• :val-size is the default size of the value in bytes, Datalevin will automatically increase the size if a larger value is transacted.

• :flags is a vector of LMDB Dbi flag keywords, may include :reversekey, :dupsort, integerkey, dupfixed, integerdup, reversedup, or create, default is [:create], see LMDB documentation.

open-kv clj

(open-kv dir)

(open-kv dir opts)

Open a LMDB key-value database, return the connection.

dir is a directory path or a dtlv connection URI string. opts is an option map that may have the following keys:

• :mapsize is the initial size of the database. This will be expanded as needed
• :flags is a vector of keywords corresponding to LMDB environment flags, e.g. :rdonly-env for MDB_RDONLY_ENV, :nosubdir for MDB_NOSUBDIR, and so on. See LMDB Documentation

Please note:

LMDB uses POSIX locks on files, and these locks have issues if one process opens a file multiple times. Because of this, do not mdb_env_open() a file multiple times from a single process. Instead, share the LMDB environment that has opened the file across all threads. Otherwise, if a single process opens the same environment multiple times, closing it once will remove all the locks held on it, and the other instances will be vulnerable to corruption from other processes.'

Therefore, a LMDB connection should be managed as a stateful resource. Multiple connections to the same DB in the same process are not recommended. The recommendation is to use a mutable state management library, for example, in Clojure, use component, mount, integrant, or something similar to hold on to and manage the connection.

opts clj

(opts conn)

Return the option map of the Datalog DB

pull clj

(pull db selector eid)

Fetches data from a Datalog database using recursive declarative description. See docs.datomic.com/on-prem/pull.html.

Unlike entity, returns plain Clojure map (not lazy).

Usage:

          (pull db [:db/id, :name, :likes, {:friends [:db/id :name]}] 1)
          ; => {:db/id   1,
          ;     :name    "Ivan"
          ;     :likes   [:pizza]
          ;     :friends [{:db/id 2, :name "Oleg"}]}

pull-many clj

(pull-many db selector eids)

Same as pull, but accepts sequence of ids and returns sequence of maps.

Usage:

       (pull-many db [:db/id :name] [1 2])
       ; => [{:db/id 1, :name "Ivan"}
       ;     {:db/id 2, :name "Oleg"}]

put-buffer clj

(put-buffer bf x)

(put-buffer bf x x-type)

Put the given type of data x in buffer bf. x-type can be one of the following data types:

• :data (default), arbitrary EDN data, avoid this as keys for range queries
• :string, UTF-8 string
• :long, 64 bits integer
• :float, 32 bits IEEE754 floating point number
• :double, 64 bits IEEE754 floating point number
• :byte, single byte
• :bytes, byte array
• :keyword, EDN keyword
• :symbol, EDN symbol
• :boolean, true or false
• :instant, timestamp, same as java.util.Date
• :uuid, UUID, same as java.util.UUID

If the value is to be put in a LMDB key buffer, it must be less than 511 bytes.

q clj

(q query & inputs)

Executes a Datalog query. See docs.datomic.com/on-prem/query.html.

Usage:

(q '[:find ?value
     :where [_ :likes ?value]
     :timeout 5000]
   db)
; => #{["fries"] ["candy"] ["pie"] ["pizza"]}

range-count clj

(range-count db dbi-name k-range)

(range-count db dbi-name k-range k-type)

Return the number of kv pairs in the specified key range in the key-value store, does not process the kv pairs.

k-range is a vector [range-type k1 k2], range-type can be one of :all, :at-least, :at-most, :closed, :closed-open, :greater-than, :less-than, :open, :open-closed, plus backward variants that put a -back suffix to each of the above, e.g. :all-back;

k-type and v-type are data types of k and v, respectively. The allowed data types are described in read-buffer.

Examples:

      (range-count lmdb "c" [:at-least 9] :long)
      ;;==> 10 

range-filter clj

(range-filter db dbi-name pred k-range)

(range-filter db dbi-name pred k-range k-type)

(range-filter db dbi-name pred k-range k-type v-type)

(range-filter db dbi-name pred k-range k-type v-type ignore-key?)

Return a seq of kv pair in the specified key range in the key-value store, for only those return true value for (pred x), where pred is a function, and x is an IKV, with both key and value fields being a ByteBuffer.

pred can use read-buffer to read the buffer content.

To access store on a server, [[interpret.inter-fn]] should be used to define the pred.

k-range is a vector [range-type k1 k2], range-type can be one of :all, :at-least, :at-most, :closed, :closed-open, :greater-than, :less-than, :open, :open-closed, plus backward variants that put a -back suffix to each of the above, e.g. :all-back;

k-type and v-type are data types of k and v, respectively. The allowed data types are described in read-buffer.

Only the value will be returned if ignore-key? is true; If value is to be ignored, put :ignore as v-type

Examples:

      (require '[datalevin.interpret :as i])

      (def pred (i/inter-fn [kv]
                 (let [^long lk (read-buffer (k kv) :long)]
                  (> lk 15)))

      (range-filter lmdb "a" pred [:less-than 20] :long :long)
      ;;==> [[16 2] [17 3]]

      ;; ignore key
      (range-filter lmdb "a" pred [:greater-than 9] :long :data true)
      ;;==> [16 17] 

range-filter-count clj

(range-filter-count db dbi-name pred k-range)

(range-filter-count db dbi-name pred k-range k-type)

Return the number of kv pairs in the specified key range in the key-value store, for only those return true value for (pred x), where pred is a function, and xis an IKV, with both key and value fields being a ByteBuffer. Does not process the kv pairs.

 `pred` can use [[read-buffer]] to read the buffer content.

  To access store on a server, [[interpret.inter-fn]] should be used to define

the pred.

`k-type` indicates data type of `k` and the allowed data types are described
in [[read-buffer]].

 `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of
 `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`,
 `:less-than`, `:open`, `:open-closed`, plus backward variants that put a
 `-back` suffix to each of the above, e.g. `:all-back`;

 Examples:

          (require '[datalevin.interpret :as i])

          (def pred (i/inter-fn [kv]
                     (let [^long lk (read-buffer (k kv) :long)]
                      (> lk 15))))

          (range-filter-count lmdb "a" pred [:less-than 20] :long)
          ;;==> 3

read-buffer clj

(read-buffer bf)

(read-buffer bf v-type)

Get the given type of data from buffer bf, v-type can be one of the following data types:

• :data (default), arbitrary EDN data
• :string, UTF-8 string
• :long, 64 bits integer
• :float, 32 bits IEEE754 floating point number
• :double, 64 bits IEEE754 floating point number
• :byte, single byte
• :bytes, an byte array
• :keyword, EDN keyword
• :symbol, EDN symbol
• :boolean, true or false
• :instant, timestamp, same as java.util.Date
• :uuid, UUID, same as java.util.UUID

remove-doc clj

(remove-doc engine doc-ref)

Remove a document referred to by doc-ref from the search engine index. A slow operation.

reset-conn! clj

(reset-conn! conn db)

(reset-conn! conn db tx-meta)

Forces underlying conn value to become a Datalog db. Will generate a tx-report that will remove everything from old value and insert everything from the new one.

resolve-tempid clj

(resolve-tempid _db tempids tempid)

Does a lookup in tempids map, returning an entity id that tempid was resolved to.

Exists for Datomic API compatibility. Prefer using map lookup directly if possible.

retract clj

(retract ent attr)

(retract ent attr value)

Remove an attribute from an entity of a Datalog database

rseek-datoms clj

(rseek-datoms db index)

(rseek-datoms db index c1)

(rseek-datoms db index c1 c2)

(rseek-datoms db index c1 c2 c3)

(rseek-datoms db index c1 c2 c3 c4)

Same as seek-datoms, but goes backwards until the beginning of the index.

schema clj

(schema conn)

Return the schema of Datalog DB

search clj

(search engine query)

(search engine query opts)

Issue a query to the search engine. query is a string of words.

 `opts` map may have these keys:

  * `:display` can be one of `:refs` (default), `:offsets`.
    - `:refs` return a lazy sequence of `doc-ref` ordered by relevance.
    - `:offsets` return a lazy sequence of
      `[doc-ref [term1 [offset ...]] [term2 [...]] ...]`,
      ordered by relevance. `term` and `offset` can be used to
      highlight the matched terms and their locations in the documents.
  * `:top` is an integer (default 10), the number of results desired.
  * `:doc-filter` is a boolean function that takes a `doc-ref` and
     determines whether or not to include the corresponding document in the
     results (default is `(constantly true)`)

search-index-writer clj

(search-index-writer lmdb)

(search-index-writer lmdb opts)

Create a writer for writing documents to the search index in bulk. The search index is stored in the passed-in key value database opened by open-kv. See also write and commit.

opts is an option map that may contain these keys:

• :domain is an identifier string, indicates the domain of this search engine. This way, multiple independent search engines can reside in the same key-value database, each with its own domain identifier.

• :analyzer is a function that takes a text string and return a seq of [term, position, offset], where term is a word, position is the sequence number of the term, and offset is the character offset of this term.

seek-datoms clj

(seek-datoms db index)

(seek-datoms db index c1)

(seek-datoms db index c1 c2)

(seek-datoms db index c1 c2 c3)

(seek-datoms db index c1 c2 c3 c4)

Similar to datoms, but will return datoms starting from specified components and including rest of the database until the end of the index.

If no datom matches passed arguments exactly, iterator will start from first datom that could be considered “greater” in index order.

Usage:

(seek-datoms db :eavt 1)
; => (#datalevin/Datom [1 :friends 2]
;     #datalevin/Datom [1 :likes "fries"]
;     #datalevin/Datom [1 :likes "pizza"]
;     #datalevin/Datom [1 :name "Ivan"]
;     #datalevin/Datom [2 :likes "candy"]
;     #datalevin/Datom [2 :likes "pie"]
;     #datalevin/Datom [2 :likes "pizza"])

(seek-datoms db :eavt 1 :name)
; => (#datalevin/Datom [1 :name "Ivan"]
;     #datalevin/Datom [2 :likes "candy"]
;     #datalevin/Datom [2 :likes "pie"]
;     #datalevin/Datom [2 :likes "pizza"])

(seek-datoms db :eavt 2)
; => (#datalevin/Datom [2 :likes "candy"]
;     #datalevin/Datom [2 :likes "pie"]
;     #datalevin/Datom [2 :likes "pizza"])

; no datom [2 :likes "fish"], so starts with one immediately following such in index
(seek-datoms db :eavt 2 :likes "fish")
; => (#datalevin/Datom [2 :likes "pie"]
;     #datalevin/Datom [2 :likes "pizza"])

stat clj

(stat db)

(stat db dbi-name)

Return the statitics of the unnamed top level database or a named DBI (i.e. sub-database) of the key-value store as a map:

• :psize is the size of database page
• :depth is the depth of the B-tree
• :branch-pages is the number of internal pages
• :leaf-pages is the number of leaf pages
• :overflow-pages is the number of overflow-pages
• :entries is the number of data entries

tempid clj

(tempid part)

(tempid part x)

Allocates and returns an unique temporary id (a negative integer). Ignores part. Returns x if it is specified.

Exists for Datomic API compatibility. Prefer using negative integers directly if possible.

touch clj

(touch e)

Forces all entity attributes to be eagerly fetched and cached. Only usable for debug output.

Usage:

         (entity db 1) ; => {:db/id 1}
         (touch (entity db 1)) ; => {:db/id 1, :dislikes [:pie], :likes [:pizza]}

transact clj

(transact conn tx-data)

(transact conn tx-data tx-meta)

Same as transact!, but returns an immediately realized future.

Exists for Datomic API compatibility. Prefer using transact! if possible.

transact! clj

(transact! conn tx-data)

(transact! conn tx-data tx-meta)

Applies transaction to the underlying Datalog database of a connection.

Returns transaction report, a map:

 { :db-before ...
   :db-after  ...
   :tx-data   [...]     ; plain datoms that were added/retracted from db-before
   :tempids   {...}     ; map of tempid from tx-data => assigned entid in db-after
   :tx-meta   tx-meta } ; the exact value you passed as `tx-meta`

Note! conn will be updated in-place and is not returned from transact!.

Usage:

; add a single datom to an existing entity (1)
(transact! conn [[:db/add 1 :name "Ivan"]])

; retract a single datom
(transact! conn [[:db/retract 1 :name "Ivan"]])

; retract single entity attribute
(transact! conn [[:db.fn/retractAttribute 1 :name]])

; ... or equivalently (since Datomic changed its API to support this):
(transact! conn [[:db/retract 1 :name]])

; retract all entity attributes (effectively deletes entity)
(transact! conn [[:db.fn/retractEntity 1]])

; create a new entity (`-1`, as any other negative value, is a tempid
; that will be replaced with Datalevin to a next unused eid)
(transact! conn [[:db/add -1 :name "Ivan"]])

; check assigned id (here `*1` is a result returned from previous `transact!` call)
(def report *1)
(:tempids report) ; => {-1 296}

; check actual datoms inserted
(:tx-data report) ; => [#datalevin/Datom [296 :name "Ivan"]]

; tempid can also be a string
(transact! conn [[:db/add "ivan" :name "Ivan"]])
(:tempids *1) ; => {"ivan" 297}

; reference another entity (must exist)
(transact! conn [[:db/add -1 :friend 296]])

; create an entity and set multiple attributes (in a single transaction
; equal tempids will be replaced with the same unused yet entid)
(transact! conn [[:db/add -1 :name "Ivan"]
                 [:db/add -1 :likes "fries"]
                 [:db/add -1 :likes "pizza"]
                 [:db/add -1 :friend 296]])

; create an entity and set multiple attributes (alternative map form)
(transact! conn [{:db/id  -1
                  :name   "Ivan"
                  :likes  ["fries" "pizza"]
                  :friend 296}])

; update an entity (alternative map form). Can’t retract attributes in
; map form. For cardinality many attrs, value (fish in this example)
; will be added to the list of existing values
(transact! conn [{:db/id  296
                  :name   "Oleg"
                  :likes  ["fish"]}])

; ref attributes can be specified as nested map, that will create netsed entity as well
(transact! conn [{:db/id  -1
                  :name   "Oleg"
                  :friend {:db/id -2
                           :name "Sergey"}])

; reverse attribute name can be used if you want created entity to become
; a value in another entity reference
(transact! conn [{:db/id  -1
                  :name   "Oleg"
                  :_friend 296}])
; equivalent to
(transact! conn [{:db/id  -1, :name   "Oleg"}
                 {:db/id 296, :friend -1}])
; equivalent to
(transact! conn [[:db/add  -1 :name   "Oleg"]
                 {:db/add 296 :friend -1]])

transact-async clj

(transact-async conn tx-data)

(transact-async conn tx-data tx-meta)

Calls transact! on a future thread pool, returning immediately.

transact-kv clj

(transact-kv db txs)

Update DB, insert or delete key value pairs in the key-value store.

txs is a seq of [op dbi-name k v k-type v-type flags] when op is :put, for insertion of a key value pair k and v; or [op dbi-name k k-type] when op is :del, for deletion of key k;

dbi-name is the name of the DBI (i.e sub-db) to be transacted, a string.

k-type, v-type and flags are optional.

k-type indicates the data type of k, and v-type indicates the data type of v. The allowed data types are described in put-buffer

:flags is a vector of LMDB Write flag keywords, may include :nooverwrite, :nodupdata, :current, :reserve, :append, :appenddup, :multiple, see LMDB documentation. Pass in :append when the data is sorted to gain better write performance.

Example:

    (transact-kv
      lmdb
      [ [:put "a" 1 2]
      [:put "a" 'a 1]
      [:put "a" 5 {}]
      [:put "a" :annunaki/enki true :attr :data]
      [:put "a" :datalevin ["hello" "world"]]
      [:put "a" 42 (d/datom 1 :a/b {:id 4}) :long :datom]
      [:put "a" (byte 0x01) #{1 2} :byte :data]
      [:put "a" (byte-array [0x41 0x42]) :bk :bytes :data]
      [:put "a" [-1 -235254457N] 5]
      [:put "a" :a 4]
      [:put "a" :bv (byte-array [0x41 0x42 0x43]) :data :bytes]
      [:put "a" :long 1 :data :long]
      [:put "a" 2 3 :long :long]
      [:del "a" 1]
      [:del "a" :non-exist] ])

tx-data->simulated-report clj

(tx-data->simulated-report db tx-data)

Returns a transaction report without side-effects. Useful for obtaining the would-be db state and the would-be set of datoms.

unhexify-string clj

(unhexify-string s)

Turn a hexified string back into a normal string

unlisten! clj

(unlisten! conn key)

Removes registered listener from connection. See also listen!.

update-schema clj

(update-schema conn schema-update)

(update-schema conn schema-update del-attrs)

(update-schema conn schema-update del-attrs rename-map)

Update the schema of an open connection to a Datalog db.

• schema-update is a map from attribute keywords to maps of corresponding properties.

• del-attrs is a set of attributes to be removed from the schema, if there is no datoms associated with them, otherwise an exception will be thrown.

• rename-map is a map of old attributes to new attributes, for renaming attributes

Return the updated schema.

Example:

  (update-schema conn {:new/attr {:db/valueType :db.type/string}})
  (update-schema conn {:new/attr {:db/valueType :db.type/string}}
                      #{:old/attr1 :old/attr2})
  (update-schema conn nil nil {:old/attr :new/attr}) 

v clj

(v kv)

Value of a key value pair

visit clj

(visit db dbi-name pred k-range)

(visit db dbi-name pred k-range k-type)

Call visitor function on each kv pairs in the specified key range, presumably for side effects. Return nil. Each kv pair is an IKV, with both key and value fields being a ByteBuffer. visitor function can use read-buffer to read the buffer content.

If visitor function returns a special value :datalevin/terminate-visit, the visit will stop immediately.

For client/server usage, [[interpret.inter-fn]] should be used to define the visitor function. For babashka pod usage, defpodfn should be used.

k-type indicates data type of k and the allowed data types are described in read-buffer.

k-range is a vector [range-type k1 k2], range-type can be one of :all, :at-least, :at-most, :closed, :closed-open, :greater-than, :less-than, :open, :open-closed, plus backward variants that put a -back suffix to each of the above, e.g. :all-back;

Examples:

      (require '[datalevin.interpret :as i])
      (import '[java.util Hashmap])

      (def hashmap (HashMap.))
      (def visitor (i/inter-fn [kv]
                     (let [^long k (b/read-buffer (l/k kv) :long)
                           ^long v (b/read-buffer (l/v kv) :long)]
                          (.put hashmap k v))))
      (visit lmdb "a" visitor [:closed 11 19] :long)

with-conn clj macro

(with-conn spec & body)

Evaluate body in the context of an connection to the Datalog database.

If the database does not exist, this will create it. If it is closed, this will open it. However, the connection will be closed in the end of this call. If a database needs to be kept open, use create-conn and hold onto the returned connection. See also create-conn and get-conn

spec is a vector of an identifier of the database connection, a path or dtlv URI string, and optionally a schema map.

Example:

    (with-conn [conn "my-data-path"]
      ;; body)

    (with-conn [conn "my-data-path" {:likes {:db/cardinality :db.cardinality/many}}]
      ;; body)

write clj

(write writer doc-ref doc-text)

Write a document to search index.