User facing API for Datalevin library features
User facing API for Datalevin library features
(add ent attr value)
Add an attribute value to an entity of a Datalog database
Add an attribute value to an entity of a Datalog database
(add-doc engine doc-ref doc-text)
Add a document to the search engine, doc-ref
can be
arbitrary Clojure data that uniquely refers to the document in the system.
doc-text
is the content of the document as a string. The search engine
does not store the original text, and assumes that caller can retrieve them
by doc-ref
. This function is for online update of search engine index.
Search index is updated with the new text if the doc-ref
already exists.
For index creation of bulk data, use search-index-writer
.
Add a document to the search engine, `doc-ref` can be arbitrary Clojure data that uniquely refers to the document in the system. `doc-text` is the content of the document as a string. The search engine does not store the original text, and assumes that caller can retrieve them by `doc-ref`. This function is for online update of search engine index. Search index is updated with the new text if the `doc-ref` already exists. For index creation of bulk data, use `search-index-writer`.
(clear conn)
Close the Datalog database, then clear all data, including schema.
Close the Datalog database, then clear all data, including schema.
(clear-dbi db dbi-name)
Clear data in the DBI (i.e sub-database) of the key-value store, but leave it open
Clear data in the DBI (i.e sub-database) of the key-value store, but leave it open
(clear-docs engine)
Remove all documents from the search engine index. It is useful because rebuilding search index may be faster than updating some documents.
Remove all documents from the search engine index. It is useful because rebuilding search index may be faster than updating some documents.
(close conn)
Close the connection to a Datalog db
Close the connection to a Datalog db
(closed-kv? db)
Return true if this key-value store is closed
Return true if this key-value store is closed
(closed? conn)
Return true when the underlying Datalog DB is closed or when conn
is nil or contains nil
Return true when the underlying Datalog DB is closed or when `conn` is nil or contains nil
(commit writer)
Commit writes to search index, must be called after writing all documents.
Commit writes to search index, must be called after writing all documents.
(conn-from-datoms datoms)
(conn-from-datoms datoms dir)
(conn-from-datoms datoms dir schema)
(conn-from-datoms datoms dir schema opts)
Create a mutable reference to a Datalog database with the given datoms added to it.
dir
could be a local directory path or a dtlv connection URI string.
opts
map has keys:
:validate-data?
, a boolean, instructing the system to validate data type during transaction. Default is false
.
:auto-entity-time?
, a boolean indicating whether to maintain :db/created-at
and :db/updated-at
values for each entity. Default is false
.
:search-opts
, an option map that will be passed to the built-in full-text search engine
:kv-opts
, an option map that will be passed to the underlying kV store
Create a mutable reference to a Datalog database with the given datoms added to it. `dir` could be a local directory path or a dtlv connection URI string. `opts` map has keys: * `:validate-data?`, a boolean, instructing the system to validate data type during transaction. Default is `false`. * `:auto-entity-time?`, a boolean indicating whether to maintain `:db/created-at` and `:db/updated-at` values for each entity. Default is `false`. * `:search-opts`, an option map that will be passed to the built-in full-text search engine * `:kv-opts`, an option map that will be passed to the underlying kV store
(conn-from-db db)
Creates a mutable reference to a given Datalog database. See create-conn
.
Creates a mutable reference to a given Datalog database. See [[create-conn]].
(conn? conn)
Returns true
if this is an open connection to a local Datalog db, false
otherwise.
Returns `true` if this is an open connection to a local Datalog db, `false` otherwise.
(copy db dest)
(copy db dest compact?)
Copy a Datalog or key-value database to a destination directory path, optionally compact while copying, default not compact.
Copy a Datalog or key-value database to a destination directory path, optionally compact while copying, default not compact.
(create-conn)
(create-conn dir)
(create-conn dir schema)
(create-conn dir schema opts)
Creates a mutable reference (a “connection”) to a Datalog database at the given
location and opens the database. Creates the database if it doesn't
exist yet. Update the schema if one is given. Return the connection.
dir
could be a local directory path or a dtlv connection URI string.
opts
map may have keys:
:validate-data?
, a boolean, instructing the system to validate data type during transaction. Default is false
.
:auto-entity-time?
, a boolean indicating whether to maintain :db/created-at
and :db/updated-at
values for each entity. Default is false
.
:search-opts
, an option map that will be passed to the built-in full-text search engine
:kv-opts
, an option map that will be passed to the underlying kV store
Please note that the connection should be managed like a stateful resource. Application should hold on to the same connection rather than opening multiple connections to the same database in the same process.
Connections are lightweight in-memory structures (~atoms). See also
transact!
, db
, close
, get-conn
, and open-kv
.
To access underlying DB, deref: @conn
.
Usage:
(create-conn)
(create-conn "/tmp/test-create-conn")
(create-conn "/tmp/test-create-conn" {:likes {:db/cardinality :db.cardinality/many}})
(create-conn "dtlv://datalevin:secret@example.host/mydb" {} {:auto-entity-time? true})
Creates a mutable reference (a “connection”) to a Datalog database at the given location and opens the database. Creates the database if it doesn't exist yet. Update the schema if one is given. Return the connection. `dir` could be a local directory path or a dtlv connection URI string. `opts` map may have keys: * `:validate-data?`, a boolean, instructing the system to validate data type during transaction. Default is `false`. * `:auto-entity-time?`, a boolean indicating whether to maintain `:db/created-at` and `:db/updated-at` values for each entity. Default is `false`. * `:search-opts`, an option map that will be passed to the built-in full-text search engine * `:kv-opts`, an option map that will be passed to the underlying kV store Please note that the connection should be managed like a stateful resource. Application should hold on to the same connection rather than opening multiple connections to the same database in the same process. Connections are lightweight in-memory structures (~atoms). See also [[transact!]], [[db]], [[close]], [[get-conn]], and [[open-kv]]. To access underlying DB, deref: `@conn`. Usage: (create-conn) (create-conn "/tmp/test-create-conn") (create-conn "/tmp/test-create-conn" {:likes {:db/cardinality :db.cardinality/many}}) (create-conn "dtlv://datalevin:secret@example.host/mydb" {} {:auto-entity-time? true})
(datom e a v)
(datom e a v tx)
(datom e a v tx added)
Low-level fn to create raw datoms in a Datalog db.
Optionally with transaction id (number) and added
flag (true
for addition, false
for retraction).
See also init-db
.
Low-level fn to create raw datoms in a Datalog db. Optionally with transaction id (number) and `added` flag (`true` for addition, `false` for retraction). See also [[init-db]].
(datom-a d)
Return the attribute of a datom
Return the attribute of a datom
(datom-e d)
Return the entity id of a datom
Return the entity id of a datom
(datom? x)
Returns true
if the given value is a datom, false
otherwise.
Returns `true` if the given value is a datom, `false` otherwise.
(datoms db index)
(datoms db index c1)
(datoms db index c1 c2)
(datoms db index c1 c2 c3)
(datoms db index c1 c2 c3 c4)
Index lookup in Datalog db. Returns a sequence of datoms (lazy iterator over actual DB index) which components (e, a, v) match passed arguments.
Datoms are sorted in index sort order. Possible index
values are: :eav
, :ave
, or :vea
(only available for :db.type/ref datoms).
Usage:
; find all datoms for entity id == 1 (any attrs and values)
; sort by attribute, then value
(datoms db :eav 1)
; => (#datalevin/Datom [1 :friends 2]
; #datalevin/Datom [1 :likes "fries"]
; #datalevin/Datom [1 :likes "pizza"]
; #datalevin/Datom [1 :name "Ivan"])
; find all datoms for entity id == 1 and attribute == :likes (any values)
; sorted by value
(datoms db :eav 1 :likes)
; => (#datalevin/Datom [1 :likes "fries"]
; #datalevin/Datom [1 :likes "pizza"])
; find all datoms for entity id == 1, attribute == :likes and value == "pizza"
(datoms db :eav 1 :likes "pizza")
; => (#datalevin/Datom [1 :likes "pizza"])
; find all datoms that have attribute == `:likes` and value == `"pizza"` (any entity id)
(datoms db :ave :likes "pizza")
; => (#datalevin/Datom [1 :likes "pizza"]
; #datalevin/Datom [2 :likes "pizza"])
; find all datoms sorted by entity id, then attribute, then value
(datoms db :eav) ; => (...)
Useful patterns:
; get all values of :db.cardinality/many attribute
(->> (datoms db :eav eid attr) (map :v))
; lookup entity ids by attribute value
(->> (datoms db :ave attr value) (map :e))
; find N entities with lowest attr value (e.g. 10 earliest posts)
(->> (datoms db :ave attr) (take N))
; find N entities with highest attr value (e.g. 10 latest posts)
(->> (datoms db :ave attr) (reverse) (take N))
Gotchas:
Index lookup in Datalog db. Returns a sequence of datoms (lazy iterator over actual DB index) which components (e, a, v) match passed arguments. Datoms are sorted in index sort order. Possible `index` values are: `:eav`, `:ave`, or `:vea` (only available for :db.type/ref datoms). Usage: ; find all datoms for entity id == 1 (any attrs and values) ; sort by attribute, then value (datoms db :eav 1) ; => (#datalevin/Datom [1 :friends 2] ; #datalevin/Datom [1 :likes "fries"] ; #datalevin/Datom [1 :likes "pizza"] ; #datalevin/Datom [1 :name "Ivan"]) ; find all datoms for entity id == 1 and attribute == :likes (any values) ; sorted by value (datoms db :eav 1 :likes) ; => (#datalevin/Datom [1 :likes "fries"] ; #datalevin/Datom [1 :likes "pizza"]) ; find all datoms for entity id == 1, attribute == :likes and value == "pizza" (datoms db :eav 1 :likes "pizza") ; => (#datalevin/Datom [1 :likes "pizza"]) ; find all datoms that have attribute == `:likes` and value == `"pizza"` (any entity id) (datoms db :ave :likes "pizza") ; => (#datalevin/Datom [1 :likes "pizza"] ; #datalevin/Datom [2 :likes "pizza"]) ; find all datoms sorted by entity id, then attribute, then value (datoms db :eav) ; => (...) Useful patterns: ; get all values of :db.cardinality/many attribute (->> (datoms db :eav eid attr) (map :v)) ; lookup entity ids by attribute value (->> (datoms db :ave attr value) (map :e)) ; find N entities with lowest attr value (e.g. 10 earliest posts) (->> (datoms db :ave attr) (take N)) ; find N entities with highest attr value (e.g. 10 latest posts) (->> (datoms db :ave attr) (reverse) (take N)) Gotchas: - Index lookup is usually more efficient than doing a query with a single clause. - Resulting iterator is calculated in constant time and small constant memory overhead.
(db conn)
Returns the underlying Datalog database object from a connection. Note that Datalevin does not have "db as a value" feature, the returned object is NOT a database value, but a reference to the database object.
Exists for Datomic API compatibility.
Returns the underlying Datalog database object from a connection. Note that Datalevin does not have "db as a value" feature, the returned object is NOT a database value, but a reference to the database object. Exists for Datomic API compatibility.
(db? x)
Returns true
if the given value is a Datalog database. Has the side effect of updating the cache of the db to the most recent. Return false
otherwise.
Returns `true` if the given value is a Datalog database. Has the side effect of updating the cache of the db to the most recent. Return `false` otherwise.
(dir db)
Return the path or URI string of the key-value store
Return the path or URI string of the key-value store
(doc-count engine)
Return the number of documents in the search index
Return the number of documents in the search index
(doc-indexed? engine doc-ref)
Test if a doc-ref
is already in the search index
Test if a `doc-ref` is already in the search index
(doc-refs engine)
Return a seq of doc-ref
in the search index
Return a seq of `doc-ref` in the search index
(drop-dbi db dbi-name)
Clear data in the DBI (i.e. sub-database) of the key-value store, then delete it
Clear data in the DBI (i.e. sub-database) of the key-value store, then delete it
(empty-db)
(empty-db dir)
(empty-db dir schema)
(empty-db dir schema opts)
Open a Datalog database at the given location. dir
could be a local directory path or a dtlv connection URI string. Creates an empty database there if it does not exist yet. Update the schema if one is given. Return reference to the database.
opts
map has keys:
:validate-data?
, a boolean, instructing the system to validate data type during transaction. Default is false
.
:auto-entity-time?
, a boolean indicating whether to maintain :db/created-at
and :db/updated-at
values for each entity. Default is false
.
:search-opts
, an option map that will be passed to the built-in full-text search engine
:kv-opts
, an option map that will be passed to the underlying kV store
Usage:
(empty-db)
(empty-db "/tmp/test-empty-db")
(empty-db "/tmp/test-empty-db" {:likes {:db/cardinality :db.cardinality/many}})
(empty-db "dtlv://datalevin:secret@example.host/mydb" {} {:auto-entity-time? true :search-engine {:analyzer blank-space-analyzer}})
Open a Datalog database at the given location. `dir` could be a local directory path or a dtlv connection URI string. Creates an empty database there if it does not exist yet. Update the schema if one is given. Return reference to the database. `opts` map has keys: * `:validate-data?`, a boolean, instructing the system to validate data type during transaction. Default is `false`. * `:auto-entity-time?`, a boolean indicating whether to maintain `:db/created-at` and `:db/updated-at` values for each entity. Default is `false`. * `:search-opts`, an option map that will be passed to the built-in full-text search engine * `:kv-opts`, an option map that will be passed to the underlying kV store Usage: (empty-db) (empty-db "/tmp/test-empty-db") (empty-db "/tmp/test-empty-db" {:likes {:db/cardinality :db.cardinality/many}}) (empty-db "dtlv://datalevin:secret@example.host/mydb" {} {:auto-entity-time? true :search-engine {:analyzer blank-space-analyzer}})
(entid db eid)
Given lookup ref [unique-attr value]
, returns numberic entity id.
db
is a Datalog database.
If entity does not exist, returns nil
.
For numeric eid
returns eid
itself (does not check for entity
existence in that case).
Given lookup ref `[unique-attr value]`, returns numberic entity id. `db` is a Datalog database. If entity does not exist, returns `nil`. For numeric `eid` returns `eid` itself (does not check for entity existence in that case).
(entity db eid)
Retrieves an entity by its id from a Datalog database. Entities are lazy map-like structures to navigate Datalevin database content.
db
is a Datalog database.
For eid
pass entity id or lookup attr:
(entity db 1)
(entity db [:unique-attr :value])
If entity does not exist, nil
is returned:
(entity db 100500) ; => nil
Creating an entity by id is very cheap, almost no-op, as attr access is on-demand:
(entity db 1) ; => {:db/id 1}
Entity attributes can be lazily accessed through key lookups:
(:attr (entity db 1)) ; => :value
(get (entity db 1) :attr) ; => :value
Cardinality many attributes are returned sequences:
(:attrs (entity db 1)) ; => [:v1 :v2 :v3]
Reference attributes are returned as another entities:
(:ref (entity db 1)) ; => {:db/id 2}
(:ns/ref (entity db 1)) ; => {:db/id 2}
References can be walked backwards by prepending _
to name part
of an attribute:
(:_ref (entity db 2)) ; => [{:db/id 1}]
(:ns/_ref (entity db 2)) ; => [{:db/id 1}]
Reverse reference lookup returns sequence of entities unless
attribute is marked as :db/component
:
(:_component-ref (entity db 2)) ; => {:db/id 1}
Entity gotchas:
touch
.Retrieves an entity by its id from a Datalog database. Entities are lazy map-like structures to navigate Datalevin database content. `db` is a Datalog database. For `eid` pass entity id or lookup attr: (entity db 1) (entity db [:unique-attr :value]) If entity does not exist, `nil` is returned: (entity db 100500) ; => nil Creating an entity by id is very cheap, almost no-op, as attr access is on-demand: (entity db 1) ; => {:db/id 1} Entity attributes can be lazily accessed through key lookups: (:attr (entity db 1)) ; => :value (get (entity db 1) :attr) ; => :value Cardinality many attributes are returned sequences: (:attrs (entity db 1)) ; => [:v1 :v2 :v3] Reference attributes are returned as another entities: (:ref (entity db 1)) ; => {:db/id 2} (:ns/ref (entity db 1)) ; => {:db/id 2} References can be walked backwards by prepending `_` to name part of an attribute: (:_ref (entity db 2)) ; => [{:db/id 1}] (:ns/_ref (entity db 2)) ; => [{:db/id 1}] Reverse reference lookup returns sequence of entities unless attribute is marked as `:db/component`: (:_component-ref (entity db 2)) ; => {:db/id 1} Entity gotchas: - Entities print as map, but are not exactly maps (they have compatible get interface though). - Entities retain reference to the database. - Creating an entity by id is very cheap, almost no-op (attributes are looked up on demand). - Comparing entities just compares their ids. Be careful when comparing entities taken from differenct dbs or from different versions of the same db. - Accessed entity attributes are cached on entity itself (except backward references). - When printing, only cached attributes (the ones you have accessed before) are printed. See [[touch]].
(entity-db entity)
Returns a Datalog db that entity was created from.
Returns a Datalog db that entity was created from.
(entries db dbi-name)
Get the number of data entries in a DBI (i.e. sub-db) of the key-value store
Get the number of data entries in a DBI (i.e. sub-db) of the key-value store
(get-conn dir)
(get-conn dir schema)
(get-conn dir schema opts)
Obtain an open connection to a Datalog database. dir
could be a local directory path or a dtlv connection URI string. Create the database if it does not exist. Reuse the same connection if a connection to the same database already exists. Open the database if it is closed. Return the connection.
See also create-conn
and with-conn
Obtain an open connection to a Datalog database. `dir` could be a local directory path or a dtlv connection URI string. Create the database if it does not exist. Reuse the same connection if a connection to the same database already exists. Open the database if it is closed. Return the connection. See also [[create-conn]] and [[with-conn]]
(get-first db dbi-name k-range)
(get-first db dbi-name k-range k-type)
(get-first db dbi-name k-range k-type v-type)
(get-first db dbi-name k-range k-type v-type ignore-key?)
Return the first kv pair in the specified key range in the key-value store;
k-range
is a vector [range-type k1 k2]
, range-type
can be one of
:all
, :at-least
, :at-most
, :closed
, :closed-open
, :greater-than
,
:less-than
, :open
, :open-closed
, plus backward variants that put a
-back
suffix to each of the above, e.g. :all-back
;
k-type
and v-type
are data types of k
and v
, respectively.
The allowed data types are described in read-buffer
.
Only the value will be returned if ignore-key?
is true
;
If value is to be ignored, put :ignore
as v-type
If both key and value are ignored, return true if found an entry, otherwise return nil.
Examples:
(get-first lmdb "c" [:all] :long :long)
;;==> [0 1]
;; ignore value
(get-first lmdb "c" [:all-back] :long :ignore)
;;==> [999 nil]
;; ignore key
(get-first lmdb "a" [:greater-than 9] :long :data true)
;;==> {:some :data}
;; ignore both, this is like testing if the range is empty
(get-first lmdb "a" [:greater-than 5] :long :ignore true)
;;==> true
Return the first kv pair in the specified key range in the key-value store; `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; `k-type` and `v-type` are data types of `k` and `v`, respectively. The allowed data types are described in [[read-buffer]]. Only the value will be returned if `ignore-key?` is `true`; If value is to be ignored, put `:ignore` as `v-type` If both key and value are ignored, return true if found an entry, otherwise return nil. Examples: (get-first lmdb "c" [:all] :long :long) ;;==> [0 1] ;; ignore value (get-first lmdb "c" [:all-back] :long :ignore) ;;==> [999 nil] ;; ignore key (get-first lmdb "a" [:greater-than 9] :long :data true) ;;==> {:some :data} ;; ignore both, this is like testing if the range is empty (get-first lmdb "a" [:greater-than 5] :long :ignore true) ;;==> true
(get-range db dbi-name k-range)
(get-range db dbi-name k-range k-type)
(get-range db dbi-name k-range k-type v-type)
(get-range db dbi-name k-range k-type v-type ignore-key?)
Return a seq of kv pairs in the specified key range in the key-value store;
k-range
is a vector [range-type k1 k2]
, range-type
can be one of
:all
, :at-least
, :at-most
, :closed
, :closed-open
, :greater-than
,
:less-than
, :open
, :open-closed
, plus backward variants that put a
-back
suffix to each of the above, e.g. :all-back
;
k-type
and v-type
are data types of k
and v
, respectively.
The allowed data types are described in read-buffer
.
Only the value will be returned if ignore-key?
is true
,
default is false
;
If value is to be ignored, put :ignore
as v-type
Examples:
(get-range lmdb "c" [:at-least 9] :long :long)
;;==> [[10 11] [11 15] [13 14]]
;; ignore value
(get-range lmdb "c" [:all-back] :long :ignore)
;;==> [[999 nil] [998 nil]]
;; ignore keys, only return values
(get-range lmdb "a" [:closed 9 11] :long :long true)
;;==> [10 11 12]
;; out of range
(get-range lmdb "c" [:greater-than 1500] :long :ignore)
;;==> []
Return a seq of kv pairs in the specified key range in the key-value store; `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; `k-type` and `v-type` are data types of `k` and `v`, respectively. The allowed data types are described in [[read-buffer]]. Only the value will be returned if `ignore-key?` is `true`, default is `false`; If value is to be ignored, put `:ignore` as `v-type` Examples: (get-range lmdb "c" [:at-least 9] :long :long) ;;==> [[10 11] [11 15] [13 14]] ;; ignore value (get-range lmdb "c" [:all-back] :long :ignore) ;;==> [[999 nil] [998 nil]] ;; ignore keys, only return values (get-range lmdb "a" [:closed 9 11] :long :long true) ;;==> [10 11 12] ;; out of range (get-range lmdb "c" [:greater-than 1500] :long :ignore) ;;==> []
(get-some db dbi-name pred k-range)
(get-some db dbi-name pred k-range k-type)
(get-some db dbi-name pred k-range k-type v-type)
(get-some db dbi-name pred k-range k-type v-type ignore-key?)
Return the first kv pair that has logical true value of (pred x)
in
the key-value store, where pred
is a function, x
is an IKV
fetched from the store, with both key and value fields being a
ByteBuffer
.
pred
can use read-buffer
to read the content.
To access store on a server, [[interpret.inter-fn]] should be used to define the pred
.
k-range
is a vector [range-type k1 k2]
, range-type
can be one of
:all
, :at-least
, :at-most
, :closed
, :closed-open
, :greater-than
,
:less-than
, :open
, :open-closed
, plus backward variants that put a
-back
suffix to each of the above, e.g. :all-back
;
k-type
and v-type
are data types of k
and v
, respectively.
The allowed data types are described in read-buffer
.
Only the value will be returned if ignore-key?
is true
;
If value is to be ignored, put :ignore
as v-type
Examples:
(require '[datalevin.interpret :as i])
(def pred (i/inter-fn [kv]
(let [^long lk (read-buffer (k kv) :long)]
(> lk 15)))
(get-some lmdb "c" pred [:less-than 20] :long :long)
;;==> [16 2]
;; ignore key
(get-some lmdb "c" pred [:greater-than 9] :long :data true)
;;==> 16
Return the first kv pair that has logical true value of `(pred x)` in the key-value store, where `pred` is a function, `x` is an `IKV` fetched from the store, with both key and value fields being a `ByteBuffer`. `pred` can use [[read-buffer]] to read the content. To access store on a server, [[interpret.inter-fn]] should be used to define the `pred`. `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; `k-type` and `v-type` are data types of `k` and `v`, respectively. The allowed data types are described in [[read-buffer]]. Only the value will be returned if `ignore-key?` is `true`; If value is to be ignored, put `:ignore` as `v-type` Examples: (require '[datalevin.interpret :as i]) (def pred (i/inter-fn [kv] (let [^long lk (read-buffer (k kv) :long)] (> lk 15))) (get-some lmdb "c" pred [:less-than 20] :long :long) ;;==> [16 2] ;; ignore key (get-some lmdb "c" pred [:greater-than 9] :long :data true) ;;==> 16
(get-value db dbi-name k)
(get-value db dbi-name k k-type)
(get-value db dbi-name k k-type v-type)
(get-value db dbi-name k k-type v-type ignore-key?)
Get kv pair of the specified key k
in the key-value store.
k-type
and v-type
are data types of k
and v
, respectively.
The allowed data types are described in read-buffer
.
If ignore-key?
is true (default true
), only return the value,
otherwise return [k v]
, where v
is the value
Examples:
(get-value lmdb "a" 1)
;;==> 2
;; specify data types
(get-value lmdb "a" :annunaki/enki :attr :data)
;;==> true
;; return key value pair
(get-value lmdb "a" 1 :data :data false)
;;==> [1 2]
;; key doesn't exist
(get-value lmdb "a" 2)
;;==> nil
Get kv pair of the specified key `k` in the key-value store. `k-type` and `v-type` are data types of `k` and `v`, respectively. The allowed data types are described in [[read-buffer]]. If `ignore-key?` is true (default `true`), only return the value, otherwise return `[k v]`, where `v` is the value Examples: (get-value lmdb "a" 1) ;;==> 2 ;; specify data types (get-value lmdb "a" :annunaki/enki :attr :data) ;;==> true ;; return key value pair (get-value lmdb "a" 1 :data :data false) ;;==> [1 2] ;; key doesn't exist (get-value lmdb "a" 2) ;;==> nil
(hexify-string s)
Turn a string into a hexified string
Turn a string into a hexified string
(index-range db attr start end)
Returns part of :avet
index between [_ attr start]
and [_ attr end]
in AVET sort order.
Same properties as datoms
.
Usage:
(index-range db :likes "a" "zzzzzzzzz")
; => (#datalevin/Datom [2 :likes "candy"]
; #datalevin/Datom [1 :likes "fries"]
; #datalevin/Datom [2 :likes "pie"]
; #datalevin/Datom [1 :likes "pizza"]
; #datalevin/Datom [2 :likes "pizza"])
(index-range db :likes "egg" "pineapple")
; => (#datalevin/Datom [1 :likes "fries"]
; #datalevin/Datom [2 :likes "pie"])
Useful patterns:
; find all entities with age in a specific range (inclusive)
(->> (index-range db :age 18 60) (map :e))
Returns part of `:avet` index between `[_ attr start]` and `[_ attr end]` in AVET sort order. Same properties as [[datoms]]. Usage: (index-range db :likes "a" "zzzzzzzzz") ; => (#datalevin/Datom [2 :likes "candy"] ; #datalevin/Datom [1 :likes "fries"] ; #datalevin/Datom [2 :likes "pie"] ; #datalevin/Datom [1 :likes "pizza"] ; #datalevin/Datom [2 :likes "pizza"]) (index-range db :likes "egg" "pineapple") ; => (#datalevin/Datom [1 :likes "fries"] ; #datalevin/Datom [2 :likes "pie"]) Useful patterns: ; find all entities with age in a specific range (inclusive) (->> (index-range db :age 18 60) (map :e))
(init-db datoms)
(init-db datoms dir)
(init-db datoms dir schema)
(init-db datoms dir schema opts)
Low-level function for creating a Datalog database quickly from a trusted sequence of datoms, useful for bulk data loading. dir
could be a local directory path or a dtlv connection URI string. Does no validation on inputs, so datoms
must be well-formed and match schema.
opts
map has keys:
:validate-data?
, a boolean, instructing the system to validate data type during transaction. Default is false
.
:auto-entity-time?
, a boolean indicating whether to maintain :db/created-at
and :db/updated-at
values for each entity. Default is false
.
:search-opts
, an option map that will be passed to the built-in full-text search engine
:kv-opts
, an option map that will be passed to the underlying kV store
See also [[datom]], [[new-search-engine]].
Low-level function for creating a Datalog database quickly from a trusted sequence of datoms, useful for bulk data loading. `dir` could be a local directory path or a dtlv connection URI string. Does no validation on inputs, so `datoms` must be well-formed and match schema. `opts` map has keys: * `:validate-data?`, a boolean, instructing the system to validate data type during transaction. Default is `false`. * `:auto-entity-time?`, a boolean indicating whether to maintain `:db/created-at` and `:db/updated-at` values for each entity. Default is `false`. * `:search-opts`, an option map that will be passed to the built-in full-text search engine * `:kv-opts`, an option map that will be passed to the underlying kV store See also [[datom]], [[new-search-engine]].
(list-dbis db)
List the names of the sub-databases in the key-value store
List the names of the sub-databases in the key-value store
(listen! conn callback)
(listen! conn key callback)
Listen for changes on the given connection to a Datalog db. Whenever a transaction is applied
to the database via transact!
, the callback is called with the transaction
report. key
is any opaque unique value.
Idempotent. Calling listen!
with the same key twice will override old
callback with the new value.
Returns the key under which this listener is registered. See also unlisten!
.
Listen for changes on the given connection to a Datalog db. Whenever a transaction is applied to the database via [[transact!]], the callback is called with the transaction report. `key` is any opaque unique value. Idempotent. Calling [[listen!]] with the same key twice will override old callback with the new value. Returns the key under which this listener is registered. See also [[unlisten!]].
(new-search-engine lmdb)
(new-search-engine lmdb opts)
Create a search engine. The search index is stored in the passed-in
key-value database opened by open-kv
.
opts
is an option map that may contain these keys:
:domain
is an identifier string, indicates the domain of this search engine.
This way, multiple independent search engines can reside in the same
key-value database, each with its own domain identifier.
:analyzer
is a function that takes a text string and return a seq of
[term, position, offset], where term is a word, position is the sequence
number of the term in the document, and offset is the character offset of
the term in the document. E.g. for a blank space analyzer and the document
"The quick brown fox jumps over the lazy dog", ["quick" 1 4] would be
the second entry of the resulting seq.
:query-analyzer
is a similar function that overrides the analyzer at
query time (and not indexing time). Mostly useful for autocomplete search in
conjunction with the datalevin.search-utils/prefix-token-filter
.
See datalevin.search-utils
for some functions to customize search.
Create a search engine. The search index is stored in the passed-in key-value database opened by [[open-kv]]. `opts` is an option map that may contain these keys: * `:domain` is an identifier string, indicates the domain of this search engine. This way, multiple independent search engines can reside in the same key-value database, each with its own domain identifier. * `:analyzer` is a function that takes a text string and return a seq of [term, position, offset], where term is a word, position is the sequence number of the term in the document, and offset is the character offset of the term in the document. E.g. for a blank space analyzer and the document "The quick brown fox jumps over the lazy dog", ["quick" 1 4] would be the second entry of the resulting seq. * `:query-analyzer` is a similar function that overrides the analyzer at query time (and not indexing time). Mostly useful for autocomplete search in conjunction with the `datalevin.search-utils/prefix-token-filter`. See [[datalevin.search-utils]] for some functions to customize search.
(open-dbi db dbi-name)
(open-dbi db dbi-name opts)
Open a named DBI (i.e. sub-db) in the key-value store. opts
is an option map that may have the following keys:
:validate-data?
, a boolean, instructing the system to validate data type during transaction. Default is false
.
:key-size
is the max size of the key in bytes, cannot be greater than 511, default is 511.
:val-size
is the default size of the value in bytes, Datalevin will automatically increase the size if a larger value is transacted.
:flags
is a vector of LMDB Dbi flag keywords, may include :reversekey
, :dupsort
, integerkey
, dupfixed
, integerdup
, reversedup
, or create
, default is [:create]
, see LMDB documentation.
Open a named DBI (i.e. sub-db) in the key-value store. `opts` is an option map that may have the following keys: * `:validate-data?`, a boolean, instructing the system to validate data type during transaction. Default is `false`. * `:key-size` is the max size of the key in bytes, cannot be greater than 511, default is 511. * `:val-size` is the default size of the value in bytes, Datalevin will automatically increase the size if a larger value is transacted. * `:flags` is a vector of LMDB Dbi flag keywords, may include `:reversekey`, `:dupsort`, `integerkey`, `dupfixed`, `integerdup`, `reversedup`, or `create`, default is `[:create]`, see [LMDB documentation](http://www.lmdb.tech/doc/group__mdb__dbi__open.html).
(open-kv dir)
(open-kv dir opts)
Open a LMDB key-value database, return the connection.
dir
is a directory path or a dtlv connection URI string.
opts
is an option map that may have the following keys:
:mapsize
is the initial size of the database. This will be expanded as needed:flags
is a vector of keywords corresponding to LMDB environment flags, e.g.
:rdonly-env
for MDB_RDONLY_ENV, :nosubdir
for MDB_NOSUBDIR, and so on. See LMDB DocumentationPlease note:
LMDB uses POSIX locks on files, and these locks have issues if one process opens a file multiple times. Because of this, do not mdb_env_open() a file multiple times from a single process. Instead, share the LMDB environment that has opened the file across all threads. Otherwise, if a single process opens the same environment multiple times, closing it once will remove all the locks held on it, and the other instances will be vulnerable to corruption from other processes.'
Therefore, a LMDB connection should be managed as a stateful resource. Multiple connections to the same DB in the same process are not recommended. The recommendation is to use a mutable state management library, for example, in Clojure, use component, mount, integrant, or something similar to hold on to and manage the connection.
Open a LMDB key-value database, return the connection. `dir` is a directory path or a dtlv connection URI string. `opts` is an option map that may have the following keys: * `:mapsize` is the initial size of the database. This will be expanded as needed * `:flags` is a vector of keywords corresponding to LMDB environment flags, e.g. `:rdonly-env` for MDB_RDONLY_ENV, `:nosubdir` for MDB_NOSUBDIR, and so on. See [LMDB Documentation](http://www.lmdb.tech/doc/group__mdb__env.html) Please note: > LMDB uses POSIX locks on files, and these locks have issues if one process > opens a file multiple times. Because of this, do not mdb_env_open() a file > multiple times from a single process. Instead, share the LMDB environment > that has opened the file across all threads. Otherwise, if a single process > opens the same environment multiple times, closing it once will remove all > the locks held on it, and the other instances will be vulnerable to > corruption from other processes.' Therefore, a LMDB connection should be managed as a stateful resource. Multiple connections to the same DB in the same process are not recommended. The recommendation is to use a mutable state management library, for example, in Clojure, use [component](https://github.com/stuartsierra/component), [mount](https://github.com/tolitius/mount), [integrant](https://github.com/weavejester/integrant), or something similar to hold on to and manage the connection.
(opts conn)
Return the option map of the Datalog DB
Return the option map of the Datalog DB
(pull db selector eid)
Fetches data from a Datalog database using recursive declarative description. See docs.datomic.com/on-prem/pull.html.
Unlike entity
, returns plain Clojure map (not lazy).
Usage:
(pull db [:db/id, :name, :likes, {:friends [:db/id :name]}] 1)
; => {:db/id 1,
; :name "Ivan"
; :likes [:pizza]
; :friends [{:db/id 2, :name "Oleg"}]}
Fetches data from a Datalog database using recursive declarative description. See [docs.datomic.com/on-prem/pull.html](https://docs.datomic.com/on-prem/pull.html). Unlike [[entity]], returns plain Clojure map (not lazy). Usage: (pull db [:db/id, :name, :likes, {:friends [:db/id :name]}] 1) ; => {:db/id 1, ; :name "Ivan" ; :likes [:pizza] ; :friends [{:db/id 2, :name "Oleg"}]}
(pull-many db selector eids)
Same as pull
, but accepts sequence of ids and returns
sequence of maps.
Usage:
(pull-many db [:db/id :name] [1 2])
; => [{:db/id 1, :name "Ivan"}
; {:db/id 2, :name "Oleg"}]
Same as [[pull]], but accepts sequence of ids and returns sequence of maps. Usage: (pull-many db [:db/id :name] [1 2]) ; => [{:db/id 1, :name "Ivan"} ; {:db/id 2, :name "Oleg"}]
(put-buffer bf x)
(put-buffer bf x x-type)
Put the given type of data x
in buffer bf
. x-type
can be
one of the following data types:
:data
(default), arbitrary EDN data, avoid this as keys for range queries:string
, UTF-8 string:long
, 64 bits integer:float
, 32 bits IEEE754 floating point number:double
, 64 bits IEEE754 floating point number:byte
, single byte:bytes
, byte array:keyword
, EDN keyword:symbol
, EDN symbol:boolean
, true
or false
:instant
, timestamp, same as java.util.Date
:uuid
, UUID, same as java.util.UUID
If the value is to be put in a LMDB key buffer, it must be less than 511 bytes.
Put the given type of data `x` in buffer `bf`. `x-type` can be one of the following data types: - `:data` (default), arbitrary EDN data, avoid this as keys for range queries - `:string`, UTF-8 string - `:long`, 64 bits integer - `:float`, 32 bits IEEE754 floating point number - `:double`, 64 bits IEEE754 floating point number - `:byte`, single byte - `:bytes`, byte array - `:keyword`, EDN keyword - `:symbol`, EDN symbol - `:boolean`, `true` or `false` - `:instant`, timestamp, same as `java.util.Date` - `:uuid`, UUID, same as `java.util.UUID` If the value is to be put in a LMDB key buffer, it must be less than 511 bytes.
(q query & inputs)
Executes a Datalog query. See docs.datomic.com/on-prem/query.html.
Usage:
(q '[:find ?value
:where [_ :likes ?value]
:timeout 5000]
db)
; => #{["fries"] ["candy"] ["pie"] ["pizza"]}
Executes a Datalog query. See [docs.datomic.com/on-prem/query.html](https://docs.datomic.com/on-prem/query.html). Usage: ``` (q '[:find ?value :where [_ :likes ?value] :timeout 5000] db) ; => #{["fries"] ["candy"] ["pie"] ["pizza"]} ```
(range-count db dbi-name k-range)
(range-count db dbi-name k-range k-type)
Return the number of kv pairs in the specified key range in the key-value store, does not process the kv pairs.
k-range
is a vector [range-type k1 k2]
, range-type
can be one of
:all
, :at-least
, :at-most
, :closed
, :closed-open
, :greater-than
,
:less-than
, :open
, :open-closed
, plus backward variants that put a
-back
suffix to each of the above, e.g. :all-back
;
k-type
and v-type
are data types of k
and v
, respectively.
The allowed data types are described in read-buffer
.
Examples:
(range-count lmdb "c" [:at-least 9] :long)
;;==> 10
Return the number of kv pairs in the specified key range in the key-value store, does not process the kv pairs. `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; `k-type` and `v-type` are data types of `k` and `v`, respectively. The allowed data types are described in [[read-buffer]]. Examples: (range-count lmdb "c" [:at-least 9] :long) ;;==> 10
(range-filter db dbi-name pred k-range)
(range-filter db dbi-name pred k-range k-type)
(range-filter db dbi-name pred k-range k-type v-type)
(range-filter db dbi-name pred k-range k-type v-type ignore-key?)
Return a seq of kv pair in the specified key range in the key-value store, for only those
return true value for (pred x)
, where pred
is a function, and x
is an IKV
, with both key and value fields being a ByteBuffer
.
pred
can use read-buffer
to read the buffer content.
To access store on a server, [[interpret.inter-fn]] should be used to define the pred
.
k-range
is a vector [range-type k1 k2]
, range-type
can be one of
:all
, :at-least
, :at-most
, :closed
, :closed-open
, :greater-than
,
:less-than
, :open
, :open-closed
, plus backward variants that put a
-back
suffix to each of the above, e.g. :all-back
;
k-type
and v-type
are data types of k
and v
, respectively.
The allowed data types are described in read-buffer
.
Only the value will be returned if ignore-key?
is true
;
If value is to be ignored, put :ignore
as v-type
Examples:
(require '[datalevin.interpret :as i])
(def pred (i/inter-fn [kv]
(let [^long lk (read-buffer (k kv) :long)]
(> lk 15)))
(range-filter lmdb "a" pred [:less-than 20] :long :long)
;;==> [[16 2] [17 3]]
;; ignore key
(range-filter lmdb "a" pred [:greater-than 9] :long :data true)
;;==> [16 17]
Return a seq of kv pair in the specified key range in the key-value store, for only those return true value for `(pred x)`, where `pred` is a function, and `x` is an `IKV`, with both key and value fields being a `ByteBuffer`. `pred` can use [[read-buffer]] to read the buffer content. To access store on a server, [[interpret.inter-fn]] should be used to define the `pred`. `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; `k-type` and `v-type` are data types of `k` and `v`, respectively. The allowed data types are described in [[read-buffer]]. Only the value will be returned if `ignore-key?` is `true`; If value is to be ignored, put `:ignore` as `v-type` Examples: (require '[datalevin.interpret :as i]) (def pred (i/inter-fn [kv] (let [^long lk (read-buffer (k kv) :long)] (> lk 15))) (range-filter lmdb "a" pred [:less-than 20] :long :long) ;;==> [[16 2] [17 3]] ;; ignore key (range-filter lmdb "a" pred [:greater-than 9] :long :data true) ;;==> [16 17]
(range-filter-count db dbi-name pred k-range)
(range-filter-count db dbi-name pred k-range k-type)
Return the number of kv pairs in the specified key range in the
key-value store, for only those return true value for (pred x)
, where pred
is a
function, and x
is an IKV
, with both key and value fields being a ByteBuffer
.
Does not process the kv pairs.
`pred` can use [[read-buffer]] to read the buffer content.
To access store on a server, [[interpret.inter-fn]] should be used to define
the pred
.
`k-type` indicates data type of `k` and the allowed data types are described
in [[read-buffer]].
`k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of
`:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`,
`:less-than`, `:open`, `:open-closed`, plus backward variants that put a
`-back` suffix to each of the above, e.g. `:all-back`;
Examples:
(require '[datalevin.interpret :as i])
(def pred (i/inter-fn [kv]
(let [^long lk (read-buffer (k kv) :long)]
(> lk 15))))
(range-filter-count lmdb "a" pred [:less-than 20] :long)
;;==> 3
Return the number of kv pairs in the specified key range in the key-value store, for only those return true value for `(pred x)`, where `pred` is a function, and `x`is an `IKV`, with both key and value fields being a `ByteBuffer`. Does not process the kv pairs. `pred` can use [[read-buffer]] to read the buffer content. To access store on a server, [[interpret.inter-fn]] should be used to define the `pred`. `k-type` indicates data type of `k` and the allowed data types are described in [[read-buffer]]. `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; Examples: (require '[datalevin.interpret :as i]) (def pred (i/inter-fn [kv] (let [^long lk (read-buffer (k kv) :long)] (> lk 15)))) (range-filter-count lmdb "a" pred [:less-than 20] :long) ;;==> 3
(read-buffer bf)
(read-buffer bf v-type)
Get the given type of data from buffer bf
, v-type
can be
one of the following data types:
:data
(default), arbitrary EDN data:string
, UTF-8 string:long
, 64 bits integer:float
, 32 bits IEEE754 floating point number:double
, 64 bits IEEE754 floating point number:byte
, single byte:bytes
, an byte array:keyword
, EDN keyword:symbol
, EDN symbol:boolean
, true
or false
:instant
, timestamp, same as java.util.Date
:uuid
, UUID, same as java.util.UUID
Get the given type of data from buffer `bf`, `v-type` can be one of the following data types: - `:data` (default), arbitrary EDN data - `:string`, UTF-8 string - `:long`, 64 bits integer - `:float`, 32 bits IEEE754 floating point number - `:double`, 64 bits IEEE754 floating point number - `:byte`, single byte - `:bytes`, an byte array - `:keyword`, EDN keyword - `:symbol`, EDN symbol - `:boolean`, `true` or `false` - `:instant`, timestamp, same as `java.util.Date` - `:uuid`, UUID, same as `java.util.UUID`
(remove-doc engine doc-ref)
Remove a document referred to by doc-ref
from the search
engine index. A slow operation.
Remove a document referred to by `doc-ref` from the search engine index. A slow operation.
(reset-conn! conn db)
(reset-conn! conn db tx-meta)
Forces underlying conn
value to become a Datalog db
. Will generate a tx-report that
will remove everything from old value and insert everything from the new one.
Forces underlying `conn` value to become a Datalog `db`. Will generate a tx-report that will remove everything from old value and insert everything from the new one.
(resolve-tempid _db tempids tempid)
Does a lookup in tempids map, returning an entity id that tempid was resolved to.
Exists for Datomic API compatibility. Prefer using map lookup directly if possible.
Does a lookup in tempids map, returning an entity id that tempid was resolved to. Exists for Datomic API compatibility. Prefer using map lookup directly if possible.
(retract ent attr)
(retract ent attr value)
Remove an attribute from an entity of a Datalog database
Remove an attribute from an entity of a Datalog database
(rseek-datoms db index)
(rseek-datoms db index c1)
(rseek-datoms db index c1 c2)
(rseek-datoms db index c1 c2 c3)
(rseek-datoms db index c1 c2 c3 c4)
Same as seek-datoms
, but goes backwards until the beginning of the index.
Same as [[seek-datoms]], but goes backwards until the beginning of the index.
(schema conn)
Return the schema of Datalog DB
Return the schema of Datalog DB
(search engine query)
(search engine query opts)
Issue a query
to the search engine. query
is a string of
words.
`opts` map may have these keys:
* `:display` can be one of `:refs` (default), `:offsets`.
- `:refs` return a lazy sequence of `doc-ref` ordered by relevance.
- `:offsets` return a lazy sequence of
`[doc-ref [term1 [offset ...]] [term2 [...]] ...]`,
ordered by relevance. `term` and `offset` can be used to
highlight the matched terms and their locations in the documents.
* `:top` is an integer (default 10), the number of results desired.
* `:doc-filter` is a boolean function that takes a `doc-ref` and
determines whether or not to include the corresponding document in the
results (default is `(constantly true)`)
Issue a `query` to the search engine. `query` is a string of words. `opts` map may have these keys: * `:display` can be one of `:refs` (default), `:offsets`. - `:refs` return a lazy sequence of `doc-ref` ordered by relevance. - `:offsets` return a lazy sequence of `[doc-ref [term1 [offset ...]] [term2 [...]] ...]`, ordered by relevance. `term` and `offset` can be used to highlight the matched terms and their locations in the documents. * `:top` is an integer (default 10), the number of results desired. * `:doc-filter` is a boolean function that takes a `doc-ref` and determines whether or not to include the corresponding document in the results (default is `(constantly true)`)
(search-index-writer lmdb)
(search-index-writer lmdb opts)
Create a writer for writing documents to the search index in bulk.
The search index is stored in the passed-in key value database opened
by open-kv
. See also write
and commit
.
opts
is an option map that may contain these keys:
:domain
is an identifier string, indicates the domain of this search engine.
This way, multiple independent search engines can reside in the same
key-value database, each with its own domain identifier.
:analyzer
is a function that takes a text string and return a seq of
[term, position, offset], where term is a word, position is the sequence
number of the term, and offset is the character offset of this term.
Create a writer for writing documents to the search index in bulk. The search index is stored in the passed-in key value database opened by [[open-kv]]. See also [[write]] and [[commit]]. `opts` is an option map that may contain these keys: * `:domain` is an identifier string, indicates the domain of this search engine. This way, multiple independent search engines can reside in the same key-value database, each with its own domain identifier. * `:analyzer` is a function that takes a text string and return a seq of [term, position, offset], where term is a word, position is the sequence number of the term, and offset is the character offset of this term.
(seek-datoms db index)
(seek-datoms db index c1)
(seek-datoms db index c1 c2)
(seek-datoms db index c1 c2 c3)
(seek-datoms db index c1 c2 c3 c4)
Similar to datoms
, but will return datoms starting from specified components and including rest of the database until the end of the index.
If no datom matches passed arguments exactly, iterator will start from first datom that could be considered “greater” in index order.
Usage:
(seek-datoms db :eavt 1)
; => (#datalevin/Datom [1 :friends 2]
; #datalevin/Datom [1 :likes "fries"]
; #datalevin/Datom [1 :likes "pizza"]
; #datalevin/Datom [1 :name "Ivan"]
; #datalevin/Datom [2 :likes "candy"]
; #datalevin/Datom [2 :likes "pie"]
; #datalevin/Datom [2 :likes "pizza"])
(seek-datoms db :eavt 1 :name)
; => (#datalevin/Datom [1 :name "Ivan"]
; #datalevin/Datom [2 :likes "candy"]
; #datalevin/Datom [2 :likes "pie"]
; #datalevin/Datom [2 :likes "pizza"])
(seek-datoms db :eavt 2)
; => (#datalevin/Datom [2 :likes "candy"]
; #datalevin/Datom [2 :likes "pie"]
; #datalevin/Datom [2 :likes "pizza"])
; no datom [2 :likes "fish"], so starts with one immediately following such in index
(seek-datoms db :eavt 2 :likes "fish")
; => (#datalevin/Datom [2 :likes "pie"]
; #datalevin/Datom [2 :likes "pizza"])
Similar to [[datoms]], but will return datoms starting from specified components and including rest of the database until the end of the index. If no datom matches passed arguments exactly, iterator will start from first datom that could be considered “greater” in index order. Usage: (seek-datoms db :eavt 1) ; => (#datalevin/Datom [1 :friends 2] ; #datalevin/Datom [1 :likes "fries"] ; #datalevin/Datom [1 :likes "pizza"] ; #datalevin/Datom [1 :name "Ivan"] ; #datalevin/Datom [2 :likes "candy"] ; #datalevin/Datom [2 :likes "pie"] ; #datalevin/Datom [2 :likes "pizza"]) (seek-datoms db :eavt 1 :name) ; => (#datalevin/Datom [1 :name "Ivan"] ; #datalevin/Datom [2 :likes "candy"] ; #datalevin/Datom [2 :likes "pie"] ; #datalevin/Datom [2 :likes "pizza"]) (seek-datoms db :eavt 2) ; => (#datalevin/Datom [2 :likes "candy"] ; #datalevin/Datom [2 :likes "pie"] ; #datalevin/Datom [2 :likes "pizza"]) ; no datom [2 :likes "fish"], so starts with one immediately following such in index (seek-datoms db :eavt 2 :likes "fish") ; => (#datalevin/Datom [2 :likes "pie"] ; #datalevin/Datom [2 :likes "pizza"])
(stat db)
(stat db dbi-name)
Return the statitics of the unnamed top level database or a named DBI (i.e. sub-database) of the key-value store as a map:
:psize
is the size of database page:depth
is the depth of the B-tree:branch-pages
is the number of internal pages:leaf-pages
is the number of leaf pages:overflow-pages
is the number of overflow-pages:entries
is the number of data entriesReturn the statitics of the unnamed top level database or a named DBI (i.e. sub-database) of the key-value store as a map: * `:psize` is the size of database page * `:depth` is the depth of the B-tree * `:branch-pages` is the number of internal pages * `:leaf-pages` is the number of leaf pages * `:overflow-pages` is the number of overflow-pages * `:entries` is the number of data entries
(tempid part)
(tempid part x)
Allocates and returns an unique temporary id (a negative integer). Ignores part
. Returns x
if it is specified.
Exists for Datomic API compatibility. Prefer using negative integers directly if possible.
Allocates and returns an unique temporary id (a negative integer). Ignores `part`. Returns `x` if it is specified. Exists for Datomic API compatibility. Prefer using negative integers directly if possible.
(touch e)
Forces all entity attributes to be eagerly fetched and cached. Only usable for debug output.
Usage:
(entity db 1) ; => {:db/id 1}
(touch (entity db 1)) ; => {:db/id 1, :dislikes [:pie], :likes [:pizza]}
Forces all entity attributes to be eagerly fetched and cached. Only usable for debug output. Usage: (entity db 1) ; => {:db/id 1} (touch (entity db 1)) ; => {:db/id 1, :dislikes [:pie], :likes [:pizza]}
(transact! conn tx-data)
(transact! conn tx-data tx-meta)
Applies transaction to the underlying Datalog database of a connection.
Returns transaction report, a map:
{ :db-before ...
:db-after ...
:tx-data [...] ; plain datoms that were added/retracted from db-before
:tempids {...} ; map of tempid from tx-data => assigned entid in db-after
:tx-meta tx-meta } ; the exact value you passed as `tx-meta`
Note! conn
will be updated in-place and is not returned from transact!
.
Usage:
; add a single datom to an existing entity (1)
(transact! conn [[:db/add 1 :name "Ivan"]])
; retract a single datom
(transact! conn [[:db/retract 1 :name "Ivan"]])
; retract single entity attribute
(transact! conn [[:db.fn/retractAttribute 1 :name]])
; ... or equivalently (since Datomic changed its API to support this):
(transact! conn [[:db/retract 1 :name]])
; retract all entity attributes (effectively deletes entity)
(transact! conn [[:db.fn/retractEntity 1]])
; create a new entity (`-1`, as any other negative value, is a tempid
; that will be replaced with Datalevin to a next unused eid)
(transact! conn [[:db/add -1 :name "Ivan"]])
; check assigned id (here `*1` is a result returned from previous `transact!` call)
(def report *1)
(:tempids report) ; => {-1 296}
; check actual datoms inserted
(:tx-data report) ; => [#datalevin/Datom [296 :name "Ivan"]]
; tempid can also be a string
(transact! conn [[:db/add "ivan" :name "Ivan"]])
(:tempids *1) ; => {"ivan" 297}
; reference another entity (must exist)
(transact! conn [[:db/add -1 :friend 296]])
; create an entity and set multiple attributes (in a single transaction
; equal tempids will be replaced with the same unused yet entid)
(transact! conn [[:db/add -1 :name "Ivan"]
[:db/add -1 :likes "fries"]
[:db/add -1 :likes "pizza"]
[:db/add -1 :friend 296]])
; create an entity and set multiple attributes (alternative map form)
(transact! conn [{:db/id -1
:name "Ivan"
:likes ["fries" "pizza"]
:friend 296}])
; update an entity (alternative map form). Can’t retract attributes in
; map form. For cardinality many attrs, value (fish in this example)
; will be added to the list of existing values
(transact! conn [{:db/id 296
:name "Oleg"
:likes ["fish"]}])
; ref attributes can be specified as nested map, that will create netsed entity as well
(transact! conn [{:db/id -1
:name "Oleg"
:friend {:db/id -2
:name "Sergey"}])
; reverse attribute name can be used if you want created entity to become
; a value in another entity reference
(transact! conn [{:db/id -1
:name "Oleg"
:_friend 296}])
; equivalent to
(transact! conn [{:db/id -1, :name "Oleg"}
{:db/id 296, :friend -1}])
; equivalent to
(transact! conn [[:db/add -1 :name "Oleg"]
{:db/add 296 :friend -1]])
Applies transaction to the underlying Datalog database of a connection. Returns transaction report, a map: { :db-before ... :db-after ... :tx-data [...] ; plain datoms that were added/retracted from db-before :tempids {...} ; map of tempid from tx-data => assigned entid in db-after :tx-meta tx-meta } ; the exact value you passed as `tx-meta` Note! `conn` will be updated in-place and is not returned from [[transact!]]. Usage: ; add a single datom to an existing entity (1) (transact! conn [[:db/add 1 :name "Ivan"]]) ; retract a single datom (transact! conn [[:db/retract 1 :name "Ivan"]]) ; retract single entity attribute (transact! conn [[:db.fn/retractAttribute 1 :name]]) ; ... or equivalently (since Datomic changed its API to support this): (transact! conn [[:db/retract 1 :name]]) ; retract all entity attributes (effectively deletes entity) (transact! conn [[:db.fn/retractEntity 1]]) ; create a new entity (`-1`, as any other negative value, is a tempid ; that will be replaced with Datalevin to a next unused eid) (transact! conn [[:db/add -1 :name "Ivan"]]) ; check assigned id (here `*1` is a result returned from previous `transact!` call) (def report *1) (:tempids report) ; => {-1 296} ; check actual datoms inserted (:tx-data report) ; => [#datalevin/Datom [296 :name "Ivan"]] ; tempid can also be a string (transact! conn [[:db/add "ivan" :name "Ivan"]]) (:tempids *1) ; => {"ivan" 297} ; reference another entity (must exist) (transact! conn [[:db/add -1 :friend 296]]) ; create an entity and set multiple attributes (in a single transaction ; equal tempids will be replaced with the same unused yet entid) (transact! conn [[:db/add -1 :name "Ivan"] [:db/add -1 :likes "fries"] [:db/add -1 :likes "pizza"] [:db/add -1 :friend 296]]) ; create an entity and set multiple attributes (alternative map form) (transact! conn [{:db/id -1 :name "Ivan" :likes ["fries" "pizza"] :friend 296}]) ; update an entity (alternative map form). Can’t retract attributes in ; map form. For cardinality many attrs, value (fish in this example) ; will be added to the list of existing values (transact! conn [{:db/id 296 :name "Oleg" :likes ["fish"]}]) ; ref attributes can be specified as nested map, that will create netsed entity as well (transact! conn [{:db/id -1 :name "Oleg" :friend {:db/id -2 :name "Sergey"}]) ; reverse attribute name can be used if you want created entity to become ; a value in another entity reference (transact! conn [{:db/id -1 :name "Oleg" :_friend 296}]) ; equivalent to (transact! conn [{:db/id -1, :name "Oleg"} {:db/id 296, :friend -1}]) ; equivalent to (transact! conn [[:db/add -1 :name "Oleg"] {:db/add 296 :friend -1]])
(transact-async conn tx-data)
(transact-async conn tx-data tx-meta)
Calls transact!
on a future thread pool, returning immediately.
Calls [[transact!]] on a future thread pool, returning immediately.
(transact-kv db txs)
Update DB, insert or delete key value pairs in the key-value store.
txs
is a seq of [op dbi-name k v k-type v-type flags]
when op
is :put
, for insertion of a key value pair k
and v
;
or [op dbi-name k k-type]
when op
is :del
, for deletion of key k
;
dbi-name
is the name of the DBI (i.e sub-db) to be transacted, a string.
k-type
, v-type
and flags
are optional.
k-type
indicates the data type of k
, and v-type
indicates the data type
of v
. The allowed data types are described in put-buffer
:flags
is a vector of LMDB Write flag keywords, may include :nooverwrite
, :nodupdata
, :current
, :reserve
, :append
, :appenddup
, :multiple
, see LMDB documentation.
Pass in :append
when the data is sorted to gain better write performance.
Example:
(transact-kv
lmdb
[ [:put "a" 1 2]
[:put "a" 'a 1]
[:put "a" 5 {}]
[:put "a" :annunaki/enki true :attr :data]
[:put "a" :datalevin ["hello" "world"]]
[:put "a" 42 (d/datom 1 :a/b {:id 4}) :long :datom]
[:put "a" (byte 0x01) #{1 2} :byte :data]
[:put "a" (byte-array [0x41 0x42]) :bk :bytes :data]
[:put "a" [-1 -235254457N] 5]
[:put "a" :a 4]
[:put "a" :bv (byte-array [0x41 0x42 0x43]) :data :bytes]
[:put "a" :long 1 :data :long]
[:put "a" 2 3 :long :long]
[:del "a" 1]
[:del "a" :non-exist] ])
Update DB, insert or delete key value pairs in the key-value store. `txs` is a seq of `[op dbi-name k v k-type v-type flags]` when `op` is `:put`, for insertion of a key value pair `k` and `v`; or `[op dbi-name k k-type]` when `op` is `:del`, for deletion of key `k`; `dbi-name` is the name of the DBI (i.e sub-db) to be transacted, a string. `k-type`, `v-type` and `flags` are optional. `k-type` indicates the data type of `k`, and `v-type` indicates the data type of `v`. The allowed data types are described in [[put-buffer]] `:flags` is a vector of LMDB Write flag keywords, may include `:nooverwrite`, `:nodupdata`, `:current`, `:reserve`, `:append`, `:appenddup`, `:multiple`, see [LMDB documentation](http://www.lmdb.tech/doc/group__mdb__put.html). Pass in `:append` when the data is sorted to gain better write performance. Example: (transact-kv lmdb [ [:put "a" 1 2] [:put "a" 'a 1] [:put "a" 5 {}] [:put "a" :annunaki/enki true :attr :data] [:put "a" :datalevin ["hello" "world"]] [:put "a" 42 (d/datom 1 :a/b {:id 4}) :long :datom] [:put "a" (byte 0x01) #{1 2} :byte :data] [:put "a" (byte-array [0x41 0x42]) :bk :bytes :data] [:put "a" [-1 -235254457N] 5] [:put "a" :a 4] [:put "a" :bv (byte-array [0x41 0x42 0x43]) :data :bytes] [:put "a" :long 1 :data :long] [:put "a" 2 3 :long :long] [:del "a" 1] [:del "a" :non-exist] ])
(tx-data->simulated-report db tx-data)
Returns a transaction report without side-effects. Useful for obtaining the would-be db state and the would-be set of datoms.
Returns a transaction report without side-effects. Useful for obtaining the would-be db state and the would-be set of datoms.
(unhexify-string s)
Turn a hexified string back into a normal string
Turn a hexified string back into a normal string
(unlisten! conn key)
Removes registered listener from connection. See also listen!
.
Removes registered listener from connection. See also [[listen!]].
(update-schema conn schema-update)
(update-schema conn schema-update del-attrs)
(update-schema conn schema-update del-attrs rename-map)
Update the schema of an open connection to a Datalog db.
schema-update
is a map from attribute keywords to maps of corresponding
properties.
del-attrs
is a set of attributes to be removed from the schema, if there is
no datoms associated with them, otherwise an exception will be thrown.
rename-map
is a map of old attributes to new attributes, for renaming
attributes
Return the updated schema.
Example:
(update-schema conn {:new/attr {:db/valueType :db.type/string}})
(update-schema conn {:new/attr {:db/valueType :db.type/string}}
#{:old/attr1 :old/attr2})
(update-schema conn nil nil {:old/attr :new/attr})
Update the schema of an open connection to a Datalog db. * `schema-update` is a map from attribute keywords to maps of corresponding properties. * `del-attrs` is a set of attributes to be removed from the schema, if there is no datoms associated with them, otherwise an exception will be thrown. * `rename-map` is a map of old attributes to new attributes, for renaming attributes Return the updated schema. Example: (update-schema conn {:new/attr {:db/valueType :db.type/string}}) (update-schema conn {:new/attr {:db/valueType :db.type/string}} #{:old/attr1 :old/attr2}) (update-schema conn nil nil {:old/attr :new/attr})
(visit db dbi-name pred k-range)
(visit db dbi-name pred k-range k-type)
Call visitor
function on each kv pairs in the specified key range, presumably for side effects. Return nil. Each kv pair is an IKV
, with both key and value fields being a ByteBuffer
. visitor
function can use read-buffer
to read the buffer content.
If visitor
function returns a special value :datalevin/terminate-visit
, the visit will stop immediately.
For client/server usage, [[interpret.inter-fn]] should be used to define the visitor
function. For babashka pod usage, defpodfn
should be used.
k-type
indicates data type of k
and the allowed data types are described
in read-buffer
.
k-range
is a vector [range-type k1 k2]
, range-type
can be one of
:all
, :at-least
, :at-most
, :closed
, :closed-open
, :greater-than
,
:less-than
, :open
, :open-closed
, plus backward variants that put a
-back
suffix to each of the above, e.g. :all-back
;
Examples:
(require '[datalevin.interpret :as i])
(import '[java.util Hashmap])
(def hashmap (HashMap.))
(def visitor (i/inter-fn [kv]
(let [^long k (b/read-buffer (l/k kv) :long)
^long v (b/read-buffer (l/v kv) :long)]
(.put hashmap k v))))
(visit lmdb "a" visitor [:closed 11 19] :long)
Call `visitor` function on each kv pairs in the specified key range, presumably for side effects. Return nil. Each kv pair is an `IKV`, with both key and value fields being a `ByteBuffer`. `visitor` function can use [[read-buffer]] to read the buffer content. If `visitor` function returns a special value `:datalevin/terminate-visit`, the visit will stop immediately. For client/server usage, [[interpret.inter-fn]] should be used to define the `visitor` function. For babashka pod usage, `defpodfn` should be used. `k-type` indicates data type of `k` and the allowed data types are described in [[read-buffer]]. `k-range` is a vector `[range-type k1 k2]`, `range-type` can be one of `:all`, `:at-least`, `:at-most`, `:closed`, `:closed-open`, `:greater-than`, `:less-than`, `:open`, `:open-closed`, plus backward variants that put a `-back` suffix to each of the above, e.g. `:all-back`; Examples: (require '[datalevin.interpret :as i]) (import '[java.util Hashmap]) (def hashmap (HashMap.)) (def visitor (i/inter-fn [kv] (let [^long k (b/read-buffer (l/k kv) :long) ^long v (b/read-buffer (l/v kv) :long)] (.put hashmap k v)))) (visit lmdb "a" visitor [:closed 11 19] :long)
(with-conn spec & body)
Evaluate body in the context of an connection to the Datalog database.
If the database does not exist, this will create it. If it is closed,
this will open it. However, the connection will be closed in the end of
this call. If a database needs to be kept open, use create-conn
and
hold onto the returned connection. See also create-conn
and get-conn
spec
is a vector of an identifier of the database connection, a path or
dtlv URI string, and optionally a schema map.
Example:
(with-conn [conn "my-data-path"]
;; body)
(with-conn [conn "my-data-path" {:likes {:db/cardinality :db.cardinality/many}}]
;; body)
Evaluate body in the context of an connection to the Datalog database. If the database does not exist, this will create it. If it is closed, this will open it. However, the connection will be closed in the end of this call. If a database needs to be kept open, use `create-conn` and hold onto the returned connection. See also [[create-conn]] and [[get-conn]] `spec` is a vector of an identifier of the database connection, a path or dtlv URI string, and optionally a schema map. Example: (with-conn [conn "my-data-path"] ;; body) (with-conn [conn "my-data-path" {:likes {:db/cardinality :db.cardinality/many}}] ;; body)
(write writer doc-ref doc-text)
Write a document to search index.
Write a document to search index.
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close