LMDB — juxt/crux-test 21.06-1.17.1

LMDB

LMDB can be used as the data store for Crux’s query indices, but can also be used as a transaction log and/or document store in single node clusters.

LMDB tends to provide faster queries than RocksDB in exchange for a slower ingest rate.

Project Dependency

In order to use LMDB within Crux, you must first add LMDB as a project dependency:

deps.edn

juxt/crux-lmdb {:mvn/version "{crux_version}-alpha"}

pom.xml

<dependency>
    <groupId>juxt</groupId>
    <artifactId>crux-lmdb</artifactId>
    <version>{crux_version}-alpha</version>
</dependency>

Using LMDB

Replace the implementation of the desired component with crux.lmdb/->kv-store

JSON

{
  "crux/index-store": {
    "kv-store": {
      "crux/module": "crux.lmdb/->kv-store",
      "db-dir": "/tmp/lmdb"
    }
  },

  "crux/document-store": { ... },
  "crux/tx-log": { ... }
}

Clojure

{:crux/index-store {:kv-store {:crux/module 'crux.lmdb/->kv-store
                               :db-dir (io/file "/tmp/lmdb")}}
 :crux/document-store {...}
 :crux/tx-log {...}}

EDN

{:crux/index-store {:kv-store {:crux/module crux.lmdb/->kv-store
                               :db-dir "/tmp/lmdb"}}
 :crux/document-store {...}
 :crux/tx-log {...}}

It is generally advised to use independent LMDB instances for each component, although using a single instance for the transaction log and document store is possible. Do not share the LMDB instance used for the index store with other components as you cannot then perform Crux version upgrades.

Parameters

db-dir (required, string/File/Path): path to LMDB data directory
sync? (boolean, default false): sync to disk after every write
env-flags (int): LMDB flags
env-mapsize(int): LMDB map size