RocksDB — juxt/crux-core 20.11-1.13.1-beta-SNAPSHOT

RocksDB

RocksDB is often used as the data store for Crux’s query indices, but can also be used as a transaction log and/or document store in single node clusters.

Project Dependency

In order to use RocksDB within Crux, you must first add RocksDB as a project dependency:

deps.edn

juxt/crux-rocksdb {:mvn/version "{crux_version}-beta"}

pom.xml

<dependency>
    <groupId>juxt</groupId>
    <artifactId>crux-rocksdb</artifactId>
    <version>{crux_version}-beta</version>
</dependency>

Using RocksDB

Replace the implementation of the desired component with crux.rocksdb/->kv-store

JSON

{
  "crux/index-store": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "db-dir": "/tmp/rocksdb"
    }
  },

  "crux/document-store": { ... },
  "crux/tx-log": { ... }
}

Clojure/EDN

{:crux/index-store {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                               :db-dir (io/file "/tmp/rocksdb")}}
 :crux/document-store {...}
 :crux/tx-log {...}}

Dependencies

metrics (function, default no-op): enable RocksDB metrics.

Parameters

db-dir (required, string/File/Path): path to RocksDB data directory
sync? (boolean, default false): sync to disk after every write
disable-wal? (boolean): disables the write-ahead log
db-options (RocksDB Options object): extra options to pass directly to RocksDB.

Monitoring RocksDB

To include RocksDB metrics in monitoring, override the metrics dependency:

JSON

{
  "crux/index-store": {
    "kv-store": {
      "crux/module": "crux.rocksdb/->kv-store",
      "metrics": {
        "crux/module": "crux.rocksdb.metrics/->metrics"
      }
      ...
    }
  },

  "crux/document-store": { ... },
  "crux/tx-log": { ... }
}

Clojure/EDN

{:crux/index-store {:kv-store {:crux/module 'crux.rocksdb/->kv-store
                               :metrics {:crux/module 'crux.rocksdb.metrics/->metrics}}
 :crux/document-store {...}
 :crux/tx-log {...}}

Parameters

instance-name (string, default "rocksdb"): unique name for this instance of RocksDB, used in metrics domains
sample-window (duration, default 3s): sample window of statistics collector