Quickstart

Quickstart

This quickstart guide shows you how to set up XTDB, ingest a document, and query it. In a few minutes, you will have a single XTDB node that persists records to disk using an in-process RocksDB instance. This means you do not need to install RocksDB separately — it will just work out of the box.

Examples are provided in Java and Clojure. Your data will be stored on the same computer your Java or Clojure process is running. This configuration is sufficient for fault-tolerant, low-load production services. It is an easy way to get familiar with XTDB in a minimal production-like environment. The section "Advanced XTDB Configuration" below contains more advanced configuration options.

This guide was extracted from the "XTDB-in-a-Box" guide, which has minimal (working) project examples for Clojure and Java you can see in the crux-in-a-box GitHub repo. If you prefer to see all the code at once, head there first. We also have thorough Reference Docs, if that’s your speed.

Each step below contains some advanced options in collapsible notes. You can safely ignore all of these if you just want to get up and running quickly. You can always revisit them later.

1. Setup

Install Dependencies

Ensure you are using Java 8 or higher:

java -version

All System Requirements

Java: JDK 8 or higher
- JDK 16 requires an extra JVM option to access JVM internals due to JEP 396. (See xtdb#1462.)
Clojure: Clojure 1.10.3 or higher
macOS: 10.14.3 (Mojave) or higher
- MacOS 10.14.3 is a RocksDB libc dependency for macOS users. (See rocksdb#5064.)
- JDK: You may require AdoptOpenJDK on certain macOS versions. (See xtdb#894.)
- Native Arm64 RocksDB Java is currently unspported on Apple M1 processors, causing an UnsatisfiedLinkError. M1 users can still run an x86 JVM on Rosetta, however. (See rocksdb#7720.)
Linux: No known version constraints
Windows: No known version constraints

Add XTDB to your Clojure or Java project:

deps.edn

com.xtdb/xtdb-core {:mvn/version "{xtdb_version}"}
com.xtdb/xtdb-rocksdb {:mvn/version "{xtdb_version}"}

project.clj

[com.xtdb/xtdb-core "{xtdb_version}"]
[com.xtdb/xtdb-rocksdb "{xtdb_version}"]

pom.xml

<dependency>
    <groupId>com.xtdb</groupId>
    <artifactId>xtdb-core</artifactId>
    <version>{xtdb_version}</version>
</dependency>
<dependency>
    <groupId>com.xtdb</groupId>
    <artifactId>xtdb-rocksdb</artifactId>
    <version>{xtdb_version}</version>
</dependency>

Advanced Installation

This guide will walk you through the installation process. If you would prefer to see all available installation options — including pre-built JARs, custom Docker containers, and Clojure CLI tooling — you will want to read the Reference Docs: Installation page.

Advanced Logging

Ignore this note if you just want to get up and running quickly. It’s completely optional, but nice to have.

Without setting up logging, you will see a SLF4J: Defaulting to no-operation (NOP) logger implementation message on STDOUT from XTDB. This is harmless, but in a real application you will want to configure Logback with the SLF4J API to see INFO messages from XTDB.

To do this in Clojure:

Add Logback, SLF4J, and tools.logging as dependencies
Set the clojure.tools.logging.factory JVM option
Create a crux-in-a-box/resources/logback.xml file

To do this in Java:

Add Logback, SLF4J, and tools.logging as dependencies
Set the clojure.tools.logging.factory JVM option
Create a crux-in-a-box/src/main/resources/logback.xml file

Configure XTDB

The following configuration uses an in-process RocksDB instance to write your data to disk. It will save your data in a directory named ./data/dev, relative to your project root. You do not need to create this directory. XTDB will create it for you.

Clojure

(ns xtdb-in-a-box.db
  (:require [clojure.java.io :as io]
            [xtdb.api :as xt]))

(defn start-xtdb! []
  (letfn [(kv-store [dir]
            {:kv-store {:xtdb/module 'xtdb.rocksdb/->kv-store
                        :db-dir (io/file dir)
                        :sync? true}})]
    (xt/start-node
     {:xtdb/tx-log (kv-store "data/dev/tx-log")
      :xtdb/document-store (kv-store "data/dev/doc-store")
      :xtdb/index-store (kv-store "data/dev/index-store")})))

(def xtdb-node (start-xtdb!))

(defn stop-xtdb! []
  (.close xtdb-node))

Java

package xtdbinabox;

import java.io.File;
import java.io.IOException;
import xtdb.api.XTDB;
import xtdb.api.IXtdb;

import java.util.HashMap;
import java.util.List;

import xtdb.api.IXtdbDatasource;
import xtdb.api.ICursor;
import xtdb.api.XtdbDocument;
import xtdb.api.TransactionInstant;
import xtdb.api.tx.Transaction;

class Box {
    public static void main(String[] args) {
        try(IXtdb xtdbNode = IXtdb.startNode(new File("config.json"))) {
            System.out.println("XTDB Started.");
            // ... ingest goes here
            // ... query goes here
        }
        catch (IOException e) {
            // ...
        }
    }
}

config.json

{
  "xtdb/index-store": {
    "kv-store": {
      "xtdb/module": "xtdb.rocksdb/->kv-store",
      "db-dir": "data/index-store"
    }
  },
  "xtdb/document-store": {
    "kv-store": {
      "xtdb/module": "xtdb.rocksdb/->kv-store",
      "db-dir": "data/doc-store"
    }
  },
  "xtdb/tx-log": {
    "kv-store": {
      "xtdb/module": "xtdb.rocksdb/->kv-store",
      "db-dir": "data/tx-log"
    }
  },
  "xtdb.lucene/lucene-store": {
    "db-dir": "data/dev/lucene-dir"
  },
  "xtdb.http-server/server": {
    "port": 9999
  }
}

Advanced Config Libraries

This guide gets you up and running quickly by hard-coding configuration values. Store these configuration values with your favourite configuration library. In Clojure, you might use Aero.

This guide also does not make any assumptions about how you manage stateful services. XTDB is a stateful service, however, and you should store the XTDB node in your favourite state management library. In Clojure, you might use Mount.

Advanced XTDB Configuration

XTDB configuration has three components:

Transaction Log
Document Store
Index Store

For more advanced configuration, XTDB allows you to choose the underlying data storage technology (for each of the three components) from a number of different modules. The Reference Docs contain a list of available modules, each with instructions on how to configure them.

2. Ingest

Clojure REPL

xtdb-in-a-box.db> (xt/submit-tx xtdb-node [[::xt/put
                                            {:xt/id "hi2u"
                                             :user/name "zig"}]])
;; => #::xt{:tx-id 0, :tx-time #inst "2021-03-11T02:27:09.176-00:00"}

Java

HashMap<String, Object> data = new HashMap<>();
data.put("user/name", "zig");
XtdbDocument document = XtdbDocument.create("hi2u", data);
TransactionInstant transaction = node.submitTx(Transaction.buildTx(tx -> {
    tx.put(document);
}));
System.out.println(data.toString());

Advanced Transactions

This is the simplest possible transaction. XTDB has more advanced transaction features including synchronous awaits, eviction, transaction functions, speculative transactions, and bitemporal valid-time put/delete. You can read about them in Reference Docs: Transactions.

3. Query

Clojure REPL

xtdb-in-a-box.db> (xt/q (xt/db xtdb-node) '{:find [e]
                                            :where [[e :user/name "zig"]]} )
;; => #{["hi2u"]}

xtdb-in-a-box.db> (stop-xtdb!)
;; => nil

Java

String query = "{:find [e] :where [[e :user/name \"zig\"]]}";
IXtdbDatasource db = node.db();
ICursor<List<?>> results = db.openQuery(query);
if (results.hasNext()) {
    List<?> result = results.next();
    System.out.println(result.toString());
}
db.close();
node.close();

Advanced Queries

This is the simplest possible query. XTDB has very powerful bitemporal graph queries with Datalog and SQL. To learn more about aggregates, pull syntax, returning maps, binding, subqueries, predicates, ordering, pagination, rules, bitemporal time-travel, streaming, and entity history you can read more in Reference Docs: Queries and Reference Docs: SQL.

1. Setup

Install Dependencies

Configure XTDB

2. Ingest

3. Query

Further Reading