Building — com.xtdb/xtdb-core 1.19.0

Building

build is a module within the XTDB repo with two key objectives:

Providing pre-configured JARs and Docker containers to start a basic implementation of XTDB with no prior knowledge required.
Providing a mechanism to spin up custom XTDB artifacts - with two necessary inputs:
- A deps.edn file containing the dependencies that the node requires.
- A xtdb.edn file, containing the configuration options to start the node with.

Preconfigured Artifacts

`xtdb-in-memory`

The xtdb-in-memory artifact starts up a basic, in-memory XTDB node, with both a HTTP Server (open on port 3000) and the XTDB SQL module with a SQL server (open on port 1501).

It can be downloaded:

as a Docker image from JUXT’s Docker Hub
as an uberjar, xtdb-in-memory.jar, from the relevant GitHub release

Communication with the node is done via the XTDB REST API - see the HTTP docs for more information.

Building Artifacts

Alongside the JARs deployed on the GitHub releases is xtdb-builder.tar.gz - the scripts within this archive can be used to build a custom XTDB JAR or Docker container.

Building a JAR (Clojure CLI tooling)

The clj-uberjar folder contains a number of files:

a deps.edn file to configure the Maven dependencies required
a xtdb.edn file to configure the node itself
a resources/logback.xml to configure logging output
a build-uberjar.sh script to build the JAR.

To use rocksdb as the index store, document store and transaction-log store of the node:

Add xtdb-rocksdb as a dependency in deps.edn:

...
com.xtdb/xtdb-rocksdb {:mvn/version "{xtdb_version}"}
...

In xtdb.edn, override the old topology with the following:

{:xtdb/index-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/tmp/xtdb/indexes"}}
 :xtdb/document-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/tmp/xtdb/documents"}}
 :xtdb/tx-log {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/tmp/xt/tx-log"}}
 :xtdb.http-server/server {}}

To build the JAR, run the build-uberjar.sh script. You can optionally pass the environment variable UBERJAR_NAME to the script (for example, UBERJAR_NAME=xtdb-rocks.jar ./build-uberjar.sh), otherwise the built uberjar will be called xtdb.jar. To run the clojure uberjar, use the following command java -jar xtdb.jar. This will now start up a node with both a HTTP server and persistent storage.

Building a JAR (Maven tooling)

Similarly to building a JAR using the Clojure CLI tooling, we can also build an uberjar using Maven. In the mvn-uberjar directory, add dependencies to the pom.xml file, update the xtdb.edn file as before, and then run build-uberjar.sh to create the uberjar. To run the maven generated uberjar, use the following command: java -jar xtdb.jar

Building a Docker Container

In the docker directory, there are a similar set of files to the uberjar examples above, as well as a Dockerfile and a build-docker.sh script.

As with building a JAR, to add rocksdb as the KV store - start by adding a dependency on xtdb-rocksdb within deps.edn.

Override the topology within xtdb.edn with the following:

{:xtdb/index-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/var/lib/xtdb/indexes"}}
 :xtdb/document-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/var/lib/xtdb/documents"}}
 :xtdb/tx-log {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/var/lib/xt/tx-log"}}
 :xtdb.http-server/server {}}

To build your Docker container, run the build-docker.sh script. You can optionally pass the environment variables IMAGE_NAME and IMAGE_VERSION to tag the container with (by default, the custom Docker container is called xtdb-custom:latest).