Building — com.github.zackteo/xtdb-http-server 1.21.1-dev-SNAPSHOT-3

Building

build is a module within the XTDB repo with two key objectives:

Providing pre-configured JARs and Docker containers to start a basic implementation of XTDB with no prior knowledge required.
Providing a mechanism to spin up custom XTDB artifacts - with two necessary inputs:
- A deps.edn file containing the dependencies that the node requires.
- A xtdb.edn file, containing the configuration options to start the node with.

Preconfigured Artifacts

`xtdb-in-memory`

The xtdb-in-memory artifact starts up a basic, in-memory XTDB node, with both a HTTP Server (open on port 3000) and the XTDB SQL module with a SQL server (open on port 1501).

It can be downloaded:

as a Docker image from JUXT’s Docker Hub
as an uberjar, xtdb-in-memory.jar, from the relevant GitHub release

Communication with the node is done via the XTDB REST API. See the HTTP module documentation for more information.

Building Artifacts

Alongside the JARs deployed on the GitHub releases is xtdb-builder.tar.gz - the scripts within this archive can be used to build a custom XTDB JAR or Docker container.

Building a JAR (Clojure CLI tooling)

The clj-uberjar folder contains a number of files:

a deps.edn file to configure the Maven dependencies required
a xtdb.edn file to configure the node itself
a resources/logback.xml to configure logging output
a build-uberjar.sh script to build the JAR.

To use rocksdb as the index store, document store and transaction-log store of the node:

Add xtdb-rocksdb as a dependency in deps.edn:

...
com.xtdb/xtdb-rocksdb {:mvn/version "{xtdb_version}"}
...

In xtdb.edn, override the old topology with the following:

{:xtdb/index-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/tmp/xtdb/indexes"}}
 :xtdb/document-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/tmp/xtdb/documents"}}
 :xtdb/tx-log {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/tmp/xt/tx-log"}}
 :xtdb.http-server/server {}}

To build the JAR, run the build-uberjar.sh script. You can optionally pass the environment variable UBERJAR_NAME to the script (for example, UBERJAR_NAME=xtdb-rocks.jar ./build-uberjar.sh), otherwise the built uberjar will be called xtdb.jar. To run the clojure uberjar, use the following command java -jar xtdb.jar. This will now start up a node with both a HTTP server and persistent storage.

Building a JAR (Maven tooling)

Similarly to building a JAR using the Clojure CLI tooling, we can also build an uberjar using Maven. In the mvn-uberjar directory, add dependencies to the pom.xml file, update the xtdb.edn file as before, and then run build-uberjar.sh to create the uberjar. To run the maven generated uberjar, use the following command: java -jar xtdb.jar

Building a Docker Container

In the docker directory, there are a similar set of files to the uberjar examples above, as well as a Dockerfile and a build-docker.sh script.

As with building a JAR, to add rocksdb as the KV store - start by adding a dependency on xtdb-rocksdb within deps.edn.

Override the topology within xtdb.edn with the following:

{:xtdb/index-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/var/lib/xtdb/indexes"}}
 :xtdb/document-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/var/lib/xtdb/documents"}}
 :xtdb/tx-log {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store, :db-dir "/var/lib/xt/tx-log"}}
 :xtdb.http-server/server {}}

To build your Docker container, run the build-docker.sh script. You can optionally pass the environment variables IMAGE_NAME and IMAGE_VERSION to tag the container with (by default, the custom Docker container is called xtdb-custom:latest).