Command Line XTDB — com.github.zackteo/xtdb-http-server 1.21.1-dev-SNAPSHOT-3

Command Line XTDB

Without writing a single line of code, this guide will show you how to spin up an in-memory XTDB node on your development machine in less than 2 minutes. Then we’ll show you how to talk to it over HTTP from the command line.

This guide will be skipping a large amount of information about XTDB, how it works, and much of its available functionality. For more information about the XTDB HTTP API, please take a look at the XTDB HTTP module documentation. Throughout this guide, you will see links to targeted extra reading and information, should you wish to read more.

Part 1: Getting an XTDB node running on your local machine

This guide will install XTDB using the Clojure CLI tools. You can find in the other methods of installing XTDB in the XTDB installation docs.

If you do not have the Clojure CLI tools installed, installers are available from the Clojure Getting Started Guide. Don’t worry, you don’t need to know Clojure or the Clojure toolchain to run XTDB. Just copy the commands below.

(1) First, create a directory to house the settings for your new XTDB node by running the following in your terminal:

mkdir xtdb-cli && cd xtdb-cli

(2) Next, create two new files within this directory named deps.edn and xtdb.edn with the following configuration code:

;; deps.edn

{:deps {org.clojure/clojure {:mvn/version "1.10.3"}
        com.xtdb/xtdb-core {:mvn/version "{xtdb_version}"}
        com.xtdb/xtdb-http-server {:mvn/version "{xtdb_version}"}}}

;; xtdb.edn

{:xtdb.http-server/server {:port 3000}} (1)

1	This configuration tells XTDB to start an HTTP server on port 3000.

(3) Last, and still from within xtdb-cli, run this command in your terminal:

clojure -M -m xtdb.main (1)

1	You will probably see a `SLF4J: Failed to load class` error. Just ignore it.

Two minutes are up and your XTDB node is running. Nice work! To check that everything is working, head over to http://localhost:3000 where you should see the XTDB console.

Part 2: Talking to XTDB over HTTP

At the moment, the database is empty. Before we can start to play with some of the features that XTDB has to offer, we need to add some data. We will be using the REST API to send and receive JSON via curl.

For this guide, we will be working with facts about the current president of the United States. At the time of writing, the current president is Joe Biden. Lets add that to the database:

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"tx-ops": [
           ["put", { (1)
                "xt/id": "president", (2)
                "name": "Joe", (3)
                "last-name": "Biden" (4)
            }]
         ]}' \
     http://localhost:3000/_xtdb/submit-tx

1	`put` - The type of operation we are performing
2	The ID for the document we are adding
3	The name to be associated with `president`
4	The last name to be associated with the `president`

We see that a transaction has been received by our locally running XTDB node. Now lets retrieve the entity:

curl -X GET \
     -H "Accept: application/json" \
     http://localhost:3000/_xtdb/entity\?eid=president (1)

1	GET the entity by the same ID we used above

We see that the data we just put in the database is successfully showing up at the ID president.

Joe Biden hasn’t always been the president of the United States.

With XTDB, we can add data to the past (or the future) by passing a 'valid time' along with the document. Notice that we are using the same ID as before, something that wouldn’t work in a traditional database.

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"tx-ops": [
           ["put", {
                "xt/id": "president", (1)
                "name": "Donald",
                "last-name": "Trump"
            }, "2017-01-20"] (2)
         ]}' \
     http://localhost:3000/_xtdb/submit-tx

1	The same ID as our first `put` operation
2	The valid time for the document being added

The above put operation places Donald Trump at the ID president from a valid date of 2017-01-20, his inauguration date.

Now lets retrieve the president entity again:

curl -X GET \
     -H "Accept: application/json" \
     http://localhost:3000/_xtdb/entity\?eid=president

We get the same result - Joe Biden!

The GET request of /entity returns the data that is currently valid for the ID president.

This is the first example of the power of a temporal database. We are able to ingest data to provide a better history of data should we want to look back.

To retrieve the Donald Trump document we just added, we can pass a valid time with our GET request:

curl -X GET \
     -H "Accept: application/json" \
     http://localhost:3000/_xtdb/entity\?eid=president\&valid-time=2020-10-10 (1)

1	We pass a valid time of `2020-10-10`

The above operation allows us to find out who the president was on the date passed. XTDB will return Donald Trump for any date between 2017-01-20 and the valid time on the Joe Biden document.

We can see all of our transactions by passing some more arguments to the same /entity endpoint:

curl -X GET \
     -H "Accept: application/json" \
     http://localhost:3000/_xtdb/entity\?eid=president\&history=true\&sortOrder=desc\&withDocs=true (1)

1	`history=true` returns all historical documents for this entity

If you look closely at the Joe Biden document, the validTime is equal to today’s date. This is because we did not explicitly provide a 'valid time' with our initial put. By default, XTDB assumes validTime is equal to the current date-time unless otherwise specified.

Let’s correct the entry for Joe Biden by passing the correct 'valid time':

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"tx-ops": [
           ["put", {
                "xt/id": "president",
                "name": "Joe",
                "last-name": "Biden"
            }, "2021-01-20"] (1)
         ]}' \
     http://localhost:3000/_xtdb/submit-tx

1	Again, `"2021-01-20"` is our explicit 'valid time' parameter.

We now have a historically accurate dataset for the last two presidents of the United States that we can query over a temporal plane! You can run the history query again to validate our correction.

Using XTDB, we can do powerful queries over the temporal plane and retrieve data as if we had travelled back in time.^[1] XTDB achieves this by maintaining an immutable (write-only) transaction log. This makes XTDB a powerful asset where auditing is important or when looking back at historical data is valuable. Better still, XTDB can build on top of many different DB solutions allowing you to retain the infrastructure you already know and love!

In this guide, we used 3 operations: storing documents with submit-tx, retrieving documents with entity, and retrieving document histories with history=true. The complete set of 18 REST operations, including RESTful Datalog queries, are explained in the XTDB REST API documentation.

XTDB also supports a number of other protocols and features:

Now that you’ve dipped your toes in the temporal data waters, we encourage you to experiment with the more advanced features of XTDB!

1. XTDB is actually more sophisticated than this and supports two-dimensional temporal plane: bitemporality. This is a short guide, which is why an explanation wasn’t included here. If you would like to read more, we have an explanation of bitemporality