XTDB offers a REST API layer in the xtdb-http-server
module that allows you to send transactions and run queries over HTTP.
Using XTDB in this manner is a valid use-case but it cannot support all of the features and benefits that running the XTDB node inside of your application provides, in particular the ability to efficiently combine custom code with multiple in-process Datalog queries.
Your application only needs to communicate with one XTDB node when using the REST API. Multiple XTDB nodes can placed be behind a HTTP load balancer to spread the writes and reads over a horizontally-scaled cluster transparently to the application. Each XTDB node in such a cluster will be independently catching up with the head of the transaction log, and since different queries might go to different nodes, you have to be slightly conscious of read consistency when designing your application to use XTDB in this way. Fortunately, you can readily achieve read-your-writes consistency with the ability to query consistent point-in-time snapshots using specific temporal coordinates.
The REST API also provides an experimental endpoint for SPARQL 1.1 Protocol queries under /_xtdb/sparql/
, rewriting the query into the XTDB Datalog dialect.
Only a small subset of SPARQL is supported and no other RDF features are available.
com.xtdb/xtdb-http-server {:mvn/version "{xtdb_version}"}
You can start up a Jetty HTTP/HTTP2 server on a node by including xtdb.http-server/server
in your node configuration, optionally passing the server port:
{
"xtdb.http-server/server": {
"port": 3000,
...
}
}
{:xtdb.http-server/server {:port 3000
...}
{:xtdb.http-server/server {:port 3000
...}
port
(int, default 3000)
read-only?
(boolean, default false): start the HTTP server in read-only mode
jwks
(string): JSON Web Token (JWT) key set to authorise requests against - {"keys": […]}
jetty-opts
(map): further options to pass to the Ring Jetty adapter
XTDB also exposes a Ring handler that you can include within your own Ring-compatible server, with parameters as above:
(xtdb.http-server/->xtdb-handler xtdb-node {...})
In addition to calling the HTTP endpoints directly you can also use the remote API client, which implements the same interfaces/protocols as a local XTDB node, where possible.
com.xtdb/xtdb-http-client {:mvn/version "{xtdb_version}"}
To connect to a pre-existing remote node, you need a URL to the node and the above on your classpath.
We can then call xtdb.api/new-api-client
, passing the URL.
If the node was started on localhost:3000
, you can connect to it by doing the following:
link:example$test/xtdb/docs/examples_test.clj[role=include]
All of the REST endpoints return application/edn
, application/json
and application/transit+json
.
Individual endpoints may return additional types - see their docs below, or see the
Swagger documentation here.
All endpoints with query-parameters accept them in both a kebab-case and camel cased format, (ie: if valid-time is taken, validTime will also be taken)
|
/_xtdb/status
Returns the current status information of the node.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/status
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/status
{
"version": "{xtdb_version}",
"revision": null,
"indexVersion": 13,
"consumerState": null,
"kvStore": "xtdb.rocksdb.RocksKv",
"estimateNumKeys": 3,
"size": 1326658
}
{:xtdb.version/version "{xtdb_version}",
:xtdb.version/revision nil, :xtdb.index/index-version 18,
:xtdb.tx-log/consumer-state nil,
:xtdb.kv/kv-store "xtdb.rocksdb.RocksKv",
:xtdb.kv/estimate-num-keys 3,
:xtdb.kv/size 132665
/_xtdb/entity
Returns the document map for a particular entity.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/entity?eid=tommy
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/entity?eid-edn=:tommy
One of the following:
eid-edn
(EDN formatted XTDB ID)
eid-json
(JSON formatted XTDB ID)
eid
(String IDs)
valid-time
(date, defaulting to now)
tx-time
(date, defaulting to latest transaction time)
tx-id
(date, defaulting to latest transaction id)
{
"xt/id": "tommy",
"first-name": "Tommy",
"last-name": "Tutorial"
}
{:xt/id :tommy,
:first-name "Tommy",
:last-name "Tutorial"}
/_xtdb/entity?history=true
Returns the history of a particular entity.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/entity?eid=tommy&history=true&sortOrder=asc
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/entity?eid-edn=:tommy&history=true&sort-order=asc
One of the following:
eid-edn
(EDN formatted XTDB ID)
eid-json
(JSON formatted XTDB ID)
eid
(String IDs)
sort-order
(either asc
or desc
)
with-corrections
(boolean, default false): includes bitemporal corrections in the response, inline, sorted by valid-time then tx-id
with-docs
(boolean, default false): includes the documents in the response sequence, under the doc
key
start-valid-time
, start-tx-time
, start-tx-id
(inclusive, default unbounded): bitemporal co-ordinates to start at
end-valid-time
, end-tx-time
, end-tx-id
(exclusive, default unbounded): bitemporal co-ordinates to stop at
[
{
"txTime": "2020-10-16T14:24:17Z",
"txId": 3,
"validTime": "2020-10-16T14:24:17Z",
"contentHash": "99747f80357c336ee5efd073c878313bf85b07f9"
},
{
"txTime": "2020-10-16T14:29:31Z",
"txId": 4,
"validTime": "2020-10-16T14:29:31Z",
"contentHash": "88d61c8de82eda2a53784bb0438e1a751cd68f96"
},
{
"txTime": "2020-10-16T14:29:35Z",
"txId": 5,
"validTime": "2020-10-16T14:29:35Z",
"contentHash": "99747f80357c336ee5efd073c878313bf85b07f9"
}
]
({:xtdb.api/tx-time #inst "2020-10-16T14:24:17.025-00:00",
:xtdb.api/tx-id 3,
:xtdb.api/valid-time #inst "2020-10-16T14:24:17.025-00:00",
:xtdb.api/content-hash #xtdb/id "99747f80357c336ee5efd073c878313bf85b07f9"}
{:xtdb.api/tx-time #inst "2020-10-16T14:29:31.928-00:00",
:xtdb.api/tx-id 4,
:xtdb.api/valid-time #inst "2020-10-16T14:29:31.928-00:00",
:xtdb.api/content-hash #xtdb/id "88d61c8de82eda2a53784bb0438e1a751cd68f96"}
{:xtdb.api/tx-time #inst "2020-10-16T14:29:35.664-00:00",
:xtdb.api/tx-id 5,
:xtdb.api/valid-time #inst "2020-10-16T14:29:35.664-00:00",
:xtdb.api/content-hash #xtdb/id "99747f80357c336ee5efd073c878313bf85b07f9"})
/_xtdb/entity-tx
Returns the transaction details for an entity - returns a map containing the tx-id and tx-time.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/entity-tx?eid=tommy
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/entity-tx?eid-edn=:tommy
One of the following:
eid-edn
(EDN formatted XTDB ID)
eid-json
(JSON formatted XTDB ID)
eid
(String IDs)
valid-time
(date, defaulting to now)
tx-time
(date, defaulting to latest transaction time)
tx-id
(date, defaulting to latest transaction id)
{
"id": "5aeebab117b892fa42002146e4c62be676bc4621",
"contentHash": "99747f80357c336ee5efd073c878313bf85b07f9",
"validTime": "2020-10-16T14:29:35Z",
"txTime": "2020-10-16T14:29:35Z",
"txId": 5
}
{:xt/id #xtdb/id "5aeebab117b892fa42002146e4c62be676bc4621",
:xtdb.api/content-hash #xtdb/id "99747f80357c336ee5efd073c878313bf85b07f9",
:xtdb.api/valid-time #inst "2020-10-16T14:29:35.664-00:00",
:xtdb.api/tx-time #inst "2020-10-16T14:29:35.664-00:00",
:xtdb.api/tx-id 5}
/_xtdb/query
Takes a datalog query and returns its results. Results are also available in text/csv
and text/tsv
formats (can force negotiation of these by using the /_xtdb/query.csv
and /_xtdb/query.tsv
endpoints respectively).
curl -g \
-X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/query?queryEdn={%3Afind+[e]+%3Awhere+[[e+%3Axt/id+_]]}
curl -g \
-X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/query?query-edn={%3Afind+[e]+%3Awhere+[[e+%3Axt/id+_]]}
query-edn
(URL encoded datalog query)
valid-time
(date, defaulting to now)
tx-time
(date, defaulting to latest transaction time)
tx-id
(date, defaulting to latest transaction id)
in-args-edn
(EDN URL encoded :in binding arguments)
in-args-json
(JSON URL encoded :in binding arguments)
[["tommy"],["james"]]
([:tommy] ["james"])
/_xtdb/query
Takes a datalog query and returns its results. Results are also available in text/csv
and text/tsv
formats (can force negotiation of these by using the /_xtdb/query.csv
and /_xtdb/query.tsv
endpoints respectively).
curl -g \
-X POST \
-H "Accept: application/edn" \
-H "Content-Type: application/edn" \
-d '{:query {:find [e first-name] :where [[e :first-name first-name] [e :last-name "Tutorial"]]}}' \
$XTDB_URL/_xtdb/query
You can also accept application/json from this endpoint, but currently the only supported Content-Type for posting queries is application/edn .
|
query
(EDN encoded datalog query)
in-args
(EDN encoded :in binding arguments)
valid-time
(date, defaulting to now)
tx-time
(date, defaulting to latest transaction time)
tx-id
(date, defaulting to latest transaction id)
[["tommy","Tommy"]]
([:tommy "Tommy"])
/_xtdb/attribute-stats
Returns frequencies of indexed attributes
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/attribute-stats
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/attribute-stats
{
"age": 1,
"firstName": 1,
"lastName": 1,
"xt/id": 3,
"first-name": 2,
"last-name": 2
}
{:age 1,
:firstName 1,
:lastName 1,
:xt/id 3,
:first-name 2,
:last-name 2}
/_xtdb/sync
Wait until the Kafka consumer’s lag is back to 0 (i.e. when it no longer has pending transactions to write). Returns the transaction time of the most recent transaction.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/sync?timeout=500
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/sync?timeout=500
timeout
(integer): specified in milliseconds
{"txTime":"2020-10-16T14:29:35Z"}
{:xtdb.api/tx-time #inst "2020-10-16T14:29:35.664-00:00"}
/_xtdb/await-tx
Waits until the node has indexed a transaction that is at or past the supplied tx-id. Returns the most recent tx indexed by the node.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/await-tx?txId=1
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/await-tx?tx-id=1
tx-id
(integer): tx-id of transaction to wait for
timeout
(integer): specified in milliseconds (defaulting to 10 seconds)
{"txId":5,"txTime":"2020-10-16T14:29:35Z"}
{:xtdb.api/tx-id 5, :xtdb.api/tx-time #inst "2020-10-16T14:29:35.664-00:00"}
/_xtdb/await-tx-time
Blocks until the node has indexed a transaction that is past the supplied tx-time. The returned date is the latest index time when this node has caught up as of this call.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/await-tx-time?tx-time=2020-10-16T14:29:35Z
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/await-tx-time?tx-time=2020-10-16T14:29:35Z
tx-time
(date): tx-time of to wait for
timeout
(integer): specified in milliseconds (defaulting to 10 seconds)
{"txTime":"2020-10-16T14:29:35Z"}
{:xtdb.api/tx-time #inst "2020-10-16T14:29:35.664-00:00"}
/_xtdb/tx-log
Returns a list of all transactions, from oldest to newest transaction time - optionally including documents.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/tx-log
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/tx-log
after-tx-id
(integer, default unbounded): transaction id to start after.
with-ops?
(boolean, defaults to false): should the operations with documents be included?
[
{
"txId": 0,
"txTime": "2020-10-16T09:02:43Z",
"txEvents": [
[
"put",
"83bed47ace572cb94c2f137f58bce73b9b7d0039",
"f441402b3c5d37365203947aabe85cf471498bf0",
"2020-06-20T20:05:50Z"
]
]
},
{
"txId": 1,
"txTime": "2020-10-16T09:28:27Z",
"txEvents": [
[
"put",
"83bed47ace572cb94c2f137f58bce73b9b7d0039",
"f441402b3c5d37365203947aabe85cf471498bf0",
"2020-06-20T20:05:50Z"
]
]
}
]
({:xtdb.api/tx-id 0,
:xtdb.api/tx-time #inst "2020-10-16T09:02:43.429-00:00",
:xtdb.api/tx-events [
[:xtdb.api/put
#xtdb/id "83bed47ace572cb94c2f137f58bce73b9b7d0039"
#xtdb/id "f441402b3c5d37365203947aabe85cf471498bf0"
#inst "2020-06-20T20:05:50.000-00:00"]]}
{:xtdb.api/tx-id 1,
:xtdb.api/tx-time #inst "2020-10-16T09:28:27.785-00:00",
:xtdb.api/tx-events [
[:xtdb.api/put
#xtdb/id "83bed47ace572cb94c2f137f58bce73b9b7d0039"
#xtdb/id "f441402b3c5d37365203947aabe85cf471498bf0"
#inst "2020-06-20T20:05:50.000-00:00"]]})
/_xtdb/submit-tx
Takes a vector of transactions (any combination of put
, delete
, match
, evict
and fn
) and executes them in order. This is the only "write" endpoint.
curl -X POST \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{"tx-ops": [["put", {"xt/id": "ivan", "name": "Ivan", "last-name": "Petrov"}],
["put", {"xt/id": "boris", "name": "Boris", "last-name": "Petrov"}],
["delete", "maria", "2012-05-07T14:57:08.462-00:00"]]}' \
$XTDB_URL/_xtdb/submit-tx
Note: Transaction functions should be submitted as strings containing clojure code, and read in as EDN.
curl -X POST \
-H "Content-Type: application/edn" \
-H "Accept: application/edn" \
-d '{:tx-ops [[:xtdb.api/put {:xt/id :ivan, :name "Ivan" :last-name "Petrov"}],
[:xtdb.api/put {:xt/id :boris, :name "Boris" :last-name "Petrov"}],
[:xtdb.api/delete :maria #inst "2012-05-07T14:57:08.462-00:00"]]}' \
$XTDB_URL/_xtdb/submit-tx
tx-ops
(Content-type formatted list of transaction ops to send)
{"txId":6,"txTime":"2020-10-19T09:21:29Z"}
{:xtdb.api/tx-id 6, :xtdb.api/tx-time #inst "2020-10-19T09:21:29Z"}
/_xtdb/tx-committed
Checks if a submitted tx was successfully committed, returning a map with tx-committed and either true
or false
(or a NodeOutOfSyncException
exception response if the node has not yet indexed the transaction).
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/tx-committed?txId=1
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/tx-committed?tx-id=1
tx-id
(integer): tx-id of transaction to check
{"txCommitted?":true}
{:tx-committed? true}
/_xtdb/latest-completed-tx
Returns the latest transaction to have been indexed by this node.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/latest-completed-tx
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/latest-completed-tx
{"txId":5,"txTime":"2020-10-16T14:29:35Z"}
{:xtdb.api/tx-id 5, :xtdb.api/tx-time #inst "2020-10-16T14:29:35.664-00:00"}
/_xtdb/latest-submitted-tx
Returns the latest transaction to have been submitted to this cluster.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/latest-submitted-tx
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/latest-submitted-tx
{"txId":5}
{:xtdb.api/tx-id 5}
/_xtdb/active-queries
Returns a list of currently running queries.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/active-queries
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/active-queries
[
{
"status": "in-progress",
"queryId": "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
"query": "{:find [e first-name], :where [[e :first-name first-name] [e :last-name \"Tutorial\"]]}",
"startedAt": "2020-10-16T15:48:52Z",
"finishedAt": null,
"error": null
}
]
({:status :in-progress
:query-id "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
:query {:find [e first-name], :where [[e :first-name first-name] [e :last-name "Tutorial"]]},
:started-at #inst "2020-10-16T15:48:52.656-00:00",
:finished-at nil
:error nil})
/_xtdb/recent-queries
Returns a list of recently completed/failed queries.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/recent-queries
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/recent-queries
[
{
"status": "completed",
"queryId": "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
"query": "{:find [e first-name], :where [[e :first-name first-name] [e :last-name \"Tutorial\"]]}",
"startedAt": "2020-10-16T15:48:52Z",
"finishedAt": "2020-10-16T15:48:52Z",
"error": null
}
]
({:status :completed,
:query-id "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
:query {:find [e first-name], :where [[e :first-name first-name] [e :last-name "Tutorial"]]},
:started-at #inst "2020-10-16T15:48:52.656-00:00",
:finished-at #inst "2020-10-16T15:48:52.835-00:00",
:error nil})
/_xtdb/slowest-queries
Returns a list of slowest completed/failed queries ran on the node.
curl -X GET \
-H "Accept: application/json" \
$XTDB_URL/_xtdb/slowest-queries
curl -X GET \
-H "Accept: application/edn" \
$XTDB_URL/_xtdb/slowest-queries
[
{
"status": "completed",
"queryId": "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
"query": "{:find [e first-name], :where [[e :first-name first-name] [e :last-name \"Tutorial\"]]}",
"startedAt": "2020-10-16T15:48:52Z",
"finishedAt": "2020-10-16T15:48:52Z",
"error": null
}
]
({:status :completed,
:query-id "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
:query {:find [e first-name], :where [[e :first-name first-name] [e :last-name "Tutorial"]]},
:started-at #inst "2020-10-16T15:48:52.656-00:00",
:finished-at #inst "2020-10-16T15:48:52.835-00:00",
:error nil})
Can you improve this documentation? These fine people already did:
James Henderson, Daniel Mason, Dan Mason & AlistairEdit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close