Liking cljdoc? Tell your friends :D

HTTP

Crux offers a REST API layer in the crux-http-server module that allows you to send transactions and run queries over HTTP.

Remote Cluster Mode

Using Crux in this manner is a valid use-case but it cannot support all of the features and benefits that running the Crux node inside of your application provides, in particular the ability to efficiently combine custom code with multiple in-process Datalog queries.

Your application only needs to communicate with one Crux node when using the REST API. Multiple Crux nodes can placed be behind a HTTP load balancer to spread the writes and reads over a horizontally-scaled cluster transparently to the application. Each Crux node in such a cluster will be independently catching up with the head of the transaction log, and since different queries might go to different nodes, you have to be slightly conscious of read consistency when designing your application to use Crux in this way. Fortunately, you can readily achieve read-your-writes consistency with the ability to query consistent point-in-time snapshots using specific temporal coordinates.

The REST API also provides an experimental endpoint for SPARQL 1.1 Protocol queries under /_crux/sparql/, rewriting the query into the Crux Datalog dialect. Only a small subset of SPARQL is supported and no other RDF features are available.

Starting an HTTP Server

Project Dependency

juxt/crux-http-server {:mvn/version "{crux_version}-alpha"}

You can start up a HTTP server on a node by including crux.http-server/server in your node configuration, optionally passing the server port:

JSON
{
  "crux.http-server/server": {
    "port": 3000,
    ...
  }
}
Clojure/EDN
{:crux.http-server/server {:port 3000
                           ...}

Parameters

  • port (int, default 3000)

  • read-only? (boolean, default false): start the HTTP server in read-only mode

  • jwks (string): JSON Web Token (JWT) key set to authorise requests against - {"keys": […​]}

Ring Handler

Crux also exposes a Ring handler that you can include within your own Ring-compatible server, with parameters as above:

(crux.http-server/->crux-handler crux-node {...})

Using a Remote API Client

In addition to calling the HTTP endpoints directly you can also use the remote API client, which implements the same interfaces/protocols as a local Crux node, where possible.

Project Dependency

juxt/crux-http-client {:mvn/version "{crux_version}-beta"}

To connect to a pre-existing remote node, you need a URL to the node and the above on your classpath. We can then call crux.api/new-api-client, passing the URL. If the node was started on localhost:3000, you can connect to it by doing the following:

link:example$src/docs/examples.clj[role=include]

Using the REST API

All of the REST endpoints return application/edn, application/json and application/transit+json. Individual endpoints may return additional types - see their docs below, or see the Swagger documentation here.

All endpoints with query-parameters accept them in both a kebab-case and camel cased format, (ie: if valid-time is taken, validTime will also be taken)

GET /_crux/status

Returns the current status information of the node.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/status
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/status

Response

JSON
{
    "version": "20.09-1.11.1-beta-SNAPSHOT",
    "revision": null,
    "indexVersion": 13,
    "consumerState": null,
    "kvStore": "crux.rocksdb.RocksKv",
    "estimateNumKeys": 3,
    "size": 1326658
}
EDN
{:crux.version/version "20.09-1.11.1-beta-SNAPSHOT",
 :crux.version/revision nil, :crux.index/index-version 13,
 :crux.doc-log/consumer-state nil,
 :crux.tx-log/consumer-state nil,
 :crux.kv/kv-store "crux.rocksdb.RocksKv",
 :crux.kv/estimate-num-keys 3,
 :crux.kv/size 132665

GET /_crux/entity

Returns the document map for a particular entity.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/entity?eid=tommy
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/entity?eid-edn=:tommy

Query Parameters

Required Parameters
  • One of the following:

    • eid-edn (EDN formatted Crux ID)

    • eid-json (JSON formatted Crux ID)

    • eid (String IDs)

Optional Parameters
  • valid-time (date, defaulting to now)

  • tx-time (date, defaulting to latest transaction time)

  • tx-id (date, defaulting to latest transaction id)

Response

JSON
{
    "crux.db/id": "tommy",
    "first-name": "Tommy",
    "last-name": "Tutorial"
}
EDN
{:crux.db/id :tommy,
 :first-name "Tommy",
 :last-name "Tutorial"}

GET /_crux/entity?history=true

Returns the history of a particular entity.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/entity?eid=tommy&history=true&sortOrder=asc
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/entity?eid-edn=:tommy&history=true&sort-order=asc

Query Parameters

Required Parameters
  • One of the following:

    • eid-edn (EDN formatted Crux ID)

    • eid-json (JSON formatted Crux ID)

    • eid (String IDs)

  • sort-order (either asc or desc)

Optional Parameters
  • with-corrections (boolean, default false): includes bitemporal corrections in the response, inline, sorted by valid-time then tx-id

  • with-docs (boolean, default false): includes the documents in the response sequence, under the doc key

  • start-valid-time, start-tx-time, start-tx-id (inclusive, default unbounded): bitemporal co-ordinates to start at

  • end-valid-time, end-tx-time, end-tx-id (exclusive, default unbounded): bitemporal co-ordinates to stop at

Response

JSON
[
    {
        "txTime": "2020-10-16T14:24:17Z",
        "txId": 3,
        "validTime": "2020-10-16T14:24:17Z",
        "contentHash": "99747f80357c336ee5efd073c878313bf85b07f9"
    },
    {
        "txTime": "2020-10-16T14:29:31Z",
        "txId": 4,
        "validTime": "2020-10-16T14:29:31Z",
        "contentHash": "88d61c8de82eda2a53784bb0438e1a751cd68f96"
    },
    {
        "txTime": "2020-10-16T14:29:35Z",
        "txId": 5,
        "validTime": "2020-10-16T14:29:35Z",
        "contentHash": "99747f80357c336ee5efd073c878313bf85b07f9"
    }
]
EDN
({:crux.tx/tx-time #inst "2020-10-16T14:24:17.025-00:00",
  :crux.tx/tx-id 3,
  :crux.db/valid-time #inst "2020-10-16T14:24:17.025-00:00",
  :crux.db/content-hash #crux/id "99747f80357c336ee5efd073c878313bf85b07f9"}
 {:crux.tx/tx-time #inst "2020-10-16T14:29:31.928-00:00",
  :crux.tx/tx-id 4,
  :crux.db/valid-time #inst "2020-10-16T14:29:31.928-00:00",
  :crux.db/content-hash #crux/id "88d61c8de82eda2a53784bb0438e1a751cd68f96"}
 {:crux.tx/tx-time #inst "2020-10-16T14:29:35.664-00:00",
  :crux.tx/tx-id 5,
  :crux.db/valid-time #inst "2020-10-16T14:29:35.664-00:00",
  :crux.db/content-hash #crux/id "99747f80357c336ee5efd073c878313bf85b07f9"})

GET /_crux/entity-tx

Returns the transaction details for an entity - returns a map containing the tx-id and tx-time.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/entity-tx?eid=tommy
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/entity-tx?eid-edn=:tommy

Query Parameters

Required Parameters
  • One of the following:

    • eid-edn (EDN formatted Crux ID)

    • eid-json (JSON formatted Crux ID)

    • eid (String IDs)

Optional Parameters
  • valid-time (date, defaulting to now)

  • tx-time (date, defaulting to latest transaction time)

  • tx-id (date, defaulting to latest transaction id)

Response

JSON
{
    "id": "5aeebab117b892fa42002146e4c62be676bc4621",
    "contentHash": "99747f80357c336ee5efd073c878313bf85b07f9",
    "validTime": "2020-10-16T14:29:35Z",
    "txTime": "2020-10-16T14:29:35Z",
    "txId": 5
}
EDN
{:crux.db/id #crux/id "5aeebab117b892fa42002146e4c62be676bc4621",
 :crux.db/content-hash #crux/id "99747f80357c336ee5efd073c878313bf85b07f9",
 :crux.db/valid-time #inst "2020-10-16T14:29:35.664-00:00",
 :crux.tx/tx-time #inst "2020-10-16T14:29:35.664-00:00",
 :crux.tx/tx-id 5}

GET /_crux/query

Takes a datalog query and returns its results. Results are also available in text/csv and text/tsv formats (can force negotiation of these by using the /_crux/query.csv and /_crux/query.tsv endpoints respectively).

Request

JSON
curl -g \
     -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/query?queryEdn={%3Afind+[e]+%3Awhere+[[e+%3Acrux.db/id+_]]}
EDN
curl -g \
     -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/query?query-edn={%3Afind+[e]+%3Awhere+[[e+%3Acrux.db/id+_]]}

Query Parameters

Required Parameters
  • query-edn (URL encoded datalog query)

Optional Parameters
  • valid-time (date, defaulting to now)

  • tx-time (date, defaulting to latest transaction time)

  • tx-id (date, defaulting to latest transaction id)

Response

JSON
[["tommy"],["james"]]
EDN
([:tommy] ["james"])

POST /_crux/query

Takes a datalog query and returns its results. Results are also available in text/csv and text/tsv formats (can force negotiation of these by using the /_crux/query.csv and /_crux/query.tsv endpoints respectively).

Request

EDN
curl -g \
     -X POST \
     -H "Accept: application/edn" \
     -H "Content-Type: application/edn" \
     -d '{:query {:find [e first-name] :where [[e :first-name first-name] [e :last-name "Tutorial"]]}}' \
     $CRUX_URL/_crux/query
You can also accept application/json from this endpoint, but currently the only supported Content-Type for posting queries is application/edn.

Parameters

Body Parameters
Required Parameters
  • query (EDN encoded datalog query)

Query Parameters
Optional Parameters
  • valid-time (date, defaulting to now)

  • tx-time (date, defaulting to latest transaction time)

  • tx-id (date, defaulting to latest transaction id)

Response

JSON
[["tommy","Tommy"]]
EDN
([:tommy "Tommy"])

GET /_crux/attribute-stats

Returns frequencies of indexed attributes

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/attribute-stats
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/attribute-stats

Response

JSON
{
    "age": 1,
    "firstName": 1,
    "lastName": 1,
    "crux.db/id": 3,
    "first-name": 2,
    "last-name": 2
}
EDN
{:age 1,
 :firstName 1,
 :lastName 1,
 :crux.db/id 3,
 :first-name 2,
 :last-name 2}

GET /_crux/sync

Wait until the Kafka consumer’s lag is back to 0 (i.e. when it no longer has pending transactions to write). Returns the transaction time of the most recent transaction.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/sync?timeout=500
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/sync?timeout=500

Query Parameters

Optional Parameters
  • timeout (integer): specified in milliseconds

Response

JSON
{"txTime":"2020-10-16T14:29:35Z"}
EDN
{:crux.tx/tx-time #inst "2020-10-16T14:29:35.664-00:00"}

GET /_crux/await-tx

Waits until the node has indexed a transaction that is at or past the supplied tx-id. Returns the most recent tx indexed by the node.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/await-tx?txId=1
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/await-tx?tx-id=1

Query Parameters

Required Parameters
  • tx-id (integer): tx-id of transaction to wait for

Optional Parameters
  • timeout (integer): specified in milliseconds (defaulting to 10 seconds)

Response

JSON
{"txId":5,"txTime":"2020-10-16T14:29:35Z"}
EDN
{:crux.tx/tx-id 5, :crux.tx/tx-time #inst "2020-10-16T14:29:35.664-00:00"}

GET /_crux/await-tx-time

Blocks until the node has indexed a transaction that is past the supplied tx-time. The returned date is the latest index time when this node has caught up as of this call.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/await-tx-time?tx-time=2020-10-16T14:29:35Z
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/await-tx-time?tx-time=2020-10-16T14:29:35Z

Query Parameters

Required Parameters
  • tx-time (date): tx-time of to wait for

Optional Parameters
  • timeout (integer): specified in milliseconds (defaulting to 10 seconds)

Response

JSON
{"txTime":"2020-10-16T14:29:35Z"}
EDN
{:crux.tx/tx-time #inst "2020-10-16T14:29:35.664-00:00"}

GET /_crux/tx-log

Returns a list of all transactions, from oldest to newest transaction time - optionally including documents.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
      $CRUX_URL/_crux/tx-log
EDN
curl -X GET \
     -H "Accept: application/edn" \
      $CRUX_URL/_crux/tx-log

Query Parameters

Optional Parameters
  • after-tx-id (integer, default unbounded): transaction id to start after.

  • with-ops? (boolean, defaults to false): should the operations with documents be included?

Response

JSON
[
    {
        "txId": 0,
        "txTime": "2020-10-16T09:02:43Z",
        "txEvents": [
            [
                "put",
                "83bed47ace572cb94c2f137f58bce73b9b7d0039",
                "f441402b3c5d37365203947aabe85cf471498bf0",
                "2020-06-20T20:05:50Z"
            ]
        ]
    },
    {
        "txId": 1,
        "txTime": "2020-10-16T09:28:27Z",
        "txEvents": [
            [
                "put",
                "83bed47ace572cb94c2f137f58bce73b9b7d0039",
                "f441402b3c5d37365203947aabe85cf471498bf0",
                "2020-06-20T20:05:50Z"
            ]
        ]
    }
]
EDN
({:crux.tx/tx-id 0,
  :crux.tx/tx-time #inst "2020-10-16T09:02:43.429-00:00",
  :crux.tx.event/tx-events [[:crux.tx/put
  			     #crux/id "83bed47ace572cb94c2f137f58bce73b9b7d0039"
			     #crux/id "f441402b3c5d37365203947aabe85cf471498bf0"
			     #inst "2020-06-20T20:05:50.000-00:00"]]}
 {:crux.tx/tx-id 1,
  :crux.tx/tx-time #inst "2020-10-16T09:28:27.785-00:00",
  :crux.tx.event/tx-events [[:crux.tx/put
  			     #crux/id "83bed47ace572cb94c2f137f58bce73b9b7d0039"
			     #crux/id "f441402b3c5d37365203947aabe85cf471498bf0"
			     #inst "2020-06-20T20:05:50.000-00:00"]]})

POST /_crux/submit-tx

Takes a vector of transactions (any combination of put, delete, match, evict and fn) and executes them in order. This is the only "write" endpoint.

Request

JSON
curl -X POST \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"tx-ops": [["put", {"crux.db/id": "ivan", "name": "Ivan", "last-name": "Petrov"}],
          	    ["put", {"crux.db/id": "boris", "name": "Boris", "last-name": "Petrov"}],
          	    ["delete", "maria", "2012-05-07T14:57:08.462-00:00"]]}' \
     $CRUX_URL/_crux/submit-tx

Note: Transaction functions should be submitted as strings containing clojure code, and read in as EDN.

EDN
curl -X POST \
     -H "Content-Type: application/edn" \
     -H "Accept: application/edn" \
     -d '{:tx-ops [[:crux.tx/put {:crux.db/id :ivan, :name "Ivan" :last-name "Petrov"}],
          	  [:crux.tx/put {:crux.db/id :boris, :name "Boris" :last-name "Petrov"}],
          	  [:crux.tx/delete :maria  #inst "2012-05-07T14:57:08.462-00:00"]]}' \
     $CRUX_URL/_crux/submit-tx

Parameters

Body Parameters
Required Parameters
  • tx-ops (Content-type formatted list of transaction ops to send)

Response

JSON
{"txId":6,"txTime":"2020-10-19T09:21:29Z"}
EDN
{:crux.tx/tx-id 6, :crux.tx/tx-time #inst "2020-10-19T09:21:29Z"}

GET /_crux/tx-committed

Checks if a submitted tx was successfully committed, returning a map with tx-committed and either true or false (or a NodeOutOfSyncException exception response if the node has not yet indexed the transaction).

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/tx-committed?txId=1
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/tx-committed?tx-id=1

Query Parameters

Required Parameters
  • tx-id (integer): tx-id of transaction to check

Response

JSON
{"txCommitted?":true}
EDN
{:tx-committed? true}

GET /_crux/latest-completed-tx

Returns the latest transaction to have been indexed by this node.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/latest-completed-tx
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/latest-completed-tx

Response

JSON
{"txId":5,"txTime":"2020-10-16T14:29:35Z"}
EDN
{:crux.tx/tx-id 5, :crux.tx/tx-time #inst "2020-10-16T14:29:35.664-00:00"}

GET /_crux/latest-submitted-tx

Returns the latest transaction to have been submitted to this cluster.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/latest-submitted-tx
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/latest-submitted-tx

Response

JSON
{"txId":5}
EDN
{:crux.tx/tx-id 5}

GET /_crux/active-queries

Returns a list of currently running queries.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/active-queries
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/active-queries

Response

JSON
[
    {
        "status": "in-progress",
        "queryId": "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
        "query": "{:find [e first-name], :where [[e :first-name first-name] [e :last-name \"Tutorial\"]]}",
        "startedAt": "2020-10-16T15:48:52Z",
        "finishedAt": null,
        "error": null
    }
]
EDN
({:status :in-progress
  :query-id "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
  :query {:find [e first-name], :where [[e :first-name first-name] [e :last-name "Tutorial"]]},
  :started-at #inst "2020-10-16T15:48:52.656-00:00",
  :finished-at nil
  :error nil})

GET /_crux/recent-queries

Returns a list of recently completed/failed queries.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/recent-queries
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/recent-queries

Response

JSON
[
    {
        "status": "completed",
        "queryId": "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
        "query": "{:find [e first-name], :where [[e :first-name first-name] [e :last-name \"Tutorial\"]]}",
        "startedAt": "2020-10-16T15:48:52Z",
        "finishedAt": "2020-10-16T15:48:52Z",
        "error": null
    }
]
EDN
({:status :completed,
  :query-id "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
  :query {:find [e first-name], :where [[e :first-name first-name] [e :last-name "Tutorial"]]},
  :started-at #inst "2020-10-16T15:48:52.656-00:00",
  :finished-at #inst "2020-10-16T15:48:52.835-00:00",
  :error nil})

GET /_crux/slowest-queries

Returns a list of slowest completed/failed queries ran on the node.

Request

JSON
curl -X GET \
     -H "Accept: application/json" \
     $CRUX_URL/_crux/slowest-queries
EDN
curl -X GET \
     -H "Accept: application/edn" \
     $CRUX_URL/_crux/slowest-queries

Response

JSON
[
    {
        "status": "completed",
        "queryId": "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
        "query": "{:find [e first-name], :where [[e :first-name first-name] [e :last-name \"Tutorial\"]]}",
        "startedAt": "2020-10-16T15:48:52Z",
        "finishedAt": "2020-10-16T15:48:52Z",
        "error": null
    }
]
EDN
({:status :completed,
  :query-id "ae17c599-dcd2-47ee-bebd-47a3122f8d34",
  :query {:find [e first-name], :where [[e :first-name first-name] [e :last-name "Tutorial"]]},
  :started-at #inst "2020-10-16T15:48:52.656-00:00",
  :finished-at #inst "2020-10-16T15:48:52.835-00:00",
  :error nil})

Can you improve this documentation? These fine people already did:
Daniel Mason, James Henderson & Dan Mason
Edit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close