- Command Line
-
On the command line, you can supply a JSON/EDN configuration file using
-f <file> - Java
-
For a Java in-process node, the modules are configured using the supplied Configurator, a file, or a classpath resource:
link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] - Kotlin
-
For a Kotlin in-process node, the modules are configured using the supplied Configurator, a file, or a classpath resource:
link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] - Clojure
-
For a Clojure in-process node, the
start-nodefunction accepts a module tree, a file, or a resource.link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include]
Liking cljdoc? Tell your friends :D
Configuration
A Crux node consists of a number of modules, which can be independently configured and augmented.
Once you have an in-memory Crux node set up, you can then start to configure the various modules - either through a JSON config file, EDN config file, or programmatically:
Without any explicit configuration, Crux will start an in-memory node.
At this point, you can start submitting transactions and running queries!
Modules
Crux has three main pluggable components - the transaction log, the document store, and the query index store. All three are backed by local KV stores by default, but they can be independently configured and overridden - you might choose to host the transaction log in Kafka, the document store in AWS’s S3, and the query indices in RocksDB.
| Transaction Log | Document Store | Index Store | |
|---|---|---|---|
✓ | |||
✓ | |||
✓ | |||
✓ | ✓ | ||
✓ | ✓ | ||
In-memory KV | ✓ | ✓ | ✓ |
LMDB (KV) | ✓ | ✓ | ✓ |
RocksDB (KV) | ✓ | ✓ | ✓ |
✓ | ✓ | ✓ |
For specific details and examples of how to configure each of these modules, see their individual sections.
Each module has both an underlying implementation and overridable parameters - for each module, you can choose to keep the implementation and override its parameters, or you can choose to override the implementation entirely.
To add the HTTP server module, and specify its port:
- Java
-
link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] - Kotlin
-
link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] - JSON
-
link:example$test/crux/docs/examples/configuration/config.json[role=include] - Clojure
-
link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] - EDN
-
link:example$test/crux/docs/examples/configuration/config.edn[role=include]
Overriding the module implementation
To override the underlying implementation, specify the factory function of the new implementation.
For example, using S3’s crux.s3/->document-store factory:
- Java
-
link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] - Kotlin
-
link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] - JSON
-
link:example$test/crux/docs/examples/configuration/config_override.json[role=include] - Clojure
-
link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] - EDN
-
link:example$test/crux/docs/examples/configuration/config_override.edn[role=include]
Nested modules
Modules in Crux form an arbitrarily nested tree - parent modules depend on child modules.
For example, the default implementations of the three main Crux modules are KV store backed implementations - the KV transaction log, the KV document store and the KV index store.
Each of these implementations depends on being given a concrete KV store implementation - by default, an in-memory KV store.
To override the implementation and parameters of this KV store (for example, to replace it with RocksDB), we override its kv-store dependency, replacing the implementation of the nested module:
- Java
-
link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] - Kotlin
-
link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] - JSON
-
link:example$test/crux/docs/examples/configuration/config_nested.json[role=include] - Clojure
-
link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] - EDN
-
link:example$test/crux/docs/examples/configuration/config_nested.edn[role=include]
|
The tx-log and document-store are considered 'golden stores'. The query indices can, should you wish to, be thrown away and rebuilt from these golden stores. Ensure that you either persist both or neither of these golden stores. If not, Crux will work fine until you restart the node, at which point some will evaporate, but others will remain. Crux tends to get rather confused in this situation! Likewise, if you persist the query indices, you’ll need to persist both the golden stores. |
Sharing modules - references
When two modules depend on a similar type of module, by default, they get an instance each. For example, if we were to write the following, the transaction log and the document store would get their own RocksDB instance:
link:example$test/crux/docs/examples/configuration/sharing_modules_naive.json[role=include]We can store both the transaction log and the document store in the same KV store, to save ourselves some hassle. We specify a new top-level module, and then refer to it by name where required:
- Java
-
link:example$test/crux/docs/examples/configuration/ConfigurationTest.java[role=include] - Kotlin
-
link:example$test/crux/docs/examples/configuration/KConfigurationTest.kt[role=include] - JSON
-
link:example$test/crux/docs/examples/configuration/config_shared.json[role=include] - Clojure
-
link:example$test/crux/docs/examples/configuration/configuration_test.clj[role=include] - EDN
-
link:example$test/crux/docs/examples/configuration/config_shared.edn[role=include]
Writing your own module (Clojure)
Crux modules are (currently) vanilla 1-arg Clojure functions with some optional metadata to specify dependencies and arguments.
By convention, these are named ->your-component, to signify that it’s returning an instance of your component.
If the value returned implements AutoCloseable/Closeable, the module will be closed when the Crux node is stopped.
The most basic component would be just a Clojure function, returning the started module:
(defn ->server [opts]
;; start your server
)You can specify arguments using the :crux.system/args metadata key - this example declares a required :port option, checked against the given spec, defaulting to 3000:
(require '[crux.system :as sys])
(defn ->server {::sys/args {:port {:spec ::sys/int
:doc "Port to start the server on"
:required? true
:default 3000}}}
[{:keys [port] :as options}]
;; start your server
)You can specify dependencies using :crux.system/deps - a map of the dependency key to its options.
The options takes the same form as the end-user options - you can specify :crux/module for the default implementation, as well as any parameters.
The started dependencies are passed to you as part of the function’s parameter, with the args.
Bear in mind that any options you do specify can be overridden by end-users!
(defn ->server {::sys/deps {:other-module {:crux/module `->other-module
:param "value"}
...}}
[{:keys [other-module]}]
;; start your server
)You can also use refs - for example, to depend on the Crux node:
(defn ->server {::sys/deps {:crux-node :crux/node}
::sys/args {:spec ::sys/int
:doc "Port to start the server on"
:required? true
:default 3000}}
[{:keys [crux-node] :as options}]
;; start your server
)
1. via third-party
crux-xodus module
Can you improve this documentation? These fine people already did:
James Henderson, Daniel Mason, Alistair, Dan Mason & Henrik MohrEdit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close