link:example$deps.edn[role=include]Liking cljdoc? Tell your friends :D
Monitoring Crux
Crux can display metrics through a variety of interfaces.
Internally, it uses dropwizard’s metrics library to register all the metrics and then passes the registry around to reporters to display the data in a suitable application.
Project Dependency
In order to use any of the crux-metrics reporters, you will need to include the following dependency on crux-metrics:
The various types of metric reporters bring in their own sets of dependencies, so we expect these to be provided by the user in their own project (in order to keep the core of crux-metrics as lightweight as possible).
Reporters requiring further dependencies will have an 'additional dependencies' section.
Getting Started
By default indexer and query metrics are included. It is also
possible to add rocksdb metrics when it is being used. These arguments can be
used whenever any of the topologies to display metrics are included.
| Field | Property | Default | Description |
|---|---|---|---|
|
|
| Includes indexer metrics in the metrics registry |
|
|
| Includes query metrics in the metrics registry |
RocksDB metrics
To include the RocksDB metrics when monitoring the
'crux.kv.rocksdb/kv-store-with-metrics module should be included in the
topology map (in place of 'crux.kv.rocksdb/kv-store):
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.kv.rocksdb/kv-store-with-metrics
...]
...})Reporters
Crux currently supports the following outputs:
-
Console
stdout -
CSV file
-
Prometheus (reporter & http exporter)
Console
This component logs metrics to sysout at regular intervals.
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.metrics.dropwizard.console/reporter]
...
})| Field | Property | Description |
|---|---|---|
|
| Interval in seconds between output dump |
|
| Unit which rates are displayed |
|
| Unit which durations are displayed |
CSV
This component logs metrics to a csv file at regular intervals. Only filename is required.
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.metrics.dropwizard.csv/reporter]
:crux.metrics.dropwizard.csv/file-name "csv-out"
...
})| Field | Property | Required | Description |
|---|---|---|---|
|
|
| Output folder name (must already exist) |
|
|
| Interval in seconds between file write |
|
|
| Unit which rates are displayed |
|
|
| Unit which durations are displayed |
JMX
Provides JMX mbeans output.
Additional Dependencies
You will need to add the following dependencies, alongside crux-metrics, in your project:
link:example$deps.edn[role=include]Getting Started
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.metrics.dropwizard.jmx/reporter]
...
})| Field | Property | Description |
|---|---|---|
|
| Change metrics domain group |
|
| Unit which rates are displayed |
|
| Unit which durations are displayed |
Prometheus
Additional Dependencies
You will need to add the following dependencies, alongside crux-metrics, in your project:
link:example$deps.edn[role=include]HTTP-Exporter
The prometheus http exporter starts a standalone server hosting prometheus metrics by default at http://localhost:8080/metrics. The port can be changed with an argument, and jvm metrics can be included in the dump.
Getting Started
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.metrics.dropwizard.prometheus/http-exporter]
...
})| Field | Property | Description |
|---|---|---|
|
| Desired port number for prometheus client server. Defaults to |
|
| If |
Reporter
This component pushes prometheus metrics to a specified pushgateway at
regular durations (by default 1 second).
Getting Started
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.metrics.dropwizard.prometheus/reporter]
:crux.metric.dropwizard.prometheus/pushgateway "localhost:9090"
...
})| Field | Property | Description |
|---|---|---|
|
| Address of the prometheus server. This field is required |
|
| Time in ISO-8601 standard between metrics push. Defaults to "PT1S". |
|
| Prefix all metric titles with this string |
AWS Cloudwatch metrics
Pushes metrics to Cloudwatch. This is indented to be used with a crux node running inside a EBS/Fargate instance. It attempts to get the relevant credentials through system variables. Crux uses this in its aws benchmarking system which can be found here.
Additional Dependencies
You will need to add the following dependencies, alongside crux-metrics, in your project:
link:example$deps.edn[role=include]Getting Started
(api/start-node {:crux.node/topology ['crux.standalone/topology
'crux.metrics.dropwizard.cloudwatch/reporter]
...
})| Field | Property | Description |
|---|---|---|
|
| Time between metrics push |
|
| When |
|
| Should jvm metrics be included in the pushed metrics? |
|
| Should jvm metrics be included in the pushed metrics? |
|
| Cloudwatch region for uploading metrics. Not required inside a EBS/Fargate instance but needed for local testing. |
|
| A list of strings to ignore specific metrics, in gitignore format. e.g. |
Tips for running
To upload metrics to Cloudwatch locally the desired region needs to be
specified with :crux.metrics.dropwizard.prometheus/region, and your aws
credentials at ~/.aws/credentials need to be visible (If ran in docker, mount
these as a volume).
When ran on aws if using cloudformation the node needs to have the permission
'cloudwatch:PutMetricData'. For a example see Crux’s benchmarking system
here.
Can you improve this documentation?Edit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close