signal.data-platform-telemetry.metrics — com.signal-ai/data-platform-telemetry 0.4.0

*default-registry* clj

add-exponential-buckets-past-maximum clj

(add-exponential-buckets-past-maximum base-buckets)

(add-exponential-buckets-past-maximum base-buckets max-time-seconds)

(add-exponential-buckets-past-maximum base-buckets max-time-seconds exponent)

Adds exponential buckets with the given exponent (defaults to 2) to the given list of buckets. The last bucket will be greater than the given value.

decrement-messages-in-flight clj

(decrement-messages-in-flight registry)

(decrement-messages-in-flight registry n)

Decrement the number of messages currently being processed by the service.

get-default-histogram-buckets clj

(get-default-histogram-buckets {:keys [max-batch-size] :or {max-batch-size 50}})

Returns the default buckets for histograms for the metrics library.

Buckets can subsequently be overwritten by overwriting a given key.

increment-messages-in-flight clj

(increment-messages-in-flight registry)

(increment-messages-in-flight registry n)

Increment the number of messages currently being processed by the service.

init clj

(init)

(init {:keys [registry ring? jvm?]
       :or {ring? true jvm? true registry *default-registry*}})

Initialises a Prometheus metrics registry and returns it.

message-ack-time clj macro

(message-ack-time registry & body)

Track the total time taken by the macro body to acknowledge (i.e. output) the message result, e.g. writing to S3 and publishing to SNS.

message-process-time clj macro

(message-process-time registry & body)

Track the time taken by the macro body to run the service logic on the message.

message-publish-time clj macro

(message-publish-time registry & body)

Track the time taken by the macro body to publish a new onward message, e.g to SNS.

message-read-time clj macro

(message-read-time registry & body)

Track the time taken by the macro body to read data for the message from storage, e.g S3.

message-result-write-time clj macro

(message-result-write-time registry & body)

Track the time taken by the macro body to write the message result to storage, e.g S3.

message-total-time clj macro

(message-total-time registry & body)

Track the total time taken by the macro body to run the read the message, run logic, and acknoweledge it.

observe-message-ack-time clj

Track the total time taken to acknowledge (i.e. output) the message result, e.g. writing to S3 and publishing to SNS.

observe-message-batch-size clj

(observe-message-batch-size registry size)

Observe the size of a batch being processed, e.g. 50 for an article mini-batch, or 1 for a single document.

observe-message-process-time clj

Track the time taken to run the service logic on the message.

observe-message-publish-time clj

Track the time taken to publish a new onward message, e.g to SNS.

observe-message-read-time clj

Track the time taken to read data for the message from storage, e.g S3.

observe-message-result-write-time clj

Track the time taken to write the message result to storage, e.g S3.

observe-message-total-time clj

Track the total time taken to run the read the message, run logic, and acknoweledge it.

observe-queue-worker-error clj

(observe-queue-worker-error registry _)

Observe an error in processing.

observe-time-seconds clj

(observe-time-seconds metric registry start-timestamp-ms end-timestamp-ms)

Given a metric name and a start/end timestamp in milliseconds, observes the time in seconds between the two times.

pipeline-latency clj

(pipeline-latency registry {:keys [histogram-bucket-overrides] :as config})

Registers metrics to track latency from point-to-point in a pipeline to the given registry.

The configuration histogram-bucket-overrides.pipeline-document-latency must be provided - be sure to include the explicit value of any SLOs in your bucket distribution.

queue-reader clj

(queue-reader registry {:keys [histogram-bucket-overrides] :as config})

Registers queue reader metrics on the given registry.

queue-writer clj

(queue-writer registry {:keys [histogram-bucket-overrides] :as config})

Registers queue worker metrics on the given registry.

record-pipeline-latency clj

(record-pipeline-latency registry
                         {:keys [mode measured-from]}
                         document-received-date-time)

Observe pipeline latency metrics for a document.

These metrics are labelled by

• "mode": the mode of operation of the service if applicable, e.g. realtime, non-realtime
• "measured-from": where the latency was measured from, e.g. "ingestion" or "pipeline_start"

ring-wrap-instrumentation clj

(ring-wrap-instrumentation registry app)

Wraps a ring application in Prometheus instrumentation.

ring-wrap-metrics-expose clj

(ring-wrap-metrics-expose registry app)

Creates a ring handler which exposes Prometheus metrics for the given registry on /internal/metrics.

take-while-and-n-more clj

(take-while-and-n-more pred n coll)