Elasticsearch publisher

In order to use the library add the dependency to your project.clj

;; Leiningen project
[com.brunobonacci/mulog-elasticsearch "x.x.x"]

;; deps.edn format
{:deps { com.brunobonacci/mulog-elasticsearch {:mvn/version "x.x.x"}}}

Current version:

The events must be serializeable in JSON format (see How to JSON encode custom Java classes for more info.)

The available configuration options:

{:type :elasticsearch

 ;; Elasticsearch endpoint (REQUIRED)
 :url  "http://localhost:9200/"


 ;; The Elasticsearch version family.
 ;; one of: `:auto` `:v6.x`  `:v7.x` `:v8.x`
 :els-version   :auto

 ;; the maximum number of events which can be sent in a single
 ;; batch request to Elasticsearch
 :max-items     5000

 ;; Interval in milliseconds between publish requests.
 ;; μ/log will try to send the records to Elasticsearch
 ;; with the interval specified.
 :publish-delay 5000

 ;; Choose an indexing strategy:
 ;; between `:index-pattern` or `:data-stream`, the default is `:index-pattern`

 ;; The index pattern to use for the events
 ;; The pattern uses the Java DateTimeFormatter format:
 ;; see: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/format/DateTimeFormatter.html
 ;; :index-pattern "'mulog-'yyyy.MM.dd"

 ;; data streams are available since Elasticsearch 7.9
 ;; IMPORTANT: the data stream must be created on Elasticsearch before publishing the data
 ;; :data-stream   "mulog-stream"

 ;; extra http options to pass to the HTTP client
 :http-opts {}


 ;; Whether or not to change the attribute names
 ;; to facilitate queries and avoid type clashing
 ;; See more on that in the link below.
 :name-mangling true

 ;; a function to apply to the sequence of events before publishing.
 ;; This transformation function can be used to filter, tranform,
 ;; anonymise events before they are published to a external system.
 ;; by defatult there is no transformation.  (since v0.1.8)
 :transform identity
 }

If you would like to use the Elasticsearch Data Streams please make sure the data streams is created before starting to publish the data. Here some info on How to setup a data stream.

For more information about the index patterns check the DateTimeFormatter documentation.

How to use the publisher:

(μ/start-publisher!
  {:type :elasticsearch
   :url  "http://localhost:9200/"})

Supported versions: 6.7+, 7.x, 8.x

Read more on Elasticsearch name mangling here.