Liking cljdoc? Tell your friends :D

Kafka Connect

Introduction

A Kafka Connect plugin for transferring data between XTDB nodes and Kafka.

The XTDB source connector will publish transacations on a node to a Kafka topic, and the sink connector can receive transactions from a Kafka topic and submit them to a node.

Table 1. Currently supported data formats
Data formatSink/Source

JSON

Both

Avro

Sink

Transit

Source

EDN

Both

To get started with the connector, there are two separate guides (depending on whether you are using a full Confluent Platform installation, or a basic Kafka installation):

Source Configuration

url

  • Destination URL of XTDB HTTP end point

  • Type: String

  • Importance: High

  • Default: "http://localhost:3000"

topic

  • The Kafka topic to publish data to

  • Type: String

  • Importance: High

  • Default: "connect-test"

format

  • Format to send data out as: edn, json or transit

  • Type: String

  • Importance: Low

  • Default: "edn"

mode

  • Mode to use: tx or doc

  • Type: String

  • Importance: Low

  • Default: "tx"

batch.size

  • The maximum number of records the Source task can read from XTDB at one time.

  • Type: Int

  • Importance: LOW

  • Default: 2000

Sink Configuration

url

  • Destination URL of XTDB HTTP end point

  • Type: String

  • Importance: High

  • Default: "http://localhost:3000"

id.key

  • Record key to use as :xt/id

  • Type: String

  • Importance: Low

  • Default: "xt/id"

❮Kafka QuickstartKafka Embedded❯

Can you improve this documentation?Edit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

Keyboard shortcutsReport a problemcljdoc on GitHub
× close