jackdaw

Jackdaw is a Clojure library for the Apache Kafka distributed streaming platform. With Jackdaw, you can create and list topics with the AdminClient API. You can produce and consume records with the Producer and Consumer APIs. You can create stream processing applications with the Streams API. Jackdaw also contains functions to serialize and deserialize records as JSON, EDN, and Avro, and functions to write unit tests and integration tests.

Maintenance fork. This is a maintained continuation of fundingcircle/jackdaw (unmaintained since 2024), modernized for Apache Kafka 4.x. It is published under a new coordinate, net.clojars.savya/jackdaw. See the CHANGELOG for the 4.x migration notes.

Stack

Installation

Leiningen / Boot:

[net.clojars.savya/jackdaw "1.3.9"]

deps.edn:

net.clojars.savya/jackdaw {:mvn/version "1.3.9"}

Jackdaw resolves Confluent artifacts from the Confluent Maven repository. Add https://packages.confluent.io/maven/ to your :repositories (Leiningen) or :mvn/repos (deps.edn).

Supported versions

Jackdaw 1.3.6 requires Clojure >= 1.10, JDK 17+, and Apache Kafka 4.x / Confluent Platform 8.x brokers. (The datafy protocol sets the Clojure floor. Clojure 1.10 introduced that protocol.)

Documentation

You can find all the documentation on cljdoc.

Examples

• Pipe
• Word Count
• Simple Ledger
• Roll Dice

Contributing

We welcome any thoughts or patches - open an issue on this fork.

Run the broker-free unit suite:

clojure -M:test

Run the full suite against Kafka, schema registry, and Kafka REST Proxy:

docker compose up -d --wait
clojure -M:test --no-config
docker compose down -v

Run only tests tagged as integration against that stack:

clojure -M:test --no-config --focus-meta :integration

Related projects

To get more information about your topologies, use the Topology Grapher library to generate graphs. See an example using jackdaw for how to add it to your topology.

Releasing

This fork self-publishes to Clojars under net.clojars.savya/jackdaw.

1. Bump the version in build.clj and project.clj, then add a dated entry to CHANGELOG.md.
2. Run clojure -M:test and clojure -T:build jar.
3. Deploy: CLOJARS_USERNAME=<user> CLOJARS_PASSWORD=<clojars-deploy-token> clojure -T:build deploy.
4. Tag the release: git tag v<version> && git push --tags.

License

Copyright © 2017 Funding Circle

Maintenance fork (2026) by Savyasachi; original: https://github.com/fundingcircle/jackdaw

Distributed under the BSD 3-Clause License.