Liking cljdoc? Tell your friends :D

cddr.kafka.test

A test-machine executes sequences of test commands

Test machines can be constructed to operate against a variety of targets. For example:

  • a local development kafka cluster
  • a mock topology processor
  • a cluster shared with other users

In each of these cases, as a test-author we typically want to do the same type of thing. Inject a bunch of events into the system, wait until the system under test has finished processing, see what comes out the other end, and check that looks good.

But the mechanism by which data is injected and observed is different in each case. The test-machine exists so that the author doesn't care. The exact same test (or scenario) can be executed against a mock topology processor, a kafka cluster running on localhost, or (via smokin or the rest-proxy) a remote kafka cluster shared with other users.

A test-machine executes sequences of test commands

Test machines can be constructed to operate against a variety of targets. For
example:

  - a local development kafka cluster
  - a mock topology processor
  - a cluster shared with other users

In each of these cases, as a test-author we typically want to do the same type
of thing. Inject a bunch of events into the system, wait until the system under
test has finished processing, see what comes out the other end, and check that
looks good.

But the mechanism by which data is injected and observed is different in each
case. The test-machine exists so that the author doesn't care. The exact same
test (or scenario) can be executed against a mock topology processor, a kafka
cluster running on localhost, or (via smokin or the rest-proxy) a remote
kafka cluster shared with other users.
raw docstring

+default-executor+clj


identity-transportclj

(identity-transport _config _topics)

The identity transport simply injects input events directly into the journal.

Like all transports, this has consumer and producer keys containing processes that pick up test-commands added by run-test. However, this transport simply echo's any write commands to the journal rather than performing the write against another system.

Primarily used internally for test purposes but may also be useful for testing your watch functions. The parameters are ignored and exist only for compatibility with other transports.

The identity transport simply injects input events directly into
the journal.

Like all transports, this has consumer and producer keys containing
processes that pick up test-commands added by `run-test`. However, this
transport simply echo's any write commands to the journal rather than
performing the write against another system.

Primarily used internally for test purposes but may also be useful
for testing your watch functions. The parameters are ignored and
exist only for compatibility with other transports.
raw docstring

kafka-transportclj

(kafka-transport config topics)

The kafka transport injects input events by sending them via a KafkaProducer with direct access to the kafka cluster. Likewise it gathers output by polling a KafkaConsumer subscribed to the listed topics and adding the results to the journal.

The config is shared by the consumer and producer processes and used to construct the underlying KafkaProducer and KafkaConsumer respectively

The topics parameter represents the "topics of interest" for this transport. It should be a {"string" topic-metadata} map which tells the producer how to serialize any write commands and tells the consumer how to deserialize output messages before adding them to the journal.

The kafka transport injects input events by sending them via a
KafkaProducer with direct access to the kafka cluster. Likewise it
gathers output by polling a KafkaConsumer subscribed to the listed
topics and adding the results to the journal.

The `config` is shared by the consumer and producer processes and
used to construct the underlying KafkaProducer and KafkaConsumer
respectively

The `topics` parameter represents the "topics of interest" for
this transport. It should be a {"string" topic-metadata} map
which tells the producer how to serialize any write commands and
tells the consumer how to deserialize output messages before
adding them to the journal.
raw docstring

mock-test-driverclj

(mock-test-driver build-fn app-config)

mock-transportclj

(mock-transport config topics)

The mock transport injects input events by submitting them to a TopologyTestDriver. Likewise it gathers output by polling the test driver's .readOutput method.

The topics parameter has the same semantics as in the kafka-transport

The mock transport injects input events by submitting them to a
TopologyTestDriver. Likewise it gathers output by polling the
test driver's `.readOutput` method.

The `topics` parameter has the same semantics as in the
kafka-transport
raw docstring

rest-proxy-transportclj

(rest-proxy-transport config topics)

The rest-proxy transport injects input events by submitting them to a remote kafka cluster via the confluent REST API. Likewise it gathers output by polling the /records endpoint of the confluent REST API.

If avro topics are involved, the topics must also be configured with access to the confluent schema registry. It should be noted that both the schema registry and the rest-proxy services used in this scenario must be backed by the same kafka cluster (or at least mirror images of one another).

The topics paramter has the same semantics as in the kafka-transport

The rest-proxy transport injects input events by submitting them
to a remote kafka cluster via the confluent REST API. Likewise it
gathers output by polling the /records endpoint of the confluent
REST API.

If avro topics are involved, the topics must also be configured
with access to the confluent schema registry. It should be noted
that both the schema registry and the rest-proxy services used
in this scenario must be backed by the same kafka cluster (or
at least mirror images of one another).

The `topics` paramter has the same semantics as in the kafka-transport
raw docstring

run-testclj

(run-test machine commands)

Runs a sequence of test commands against a test-machine and returns a response map.

The response map includes

:results A sequence of execution results. One for each command attempted

:journal A snapshot of all kafka output read by the test consumer

The first parameter is a test-machine and the second is a list of commands to execute. Remember to use with-open on the test-machine to ensure that all resources are correcly torn down.

Runs a sequence of test commands against a test-machine and returns a
response map.

The response map includes

  `:results` A sequence of execution results. One for each command
             attempted

  `:journal` A snapshot of all kafka output read by the test consumer

The first parameter is a test-machine and the second is a list of
commands to execute. Remember to use `with-open` on the test-machine
to ensure that all resources are correcly torn down.
raw docstring

test-machineclj

(test-machine transport)
(test-machine transport executor)

Returns a test-machine for use in conjunction with run-test.

The test-machine is just a map to which we attach various stateful objects required during a typical test run involving an application which reads/writes to kafka.

The first parameter is a transport which can be obtained by one of the transport returning functions defined below. The transport determines how exactly the test events will be injected into the system under test.

The (optional) second parameter is an executor and if none is specified the +default-executor+ is used.

Returns a test-machine for use in conjunction with `run-test`.

The test-machine is just a map to which we attach various stateful
objects required during a typical test run involving an application
which reads/writes to kafka.

The first parameter is a `transport` which can be obtained by one of
the transport returning functions defined below. The transport
determines how exactly the test events will be injected into the system
under test.

The (optional) second parameter is an executor and if none is specified
the `+default-executor+` is used.
raw docstring

TestMachineclj


with-test-machineclj

(with-test-machine transport f)

Convenience wrapper for the test-machine.

Creates a test-machine using the supplied transport and then passes it to the supplied f. Typical usage would look something like this:

(with-test-machine (test-transport)
  (fn [machine]
    (let [{:keys [results journal]} (run-test machine test-commands)]
      (is (every? ok? results))
      (is (= (expected-topic-output test-commands)
             (actual-topic-output journal))))))
Convenience wrapper for the test-machine.

Creates a test-machine using the supplied `transport` and then
passes it to the supplied `f`. Typical usage would look something
like this:

```
(with-test-machine (test-transport)
  (fn [machine]
    (let [{:keys [results journal]} (run-test machine test-commands)]
      (is (every? ok? results))
      (is (= (expected-topic-output test-commands)
             (actual-topic-output journal))))))
```
raw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close