cddr.kafka.test
A test-machine executes sequences of test commands
Test machines can be constructed to operate against a variety of targets. For example:
- a local development kafka cluster
- a mock topology processor
- a cluster shared with other users
In each of these cases, as a test-author we typically want to do the same type of thing. Inject a bunch of events into the system, wait until the system under test has finished processing, see what comes out the other end, and check that looks good.
But the mechanism by which data is injected and observed is different in each case. The test-machine exists so that the author doesn't care. The exact same test (or scenario) can be executed against a mock topology processor, a kafka cluster running on localhost, or (via smokin or the rest-proxy) a remote kafka cluster shared with other users.
A test-machine executes sequences of test commands Test machines can be constructed to operate against a variety of targets. For example: - a local development kafka cluster - a mock topology processor - a cluster shared with other users In each of these cases, as a test-author we typically want to do the same type of thing. Inject a bunch of events into the system, wait until the system under test has finished processing, see what comes out the other end, and check that looks good. But the mechanism by which data is injected and observed is different in each case. The test-machine exists so that the author doesn't care. The exact same test (or scenario) can be executed against a mock topology processor, a kafka cluster running on localhost, or (via smokin or the rest-proxy) a remote kafka cluster shared with other users.
+default-executor+clj
identity-transportclj
(identity-transport _config _topics)The identity transport simply injects input events directly into the journal.
Like all transports, this has consumer and producer keys containing
processes that pick up test-commands added by run-test. However, this
transport simply echo's any write commands to the journal rather than
performing the write against another system.
Primarily used internally for test purposes but may also be useful for testing your watch functions. The parameters are ignored and exist only for compatibility with other transports.
The identity transport simply injects input events directly into the journal. Like all transports, this has consumer and producer keys containing processes that pick up test-commands added by `run-test`. However, this transport simply echo's any write commands to the journal rather than performing the write against another system. Primarily used internally for test purposes but may also be useful for testing your watch functions. The parameters are ignored and exist only for compatibility with other transports.
kafka-transportclj
(kafka-transport config topics)The kafka transport injects input events by sending them via a KafkaProducer with direct access to the kafka cluster. Likewise it gathers output by polling a KafkaConsumer subscribed to the listed topics and adding the results to the journal.
The config is shared by the consumer and producer processes and
used to construct the underlying KafkaProducer and KafkaConsumer
respectively
The topics parameter represents the "topics of interest" for
this transport. It should be a {"string" topic-metadata} map
which tells the producer how to serialize any write commands and
tells the consumer how to deserialize output messages before
adding them to the journal.
The kafka transport injects input events by sending them via a
KafkaProducer with direct access to the kafka cluster. Likewise it
gathers output by polling a KafkaConsumer subscribed to the listed
topics and adding the results to the journal.
The `config` is shared by the consumer and producer processes and
used to construct the underlying KafkaProducer and KafkaConsumer
respectively
The `topics` parameter represents the "topics of interest" for
this transport. It should be a {"string" topic-metadata} map
which tells the producer how to serialize any write commands and
tells the consumer how to deserialize output messages before
adding them to the journal.mock-test-driverclj
(mock-test-driver build-fn app-config)mock-transportclj
(mock-transport config topics)The mock transport injects input events by submitting them to a
TopologyTestDriver. Likewise it gathers output by polling the
test driver's .readOutput method.
The topics parameter has the same semantics as in the
kafka-transport
The mock transport injects input events by submitting them to a TopologyTestDriver. Likewise it gathers output by polling the test driver's `.readOutput` method. The `topics` parameter has the same semantics as in the kafka-transport
rest-proxy-transportclj
(rest-proxy-transport config topics)The rest-proxy transport injects input events by submitting them to a remote kafka cluster via the confluent REST API. Likewise it gathers output by polling the /records endpoint of the confluent REST API.
If avro topics are involved, the topics must also be configured with access to the confluent schema registry. It should be noted that both the schema registry and the rest-proxy services used in this scenario must be backed by the same kafka cluster (or at least mirror images of one another).
The topics paramter has the same semantics as in the kafka-transport
The rest-proxy transport injects input events by submitting them to a remote kafka cluster via the confluent REST API. Likewise it gathers output by polling the /records endpoint of the confluent REST API. If avro topics are involved, the topics must also be configured with access to the confluent schema registry. It should be noted that both the schema registry and the rest-proxy services used in this scenario must be backed by the same kafka cluster (or at least mirror images of one another). The `topics` paramter has the same semantics as in the kafka-transport
run-testclj
(run-test machine commands)Runs a sequence of test commands against a test-machine and returns a response map.
The response map includes
:results A sequence of execution results. One for each command
attempted
:journal A snapshot of all kafka output read by the test consumer
The first parameter is a test-machine and the second is a list of
commands to execute. Remember to use with-open on the test-machine
to ensure that all resources are correcly torn down.
Runs a sequence of test commands against a test-machine and returns a
response map.
The response map includes
`:results` A sequence of execution results. One for each command
attempted
`:journal` A snapshot of all kafka output read by the test consumer
The first parameter is a test-machine and the second is a list of
commands to execute. Remember to use `with-open` on the test-machine
to ensure that all resources are correcly torn down.test-machineclj
(test-machine transport)(test-machine transport executor)Returns a test-machine for use in conjunction with run-test.
The test-machine is just a map to which we attach various stateful objects required during a typical test run involving an application which reads/writes to kafka.
The first parameter is a transport which can be obtained by one of
the transport returning functions defined below. The transport
determines how exactly the test events will be injected into the system
under test.
The (optional) second parameter is an executor and if none is specified
the +default-executor+ is used.
Returns a test-machine for use in conjunction with `run-test`. The test-machine is just a map to which we attach various stateful objects required during a typical test run involving an application which reads/writes to kafka. The first parameter is a `transport` which can be obtained by one of the transport returning functions defined below. The transport determines how exactly the test events will be injected into the system under test. The (optional) second parameter is an executor and if none is specified the `+default-executor+` is used.
TestMachineclj
with-test-machineclj
(with-test-machine transport f)Convenience wrapper for the test-machine.
Creates a test-machine using the supplied transport and then
passes it to the supplied f. Typical usage would look something
like this:
(with-test-machine (test-transport)
(fn [machine]
(let [{:keys [results journal]} (run-test machine test-commands)]
(is (every? ok? results))
(is (= (expected-topic-output test-commands)
(actual-topic-output journal))))))
Convenience wrapper for the test-machine.
Creates a test-machine using the supplied `transport` and then
passes it to the supplied `f`. Typical usage would look something
like this:
```
(with-test-machine (test-transport)
(fn [machine]
(let [{:keys [results journal]} (run-test machine test-commands)]
(is (every? ok? results))
(is (= (expected-topic-output test-commands)
(actual-topic-output journal))))))
```cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close