Liking cljdoc? Tell your friends :D

jepsen.core

Entry point for all Jepsen tests. Coordinates the setup of servers, running tests, creating and resolving failures, and interpreting results.

Jepsen tests a system by running a set of singlethreaded processes, each representing a single client in the system, and a special nemesis process, which induces failures across the cluster. Processes choose operations to perform based on a generator. Each process uses a client to apply the operation to the distributed system, and records the invocation and completion of that operation in the history for the test. When the test is complete, a checker analyzes the history to see if it made sense.

Jepsen automates the setup and teardown of the environment and distributed system by using an OS and client respectively. See run! for details.

Entry point for all Jepsen tests. Coordinates the setup of servers, running
tests, creating and resolving failures, and interpreting results.

Jepsen tests a system by running a set of singlethreaded *processes*, each
representing a single client in the system, and a special *nemesis* process,
which induces failures across the cluster. Processes choose operations to
perform based on a *generator*. Each process uses a *client* to apply the
operation to the distributed system, and records the invocation and
completion of that operation in the *history* for the test. When the test is
complete, a *checker* analyzes the history to see if it made sense.

Jepsen automates the setup and teardown of the environment and distributed
system by using an *OS* and *client* respectively. See `run!` for details.
raw docstring

analyze!clj

(analyze! test)

After running the test and obtaining a history, we perform some post-processing on the history, run the checker, and write the test to disk again.

After running the test and obtaining a history, we perform some
post-processing on the history, run the checker, and write the test to disk
again.
raw docstring

client-workerclj

(client-worker test process-id node)

A worker for executing operations on clients. Takes a test, an initial process id, and a node to bind clients to.

A worker for executing operations on clients. Takes a test, an initial
process id, and a node to bind clients to.
raw docstring

conj-op!clj

(conj-op! test op)

Add an operation to a tests's history, and returns the operation.

Add an operation to a tests's history, and returns the operation.
raw docstring

invoke-op!clj

(invoke-op! op test client abort?)

Applies an operation to a client, catching client exceptions and converting them to infos. Returns a completion op, throwing if the completion is invalid.

Applies an operation to a client, catching client exceptions and converting
them to infos. Returns a completion op, throwing if the completion is
invalid.
raw docstring

log-resultsclj

(log-results test)

Logs info about the results of a test to stdout, and returns test.

Logs info about the results of a test to stdout, and returns test.
raw docstring

nemesis-apply-op!clj

(nemesis-apply-op! op test nemesis abort?)

Logs, journals, and invokes an operation, logging and journaling its completion, and returning the completed operation.

Logs, journals, and invokes an operation, logging and journaling its
completion, and returning the completed operation.
raw docstring

nemesis-invoke-op!clj

(nemesis-invoke-op! op test client abort?)

Applies an operation to a nemesis, catching exceptions and converting them to infos. Returns a completion op, throwing if the completion is invalid.

Applies an operation to a nemesis, catching exceptions and converting
them to infos. Returns a completion op, throwing if the completion is
invalid.
raw docstring

nemesis-workerclj

(nemesis-worker test)

A worker for introducing failures. Takes a test.

A worker for introducing failures. Takes a test.
raw docstring

primaryclj

(primary test)

Given a test, returns the primary node.

Given a test, returns the primary node.
raw docstring

run!clj

(run! test)

Runs a test. Tests are maps containing

:nodes A sequence of string node names involved in the test :concurrency (optional) How many processes to run concurrently :ssh SSH credential information: a map containing... :username The username to connect with (root) :password The password to use :port SSH listening port (22) :private-key-path A path to an SSH identity file (~/.ssh/id_rsa) :strict-host-key-checking Whether or not to verify host keys :logging Logging options; see jepsen.store/start-logging! :os The operating system; given by the OS protocol :db The database to configure: given by the DB protocol :client A client for the database :nemesis A client for failures :generator A generator of operations to apply to the DB :checker Verifies that the history is valid :log-files A list of paths to logfiles/dirs which should be captured at the end of the test. :nonserializable-keys A collection of top-level keys in the test which shouldn't be serialized to disk.

Tests proceed like so:

  1. Setup the operating system

  2. Try to teardown, then setup the database

  • If the DB supports the Primary protocol, also perform the Primary setup on the first node.
  1. Create the nemesis

  2. Fork the client into one client for each node

  3. Fork a thread for each client, each of which requests operations from the generator until the generator returns nil

  • Each operation is appended to the operation history
  • The client executes the operation and returns a vector of history elements
    • which are appended to the operation history
  1. Capture log files

  2. Teardown the database

  3. Teardown the operating system

  4. When the generator is finished, invoke the checker with the history

  • This generates the final report
Runs a test. Tests are maps containing

:nodes      A sequence of string node names involved in the test
:concurrency  (optional) How many processes to run concurrently
:ssh        SSH credential information: a map containing...
  :username           The username to connect with   (root)
  :password           The password to use
  :port               SSH listening port (22)
  :private-key-path   A path to an SSH identity file (~/.ssh/id_rsa)
  :strict-host-key-checking  Whether or not to verify host keys
:logging    Logging options; see jepsen.store/start-logging!
:os         The operating system; given by the OS protocol
:db         The database to configure: given by the DB protocol
:client     A client for the database
:nemesis    A client for failures
:generator  A generator of operations to apply to the DB
:checker    Verifies that the history is valid
:log-files  A list of paths to logfiles/dirs which should be captured at
            the end of the test.
:nonserializable-keys   A collection of top-level keys in the test which
                        shouldn't be serialized to disk.

Tests proceed like so:

1. Setup the operating system

2. Try to teardown, then setup the database
  - If the DB supports the Primary protocol, also perform the Primary setup
    on the first node.

3. Create the nemesis

4. Fork the client into one client for each node

5. Fork a thread for each client, each of which requests operations from
   the generator until the generator returns nil
  - Each operation is appended to the operation history
  - The client executes the operation and returns a vector of history elements
    - which are appended to the operation history

6. Capture log files

7. Teardown the database

8. Teardown the operating system

9. When the generator is finished, invoke the checker with the history
  - This generates the final report
raw docstring

run-case!clj

(run-case! test)

Spawns nemesis and clients, runs a single test case, and returns that case's history.

Spawns nemesis and clients, runs a single test case, and
returns that case's history.
raw docstring

run-workers!clj

(run-workers! workers)

Runs a set of workers through setup, running, and teardown.

Runs a set of workers through setup, running, and teardown.
raw docstring

snarf-logs!clj

(snarf-logs! test)

Downloads logs for a test.

Downloads logs for a test.
raw docstring

synchronizeclj

(synchronize test)
(synchronize test timeout-s)

A synchronization primitive for tests. When invoked, blocks until all nodes have arrived at the same point.

This is often used in IO-heavy DB setup code to ensure all nodes have completed some phase of execution before moving on to the next. However, if an exception is thrown by one of those threads, the call to synchronize will deadlock! To avoid this, we include a default timeout of 60 seconds, which can be overridden by passing an alternate timeout in seconds.

A synchronization primitive for tests. When invoked, blocks until all nodes
have arrived at the same point.

This is often used in IO-heavy DB setup code to ensure all nodes have
completed some phase of execution before moving on to the next. However, if
an exception is thrown by one of those threads, the call to `synchronize`
will deadlock! To avoid this, we include a default timeout of 60 seconds,
which can be overridden by passing an alternate timeout in seconds.
raw docstring

with-dbcljmacro

(with-db test & body)

Wraps body in DB setup and teardown.

Wraps body in DB setup and teardown.
raw docstring

with-log-snarfingcljmacro

(with-log-snarfing test & body)

Evaluates body and ensures logs are snarfed afterwards. Will also download logs in the event of JVM shutdown, so you can ctrl-c a test and get something useful.

Evaluates body and ensures logs are snarfed afterwards. Will also download
logs in the event of JVM shutdown, so you can ctrl-c a test and get something
useful.
raw docstring

with-oscljmacro

(with-os test & body)

Wraps body in OS setup and teardown.

Wraps body in OS setup and teardown.
raw docstring

with-resourcescljmacro

(with-resources [sym start stop resources] & body)

Takes a four-part binding vector: a symbol to bind resources to, a function to start a resource, a function to stop a resource, and a sequence of resources. Then takes a body. Starts resources in parallel, evaluates body, and ensures all resources are correctly closed in the event of an error.

Takes a four-part binding vector: a symbol to bind resources to, a function
to start a resource, a function to stop a resource, and a sequence of
resources. Then takes a body. Starts resources in parallel, evaluates body,
and ensures all resources are correctly closed in the event of an error.
raw docstring

Workercljprotocol

Polymorphic lifecycle for worker threads; synchronized setup, run, and teardown phases, each with error recovery. Workers are singlethreaded and may be stateful. Return value are ignored.

Polymorphic lifecycle for worker threads; synchronized setup, run, and
teardown phases, each with error recovery. Workers are singlethreaded and may
be stateful. Return value are ignored.

abort-worker!clj

(abort-worker! worker)

run-worker!clj

(run-worker! worker)

setup-worker!clj

(setup-worker! worker)

teardown-worker!clj

(teardown-worker! worker)

worker-nameclj

(worker-name worker)
raw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close