jepsen.util — jepsen 0.1.14

Kitchen sink

*relative-time-origin* clj

all-jdk-loggers clj

(all-jdk-loggers)

buf-size clj

chunk-vec clj

(chunk-vec n v)

Partitions a vector into reducibles of size n (somewhat like partition-all) but uses subvec for speed.

(chunk-vec 2 [1])     ; => ([1])
(chunk-vec 2 [1 2 3]) ; => ([1 2] [3])

coll clj

(coll thing-or-things)

Wraps non-coll things into singleton lists, and leaves colls as themselves. Useful when you can take either a single thing or a sequence of things.

compare< clj

(compare< a b)

Like <, but works on any comparable objects, not just numbers.

concat-files! clj

(concat-files! out fs)

Appends contents of all fs, writing to out. Returns fs.

contains-many? clj

(contains-many? m & ks)

Takes a map and any number of keys, returning true if all of the keys are present. Ex. (contains-many? {:a 1 :b 2 :c 3} :a :b :c) => true

default clj

(default m k v)

Like assoc, but only fills in values which are NOT present in the map.

drop-common-proper-prefix clj

(drop-common-proper-prefix cs)

Given a collection of sequences, removes the longest common proper prefix from each one.

exception? clj

(exception? x)

Is x an Exception?

fast-last clj

(fast-last coll)

Like last, but O(1) on counted collections.

fcatch clj

(fcatch f)

Takes a function and returns a version of it which returns, rather than throws, exceptions.

fraction clj

(fraction a b)

a/b, but if b is zero, returns unity.

get-named-lock! clj

(get-named-lock! locks name)

Given a pool of locks, and a lock name, returns the object used for locking in that pool. Creates the lock if it does not already exist.

history->latencies clj

(history->latencies history)

Takes a history--a sequence of operations--and emits the same history but with every invocation containing two new keys:

:latency the time in nanoseconds it took for the operation to complete. :completion the next event for that process

inc* clj

(inc* x)

Like inc, but (inc nil) => 1.

integer-interval-set-str clj

(integer-interval-set-str set)

Takes a set of integers and yields a sorted, compact string representation.

lazy-atom clj

(lazy-atom f)

An atom with lazy state initialization. Calls (f) on first use to provide the initial value of the atom. Only supports swap/reset/deref. Reset bypasses lazy initialization. If f throws, behavior is undefined (read: proper fucked).

letr clj macro

(letr bindings & body)

Let bindings, plus early return.

You want to do some complicated, multi-stage operation assigning lots of variables--but at different points in the let binding, you need to perform some conditional check to make sure you can proceed to the next step. Ordinarily, you'd intersperse let and if statements, like so:

(let [res (network-call)]
  (if-not (:ok? res)
    :failed-network-call

    (let [people (:people (:body res))]
      (if (zero? (count people))
        :no-people

        (let [res2 (network-call-2 people)]
          ...

This is a linear chain of operations, but we're forced to nest deeply because we have no early-return construct. In ruby, we might write

res = network_call
return :failed_network_call if not x.ok?

people = res[:body][:people]
return :no-people if people.empty?

res2 = network_call_2 people
...

which reads the same, but requires no nesting thanks to Ruby's early return. Clojure's single-return is usually a boon to understandability, but deep linear branching usually means something like

• Deep nesting (readability issues)
• Function chaining (lots of arguments for bound variables)
• Throw/catch (awkward exception wrappers)
• Monadic interpreter (slow, indirect)

This macro lets you write:

(letr [res    (network-call)
       _      (when-not (:ok? res) (return :failed-network-call))
       people (:people (:body res))
       _      (when (zero? (count people)) (return :no-people))
       res2   (network-call-2 people)]
  ...)

letr works like let, but if (return x) is ever returned from a binding, letr returns x, and does not evaluate subsequent expressions.

If something other than (return x) is returned from evaluating a binding, letr binds the corresponding variable as normal. Here, we use _ to indicate that we're not using the results of (when ...), but this is not mandatory. You cannot use a destructuring bind for a return expression.

letr is not a true early return--(return x) must be a terminal expression for it to work--like (recur). For example,

(letr [x (do (return 2) 1)]
  x)

returns 1, not 2, because (return 2) was not the terminal expression.

return only works within letr's bindings, not its body.

letr-let-if clj

(letr-let-if groups body)

Takes a sequence of binding groups and a body expression, and emits a let for the first group, an if statement checking for a return, and recurses; ending with body.

letr-partition-bindings clj

(letr-partition-bindings bindings)

Takes a vector of bindings [sym expr, sym' expr, ...]. Returns binding-groups: a sequence of vectors of bindgs, where the final binding in each group has an early return. The final group (possibly empty!) contains no early return.

letr-rewrite-return clj

(letr-rewrite-return expr)

Rewrites (return x) to (Return. x) in expr. Returns a pair of [changed? expr], where changed is whether the expression contained a return.

linear-time-nanos clj

(linear-time-nanos)

A linear time source in nanoseconds.

local-time clj

(local-time)

Drops millisecond resolution

log clj

(log & things)

log-op clj

(log-op op)

Logs an operation and returns it.

log-print clj

(log-print _ & things)

logger clj

longest-common-prefix clj

(longest-common-prefix cs)

Given a collection of sequences, finds the longest sequence which is a prefix of every sequence given.

majority clj

(majority n)

Given a number, returns the smallest integer strictly greater than half.

map-keys clj

(map-keys f m)

Maps keys in a map.

map-kv clj

(map-kv f m)

Takes a function (f [k v]) which returns [k v], and builds a new map by applying f to every pair.

map-vals clj

(map-vals f m)

Maps values in a map.

max-by clj

(max-by f coll)

Finds the maximum element of a collection based on some (f element), which returns Comparables. If coll is empty, returns nil.

maybe-number clj

(maybe-number s)

Tries reading a string as a long, then double, then string. Passes through nil. Useful for getting nice values out of stats APIs that just dump a bunch of heterogenously-typed strings at you.

meh clj macro

(meh & body)

Returns, rather than throws, exceptions.

min-by clj

(min-by f coll)

Finds the minimum element of a collection based on some (f element), which returns Comparables. If coll is empty, returns nil.

ms->nanos clj

(ms->nanos ms)

mute clj macro

(mute & body)

mute-jdk clj macro

(mute-jdk & body)

name+ clj

(name+ x)

Tries name, falls back to pr-str.

named-locks clj

(named-locks)

Creates a mutable data structure which backs a named locking mechanism.

Named locks are helpful when you need to coordinate access to a dynamic pool of resources. For instance, you might want to prohibit multiple threads from executing a command on a remote node at once. Nodes are uniquely identified by a string name, so you could write:

(defonce node-locks (named-locks))

...
(defn start-db! [node]
  (with-named-lock node-locks node
    (c/exec :service :meowdb :start)))

Now, concurrent calls to start-db! will not execute concurrently.

The structure we use to track named locks is an atom wrapping a map, where the map's keys are any object, and the values are canonicalized versions of that same object. We use standard Java locking on the canonicalized versions. This is basically an arbitrary version of string interning.

nanos->ms clj

(nanos->ms nanos)

nanos->secs clj

(nanos->secs nanos)

nemesis-intervals clj

(nemesis-intervals history)

(nemesis-intervals history opts)

Given a history where a nemesis goes through :f :start and :f :stop type transitions, constructs a sequence of pairs of start and stop ops. Since a nemesis usually goes :start :start :stop :stop, we construct pairs of the first and third, then second and fourth events. Where no :stop op is present, we emit a pair like [start nil]. Optionally, a map of start and stop sets may be provided to match on user-defined :start and :stop keys.

Multiple starts are ended by the same pair of stops, so :start1 :start2 :start3 :start4 :stop1 :stop2 yields:

[start1 stop1] [start2 stop2] [start3 stop1] [start4 stop2]

op->str clj

(op->str op)

Format an operation as a string.

poly-compare clj

(poly-compare a b)

Comparator function for sorting heterogenous collections.

polysort clj

(polysort coll)

Sort, but on heterogenous collections.

pprint-str clj

(pprint-str x)

print-history clj

(print-history history)

(print-history printer history)

Prints a history to the console.

prn-op clj

(prn-op op)

Prints an operation to the console.

processors clj

(processors)

How many processors on this platform?

pwrite-history! clj

(pwrite-history! f history)

(pwrite-history! f printer history)

Writes history, taking advantage of more cores.

rand-nth-empty clj

(rand-nth-empty coll)

Like rand-nth, but returns nil if the collection is empty.

random-nonempty-subset clj

(random-nonempty-subset nodes)

A randomly selected, randomly ordered, non-empty subset of the given collection.

real-pmap clj

(real-pmap f coll)

Like pmap, but launches futures instead of using a bounded threadpool.

relative-time-nanos clj

(relative-time-nanos)

Time in nanoseconds since relative-time-origin

retry clj macro

(retry dt & body)

Evals body repeatedly until it doesn't throw, sleeping dt seconds.

secs->nanos clj

(secs->nanos s)

sequential clj

(sequential thing-or-things)

Wraps non-sequential things into singleton lists, and leaves sequential things or nil as themselves. Useful when you can take either a single thing or a sequence of things.

sleep clj

(sleep dt)

High-resolution sleep; takes a (possibly fractional) time in ms.

spy clj

(spy x)

time- clj macro

(time- & body)

timeout clj macro

(timeout millis timeout-val & body)

Times out body after n millis, returning timeout-val.

with-named-lock clj macro

(with-named-lock locks name & body)

Given a lock pool, and a name, locks that name in the pool for the duration of the body.

with-relative-time clj macro

(with-relative-time & body)

Binds relative-time-origin at the start of body.

with-retry clj macro

(with-retry initial-bindings & body)

It's really fucking inconvenient not being able to recur from within (catch) expressions. This macro wraps its body in a (loop [bindings] (try ...)). Provides a (retry & new bindings) form which is usable within (catch) blocks: when this form is returned by the body, the body will be retried with the new bindings.

with-thread-name clj macro

(with-thread-name thread-name & body)

Sets the thread name for duration of block.

write-history! clj

(write-history! f history)

(write-history! f printer history)

Writes a history to a file.