exoscale.checkmate — exoscale/checkmate 1.0.0-alpha7

3 main states

• success (all good)
• error (bad, potentially retry)
• failure (terminal failure)

• Conditions control if we should retry and how to setup the ctx at first call, and how to update the ctx upon failures (inc retry counter for instance). Conditions are in effect, composable and should remain pure (ctx in/out).

You can for instance do:

;; will retry up to 3 times with a 1000ms delay between tries (cm/do! #(do-something) [(delayed-retries (constant-backoff-delays 1000)) (max-retries 3)])

• Effects can apply side effects at various stages (ex sleep, update state machine), they are purely for io, as opposed to Conditions

constant-backoff-delays clj

(constant-backoff-delays ms)

delays clj

(delays coll)

exponential-backoff-delays clj

(exponential-backoff-delays x)

fail-on clj

(fail-on pred)

Set to failed on (pred ret) -> true Similar to retry-on, the other way around. Fails on match.

max-in-flight clj

(max-in-flight max)

max-retries clj

(max-retries max)

progressive-backoff-delays clj

(progressive-backoff-delays x)

rate-limited-retries clj

(rate-limited-retries
  {:keys [max-errors error-rate exception]
   :or {max-errors 10
        error-rate 1000
        exception (fn [ctx]
                      (ex-info "Rejected execution, max rate reached"
                               {:type :exoscale.ex/busy
                                :exoscale.checkmate/ctx ctx}))}})

!! This condition should be shared for all runner calls, since it's stateful

Only allows a max rate of errors until it plain rejects calls until rate can be satisfied again:

Example:

max-errors : 10 error-rate : 1000ms

Allows up to 10 successive errors, upon token exhaustion will only allow 1 error per second in burst, until it fills again to 10.

So this allows burst'iness up to a point.

If you create a lot or rate-limited-retries conditions you need to remember to async/close! the associated chan, or pass it as argument. The chan is also available via Condition ctx, in case you want to enable temporary burst'iness, early flush or closing

retry-on clj

(retry-on x)

Allow retries on (pred e) -> true, useful to limit retries to a class of errors ex: (retry-on #(instance? TimeoutException %)). x can be a function, a Class name (ex: some exception type), or a keyword in that case it will match (exoscale.ex/type? x %).

retry-on* clj

(retry-on* pred)

like retry-on, but runs on the whole ctx instead of just result

run clj

(run f conditions)

(run f conditions opts)

Synchronous runner, take function f and runs it. Upon error will check Conditions passed as arguments to know if it warrants a retry, otherwise it triggers a failure and rethrows the last error.

At every stage hooks are invoked, and potentially Effects from Conditions.

• hook: map of allow to setup event at various stages (logging, reporting)
• conditions: sequence of Conditions to be used

timeout clj

(timeout ms)

(timeout ms {:as _opts :keys [exception]})