A Clojure wrapper over the great library Resilience4j to provide fault tolerance to your service. Picked a lot of ideas from another great library diehard.
As you may expected, it has Circuit Breaker, Retry, Bulkhead, Rate Limiter.
And there's also an example to show how to use them all together here and how to handle exceptions here.
(require '[resilience.breaker :as breaker]
'[resilience.core :as resilience])
;; define a breaker
;; when over 80% of the last 30 requests failed, the breaker will open for one second
(breaker/defbreaker my-breaker
{:failure-rate-threshold 80
:ring-buffer-size-in-closed-state 30
:wait-millis-in-open-state 1000})
;; do something with the protection of the breaker
(resilience/execute-with-breaker my-breaker
(do-something)
(do-another-thing))
(require '[resilience.retry :as retry]
'[resilience.core :as resilience])
;; define a retry
;; retry the request at most 3 times with a one second interval when any exception occurs
(retry/defretry my-retry
{:max-attempts 3
:wait-millis 1000})
;; do something with the protection of the retry
(resilience/execute-with-retry my-retry
(do-something)
(do-another-thing))
(require '[resilience.bulkhead :as bulkhead]
'[resilience.core :as resilience])
;; define a bulkhead
;; allow at most 3 parallel executions
(bulkhead/defbulkhead my-bulkhead
{:max-concurrent-calls 3
:max-wait-millis 1000})
;; do something with the protection of the bulkhead
(resilience/execute-with-bulkhead my-bulkhead
(do-something)
(do-another-thing))
(require '[resilience.ratelimiter :as ratelimiter]
'[resilience.core :as resilience])
;; define a ratelimiter
;; allow at most 10 requests per second
(ratelimiter/defratelimiter my-ratelimiter
{:limit-for-period 10
:limit-refresh-period-millis 1000
:timeout-millis 1000})
;; do something with the protection of the ratelimiter
(resilience/execute-with-ratelimiter my-ratelimiter
(do-something)
(do-another-thing))
(require '[resilience.breaker :as breaker]
'[resilience.bulkhead :as bulkhead]
'[resilience.retry :as retry]
'[resilience.ratelimiter :as ratelimiter]
'[resilience.core :as resilience])
;; define a breaker
(breaker/defbreaker my-breaker
{:failure-rate-threshold 80
:ring-buffer-size-in-closed-state 30
:wait-millis-in-open-state 1000})
;; define a retry
(retry/defretry my-retry
{:max-attempts 3
:wait-millis 1000})
;; define a bulkhead
(bulkhead/defbulkhead my-bulkhead
{:max-concurrent-calls 3
:max-wait-millis 1000})
;; define a ratelimiter
(ratelimiter/defratelimiter my-ratelimiter
{:limit-for-period 10
:limit-refresh-period-millis 1000
:timeout-millis 1000})
;; use them all together
(resilience/with-resilience-family
[:retry my-retry :breaker my-breaker :bulkhead my-bulkhead :rate-limiter my-ratelimiter]
(do-something)
(do-another-thing))
;; an alternative way
(resilience/execute
(do (do-something)
(do-another-thing))
(resilience/with-retry my-retry)
(resilience/with-breaker my-breaker)
(resilience/with-bulkhead my-bulkhead)
(resilience/with-rate-limiter my-ratelimiter))
Please note that the second parameter passed to with-resilience-family
is a list. It means the order of the family members is matters. Such as usually you need to put retry policy before circuit breaker to insure that when the circuit breaker is open you do not retry further in vain. And the same is for using execute
in which you need to mind the order of those with-*
forms.
What we missing until now is how to handle exceptions thrown by resilience family members. Most of resilience family members have their corresponding exception which will be thrown when certain conditions match. Such as when circuit breaker is open, the subsequent requests will trigger CircuitBreakerOpenException
in circuit breaker. And for bulkhead, when bulkhead is full, the subsequent parallel requests will trigger BulkheadFullException
in bulkhead. What is matters here is that sometimes you need to handle all these exceptions respectively which may force you to add try ... catch
blocks to protect your codes then make them not as concise as above examples. After adding exception handling stuff, the example before looks like this:
(try
(resilience/with-resilience-family
[:retry my-retry :breaker my-breaker :bulkhead my-bulkhead :rate-limiter my-ratelimiter]
(do-something)
(do-another-thing))
(catch CircuitBreakerOpenException ex
(log-circuit-breaker-open-and-return-a-fallback-value ex))
(catch BulkheadFullException ex
(log-bulkhead-full-and-return-a-fallback-value ex))
(catch RequestNotPermitted ex
(log-request-not-permitted-and-return-a-fallback-value ex))
(catch Exception ex
(log-unexpected-exception-and-return-a-fallback-value ex)))
If you don't like to use try ... catch
block, you can choose to use execute
with recover
form like this:
(resilience/execute
(do (do-something)
(do-another-thing))
(resilience/with-retry my-retry)
(resilience/with-breaker my-breaker)
(resilience/with-bulkhead my-bulkhead)
(resilience/with-rate-limiter my-ratelimiter)
(resilience/recover-from CircuitBreakerOpenException log-circuit-breaker-open-and-return-a-fallback-value)
(resilience/recover-from BulkheadFullException log-bulkhead-full-and-return-a-fallback-value)
(resilience/recover-from RequestNotPermitted log-request-not-permitted-and-return-a-fallback-value)
(resilience/recover log-unexpected-exception-and-return-a-fallback-value))
There's not much difference between them. Usually you can just choose any one style of them and stick on it.
All of Circuit Breaker, Retry, Rate Limiter can register to their corresponding Registry
and use Registry
as a management tool. Take Circuit Breaker as an example. You can use Circuit Breaker Registry
to create, retrieve and monitor all your Circuit Breaker
instances registered to it. Like:
(require '[resilience.breaker :as breaker]
'[resilience.core :as resilience])
;; define a Circuit Breaker Registry
(breaker/defregistry my-breaker-registry
{:failure-rate-threshold 80
:wait-millis-in-open-state 1000
:ring-buffer-size-in-closed-state 30})
;; define a breaker and register it to the Circuit Breaker Registry
;; the new breaker will inherit configuration from the Registry
(breaker/defbreaker my-registered-breaker {:registry my-breaker-registry})
;; then we can use the circuit breaker as usual
(resilience/execute-with-breaker my-breaker
(do-something)
(do-another-thing))
;; and from the Registry we can get all circuit breakers registered to it
(doseq [cb (breaker/get-all-breakers my-breaker-registry)]
(let [state (breaker/state cb)
metrics (breaker/metrics cb)]
(do-some-monitoring-stuff cb state metrics)))
Can you improve this documentation?Edit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close