Whew
A zero-deps library that wraps Java's CompletableFuture class. Provides
functions and macros to drag futures through handlers, looping over them,
mapping and so on. Deals with nested futures (when a future returns a future and
so on).
The library might remind you Manifold or Auspex. But the API is a bit different and it handles some special cases.
Installation
Requires Java version at least 8. Add a dependency:
;; lein
[com.github.igrishaev/whew "0.1.0"]
;; deps
com.github.igrishaev/whew {:mvn/version "0.1.0"}
Usage
Import the library:
(ns org.some.project
(:require
[whew.core :as $]))
Whew provides functions and macros named after their Clojure counterparts,
e.g. map, future, loop, etc. Never :use this library but rather
:require it using an alias. Here and below we will use $.
A quick demo. Let's prepare a function that makes some IO, for example fetches JSON from network:
(defn get-json [code]
(-> (format "https://http.dog/%d.json" code)
(http/get {:as :json
:throw-exceptions true})
:body))
Here is how you run it as a future:
(def -f ($/future
(get-json 101)))
The future macro accepts arbitrary block of code and wraps it into a future
that gets executed in the background. Now deref it, and you'll get a result:
@-f
{:image
{:avif "https://http.dog/101.avif"
:jpg "https://http.dog/101.jpg"
:jxl "https://http.dog/101.jxl"
:webp "https://http.dog/101.webp"}
:status_code 101
:title "Switching Protocols"
:url "https://http.dog/101"}
By derefing a future, you freeze the current thread forcing it to wait until the future is ready. But you can assign a post-processing handler which gets run in the background with a result of a future. Adding a handler returns a new future. Below, we compose a bit of HTML markup out from a fetched data:
(-> -f
($/then [data]
(-> data :image :jpg))
($/then [url]
[:a {:src url}
"Click me!"])
(deref))
[:a {:src "https://http.dog/101.jpg"} "Click me!"]
Each then handler accepts a future and a binding symbol. It is bound to a
result of a previous future. The block of code can return either a flat value or
a future as well:
(-> ($/future
(get-json 201))
($/then [data]
($/future
(do-something-else (:url data))))
($/then [response]
...))
You can enqueue as many then handles as you want.
Should any of them throw an exception, a future becomes failed, and no more
further handlers apply. If you deref such a failed future, you'll get an
exception. To recover from it, there is another catch handler:
(-> ($/future
(get-json 333)) ;; weird code
($/catch [e]
{:error true
:message (ex-message e)})
(deref))
{:error true, :message "clj-http: status 404"}
The e symbol is bound to an exception value occurred before. What's important,
it will be an unwrapped exception! By default, the CompletableFuture class
wraps any runtime exception into various classes like ExecutionException or
CompletionException. If you branch your logic depending on exception class or
inheritance, you'll need to get an ex-cause first. But the catch macro
handles it for you: the e variable will be of the right class.
Whew provides a number of various macros to express logic throughout futures: mixing and chaining them, zipping, waiting multiple futures for completion and so on. The section below describes these in detail.
API
Creating Futures
Both future and future-async macros take an arbitrary block of code and
return a future that carries a result of this block:
($/future 42)
($/future-async
(let [...]
(do-long-job ...)))
A future might return a future which returns a future which returs a future and so on:
($/future
($/future
($/future
($/future 42))))
To handle the result in a standard way, you'll have to deref it four times:
@@@@($/future
($/future
($/future
($/future 42))))
But Whew handles such cases:
(-> ($/future
($/future
($/future
($/future 42))))
($/then [x] (inc x)) ;; x = 42
(deref))
43
There is a special deref function that takes folding into account:
(-> ($/future
($/future
($/future
($/future 42))))
($/deref))
42
Usually you don't need to deref futures explicitly neither with the standard
deref nor the custom $/deref.
The future-sync macro takes a block of code but executes it immediately in the
same thread and produces a completed future. When a future is completed, it
means it can be deref-fed right now without delay. This is useful when you
want just to mimic a future.
The block of code is implicitly wrapped with try/catch. Should an exception pop up, the result will be a failed future with this exception:
The following piece of code will throw immediately:
(def -f ($/future-sync (/ 0 0)))
($/failed? -f)
true
@-f
;; Execution error (ArithmeticException)...
;; Divide by zero
The ->future function turns any value into a completed future. Any Throwable
instance produces a failed future: the one than cannot be propagated through
then handlers, but only catch.
(-> ($/->future 1)
($/then [x]
(inc x))
($/deref))
;; 2
(-> ($/->future (ex-info "boom" {:a 1}))
($/then [x]
(inc x))
($/catch [e]
{:data (ex-data e)})
($/deref))
;; {:data {:a 1}}
The future? predicate checks if a g value is a future:
($/future? ($/future 1))
true
($/future? 1)
false
The failed? predicated checks if a future has failed. Pay attention that in
the beginning, we don't know it untill it really has:
(def -f ($/future
(Thread/sleep 5000)
(/ 0 0)))
($/failed? -f)
false
;; wait for 5 seconds
($/failed? -f)
true
On Executors
By default, the CompletableFuture class relies on the
ForkJoinPool/commonPool executor although it's possible to override it. By
running benchmarks, I noticed that the standard
clojure.core.Agent/soloExecutor used for built-in Clojure futures and agents
is more robust. Thus, when you spawn futures using ($/future) and
($/future-async) macros, the soloExecutor executor is used.
The future-via macro acts like future-async but accepts a custom Executor
instance to work with. Any further then and catch handlers will be served
under the same executor as well. Here is an example of using a custom
two-threaded executor:
(with-open [executor
(Executors/newFixedThreadPool 2)]
(-> ($/future-via [executor]
(let [a 1 b 2]
(+ a b)))
($/then [x]
...)))
You can also use a virtual executor that relies on virtual threads available since Java 21:
(with-open [e (Executors/newVirtualThreadPerTaskExecutor)]
($/future-via [a]
...))
The default executor which Whew relies on is stored in the
whew.core/EXECUTOR_DEFAULT variable, and it's initial value is
Agent/soloExecutor. You can switch it globally to something else as follows:
($/set-executor! some-executor)
At the moment, Whew provides the following Executor constants:
$/EXECUTOR_CLJ_SOLO: a built-itAgent/soloExecutorClojure executor;$/EXECUTOR_CLJ_POOLED: a built-itAgent/pooledExecutorClojure executor;$/EXECUTOR_FJ_COMMON: a default ForkJoin common pool.
Chaining Futures
Then macro provides a new future based on the previous one:
@(-> ($/future 1) ($/then [x] (inc x)))
2
It works with non-future values as well. Internally, they get transformed into a completed future:
@(-> 1 ($/then [x] (inc x)))
2
A macro then-fn acts the same but accepts a function which gets applied to a
result of a previous future:
@(-> 1 ($/then-fn inc))
2
You can pass additional arguments too:
@(-> 1 ($/then-fn + 100))
101
The catch macro handles an exception occurred beforehand. Pay attention that
the second inc form didn't work because it doesn't apply to a failed future:
@(-> ($/future 1)
($/then-fn inc) ;; works
($/then-fn / 0) ;; fails
($/then-fn inc) ;; unreached
($/catch [e] ;; recovered
:this-is-fine))
:this-is-fine
As it was mentioned above, the macro unwraps exceptions. The e variable will
be an instance of ArithmeticException but not ExecutionException.
The catch-fn macro acts the same but accepts 1-argument function that handles
an exception:
@(-> ($/future 1)
($/then [x]
(throw (ex-info "boom" {:foo 1})))
($/catch-fn ex-data)
($/then [data]
{:ex-data data}))
{:ex-data {:foo 1}}
It accepts additional arguments like then-fn does but usually they're not
needed.
The handle macro captures both a result and an exception as a pair:
@(-> ($/future 1)
($/handle [r e]
{:result r :exception e}))
{:result 1 :exception nil}
Usually you check if an exception is nil to decide the logic. The handle-fn
macro is similar but accepts a 2-arity function which handles both a result and
an exception:
@(-> ($/future 1)
($/handle-fn
(fn [r e]
{:result r :exception e})))
{:result 1 :exception nil}
In both cases, exceptions are unwrapped.
Dereferencing
The standard @ operator and the deref function get a value from a future but
don't take multiple levels into account:
@($/future
($/future
($/future
($/future 1))))
#object[...CompletableFuture 0x287238e4 "...[Completed normally]"]
So you've to go to the end:
@@@@($/future
($/future
($/future
($/future 1))))
1
But the $/deref function from Whew does the same with no issues:
($/deref
($/future
($/future
($/future
($/future 1)))))
1
To not hang forever, it accepts an amount of milliseconds (how long to wait) and a default value to return on timeout:
($/deref ($/future
(Thread/sleep 1000)
:done)
100
:too-long)
:too-long
Folding
Folding a future means removing its unnecessarily levels, for example:
;; this
($/future ($/future ($/future 1)))
;; becomes this
($/future 1)
When a future is folded, only one deref is required to obtain a value. The
fold function does it:
@($/fold ($/future ($/future ($/future 1))))
;; 1
The deref function described above is nothing but a combo of fold and .get
invocations.
Usually you don't need to fold futures manually because then, catch and
other macros do it for you.
Zipping, Any-of, One-of
The zip macro accepts a number of forms. Each form is turned into an async
future. The result is a future that gets completed when all the futures complete
(either successfully or with an exception). It gets a vector of plain values:
(-> ($/zip 1 ($/future ($/future 2)) 3)
($/then [vs]
{:values vs})
(deref))
{:values [1 2 3]}
Should any future fail, so the entire future does with the same exception:
(-> ($/zip 1 ($/future ($/future (/ 0 0))) 3)
($/then [vs]
{:values vs})
($/catch [e]
{:error (ex-message e)})
(deref))
{:error "Divide by zero"}
The all-of function acts the same: it accepts a collection of futures and
returns a future with completed items:
(-> ($/all-of [1 ($/future ($/future 2)) 3])
($/then [vs]
{:values vs})
(deref))
{:values [1 2 3]}
The difference is, all-of is a function but not a macro. It's useful when you
have a collection of futures produced by other functions.
The any-of function takes a collection of futures and waits for the first
completed one. The result is a future that carries a value of this future:
@($/any-of [($/future
(Thread/sleep 300)
:A)
($/future
(Thread/sleep 200)
:B)
($/future
(Thread/sleep 100)
:C)])
;; C
The result is :C because the third future took less time to complete.
The Let Macro
The let macro mimics the one from the standard Clojure library, but:
- any value (in the right binding part) can return a future;
- bindings must not depend on each other;
- the body is executed when all the futures are completed;
- the body has access to derefed values but not futures.
Here is a small demo. We use the get-json function that fetches a piece of
data by a numeric code.
@($/let [resp-101 ($/future (get-json 101))
resp-202 ($/future (get-json 202))
resp-404 ($/future (get-json 404))]
{101 resp-101
202 resp-202
404 resp-404})
{101
{:image
{:jpg "https://http.dog/101.jpg"}
:title "Switching Protocols"
:url "https://http.dog/101"}
202
{:image
{:jpg "https://http.dog/202.jpg"}
:title "Accepted"
:url "https://http.dog/202"}
404
{:image
{:jpg "https://http.dog/404.jpg"}
:title "Not Found"
:url "https://http.dog/404"}}
What's important, all three get-json invocations are done in parallel. When
every of them is ready, the body gets access to their values.
Should any binding fail, the entire let node fails as well:
@(-> ($/let [resp-101 ($/future (get-json 101))
resp-321 ($/future (get-json 321)) ;; wrong code
resp-404 ($/future (get-json 404))]
{101 resp-101
321 resp-321
404 resp-404})
($/catch [e]
(-> e
ex-data
(select-keys [:status :reason-phrase]))))
{:status 404 :reason-phrase "Not Found"}
If you unsure about a certain binding, protect it with a catch macro:
@(-> ($/let [resp-101 ($/future (get-json 101))
resp-321 (-> ($/future (get-json 321))
($/catch [e]
{:error true
:code 321}))
resp-404 ($/future (get-json 404))]
{101 resp-101
321 resp-321
404 resp-404}))
{101
{:title "Switching Protocols"
:url "https://http.dog/101"}
321 {:error true, :code 321}
404
{:title "Not Found"
:url "https://http.dog/404"}}
For & Map
The for macro acts like the standard for but wraps each body expression into
a future. The result is a future holding all completed values. Here is how we
collect responses for given codes:
@($/for [code [100 101 200 201 202 500]]
(get-json code))
[{...} {...} {...} {...} {...} {...}]
Should any body expression fail, the entire future fails as well. Use the
catch macro to handle an exception.
The macro supports special :let, :when and other options like the standart
for does:
@($/for [code [100 101 200 201 202 500]
:when (>= code 500)]
(get-json code))
[{:status_code 500}]
Loop & Recur
Sometimes you need a future that returns a future that returs... and so on until
something happens. This is where the loop macro helps. It reminds the standard
loop/recur combo but has the following features:
- it returns a future that is executed in the background;
- use a special
$/recurform but not the standardrecurfromclojure.core; - the body can produce a future;
- bindings can be futures as well.
The example below fetches JSON data one by one. Every time a future gets
completed, it performs the same block of code using bindings passed through the
$/recur form.
@($/loop [codes [100 101 200 201 202 500]
acc []]
(if-let [code (first codes)]
(let [result (get-json code)]
(println (format "result for status %d" code))
($/recur (next codes) (conj acc result)))
acc))
;; result for status 100
;; result for status 101
;; result for status 200
;; result for status 201
;; result for status 202
;; result for status 500
[{...} {...} ...]
The loop macro is used rarely with futures because other facilities are
usually enough. Loop is needed when you don't have an entire dataset at once,
and fetch it on the fly. A good example is pagination: you fetch data by chunks
and accumulate until the result is empty. Thus, you cannot run multiple futures
at once because you don't know for how long to proceed. This kind of fetching
can be expressed as follows:
(def PAGE_SIZE 100)
(-> ($/loop [acc []
off 0]
(let [result (fetch-items {:offset off :size PAGE_SIZE})
items (-> result :response :items)]
(if (seq items)
($/recur (into acc items) (+ off PAGE_SIZE))
acc)))
($/then [items]
(process-items items))
($/catch [e]
(log/errorf e "error: %s" 42)
(report-error e)))
Timeout & Cancelling
Any future can be limited in time by two strateges. First, it fails with a timeout exception, and it's up to you how to handle this. Second, you specify a default value for this future which comes into play on timeout.
The timeout macro has two bodies for both cases. The first form accepts a
future and a number of milliseconds. It assigns a timeout to a future. The
following example will fail because the sleep time is longer than the timeout:
@(-> ($/future
(Thread/sleep 1000))
($/timeout 100)
($/catch [e]
{:error (type e)}))
{:error java.util.concurrent.TimeoutException}
Pay attention that the TimeoutException instance has no message: the
(ex-message e) invocation will return nil.
The macro accepts an arbitrary block of code. The result becomes a value for a future when it breaches timeout:
@(-> ($/future
(Thread/sleep 1000))
($/timeout 100
(let [a 1 b 2]
(println "recovering from timeout")
{:some ["other" :value]}))
($/catch [e]
{:error (type e)})
($/then [x]
(println "final handler")
{:data x}))
;; recovering from timeout
;; final handler
;; {:data {:some ["other" :value]}}
Most likely you don't need to set timeouts explicitly: modern HTTP clients allow to specify socket timings when making calls. The same applies to any libraries working with sockets. But in rare cases, an explicit timeout might help.
Canceling is something opposite to a timeout. It is when you ask to reject a
future spawned previously. Canceling a completed future has no effect. But if it
has not been completed or failed before, a cancellation request completes a
future with CancellationException. Later or, such a future can be checked for
cancellation state with a predicate.
The cancel function tries to cancel a future. The result is a boolean value
meaning if the attempt was successful or not. Canceling a non-future value has
no effect, and the result will be nil:
(def -f ($/future 1))
($/cancel -f)
false ;; already completed
Let's try a slow future:
(def -f ($/future
(Thread/sleep 5000)
(println "DONE")))
($/cancel -f)
($/cancelled? -f)
true
@-f ;; throws
;; Execution error (CancellationException)
Pay attention that you will see the DONE line printed! This is because a
future was in the middle of processing, and the CompletableFuture class
doesn't interrupt calculation.
If you emit a cancellation request before a future has been started there won't be any background cancellation. The following tests proves it:
(let [p1 (promise)
p2 (promise)]
(with-open [e (Executors/newFixedThreadPool 1)]
(let [f1 ($/future-via [e]
(Thread/sleep 2000)
(println "DONE 1")
(deliver p1 true))
_ (Thread/sleep 500)
f2 ($/future-via [e]
(Thread/sleep 2000)
(println "DONE 2")
(deliver p2 true))]
(is ($/cancel f1))
(is ($/cancel f2))))
(is (realized? p1))
(is (not (realized? p2))))
Above, we have an executor with one thread only. We spawn a future f1 which
takes 2 seconds to complete, and wait for half of a second to let it start. The
second future f2 will stay in a queue until f1 is done. Then we cancel both
futures. The f1 accepts a cancellation request in the middle of processing. It
gets canceled although the work is done: you'll see the "DONE 1" printing and
the p1 promise will be delivered. But as f2 has not been started, it gets
removed from a queue of the executor. You won't see "DONE 2", nor the p2
promise will be delivered.
Misc
©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©
Ivan Grishaev, 2025. © UNLICENSE ©
©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©©
Can you improve this documentation?Edit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close