com.climate.claypoole.lazy — org.clj-commons/claypoole 1.2.2

Lazy threadpool tools for Clojure.

This namespace provides lazy versions of the parallel functions from com.climate.claypoole. They will only run the outputs you realize, plus a few more (a buffer), just like core pmap. The buffer is used to help keep the threadpool busy.

Each parallel function also comes with a -buffer variant that allows you to specify the buffer size. The non -buffer forms use the threadpool size as their buffer size.

To use the threadpool most efficiently with these lazy functions, prefer the unordered versions (e.g. upmap), since the ordered ones may starve the threadpool of work. For instance, this pmap will take 6 milliseconds to run: (lazy/pmap 2 #(Thread/sleep %) [4 3 2 1]) That's because it will not realize the 2 task until the 4 task is complete, so one thread in the pool will sit idle for 1 millisecond.

pcalls clj

(pcalls pool & fs)

Like clojure.core.pcalls, except it's lazy and it takes a threadpool. For more detail on its parallelism and on its threadpool argument, see pmap.

pcalls-buffer clj

(pcalls-buffer pool buffer & fs)

Like clojure.core.pcalls, except it's lazy and it takes a threadpool and a buffer size. For more detail on these arguments, see pmap-buffer.

pdoseq clj macro

(pdoseq pool bindings & body)

Like doseq, but in parallel. Unlike the streaming sequence functions (e.g. pmap), pdoseq blocks until all the work is done.

Similar to pfor, only the body is done in parallel. For more details, see pfor.

pfor clj macro

(pfor pool bindings & body)

A parallel version of for. It is like for, except it takes a threadpool and is parallel. For more detail on its parallelism and on its threadpool argument, see pmap.

Note that while the body is executed in parallel, the bindings are executed in serial, so while this will call complex-computation in parallel: (pfor pool [i (range 1000)] (complex-computation i)) this will not have useful parallelism: (pfor pool [i (range 1000) :let [result (complex-computation i)]] result)

You can use the special binding :priority (which must be the last binding) to set the priorities of the tasks. (upfor (priority-threadpool 10) [i (range 1000) :priority (inc i)] (complex-computation i))

pfor-buffer clj macro

(pfor-buffer pool buffer bindings & body)

Like pfor, but it takes a buffer size; see pmap-buffer for information about this argument.

pmap clj

(pmap pool f & colls)

A lazy pmap where the work happens in a threadpool, just like core pmap, but using Claypoole futures in the given threadpool.

Unlike core pmap, it doesn't assume the buffer size is nprocessors + 2; instead, it tries to fill the pool.

If you (doall (take 2 (pmap 10 inc (range 1000)))) then 12 inputs and outputs will be realized--the 2 you asked for, plus the 10 that are run in the buffer to keep the threadpool busy.

pmap-buffer clj

(pmap-buffer pool buffer-size f & colls)

A lazy pmap where the work happens in a threadpool, just like core pmap, but using Claypoole futures.

Unlike core pmap, it doesn't assume the buffer size is nprocessors + 2; instead, you must specify how many tasks ahead will be run in the background.

If you (doall (take 2 (pmap-buffer pool 10 inc (range 1000)))) then 12 inputs and outputs will be realized--the 2 you asked for, plus the 10 that are run in the buffer to keep the threadpool busy.

prun! clj

(prun! pool proc coll)

Like run!, but in parallel. Unlike the streaming sequence functions (e.g. pmap), prun! blocks until all the work is done.

pvalues clj macro

(pvalues pool & exprs)

Like clojure.core.pvalues, except it's lazy and it takes a threadpool. For more detail on its parallelism and on its threadpool argument, see pmap.

pvalues-buffer clj macro

(pvalues-buffer pool buffer & exprs)

Like clojure.core.pvalues, except it's lazy and it takes a threadpool and a buffer size. For more detail on its parallelism and on its arguments, see pmap-buffer.

upcalls clj

(upcalls pool & fs)

Like clojure.core.pcalls, except it's lazy, it takes a threadpool, and it returns results ordered by completion time. For more detail on its parallelism and on its threadpool argument, see upmap.

upcalls-buffer clj

(upcalls-buffer pool buffer & fs)

Like clojure.core.pcalls, except it's lazy, it takes a threadpool and a buffer size, and it returns results ordered by completion time. For more detail on its parallelism and on its arguments, see upmap-buffer.

upfor clj macro

(upfor pool bindings & body)

Like pfor, except the return value is a sequence of results ordered by completion time, not by input order.

upfor-buffer clj macro

(upfor-buffer pool buffer bindings & body)

Like upfor, but it takes a buffer size; see pmap-buffer for information about this argument.

upmap clj

(upmap pool f & colls)

Like pmap, but with results returned in the order they completed.

Note that unlike core pmap, it doesn't assume the buffer size is nprocessors

• 2; instead, it tries to fill the pool.

upmap-buffer clj

(upmap-buffer pool buffer-size f & colls)

Like pmap-buffer, but with results returned in the order they completed.

Note that unlike core pmap, it doesn't assume the buffer size is nprocessors

• 2; instead, you must specify how many tasks ahead will be run in the background.

upvalues clj macro

(upvalues pool & exprs)

Like clojure.core.pvalues, except it's lazy, it takes a threadpool, and it returns results ordered by completion time. For more detail on its parallelism and on its threadpool argument, see upmap.

upvalues-buffer clj macro

(upvalues-buffer pool buffer & exprs)

Like clojure.core.pvalues, except it's lazy, it takes a threadpool and a buffer size, and it returns results ordered by completion time. For more detail on its parallelism and on its arguments, see upmap-buffer.