Liking cljdoc? Tell your friends :D

com.climate.claypoole

Threadpool tools for Clojure. Claypoole provides parallel functions and macros that use threads from a pool and otherwise act like key builtins like future, pmap, for, and so on. See the file README.md for an introduction.

A threadpool is just an ExecutorService with a fixed number of threads. In general, you can use your own ExecutorService in place of any threadpool, and you can treat a threadpool as you would any other ExecutorService.

Threadpool tools for Clojure. Claypoole provides parallel functions and
macros that use threads from a pool and otherwise act like key builtins like
future, pmap, for, and so on. See the file README.md for an introduction.

A threadpool is just an ExecutorService with a fixed number of threads. In
general, you can use your own ExecutorService in place of any threadpool, and
you can treat a threadpool as you would any other ExecutorService.
raw docstring

*default-pmap-buffer*clj

This is an advanced configuration option. You probably don't need to set this!

When doing a pmap, Claypoole pushes input tasks into the threadpool. It normally tries to keep the threadpool full, plus it adds a buffer of size nthreads. If it can't find out the number of threads in the threadpool, it just tries to keep default-pmap-buffer tasks in the pool.

This is an advanced configuration option. You probably don't need to set
this!

When doing a pmap, Claypoole pushes input tasks into the threadpool. It
normally tries to keep the threadpool full, plus it adds a buffer of size
nthreads. If it can't find out the number of threads in the threadpool, it
just tries to keep *default-pmap-buffer* tasks in the pool.
sourceraw docstring

*parallel*clj

A dynamic binding to disable parallelism. If you do (binding [parallel false] body) then the body will have no parallelism. Disabling parallelism this way is handy for testing.

A dynamic binding to disable parallelism. If you do
  (binding [*parallel* false]
    body)
then the body will have no parallelism. Disabling parallelism this way is
handy for testing.
sourceraw docstring

completable-futurecljmacro

(completable-future pool & body)

Like clojure.core/future, but using a threadpool and returns a CompletableFuture.

The threadpool may be one of 3 things:

  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in agent threadpool, the same threadpool used by an ordinary clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in serial. This may be helpful during profiling, for example.
Like clojure.core/future, but using a threadpool and returns a CompletableFuture.

The threadpool may be one of 3 things:
  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in
     agent threadpool, the same threadpool used by an ordinary
     clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in
     serial. This may be helpful during profiling, for example.
sourceraw docstring

completable-future-callclj

(completable-future-call pool f)

Like clojure.core/future-call, but using a threadpool, and returns a CompletableFuture.

The threadpool may be one of 3 things:

  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in agent threadpool, the same threadpool used by an ordinary clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in serial. This may be helpful during profiling, for example.
Like clojure.core/future-call, but using a threadpool, and returns a CompletableFuture.

The threadpool may be one of 3 things:
1. An ExecutorService, e.g. one created by threadpool.
2. The keyword :builtin. In this case, the future will use the built-in
agent threadpool, the same threadpool used by an ordinary
clojure.core/future.
3. The keyword :serial. In this case, the computation will be performed in
serial. This may be helpful during profiling, for example.
sourceraw docstring

futurecljmacro

(future pool & body)

Like clojure.core/future, but using a threadpool.

The threadpool may be one of 3 things:

  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in agent threadpool, the same threadpool used by an ordinary clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in serial. This may be helpful during profiling, for example.
Like clojure.core/future, but using a threadpool.

The threadpool may be one of 3 things:
  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in
     agent threadpool, the same threadpool used by an ordinary
     clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in
     serial. This may be helpful during profiling, for example.
sourceraw docstring

future-callclj

(future-call pool f)

Like clojure.core/future-call, but using a threadpool.

The threadpool may be one of 3 things:

  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in agent threadpool, the same threadpool used by an ordinary clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in serial. This may be helpful during profiling, for example.
Like clojure.core/future-call, but using a threadpool.

The threadpool may be one of 3 things:
  1. An ExecutorService, e.g. one created by threadpool.
  2. The keyword :builtin. In this case, the future will use the built-in
     agent threadpool, the same threadpool used by an ordinary
     clojure.core/future.
  3. The keyword :serial. In this case, the computation will be performed in
     serial. This may be helpful during profiling, for example.
sourceraw docstring

ncpusclj

(ncpus)

Get the number of available CPUs.

Get the number of available CPUs.
sourceraw docstring

pcallsclj

(pcalls pool & fs)

Like clojure.core.pcalls, except it takes a threadpool. For more detail on its parallelism and on its threadpool argument, see pmap.

Like clojure.core.pcalls, except it takes a threadpool. For more detail on
its parallelism and on its threadpool argument, see pmap.
sourceraw docstring

pdoseqcljmacro

(pdoseq pool bindings & body)

Like doseq, but in parallel. Unlike the streaming sequence functions (e.g. pmap), pdoseq blocks until all the work is done.

Similar to pfor, only the body is done in parallel. For more details, see pfor.

Like doseq, but in parallel. Unlike the streaming sequence functions (e.g.
pmap), pdoseq blocks until all the work is done.

Similar to pfor, only the body is done in parallel. For more details, see
pfor.
sourceraw docstring

pforcljmacro

(pfor pool bindings & body)

A parallel version of for. It is like for, except it takes a threadpool and is parallel. For more detail on its parallelism and on its threadpool argument, see pmap.

Note that while the body is executed in parallel, the bindings are executed in serial, so while this will call complex-computation in parallel: (pfor pool [i (range 1000)] (complex-computation i)) this will not have useful parallelism: (pfor pool [i (range 1000) :let [result (complex-computation i)]] result)

You can use the special binding :priority (which must be the last binding) to set the priorities of the tasks. (upfor (priority-threadpool 10) [i (range 1000) :priority (inc i)] (complex-computation i))

A parallel version of for. It is like for, except it takes a threadpool and
is parallel. For more detail on its parallelism and on its threadpool
argument, see pmap.

Note that while the body is executed in parallel, the bindings are executed
in serial, so while this will call complex-computation in parallel:
    (pfor pool [i (range 1000)] (complex-computation i))
this will not have useful parallelism:
    (pfor pool [i (range 1000) :let [result (complex-computation i)]] result)

You can use the special binding :priority (which must be the last binding) to
set the priorities of the tasks.
    (upfor (priority-threadpool 10) [i (range 1000)
                                     :priority (inc i)]
      (complex-computation i))
sourceraw docstring

pmapclj

(pmap pool f & arg-seqs)

Like clojure.core.pmap, except:

  1. It is eager, returning an ordered sequence of completed results as they are available.
  2. It uses a threadpool to control the desired level of parallelism.

A word of caution: pmap will consume the entire input sequence and produce as much output as possible--if you pmap over (range) you'll get an Out of Memory Exception! Use this when you definitely want the work to be done.

The threadpool may be one of 4 things:

  1. An ExecutorService, e.g. one created by threadpool.
  2. An integer. In this case, a threadpool will be created, and it will be destroyed when all the pmap tasks are complete.
  3. The keyword :builtin. In this case, pmap will use the Clojure built-in agent threadpool. For pmap, that's probably not what you want, as you'll likely create a thread per task.
  4. The keyword :serial. In this case, the computations will be performed in serial via (doall map). This may be helpful during profiling, for example.
Like clojure.core.pmap, except:
  1. It is eager, returning an ordered sequence of completed results as they
     are available.
  2. It uses a threadpool to control the desired level of parallelism.

A word of caution: pmap will consume the entire input sequence and produce as
much output as possible--if you pmap over (range) you'll get an Out of Memory
Exception! Use this when you definitely want the work to be done.

The threadpool may be one of 4 things:
  1. An ExecutorService, e.g. one created by threadpool.
  2. An integer. In this case, a threadpool will be created, and it will be
     destroyed when all the pmap tasks are complete.
  3. The keyword :builtin. In this case, pmap will use the Clojure built-in
     agent threadpool. For pmap, that's probably not what you want, as you'll
     likely create a thread per task.
  4. The keyword :serial. In this case, the computations will be performed in
     serial via (doall map). This may be helpful during profiling, for example.
sourceraw docstring

priority-threadpoolclj

(priority-threadpool
  n
  &
  {:keys [default-priority] :as args :or {default-priority 0}})

Make a threadpool that chooses tasks based on their priorities.

Assign priorities to tasks by wrapping the pool with with-priority or with-priority-fn. You can also set a default priority with keyword argument :default-priority.

Otherwise, this uses the same keyword arguments as threadpool, and functions just like any other ExecutorService.

Make a threadpool that chooses tasks based on their priorities.

Assign priorities to tasks by wrapping the pool with with-priority or
with-priority-fn. You can also set a default priority with keyword argument
:default-priority.

Otherwise, this uses the same keyword arguments as threadpool, and functions
just like any other ExecutorService.
sourceraw docstring

priority-threadpool?clj

(priority-threadpool? pool)

Returns true iff the argument is a priority-threadpool.

Returns true iff the argument is a priority-threadpool.
sourceraw docstring

prun!clj

(prun! pool proc coll)

Like run!, but in parallel. Unlike the streaming sequence functions (e.g. pmap), prun! blocks until all the work is done.

Like run!, but in parallel. Unlike the streaming sequence functions (e.g.
pmap), prun! blocks until all the work is done.
sourceraw docstring

pvaluescljmacro

(pvalues pool & exprs)

Like clojure.core.pvalues, except it takes a threadpool. For more detail on its parallelism and on its threadpool argument, see pmap.

Like clojure.core.pvalues, except it takes a threadpool. For more detail on
its parallelism and on its threadpool argument, see pmap.
sourceraw docstring

serial?clj

(serial? pool)

Check if we should run computations on this threadpool in serial.

Check if we should run computations on this threadpool in serial.
sourceraw docstring

shutdownclj

(shutdown pool)

Syntactic sugar to stop a pool cleanly. This will stop the pool from accepting any new requests.

Syntactic sugar to stop a pool cleanly. This will stop the pool from
accepting any new requests.
sourceraw docstring

shutdown!clj

(shutdown! pool)

Syntactic sugar to forcibly shutdown a pool. This will kill any running threads in the pool!

Syntactic sugar to forcibly shutdown a pool. This will kill any running
threads in the pool!
sourceraw docstring

shutdown?clj

(shutdown? pool)

Syntactic sugar to test if a pool is shutdown.

Syntactic sugar to test if a pool is shutdown.
sourceraw docstring

thread-factoryclj

(thread-factory & {:keys [daemon thread-priority] pool-name :name :as args})

Create a ThreadFactory with keyword options including thread daemon status :daemon, the thread name format :name (a string for format with one integer), and a thread priority :thread-priority.

This is exposed as a public function because it's handy if you're instantiating your own ExecutorServices.

Create a ThreadFactory with keyword options including thread daemon status
:daemon, the thread name format :name (a string for format with one integer),
and a thread priority :thread-priority.

This is exposed as a public function because it's handy if you're
instantiating your own ExecutorServices.
sourceraw docstring

threadpoolclj

(threadpool n
            &
            {:keys [daemon thread-priority] pool-name :name :or {daemon true}})

Make a threadpool. It should be shutdown when no longer needed.

A threadpool is just an ExecutorService with a fixed number of threads. In general, you can use your own ExecutorService in place of any threadpool, and you can treat a threadpool as you would any other ExecutorService.

This takes optional keyword arguments: :daemon, a boolean indicating whether the threads are daemon threads, which will automatically die when the JVM exits, defaults to true) :name, a string giving the pool name, which will be the prefix of each thread name, resulting in threads named "name-0", "name-1", etc. Defaults to "claypoole-[pool-number]". :thread-priority, an integer in [Thread/MIN_PRIORITY, Thread/MAX_PRIORITY]. The effects of thread priority are system-dependent and should not be confused with Claypoole's priority threadpools that choose tasks based on a priority. For more info about Java thread priority see http://www.javamex.com/tutorials/threads/priority_what.shtml

Note: Returns a ScheduledExecutorService rather than just an ExecutorService because it's the same thing with a few bonus features.

Make a threadpool. It should be shutdown when no longer needed.

A threadpool is just an ExecutorService with a fixed number of threads. In
general, you can use your own ExecutorService in place of any threadpool, and
you can treat a threadpool as you would any other ExecutorService.

This takes optional keyword arguments:
  :daemon, a boolean indicating whether the threads are daemon threads,
           which will automatically die when the JVM exits, defaults to
           true)
  :name, a string giving the pool name, which will be the prefix of each
           thread name, resulting in threads named "name-0",
           "name-1", etc. Defaults to "claypoole-[pool-number]".
  :thread-priority, an integer in [Thread/MIN_PRIORITY, Thread/MAX_PRIORITY].
           The effects of thread priority are system-dependent and should not
           be confused with Claypoole's priority threadpools that choose
           tasks based on a priority. For more info about Java thread
           priority see
           http://www.javamex.com/tutorials/threads/priority_what.shtml

Note: Returns a ScheduledExecutorService rather than just an ExecutorService
because it's the same thing with a few bonus features.
sourceraw docstring

threadpool?clj

(threadpool? pool)

Returns true iff the argument is a threadpool.

Returns true iff the argument is a threadpool.
sourceraw docstring

upcallsclj

(upcalls pool & fs)

Like clojure.core.pcalls, except it takes a threadpool and returns results ordered by completion time. For more detail on its parallelism and on its threadpool argument, see upmap.

Like clojure.core.pcalls, except it takes a threadpool and returns results
ordered by completion time. For more detail on its parallelism and on its
threadpool argument, see upmap.
sourceraw docstring

upforcljmacro

(upfor pool bindings & body)

Like pfor, except the return value is a sequence of results ordered by completion time, not by input order.

Like pfor, except the return value is a sequence of results ordered by
*completion time*, not by input order.
sourceraw docstring

upmapclj

(upmap pool f & arg-seqs)

Like pmap, except that the return value is a sequence of results ordered by completion time, not by input order.

Like pmap, except that the return value is a sequence of results ordered by
*completion time*, not by input order.
sourceraw docstring

upvaluescljmacro

(upvalues pool & exprs)

Like clojure.core.pvalues, except it takes a threadpool and returns results ordered by completion time. For more detail on its parallelism and on its threadpool argument, see upmap.

Like clojure.core.pvalues, except it takes a threadpool and returns results
ordered by completion time. For more detail on its parallelism and on its
threadpool argument, see upmap.
sourceraw docstring

with-priorityclj

(with-priority pool priority)

Make a priority-threadpool wrapper with a given fixed priority.

All tasks run with this pool wrapper will have the given priority. e.g.

(def t1 (future (with-priority p 1) 1)) (def t2 (future (with-priority p 2) 2)) (def t3 (future (with-priority p 3) 3))

will use pool p to run these tasks with priorities 1, 2, and 3 respectively.

If you nest priorities, the outermost one "wins", so this task will be run at priority 3:

(def wp (with-priority p 1)) (def t1 (future (with-priority (with-priority wp 2) 3) :result))

Make a priority-threadpool wrapper with a given fixed priority.

All tasks run with this pool wrapper will have the given priority. e.g.

  (def t1 (future (with-priority p 1) 1))
  (def t2 (future (with-priority p 2) 2))
  (def t3 (future (with-priority p 3) 3))

will use pool p to run these tasks with priorities 1, 2, and 3 respectively.

If you nest priorities, the outermost one "wins", so this task will be run
at priority 3:

  (def wp (with-priority p 1))
  (def t1 (future (with-priority (with-priority wp 2) 3) :result))
sourceraw docstring

with-priority-fnclj

(with-priority-fn pool priority-fn)

Make a priority-threadpool wrapper that uses a given priority function.

The priority function is applied to a pmap'd function's arguments. e.g.

(upmap (with-priority-fn pool (fn [x _] x)) + [6 5 4] [1 2 3])

will use pool to run tasks [(+ 6 1) (+ 5 2) (+ 4 3)] with priorities [6 5 4].

Make a priority-threadpool wrapper that uses a given priority function.

The priority function is applied to a pmap'd function's arguments. e.g.

  (upmap (with-priority-fn pool (fn [x _] x))
         + [6 5 4] [1 2 3])

will use pool to run tasks [(+ 6 1) (+ 5 2) (+ 4 3)]
with priorities [6 5 4].
sourceraw docstring

with-shutdown!cljmacro

(with-shutdown! pool-syms-and-inits & body)

Lets a threadpool from an initializer, then evaluates body in a try expression, calling shutdown! on the threadpool to forcibly shut it down at the end.

The threadpool initializer may be a threadpool. Alternately, it can be any threadpool argument accepted by pmap, e.g. a number, :builtin, or :serial, in which case it will create a threadpool just as pmap would.

Be aware that any unfinished jobs at the end of the body will be killed!

Examples:

(with-shutdown! [pool (threadpool 6)]
  (doall (pmap pool identity (range 1000))))
(with-shutdown! [pool1 6
                 pool2 :serial]
  (doall (pmap pool1 identity (range 1000))))

Bad example:

(with-shutdown! [pool 6]
  ;; Some of these tasks may be killed!
  (pmap pool identity (range 1000)))
Lets a threadpool from an initializer, then evaluates body in a try
expression, calling shutdown! on the threadpool to forcibly shut it down at
the end.

The threadpool initializer may be a threadpool. Alternately, it can be any
threadpool argument accepted by pmap, e.g. a number, :builtin, or :serial, in
which case it will create a threadpool just as pmap would.

Be aware that any unfinished jobs at the end of the body will be killed!

Examples:

    (with-shutdown! [pool (threadpool 6)]
      (doall (pmap pool identity (range 1000))))
    (with-shutdown! [pool1 6
                     pool2 :serial]
      (doall (pmap pool1 identity (range 1000))))

Bad example:

    (with-shutdown! [pool 6]
      ;; Some of these tasks may be killed!
      (pmap pool identity (range 1000)))
sourceraw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close