Liking cljdoc? Tell your friends :D


Some sokka terminology:


Task is the basic unit of work. Here is a quick example of what a task looks like:

{:task-id  "7p9767c9hch7e6kt18nw9iwb1"
 :task-group-id "5a978"
 :task-description "daily products data sync - 10/05/21"
 :topic "products-sync"
 :sub-topic "default"
 :timeout-ms 10000
 :status :starting
 :data {
    :operation :sync-products
    :include :all
    :batch-size 100
task-idUnique id for a task
task-group-idTasks can be grouped together using a task-group-id.
task-descriptionFriendly description of the task.
topicLogical grouping of tasks. Tasks that belong to a topic will be processed in FIFO order (not strictly).
sub-topicFurther classification to help organize and list tasks better.
statusCurrent status of the task. valid statuses are :starting, :running, :snoozed, :failed and :terminted.
timeout-msTime in ms after which an executing task would be cancelled and marked as failed
dataThe task definition that describes what needs to be done. The value must be a valid edn.


TaskQ is a durable queue that implements the [[verbo.sokka.TaskQ]] protocol. Tasks are organized by topic and sub-topic.


A worker is a thread that reserves (dequeues) tasks from a topic and executes it. The worker is responsible for managing the life-cycle of a task.

A typical life-cycle is:

  • Process: reserve - execute - update status (terminate/fail/snooze)
  • Keepalive: periodically extend the lease of the task.


Reservation is the process of acquiring a lock/lease on a task. The actual leasing mechanism will depend on the TaskQ implementation. For eg., the default DynamoDB implementation uses leases (time based locks).

Extending Lease (Keepalive)

A worker must periodically 'extend' the lease of the task that it has reserved. Lease extension usually comprises of the following steps:

  • Check that the lease held is still valid. If the lease was revoked by a supervisor elsewhere, abort the current execution.
  • Update the lease for the configured lease-time.

Lease Supervision

Locking implementations in distributed systems are prone to issues caused by clock skew. A process with skewed time can cause the active and valid lease held by other processes to be revoked. This can cause thrashing of tasks and results in the same task being attempted to run multiple times wasting system resources and delaying sucessful task execution.

Sokka provides an out of the box solution that solves the clock skew issue for implementations that use time based locks (see: verbo.sokka.supervisor). The solution is to maintain an in memory list of tasks with active lease and a projected expiry time based on the current time of the process/instance. If the task still has the same record version after the expiry, it is considered stale and its lease is forcefully revoked freeing it up to be reserved again.

Can you improve this documentation?Edit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close