ablauf.job — exoscale/ablauf 0.3.6

An abstract program runtime for ASTs produced by ablauf.job.ast. This exposes two functions for the outside world: make and restart

You are expected to create a program to be ran with the functions available in the ablauf.job.ast namespace. This program is then initialized with the rough equivalent of an instruction pointer and a context.

This namespace makes no guess as to how program instructions are actually performed, instead, for each position of the program, it yields the instructions to carry out. There might be more than one if parallelism is used.

The logical flow is thus:

• call restart on a program with an empty result list
• carry out resulting actions
• call restart on the result of the actions

For an actual full fledged program runner, a runner using manifold as the underlying dispatching engine is provided in ablauf.job.manifold, an SQL backed one in ablauf.job.sql.

abort clj

(abort job)

(abort job reason)

Given a job, mark pending all pending leafs as failure with the proper reason. Yields an updated job.

ast-zip clj

(ast-zip ast)

Given a well-formed AST (as per ablauf.job.ast), yield a zipper

augment clj

(augment context {:ast/keys [augment] :exec/keys [result output]})

When an AST node contains an augment key, process it to augment the resulting context. Augments have a source: a function of the output, a keyword or vector, of keyword pointing to a path in the output. Augments also have a destination a key or key vector of the position in which to augment the context.

augmentable? clj

(augmentable? {:ast/keys [augment] :as node})

Predicate to test for an augmentable node

done? clj

(done? job)

Predicate to test for completion of a (sub)job Here is the per-status breakdown:

status	done?
:job/failure	Y
:job/success	Y
:job/pending	N

eligible? clj

(eligible? job)

Predicate to test for pending completion of a (sub)job. Here is the per-status breakdown:

status	eligible?
:job/failure	N
:job/success	N
:job/pending	Y

failed? clj

(failed? job)

Predicate to test for failure of a (sub)job Here is the per-status breakdown:

status	failed?
:job/failure	Y
:job/success	N
:job/pending	N

find-dispatchs clj

(find-dispatchs job)

index-ast clj

(index-ast pos)

Uniquely identifies job nodes, for later merging

make clj

(make ast)

Creates a job, suitable for restart from a valid AST as produced by functions in ablauf.job.ast

make-with-context clj

(make-with-context ast context)

Creates a job, attaching an initial context map, as for make, this creates a tuple suitable for restart

merge-dispatchs clj

(merge-dispatchs job nodes)

merge-results clj

(merge-results job context nodes)

Updates a job given a list of node updates. Node updates either come from an action dispatch return, or from newly found dispatchs.

pending? clj

(pending? job)

Predicate to test for pending completion of a (sub)job Here is the per-status breakdown:

status	pending?
:job/failure	N
:job/success	N
:job/pending	Y

prepare-replay clj

(prepare-replay job)

It will traverse the whole job and:

1. mark as unstarted all (pending or failed) idempotent leafs
2. mark as failure all pending non idempotent leafs

Yields the modified job

prune clj

(prune ast)

Remove empties :ast/nodes leaves

reload clj

(reload ast context)

Yield a job ready for restart from a data ast and an optional context.

remove-nodes-by clj

(remove-nodes-by job f)

Removes any node that returns true for (f node). Can be a leaf or not.

restart clj

(restart [job context] results)

Given a job, and node updates for it, figure out the next course of action to take.

Yields an updated job and potential side-effects. The result has the following structure:

[job context dispatchs]

This allows using the return of restart as an accumulator for reductions or similar functions.

status clj

(status ast)

Get the job status from an ast.

Status predicates breakdown: | status | done? | failed? | eligible? | pending? | |--------|---------|-----------|-------------|------------| | :job/failure | Y | Y | N | N | | :job/success | Y | N | N | N | | :job/pending | N | N | Y | Y |

unzip clj

(unzip zipper)

Get back to a data AST from a program's zipper