cascalog.cascading.flow — cascalog/cascalog-core 3.0.0

*stats-fn* clj

all-to-memory clj

(all-to-memory & args)

Return the results of the supplied workflows as data structures. Accepts many workflows, and (optionally) a flow name as the first argument.

assert-success! clj

(assert-success! sm)

Inputs: [sm :- stats/StatsMap]

compile-flow clj

(compile-flow & args)

Inputs: [& args] Returns: ClojureFlow

Attaches output taps to some number of subqueries and creates a Cascading flow. The flow can be executed with .complete, or introspection can be done on the flow.

Syntax: (compile-flow sink1 query1 sink2 query2 ...) or (compile-flow flow-name sink1 query1 sink2 query2)

If the first argument is a string, that will be used as the name for the query and will show up in the JobTracker UI.

compile-hadoop clj

(compile-hadoop fd)

Inputs: [fd :- FlowDef] Returns: HadoopFlow

Compiles the supplied FlowDef into a Hadoop flow.

flow-def clj

(flow-def {:keys [source-map sink-map trap-map tails name]})

Inputs: [{:keys [source-map sink-map trap-map tails name]} :- ClojureFlow] Returns: FlowDef

Generates an instance of FlowDef off of the supplied ClojureFlow.

graph clj

(graph flow path)

Inputs: [flow :- ClojureFlow path :- s/Str] Returns: ClojureFlow

Writes a dotfile for the flow at hand to the supplied path.

handle-stats! clj

(handle-stats! sm)

Inputs: [sm :- stats/StatsMap] Returns: stats/StatsMap

If the stats function is bound, passes the supplied stats map in and returns it unchanged.

IRunnable clj protocol

All runnable items should implement this function.

run! clj

(run! x)

Runs the flow and returns an instance of cascalog.cascading.stats/StatsMap.

jcompile-flow clj

(jcompile-flow & args)

Inputs: [& args] Returns: HadoopFlow

jflow-def clj

(jflow-def & args)

Inputs: [& args] Returns: FlowDef

to-memory clj

(to-memory m)

Executes the supplied flow and returns the results as a sequence of tuples.

with-stats clj macro

(with-stats f & body)

Executes the supplied body with the supplied stats fn.