mx.interware.arp.streams.stateful — mx.interware/caudal 0.7.1

acum-stats clj

(acum-stats state-key avg-key variance-key stdev-key count-key & children)

Stream function that acumulates stats (mean,variance,stdev,count) recomputing them, it receives an event that is decorated with mean,stdev,variance and count and takes information from the state and the resultin mixing is stored in the state, then the event with the new values is propagated to children

Arguments:
state-key: Key name to store accumulating mean,stdev,variance and count avg-key: Key where the mean is in the event and in the state
variance-key: Key where the variance is in the event and in the state
stdev-key: Key where the stdev is in the event and in the state
count-key: Key where the count is in the event and in the state
children: List of streams to propagate result
-- if event is missign required fields, the event is propagated with :error set to message error

batch clj

(batch state-key n dt & children)

Streamer function that propagates a vector of events once n elements are stored in or dt time has passed

Arguments:
state-key: Key name to store
n: Max number of elements stored
dt: Max time to propagate
children: Children streamer functions to be propagated

changed clj

(changed state-key change-pred & children)

Streamer function that checks if a value of an event has changed using a predicate

Arguments:
state-key: State key to store
change-pred: predicate for evaluating
children: Children streamer functions to be propagated

concurrent-meter clj

(concurrent-meter state-key
                  init-tx-pred
                  end-tx-pred
                  tx-id-fn
                  &
                  [concurrent-fld-cntr & children :as all-children])

Stream function that stores in event the current number of transactions opened
when an initial event passes, the counter increments, if an end event paseses, the counter decrements

Arguments:
state-key: State key name to store the counter
init-tx-pred: Predicate to detect the initial event of an operation
init-tx-end: Predicate to detect the finish event of an operation
*tx-id-fn: Function to get the transaction id
*[]: list that contains:

• event key name to store the meter
• children functions to propagate

counter clj

(counter state-key event-counter-key & children)

Streamer function that appends to an event the count of passed events

Arguments:
state-key: State key to store the count
event-counter-key: Event key to hold the count
children: Children streamer functions to be propagated

dump-every clj

(dump-every state-key file-name-prefix date-format cycle dir-path)

Stream function that writes to the file system information stored y the state.

Arguments:
state-key: Key of the state to write to the file system
file-name-prefix, date-format and dir-path: are used to copute the file name like this:

• Arquimedes take the JVM time and formats it using a java.text.SimpleDateFormat with date-format and
• concatenates it with '-' and file-name-prefix, then appends to it the current 'by string list' then
• appends '.edn' the file will be located at the dir-path directory.
  cycle: represents a long number for milliseconds or a vetor with a number ant time messure like:
• [5 :minutes] or [1 :day]
• cycle will thel arquimedes to write this file width this frequency posibly overwriting it.

ewma-timeless clj

(ewma-timeless state-key r & [metric-key & children :as all-children])

Streamer function that normalizes the metric value, it calculates the ewma which represents the exponential waited moving average and is calculated by the next equation:
(last-metric * r) + (metric-history * (1-r)), events with no metric will be ignored 

Arguments:
state-key: state key to store the metric
r: Ratio value
(metric-key): Optional for key holding metric, default :metric children the streamer functions to propagate

log-matcher clj

(log-matcher state-key
             init-pred
             end-pred
             tx-id-fn
             timeout-delta
             &
             [ts-key & children :as all-children])

Stream function that correlates initial and end events, extract the metric value and calculates the metric difference

Arguments:
state-key: Key name to store the time
init-tx-pred: Predicate to detect the initial event of an operation
init-tx-end: Predicate to detect the finish event of an operation
tx-id-fn: Function to get the transaction id
timeout-delta: Time to wait the end event in milliseconds, when timeout, it will discard the event
[]:List that contains the key name to store the difference miliseconds and childs to propagate

mixer clj

(mixer state-key delay ts-key priority-fn & children)

moving-time-window clj

(moving-time-window state-key n & [time-key & children :as all-children])

Streamer function that propagate events to a vector of streamers using a moving time window delay

Arguments:
state-key: State key
n: Max number of events
(time-key): Optional key holding the time i millis, default :time & children: functions to propagate

rate clj

(rate state-key ts-key rate-key days bucket-size & children)

Stream function that matains a matrix fo days rows and 60*24 columns, and stores it in the state, then for each event that passes through increments the number in the position of row 0 and collum equals to the number of minute in the day, remainder rows contain past days minus 1 (days shold be less than 60 '2 months').

Arguments:
state-key: Key of the state to hold the matrix
ts-key: keyword of the timestamp in millis
rate-key: keyword holding the matrix when the outgoing event is propagated to children
days: Number of days to hold (number of rows in the matrix) minutes2group: number of minutes to group, larger numbers require less memory (ex: 15 means we know how many events are dividing the hour in 4 (15 minutes) it is required that this number divides 60 with no fraction(MEJORAR).
children: List of streams to propagate result
-- if event dosent have 'ts-key' event is ignored!!

reduce-with clj

(reduce-with state-key fun & children)

Streamer function that reduces values of an event using a function and propagates it

Arguments:
state-key: Key name to reduce
fun: function that reduces the values
children: Children streamer functions to be propagated

rollup clj

(rollup state-key n dt & children)

Streamer function that propagates n events each dt time

Arguments:
state-key: Key name to roll back
n: Max number of elements stored
dt: Max time to perform operations children: Children streamer functions to be propagated