nl.jomco.spider
apply-templclj
(apply-templ val env)Apply template expressions in val string with given env.
Expression are place within { and } characters, parsed using
read-string and evaluated using evaluate and env.
Example:
(apply-templ "1 + {x} = {(+ 1 x)}" {'+ +, 'x 2}) ;; -> "1 + 2 = 3"
Apply template expressions in `val` string with given `env`.
Expression are place within `{` and `}` characters, parsed using
`read-string` and evaluated using `evaluate` and `env`.
Example:
(apply-templ "1 + {x} = {(+ 1 x)}" {'+ +, 'x 2})
;; -> "1 + 2 = 3"entriesclj
(entries coll)Return key => value pairs for each entry in coll. For vectors, returns index => value pairs.
Return key => value pairs for each entry in coll. For vectors, returns index => value pairs.
evaluateclj
(evaluate expr env)Evaluate parsed expr with given env.
Apart from the values and functions given in env, special
forms (if PRED TRUE-FN FALSE-FN), (and PRED-1 PRED-2 ..)
and (or PRED-1 PRED-2 ..) are also interpreted. Both and and
or return a boolean.
Examples:
(evaluate '(+ 1 2) {'+ +}) ;; => 3 (evaluate '(+ 1 2 x) {'+ +, 'x 3}) ;; => 6 (evaluate '(if (or (< 0 x 5) (< 10 x 15)) "yes" "no") {'< <, 'x 3}) ;; => "yes"
Evaluate parsed `expr` with given `env`.
Apart from the values and functions given in `env`, special
forms `(if PRED TRUE-FN FALSE-FN)`, `(and PRED-1 PRED-2 ..)`
and `(or PRED-1 PRED-2 ..)` are also interpreted. Both `and` and
`or` return a boolean.
Examples:
(evaluate '(+ 1 2) {'+ +})
;; => 3
(evaluate '(+ 1 2 x) {'+ +, 'x 3})
;; => 6
(evaluate '(if (or (< 0 x 5) (< 10 x 15)) "yes" "no") {'< <, 'x 3})
;; => "yes"generateclj
(generate templs matches env)Generate values from templs using matches (a result from
multi-select) and env (for evaluate).
Generate values from `templs` using `matches` (a result from `multi-select`) and `env` (for `evaluate`).
harvestclj
(harvest interaction rules base-url env)Returns a set of requests generated from the given
interaction (created by interact) and rules using select and
generate.
Returns a set of requests generated from the given `interaction` (created by `interact`) and `rules` using `select` and `generate`.
interactclj
(interact request)Make given HTTP request and return an interaction of format:
{:request {:method :get, :headers { .. }, :body { .. }} :response {:status 200, :headers { .. }, :body { .. }}
Make given HTTP `request` and return an interaction of format:
{:request {:method :get, :headers { .. }, :body { .. }}
:response {:status 200, :headers { .. }, :body { .. }}make-urlclj
(make-url base-url path)Navigate on base-url with path.
Navigate on `base-url` with `path`.
multi-selectclj
(multi-select data rules)Run rules (a collection of queries for select against data and
join the results.
Like select, returns a (possibly empty, for no matches) relation of bindings
Run rules (a collection of queries for `select` against `data` and join the results. Like select, returns a (possibly empty, for no matches) relation of bindings
selectclj
(select data query)Select query from data. A query is a list of terms navigating
into the data. Query terms which are symbols starting with a ?
character are considered to be placeholders.
Returns a clojure.set compatible relation result:
- Empty set when no matches are found
- #{{}} (a set containing an empty map) for an exact match
- A set of maps with bindings for every match with placeholders.
Bindings are unified; if a placeholder appears multiple times, the value for that placeholder must be the same for every appearance in a match.
Examples:
(select {:name "fred", :friends [{:name "barney"}]} '[:name ?name]) ;; => #{{?name "fred"}}
(select {:name "fred", :friends [{:name "barney" :name "dino"}]} '[:friends ?i :name ?name]) ;; => #{{?i 0, ?name "barney"} {?i 1, ?name "dino"}}
Select `query` from `data`. A query is a list of terms navigating
into the data. Query terms which are symbols starting with a \?
character are considered to be placeholders.
Returns a `clojure.set` compatible relation result:
- Empty set when no matches are found
- #{{}} (a set containing an empty map) for an exact match
- A set of maps with bindings for every match with placeholders.
Bindings are unified; if a placeholder appears multiple times, the
value for that placeholder must be the same for every appearance in
a match.
Examples:
(select {:name "fred",
:friends [{:name "barney"}]}
'[:name ?name])
;; => #{{?name "fred"}}
(select {:name "fred",
:friends [{:name "barney"
:name "dino"}]}
'[:friends ?i :name ?name])
;; => #{{?i 0, ?name "barney"}
{?i 1, ?name "dino"}}stepclj
(step {:keys [pool seen]}
rules
&
{:keys [interact env follow?]
:or {interact interact follow? any? env default-env}})Take a spider step with pool being a set of requests to consider
one of, seen the requests already probed, return a map containing
new versions of pool and seen, and the interaction just taken or
nil when no more requests left to consider.
Use option interact to provide an alternative to the interact
function, and option env to provide an alternative environment for
evaluate (see also default-env). If follow? option is
provided, only consider generated requests for which (follow? request) is true.
Take a spider step with `pool` being a set of requests to consider one of, `seen` the requests already probed, return a map containing new versions of `pool` and `seen`, and the interaction just taken or `nil` when no more requests left to consider. Use option `interact` to provide an alternative to the `interact` function, and option `env` to provide an alternative environment for `evaluate` (see also `default-env`). If `follow?` option is provided, only consider generated requests for which `(follow? request)` is true.
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close