Liking cljdoc? Tell your friends :D

nl.jomco.spider


apply-templclj

(apply-templ val env)

Apply template expressions in val string with given env. Expression are place within { and } characters, parsed using read-string and evaluated using evaluate and env.

Example:

(apply-templ "1 + {x} = {(+ 1 x)}" {'+ +, 'x 2}) ;; -> "1 + 2 = 3"

Apply template expressions in `val` string with given `env`.
Expression are place within `{` and `}` characters, parsed using
`read-string` and evaluated using `evaluate` and `env`.

Example:

  (apply-templ "1 + {x} = {(+ 1 x)}" {'+ +, 'x 2})
  ;; -> "1 + 2 = 3"
sourceraw docstring

clientclj

source

default-envclj

source

entriesclj

(entries coll)

Return key => value pairs for each entry in coll. For vectors, returns index => value pairs.

Return key => value pairs for each entry in coll. For vectors,
returns index => value pairs.
sourceraw docstring

evaluateclj

(evaluate expr env)

Evaluate parsed expr with given env.

Apart from the values and functions given in env, special forms (if PRED TRUE-FN FALSE-FN), (and PRED-1 PRED-2 ..) and (or PRED-1 PRED-2 ..) are also interpreted. Both and and or return a boolean.

Examples:

(evaluate '(+ 1 2) {'+ +}) ;; => 3 (evaluate '(+ 1 2 x) {'+ +, 'x 3}) ;; => 6 (evaluate '(if (or (< 0 x 5) (< 10 x 15)) "yes" "no") {'< <, 'x 3}) ;; => "yes"

Evaluate parsed `expr` with given `env`.

Apart from the values and functions given in `env`, special
forms `(if PRED TRUE-FN FALSE-FN)`, `(and PRED-1 PRED-2 ..)`
and `(or PRED-1 PRED-2 ..)` are also interpreted.  Both `and` and
`or` return a boolean.

Examples:

  (evaluate '(+ 1 2) {'+ +})
  ;; => 3
  (evaluate '(+ 1 2 x) {'+ +, 'x 3})
  ;; => 6
  (evaluate '(if (or (< 0 x 5) (< 10 x 15)) "yes" "no") {'< <, 'x 3})
  ;; => "yes"
sourceraw docstring

generateclj

(generate templs matches env)

Generate values from templs using matches (a result from multi-select) and env (for evaluate).

Generate values from `templs` using `matches` (a result from
`multi-select`) and `env` (for `evaluate`).
sourceraw docstring

harvestclj

(harvest interaction rules base-url env)

Returns a set of requests generated from the given interaction (created by interact) and rules using select and generate.

Returns a set of requests generated from the given
`interaction` (created by `interact`) and `rules` using `select` and
`generate`.
sourceraw docstring

interactclj

(interact request)

Make given HTTP request and return an interaction of format:

{:request {:method :get, :headers { .. }, :body { .. }} :response {:status 200, :headers { .. }, :body { .. }}

Make given HTTP `request` and return an interaction of format:

{:request {:method :get, :headers { .. }, :body { .. }}
 :response {:status 200, :headers { .. }, :body { .. }}
sourceraw docstring

make-urlclj

(make-url base-url path)

Navigate on base-url with path.

Navigate on `base-url` with `path`.
sourceraw docstring

multi-selectclj

(multi-select data rules)

Run rules (a collection of queries for select against data and join the results.

Like select, returns a (possibly empty, for no matches) relation of bindings

Run rules (a collection of queries for `select` against `data` and
join the results.

Like select, returns a (possibly empty, for no matches) relation of
bindings
sourceraw docstring

placeholder?clj

(placeholder? x)
source

selectclj

(select data query)

Select query from data. A query is a list of terms navigating into the data. Query terms which are symbols starting with a ? character are considered to be placeholders.

Returns a clojure.set compatible relation result:

  • Empty set when no matches are found
  • #{{}} (a set containing an empty map) for an exact match
  • A set of maps with bindings for every match with placeholders.

Bindings are unified; if a placeholder appears multiple times, the value for that placeholder must be the same for every appearance in a match.

Examples:

(select {:name "fred", :friends [{:name "barney"}]} '[:name ?name]) ;; => #{{?name "fred"}}

(select {:name "fred", :friends [{:name "barney" :name "dino"}]} '[:friends ?i :name ?name]) ;; => #{{?i 0, ?name "barney"} {?i 1, ?name "dino"}}

Select `query` from `data`.  A query is a list of terms navigating
into the data.  Query terms which are symbols starting with a \?
character are considered to be placeholders.

Returns a `clojure.set` compatible relation result:

- Empty set when no matches are found
- #{{}} (a set containing an empty map) for an exact match
- A set of maps with bindings for every match with placeholders.

Bindings are unified; if a placeholder appears multiple times, the
value for that placeholder must be the same for every appearance in
a match.

Examples:

  (select {:name "fred",
           :friends [{:name "barney"}]}
          '[:name ?name])
  ;; => #{{?name "fred"}}

  (select {:name "fred",
           :friends [{:name "barney"
                      :name "dino"}]}
          '[:friends ?i :name ?name])
  ;; => #{{?i 0, ?name "barney"}
          {?i 1, ?name "dino"}}
sourceraw docstring

stepclj

(step {:keys [pool seen]}
      rules
      &
      {:keys [interact env follow?]
       :or {interact interact follow? any? env default-env}})

Take a spider step with pool being a set of requests to consider one of, seen the requests already probed, return a map containing new versions of pool and seen, and the interaction just taken or nil when no more requests left to consider.

Use option interact to provide an alternative to the interact function, and option env to provide an alternative environment for evaluate (see also default-env). If follow? option is provided, only consider generated requests for which (follow? request) is true.

Take a spider step with `pool` being a set of requests to consider
one of, `seen` the requests already probed, return a map containing
new versions of `pool` and `seen`, and the interaction just taken or
`nil` when no more requests left to consider.

Use option `interact` to provide an alternative to the `interact`
function, and option `env` to provide an alternative environment for
`evaluate` (see also `default-env`).  If `follow?` option is
provided, only consider generated requests for which `(follow?
request)` is true.
sourceraw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close