clementine.core — clementine 0.0.1

Lightweight web crawler with an aritrary handler.

crawl clj

(crawl config)

fetch-dom clj

(fetch-dom url)

Retrieves DOM at given url

fetch-urls clj

(fetch-urls url link-selector)

Fetches urls on page

go-crawl clj

(go-crawl url handler link-selector max-depth visiting visited)

Passes all reachable urls from url to handler fn

http-opts clj

is-http clj

(is-http url)

Returns true iff url is valid and uses http

resolve-path clj

(resolve-path url other)