Liking cljdoc? Tell your friends :D
Clojure only.

pegasus.defaults

Contains default components

Contains default components
raw docstring

add-location-configclj

(add-location-config user-config)

Given a user-config, returns a new config with location info added to it

Given a user-config, returns a new config
with location info added to it
sourceraw docstring

add-state-portionsclj

(add-state-portions config)
source

add-structs-configclj

(add-structs-config user-config)
source

build-pipeline-configclj

(build-pipeline-config user-config)
source

close-wrtrclj

(close-wrtr config)
source

config-logsclj

(config-logs config)
source

default-filterclj

(default-filter obj config)

By default, we ignore robots.txt urls

By default, we ignore robots.txt urls
sourceraw docstring

default-location-configclj

source

default-pipeline-configclj

source

default-stop-checkclj

(default-stop-check obj config)

Stops at 100 documents. This function is part of the crawl pipeline but doesn't use any of the objects being passed through it; it only looks at the crawl-state.

Stops at 100 documents. This
function is part of the crawl
pipeline but doesn't use any of the objects being
passed through it; it only looks at
the crawl-state.
sourceraw docstring

default-visited-checkclj

(default-visited-check obj queue visited)
source

enforce-pipeline-checkclj

(enforce-pipeline-check a-config)
source

get-requestclj

(get-request url user-agent)
(get-request url user-agent socket-timeout conn-timeout)
source

mark-stopclj

(mark-stop config)
source

not-enqueuedclj

(not-enqueued a-uri config)
source

not-visitedclj

(not-visited a-uri config)
source

robots-filterclj

(robots-filter a-uri config)
source

zero-enqueued?clj

(zero-enqueued? q)
source

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
←Move to previous article
→Move to next article
Ctrl+/Jump to the search field
Raise an issueBrowse cljdoc sourceChat on Slack
× close