Blat

In Yiddish, בלאַט means page.

Definition

Leverage pagination APIs with core.async.

Documentation

This library is comprised of one function called fetch.

Usage example

Example I

Given a function that knows how to operate asynchronously on a page in a pagination API, for example, the tMDB API:

(require '[clojure.core.async :as a]
	 '[clj-http.client :as client])

(defn movies [query page c]
  (client/get (str api-endpoint "/search/movie") {:query-params {"api_key" api-key
								 "query" query
								 "page" page}
						  :async? true
						  :as :json}
	      (fn [resp] (a/onto-chan c (:results (:body resp))))
	      (fn [e] (a/>!! c (.getMessage e))))

It is then possible to quickly operate on any number of pages.

(require ' [blet.core :refer [fetch]])
(fetch (partial movies "love") 1 50)

Note: start is inclusive, while end is exclusive, like Clojure's range function.

Example II

Given a function that knows how to operate both synchronously and asynchronously on a page in a pagination API, for example, the tMDB API:

(require '[clojure.core.async :as a]
	 '[clj-http.client :as client])

(defn movies
  ([query page]
   (-> (client/get (str api-endpoint "/search/movie") {:query-params {"api_key" api-key
								     "query" query
								     "page" page}
						      :as :json})
      :body))
  ([query page c]
  (client/get (str api-endpoint "/search/movie") {:query-params {"api_key" api-key
								 "query" query
								 "page" page}
						  :async? true
						  :as :json}
	      (fn [resp] (a/onto-chan c (:results (:body resp))))
	      (fn [e] (a/>!! c (.getMessage e))))))

You can then use the blat library to retrieve all pages concurrently without knowing beforehand how many pages of results are available, leveraging the API instead.

(require ' [blet.core :refer [fetch]])
(defn fetch-all [query]
  (let [f (partial movies query)
	{results :results total :total_pages} (f 1)]
    (fetch f 2 (inc total) results)))

The first use of find is synchronous. It is called with two arguments, query and the first page of results. The results are destructured to get the number of total pages and the initial results. The second use of find is asynchronous via fetch, which internally calls it with an arity of 3, the third argument being a channel.