with-http

A powerful macro to stub HTTP calls with a local Jetty server. Declarative, flexible, and extremely useful. I've been copying it through many projects, and now it's time to ship it as a standalone library.

ToC

• Installation
• About
• The App routes
• Default handler
• Basic test
• JSON
• Slow responses
• Files & Resources
• Capturing requests
• License

Installation

Lein:

[com.github.igrishaev/with-http "0.1.0"]

Deps.edn

{com.github.igrishaev/with-http {:mvn/version "0.1.0"}}

Pay attention: since the library is primarily used for tests, put the dependency in the corresponding profile or alias. Storing it in global dependencies is not a good idea as it becomes a part of the production code otherwise.

About

The library provides a with-http macro of the following form:

(with-http [port app]
  ...body)

The port is the number (1..65535) and the app is a map of routes. When entering the macro, it spawns a local Jetty server on that port in the background. The app map tells the server how to respond to calls.

Now that you have a running server, point your HTTP API clients to http://localhost:<port> to imitate real network interaction. For example, for prod, a third-party base URL is https://api.some.cool.service but for tests, it is http://localhost:8088. This can be done using environment variables or the Aero library.

Why not use with-redefs, would you ask? Well, although with-redefs looks like a solution at first glance, it's questionable. Using with-redefs means lying to yourself. You temporarily mute some pieces of the codebase pretending it's OK, but it's not.

Often, bugs lurk in the code that you actually substitute using with-redefs, namely:

• you've messed up with MD5/SHA/etc algorithms to sign a request. Calling localhost would trigger that code and lead to an exception, but with-redefs would not.

• you process the response poorly, e.g. not taking Content-Type header or non-200 status code into account.

• you cannot imitate delays and timeout exceptions when interacting with HTTP API.

The good news is, that the with-http macro can test all the cases mentioned above and much more.

The App routes

The app parameter is a two-level map of the form:

{path {method response}}

For example:

{"/foo" {:get {:status 200 :body "it was GET"}
         :post {:status 201 :body "it was POST"}}}

Calling GET /foo and POST /foo would return 200 and 201 status codes with different messages.

The response might be:

• a Ring map;
• a Ring handler function that accepts a request map and returns a response map;
• an instance of java.io.File;
• a resource: (clojure.java.io/resource "some/file.txt");
• a string.

Other examples:

{"/foo" {:get (fn [{:keys [params]}]
                (log/infof "Params: %s" params)
                {:status 200 :body "OK"})}}

{"/some/json" {:get (io/resource "file.json")}}

The path might be a vector as well. During the preparation step, it will be compiled into a string, for example:

{["/foo/bar/" 42 "/test"]
 {:get {:status 200 :body "hello"}}}

;; becomes

{"/foo/bar/42/test"
 {:get {:status 200 :body "hello"}}}

This is useful when the paths contain parameters.

The make-url function helps to build a local URL like http://localhost:<port>/<path>. Its second path argument is either a string or a vector which gets compiled into a string:

(make-url PORT "/foo?a=1&b=2")
;; http://localhost:8899/foo?a=1&b=2

(make-url 8899 ["/users/" 42 "/reports/" 99999])
;; http://localhost:8899/users/42/reports/99999

Default handler

The App mapping has a default handler which gets triggered when the client calls a non-existing route. By default, it's 404 status page with the following JSON payload:

(def NOT-FOUND
  {:status 404
   :body {:error "with-http: route not found"}})

You can override it by adding the :default key to the app map. The value might be a map, a function, a file and so on.

{"/foo" {:get {:status 200 :body "hello"}}
         :default (fn [_]
                    {:status 202 :body "I'm the default!"})}

Basic test

A simple test to ensure the macro works:

(deftest test-with-http-test-json

  (let [app
        {"/foo" {:get {:status 200
                       :body "test"}}}

        url
        (make-url PORT "/foo")

        {:keys [status body]}
        (with-http [PORT app]
          (client/get url))]

    (is (= 200 status))
    (is (= "test" body))))

JSON

The Ring handler function produced from the app mapping is wrapped with wrap-json-response and wrap-json-params middleware layers. It means the body of the response might be a collection that gets dumped into JSON:

(deftest test-with-http-test-json

  (let [body
        {:hello [1 "test" true]}

        app
        {"/foo" {:get {:status 200
                       :body body}}}

        url
        (make-url PORT "/foo")

        {:keys [status body]}
        (with-http [PORT app]
          (client/get url {:as :json}))]

    (is (= 200 status))
    (is (= {:hello [1 "test" true]} body))))

Slow responses

To imitate slow responses, provide a function that sleeps for a certain amount of time:

{"/foo" {:get (fn [_]
                (Thread/sleep 10000)
                {:status 200 :body "OK"})}}

Then ensure you pass the timeout limit into your API call.

Files & Resources

Storing JSON responses in files is a good idea. Here is how you can serve them with the macro:

{"/foo" {:get (io/file "dev-resources/test.txt")}}

or

{"/foo" {:get (io/file "dev-resources/test.json")}}

Capturing requests

Another trick to improve your tests: ensure you pass the right parameters or headers to the HTTP API. Provide an atom and a handler function closed over that atom. Each time you receive a request, save it's data to the atom and then validate them:

(deftest test-with-http-capture-params

  (let [capture!
        (atom nil)

        app
        {"/foo" {:get (fn [{:keys [params]}]
                        (reset! capture! params)
                        {:status 200 :body "OK"})}}

        url
        (make-url PORT "/foo?a=1&b=2")

        {:keys [status body]}
        (with-http [PORT app]
          (client/get url))]

    (is (= 200 status))
    (is (= "OK" body))
    (is (= {:a "1" :b "2"} @capture!))))

License

Copyright © 2023 Ivan Grishaev

This program and the accompanying materials are made available under the terms of the Eclipse Public License 2.0 which is available at http://www.eclipse.org/legal/epl-2.0.

This Source Code may also be made available under the following Secondary Licenses when the conditions for such availability set forth in the Eclipse Public License, v. 2.0 are satisfied: GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version, with the GNU Classpath Exception which is available at https://www.gnu.org/software/classpath/license.html.