Liking cljdoc? Tell your friends :D

Your First API

Welcome Back

This is the fourth part in the tutorial. By now, you’ve built a simple service that vends out friendly greetings by name and can handle a few content types.

We’re going to leave our greeting service alone for the time being and turn our attention to a REST API for some entities. To keep things simple, we’re going to keep the entities in memory. Even so, we will use the same kind of techniques you would use in a real application.

What You Will Learn

After reading this guide, you will be able to:

  • Use path parameters to identify items

  • Generate URLs for new items

  • Encode and decode entity bodies

Guide Assumptions

This guide is for beginners who are new to Pedestal and may be new to Clojure. It doesn’t assume any prior experience with a Clojure-based web framework. You should be familiar with the basics of HTTP: URLs, response codes, and content types.

You do not need experience with any particular database or persistence framework.

This guide also assumes that you are in a Unix-like development environment, with Java installed. We’ve tested it on Mac OS X and Linux (any flavor) with great results. We haven’t yet tried it on the Windows Subsystem for Linux, but would love to hear from you if you’ve succeeded with it there.

Getting Help if You’re Stuck

partial$getting-help.adoc

Where We Are Going

We will start a new project this time. The service will be a simple "to-do" list that persists only in memory. We will use the same techniques that we would use to interact with a real database.

We’re going to make a REST style API. Our API will have the following routes:

RouteVerbActionResponse

/todo

POST

Create a new list

201 with URL of new list

/todo

GET

Return query form

200 with static page

/todo/:list-id

GET

View a list

200 with all items

/todo/:list-id/:item-id

GET

View an item

200 with item

/todo/:list-id/:item-id

PUT

Update an item

200 with updated item

Setting Up

We will start this project with another empty directory. (Still avoiding the template projects for now.)

I’m going to call my project 'todo-api'. Feel free to call yours something different, but it’s up to you to do the mental translation from here on out.

$ mkdir todo-api
$ cd todo-api
$ mkdir src

This time, we’ll set up our deps.edn file first. It will be exactly the same as our last one:

deps.edn
link:example$todo-api/deps.edn[role=include]

Our new source file will start off with the usual namespace declaration:

src/main.clj
link:example$todo-api/src/main.clj[role=include]

This adds new namespaces that we’ll use later in this tutorial.

Partial Functions

We’re going to use some of the same structures from previous tutorials, such as an ok function to build a successful response, but with some differences:

src/main.clj
link:example$todo-api/src/main.clj[role=include]

clj:partial[] is a Clojure function that takes a function and returns a new function that needs fewer arguments - you supply the initial arguments in the call to partial. The returned function keeps track of those initial arguments, and when invoked, "glues together" the initial arguments and any additional arguments provided to it, and calls the original function: response in this case.

The & {:as headers} forms in the response function’s parameter list is a bit advanced; it’s based on destructuring, but in practice all it means is that after the status and body arguments, you may pass key/value pairs that will be used as the headers of the response map built by response. For example, you could have a handler function return (ok "success" "Content-Type" "text/plain").

Although created with def, the symbols ok, created, and accepted, are all functions, as if they were created using defn. Remember: Clojure functions are still values!

Why do any of this? For all that web developers generally know the HTTP status codes in their sleep [1], having these three functions results in code that is more intentful: the semantics become visible directly in the code.

And you probably did go look up code 201.

Routes Up Front

Second, we’re going to define all our routes right away, but have them all use an echo handler so we can test the routes separately from everything else[2].

src/main.clj
link:example$todo-api/src/main.clj[role=include]

Pedestal makes it possible to invoke these routes directly from the REPL, without involving HTTP at all. This is a nice way to break your work up into incremental steps. It goes together nicely with the interactive development support we introduced in the Hello World, With Parameters tutorial.

Let’s start the service in this early, echo-only mode:

$ clj
Clojure 1.12.0
user=> (require 'main)
nil
user=> (main/start)
#object[io.pedestal.http.http_kit$create_connector$reify__15862 0x63cd8f26 "io.pedestal.http.http_kit$create_connector$reify__15862@63cd8f26"]
user=>

Poking the Routes

Let’s try out the routes that we defined in the main namespace. We’ll do that using api:response-for[ns=io.pedestal.service.test]. This function exercises our Pedestal connection without going through any actual HTTP requests.

By extracting the connector (it’s in the main/*connector atom) we can pass it to test/response-for along with the HTTP verb (as a keyword), and the URL:

user=> (require '[io.pedestal.service.test :as test])
nil
user=> (test/response-for @main/*connector :get "/todo")
[main] INFO io.pedestal.service.interceptors - {:msg "GET /todo", :line 40}
{:status 200, :body "{:headers {}, :async-channel #object[io.pedestal.http.http_kit.impl$mock_channel$reify__15583 0x4fbdd61f \"io.pedestal.http.http_kit.impl$mock_channel$reify__15583@4fbdd61f\"], :server-port -1, :path-info \"/todo\", :url-for #object[clojure.lang.Delay 0xc7ba58d {:status :pending, :val nil}], :uri \"/todo\", :server-name \"localhost\", :query-string nil, :path-params {}, :body #object[java.io.InputStream$1 0x2d579733 \"java.io.InputStream$1@2d579733\"], :scheme :http, :request-method :get, :headers {:strict-transport-security "max-age=31536000; includeSubdomains", :x-frame-options "DENY", :x-content-type-options "nosniff", :x-xss-protection "1; mode=block", :x-download-options "noopen", :x-permitted-cross-domain-policies "none", :content-security-policy "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", :content-type "application/edn"}}
Part of this output — [main] INFO …​ — comes from built-in Pedestal interceptors for logging requests; the logging output gets mixed into the output from the REPL.

The response is somewhat overwhelmingly verbose because we are using the echo interceptor, so the entire incoming request converted to a string and is nested inside the response. If we omit the response body, it’s easier to observe the most useful parts of the response:

user=> (dissoc *1 :body)
{:status 200, :headers {:strict-transport-security "max-age=31536000; includeSubdomains", :x-frame-options "DENY", :x-content-type-options "nosniff", :x-xss-protection "1; mode=block", :x-download-options "noopen", :x-permitted-cross-domain-policies "none", :content-security-policy "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", :content-type "application/edn"}}
Inside the REPL, Clojure keeps track of the most recent result it printed in the symbol *1. That makes it a snap to dig down into that result. There’s also *2 and *3.

Now we can see the status code (200, OK) and the headers. Not too bad.

We’re going to test quite a few of these routes this way, so let’s add a helper function:

link:example$todo-api/src/main.clj[role=include]

Now, after we reload the namespace with (require :reload 'main), we can experiment a bit easier.

What happens if we ask for a route that doesn’t exist? Or a verb that isn’t supported on a real route?

user=> (require :reload 'main)
nil
user=> (main/test-request :get "/does-not-exist")
[main] INFO io.pedestal.service.interceptors - {:msg "GET /does-not-exist", :line 40}
{:status 404, :headers {:content-type "text/plain"}, :body "Not Found"}

This 404 response comes from Pedestal itself [3] function]. That is the default response when the router cannot find a route that matches the URL pattern. (For much more about routers, see the reference:routing-quick-reference.adoc).

Path Parameters

Some of our routes have parameters in them. Those look like keywords embedded in the URL patterns. That tells Pedestal to match any value in that position (except a slash!) and bind it to that parameter in the reference:request-map.adoc. We should be able to see those parameters in the requests that echo will send back to us. This is one of the ways we can make sure that routes do what we think they are going to do, before we write the real logic.

user=> (main/test-request :get "/todo/abcdef/12345")
[main] INFO io.pedestal.service.interceptors - {:msg "GET /todo/abcdef/12345", :line 40}
{:status 200, :body "{:headers {}, :async-channel #object[io.pedestal.http.http_kit.impl$mock_channel$reify__15583 0x6801b4a4 \"io.pedestal.http.http_kit.impl$mock_channel$reify__15583@6801b4a4\"], :server-port -1, :path-info \"/todo/abcdef/12345\", :url-for #object[clojure.lang.Delay 0x54eaf69d {:status :pending, :val nil}], :uri \"/todo/abcdef/12345\", :server-name \"localhost\", :query-string nil, :path-params {:list-id \"abcdef\", :item-id \"12345\"}, :body #object[java.io.InputStream$1 0x647b7f63 \"java.io.InputStream$1@647b7f63\"], :scheme :http, :request-method :get, :headers {:strict-transport-security "max-age=31536000; includeSubdomains", :x-frame-options "DENY", :x-content-type-options "nosniff", :x-xss-protection "1; mode=block", :x-download-options "noopen", :x-permitted-cross-domain-policies "none", :content-security-policy "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", :content-type "application/edn"}}

The interesting part here is deeply buried inside the request map (which we see as a giant string inside the response body). We can pull it out with a bit of Clojure code, but I’ll just highlight it here. (If you’re interested in an exercise, try using clj:*[ns=clojure.edn] to parse the response body).

Inside the request map, there is a :path-params map which looks like this:

:path-params {:list-id "abcdef", :item-id "12345"}

Pedestal extracted these parameters from the URL and bound them to the request for us. Note that the values are always strings, because URLs are strings. If we want to treat "12345" as a number, that’s up to us. But our application had better not break if some sly user sends us a request with something other than numbers there.

Create List

Before we can do anything else, we need a way to create a TODO list - later we’ll handle adding items to the list.

This is a POST to the /todo URL. In the interest of simplicity, the only part of a request body we need is a name field. Also in the interest of simplicity, we are not going to deal with content negotiation yet.

What do we need to do in order to create a list?

  1. Make a new data structure for the list.

  2. Add it to our repository.

Without getting too far ahead of ourselves, we’d like to avoid doing all of the above in a single handler function, and instead, break this up into several interceptors, each of which is (as much as possible) free of side effects, and easily testable.

The approach we are taking here is especially well suited when using Datomic or Datalevin for persistent storage, but can work even when using a more traditional database.

Let’s break the behavior up into multiple steps, and give each step a provisional name:

  1. A attaches a database connection to the request.

  2. B looks up an item of interest and attaches it to the request.

  3. C makes some decisions and attaches the result of those decisions to the request.

  4. D executes a transaction with those decisions.

This way, A and D are quite generic. B is somewhat specific to the particular route — is it a list or a list item that’s being looked up?. That leaves C which is most specific to a single route, and might event be a simple handler function.

For this guide, we are going to cheat and create an in-memory repository:

src/main.clj
link:example$todo-api/src/main.clj[role=include]
1Using clj:defonce[] here ensures that the value, the database atom, will survive when we reload the namespace.

Next, we’ll implement roles A and D in a single interceptor:

src/main.clj
link:example$todo-api/src/main.clj[role=include]
1@*database gets the current value of the atom. We attach it to the request inside the context so that other interceptors can see it. This is step A from above.
2The :tx-data key on the context may be set by other interceptors. This is step D from above.
3clj:if-let[] combines a let with an if test. The swap! here is our stand-in for a real database transaction. The clj:swap![] modifies the value stored in the *database atom, and returns the new value.
4We attach the updated database value in case any other interceptors need to use it. We use the latest version of the database (returned from swap!).
5If there wasn’t any tx-data, we just return the context without changes. If you forget this and return nil, every request will respond with 404.

This may be a simple tutorial where we’re feeding in one request at a time, manually …​ but one aspect to working in Clojure is just how well even such simple code scales to real world realities. This interceptor would work quite well in a busy server with multiple request processing threads executing simulataneously. The @*database expression captures a consistent snapshot of the ever-changing database stored in the atom; any interceptors in the queue will all work from a single consistent view of the database. No sharing, no race conditions, no impossible to reproduce bugs caused by mutable data shared across multiple threads.

Now we need a handler to receive the request, extract a name, and create the actual list.

src/main.clj
link:example$todo-api/src/main.clj[role=include]

link:example$todo-api/src/main.clj[role=include]
1If there was a query parameter, use it. Otherwise, use a placeholder.
2Generate a unique ID. In a real service, the DB itself would do this for us.
3Don’t modify the DB here. Attach data to the context that the db-interceptor will use.
4We modify the route definition to use both new interceptors, in order.

There are a couple of things to notice about this updated route definition. First, instead of using the echo handler function here, we have a vector with our db-interceptor and list-create interceptors. Recall from the introduction to interceptors that interceptors have two functions. The :enter functions get called from left to right, then the :leave functions get called from right to left. That means calls to the db-interceptor bracket the call to the list-create interceptor.

The second thing about the new route is that we removed the :route-name :list-create clause. Pedestal requires that every route must have a unique name. When we used the echo interceptor everywhere, there was no way for Pedestal to create unique, meaningful names for the routes so we had to help it out. Now, list-create has its own name so we can leverage Pedestal’s default behavior: use the name of the last interceptor in the vector.

When you use a handler function in a route and don’t supply an explicit :route-name, Pedestal will attempt to extract a route name from the function. You can supply meta data to specify a better name than the default.

First Attempt

Let’s give this a try.

user> (main/test-request :post "/todo?name=A-List")
[main] INFO io.pedestal.service.interceptors - {:msg "POST /todo", :line 40}
{:status 404, :headers {:content-type "text/plain"}, :body "Not Found"}

Uh oh. What happened here? We just tried this route a minute ago. Was it the query parameter?

user> (main/test-request :post "/todo")
[main] INFO io.pedestal.service.interceptors - {:msg "POST /todo", :line 40}
{:status 404, :headers {:content-type "text/plain"}, :body "Not Found"}

Nope.

If you check the contents of your in-memory database by running @main/*database in the REPL, you’ll see the new items (one for each POST request) in there:

user=> @main/*database
{"l15892" {:name "A-List", :items {}},
 "l15895" {:name "Unnamed List", :items {}}}

So why did Pedestal return a 404 Not found response?

The problem is that neither of the list-create and db-interceptor interceptors attached any kind of response to the context. Pedestal looked at the context after running all the interceptors, saw that there was no response, and sent the 404 as a fallback. We need to attach a response somewhere.

Providing a proper response

In REST apis, it is expected that a POST request that creates an entity will then return a response with a 201 Created status code, and include a Location header that points to the new resource.

We have defined a route for that, :get /todo/:list-id, which we’ve named :list-view, and, knowing the list id, it’s quite straight forward to assemble that string.

But should our handler code be responsible for doing so? Anyone with experience in web development knows that things shift around constantly, and should a change to the routing necessitate changes to some scattered collection of handlers? Even if the routing table is stable, we want to avoid duplicated code.

Pedestal provides the api:url-for[ns=io.pedestal.http.route] function to generate a URL from route name, and will even interpolate parameters into the URL string. So our list-create interceptor can point the client over to a route that will render the new list.

Of course, it would be nice if the response body also included the current state of the just-created list. That saves the client a round trip and saves the service a request. For now, we can do this all in a single interceptor.

src/main.clj
link:example$todo-api/src/main.clj[role=include]
1Generate URL for route :list-view and the new ID we just generated. Pedestal uses some hidden machinery[4] to get the routing table, which is needed to construct the URL.
2Return a 201 response, with an extra header (Location).

And the results (manually formatted for readability):

user==> (main/test-request :post "/todo?name=A")
{:status 201,
  :body "{:name \"A\", :items {}}", (1)
  :headers {:x-xss-protection "1; mode=block", (2)
            :x-content-type-options "nosniff",
            :x-permitted-cross-domain-policies "none",
            :x-download-options "noopen",
            :x-frame-options "DENY",
            :strict-transport-security "max-age=31536000; includeSubdomains",
            :content-security-policy "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;",
            :content-type "application/edn",
            :location "/todo/l15959"}} (3)
1The created list, as Clojure map, encoded as a string.
2The api:response-for[ns=io.pedestal.service.test] function converts headers from Camel-Case strings to lower-case keywords. It also ensures that the :body is a string (by default).
3The location, generated by url-for.

In general, adding something to the database might cause it to change. Sometimes there are triggers or stored procedures that change it. Other times it’s just an ID being assigned. Usually, instead of attaching the new entity here, I would attach its ID to the context and use another interceptor later in the chain to look up the entity after db-interceptor executes the transaction.

This becomes especially important when you are updating existing entities that could have concurrent changes from many sources.

View List

Now that we’ve done one route, this will go a bit faster. The connection from route to interceptor to entity should be a bit more evident at this point.

To view a list, we need to look it up from the "database", as provided in the :database key of the request map.

If found, we want to return the found to-do list …​ but indirectly.

src/main.clj
link:example$todo-api/src/main.clj[role=include]
1Query the in-memory database.
2Extract the path parameter, if present.
3Lookup the matching list if the parameter was provided.
4If a list was found, add it to the context as :result; otherwise do nothing resulting in a 404 response.

Adding the :result key to the context does not terminate execution of the interceptor chain; that’s triggered by the :response key. And if we don’t provide a response, we’re back in 404 land.

Once again, we’ll lean on interceptors to keep our code reusable.

The Entity View

The entity-render interceptor provides the missing part; when a prior interceptor has attached a :result to the context, it will build and attach a response.

src/main.clj
link:example$todo-api/src/main.clj[role=include]

But to make this work, we need to include it in the :list-view route:

src/main.clj
link:example$todo-api/src/main.clj[role=include]

After once again reloading code and restarting the connector, you can give the endpoint a try:

user=> (main/test-request :get "/todo/l15892")
[main] INFO io.pedestal.service.interceptors - {:msg "GET /todo/l15892", :line 40}
{:status 200, :body "{:name \"A-List\", :items {}}", :headers {:strict-transport-security "max-age=31536000; includeSubdomains", :x-frame-options "DENY", :x-content-type-options "nosniff", :x-xss-protection "1; mode=block", :x-download-options "noopen", :x-permitted-cross-domain-policies "none", :content-security-policy "object-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'strict-dynamic' https: http:;", :content-type "application/edn"}}

There in the body is the retrieved list.

Add Item to List

As we build up a set of interceptors and utility functions, we find that we can go ever faster. Next up, we’ll look at adding a new item to an existing list.

To add an item to a list we will perform the following steps:

  1. Check that the list exists

  2. Create a new item for the list

  3. Add that item to the list

  4. Return a response with the new item

Someone with a seasoned eye may notice that we have multiple steps that could cause problems in heavily used, multi-threaded server:

  • What if the list is deleted by some other thread between steps 1 and 2?

  • What if another item is added to the list between steps 1 and 3 (do we lose one of the items?)

This is the kind of thing that is really important to do atomically. We can take advantage of that Clojure atom to execute a function atomically. That’s what the swap! call does for us.

But since that function might not succeed, it means we don’t know enough to create the response inside the list-item-create interceptor. We need to defer response creation until after the db-interceptor runs its :leave function. Basically, we need something to look up an item by its list ID and item ID, then render the result.

That’s exactly what we would do for the :list-item-view route! We can get some reuse here. If list-item-create can add the item to the database, then the item will exist. If the item can’t be added, it will result in a 404 response.

We just need to do something a little funky-looking with the list-item-view interceptor. We need to put its behavior in the :leave function instead of the :enter function.

Here’s the pair of new interceptors, plus the updated route table.

src/main.clj
link:example$todo-api/src/main.clj[role=include]
1New generic interceptor that provides a 200 OK response with EDN body out of any Clojure data structure.
2New interceptor that looks up list items but doesn’t attach a response (since that’s `entity-render’s job).
3Instead, list-item-view attaches the result to the context where entity-render expects to find it.
4This interceptor creates an item and sets up tx-data to be executed.
5This is where list-item-create passes a parameter to list-item-view.

And, again, we must add the necessary interceptor to the :list-item-create route:

src/main.clj
link:example$todo-api/src/main.clj[role=include]

To understand the interceptors for the :list-item-create route, we need to think about :enter functions from left to right and :leave functions from right to left.

PhaseInterceptorAction

Enter

entity-render

None.

Enter

list-item-view

None.

Enter

db-interceptor

Attach the database value to the context.

Enter

list-item-create

Attach tx-data to the context and a path-param to identify the item.

Leave

list-item-create

None.

Leave

db-interceptor

Execute the tx-data, attach updated DB value to context.

Leave

list-item-view

Find the (new) item by path-params, attach it to the context as :result.'

Leave

entity-render

Find :result on the context, make an OK response.

A defect here is that when adding a list item, we don’t return a 201 response and a Location header as we do when adding a list. Implementing that should be an additional section in this tutorial.

Splitting the work into these pipelined stages allows us to make very small, very targeted functions that collaborate via the context.

The rest of the routes follow these same basic patterns. Take a few minutes to see if you can create the interceptors for the next few.

The Whole Shebang

Here is the final version of the code. Everything is cleaned, folded, and sorted nicely.

src/main.clj
link:example$todo-api/src/main.clj[role=include]

The Path So Far

In this guide, we have built a new API with an in-memory database. You’re ready to go get funding and launch!

We covered some more advanced concepts with interceptors this time:

  • Passing results from one interceptor to another

  • Reusing interceptors in different routes

  • Separating query from transaction with :enter and :leave functions

  • Testing our routes interactively at a REPL.

Where To Next?

Congratulations! You’ve finished the whole getting started trail.

To go deep on some of these topics, check out any of the following references:

  • reference:interceptors.adoc

  • reference:routing-quick-reference.adoc

  • reference:default-interceptors.adoc

1. But do you know what status code 418 is for?
2. See the Hello World With Content Types tutorial for an explanation of the echo interceptor.
3. Technically, it comes from an interceptor added by the api:with-default-interceptors[ns=io.pedestal.connector
4. A private, thread-bound var.
❮What is an Interceptor?getting-help❯

Can you improve this documentation? These fine people already did:
Howard M. Lewis Ship, Howard Lewis Ship & Marduk BolañosEdit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

Keyboard shortcutsReport a problemcljdoc on GitHub
× close