Security — yada 1.2.16

Security

Built into the library, yada offers a complete standards-based set of security features for today’s secure applications and content delivery.

In yada, resources are self-contained and are individually protected from unauthorized access:

One design goal of HTTP is to separate resource identification from request semantics.

— RFC 7231 Section 2

Security is part of the semantics of a resource, so is part of resource itself, rather than coupled to the URI and routing. This approach improves the cohesion of the web resource, which can be tested independently.

As in all other areas, yada aims for 100% compliance with core HTTP standards when it comes to security, notably RFC 7235. Also, since HTTP APIs are nowadays used to facilitate transactional integration between systems via the user’s browser, it is important that yada fully supports systems that offer APIs to other applications, across origins, as standardised by CORS.

Security aspects for a resource are specified in its model’s :access-control entry.

Depending on your use-case, you may decide to create this map independently and merge it into groups of yada resource maps. In yada, it is typical to see resource maps created programmatically—they are just maps of data after all.

It is worth taking some time to understand what is involved in processing an HTTP request in yada that relate to authentication and authorization. The relevant portion of the default request-processing interceptor chain in yada, showing the order of processing. shows a simplified portion of the sequence of steps.

The relevant portion of the default request-processing interceptor chain in yada, showing the order of processing.

skinparam monochrome true
(*) --> "parse-parameters"
note right
Query, path parameters,
request header & body
parsed. ❶
end note
"parse-parameters" --> "authenticate"
note right
Request credentials verified
(e.g. roles). ❷
end note
"authenticate" --> "get-properties"
note right
Resource properties established,
(e.g. resource owner). ❸
end note
"get-properties" --> "authorize"
note right
Request authorized by checking
credentials against resource's
properties. ❹
end note
"authorize" --> (*)
note right
Request processing
continues ❺
end note

1	First, the values of declared parameters (query, path, header, form & body) are taken from the request and checked for validity. These parameters may be used later, both in the identification of the user and the resource being addressed. Invalid parameters would cause a response with a `400 Bad Request` status.
2	Next, the request is authenticated according to an authentication scheme. This involves inspecting the request for claims about the identity of the user (and verifying that these claims are genuine and trustworthy). Unless there is something suspicious about the request’s claims, such as the detection of a forgery attempt, no decision is made at this stage about whether the request should be accepted or rejected.
3	Next, the resource’s properties are determined, such as existence, last modification time and, in particular, attributes governing ownership and required access conditions. These may be ascertained solely from the request or might involve one or more requests to other sources, such as databases.
4	After this, an authorization step is carried out to determine whether the credentials carried by the request, if any, are sufficient to allow access to the resource. If not, a response is returned with a 401 or 403 status code—a `401 Unauthorized` if no credentials are present, and a `403 Forbidden` if they are. A 401 gives the user-agent the hint that it should attempt to capture authentication data from the user and retry the request.
5	The request processing proceeds. Any response generated may depend on information established by these steps. For example, certain information might be filtered out of a response to requests that don’t have sufficient authorization.

Note that this design supports all of the following cases:

A resource is publicly accessible.
A resource is publicly accessible but is rendered differently for a authenticated user.
A resource cannot be accessed without authentication.
A resource cannot be accessed without authentication and the user having sufficient access rights.

A tale of two bank accounts

To understand the reasons why requests are processed in this way, here’s a practical example.

Imagine tasked with the problem of securing a banking website.

A bank has decided that the URL for accessing account balance information should be https://bigbank.com/accounts/<account-number>/balances.html.

All requests are checked to ensure that the request is from a properly logged-in user.

Let’s say Alice’s bank account number is 12345678. By this design, as long as Alice is logged in, Alice can access her account balance via a GET request to https://bigbank.com/accounts/12345678/balances.html.

Can you spot a security issue here?

The problem is that Alice can also access Bob’s bank account balances.

This may seem like an obvious mistake but this type of vulnerability has historically been extremely common and responsible for a great many hacks ^[1]. It is still a common contributor to OWASP 2017’s #5 most critical security issue: Broken Access Control.

One of the causes is that many web frameworks don’t provide good support for this example use-case, and still rely on declaratively security on the URL itself. This is another reason why it’s important to understand security in terms of a resource rather than a URI pattern. It’s only once you have fully identified a web resource that you can determine whether access to it should be granted to a request.

Realms

In HTTP, resources can exist inside a protection space determined by one or more realms. Each resource declares the realm (or realms) it is protected by, as part of the :access-control entry of its resource-model.

yada supports multiple realms. By default, there is a single realm in operation, named default. However, you can group multiple authentication schemes together with an authorization model into a separate realm. Note each realm can contain multiple authentication schemes (it might be that a realm offers a choice of how to authenticate).

{:access-control
  {:realms
    {"Gondor" {:authentication-schemes […]
               :authorization {…}}
     "Mordor" {:authentication-schemes {…}
               :authorization {…}}}}}

Authentication

Authentication is the act of establishing the credentials of a user, by checking the claims in the request.

As has already been explained, the act of authentication alone does not approve or deny the request—that requires the act of authorization which is covered in the next section.

Each realm declares one or more authentication schemes. An authentication scheme determines how the request’s credentials are established. Credentials contain information such as the user’s identity, roles and privileges, which can be used to deny the request, or if approved, may affect the nature of the response.

yada supports a number of built-in authentication schemes (Basic, Digest, Cookie, JWT) and also allows for custom authentication schemes.

Here is an example of a custom authentication scheme:

{:scheme "my-authentication-scheme" (1)
 :authenticate (fn [ctx] {:user "Alice"}) (2)
}

1	The scheme name.
2	An `authenticate` function, which is optional. For illustrative purposes, this one simply returns a map containing "Alice".

The scheme name can be a keyword but if it’s a string in will be used in a WWW-Authenticate header value in the event of a 401 Unauthorized response (authorization is covered later in Authorization).

The :authenticate function, if present, will specify a function that will be invoked with the yada context and should return credentials (which can be anything, but usually a map). Returning nil from this function indicates that the request contains no credentials, which determines whether a 401 or 403 is returned later if the authorization check fails.

If the :authenticate function is missing, the authentication method will be determined by the scheme, via the yada.security/verify multimethod. These might be built-in or methods provided by yada extensions (or user-code) via Clojure’s defmethod.

Authorization

Authorization is the act of allowing a user access to a resource.

This may require knowledge about the user only (for example, in Role-based access control) or may (additionally) depend on properties of the resource identified by the HTTP request’s URI (as part of an Attribute-based access control authorization scheme). In either case, we assume that the user has already been authenticated, and we are confident that their credentials are genuine.

In yada, the resource’s properties are determined prior to the authorization step, since it may be necessary to use these properties in the authorization decision.

Custom authorization with a validate function

A custom authorization scheme can be provided by specifying a validate function, which takes the yada context and the credentials established in the realm during the authentication step:

:authorization {:validate (fn [ctx creds] …)}

If the authorization succeeds, the validate function must return the yada context, possibly augmented to provide extra data that can might be needed when generating the response.

If the request should be rejected, the validate function should return nil or false. Either a 401 Unauthorized response will then be returned to the user-agent (in the case that no authentication claims were sent with the request) or a 403 Forbidden (in the case that they were).

It is a common case for a resource to require a valid logged-in user. This is easy to implement:

:authorization {:validate (fn [ctx creds] (when creds ctx))}

Custom authorization with a defmethod

As an alternative to specifying a function, a global custom authorization scheme can be defined via a defmethod.

This is no longer a recommended approach and exists solely for reasons of compatibility with previous yada releases, see Which should I choose: function or defmethod?

First, decide on a keyword that will be used to dispatch your authorization function. In this example, we’ve chosen :my/custom-authorization.

Now declare the authorization function that will be called by yada during request processing. This is a defmethod, as follows:

(defmethod yada.authorization/validate
  :my/custom-authorization
  [ctx credentials authorization]
…
)

The credentials argument contains all the verified credentials sent in the request.

Now add an :authorization map to the :access-control part of your resource model. The map must contain a :scheme value specific to your resource model, along with any extra parameters you want to be passed as the authorization argument to your authorization function. In this example, we want to pass the :my/ensure parameter set to [:same-account]. You can specify anything you like to be passed as parameters (there are no schema restrictions here).

{:access-control
 {:authorization
  {:scheme :my/custom-authorization
   :my/ensure [:same-account]}}}

Which should I choose: function or defmethod?

Given these two alternatives to custom authorization, which one should you choose?

Earlier versions of yada only provided the defmethod way of providing your own code, both for authorization and authentication.

The problem with a defmethod is that it exists at the top-level of a namespace and cannot access surrounding lexical scope. That makes it hard to access resources such as databases that might be required by the authentication and authorization steps.

One popular workaround was to inject database connections into the yada context via a custom interceptor at the beginning of the chain. However, this has provied a tricky, cumbersome and error-prone approach.

Functions are therefore recommeded rather than the defmethod approach.

Basic Authentication

Basic Authentication is supported by all web browsers and serves as a quick way of securing a resource. Basic Authentication is defined in RFC 2617.

Basic Authentication shows a resource that is protected with Basic Authentication.

Example 1. Basic Authentication

Assume we have a map of users:

{"alice" {:yada.example-auth/name "Alice Roberts"
          :yada.example-auth/password "Seeshai6"}
 "bob2" {:yada.example-auth/name "Bob Mortimer"
         :yada.example-auth/password "bohthoM6"}}

The function below creates a yada resource. The function takes the map of users an argument and returns a resource which will be accessible only via successful authentication.

(ns yada.example-auth
  (:require [yada.yada :as yada]))

(defn basic-auth-example-resource [users]
  (yada/resource
   {:access-control
    {:realms
     {"default"
      {:authentication-schemes
       [{:scheme "Basic" ; (1)
         :verify
         (fn [[user-id given-password]]
           (when-let [user (get users user-id)]
             (when (= given-password (::password user))
               {::user (dissoc user ::password)})))}] ; (2)
       :authorization
       {:validate
        (fn [ctx creds]
	  (when creds ctx))}}}} ; (3)

    :methods
    {:get
     {:produces
      {:media-type "text/plain" :charset "utf8"}
      :response
      (fn [ctx]
        (format
         "Welcome %s" ; (4)
         (-> ctx :authentication (get "default") ::user ::name)))}}}))

1	The :scheme is declared as `"Basic"`
2	If the user and password match, a map is returned containing the credentials (in this case, the map representing the user, minus the password)
3	If credentials exist, the authorization step succeeds.
4	The user’s name is generated in the response body.

How Basic authentication works

Let’s take a moment to dig a bit deeper into the implementation of this method:

link:../src/yada/security.clj[role=include]

1	Note the multimethod is dispatched on the value of `:scheme`, and the `verify` function extracted.
2	The `Authorization` header is extracted from the request, via a regular expresssion.
3	The base64-encoded credentials are extracted.
4	The `verify` function is called with the decoded `user` and `password` values.

Note that the implementation does much of the heavy-lifting so that the verify function can be easier to implement.

Digest authentication

(coming soon)

We can also use cookies to present authentication credentials. The advantage of cookies is that they can be set by the server based on custom authentication interaction with the user, such as the submission of a login-form.

To protect a site with cookies:

{:access-control
  {:scheme :cookie
   :cookie "session"
   :verify (fn [cookie] …}}

JWT authentication

(coming soon)

Form-based logins

Basic Authentication has a number of weaknesses, such as the difficulty of logging out and the lack of control that a website has over the fields presented to a human. Therefore, the vast majority of websites prefer to use a custom login form generated in HTML.

You can think of a login form as a resource that lets the user present one set of credentials in order to acquire additional ones. The credentials the user presents, via a form, are verified and if they are true, a cookie is generated that certifies this. This cookie provides the certification to subsequent requests in which it is sent.

Let’s start by building this login resource that will provide a login form page to browsers and verify the form data when that form is submitted.

Here’s a simplistic but viable resource model for the two methods involved:

(require
 '[buddy.sign.jwt :as jwt]
 '[schema.core :as s]
 '[hiccup.core :refer [html])

{:methods
 {:post
  {:consumes "application/x-www-form-urlencoded"
   :parameters {:form
                {:user s/Str :password s/Str}}

   :response
   (fn [ctx]
     (let [{:keys [user password]} (get-in ctx [:parameters :form])]
       (if (valid-user user password)
         (assoc (:response ctx)
                :cookies {"session"
                          {:value
                           (jwt/sign {:user user} "lp0fTc2JMtx8")}})
         "Try again!")))}
  :get
  {:produces "text/html"
   :response (html
              [:form {:method :post}
               [:input {:name "user" :type :text}]
               [:input {:name "password" :type :password}]
               [:input {:type :submit}]])}}}

The POST method method consumes incoming URL-encoded data (the classic way a browser sends form data). It de-structures the two parameters (user and password) from the form parameters.

We then determine if the user and password are valid (we don’t explain here how this is done, but assume a valid-user function exists that can tell us). If the user is valid we associate a new cookie called "session" with the response. By starting with the :response value of the request context, we ensure yada interprets our return value as a Ring response rather than some other value.

We use Buddy’s sign function to sign and encoded the cookie’s value as a JSON string. We only specify the credentials as {:user user} in this case, but we could put much more into that map. The sign function requires us to provide a secret symmetric key that we can use for both signing and verification, but the library does allow us asymmetric key options too.

The other method, GET, simply produces a form for user-agents that can render HTML (browsers, typically) to post back. For reasons of cohesion, it’s a good idea to provide these two methods in the same resource to encapsulate and dedupe the fields which are relevant to both the GET and the POST.

Logout

The recommended way of logging out is to remove the session.

Bearer authentication (OAuth2)

(coming soon)

Multifactor authentication

(coming soon)

Built-in role-based authorization

By default, yada will use a declarative role-based authorization scheme.

Any method can be protected by declaring a role or set of roles in its model.

{:access-control
 {:authorization
  {:methods
   {:post :accounts/create-transaction}}}}

If multiple roles are involved, they can be composed inside vectors using simple predicate logic.

{:access-control
 {:authorization
  {:methods
   {:post [:or [:and :accounts/user
                     :accounts/create-transaction]
               :superuser}}}}

Only the simple boolean 'operators' of :and, :or and :not are allowed in this authorization scheme. This keeps the role definitions declarative and easy to extract and process by other tooling.

Of course, authentication information is available in the request context when a method is invoked, so any method may apply its own custom authorization logic as necessary. However, yada encourages developers to adopt a declarative approach to resources wherever possible, to maximise the integration opportunities with other libraries and tools.

yada fully supports Cross-Origin Resource Sharing (CORS) allowing you to provide APIs that are accessible from other origins.

For example, you may be creating an API that you wish other websites to make use of, by allowing browsers visiting those websites access to your API.

CORS is specified in the :access-control section of the resource-model.

{:access-control
 {:allow-origin "*"
  :allow-credentials false
  :expose-headers #{"X-Custom"}
  :allow-methods #{:get :post}
  :allow-headers ["Api-Key"]
 }}

With the exception of :allow-credentials (which must be a boolean), any of the values can be declared as single-arity functions, which are called with the request-context as an argument to determine the value for the corresponding response header.

HTTP Strict Transport Security (HSTS)

clojure {:strict-transport-security {:max-age 12000}}

Defaults to a maximum age of 31536000.

The HSTS header is only set if the scheme is HTTPS or the service is behind a proxy (determined by the presence of the X-Forwarded-For request header).

Content Security Policy

{:content-security-policy "url-src"}

Defaults to default-src https: data: 'unsafe-inline' 'unsafe-eval'.

Clickjacking prevention

A browser’s iframe can be used for 'click-jacking'. By default yada tells browsers not to allow this. The default value is SAMEORIGIN, unless you override it in the resource-model.

{:x-frame-options "NONE"}

Cross-site Scripting (XSS) protection

yada also sets the X-Xss-Protection response header to 1; mode=block. This can be overridden in the resource model.

{:x-content-type-options "0"}

Media-type sniffing protection

By default, yada sets the X-Content-Type-Options response header to nosniff. This tells browsers not to try to attempt to determine the content-type of the response body.

Since yada sets the Content-Type header according to HTTP standards, there should never be a need for a browser to 'sniff' the response body for this information, preventing an attack that might exploit some vulnerability in this process.

1. It is surprising that the simple editing of a URL in the browser by the user should be considered 'hacking' at all