Muuntaja

Clojure library for fast http format negotiation, encoding and decoding. Standalone library, but ships with adapters for ring (async) middleware & Pedestal-style interceptors. Explicit & extendable, supporting out-of-the-box JSON, EDN and Transit (both JSON & Msgpack). Ships with optional adapters for MessagePack and YAML.

Based on ring-middleware-format, but a complete rewrite (and up to 30x faster).

Rationale

• both standalone & http (middleware & interceptors)
• explicit configuration over mutable state (e.g. multimethods)
• fast with good defaults
• extendable & pluggable: new formats, behavior
• typed exceptions - caught elsewhere
• support runtime docs (like swagger) & inspection (negotiation results)
• support runtime configuration (negotiation overrides)

Modules

• metosin/muuntaja - the core abstractions + Jsonista JSON, EDN and Transit formats
• metosin/muuntaja-cheshire - optional Cheshire JSON format
• metosin/muuntaja-msgpack - Messagepack format
• metosin/muuntaja-yaml - YAML format

Posts

• Muuntaja, a boring library everyone should use

Check the docs on cljdoc.xyz for detailed API documentation as well as more guides on how to use Muuntaja.

Latest version

[metosin/muuntaja "0.6.0-alpha1"]

Optionally, the parts can be required separately:

[metosin/muuntaja-cheshire "0.6.0-alpha1"]
[metosin/muuntaja-msgpack "0.6.0-alpha1"]
[metosin/muuntaja-yaml "0.6.0-alpha1"]

Muuntaja requires Java 1.8+

Quickstart

Ring

(require '[muuntaja.middleware :as middleware])

(defn echo [request]
  {:status 200
   :body (:body-params request)})

; with defaults
(def app (middleware/wrap-format echo))

(app {:headers
      {"content-type" "application/edn"
       "accept" "application/json"}
      :body "{:kikka 42}"})
; {:status 200,
;  :body #object[java.io.ByteArrayInputStream]
;  :muuntaja/format "application/json",
;  :headers {"Content-Type" "application/json; charset=utf-8"}}

There is a more detailed Ring guide too. See also differences to ring-middleware-format & ring-json if you are migrating from those.

Interceptors

See muuntaja.interceptor.

Standalone

Create a Muuntaja and use it to encode & decode JSON:

(require '[muuntaja.core :as m])

;; with defaults
(def m (m/create))

(->> {:kikka 42}
     (m/encode m "application/json")
     slurp)
; => "{\"kikka\":42}"

(->> {:kikka 42}
     (m/encode m "application/json")
     (m/decode m "application/json"))
; => {:kikka 42}

With custom EDN decoder options:

(def m
  (m/create
    (assoc-in
      m/default-options
      [:formats "application/edn" :decoder-opts]
      {:readers {'INC inc}})))

(->> "{:value #INC 41}"
     (m/decode m "application/edn"))
; => {:value 42}

Defining a encoding function for Transit-json:

(def encode-transit-json
  (m/encoder m "application/transit+json"))

(slurp (encode-transit-json {:kikka 42}))
; => "[\"^ \",\"~:kikka\",42]"

Encode format

By default, encoding writes value into a java.io.ByteArrayInputStream. This can be changed with a :return option, accepting the following values:

value	description
:input-stream	encodes into java.io.ByteArrayInputStream (default)
:bytes	encodes into byte[]. Faster than Strean, enables NIO for servers supporting it
:output-stream	encodes lazily into java.io.OutputStream via a callback function

All return types satisfy the following Protocols & Interfaces:

• ring.protocols.StreamableResponseBody, Ring 1.6.0+ will stream these for you
• clojure.io.IOFactory, so you can slurp the response

:input-stream

(def m (m/create (assoc m/default-options :return :input-stream)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; #object[java.io.ByteArrayInputStream]

:bytes

(def m (m/create (assoc m/default-options :return :bytes)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; #object["[B" 0x31f5d734 "[B@31f5d734"]

:output-stream

(def m (m/create (assoc m/default-options :return :output-stream)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; <<StreamableResponse>>

Format-based return

(def m (m/create (assoc-in m/default-options [:formats "application/edn" :return] :output-stream)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; #object[java.io.ByteArrayInputStream]

(->> {:kikka 42}
     (m/encode m "application/edn"))
; <<StreamableResponse>>

HTTP format negotiation

HTTP format negotiation is done via request headers for both request (content-type, including the charset) and response (accept and accept-charset). With the default options, a full match on the content-type is required, e.g. application/json. Adding a :matches regexp for formats enables more loose matching. See Configuration docs for more info.

Results of the negotiation are published into request & response under namespaced keys for introspection. These keys can also be set manually, overriding the content negotiation process.

Exceptions

When something bad happens, an typed exception is thrown. You should handle it elsewhere. Thrown exceptions have an ex-data with the following :type value (plus extra info to generate descriptive erros to clients):

• :muuntaja/decode, input can't be decoded with the negotiated format & charset.
• :muuntaja/request-charset-negotiation, request charset is illegal.
• :muuntaja/response-charset-negotiation, could not negotiate a charset for the response.
• :muuntaja/response-format-negotiation, could not negotiate a format for the response.

Server Spec

Request

• :muuntaja/format, format name that was used to decode the request body, e.g. application/json. If the key is already present in the request map, muuntaja middleware/interceptor will skip the decoding process.
• :muuntaja/request, client-negotiated request format and charset as FormatAndCharset record. Will be used in the response pipeline.
• :muuntaja/response, client-negotiated response format and charset as FormatAndCharset record. Will be used in the response pipeline.
• :body-params decoded body is here.

Response

• :muuntaja/encode?, if set to true, the response body will be encoded regardles of the type (primitives!)
• :muuntaja/format, format name that was used to encode the response body, e.g. application/json. If the key is already present in the response map, muuntaja middleware/interceptor will skip the encoding process.
• :muuntaja/content-type, handlers can use this to override the negotiated content-type for response encoding, e.g. setting it to application/edn will cause the response to be formatted in JSON.

Options

Default options

{:http {:extract-content-type extract-content-type-ring
        :extract-accept-charset extract-accept-charset-ring
        :extract-accept extract-accept-ring
        :decode-request-body? (constantly true)
        :encode-response-body? encode-collections-with-override}

 :allow-empty-input? true
 :return :input-stream

 :default-charset "utf-8"
 :charsets available-charsets

 :default-format "application/json"
 :formats {"application/json" json-format/json-format
           "application/edn" edn-format/edn-format
           "application/transit+json" transit-format/transit-json-format
           "application/transit+msgpack" transit-format/transit-msgpack-format}}

Profiling

YourKit supports open source projects with its full-featured Java Profiler. YourKit, LLC is the creator of YourKit Java Profiler and YourKit .NET Profiler, innovative and intelligent tools for profiling Java and .NET applications.

License

Picture

By Unknown. The drawing is signed "E. Ducretet", indicating that the apparatus was made by Eugene Ducretet, a prominent Paris scientific instrument manufacturer and radio researcher. The drawing was undoubtedly originally from the Ducretet instrument catalog. [Public domain], via Wikimedia Commons.

Original Code (ring-middleware-format)

Copyright © 2011, 2012, 2013, 2014 Nils Grunwald
Copyright © 2015, 2016 Juho Teperi

This library

Copyright © 2016-2017 Metosin Oy

Distributed under the Eclipse Public License, the same as Clojure.