Edamame

Configurable EDN/Clojure parser with location metadata.

Reasons to use edamame

• You want to include locations in feedback about Clojure and EDN files
• You want to parse Clojure-like expressions without any evaluation
• Anonymous functions are read deterministically
• Highly configurable
• Auto-resolve aliased keywords based on the ns form

This library works with:

• Clojure on the JVM
• GraalVM compiled binaries
• ClojureScript (including self-hosted and advanced compiled)

Installation

Use as a dependency:

Projects

Project using edamame:

• clerk
• malli
• poly
• SCI
• splint
• zprint
• zen

API

See API.

Usage

(require '[edamame.core :as e :refer [parse-string]])

Location metadata

Locations are attached as metadata:

(def s "
[{:a 1}
 {:b 2}]")
(map meta (parse-string s))
;;=>
({:row 2, :col 2, :end-row 2, :end-col 8}
 {:row 3, :col 2, :end-row 3, :end-col 8})

(->> "{:a {:b {:c [a b c]}}}"
     parse-string
     (tree-seq coll? #(if (map? %) (vals %) %))
     (map meta))
;;=>
({:row 1, :col 1, :end-row 1, :end-col 23}
 {:row 1, :col 5, :end-row 1, :end-col 22}
 {:row 1, :col 9, :end-row 1, :end-col 21}
 {:row 1, :col 13, :end-row 1, :end-col 20}
 {:row 1, :col 14, :end-row 1, :end-col 15}
 {:row 1, :col 16, :end-row 1, :end-col 17}
 {:row 1, :col 18, :end-row 1, :end-col 19})

You can control on which elements locations get added using the :location? option.

Parser options

Edamame's API consists of two functions: parse-string which parses a the first form from a string and parse-string-all which parses all forms from a string. Both functions take the same options. See the docstring of parse-string for all the options.

Examples:

(parse-string "@foo" {:deref true})
;;=> (deref foo)

(parse-string "'bar" {:quote true})
;;=> (quote bar)

(parse-string "#(* % %1 %2)" {:fn true})
;;=> (fn [%1 %2] (* %1 %1 %2))

(parse-string "#=(+ 1 2 3)" {:read-eval true})
;;=> (read-eval (+ 1 2 3))

(parse-string "#\"foo\"" {:regex true})
;;=> #"foo"

(parse-string "#'foo" {:var true})
;;=> (var foo)

(parse-string "#(alter-var-root #'foo %)" {:all true})
;;=> (fn [%1] (alter-var-root (var foo) %1))

Note that standard behavior is overridable with functions:

(parse-string "#\"foo\"" {:regex #(list 're-pattern %)})
(re-pattern "foo")

Clojure defaults

The closest defaults to how Clojure reads code:

{:all true
 :row-key :line
 :col-key :column
 :end-location false
 :location? seq?}

Reader conditionals

Process reader conditionals:

(parse-string "[1 2 #?@(:cljs [3 4])]" {:features #{:cljs} :read-cond :allow})
;;=> [1 2 3 4]

(parse-string "[1 2 #?@(:cljs [3 4])]" {:features #{:cljs} :read-cond :preserve})
;;=> [1 2 #?@(:cljs [3 4])]

(let [res (parse-string "#?@(:bb 1 :clj 2)" {:read-cond identity})]
  (prn res) (prn (meta res)))
;;=> (:bb 1 :clj 2)
;;=> {:row 1, :col 1, :end-row 1, :end-col 18, :edamame/read-cond-splicing true}

Auto-resolve

Auto-resolve keywords:

(parse-string "[::foo ::str/foo]" {:auto-resolve '{:current user str clojure.string}})
;;=> [:user/foo :clojure.string/foo]

If you don't care much about the exact value of the keyword, but just want to parse something:

(parse-string "[::foo ::str/foo]" {:auto-resolve name})
;;=> [:current/foo :str/foo]

To create options from a namespace in the process where edamame is called from:

(defn auto-resolves [ns]
  (as-> (ns-aliases ns) $
    (assoc $ :current (ns-name *ns*))
    (zipmap (keys $)
            (map ns-name (vals $)))))

(require '[clojure.string :as str]) ;; create example alias

(auto-resolves *ns*) ;;=> {str clojure.string, :current user}

(parse-string "[::foo ::str/foo]" {:auto-resolve (auto-resolves *ns*)})
;;=> [:user/foo :clojure.string/foo]

To auto-resolve keywords from the running Clojure environment:

(require '[clojure.test :as t])
(e/parse-string "::t/foo" {:auto-resolve (fn [x] (if (= :current x) *ns* (get (ns-aliases *ns*) x)))})
:clojure.test/foo

:auto-resolve-ns

To auto-magically resolve keywords based on the ns form, use :auto-resolve-ns true:

(= '[(ns foo (:require [clojure.set :as set])) :clojure.set/foo]
    (parse-string-all "(ns foo (:require [clojure.set :as set])) ::set/foo"
                      {:auto-resolve-ns true}))

(def rdr (p/reader "(ns foo (:require [clojure.set :as set])) ::set/foo"))
(def opts (p/normalize-opts {:auto-resolve-ns true}))
(= (ns foo (:require [clojure.set :as set])) (p/parse-next rdr opts))
(= :clojure.set/foo (p/parse-next rdr opts))

Syntax-quote

Syntax quoting can be enabled using the :syntax-quote option. Symbols are resolved to fully qualified symbols using :resolve-symbol which is set to identity by default:

(parse-string "`(+ 1 2 3 ~x ~@y)" {:syntax-quote true})
;;=> (clojure.core/sequence (clojure.core/seq (clojure.core/concat (clojure.core/list (quote +)) (clojure.core/list 1) (clojure.core/list 2) (clojure.core/list 3) (clojure.core/list x) y)))

(parse-string "`(+ 1 2 3 ~x ~@y)" {:syntax-quote {:resolve-symbol #(symbol "user" (name %))}})
;;=> (clojure.core/sequence (clojure.core/seq (clojure.core/concat (clojure.core/list (quote user/+)) (clojure.core/list 1) (clojure.core/list 2) (clojure.core/list 3) (clojure.core/list x) y)))

To resolve symbols in syntax quote from the running Clojure environment:

(require '[clojure.tools.reader :refer [resolve-symbol]])

(require '[clojure.test :as t])
(e/parse-string "`t/run-tests" {:syntax-quote {:resolve-symbol resolve-symbol}})
;;=> (quote clojure.test/run-tests)

Data readers

Passing data readers:

(parse-string "#js [1 2 3]" {:readers {'js (fn [v] (list 'js v))}})
(js [1 2 3])

Preserve order of map and set keys

To preserve order of map and set keys, you can use the :map and :set options:

(require '[edamame.core :as e])
(require '[flatland.ordered.map :as m])
(e/parse-string "{:a 1}" {:map m/ordered-map}) ;;=> #ordered/map ([:a 1])
(require '[clojure.data.json :as j])
(j/write-str (e/parse-string "{:a 1 :b 2 :c 3 :d 4 :e 5 :f 6 :g 7 :h 8 :i 9 :j 10 :k 11 :l 12}" {:map m/ordered-map}))
;;=> "{\"a\":1,\"b\":2,\"c\":3,\"d\":4,\"e\":5,\"f\":6,\"g\":7,\"h\":8,\"i\":9,\"j\":10,\"k\":11,\"l\":12}"

Postprocess

Postprocess read values:

(defrecord Wrapper [obj loc])

(defn iobj? [x]
  #?(:clj (instance? clojure.lang.IObj x)
     :cljs (satisfies? IWithMeta x)))

(parse-string "[1]" {:postprocess
                       (fn [{:keys [:obj :loc]}]
                         (if (iobj? obj)
                           (vary-meta obj merge loc)
                           (->Wrapper obj loc)))})

[#user.Wrapper{:obj 1, :loc {:row 1, :col 2, :end-row 1, :end-col 3}}]

This allows you to preserve metadata for objects that do not support carrying metadata. When you use a :postprocess function, it is your responsibility to attach location metadata.

Fix incomplete expressions

Edamame exposes information via ex-data in an exception in case of unmatched delimiters. This can be used to fix incomplete expressions:

(def incomplete "{:a (let [x 5")

(defn fix-expression [expr]
  (try (when (parse-string expr)
         expr)
       (catch clojure.lang.ExceptionInfo e
         (if-let [expected-delimiter (:edamame/expected-delimiter (ex-data e))]
           (fix-expression (str expr expected-delimiter))
           (throw e)))))

(fix-expression incomplete) ;; => "{:a (let [x 5])}"

Test

For the node tests, ensure clojure is installed as a command line tool as shown here. For the JVM tests you will require leiningen to be installed. Then run the following:

script/test/jvm
script/test/node
script/test/all

Credits

The code is largely inspired by rewrite-clj and derived projects.

License

Copyright © 2019-2022 Michiel Borkent

Distributed under the Eclipse Public License 1.0. This project contains code from Clojure and ClojureScript which are also licensed under the EPL 1.0. See LICENSE.