Data-driven Schemas for Clojure/Script and babashka.
STATUS: well matured alpha
Presentations:
Try the online demo, see also some 3rd Party Libraries.
We are building dynamic multi-tenant systems where data models should be first-class: they should drive the runtime value transformations, forms and processes. We should be able to edit the models at runtime, persist them and load them back from a database and over the wire, for both Clojure and ClojureScript. Think of JSON Schema, but for Clojure/Script.
Hasn't the problem been solved (many times) already?
There is Schema, which is an awesome, proven and collaborative open-source project, and we absolutely love it. We still use it in many of our projects. The sad part: serializing & de-serializing schemas is non-trivial and there is no proper support on branching.
Spec is the de facto data specification library for Clojure. It has many great ideas, but it is opinionated with macros, global registry, and it doesn't have any support for runtime transformations. Spec-tools was created to "fix" some of the things, but after five years of developing it, it's still a kind of hack and not fun to maintain.
So, we decided to spin out our own library, which would do all the things we feel is important for dynamic system development. It's based on the best parts of the existing libraries and several project-specific tools we have done over the years.
If you have expectations (of others) that aren't being met, those expectations are your own responsibility. You are responsible for your own needs. If you want things, make them.
Malli requires Clojure 1.10+ and is tested against 1.10 and 1.11.
Malli supports Vector, Map and Lite syntaxes.
The default syntax uses vectors, inspired by hiccup:
type
[type & children]
[type properties & children]
Examples:
;; just a type (String)
:string
;; type with properties
[:string {:min 1, :max 10}]
;; type with properties and children
[:tuple {:title "location"} :double :double]
;; a function schema of :int -> :int
[:=> [:cat :int] :int]
Usage:
(require '[malli.core :as m])
(def non-empty-string
(m/schema [:string {:min 1}]))
(m/schema? non-empty-string)
; => true
(m/validate non-empty-string "")
; => false
(m/validate non-empty-string "kikka")
; => true
(m/form non-empty-string)
; => [:string {:min 1}]
Alternative map-syntax, similar to cljfx:
NOTE: For now, Map syntax in considered as internal, so don't use it as a database persistency model.
;; just a type (String)
{:type :string}
;; type with properties
{:type :string
:properties {:min 1, :max 10}
;; type with properties and children
{:type :tuple
:properties {:title "location"}
:children [{:type :double}
{:type :double}]}
;; a function schema of :int -> :int
{:type :=>
:input {:type :cat, :children [{:type :int}]}
:output :int}
Usage:
(def non-empty-string
(m/from-ast {:type :string
:properties {:min 1}}))
(m/schema? non-empty-string)
; => true
(m/validate non-empty-string "")
; => false
(m/validate non-empty-string "kikka")
; => true
(m/ast non-empty-string)
; => {:type :string,
; :properties {:min 1}}
Map-syntax is also called the Schema AST.
Malli started with just the Vector syntax. It's really powerful and relatively easy to read, but not optimal for all use cases.
We introduced Map Syntax as we found out that the overhead of parsing large amount of vector-syntaxes can be a deal-breaker when running on slow single-threaded environments like Javascript on mobile phones. Map-syntax allows lazy and parseless Schema Creation.
We added Lite Syntax for simplified schema creation for special cases, like to be used with reitit coercion and for easy migration from data-specs.
Validating values against a schema:
;; with schema instances
(m/validate (m/schema :int) 1)
; => true
;; with vector syntax
(m/validate :int 1)
; => true
(m/validate :int "1")
; => false
(m/validate [:and :int [:> 6]] 7)
; => true
(m/validate [:qualified-keyword {:namespace :aaa}] :aaa/bbb)
; => true
;; optimized (pure) validation function for best performance
(def valid?
(m/validator
[:map
[:x :boolean]
[:y {:optional true} :int]
[:z :string]]))
(valid? {:x true, :z "kikka"})
; => true
Schemas can have properties:
(def Age
[:and
{:title "Age"
:description "It's an age"
:json-schema/example 20}
:int [:> 18]])
(m/properties Age)
; => {:title "Age"
; :description "It's an age"
; :json-schema/example 20}
Maps are open by default:
(m/validate
[:map [:x :int]]
{:x 1, :extra "key"})
; => true
Maps can be closed with :closed
property:
(m/validate
[:map {:closed true} [:x :int]]
{:x 1, :extra "key"})
; => false
Maps keys are not limited to keywords:
(m/validate
[:map
["status" [:enum "ok"]]
[1 :any]
[nil :any]
[::a :string]]
{"status" "ok"
1 'number
nil :yay
::a "properly awesome"})
; => true
Most core-predicates are mapped to Schemas:
(m/validate string? "kikka")
; => true
See the full list of default schemas.
You can also use decomplected maps keys and values using registry references. References must be either qualified keywords or strings.
(m/validate
[:map {:registry {::id int?
::country string?}}
::id
[:name string?]
[::country {:optional true}]]
{::id 1
:name "kikka"})
; => true
Other times, we use a map as a homogeneous index. In this case, all our key-value
pairs have the same type. For this use case, we can use the :map-of
schema.
(m/validate
[:map-of :string [:map [:lat number?] [:long number?]]]
{"oslo" {:lat 60 :long 11}
"helsinki" {:lat 60 :long 24}})
;; => true
You can use :sequential
for any homogeneous Clojure sequence, :vector
for vectors and :set
for sets.
(m/validate [:sequential any?] (list "this" 'is :number 42))
;; => true
(m/validate [:vector int?] [1 2 3])
;; => true
(m/validate [:vector int?] (list 1 2 3))
;; => false
A :tuple
describes a fixed length Clojure vector of heterogeneous elements:
(m/validate [:tuple keyword? string? number?] [:bing "bang" 42])
;; => true
Malli also supports sequence regexes like Seqexp and Spec.
The supported operators are :cat
& :catn
for concatenation / sequencing
(m/validate [:cat string? int?] ["foo" 0]) ; => true
(m/validate [:catn [:s string?] [:n int?]] ["foo" 0]) ; => true
:alt
& :altn
for alternatives
(m/validate [:alt keyword? string?] ["foo"]) ; => true
(m/validate [:altn [:kw keyword?] [:s string?]] ["foo"]) ; => true
and :?
, :*
, :+
& :repeat
for repetition:
(m/validate [:? int?] []) ; => true
(m/validate [:? int?] [1]) ; => true
(m/validate [:? int?] [1 2]) ; => false
(m/validate [:* int?] []) ; => true
(m/validate [:* int?] [1 2 3]) ; => true
(m/validate [:+ int?] []) ; => false
(m/validate [:+ int?] [1]) ; => true
(m/validate [:+ int?] [1 2 3]) ; => true
(m/validate [:repeat {:min 2, :max 4} int?] [1]) ; => false
(m/validate [:repeat {:min 2, :max 4} int?] [1 2]) ; => true
(m/validate [:repeat {:min 2, :max 4} int?] [1 2 3 4]) ; => true (:max is inclusive, as elsewhere in Malli)
(m/validate [:repeat {:min 2, :max 4} int?] [1 2 3 4 5]) ; => false
:catn
and :altn
allow naming the subsequences / alternatives
(m/explain
[:* [:catn [:prop string?] [:val [:altn [:s string?] [:b boolean?]]]]]
["-server" "foo" "-verbose" 11 "-user" "joe"])
;; => {:schema [:* [:map [:prop string?] [:val [:map [:s string?] [:b boolean?]]]]],
;; :value ["-server" "foo" "-verbose" 11 "-user" "joe"],
;; :errors ({:path [0 :val :s], :in [3], :schema string?, :value 11}
;; {:path [0 :val :b], :in [3], :schema boolean?, :value 11})}
while :cat
and :alt
just use numeric indices for paths:
(m/explain
[:* [:cat string? [:alt string? boolean?]]]
["-server" "foo" "-verbose" 11 "-user" "joe"])
;; => {:schema [:* [:cat string? [:alt string? boolean?]]],
;; :value ["-server" "foo" "-verbose" 11 "-user" "joe"],
;; :errors ({:path [0 1 0], :in [3], :schema string?, :value 11}
;; {:path [0 1 1], :in [3], :schema boolean?, :value 11})}
As all these examples show, the "seqex" operators take any non-seqex child schema to
mean a sequence of one element that matches that schema. To force that behaviour for
a seqex child :schema
can be used:
(m/validate
[:cat [:= :names] [:schema [:* string?]] [:= :nums] [:schema [:* number?]]]
[:names ["a" "b"] :nums [1 2 3]])
; => true
;; whereas
(m/validate
[:cat [:= :names] [:* string?] [:= :nums] [:* number?]]
[:names "a" "b" :nums 1 2 3])
; => true
Although a lot of effort has gone into making the seqex implementation fast
(require '[clojure.spec.alpha :as s])
(require '[criterium.core :as cc])
(let [valid? (partial s/valid? (s/* int?))]
(cc/quick-bench (valid? (range 10)))) ; Execution time mean : 27µs
(let [valid? (m/validator [:* int?])]
(cc/quick-bench (valid? (range 10)))) ; Execution time mean : 2.7µs
it is always better to use less general tools whenever possible:
(let [valid? (partial s/valid? (s/coll-of int?))]
(cc/quick-bench (valid? (range 10)))) ; Execution time mean : 1.8µs
(let [valid? (m/validator [:sequential int?])]
(cc/quick-bench (valid? (range 10)))) ; Execution time mean : 0.12µs
Using a predicate:
(m/validate string? "kikka")
Using :string
Schema:
(m/validate :string "kikka")
;; => true
(m/validate [:string {:min 1, :max 4}] "")
;; => false
Using regular expressions:
(m/validate #"a+b+c+" "abbccc")
;; => true
;; :re with string
(m/validate [:re ".{3,5}"] "abc")
;; => true
;; :re with regex
(m/validate [:re #".{3,5}"] "abc")
;; => true
;; NB: re-find semantics
(m/validate [:re #"\d{4}"] "1234567")
;; => true
;; anchor with ^...$ if you want to strictly match the whole string
(m/validate [:re #"^\d{4}$"] "1234567")
;; => false
Use :maybe
to express that an element should match some schema OR be nil
:
(m/validate [:maybe string?] "bingo")
;; => true
(m/validate [:maybe string?] nil)
;; => true
(m/validate [:maybe string?] :bingo)
;; => false
:fn
allows any predicate function to be used:
(def my-schema
[:and
[:map
[:x int?]
[:y int?]]
[:fn (fn [{:keys [x y]}] (> x y))]])
(m/validate my-schema {:x 1, :y 0})
; => true
(m/validate my-schema {:x 1, :y 2})
; => false
Detailed errors with m/explain
:
(def Address
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:map
[:street string?]
[:city string?]
[:zip int?]
[:lonlat [:tuple double? double?]]]]])
(m/explain
Address
{:id "Lillan"
:tags #{:artesan :coffee :hotel}
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322, 23.7854658]}})
; => nil
(m/explain
Address
{:id "Lillan"
:tags #{:artesan "coffee" :garden}
:address {:street "Ahlmanintie 29"
:zip 33100
:lonlat [61.4858322, nil]}})
;{:schema [:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address [:map
; [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]]]],
; :value {:id "Lillan",
; :tags #{:artesan :garden "coffee"},
; :address {:street "Ahlmanintie 29"
; :zip 33100
; :lonlat [61.4858322 nil]}},
; :errors ({:path [:tags 0]
; :in [:tags 0]
; :schema keyword?
; :value "coffee"}
; {:path [:address :city],
; :in [:address :city],
; :schema [:map
; [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]],
; :type :malli.core/missing-key}
; {:path [:address :lonlat 1]
; :in [:address :lonlat 1]
; :schema double?
; :value nil})}
Note! If you need error messages that serialize neatly to EDN/JSON, use mu/explain-data
instead.
Explain results can be humanized with malli.error/humanize
:
(require '[malli.error :as me])
(-> Address
(m/explain
{:id "Lillan"
:tags #{:artesan "coffee" :garden}
:address {:street "Ahlmanintie 29"
:zip 33100
:lonlat [61.4858322, nil]}})
(me/humanize))
;{:tags #{["should be a keyword"]}
; :address {:city ["missing required key"]
; :lonlat [nil ["should be a double"]]}}
Error messages can be customized with :error/message
and :error/fn
properties:
(-> [:map
[:id int?]
[:size [:enum {:error/message "should be: S|M|L"}
"S" "M" "L"]]
[:age [:fn {:error/fn '(fn [{:keys [value]} _] (str value ", should be > 18"))}
'(fn [x] (and (int? x) (> x 18)))]]]
(m/explain {:size "XL", :age 10})
(me/humanize
{:errors (-> me/default-errors
(assoc ::m/missing-key {:error/fn (fn [{:keys [in]} _] (str "missing key " (last in)))}))}))
;{:id ["missing key :id"]
; :size ["should be: S|M|L"]
; :age ["10, should be > 18"]}
Messages can be localized:
(-> [:map
[:id int?]
[:size [:enum {:error/message {:en "should be: S|M|L"
:fi "pitäisi olla: S|M|L"}}
"S" "M" "L"]]
[:age [:fn {:error/fn {:en '(fn [{:keys [value]} _] (str value ", should be > 18"))
:fi '(fn [{:keys [value]} _] (str value ", pitäisi olla > 18"))}}
'(fn [x] (and (int? x) (> x 18)))]]]
(m/explain {:size "XL", :age 10})
(me/humanize
{:locale :fi
:errors (-> me/default-errors
(assoc-in ['int? :error-message :fi] "pitäisi olla numero")
(assoc ::m/missing-key {:error/fn {:en '(fn [{:keys [in]} _] (str "missing key " (last in)))
:fi '(fn [{:keys [in]} _] (str "puuttuu avain " (last in)))}}))}))
;{:id ["puuttuu avain :id"]
; :size ["pitäisi olla: S|M|L"]
; :age ["10, pitäisi olla > 18"]}
Top-level humanized map-errors are under :malli/error
:
(-> [:and [:map
[:password string?]
[:password2 string?]]
[:fn {:error/message "passwords don't match"}
'(fn [{:keys [password password2]}]
(= password password2))]]
(m/explain {:password "secret"
:password2 "faarao"})
(me/humanize))
; {:malli/error ["passwords don't match"]}
Errors can be targeted using :error/path
property:
(-> [:and [:map
[:password string?]
[:password2 string?]]
[:fn {:error/message "passwords don't match"
:error/path [:password2]}
'(fn [{:keys [password password2]}]
(= password password2))]]
(m/explain {:password "secret"
:password2 "faarao"})
(me/humanize))
; {:password2 ["passwords don't match"]}
By default, only direct erroneous schema properties are used:
(-> [:map
[:foo {:error/message "entry-failure"} :int]] ;; here, :int fails, no error props
(m/explain {:foo "1"})
(me/humanize))
; => {:foo ["should be an integer"]}
Looking up humanized errors from parent schemas with custom :resolve
(BETA, subject to change):
(-> [:map
[:foo {:error/message "entry-failure"} :int]]
(m/explain {:foo "1"})
(me/humanize {:resolve me/-resolve-root-error}))
; => {:foo ["entry-failure"]}
For closed schemas, key spelling can be checked with:
(-> [:map [:address [:map [:street string?]]]]
(mu/closed-schema)
(m/explain
{:name "Lie-mi"
:address {:streetz "Hämeenkatu 14"}})
(me/with-spell-checking)
(me/humanize))
;{:address {:street ["missing required key"]
; :streetz ["should be spelled :street"]}
; :name ["disallowed key"]}
For pretty development-time error printing, try malli.dev.pretty/explain
(require '[malli.transform :as mt])
Two-way schema-driven value transformations with m/decode
and m/encode
using a m/Transformer
.
Default Transformers include: string-transformer
, json-transformer
, strip-extra-keys-transformer
, default-value-transformer
and key-transformer
.
NOTE: the included transformers are best-effort, i.e. they won't throw on bad input, they will just pass the input value through unchanged. You should make sure your schema validation catches these non-transformed values. Custom transformers should follow the same idiom.
(m/decode int? "42" mt/string-transformer)
; 42
(m/encode int? 42 mt/string-transformer)
; "42"
Transformations are recursive:
(m/decode
Address
{:id "Lillan",
:tags ["coffee" "artesan" "garden"],
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322 23.7854658]}}
mt/json-transformer)
;{:id "Lillan",
; :tags #{:coffee :artesan :garden},
; :address {:street "Ahlmanintie 29"
; :city "Tampere"
; :zip 33100
; :lonlat [61.4858322 23.7854658]}}
Transform map keys:
(m/encode
Address
{:id "Lillan",
:tags ["coffee" "artesan" "garden"],
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322 23.7854658]}}
(mt/key-transformer {:encode name}))
;{"id" "Lillan",
; "tags" ["coffee" "artesan" "garden"],
; "address" {"street" "Ahlmanintie 29"
; "city" "Tampere"
; "zip" 33100
; "lonlat" [61.4858322 23.7854658]}}
Transformers can be composed with mt/transformer
:
(def strict-json-transformer
(mt/transformer
mt/strip-extra-keys-transformer
mt/json-transformer))
(m/decode
Address
{:id "Lillan",
:EVIL "LYN"
:tags ["coffee" "artesan" "garden"],
:address {:street "Ahlmanintie 29"
:DARK "ORKO"
:city "Tampere"
:zip 33100
:lonlat [61.4858322 23.7854658]}}
strict-json-transformer)
;{:id "Lillan",
; :tags #{:coffee :artesan :garden},
; :address {:street "Ahlmanintie 29"
; :city "Tampere"
; :zip 33100
; :lonlat [61.4858322 23.7854658]}}
Schema properties can be used to override default transformations:
(m/decode
[string? {:decode/string 'str/upper-case}]
"kerran" mt/string-transformer)
; => "KERRAN"
This works too:
(m/decode
[string? {:decode {:string 'str/upper-case}}]
"kerran" mt/string-transformer)
; => "KERRAN"
Decoders and encoders as interceptors (with :enter
and :leave
stages):
(m/decode
[string? {:decode/string {:enter 'str/upper-case}}]
"kerran" mt/string-transformer)
; => "KERRAN"
(m/decode
[string? {:decode/string {:enter '#(str "olipa_" %)
:leave '#(str % "_avaruus")}}]
"kerran" mt/string-transformer)
; => "olipa_kerran_avaruus"
To access Schema (and options) use :compile
:
(m/decode
[int? {:math/multiplier 10
:decode/math {:compile (fn [schema _]
(let [multiplier (:math/multiplier (m/properties schema))]
(fn [x] (* x multiplier))))}}]
12
(mt/transformer {:name :math}))
; => 120
Going crazy:
(m/decode
[:map
{:decode/math {:enter '#(update % :x inc)
:leave '#(update % :x (partial * 2))}}
[:x [int? {:decode/math {:enter '(partial + 2)
:leave '(partial * 3)}}]]]
{:x 1}
(mt/transformer {:name :math}))
; => {:x 24}
The m/encode
and m/decode
functions work on clojure data. To go
from clojure data to JSON, you need a JSON library like
jsonista. Additionally, since
m/decode
doesn't check the schema, you need to run m/validate
(or
m/explain
) if you want to make sure your data conforms to your
schema.
To JSON:
(def Tags
(m/schema [:map
{:closed true}
[:tags [:set :keyword]]]))
(jsonista.core/write-value-as-string
(m/encode Tags
{:tags #{:bar :quux}}
mt/json-transformer))
; => "{\"tags\":[\"bar\",\"quux\"]}"
From JSON without validation:
(m/decode Tags
(jsonista.core/read-value "{\"tags\":[\"bar\",[\"quux\"]]}"
jsonista.core/keyword-keys-object-mapper)
mt/json-transformer)
; => {:tags #{:bar ["quux"]}}
From JSON with validation:
(m/explain Tags
(m/decode Tags
(jsonista.core/read-value "{\"tags\":[\"bar\",[\"quux\"]]}"
jsonista.core/keyword-keys-object-mapper)
mt/json-transformer))
; => {:schema [:map {:closed true} [:tags [:set :keyword]]],
; :value {:tags #{:bar ["quux"]}},
; :errors ({:path [:tags 0], :in [:tags ["quux"]], :schema :keyword, :value ["quux"]})}
(m/validate Tags
(m/decode Tags
(jsonista.core/read-value "{\"tags\":[\"bar\",\"quux\"]}" ; <- note! no error
jsonista.core/keyword-keys-object-mapper)
mt/json-transformer))
; => true
For performance, it's best to prebuild the validator, decoder and explainer:
(def validate-Tags (m/validator Tags))
(def decode-Tags (m/decoder Tags mt/json-transformer))
(-> (jsonista.core/read-value "{\"tags\":[\"bar\",\"quux\"]}"
jsonista.core/keyword-keys-object-mapper)
decode-Tags
validate-Tags)
; => true
Applying default values:
(m/decode [:and {:default 42} int?] nil mt/default-value-transformer)
; => 42
With custom key and type defaults:
(m/decode
[:map
[:user [:map
[:name :string]
[:description {:ui/default "-"} :string]]]]
nil
(mt/default-value-transformer
{:key :ui/default
:defaults {:map (constantly {})
:string (constantly "")}}))
; => {:user {:name "", :description "-"}}
With custom function:
(m/decode
[:map
[:os [:string {:property "os.name"}]]
[:timezone [:string {:property "user.timezone"}]]]
{}
(mt/default-value-transformer
{:key :property
:default-fn (fn [_ x] (System/getProperty x))}))
; => {:os "Mac OS X", :timezone "Europe/Helsinki"}
Optional Keys are not added by default:
(m/decode
[:map
[:name [:string {:default "kikka"}]]
[:description {:optional true} [:string {:default "kikka"}]]]
{}
(mt/default-value-transformer))
; => {:name "kikka"}
Adding optional keys too via ::mt/add-optional-keys
option:
(m/decode
[:map
[:name [:string {:default "kikka"}]]
[:description {:optional true} [:string {:default "kikka"}]]]
{}
(mt/default-value-transformer {::mt/add-optional-keys true}))
; => {:name "kikka", :description "kikka"}
Single sweep of defaults & string encoding:
(m/encode
[:map {:default {}}
[:a [int? {:default 1}]]
[:b [:vector {:default [1 2 3]} int?]]
[:c [:map {:default {}}
[:x [int? {:default 42}]]
[:y int?]]]
[:d [:map
[:x [int? {:default 42}]]
[:y int?]]]
[:e int?]]
nil
(mt/transformer
mt/default-value-transformer
mt/string-transformer))
;{:a "1"
; :b ["1" "2" "3"]
; :c {:x "42"}}
(require '[malli.util :as mu])
Updating Schema properties:
(mu/update-properties [:vector int?] assoc :min 1)
; => [:vector {:min 1} int?]
Lifted clojure.core
function to work with schemas: select-keys
, dissoc
, get
, assoc
, update
, get-in
, assoc-in
, update-in
(mu/get-in Address [:address :lonlat])
; => [:tuple double? double?]
(mu/update-in Address [:address] mu/assoc :country [:enum "fi" "po"])
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]
; [:country [:enum "fi" "po"]]]]]
(-> Address
(mu/dissoc :address)
(mu/update-properties assoc :title "Address"))
;[:map {:title "Address"}
; [:id string?]
; [:tags [:set keyword?]]]
Making keys optional or required:
(mu/optional-keys [:map [:x int?] [:y int?]])
;[:map
; [:x {:optional true} int?]
; [:y {:optional true} int?]]
(mu/required-keys [:map [:x {:optional true} int?] [:y int?]])
;[:map
; [:x int?]
; [:y int?]]
Closing and opening all :map
schemas recursively:
(def abcd
[:map {:title "abcd"}
[:a int?]
[:b {:optional true} int?]
[:c [:map
[:d int?]]]])
(mu/closed-schema abcd)
;[:map {:title "abcd", :closed true}
; [:a int?]
; [:b {:optional true} int?]
; [:c [:map {:closed true}
; [:d int?]]]]
(-> abcd
mu/closed-schema
mu/open-schema)
;[:map {:title "abcd"}
; [:a int?]
; [:b {:optional true} int?]
; [:c [:map
; [:d int?]]]]
Merging Schemas (last value wins):
(mu/merge
[:map
[:name string?]
[:description string?]
[:address
[:map
[:street string?]
[:country [:enum "finland" "poland"]]]]]
[:map
[:description {:optional true} string?]
[:address
[:map
[:country string?]]]])
;[:map
; [:name string?]
; [:description {:optional true} string?]
; [:address [:map
; [:street string?]
; [:country string?]]]]
With :and
, first child is used in merge:
(mu/merge
[:and {:type "entity"}
[:map {:title "user"}
[:name :string]]
map?]
[:map {:description "aged"} [:age :int]])
;[:and {:type "entity"}
; [:map {:title "user", :description "aged"}
; [:name :string]
; [:age :int]]
; map?]
Schema unions (merged values of both schemas are valid for union schema):
(mu/union
[:map
[:name string?]
[:description string?]
[:address
[:map
[:street string?]
[:country [:enum "finland" "poland"]]]]]
[:map
[:description {:optional true} string?]
[:address
[:map
[:country string?]]]])
;[:map
; [:name string?]
; [:description {:optional true} string?]
; [:address [:map
; [:street string?]
; [:country [:or [:enum "finland" "poland"] string?]]]]]
Adding generated example values to Schemas:
(m/walk
[:map
[:name string?]
[:description string?]
[:address
[:map
[:street string?]
[:country [:enum "finland" "poland"]]]]]
(m/schema-walker
(fn [schema]
(mu/update-properties schema assoc :examples (mg/sample schema {:size 2, :seed 20})))))
;[:map
; {:examples ({:name "", :description "", :address {:street "", :country "poland"}}
; {:name "W", :description "x", :address {:street "8", :country "finland"}})}
; [:name [string? {:examples ("" "")}]]
; [:description [string? {:examples ("" "")}]]
; [:address
; [:map
; {:examples ({:street "", :country "finland"} {:street "W", :country "poland"})}
; [:street [string? {:examples ("" "")}]]
; [:country [:enum {:examples ("finland" "poland")} "finland" "poland"]]]]]
Finding first value (prewalk):
(mu/find-first
[:map
[:x int?]
[:y [:vector [:tuple
[:or [:and {:salaisuus "turvassa"} boolean?] int?]
[:schema {:salaisuus "vaarassa"} false?]]]]
[:z [:string {:salaisuus "piilossa"}]]]
(fn [schema _ _]
(-> schema m/properties :salaisuus)))
; => "turvassa"
Finding all subschemas with paths, retaining order:
(def Schema
(m/schema
[:maybe
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:and
[:map
[:street {:optional true} string?]
[:lonlat {:optional true} [:tuple double? double?]]]
[:fn '(fn [{:keys [street lonlat]}] (or street lonlat))]]]]]))
(mu/subschemas Schema)
;[{:path [], :in [], :schema [:maybe
; [:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:and
; [:map
; [:street {:optional true} string?]
; [:lonlat {:optional true} [:tuple double? double?]]]
; [:fn (fn [{:keys [street lonlat]}] (or street lonlat))]]]]]}
; {:path [0], :in [], :schema [:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:and
; [:map
; [:street {:optional true} string?]
; [:lonlat {:optional true} [:tuple double? double?]]]
; [:fn (fn [{:keys [street lonlat]}] (or street lonlat))]]]]}
; {:path [0 :id], :in [:id], :schema string?}
; {:path [0 :tags], :in [:tags], :schema [:set keyword?]}
; {:path [0 :tags :malli.core/in], :in [:tags :malli.core/in], :schema keyword?}
; {:path [0 :address], :in [:address], :schema [:and
; [:map
; [:street {:optional true} string?]
; [:lonlat {:optional true} [:tuple double? double?]]]
; [:fn (fn [{:keys [street lonlat]}] (or street lonlat))]]}
; {:path [0 :address 0], :in [:address], :schema [:map
; [:street {:optional true} string?]
; [:lonlat {:optional true} [:tuple double? double?]]]}
; {:path [0 :address 0 :street], :in [:address :street], :schema string?}
; {:path [0 :address 0 :lonlat], :in [:address :lonlat], :schema [:tuple double? double?]}
; {:path [0 :address 0 :lonlat 0], :in [:address :lonlat 0], :schema double?}
; {:path [0 :address 0 :lonlat 1], :in [:address :lonlat 1], :schema double?}
; {:path [0 :address 1], :in [:address], :schema [:fn (fn [{:keys [street lonlat]}] (or street lonlat))]}]
Collecting unique value paths and their schema paths:
(->> Schema
(mu/subschemas)
(mu/distinct-by :id)
(mapv (juxt :in :path)))
;[[[] []]
; [[] [0]]
; [[:id] [0 :id]]
; [[:tags] [0 :tags]]
; [[:tags :malli.core/in] [0 :tags :malli.core/in]]
; [[:address] [0 :address]]
; [[:address] [0 :address 0]]
; [[:address :street] [0 :address 0 :street]]
; [[:address :lonlat] [0 :address 0 :lonlat]]
; [[:address :lonlat 0] [0 :address 0 :lonlat 0]]
; [[:address :lonlat 1] [0 :address 0 :lonlat 1]]
; [[:address] [0 :address 1]]]
Schema paths can be converted into value paths:
(mu/get-in Schema [0 :address 0 :lonlat])
; => [:tuple double? double?]
(mu/path->in Schema [0 :address 0 :lonlat])
; => [:address :lonlat]
and back, returning all paths:
(mu/in->paths Schema [:address :lonlat])
; => [[0 :address 0 :lonlat]]
There are also declarative versions of schema transforming utilities in malli.util/schemas
. These include :merge
, :union
and :select-keys
:
(def registry (merge (m/default-schemas) (mu/schemas)))
(def Merged
(m/schema
[:merge
[:map [:x :string]]
[:map [:y :int]]]
{:registry registry}))
Merged
;[:merge
; [:map [:x :string]]
; [:map [:y :int]]]
(m/deref Merged)
;[:map
; [:x :string]
; [:y :int]]
(m/validate Merged {:x "kikka", :y 6})
; => true
Writing and Reading schemas as EDN, no eval
needed.
(require '[malli.edn :as edn])
(-> [:and
[:map
[:x int?]
[:y int?]]
[:fn '(fn [{:keys [x y]}] (> x y))]]
(edn/write-string)
(doto prn) ; => "[:and [:map [:x int?] [:y int?]] [:fn (fn [{:keys [x y]}] (> x y))]]"
(edn/read-string)
(doto (-> (m/validate {:x 0, :y 1}) prn)) ; => false
(doto (-> (m/validate {:x 2, :y 1}) prn))) ; => true
;[:and
; [:map
; [:x int?]
; [:y int?]]
; [:fn (fn [{:keys [x y]}] (> x y))]]
Closed dispatch with :multi
schema and :dispatch
property:
(m/validate
[:multi {:dispatch :type}
[:sized [:map [:type keyword?] [:size int?]]]
[:human [:map [:type keyword?] [:name string?] [:address [:map [:country keyword?]]]]]]
{:type :sized, :size 10})
; true
Default branch with ::m/default
:
(def valid?
(m/validator
[:multi {:dispatch :type}
["object" [:map-of :keyword :string]]
[::m/default :string]]))
(valid? {:type "object", :key "1", :value "100"})
; => true
(valid? "SUCCESS!")
; => true
(valid? :failure)
; => false
Any (serializable) function can be used for :dispatch
:
(m/validate
[:multi {:dispatch 'first}
[:sized [:tuple keyword? [:map [:size int?]]]]
[:human [:tuple keyword? [:map [:name string?] [:address [:map [:country keyword?]]]]]]]
[:human {:name "seppo", :address {:country :sweden}}])
; true
:dispatch
values should be decoded before actual values:
(m/decode
[:multi {:dispatch :type
:decode/string '#(update % :type keyword)}
[:sized [:map [:type [:= :sized]] [:size int?]]]
[:human [:map [:type [:= :human]] [:name string?] [:address [:map [:country keyword?]]]]]]
{:type "human"
:name "Tiina"
:age "98"
:address {:country "finland"
:street "this is an extra key"}}
(mt/transformer mt/strip-extra-keys-transformer mt/string-transformer))
;{:type :human
; :name "Tiina"
; :address {:country :finland}}
To create a recursive schema, introduce a local registry and wrap all recursive positions in the registry with :ref
. Now you may reference the recursive schemas in the body of the schema.
For example, here is a recursive schema using :schema
for singly-linked lists of positive integers:
(m/validate
[:schema {:registry {::cons [:maybe [:tuple pos-int? [:ref ::cons]]]}}
::cons]
[16 [64 [26 [1 [13 nil]]]]])
; => true
Without the :ref
keyword, malli eagerly expands the schema until a stack overflow error is thrown:
(m/validate
[:schema {:registry {::cons [:maybe [:tuple pos-int? ::cons]]}}
::cons]
[16 [64 [26 [1 [13 nil]]]]])
; StackOverflowError
Mutual recursion works too. Thanks to the :schema
construct, many schemas could be defined in the local registry, the top-level one being promoted by the :schema
second parameter:
(m/validate
[:schema {:registry {::ping [:maybe [:tuple [:= "ping"] [:ref ::pong]]]
::pong [:maybe [:tuple [:= "pong"] [:ref ::ping]]]}}
::ping]
["ping" ["pong" ["ping" ["pong" ["ping" nil]]]]])
; => true
Nested registries, the last definition wins:
(m/validate
[:schema {:registry {::ping [:maybe [:tuple [:= "ping"] [:ref ::pong]]]
::pong any?}} ;; effectively unreachable
[:schema {:registry {::pong [:maybe [:tuple [:= "pong"] [:ref ::ping]]]}}
::ping]]
["ping" ["pong" ["ping" ["pong" ["ping" nil]]]]])
; => true
Schemas can be used to generate values:
(require '[malli.generator :as mg])
;; random
(mg/generate keyword?)
; => :?
;; using seed
(mg/generate [:enum "a" "b" "c"] {:seed 42})
;; => "a"
;; using seed and size
(mg/generate pos-int? {:seed 10, :size 100})
;; => 55740
;; regexs work too (only clj and if [com.gfredericks/test.chuck "0.2.10"+] available)
(mg/generate
[:re #"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,63}$"]
{:seed 42, :size 10})
; => "CaR@MavCk70OHiX.yZ"
;; :gen/elements (note, are not validated)
(mg/generate
[:and {:gen/elements ["kikka" "kukka" "kakka"]} string?]
{:seed 10})
; => "kikka"
;; portable :gen/fmap
(mg/generate
[:and {:gen/fmap '(partial str "kikka_")} string?]
{:seed 10, :size 10})
;; => "kikka_WT3K0yax2"
;; :gen/schema
(mg/generate
[:any {:gen/schema [:int {:min 10, :max 20}]}]
{:seed 10})
; => 19
;; :gen/min & :gen/max for numbers and collections
(mg/generate
[:vector {:gen/min 4, :gen/max 4} :int] '
{:seed 1})
; => [-8522515 -1433 -1 1]
;; :gen/infinite? & :gen/NaN? for :double
(mg/generate
[:double {:gen/infinite? true, :gen/NaN? true}]
{:seed 1})
; => ##Inf
(require '[clojure.test.check.generators :as gen])
;; gen/gen (note, not serializable)
(mg/generate
[:sequential {:gen/gen (gen/list gen/neg-int)} int?]
{:size 42, :seed 42})
; => (-37 -13 -13 -24 -20 -11 -34 -40 -22 0 -10)
Generated values are valid:
(mg/generate Address {:seed 123, :size 4})
;{:id "H7",
; :tags #{:v?.w.t6!.QJYk-/-?s*4
; :_7U
; :QdG/Xi8J
; :*Q-.p*8*/n-J9u}
; :address {:street "V9s"
; :city ""
; :zip 3
; :lonlat [-2.75 -0.625]}}
(m/validate Address (mg/generate Address))
; => true
Sampling values:
;; sampling
(mg/sample [:and int? [:> 10] [:< 100]] {:seed 123})
; => (25 39 51 13 53 43 57 15 26 27)
Integration with test.check:
(require '[clojure.test.check.generators :as gen])
(gen/sample (mg/generator pos-int?))
; => (2 1 2 2 2 2 8 1 55 83)
Inspired by F# Type providers:
(require '[malli.provider :as mp])
(def samples
[{:id "Lillan"
:tags #{:artesan :coffee :hotel}
:address {:street "Ahlmanintie 29"
:city "Tampere"
:zip 33100
:lonlat [61.4858322, 23.7854658]}}
{:id "Huber",
:description "Beefy place"
:tags #{:beef :wine :beer}
:address {:street "Aleksis Kiven katu 13"
:city "Tampere"
:zip 33200
:lonlat [61.4963599 23.7604916]}}])
(mp/provide samples)
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map
; [:street string?]
; [:city string?]
; [:zip number?]
; [:lonlat [:vector double?]]]]
; [:description {:optional true} string?]]
All samples are valid against the inferred schema:
(every? (partial m/validate (mp/provide samples)) samples)
; => true
For better performance, use mp/provider
:
;; 5ms
(p/bench (mp/provide samples))
;; 500µs (10x)
(let [provider (mp/provider)]
(p/bench (provider samples)))
Inferring :map-of
requires 3 identical key & value schemas:
(mp/provide
[{:a [1]
:b [1 2]
:c [1 2 3]}])
; [:map-of keyword? [:vector int?]]
This can be configured via ::mp/map-of-threshold
options:
(mp/provide
[{:a [1]
:b [1 2]}]
{::mp/map-of-threshold 2})
; [:map-of keyword? [:vector int?]]
Sample-data can be type-hinted with ::mp/hint
:
(mp/provide
[^{::mp/hint :map-of}
{:a {:b 1, :c 2}
:b {:b 2, :c 1}
:c {:b 3}
:d nil}])
;[:map-of
; keyword?
; [:maybe [:map
; [:b int?]
; [:c {:optional true} int?]]]]
By default, tuples are not inferred:
(mp/provide
[[1 "kikka" true]
[2 "kukka" true]
[3 "kakka" true]])
; [:vector some?]
There is ::mp/tuple-threshold
option:
(mp/provide
[[1 "kikka" true]
[2 "kukka" true]
[3 "kakka" false]]
{::mp/tuple-threshold 3})
; [:tuple int? string? boolean?]
Sample-data can be type-hinted with ::mp/hint
:
(mp/provide
[^{::mp/hint :tuple}
[1 "kikka" true]
["2" "kukka" true]])
; [:tuple some? string? boolean?]
By default, no decoding is applied for (leaf) values:
(mp/provide
[{:id "caa71a26-5fe1-11ec-bf63-0242ac130002"}
{:id "8aadbf5e-5fe3-11ec-bf63-0242ac130002"}])
; => [:map [:id string?]]
Adding custom decoding via ::mp/value-decoders
option:
(mp/provide
[{:id "caa71a26-5fe1-11ec-bf63-0242ac130002"
:time "2021-01-01T00:00:00Z"}
{:id "8aadbf5e-5fe3-11ec-bf63-0242ac130002"
:time "2022-01-01T00:00:00Z"}]
{::mp/value-decoders {'string? {:uuid mt/-string->uuid
'inst? mt/-string->date}}})
; => [:map [:id :uuid] [:time inst?]
Schemas can also be inferred from Clojure Destructuring Syntax.
(require '[malli.destructure :as md])
(def infer (comp :schema md/parse))
(infer '[a b & cs])
; => [:cat :any :any [:* :any]]
Malli also supports adding type hints as an extension to the normal Clojure syntax (enabled by default), inspired by Plumatic Schema.
(infer '[a :- :int, b :- :string & cs :- [:* :boolean]])
; => [:cat :int :string [:* :boolean]]
Pulling out function argument schemas from Vars:
(defn kikka
([a] [a])
([a b & cs] [a b cs]))
(md/infer #'kikka)
;[:function
; [:=> [:cat :any] :any]
; [:=> [:cat :any :any [:* :any]] :any]]
md/parse
uses the following options:
key | description |
---|---|
::md/inline-schemas | support plumatic-style inline schemas (true) |
::md/sequential-maps | support sequential maps in non-rest position (true) |
::md/references | qualified schema references used (true) |
::md/required-keys | destructured keys are required (false) |
::md/closed-maps | destructured maps are closed (false) |
A more complete example:
(infer '[a [b c & rest :as bc]
& {:keys [d e]
:demo/keys [f]
g :demo/g
[h] :h
:or {d 0}
:as opts}])
;[:cat
; :any
; [:maybe [:cat
; [:? :any]
; [:? :any]
; [:* :any]]]
; [:altn
; [:map
; [:map
; [:d {:optional true} :any]
; [:e {:optional true} :any]
; [:demo/f {:optional true}]
; [:demo/g {:optional true}]
; [:h {:optional true} [:maybe [:cat
; [:? :any]
; [:* :any]]]]]]
; [:args
; [:*
; [:alt
; [:cat [:= :d] :any]
; [:cat [:= :e] :any]
; [:cat [:= :demo/f] :demo/f]
; [:cat [:= :demo/g] :demo/g]
; [:cat [:= :h] [:maybe [:cat
; [:? :any]
; [:* :any]]]]
; [:cat :any :any]]]]]]
Schemas can be used to parse values using m/parse
and m/parser
:
m/parse
for one-time things:
(m/parse
[:* [:catn
[:prop string?]
[:val [:altn
[:s string?]
[:b boolean?]]]]]
["-server" "foo" "-verbose" true "-user" "joe"])
;[{:prop "-server", :val [:s "foo"]}
; {:prop "-verbose", :val [:b true]}
; {:prop "-user", :val [:s "joe"]}]
m/parser
to create an optimized parser:
(def Hiccup
[:schema {:registry {"hiccup" [:orn
[:node [:catn
[:name keyword?]
[:props [:? [:map-of keyword? any?]]]
[:children [:* [:schema [:ref "hiccup"]]]]]]
[:primitive [:orn
[:nil nil?]
[:boolean boolean?]
[:number number?]
[:text string?]]]]}}
"hiccup"])
(def parse-hiccup (m/parser Hiccup))
(parse-hiccup
[:div {:class [:foo :bar]}
[:p "Hello, world of data"]])
;[:node
; {:name :div
; :props {:class [:foo :bar]}
; :children [[:node
; {:name :p
; :props nil
; :children [[:primitive [:text "Hello, world of data"]]]}]]}]
Parsing returns tagged values for :orn
, :catn
, :altn
and :multi
.
(def Multi
[:multi {:dispatch :type}
[:user [:map [:size :int]]]
[::m/default :any]])
(m/parse Multi {:type :user, :size 1})
; => [:user {:type :user, :size 1}]
(m/parse Multi {:type "sized", :size 1})
; => [:malli.core/default {:type "sized", :size 1}]
The inverse of parsing, using m/unparse
and m/unparser
:
(->> [:div {:class [:foo :bar]}
[:p "Hello, world of data"]]
(m/parse Hiccup)
(m/unparse Hiccup))
;[:div {:class [:foo :bar]}
; [:p "Hello, world of data"]]
Enabling serializable function schemas requires sci as external dependency. If
it is not present, the malli function evaluator throws :sci-not-available
exception.
For ClojureScript, you also need to require sci.core
manually, either directly or via :preloads
.
For GraalVM, you need to require sci.core
manually, before requiring any malli namespaces.
(def my-schema
[:and
[:map
[:x int?]
[:y int?]]
[:fn '(fn [{:keys [x y]}] (> x y))]])
(m/validate my-schema {:x 1, :y 0})
; => true
(m/validate my-schema {:x 1, :y 2})
; => false
NOTE: sci is not termination safe so be wary of sci
functions from untrusted sources. You can explicitly disable sci with option ::m/disable-sci
and set the default options with ::m/sci-options
.
(m/validate [:fn 'int?] 1 {::m/disable-sci true})
; Execution error
; :malli.core/sci-not-available {:code int?}
Implemented with protocol malli.core/AST
. Allows lossless round-robin with faster schema creation.
NOTE: For now, the AST syntax in considered as internal, e.g. don't use it as a database persistency model.
(def ?schema
[:map
[:x boolean?]
[:y {:optional true} int?]
[:z [:map
[:x boolean?]
[:y {:optional true} int?]]]])
(m/form ?schema)
;[:map
; [:x boolean?]
; [:y {:optional true} int?]
; [:z [:map
; [:x boolean?]
; [:y {:optional true} int?]]]]
(m/ast ?schema)
;{:type :map,
; :keys {:x {:order 0
; :value {:type boolean?}},
; :y {:order 1, :value {:type int?}
; :properties {:optional true}},
; :z {:order 2,
; :value {:type :map,
; :keys {:x {:order 0
; :value {:type boolean?}},
; :y {:order 1
; :value {:type int?}
; :properties {:optional true}}}}}}}
(-> ?schema
(m/schema) ;; 3.4µs
(m/ast)
(m/from-ast) ;; 180ns (18x, lazy)
(m/form)
(= (m/form ?schema)))
; => true
Schemas can be transformed using post-walking, e.g. the Visitor Pattern.
The identity walker:
(m/walk
Address
(m/schema-walker identity))
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map
; [:street string?]
; [:city string?]
; [:zip int?]
; [:lonlat [:tuple double? double?]]]]]
Adding :title
property to schemas:
(m/walk
Address
(m/schema-walker #(mu/update-properties % assoc :title (name (m/type %)))))
;[:map {:title "map"}
; [:id [string? {:title "string?"}]]
; [:tags [:set {:title "set"} [keyword? {:title "keyword?"}]]]
; [:address
; [:map {:title "map"}
; [:street [string? {:title "string?"}]]
; [:city [string? {:title "string?"}]]
; [:zip [int? {:title "int?"}]]
; [:lonlat [:tuple {:title "tuple"} [double? {:title "double?"}] [double? {:title "double?"}]]]]]]
Transforming schemas into maps:
(m/walk
Address
(fn [schema _ children _]
(-> (m/properties schema)
(assoc :malli/type (m/type schema))
(cond-> (seq children) (assoc :malli/children children)))))
;{:malli/type :map,
; :malli/children [[:id nil {:malli/type string?}]
; [:tags nil {:malli/type :set
; :malli/children [{:malli/type keyword?}]}]
; [:address nil {:malli/type :map,
; :malli/children [[:street nil {:malli/type string?}]
; [:city nil {:malli/type string?}]
; [:zip nil {:malli/type int?}]
; [:lonlat nil {:malli/type :tuple
; :malli/children [{:malli/type double?}
; {:malli/type double?}]}]]}]]}
Transforming Schemas into JSON Schema:
(require '[malli.json-schema :as json-schema])
(json-schema/transform Address)
;{:type "object",
; :properties {:id {:type "string"},
; :tags {:type "array"
; :items {:type "string"}
; :uniqueItems true},
; :address {:type "object",
; :properties {:street {:type "string"},
; :city {:type "string"},
; :zip {:type "integer", :format "int64"},
; :lonlat {:type "array",
; :items [{:type "number"} {:type "number"}],
; :additionalItems false}},
; :required [:street :city :zip :lonlat]}},
; :required [:id :tags :address]}
Custom transformation via :json-schema
namespaced properties:
(json-schema/transform
[:enum
{:title "Fish"
:description "It's a fish"
:json-schema/type "string"
:json-schema/default "perch"}
"perch" "pike"])
;{:title "Fish"
; :description "It's a fish"
; :type "string"
; :default "perch"
; :enum ["perch" "pike"]}
Full override with :json-schema
property:
(json-schema/transform
[:map {:json-schema {:type "file"}}
[:file any?]])
; {:type "file"}
Transforming Schemas into Swagger2 Schema:
(require '[malli.swagger :as swagger])
(swagger/transform Address)
;{:type "object",
; :properties {:id {:type "string"},
; :tags {:type "array"
; :items {:type "string"}
; :uniqueItems true},
; :address {:type "object",
; :properties {:street {:type "string"},
; :city {:type "string"},
; :zip {:type "integer", :format "int64"},
; :lonlat {:type "array",
; :items {},
; :x-items [{:type "number", :format "double"}
; {:type "number", :format "double"}]}},
; :required [:street :city :zip :lonlat]}},
; :required [:id :tags :address]}
Custom transformation via :swagger
and :json-schema
namespaced properties:
(swagger/transform
[:enum
{:title "Fish"
:description "It's a fish"
:swagger/type "string"
:json-schema/default "perch"}
"perch" "pike"])
;{:title "Fish"
; :description "It's a fish"
; :type "string"
; :default "perch"
; :enum ["perch" "pike"]}
Full override with :swagger
property:
(swagger/transform
[:map {:swagger {:type "file"}}
[:file any?]])
; {:type "file"}
Schema Types are described using m/IntoSchema
protocol, which has a factory method
(-into-schema [this properties children options])
to create the actual Schema instances.
See malli.core
for example implementations.
For simple cases, there is m/-simple-schema
:
(require '[clojure.test.check.generators :as gen])
(def Over6
(m/-simple-schema
{:type :user/over6
:pred #(and (int? %) (> % 6))
:type-properties {:error/message "should be over 6"
:decode/string mt/-string->long
:json-schema/type "integer"
:json-schema/format "int64"
:json-schema/minimum 6
:gen/gen (gen/large-integer* {:min 7})}}))
(m/into-schema? Over6)
; => true
m/IntoSchema
can be both used as Schema (creating a Schema instance with nil
properties
and children) and as Schema type to create new Schema instances without needing to
register the types:
(m/schema? (m/schema Over6))
; => true
(m/schema? (m/schema [Over6 {:title "over 6"}]))
; => true
:pred
is used for validation:
(m/validate Over6 2)
; => false
(m/validate Over6 7)
; => true
:type-properties
are shared for all schema instances and are used just like Schema
(instance) properties by many Schema applications, including error messages,
value generation and json-schema transformations.
(json-schema/transform Over6)
; => {:type "integer", :format "int64", :minimum 6}
(json-schema/transform [Over6 {:json-schema/example 42}])
; => {:type "integer", :format "int64", :minimum 6, :example 42}
You can also build content-dependent schemas by using a callback function of properties children -> opts
instead of static opts
:
(def Over
(m/-simple-schema
(fn [{:keys [value]} _]
(assert (int? value))
{:type :user/over
:pred #(and (int? %) (> % value))
:type-properties {:error/fn (fn [error _] (str "should be over " value ", was " (:value error)))
:decode/string mt/-string->long
:json-schema/type "integer"
:json-schema/format "int64"
:json-schema/minimum value
:gen/gen (gen/large-integer* {:min (inc value)})}})))
(-> [Over {:value 12}]
(m/explain 10)
(me/humanize))
; => ["should be over 12, was 10"]
Schemas are looked up using a malli.registry/Registry
protocol, which is effectively a map from schema type
to a schema recipe (Schema
, IntoSchema
or vector-syntax schema). Map
s can also be used as a registry.
Custom Registry
can be passed into all/most malli public APIs via the optional options map using :registry
key. If omitted, malli.core/default-registry
is used.
;; the default registry
(m/validate [:maybe string?] "kikka")
; => true
;; registry as explicit options
(m/validate [:maybe string?] "kikka" {:registry m/default-registry})
; => true
The default immutable registry is merged from multiple parts, enabling easy re-composition of custom schema sets. See built-in schemas for list of all Schemas.
Here's an example to create a custom registry without the default core predicates and with :neg-int
and :pos-int
Schemas:
(def registry
(merge
(m/class-schemas)
(m/comparator-schemas)
(m/base-schemas)
{:neg-int (m/-simple-schema {:type :neg-int, :pred neg-int?})
:pos-int (m/-simple-schema {:type :pos-int, :pred pos-int?})}))
(m/validate [:or :pos-int :neg-int] 'kikka {:registry registry})
; => false
(m/validate [:or :pos-int :neg-int] 123 {:registry registry})
; => true
We did not register normal predicate schemas:
(m/validate pos-int? 123 {:registry registry})
; Syntax error (ExceptionInfo) compiling
; :malli.core/invalid-schema {:schema pos-int?}
Any schema can define a local registry using :registry
schema property:
(def Adult
[:map {:registry {::age [:and int? [:> 18]]}}
[:age ::age]])
(mg/generate Adult {:size 10, :seed 1})
; => {:age 92}
Local registries can be persisted:
(-> Adult
(malli.edn/write-string)
(malli.edn/read-string)
(m/validate {:age 46}))
; => true
See also Recursive Schemas.
Passing in custom options to all public methods is a lot of boilerplate. For the lazy, there is an easier way - we can swap the (global) default registry:
(require '[malli.registry :as mr])
;; the default registry
(-> m/default-registry (mr/schemas) (count))
;=> 140
;; global side-effects! free since 0.7.0!
(mr/set-default-registry!
{:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)})
(-> m/default-registry (mr/schemas) (count))
; => 3
(m/validate
[:map [:maybe [:maybe :string]]]
{:maybe "sheep"})
; => true
(m/validate :int 42)
; =throws=> :malli.core/invalid-schema {:schema :int}
NOTE: mr/set-default-registry!
is an imperative api with global side-effects. Easy, but not simple. If you want to disable the api, you can define the following compiler/jvm bootstrap:
:closure-defines {malli.registry/mode "strict"}
:jvm-opts ["-Dmalli.registry/mode=strict"]
The default schema registry is defined as a Var, so all Schema implementation (100+) are dragged in. For ClojureScript, this means the schemas implementations are not removed via Dead Code Elimination (DCE), resulting a large (37KB, zipped) js-bundle.
Malli allows the default registry to initialized with empty schemas, using the following compiler/jvm bootstrap:
:closure-defines {malli.registry/type "custom"}
:jvm-opts ["-Dmalli.registry/type=custom"]
;; with the flag set on
(-> m/default-registry (mr/schemas) (count))
; => 0
With this, you can register just what you need and rest are DCE'd. The previous example results in just a 3KB gzip bundle.
Malli supports multiple type of registries.
Just just a Map
.
(require '[malli.registry :as mr])
(mr/set-default-registry!
{:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)})
(m/validate
[:map [:maybe [:maybe :string]]]
{:maybe "sheep"})
; => true
clojure.spec introduces a mutable global registry for specs. The mutable registry in malli forced you to bring in your own state atom and functions how to work with it:
Using a custom registry atom:
(def registry*
(atom {:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)}))
(defn register! [type ?schema]
(swap! registry* assoc type ?schema))
(mr/set-default-registry!
(mr/mutable-registry registry*))
(register! :non-empty-string [:string {:min 1}])
(m/validate :non-empty-string "malli")
; => true
The mutable registry can also be passed in as an explicit option:
(def registry (mr/mutable-registry registry*))
(m/validate :non-empty-string "malli" {:registry registry})
; => true
If you know what you are doing, you can also use dynamic scope to pass in default schema registry:
(mr/set-default-registry!
(mr/dynamic-registry))
(binding [mr/*registry* {:string (m/-string-schema)
:maybe (m/-maybe-schema)
:map (m/-map-schema)
:non-empty-string [:string {:min 1}]}]
(m/validate :non-empty-string "malli"))
; => true
You can provide schemas at runtime using mr/lazy-registry
- it takes a local registry and a provider function of type registry -> schema
as arguments:
(def registry
(mr/lazy-registry
(m/default-schemas)
(fn [type registry]
;; simulates pulling CloudFormation Schemas when needed
(let [lookup {"AWS::ApiGateway::UsagePlan" [:map {:closed true}
[:Type [:= "AWS::ApiGateway::UsagePlan"]]
[:Description {:optional true} string?]
[:UsagePlanName {:optional true} string?]]
"AWS::AppSync::ApiKey" [:map {:closed true}
[:Type [:= "AWS::AppSync::ApiKey"]]
[:ApiId string?]
[:Description {:optional true} string?]]}]
(println "... loaded" type)
(some-> type lookup (m/schema {:registry registry}))))))
;; lazy multi, doesn't realize the schemas
(def CloudFormation
(m/schema
[:multi {:dispatch :Type, :lazy-refs true}
"AWS::ApiGateway::UsagePlan"
"AWS::AppSync::ApiKey"]
{:registry registry}))
(m/validate
CloudFormation
{:Type "AWS::ApiGateway::UsagePlan"
:Description "laiskanlinna"})
; ... loaded AWS::ApiGateway::UsagePlan
; => true
(m/validate
CloudFormation
{:Type "AWS::ApiGateway::UsagePlan"
:Description "laiskanlinna"})
; => true
Registries can be composed, a full example:
(require '[malli.core :as m])
(require '[malli.registry :as mr])
(def registry (atom {}))
(defn register! [type schema]
(swap! registry assoc type schema))
(mr/set-default-registry!
;; linear search
(mr/composite-registry
;; immutable registry
{:map (m/-map-schema)}
;; mutable (spec-like) registry
(mr/mutable-registry registry)
;; on the perils of dynamic scope
(mr/dynamic-registry)))
;; mutate like a boss
(register! :maybe (m/-maybe-schema))
;; ☆.。.:*・°☆.。.:*・°☆.。.:*・°☆.。.:*・°☆
(binding [mr/*registry* {:string (m/-string-schema)}]
(m/validate
[:map [:maybe [:maybe :string]]]
{:maybe "sheep"}))
; => true
See Instrumentation.
Clj-kondo is a linter for Clojure code that sparks joy.
Given functions and function Schemas:
(defn square [x] (* x x))
(m/=> square [:=> [:cat int?] nat-int?])
(defn plus
([x] x)
([x y] (+ x y)))
(m/=> plus [:function
[:=> [:cat int?] int?]
[:=> [:cat int? int?] int?]])
Generating clj-kondo
configuration from current namespace:
(require '[malli.clj-kondo :as mc])
(-> (mc/collect *ns*) (mc/linter-config))
;{:lint-as #:malli.schema{defn schema.core/defn},
; :linters
; {:type-mismatch
; {:namespaces
; {user {square {:arities {1 {:args [:int]
; :ret :pos-int}}}
; plus {:arities {1 {:args [:int]
; :ret :int},
; 2 {:args [:int :int]
; :ret :int}}}}}}}}
Emitting confing into ./.clj-kondo/configs/malli/config.edn
:
(mc/emit!)
In action:
Typed Clojure is an optional type system for Clojure.
typed.malli can consume a subset of malli schema syntax to statically type check and infer Clojure code.
See this in action in the malli-type-providers example project.
(ns typed-example.malli-type-providers
(:require [typed.clojure :as t]
[malli.core :as m]))
;; just use malli instrumentation normally
(m/=> foo [:=> [:cat :int] :int])
;; Typed Clojure will statically check `foo` against its schema (after converting it to a type)
(defn foo [t] (inc t))
;; Typed Clojure will automatically infer `foo`s type from its schema
(foo 1)
(comment (t/check-ns-clj)) ;; check this ns
Transforming Schemas into DOT Language:
(require '[malli.dot :as md])
(def Address
[:schema
{:registry {"Country" [:map
[:name [:enum :FI :PO]]
[:neighbors [:vector [:ref "Country"]]]]
"Burger" [:map
[:name string?]
[:description {:optional true} string?]
[:origin [:maybe "Country"]]
[:price pos-int?]]
"OrderLine" [:map
[:burger "Burger"]
[:amount int?]]
"Order" [:map
[:lines [:vector "OrderLine"]]
[:delivery [:map
[:delivered boolean?]
[:address [:map
[:street string?]
[:zip int?]
[:country "Country"]]]]]]}}
"Order"])
(md/transform Address)
; "digraph { ... }"
Visualized with Graphviz:
Transforming Schemas into PlantUML:
(require '[malli.plantuml :as plantuml])
(plantuml/transform Address)
; "@startuml ... @enduml"
Visualized with PlantText:
Simple syntax sugar, like data-specs, but for malli.
As the namespace suggests, it's experimental, built for reitit.
(require '[malli.experimental.lite :as l])
(l/schema
{:map1 {:x int?
:y [:maybe string?]
:z (l/maybe keyword?)}
:map2 {:min-max [:int {:min 0 :max 10}]
:tuples (l/vector (l/tuple int? string?))
:optional (l/optional (l/maybe :boolean))
:set-of-maps (l/set {:e int?
:f string?})
:map-of-int (l/map-of int? {:s string?})}})
;[:map
; [:map1
; [:map
; [:x int?]
; [:y [:maybe string?]]
; [:z [:maybe keyword?]]]]
; [:map2
; [:map
; [:min-max [:int {:min 0, :max 10}]]
; [:tuples [:vector [:tuple int? string?]]]
; [:optional {:optional true} [:maybe :boolean]]
Options can be used by binding a dynamic l/*options*
Var:
(binding [l/*options* {:registry (merge
(m/default-schemas)
{:user/id :int})}]
(l/schema {:id (l/maybe :user/id)
:child {:id :user/id}}))
;[:map
; [:id [:maybe :user/id]]
; [:child [:map [:id :user/id]]]]
Malli tries to be really, really fast.
Usually as fast (or faster) as idiomatic Clojure.
(require '[criterium.core :as cc])
(def valid {:x true, :y 1, :z "zorro"})
;; idomatic clojure (54ns)
(let [valid? (fn [{:keys [x y z]}]
(and (boolean? x)
(if y (int? y) true)
(string? z)))]
(assert (valid? valid))
(cc/quick-bench (valid? valid)))
(require '[malli.core :as m])
;; malli (39ns)
(let [valid? (m/validator
[:map
[:x :boolean]
[:y {:optional true} :int]
[:z :string]])]
(assert (valid? valid))
(cc/quick-bench (valid? valid)))
Same with Clojure Spec and Plumatic Schema:
(require '[clojure.spec.alpha :as spec])
(require '[schema.core :as schema])
(spec/def ::x boolean?)
(spec/def ::y int?)
(spec/def ::z string?)
;; clojure.spec (450ns)
(let [spec (spec/keys :req-un [::x ::z] :opt-un [::y])]
(assert (spec/valid? spec valid))
(cc/quick-bench (spec/valid? spec valid)))
;; plumatic schema (660ns)
(let [valid? (schema/checker
{:x schema/Bool
(schema/optional-key :y) schema/Int
:z schema/Str})]
(assert (not (valid? valid)))
(cc/quick-bench (valid? valid)))
Usually faster than idiomatic Clojure.
(def data {:x "true", :y "1", :z "kikka"})
(def expected {:x true, :y 1, :z "kikka"})
;; idiomatic clojure (290ns)
(let [transform (fn [{:keys [x y] :as m}]
(cond-> m
(string? x) (update :x #(Boolean/parseBoolean %))
(string? y) (update :y #(Long/parseLong %))))]
(assert (= expected (transform data)))
(cc/quick-bench (transform data)))
;; malli (72ns)
(let [schema [:map
[:x :boolean]
[:y {:optional true} int?]
[:z string?]]
transform (m/decoder schema (mt/string-transformer))]
(assert (= expected (transform data)))
(cc/quick-bench (transform data)))
Same with Clojure Spec and Plumatic Schema:
(require '[spec-tools.core :as st])
(require '[schema.coerce :as sc])
(spec/def ::x boolean?)
(spec/def ::y int?)
(spec/def ::z string?)
;; clojure.spec (19000ns)
(let [spec (spec/keys :req-un [::x ::z] :opt-un [::y])
transform #(st/coerce spec % st/string-transformer)]
(assert (= expected (transform data)))
(cc/quick-bench (transform data)))
;; plumatic schema (2200ns)
(let [schema {:x schema/Bool
(schema/optional-key :y) schema/Int
:z schema/Str}
transform (sc/coercer schema sc/string-coercion-matcher)]
(assert (= expected (transform data)))
(cc/quick-bench (transform data)))
The transformation engine is smart enough to just transform parts of the schema that need to be transformed. If there is nothing to transform, identity
function is returned.
(def json->user
(m/decoder
[:map
[:id :int]
[:name :string]
[:address [:map
[:street :string]
[:rural :boolean]
[:country [:enum "finland" "poland"]]]]]
(mt/json-transformer)))
(= identity json->user)
; => true
;; 5ns
(cc/quick-bench
(json->user
{:id 1
:name "tiina"
:address {:street "kotikatu"
:rural true
:country "poland"}}))
;; 37µs
(let [spec (s/* (s/cat :prop string?,
:val (s/alt :s string?
:b boolean?)))
parse (partial s/conform spec)]
(cc/quick-bench
(parse ["-server" "foo" "-verbose" "-verbose" "-user" "joe"])))
;; 2.4µs
(let [schema [:* [:catn
[:prop string?]
[:val [:altn
[:s string?]
[:b boolean?]]]]]
parse (m/parser schema)]
(cc/quick-bench
(parse ["-server" "foo" "-verbose" "-verbose" "-user" "joe"])))
malli.core/predicate-schemas
Contains both function values and unqualified symbol representations for all relevant core predicates. Having both representations enables reading forms from both code (function values) and EDN-files (symbols): any?
, some?
, number?
, integer?
, int?
, pos-int?
, neg-int?
, nat-int?
, pos?,
neg?, float?
, double?
, boolean?
, string?
, ident?
, simple-ident?
, qualified-ident?
, keyword?
, simple-keyword?
, qualified-keyword?
, symbol?
, simple-symbol?
, qualified-symbol?
, uuid?
, uri?
, decimal?
, inst?
, seqable?
, indexed?
, map?
, vector?
, list?
, seq?
, char?
, set?
, nil?
, false?
, true?
, zero?
, rational?
, coll?
, empty?
, associative?
, sequential?
, ratio?
, bytes?
, ifn?
and fn?
.
malli.core/class-schemas
Class-based schemas, contains java.util.regex.Pattern
& js/RegExp
.
malli.core/comparator-schemas
Comparator functions as keywords: :>
, :>=
, :<
, :<=
, :=
and :not=
.
malli.core/type-schemas
Type-like schemas: :any
, :nil
, :string
, :int
, :double
, :boolean
, :keyword
, :qualified-keyword
, :symbol
, :qualified-symbol
, and :uuid
.
malli.core/sequence-schemas
Sequence/regex-schemas: :+
, :*
, :?
, :repeat
, :cat
, :alt
, :catn
, :altn
.
malli.core/base-schemas
Contains :and
, :or
, :orn
, :not
, :map
, :map-of
, :vector
, :sequential
, :set
, :enum
, :maybe
, :tuple
, :multi
, :re
, :fn
, :ref
, :=>
, :function
and :schema
.
malli.util/schemas
:merge
, :union
and :select-keys
.
The public API of Malli has been quite stable already in pre-alpha and in alpha, we try not to break things. Still, the library is evolving and things like value destructuring could affect public APIs and most likely affect the library extenders, e.g. need to implement a new protocol method for custom schemas.
All changes (breaking or not) will be documented in the CHANGELOG and there will be a migration guide and path if needed.
The API layers and stability:
-
, e.g. malli.core/validate
. The most stable part of the library, should not change (much) in alpha-
, e.g. malli.core/-collection-schema
. Not needed with basic use cases, might evolve during the alpha, follow CHANGELOG for detailsmalli.impl
namespaces, all bets are off.Malli aims to support the LTS releases Java 8 and 11 and the latest Java release, Java 15.
We use Kaocha and cljs-test-runner as a test runners. Before running the tests, you need to install NPM dependencies.
npm install
./bin/kaocha
./bin/node
clj -Mjar
clj -Minstall
With default registry (37KB+ Gzipped)
# no sci
npx shadow-cljs run shadow.cljs.build-report app /tmp/report.html
# with sci
npx shadow-cljs run shadow.cljs.build-report app-sci /tmp/report.html
With minimal registry (2.4KB+ Gzipped)
# no sci
npx shadow-cljs run shadow.cljs.build-report app2 /tmp/report.html
# with sci
npx shadow-cljs run shadow.cljs.build-report app2-sci /tmp/report.html
clojure-lsp format
clojure-lsp clean-ns
npx shadow-cljs release app --pseudo-names
Without sci (11Mb)
./bin/native-image demo
./demo '[:set :keyword]' '["kikka" "kukka"]'
With sci (18Mb):
./bin/native-image demosci
./demosci '[:fn (fn [x] (and (int? x) (> x 10)))]]' '12'
Since version 0.8.9 malli is compatible with babashka, a native, fast starting Clojure interpreter for scripting.
You can add malli to bb.edn
:
{:deps {metosin/malli {:mvn/version "0.8.9"}}}
or directly in a babashka script:
(ns bb-malli
(:require [babashka.deps :as deps]))
(deps/add-deps '{:deps {metosin/malli {:mvn/version "0.8.9"}}})
(require '[malli.core :as malli])
(prn (malli/validate [:map [:a [:int]]] {:a 1}))
(prn (malli/explain [:map [:a [:int]]] {:a "foo"}))
Copyright © 2019-2022 Metosin Oy and contributors.
Available under the terms of the Eclipse Public License 2.0, see LICENSE
.
Can you improve this documentation? These fine people already did:
Tommi Reiman, Ambrose Bonnaire-Sergeant, Miikka Koskinen, Joel Kaasinen, Pauli Jaakkola, Vesa Marttila, erichkoliphant, Teodor Heggelund, Brett Rowberry, lread, Lassi Immonen, Jean BOUDET, Lucy Wang, Arne Brasseur, Kevin van Rooijen, apeckham, Alexis Vincent, Stanislav Yakushev, Bobby Towers, Peter Nagy, Ian Lobarev, James Conroy-Finn, Lauri Oherd, Leon Mergen, AN, Estevo U. C. Castro, Alan Thompson, Sylvain Ageneau, Ivan Kuznetsov, Maurício Szabo, Søren Sjørup, caumond, Ronny Løvtangen, Yevgeni Tsodikov, Derek Passen, Oleg, Jeroen van Dijk, Michiel Borkent, Steve Buikhuizen, Amar Sood, Juho Teperi & Ilmo RaunioEdit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close