AST definition and compiler for #dt/e expressions.
#dt/e is a reader tag that produces an AST map. datajure.core interprets these ASTs when executing dt queries. This namespace handles:
Op names are stored as keywords in the AST (e.g. :and, :>, :+) rather than symbols, because the Clojure compiler tries to resolve symbols in literal data structures. Since and/or/not are macros, the compiler rejects 'Can't take value of a macro' when it encounters them as bare symbols in the map values returned by the reader tag. Keywords are self-evaluating and avoid this entirely.
Nil-safety rules (matching spec):
AST definition and compiler for #dt/e expressions.
#dt/e is a reader tag that produces an AST map. datajure.core interprets
these ASTs when executing dt queries. This namespace handles:
- AST node constructors
- compile-expr: AST -> fn of dataset -> column/scalar
- Reader tag handler registered via resources/data_readers.clj (primary)
and register-reader! / alter-var-root (AOT/script fallback)
Op names are stored as keywords in the AST (e.g. :and, :>, :+) rather than
symbols, because the Clojure compiler tries to resolve symbols in literal
data structures. Since and/or/not are macros, the compiler rejects
'Can't take value of a macro' when it encounters them as bare symbols in
the map values returned by the reader tag. Keywords are self-evaluating
and avoid this entirely.
Nil-safety rules (matching spec):
- Comparison ops with nil arg -> false column (all rows false)
- Arithmetic ops with nil arg -> nil (becomes missing when stored in dataset)
These rules only activate when a Clojure nil literal appears in an expression.
Dataset columns with missing values are handled natively by dfn.(col-node kw)Create a column reference AST node. kw is a keyword naming the column.
Create a column reference AST node. `kw` is a keyword naming the column.
(col-refs node)Extract the set of column keywords referenced by an AST node. If a :lit node contains an embedded expr-node (from expression composition), recursively extracts its col-refs.
Extract the set of column keywords referenced by an AST node. If a :lit node contains an embedded expr-node (from expression composition), recursively extracts its col-refs.
(compile-expr node)(compile-expr node env)Compile an AST node to a fn [ds] -> column/scalar. Column keywords resolve to dataset columns; literals pass through; ops dispatch via op-table to dfn functions.
Nil-safety: if any arg to an op evaluates to nil (e.g. a nil literal), comparison ops return an all-false boolean column; arithmetic ops return nil (which becomes a missing value when stored in a dataset column).
Compile an AST node to a fn [ds] -> column/scalar. Column keywords resolve to dataset columns; literals pass through; ops dispatch via op-table to dfn functions. Nil-safety: if any arg to an op evaluates to nil (e.g. a nil literal), comparison ops return an all-false boolean column; arithmetic ops return nil (which becomes a missing value when stored in a dataset column).
(count-distinct col)Count of distinct non-nil values in a column.
Count of distinct non-nil values in a column.
(expr-node? x)Returns true if x is a #dt/e AST node (a map with :node/type).
Returns true if x is a #dt/e AST node (a map with :node/type).
(lit-node v)Create a literal value AST node. v is any scalar (number, string, keyword, set, etc.).
Create a literal value AST node. `v` is any scalar (number, string, keyword, set, etc.).
(op-node op args)Create an operation AST node. op is an op keyword (e.g. :+, :>), args is a seq of child AST nodes.
Create an operation AST node. `op` is an op keyword (e.g. :+, :>), `args` is a seq of child AST nodes.
(read-expr form)Reader tag handler for #dt/e. Returns an AST map. Called at read time.
Reader tag handler for #dt/e. Returns an AST map. Called at read time.
(register-reader!)Register the #dt/e reader tag via alter-var-root on data-readers. Fallback for AOT compilation or scripts where data_readers.clj is not picked up at startup. The primary registration mechanism is resources/data_readers.clj, which Clojure merges automatically for all threads at JVM startup.
Register the #dt/e reader tag via alter-var-root on *data-readers*. Fallback for AOT compilation or scripts where data_readers.clj is not picked up at startup. The primary registration mechanism is resources/data_readers.clj, which Clojure merges automatically for all threads at JVM startup.
(row-node row-op args)AST node for a row-wise function call. row-op is a keyword like :row/sum. args are parsed AST nodes.
AST node for a row-wise function call. row-op is a keyword like :row/sum. args are parsed AST nodes.
(wavg w v)Weighted average. Args: weight-col value-col. Skips nil pairs.
Weighted average. Args: weight-col value-col. Skips nil pairs.
(win-node win-op args)AST node for a window function call. win-op is a keyword like :win/rank. args are parsed AST nodes.
AST node for a window function call. win-op is a keyword like :win/rank. args are parsed AST nodes.
(win-refs node)Extract the set of window op keywords (e.g. :win/rank) referenced by an AST node. If a :lit node contains an embedded expr-node (from expression composition), recursively extracts its win-refs.
Extract the set of window op keywords (e.g. :win/rank) referenced by an AST node. If a :lit node contains an embedded expr-node (from expression composition), recursively extracts its win-refs.
(wsum w v)Weighted sum. Args: weight-col value-col. Skips nil pairs.
Weighted sum. Args: weight-col value-col. Skips nil pairs.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |