Statement-level translation: SELECT / INSERT / UPDATE / DELETE / CTE.
The top half of this namespace (translate-select + its join / HAVING / materialization machinery) maps a PlainSelect AST to a Datalog query-map plus row-formatting metadata the handler uses at execute time.
The middle half handles INSERT / UPDATE / DELETE: extract-value
lifts JSqlParser literal/expression nodes into Clojure values,
coerce-insert-value adapts them to the target column's
:db/valueType, and translate-insert / translate-update /
translate-delete produce tx-data + (for UPDATE/DELETE) an
eids-walk query. INSERT RETURNING and UPDATE RETURNING land in
extract-returning; CHECK / UPDATE expressions evaluated
per-row at handler time go through eval-check-predicate /
eval-update-expr.
The bottom half implements CTEs: translate-cte-branch
materializes one WITH-clause body against an enriched db so the
outer SELECT / DML can reference it as a virtual table;
translate-recursive-cte handles the WITH RECURSIVE form by
iterating until the CTE converges.
The three blocks live in one namespace because they are mutually recursive:
translate-select → translate-recursive-cte (WITH in SELECT) translate-recursive-cte → translate-select (CTE body) translate-insert → translate-select (INSERT ... SELECT) translate-insert → translate-recursive-cte (INSERT ... WITH) translate-select → extract-value (scalar subqueries)
Dependencies on already-extracted namespaces are all one-way:
stmt → expr (translate-expr, translate-predicate, …) stmt → ctx (make-ctx, col-var!, resolve-column, …) stmt → fns (aggregate lookups) stmt → params (ParamRef, from-bindings, parse-db) stmt → jsonb, schema, types (type coercion + jsonb ops)
Statement-level translation: SELECT / INSERT / UPDATE / DELETE / CTE. The top half of this namespace (translate-select + its join / HAVING / materialization machinery) maps a PlainSelect AST to a Datalog query-map plus row-formatting metadata the handler uses at execute time. The middle half handles INSERT / UPDATE / DELETE: `extract-value` lifts JSqlParser literal/expression nodes into Clojure values, `coerce-insert-value` adapts them to the target column's :db/valueType, and `translate-insert` / `translate-update` / `translate-delete` produce tx-data + (for UPDATE/DELETE) an eids-walk query. INSERT RETURNING and UPDATE RETURNING land in `extract-returning`; CHECK / UPDATE expressions evaluated per-row at handler time go through `eval-check-predicate` / `eval-update-expr`. The bottom half implements CTEs: `translate-cte-branch` materializes one WITH-clause body against an enriched db so the outer SELECT / DML can reference it as a virtual table; `translate-recursive-cte` handles the WITH RECURSIVE form by iterating until the CTE converges. The three blocks live in one namespace because they are mutually recursive: translate-select → translate-recursive-cte (WITH in SELECT) translate-recursive-cte → translate-select (CTE body) translate-insert → translate-select (INSERT ... SELECT) translate-insert → translate-recursive-cte (INSERT ... WITH) translate-select → extract-value (scalar subqueries) Dependencies on already-extracted namespaces are all one-way: stmt → expr (translate-expr, translate-predicate, …) stmt → ctx (make-ctx, col-var!, resolve-column, …) stmt → fns (aggregate lookups) stmt → params (ParamRef, *from-bindings*, *parse-db*) stmt → jsonb, schema, types (type coercion + jsonb ops)
Query-tree-local counter used to give unaliased derived relations distinct, deterministic storage namespaces. Nested translate-select calls inherit the same atom so an inner anonymous relation cannot collide with one owned by an outer SELECT.
Query-tree-local counter used to give unaliased derived relations distinct, deterministic storage namespaces. Nested translate-select calls inherit the same atom so an inner anonymous relation cannot collide with one owned by an outer SELECT.
{cte-name -> synthetic-namespace} for the WITH items in scope.
A CTE is materialised into a speculative db as ordinary attributes,
and the namespace used to be the CTE's own name — so a CTE whose name
matched a real table wrote into that table's namespace. The two then
MERGED rather than the CTE shadowing the table: scans saw the union of
both relations' rows, SELECT * listed the union of their columns,
and a CTE row whose primary key matched a base row UPSERTED onto it.
WITH t AS (...) DELETE FROM t WHERE id IN (SELECT id FROM t) deleted
the real table's rows.
Giving each CTE a synthetic namespace and keeping its user-visible
name as an ALIAS routes the whole thing through the same path as
FROM emp e, which already resolves alias-to-relation correctly. It
also leaves the base table's namespace untouched, so PostgreSQL's
escape hatch — a schema-qualified public.t still reaching the real
table — keeps working.
Bound by parse-sql around translation, and re-bound by the execute-time UPDATE/DELETE re-translation, which resolves the WHERE clause afresh and would otherwise lose the mapping.
`{cte-name -> synthetic-namespace}` for the WITH items in scope.
A CTE is materialised into a speculative db as ordinary attributes,
and the namespace used to be the CTE's own name — so a CTE whose name
matched a real table wrote into that table's namespace. The two then
MERGED rather than the CTE shadowing the table: scans saw the union of
both relations' rows, `SELECT *` listed the union of their columns,
and a CTE row whose primary key matched a base row UPSERTED onto it.
`WITH t AS (...) DELETE FROM t WHERE id IN (SELECT id FROM t)` deleted
the real table's rows.
Giving each CTE a synthetic namespace and keeping its user-visible
name as an ALIAS routes the whole thing through the same path as
`FROM emp e`, which already resolves alias-to-relation correctly. It
also leaves the base table's namespace untouched, so PostgreSQL's
escape hatch — a schema-qualified `public.t` still reaching the real
table — keeps working.
Bound by parse-sql around translation, and re-bound by the
execute-time UPDATE/DELETE re-translation, which resolves the WHERE
clause afresh and would otherwise lose the mapping.Lowercased names of CTEs (WITH items) in scope for the statement being
translated. Bound by parse-sql* so the undefined-table 42P01 check
exempts CTE references that aren't materialised into the schema — most
notably data-modifying CTE bodies (WITH x AS (INSERT … RETURNING …)),
which are skipped during WITH-fold.
Lowercased names of CTEs (WITH items) in scope for the statement being translated. Bound by parse-sql* so the undefined-table 42P01 check exempts CTE references that aren't materialised into the schema — most notably data-modifying CTE bodies (`WITH x AS (INSERT … RETURNING …)`), which are skipped during WITH-fold.
Bound by build-update-tx-for-bindings to the live db when evaluating
per-row UPDATE SET expressions. Read by the Function (concat,
etc.) and ParenthesedSelect (scalar subquery) branches of
eval-update-expr — which need a db handle to dispatch to
translate-select for inner subqueries. nil when an UPDATE has no
subquery / function-call assignments, which is the common case.
Bound by build-update-tx-for-bindings to the live db when evaluating per-row UPDATE SET expressions. Read by the Function (`concat`, etc.) and ParenthesedSelect (scalar subquery) branches of `eval-update-expr` — which need a db handle to dispatch to translate-select for inner subqueries. nil when an UPDATE has no subquery / function-call assignments, which is the common case.
(apply-compound-projections results aliases query in-args compound-exprs)Evaluate deferred SELECT projection forms and remove their hidden inputs.
Shared by the normal SELECT executor and INSERT ... SELECT, which must consume the same visible row shape. Returns [rows aliases].
Evaluate deferred SELECT projection forms and remove their hidden inputs. Shared by the normal SELECT executor and INSERT ... SELECT, which must consume the same visible row shape. Returns [rows aliases].
(apply-sql-cast inner ce)Apply a SQL CAST to a value. Returns the value cast to the target type.
Handles nil safely (returns nil). Idempotent: if inner is already the
target type, returns it unchanged (avoids lossy round-trips through str,
e.g. Date → (str d) → unparseable string).
Apply a SQL CAST to a value. Returns the value cast to the target type. Handles nil safely (returns nil). Idempotent: if `inner` is already the target type, returns it unchanged (avoids lossy round-trips through str, e.g. Date → `(str d)` → unparseable string).
(coerce-insert-value val attr schema & [db])Coerce a value to match the schema type for an attribute.
:db.type/ref columns: SQL FK semantics says INSERT/UPDATE … SET col = N writes the target's PK value (matching what the
read-side ref-deref returns). Datahike's transact requires either
an entity-id or a lookup-ref. We convert the user-supplied PK
value to a lookup-ref [target-pk-attr val] using the same
convention derive-ref-targets uses on the read side, keeping
read and write FK semantics symmetric. Falls through to the raw
value when no target is resolvable (hint-only refs without a
threaded db, or genuinely-unmapped refs).
Coerce a value to match the schema type for an attribute. `:db.type/ref` columns: SQL FK semantics says `INSERT/UPDATE … SET col = N` writes the target's PK value (matching what the read-side ref-deref returns). Datahike's transact requires either an entity-id or a lookup-ref. We convert the user-supplied PK value to a lookup-ref `[target-pk-attr val]` using the same convention `derive-ref-targets` uses on the read side, keeping read and write FK semantics symmetric. Falls through to the raw value when no target is resolvable (hint-only refs without a threaded db, or genuinely-unmapped refs).
(compound-projection-indices aliases compound-exprs)Indices that turn the physical compound-projection shape into its SQL SELECT-list shape. Hidden aggregate inputs are removed and deferred expressions are restored to the positions recorded while lowering.
Indices that turn the physical compound-projection shape into its SQL SELECT-list shape. Hidden aggregate inputs are removed and deferred expressions are restored to the positions recorded while lowering.
(correlated-select-item-spec e0 outer-aliases)Detect a correlated scalar subquery in a SELECT-list item expression e,
returning a deferral spec (without :out-pos/:alias/:oid, which the caller
adds) or nil. Two shapes:
:scalar — the item IS a scalar subquery (SELECT … <outer ref> …).:case — the item is a CASE whose THEN/ELSE contains a correlated
subquery (asyncpg's CASE WHEN typtype='c' THEN (SELECT array_agg(…) WHERE c.reltype = t.oid) END). The single-rule CASE compiler would
pre-evaluate the subquery once at parse (→ NULL); instead we defer and
let exec-select evaluate the whole CASE per outer row.:corr-refs is the set of [outer-alias col] references threaded into
:find as hidden columns so exec-select can bind from-bindings per row.
Detect a correlated scalar subquery in a SELECT-list item expression `e`, returning a deferral spec (without :out-pos/:alias/:oid, which the caller adds) or nil. Two shapes: - `:scalar` — the item IS a scalar subquery `(SELECT … <outer ref> …)`. - `:case` — the item is a CASE whose THEN/ELSE contains a correlated subquery (asyncpg's `CASE WHEN typtype='c' THEN (SELECT array_agg(…) WHERE c.reltype = t.oid) END`). The single-rule CASE compiler would pre-evaluate the subquery once at parse (→ NULL); instead we defer and let exec-select evaluate the whole CASE per outer row. `:corr-refs` is the set of [outer-alias col] references threaded into :find as hidden columns so exec-select can bind *from-bindings* per row.
(correlated-splice visible out-pos->val n-output)Assemble n-output columns from visible (non-correlation columns in
order) and out-pos->val (subquery output-position → value).
Assemble `n-output` columns from `visible` (non-correlation columns in order) and `out-pos->val` (subquery output-position → value).
(correlated-subquery-refs inner outer-aliases)Given a scalar-subquery inner (a JSqlParser Select) and the set of
outer-aliases (lowercased outer FROM aliases/table names), return the
set of [outer-alias col] correlation references the inner makes, or nil
when uncorrelated.
Detection is lexical — it finds alias.col occurrences of an OUTER alias
in the inner SQL (negative-lookbehind so xt. doesn't match alias t).
Robust enough for catalog / introspection shapes; AST-precise detection
(which would also respect inner shadowing) is a later refinement. This is
the first slice of the correlated-subquery / LATERAL executor — see
doc/design-alignment.md.
Given a scalar-subquery `inner` (a JSqlParser Select) and the set of `outer-aliases` (lowercased outer FROM aliases/table names), return the set of [outer-alias col] correlation references the inner makes, or nil when uncorrelated. Detection is lexical — it finds `alias.col` occurrences of an OUTER alias in the inner SQL (negative-lookbehind so `xt.` doesn't match alias `t`). Robust enough for catalog / introspection shapes; AST-precise detection (which would also respect inner shadowing) is a later refinement. This is the first slice of the correlated-subquery / LATERAL executor — see doc/design-alignment.md.
(enrich-schema-with-pg-array-meta schema db)Datahike's :schema map only carries :db/* keys; pgwire-side
metadata like :pg/array-elem lives as ident-entity facts. For
array column INSERTs we need that metadata available via
(get-in schema [attr :pg/array-elem]), so this helper queries
db for every ident's array-elem/ndim and merges the results into
the schema map. Memoised per schema-identity — was ~0.7 ms/row of
pure recomputation on the Pagila pg_dump replay before caching.
Datahike's `:schema` map only carries `:db/*` keys; pgwire-side metadata like `:pg/array-elem` lives as ident-entity facts. For array column INSERTs we need that metadata available via `(get-in schema [attr :pg/array-elem])`, so this helper queries db for every ident's array-elem/ndim and merges the results into the schema map. Memoised per schema-identity — was ~0.7 ms/row of pure recomputation on the Pagila pg_dump replay before caching.
(eval-check-predicate expr entity-map ns-str schema)Evaluate a CHECK-style JSqlParser Expression against an entity map and return a tri-state: true (satisfied), false (violation), or nil (unknown — PG treats as satisfied). This is distinct from eval-update-expr which returns the arithmetic value of the expression; predicates need comparison / logical ops that only make sense at enforcement time.
Column refs resolve via eval-update-expr so any entity-map-aware coercion (namespace-qualified lookups) stays consistent between SET-value evaluation and CHECK evaluation.
Evaluate a CHECK-style JSqlParser Expression against an entity map and return a tri-state: true (satisfied), false (violation), or nil (unknown — PG treats as satisfied). This is distinct from eval-update-expr which returns the arithmetic value of the expression; predicates need comparison / logical ops that only make sense at enforcement time. Column refs resolve via eval-update-expr so any entity-map-aware coercion (namespace-qualified lookups) stays consistent between SET-value evaluation and CHECK evaluation.
(eval-update-cond expr entity-map ns-str schema)Evaluate a boolean expression in UPDATE SET position -- the tests of a CASE. Three-valued: returns true / false / :null, so the caller can apply PostgreSQL's rule that a branch is taken only on TRUE.
A small evaluator rather than a reuse of the WHERE translator: this runs per ENTITY against an already-materialised entity-map, not as a datalog clause over the whole relation.
Evaluate a boolean expression in UPDATE SET position -- the tests of a CASE. Three-valued: returns true / false / :__null__, so the caller can apply PostgreSQL's rule that a branch is taken only on TRUE. A small evaluator rather than a reuse of the WHERE translator: this runs per ENTITY against an already-materialised entity-map, not as a datalog clause over the whole relation.
(eval-update-expr value-expr entity-map ns-str schema)Evaluate an UPDATE SET expression for a specific entity. For simple literals, returns the literal value. For expressions (col + 1, col * 2), evaluates against the entity's current values.
JdbcParameter placeholders return a ParamRef — the tx-build step runs substitute-params once bound values are available from Bind.
Evaluate an UPDATE SET expression for a specific entity. For simple literals, returns the literal value. For expressions (col + 1, col * 2), evaluates against the entity's current values. JdbcParameter placeholders return a ParamRef — the tx-build step runs substitute-params once bound values are available from Bind.
(eval-values-literal expr)Evaluate a literal JSqlParser expression from a VALUES row. Handles simple literals, casts, and parenthesis; returns the raw value. Anything else returns :unhandled — the caller decides whether to fall back.
A JdbcParameter returns a ParamRef that the wire layer resolves at Bind time, allowing prepared INSERTs / UPDATE FROM VALUES with ?/$N.
Evaluate a literal JSqlParser expression from a VALUES row. Handles simple literals, casts, and parenthesis; returns the raw value. Anything else returns :unhandled — the caller decides whether to fall back. A JdbcParameter returns a ParamRef that the wire layer resolves at Bind time, allowing prepared INSERTs / UPDATE FROM VALUES with ?/$N.
(extract-from-values update)If an UPDATE's FROM clause is (VALUES (...), (...)) AS alias(col1, col2, ...),
extract it. Returns {:alias str :cols [str] :rows [[literal ...] ...]} or nil.
If an UPDATE's FROM clause is `(VALUES (...), (...)) AS alias(col1, col2, ...)`,
extract it. Returns {:alias str :cols [str] :rows [[literal ...] ...]} or nil.(extract-returning returning-clause)Preserve a RETURNING target list as typed descriptors.
RETURNING is a SELECT-like projection, not a list of column names: it
admits *, table.*, arbitrary expressions and aliases. Keeping the
expression AST here lets the write executor evaluate the projection over
each affected row and lets Describe infer the same result shape before
execution.
Preserve a RETURNING target list as typed descriptors. RETURNING is a SELECT-like projection, not a list of column names: it admits `*`, `table.*`, arbitrary expressions and aliases. Keeping the expression AST here lets the write executor evaluate the projection over each affected row and lets Describe infer the same result shape before execution.
(extract-value e)(extract-value e schema db)Extract a Clojure value from a JSqlParser expression for INSERT VALUES. Optional schema+db params enable scalar subquery evaluation.
When the expression is a JdbcParameter (prepared-statement placeholder):
params/*bound-params* is bound (lexical-template fast path
or execute-time re-translation), resolve the parameter inline
to the bound value — no ParamRef leaves this function;Extract a Clojure value from a JSqlParser expression for INSERT VALUES.
Optional schema+db params enable scalar subquery evaluation.
When the expression is a JdbcParameter (prepared-statement placeholder):
- if `params/*bound-params*` is bound (lexical-template fast path
or execute-time re-translation), resolve the parameter inline
to the bound value — no ParamRef leaves this function;
- otherwise emit a ParamRef sentinel that the wire layer resolves
at Bind time against the decoded client value.(figure-colname expr)The name PostgreSQL gives an un-aliased SELECT output column.
Mirrors FigureColname (parse_target.c:1711): consult the rules,
and fall back to the literal ?column?. We used to answer with
whatever was closest to hand — the datalog variable (p1, v1), or
the expression's own SQL text (B'1001000', B'101'::varbit) —
none of which any PostgreSQL client would see.
The name PostgreSQL gives an un-aliased SELECT output column. Mirrors `FigureColname` (parse_target.c:1711): consult the rules, and fall back to the literal `?column?`. We used to answer with whatever was closest to hand — the datalog variable (`p1`, `v1`), or the expression's own SQL text (`B'1001000'`, `B'101'::varbit`) — none of which any PostgreSQL client would see.
(invalidate-enriched-schema-cache!)Clear the array-meta/typmod enriched-schema cache. Called from the server's DDL exec path.
Clear the array-meta/typmod enriched-schema cache. Called from the server's DDL exec path.
(join-type join)Determine the type of a JOIN: :inner, :left, :right, :full, or :cross. LEFT JOIN and LEFT OUTER JOIN are both :left (JSqlParser may or may not set isOuter).
Determine the type of a JOIN: :inner, :left, :right, :full, or :cross. LEFT JOIN and LEFT OUTER JOIN are both :left (JSqlParser may or may not set isOuter).
(match-aggregate-index f find-elems find-aliases)(match-aggregate-index f find-elems find-aliases resolved)Try to find the index of an aggregate function in the find-elements. For COUNT(*) → look for (count ?x), for SUM(col) → (sum ?x) or (datahike.pg.sql/filter-sum ?x). Returns the 0-based index or nil.
Matches both the raw Datalog aggregate symbol and our ns-qualified null-filtering variant (filter-sum/avg/min/max/count[-distinct]) so HAVING clauses resolve regardless of which variant the SELECT projection emitted.
Try to find the index of an aggregate function in the find-elements. For COUNT(*) → look for (count ?x), for SUM(col) → (sum ?x) or (datahike.pg.sql/filter-sum ?x). Returns the 0-based index or nil. Matches both the raw Datalog aggregate symbol and our ns-qualified null-filtering variant (filter-sum/avg/min/max/count[-distinct]) so HAVING clauses resolve regardless of which variant the SELECT projection emitted.
(materialize-derived-select! ps db schema)Given a ParenthesedSelect in FROM/JOIN position, return an enriched db that has a virtual table populated with the subquery's results.
Handles two inner shapes:
(SELECT … FROM <table-or-derived> …) — runs the inner select
against db and materializes its rows (existing behaviour).(SELECT * FROM table_function(...) [WITH ORDINALITY]) — expands
the table function directly without running a query.Returns {:db spec-db :name sub-name :alias sub-alias :aliases cols} or nil if the shape isn't recognised.
Given a ParenthesedSelect in FROM/JOIN position, return an enriched db
that has a virtual table populated with the subquery's results.
Handles two inner shapes:
1. `(SELECT … FROM <table-or-derived> …)` — runs the inner select
against `db` and materializes its rows (existing behaviour).
2. `(SELECT * FROM table_function(...) [WITH ORDINALITY])` — expands
the table function directly without running a query.
Returns {:db spec-db :name sub-name :alias sub-alias :aliases cols}
or nil if the shape isn't recognised.(materialize-recursive-cte! wi target-name db schema)Run a WITH RECURSIVE CTE rule to a fixed point and materialize the
resulting rows into a speculative db under :<target-name>/<col>
virtual attrs. Mirrors the result shape of materialize-set-op!
so callers (parse-sql) can swap implementations based on
(.isRecursive wi).
When the CTE is parameterised (a $n appears in its body, so in-args
carries ParamRef sentinels that aren't bound until Bind), DATA can't be
produced at parse time. We then enrich only the SCHEMA (column attrs,
with value-types inferred from the anchor branch) and return a
:deferred spec; the server re-runs the rule at Execute via
materialize-recursive-rows! once the params are bound.
Reuses translate-recursive-cte for the rule construction; rule
eval here is the SELECT counterpart of build-update-with-recursive-tx
in the server.
Run a WITH RECURSIVE CTE rule to a fixed point and materialize the resulting rows into a speculative db under `:<target-name>/<col>` virtual attrs. Mirrors the result shape of `materialize-set-op!` so callers (parse-sql) can swap implementations based on `(.isRecursive wi)`. When the CTE is parameterised (a `$n` appears in its body, so `in-args` carries ParamRef sentinels that aren't bound until Bind), DATA can't be produced at parse time. We then enrich only the SCHEMA (column attrs, with value-types inferred from the anchor branch) and return a `:deferred` spec; the server re-runs the rule at Execute via materialize-recursive-rows! once the params are bound. Reuses `translate-recursive-cte` for the rule construction; rule eval here is the SELECT counterpart of `build-update-with-recursive-tx` in the server.
(materialize-recursive-iterative! wi target-name db schema)FALLBACK recursive-CTE evaluator (B1): semi-naive iteration instead of a single Datalog rule. Used when materialize-recursive-cte!'s rule encoding can't represent the body (LEFT JOIN → not-join, correlated subqueries, nested recursion — e.g. asyncpg's typeinfo introspection).
Runs the anchor as an ordinary SELECT, materialises its rows under
:<target>/<col>, then repeatedly runs the recursive branch — translated
once against the seeded virtual table — folding NOVEL rows back in until a
fixed point. Each iteration is a plain query the engine already handles.
Parameterised CTEs are DEFERRED to Execute (B2-style): when the anchor has
a $n and a real FROM clause (asyncpg's FROM {typeinfo} ti WHERE ti.oid = any($1)), only the schema is enriched at parse and a :deferred
{:kind :iterative …} spec is returned; the server runs the anchor with the
bound params and iterates at Execute (materialize-recursive-iterative-rows!).
Returns the standard {:db :schema :name :alias :aliases [:deferred]} map, or
nil when it can't apply: a non-UNION shape, a parameterised TABLE-FREE
anchor (SELECT $1::int — the param constant-folds in translation, so the
rule path / B2 must own it), or any translation/eval failure.
FALLBACK recursive-CTE evaluator (B1): semi-naive iteration instead of a
single Datalog rule. Used when materialize-recursive-cte!'s rule encoding
can't represent the body (LEFT JOIN → not-join, correlated subqueries,
nested recursion — e.g. asyncpg's typeinfo introspection).
Runs the anchor as an ordinary SELECT, materialises its rows under
`:<target>/<col>`, then repeatedly runs the recursive branch — translated
once against the seeded virtual table — folding NOVEL rows back in until a
fixed point. Each iteration is a plain query the engine already handles.
Parameterised CTEs are DEFERRED to Execute (B2-style): when the anchor has
a `$n` and a real FROM clause (asyncpg's `FROM {typeinfo} ti WHERE
ti.oid = any($1)`), only the schema is enriched at parse and a `:deferred`
{:kind :iterative …} spec is returned; the server runs the anchor with the
bound params and iterates at Execute (materialize-recursive-iterative-rows!).
Returns the standard {:db :schema :name :alias :aliases [:deferred]} map, or
nil when it can't apply: a non-`UNION` shape, a parameterised TABLE-FREE
anchor (`SELECT $1::int` — the param constant-folds in translation, so the
rule path / B2 must own it), or any translation/eval failure.(materialize-recursive-iterative-rows!
{:keys [anchor recursive col-names col-types target-name row-marker]}
db)Execute-time materialisation for a DEFERRED iterative recursive CTE (see
materialize-recursive-iterative!). spec is the :deferred map with its
anchor/recursive :in-args already param-substituted by resolve-param-refs.
Runs the anchor against its parse-time enriched-db, folds the rows into the
recursive branch's enriched-db (which carries the CTE schema + any derived
tables it referenced), and iterates to a fixed point. The recursive step is
TOLERANT — if an iteration throws (e.g. an array-membership join the engine
can't resolve), we stop and keep what we have (asyncpg needs only the anchor
rows for a composite of core-typed fields). Returns the data-enriched db
(the caller's query-db, whose schema already has the CTE attrs); on anchor
failure returns db unchanged.
Execute-time materialisation for a DEFERRED iterative recursive CTE (see materialize-recursive-iterative!). `spec` is the :deferred map with its anchor/recursive :in-args already param-substituted by resolve-param-refs. Runs the anchor against its parse-time enriched-db, folds the rows into the recursive branch's enriched-db (which carries the CTE schema + any derived tables it referenced), and iterates to a fixed point. The recursive step is TOLERANT — if an iteration throws (e.g. an array-membership join the engine can't resolve), we stop and keep what we have (asyncpg needs only the anchor rows for a composite of core-typed fields). Returns the data-enriched db (the caller's query-db, whose schema already has the CTE attrs); on anchor failure returns `db` unchanged.
(materialize-recursive-rows! {:keys [rule rule-name rule-vars col-names
col-types in-params in-args target-name
row-marker]}
db)Execute-time counterpart for a DEFERRED recursive CTE (see
materialize-recursive-cte!): ground the now-bound params into the rule
(in-args already substituted by resolve-param-refs), run it to a fixed
point, coerce the rows to the parse-time col-types, and db-with the
data into db (whose schema already carries the CTE attrs from parse).
Returns the data-enriched db; on rule-eval failure returns db unchanged
so the outer query degrades to an empty CTE rather than crashing.
Execute-time counterpart for a DEFERRED recursive CTE (see materialize-recursive-cte!): ground the now-bound params into the rule (in-args already substituted by resolve-param-refs), run it to a fixed point, coerce the rows to the parse-time `col-types`, and db-with the data into `db` (whose schema already carries the CTE attrs from parse). Returns the data-enriched db; on rule-eval failure returns `db` unchanged so the outer query degrades to an empty CTE rather than crashing.
(materialize-set-op! inner target-name db schema)(materialize-set-op! inner target-name db schema alias)(materialize-set-op! inner target-name db schema alias explicit-aliases)Run a SELECT (PlainSelect, SetOperationList, or VALUES) and persist its rows
under target-name/<col> in a speculative db. Returns the same
{:db :schema :name :alias :aliases} map shape as
materialize-derived-select! so callers can swap them.
Used by both the derived-table path (FROM (...) AS t) and the CTE path (WITH t AS (...)), since SQL set operations over heterogeneous tables can't be expressed natively in Datalog — we have to flatten them into a single virtual table.
target-name is the NAMESPACE the rows are stored under; alias is
the name the user wrote. They differ for CTEs, where the namespace is
synthetic so a CTE cannot collide with a real table of the same name
— see datahike.pg.sql/cte-namespace.
Run a SELECT (PlainSelect, SetOperationList, or VALUES) and persist its rows
under `target-name/<col>` in a speculative db. Returns the same
`{:db :schema :name :alias :aliases}` map shape as
`materialize-derived-select!` so callers can swap them.
Used by both the derived-table path (FROM (...) AS t) and the CTE
path (WITH t AS (...)), since SQL set operations over heterogeneous
tables can't be expressed natively in Datalog — we have to flatten
them into a single virtual table.
`target-name` is the NAMESPACE the rows are stored under; `alias` is
the name the user wrote. They differ for CTEs, where the namespace is
synthetic so a CTE cannot collide with a real table of the same name
— see `datahike.pg.sql/cte-namespace`.(materialize-table-function tf)(materialize-table-function tf eval-fn)(materialize-table-function tf eval-fn coldefs)Produce rows for a TableFunction FROM item. Supports the common
constant-arg set-returning functions:
unnest(ARRAY[…]) (+ WITH ORDINALITY)generate_series(start,stop[,step]) over integers (+ WITH ORDINALITY)now() / current_timestamp / {statement,transaction,clock}_timestamp
(one row, current time)Returns {:aliases [col-names] :rows [[v1 v2 …] …] :vtypes [kw kw …]} or nil if the function isn't one we expand.
eval-fn resolves an argument expression to a value; it defaults to
srf-const-eval (literals only). The LATERAL nested-loop will pass an
eval-fn that resolves correlated arguments per outer row — see
srf-const-eval's note. That is why arguments flow through eval-fn here
rather than being pattern-matched as literals inline.
Produce rows for a `TableFunction` FROM item. Supports the common
constant-arg set-returning functions:
- `unnest(ARRAY[…])` (+ WITH ORDINALITY)
- `generate_series(start,stop[,step])` over integers (+ WITH ORDINALITY)
- `now()` / `current_timestamp` / `{statement,transaction,clock}_timestamp`
(one row, current time)
Returns {:aliases [col-names] :rows [[v1 v2 …] …] :vtypes [kw kw …]}
or nil if the function isn't one we expand.
`eval-fn` resolves an argument expression to a value; it defaults to
`srf-const-eval` (literals only). The LATERAL nested-loop will pass an
eval-fn that resolves correlated arguments per outer row — see
srf-const-eval's note. That is why arguments flow through eval-fn here
rather than being pattern-matched as literals inline.Re-exported from datahike.pg.sql.coerce, where it moved so the shared cast implementation (a leaf namespace) could reach it.
Re-exported from datahike.pg.sql.coerce, where it moved so the shared cast implementation (a leaf namespace) could reach it.
(recursive-cte-self-reference? wi)True when a WITH RECURSIVE item actually references its own relation.
True when a WITH RECURSIVE item actually references its own relation.
(resolve-correlated-rows parse-fn parsed rows query-db inner-schema)Resolve a parsed SELECT's deferred correlated subqueries against raw result
rows: per outer row, run each subquery with the correlation columns bound
into from-bindings, splice the value at its out-pos, and drop the hidden
_corr columns. Returns [resolved-rows resolved-aliases resolved-oids],
where resolved-oids carries each subquery's declared OID at its spliced
position (nil for visible columns) so a caller materialising the result can
type array columns from the subquery's OID rather than the runtime value
class (e.g. array_agg(atttypid) → oid[] not int8[]). No-op (returns
[rows find-aliases nil]) when there are no correlated subqueries.
Resolve a parsed SELECT's deferred correlated subqueries against raw result `rows`: per outer row, run each subquery with the correlation columns bound into *from-bindings*, splice the value at its out-pos, and drop the hidden __corr_ columns. Returns [resolved-rows resolved-aliases resolved-oids], where resolved-oids carries each subquery's declared OID at its spliced position (nil for visible columns) so a caller materialising the result can type array columns from the subquery's OID rather than the runtime value class (e.g. array_agg(atttypid) → oid[] not int8[]). No-op (returns [rows find-aliases nil]) when there are no correlated subqueries.
(run-correlated-spec parse-fn spec fb inner-schema query-db)Value of a deferred correlated SELECT item for one outer row. fb is the
per-row from-bindings. :scalar runs the subquery; :case walks branches.
Value of a deferred correlated SELECT item for one outer row. `fb` is the per-row *from-bindings*. :scalar runs the subquery; :case walks branches.
(run-recursive-rule db rule rule-name rule-vars in-params in-args)Evaluate a recursive-CTE Datalog rule to a fixed point and return the
raw result rows. in-args must already be free of ParamRef sentinels
(substituted at Bind for the parameterised path).
Forces the query planner on regardless of caller context — Datahike's base (relational) engine can't evaluate the recursive bodies translate-recursive-cte emits (head var bound through a function op then filtered by a predicate, datahike PR #825).
Evaluate a recursive-CTE Datalog rule to a fixed point and return the raw result rows. `in-args` must already be free of ParamRef sentinels (substituted at Bind for the parameterised path). Forces the query planner on regardless of caller context — Datahike's base (relational) engine can't evaluate the recursive bodies translate-recursive-cte emits (head var bound through a function op then filtered by a predicate, datahike PR #825).
(select-item-alias item)The explicit AS label of a select item, or nil.
An UNQUOTED alias is down-cased, the way PG's lexer folds every
unquoted identifier (scan.l's downcase_truncate_identifier), so
SELECT 1 AS Foo names the column foo. A quoted one keeps its
case: AS "Foo" stays Foo.
The explicit `AS` label of a select item, or nil. An UNQUOTED alias is down-cased, the way PG's lexer folds every unquoted identifier (scan.l's `downcase_truncate_identifier`), so `SELECT 1 AS Foo` names the column `foo`. A quoted one keeps its case: `AS "Foo"` stays `Foo`.
(translate-cte-branch ps
cte-name
col-names
rule-vars
rule-name
schema
db
virtual-cte-schema)Translate one branch of a recursive CTE (anchor or recursive PlainSelect) into Datalog rule body clauses.
Returns a vector of clauses for use as a rule body, with the SELECT items bound to the rule output vars.
Translate one branch of a recursive CTE (anchor or recursive PlainSelect) into Datalog rule body clauses. - cte-name: the CTE's name (e.g. "__parent_store_compute") - col-names: CTE column names in order (e.g. ["id" "parent_path"]) - rule-vars: corresponding rule output vars (e.g. [?id ?parent_path]) - rule-name: the rule's name as a symbol (for self-references) - schema, db: from the outer context - virtual-cte-schema: if non-nil, the CTE is referenceable as a virtual table in this branch (for the recursive branch only). Returns a vector of clauses for use as a rule body, with the SELECT items bound to the rule output vars.
(translate-delete delete schema)Translate a DELETE statement to Datahike retraction query + tx-data.
Translate a DELETE statement to Datahike retraction query + tx-data.
(translate-insert insert schema db)Translate an INSERT statement to Datahike transaction data. Supports single-row and multi-row VALUES, with or without column list. Handles ON CONFLICT (UPSERT) via :db.fn/call for atomic execution.
Translate an INSERT statement to Datahike transaction data. Supports single-row and multi-row VALUES, with or without column list. Handles ON CONFLICT (UPSERT) via :db.fn/call for atomic execution.
(translate-join ctx join _default-table)Add join clauses for a SQL JOIN to the context. For INNER joins with ref-based ON (a.ref_col = b.db_id), unifies the ref column variable with the target entity variable. For LEFT joins, records the ref-attr and right-table alias for later or-join wrapping in translate-select. Returns {:name str :alias str :join-type keyword :ref-attr kw :left-entity-var sym}.
Add join clauses for a SQL JOIN to the context.
For INNER joins with ref-based ON (a.ref_col = b.db_id), unifies the ref
column variable with the target entity variable.
For LEFT joins, records the ref-attr and right-table alias for later
or-join wrapping in translate-select.
Returns {:name str :alias str :join-type keyword :ref-attr kw :left-entity-var sym}.(translate-recursive-cte wi schema db)Translate a WITH RECURSIVE CTE definition into a Datalog rule. Returns: {:rule [...] :rule-name sym :col-names [...] :rule-vars [...] :in-params [...] :in-args [...]}
Translate a WITH RECURSIVE CTE definition into a Datalog rule.
Returns: {:rule [...] :rule-name sym :col-names [...] :rule-vars [...]
:in-params [...] :in-args [...]}(translate-select select schema & [db])Translate a SELECT while sharing anonymous relation identities across its complete nested query tree.
Translate a SELECT while sharing anonymous relation identities across its complete nested query tree.
(translate-select* select schema & [db])Translate a PlainSelect into a Datalog query map + metadata. Returns {:query map :find-aliases [...] :has-aggregates? bool}
Translate a PlainSelect into a Datalog query map + metadata.
Returns {:query map :find-aliases [...] :has-aggregates? bool}(translate-update update schema db)Translate an UPDATE statement to Datahike retract+assert pairs. Handles UPDATE with WITH RECURSIVE CTE — for these, the result is {:type :update-with-recursive ...} containing the rule, columns, and target table info for the server to execute.
Translate an UPDATE statement to Datahike retract+assert pairs.
Handles UPDATE with WITH RECURSIVE CTE — for these, the result is
{:type :update-with-recursive ...} containing the rule, columns,
and target table info for the server to execute.(validate-lateral-join-shapes! statement)Reject a RIGHT/FULL JOIN whose table function has a correlated column argument. PostgreSQL permits a LATERAL reference to the left side only for INNER and LEFT joins. This must run on the pristine parser AST: later translation passes may replace the table function while deriving schemas.
Reject a RIGHT/FULL JOIN whose table function has a correlated column argument. PostgreSQL permits a LATERAL reference to the left side only for INNER and LEFT joins. This must run on the pristine parser AST: later translation passes may replace the table function while deriving schemas.
(validate-recursive-cte-shape! wi)Raise PostgreSQL's structural error for a recursive CTE whose recursive reference is not combined with its anchor by UNION [ALL]. Called before fallback evaluators, whose capability probes intentionally catch errors.
Raise PostgreSQL's structural error for a recursive CTE whose recursive reference is not combined with its anchor by UNION [ALL]. Called before fallback evaluators, whose capability probes intentionally catch errors.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |