Prepared-statement parameter substitution and PG OID inference.
Three concerns live together here because they all deal with the
? / $N placeholder lifecycle:
ParamRef record + param-ref?. Emitted by translators at Parse
time inside tx-data / query structures, replaced by real values
at Execute time via substitute-params.
*bound-params* dynamic var: when bound to a 1-indexed vector
of resolved values, translator branches (e.g. the JdbcParameter
expression) resolve placeholders in-line instead of emitting
ParamRef. This lets the same translator body serve both
prepared-Parse (returns structured AST with ParamRefs) and
re-translation-during-Execute (returns fully-bound form).
OID inference (infer-param-oid-for-column,
insert-param-oids, update-param-oids, where-param-oids) —
walks a JSqlParser AST, maps each placeholder index to the PG
type OID of the column it's bound against. Consumed by
describeParams to populate the ParameterDescription message so
pgjdbc / psycopg2 / etc. size their binary binds correctly.
*parse-db* is bound by parse-sql (higher-level) to the current
db snapshot so OID inference can consult :pg/type metadata that
(:schema db) doesn't surface.
Prepared-statement parameter substitution and PG OID inference. Three concerns live together here because they all deal with the `?` / `$N` placeholder lifecycle: 1. ParamRef record + `param-ref?`. Emitted by translators at Parse time inside tx-data / query structures, replaced by real values at Execute time via `substitute-params`. 2. `*bound-params*` dynamic var: when bound to a 1-indexed vector of resolved values, translator branches (e.g. the JdbcParameter expression) resolve placeholders in-line instead of emitting ParamRef. This lets the same translator body serve both prepared-Parse (returns structured AST with ParamRefs) and re-translation-during-Execute (returns fully-bound form). 3. OID inference (`infer-param-oid-for-column`, `insert-param-oids`, `update-param-oids`, `where-param-oids`) — walks a JSqlParser AST, maps each placeholder index to the PG type OID of the column it's bound against. Consumed by describeParams to populate the ParameterDescription message so pgjdbc / psycopg2 / etc. size their binary binds correctly. `*parse-db*` is bound by parse-sql (higher-level) to the current db snapshot so OID inference can consult :pg/type metadata that (:schema db) doesn't surface.
Dynamically bound at Execute time to a 1-indexed vector (or nil if
no params). When set, translate-expr's JdbcParameter branch resolves
placeholders to concrete values in-line; otherwise (Parse time) it
emits ?pN in-param vars and records the index in ctx.
This split lets the same translator body serve both prepared-Parse and re-translation-during-Execute (UPDATE/DELETE keep where-expr as a JSqlParser AST and re-translate on each Execute).
Dynamically bound at Execute time to a 1-indexed vector (or nil if no params). When set, translate-expr's JdbcParameter branch resolves placeholders to concrete values in-line; otherwise (Parse time) it emits `?pN` in-param vars and records the index in ctx. This split lets the same translator body serve both prepared-Parse and re-translation-during-Execute (UPDATE/DELETE keep where-expr as a JSqlParser AST and re-translate on each Execute).
When bound (by build-update-tx handling UPDATE ... FROM (VALUES ...)),
a map {alias-name → {col-name → literal}} used by the Column branches
of translate-expr and eval-update-expr to substitute row-level values
for references like __tmp.col to the VALUES alias.
When bound (by build-update-tx handling UPDATE ... FROM (VALUES ...)),
a map {alias-name → {col-name → literal}} used by the Column branches
of translate-expr and eval-update-expr to substitute row-level values
for references like `__tmp.col` to the VALUES alias.Bound by parse-sql to the live db snapshot so downstream helpers (e.g. pg-type-of-attr) can consult Datahike for attribute metadata that :schema doesn't surface (:pg/type and friends). Not meant to flow beyond the parse phase — clear it before dispatching to the execute path.
Bound by parse-sql to the live db snapshot so downstream helpers (e.g. pg-type-of-attr) can consult Datahike for attribute metadata that :schema doesn't surface (:pg/type and friends). Not meant to flow beyond the parse phase — clear it before dispatching to the execute path.
Bound by parse-sql to itself so top-level translate-* entries in
datahike.pg.sql.stmt can seed :parse-sql into make-ctx without a
cyclic require on sql.clj. Downstream expression translators call
it to re-parse inner SQL strings for IN / EXISTS subqueries.
Bound by parse-sql to itself so top-level translate-* entries in datahike.pg.sql.stmt can seed `:parse-sql` into make-ctx without a cyclic require on sql.clj. Downstream expression translators call it to re-parse inner SQL strings for IN / EXISTS subqueries.
(ast-param-indices node)Recursively walk a JSqlParser AST, returning a sorted set of
1-based parameter indices (? / $N placeholders).
Uses reflection to traverse any node's zero-arg getters. This is called once per Parse to determine how many parameters a prepared statement expects — the cost is bounded by AST size, which is tiny compared to query execution.
Recursively walk a JSqlParser AST, returning a sorted set of 1-based parameter indices (`?` / `$N` placeholders). Uses reflection to traverse any node's zero-arg getters. This is called once per Parse to determine how many parameters a prepared statement expects — the cost is bounded by AST size, which is tiny compared to query execution.
(call-marker? v)True if v is a deferred function-call marker emitted by translate-*
(currently :nextval and :now). These must survive the result-
cache intact and be resolved per execute.
True if v is a deferred function-call marker emitted by translate-* (currently `:nextval` and `:now`). These must survive the result- cache intact and be resolved per execute.
(collect-table-aliases from-item joins)Given a FROM item and a sequence of JOINs, build a map
{alias-name → real-table-name}. Tables without an alias still get
an entry mapping the table name to itself (so lookup is uniform).
Given a FROM item and a sequence of JOINs, build a map
`{alias-name → real-table-name}`. Tables without an alias still get
an entry mapping the table name to itself (so lookup is uniform).(has-param-marker? sql)Fast scan: does SQL contain a ? or $N placeholder OUTSIDE a
quoted string, dollar-quoted body, or comment? When false, the
parser doesn't need to walk the AST for parameter indices — a
real win for pg_dump-style INSERTs (literal-only) where the
reflection-based AST walk dominated parse time.
pgjdbc rewrites ? to numbered $N before sending Parse, so the
on-wire SQL never has ? from a JDBC client — must detect both
forms.
Fast scan: does SQL contain a `?` or `$N` placeholder OUTSIDE a quoted string, dollar-quoted body, or comment? When false, the parser doesn't need to walk the AST for parameter indices — a real win for pg_dump-style INSERTs (literal-only) where the reflection-based AST walk dominated parse time. pgjdbc rewrites `?` to numbered `$N` before sending Parse, so the on-wire SQL never has `?` from a JDBC client — must detect both forms.
(infer-param-oid-for-column schema table-ns col-name)(infer-param-oid-for-column schema table-ns col-name db)Given a schema and a (table-namespace, column-name), return the PG OID that matches the attribute's :db/valueType, or nil if we don't know. Used by describeParams to populate ParameterDescription so drivers can size buffers correctly for typed INSERT/UPDATE/WHERE placeholders.
:pg/type on the attr — recorded by translate-create-table for types
that don't have a 1:1 Datahike mapping (date/time/timestamp all
collapse to :db.type/instant; jsonb vs json both use :db.type/string)
— takes precedence so we round-trip the original SQL type. Without
this, pgjdbc Describes a DATE column's param as timestamp (1114)
and rejects subsequent binds as "Can't change resolved type for
param: 1 from 1082 to 1114".
db is optional — pass nil and the :pg/type refinement is skipped.
Accepts schema either as the map {ident → props} returned by
pgs/schema-of, or as a seq of {:db/ident ... :db/valueType ...}
entries (the :db/add transaction form).
Given a schema and a (table-namespace, column-name), return the PG
OID that matches the attribute's :db/valueType, or nil if we don't
know. Used by describeParams to populate ParameterDescription so
drivers can size buffers correctly for typed INSERT/UPDATE/WHERE
placeholders.
:pg/type on the attr — recorded by translate-create-table for types
that don't have a 1:1 Datahike mapping (date/time/timestamp all
collapse to :db.type/instant; jsonb vs json both use :db.type/string)
— takes precedence so we round-trip the original SQL type. Without
this, pgjdbc Describes a DATE column's param as `timestamp` (1114)
and rejects subsequent binds as "Can't change resolved type for
param: 1 from 1082 to 1114".
`db` is optional — pass nil and the :pg/type refinement is skipped.
Accepts schema either as the map {ident → props} returned by
`pgs/schema-of`, or as a seq of `{:db/ident ... :db/valueType ...}`
entries (the `:db/add` transaction form).(insert-param-oids insert schema)(insert-param-oids insert schema db)Walk an INSERT AST: for each VALUES (..., ?, ...) row, positional
column i → attribute type → PG OID. Returns a map {param-index → oid}.
Only covers the flat single-row / multi-row VALUES case — which is what JDBC setObject/setString produces for the common ORM path.
When the INSERT omits the explicit column list (INSERT INTO t VALUES (?, ?, ?)), falls back to the table's declared column order
from pgs/column-info (which honours both schema entity-ID order and
the :datahike.pg/column-order hint). This is what pgjdbc's
executeBatch needs: setLong(1, …) wants param 1's OID at Describe
time, and without inferred OIDs pgjdbc's resolved-type tracker
raises Can't change resolved type for param: N from <oid> to 0.
Walk an INSERT AST: for each `VALUES (..., ?, ...)` row, positional
column i → attribute type → PG OID. Returns a map {param-index → oid}.
Only covers the flat single-row / multi-row VALUES case — which is
what JDBC setObject/setString produces for the common ORM path.
When the INSERT omits the explicit column list (`INSERT INTO t
VALUES (?, ?, ?)`), falls back to the table's declared column order
from `pgs/column-info` (which honours both schema entity-ID order and
the `:datahike.pg/column-order` hint). This is what pgjdbc's
`executeBatch` needs: setLong(1, …) wants param 1's OID at Describe
time, and without inferred OIDs pgjdbc's resolved-type tracker
raises `Can't change resolved type for param: N from <oid> to 0`.(nextval-marker? v)Back-compat alias: true only for the nextval flavour of call-marker.
Back-compat alias: true only for the nextval flavour of call-marker.
(param-ref? x)True when x is a parameter placeholder emitted by translate-* during
prepared-statement parsing. Appears in tx-data (INSERT/UPDATE VALUES)
and nested inside parsed structures that the handler walks at Bind/
Execute time to substitute real values.
True when `x` is a parameter placeholder emitted by translate-* during prepared-statement parsing. Appears in tx-data (INSERT/UPDATE VALUES) and nested inside parsed structures that the handler walks at Bind/ Execute time to substitute real values.
(resolve-nextvals! x nextval-fn)Walk x replacing every {:fn :nextval :seq-name S} marker with the
long produced by an actual nextval('S') against the live conn.
Each call commits independently via CAS-retry — same path
handle-nextval uses for SELECT nextval(...). PG semantics:
non-transactional advances; concurrent callers get distinct values.
nextval-fn is (fn [seq-name] long-or-throw). Decoupling the
resolver from the conn lets server.clj wire handle-nextval in
without params.clj taking a server.clj dependency.
Sibling shape to substitute-params: leaves functions, records,
and other opaque values alone, recurses into map values / vectors /
seqs.
Walk `x` replacing every `{:fn :nextval :seq-name S}` marker with the
long produced by an actual `nextval('S')` against the live conn.
Each call commits independently via CAS-retry — same path
`handle-nextval` uses for `SELECT nextval(...)`. PG semantics:
non-transactional advances; concurrent callers get distinct values.
`nextval-fn` is `(fn [seq-name] long-or-throw)`. Decoupling the
resolver from the conn lets server.clj wire `handle-nextval` in
without `params.clj` taking a server.clj dependency.
Sibling shape to `substitute-params`: leaves functions, records,
and other opaque values alone, recurses into map values / vectors /
seqs.(substitute-params x bound)Walk x replacing every ParamRef with the corresponding bound value
from bound (1-indexed: (->ParamRef 1) → (bound 1) ... so bound
is either a vector (we nth by idx-1) or a function idx→value).
Leaves everything else untouched.
The wire layer calls this at Execute time to resolve placeholders inside INSERT tx-data.
Maps with nil values after substitution have those keys dissoc'd.
An INSERT like INSERT INTO t (a, b) VALUES (?, ?) with
setString(1, null) ends up as {:t/a nil :t/b "x"} here —
d/transact rejects [:db/add eid :t/a nil] as :transact/syntax,
but the correct PG behaviour for a nullable column is to simply
not assert the attribute. The translate-time row-builder already
drops nil literals (NullValue), but those land as ParamRef sentinels
at parse time and only resolve to nil here.
Identity preservation: deferred call-markers ({:fn :nextval ...},
{:fn :now}) pass through unchanged — same Clojure object in,
same object out. Otherwise reduce-kv would mint new marker maps
and resolve-nextvals! would call the underlying function multiple
times when the same logical use appears in multiple parts of
tx-data (e.g. a :db.fn/call arg AND an outer entity-map via
assoc).
Walk `x` replacing every ParamRef with the corresponding bound value
from `bound` (1-indexed: `(->ParamRef 1)` → `(bound 1)` ... so `bound`
is either a vector (we nth by idx-1) or a function idx→value).
Leaves everything else untouched.
The wire layer calls this at Execute time to resolve placeholders
inside INSERT tx-data.
Maps with nil values after substitution have those keys dissoc'd.
An INSERT like `INSERT INTO t (a, b) VALUES (?, ?)` with
`setString(1, null)` ends up as `{:t/a nil :t/b "x"}` here —
`d/transact` rejects `[:db/add eid :t/a nil]` as `:transact/syntax`,
but the correct PG behaviour for a nullable column is to simply
not assert the attribute. The translate-time row-builder already
drops nil literals (NullValue), but those land as ParamRef sentinels
at parse time and only resolve to nil here.
Identity preservation: deferred call-markers (`{:fn :nextval ...}`,
`{:fn :now}`) pass through unchanged — same Clojure object in,
same object out. Otherwise reduce-kv would mint new marker maps
and resolve-nextvals! would call the underlying function multiple
times when the same logical use appears in multiple parts of
tx-data (e.g. a `:db.fn/call` arg AND an outer entity-map via
`assoc`).(unquote-ident s)Strip SQL double-quote delimiters from an identifier. PostgreSQL uses double quotes for case-sensitive or reserved-word identifiers: '"MyTable"' → 'MyTable', 'my_table' → 'my_table'
Strip SQL double-quote delimiters from an identifier. PostgreSQL uses double quotes for case-sensitive or reserved-word identifiers: '"MyTable"' → 'MyTable', 'my_table' → 'my_table'
(update-param-oids update schema)Walk an UPDATE AST: for each SET col = ?, map param index to the column attribute's PG OID.
Walk an UPDATE AST: for each SET col = ?, map param index to the column attribute's PG OID.
(where-param-oids expr schema default-table-ns)(where-param-oids expr schema default-table-ns aliases)Walk an expression tree (a WHERE clause) and for each comparison
col = ? / ? = col / col IN (?,?) / col BETWEEN ? AND ?,
map the ? param index to the column's PG OID. Best-effort.
default-table-ns is used when a column has no explicit table
qualifier. aliases is an optional {alias-name → real-table-name}
map (derived by the caller from FROM/JOIN clauses) so JOIN t a ON a.col = ? resolves a to t's real schema.
Walk an expression tree (a WHERE clause) and for each comparison
`col = ?` / `? = col` / `col IN (?,?)` / `col BETWEEN ? AND ?`,
map the `?` param index to the column's PG OID. Best-effort.
`default-table-ns` is used when a column has no explicit table
qualifier. `aliases` is an optional `{alias-name → real-table-name}`
map (derived by the caller from FROM/JOIN clauses) so `JOIN t a ON
a.col = ?` resolves `a` to `t`'s real schema.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |