datahike.pg.sql.copy — org.replikativ/pg-datahike 0.1.279

Token-driven hand-parser for COPY. JSqlParser 5.x doesn't recognise COPY at all — UnsupportedStatement — so the wire-protocol layer needs structured access before it can drive the COPY-IN sub-protocol.

Also exposes row->entity-map: shared helper used by the COPY-IN exec handler (server.clj) to turn a vector of decoded String|::null values into a Datahike entity map keyed by :<ns>/<col> attributes, with per-column string→type coercion driven by :db/valueType.

Mirrors the structure of datahike.pg.sql.database: tokenise, parse the prefix (table + optional column list + FROM/TO target), then parse the option list in either:

• Modern paren form — WITH (key [=] value [, ...])
• Legacy keyword form — [WITH] kw1 kw2 ... (e.g. WITH BINARY, WITH CSV HEADER, DELIMITER '|' NULL '\N' CSV) still in the wild from old pg_dump output.

PG syntax (from ../postgres/doc/src/sgml/ref/copy.sgml):

COPY [schema.]table [ ( col [, ...] ) ] { FROM { 'file' | PROGRAM 'cmd' | STDIN } | TO { 'file' | PROGRAM 'cmd' | STDOUT } } [ [ WITH ] ( option [, ...] ) ]

Options accepted:

FORMAT 'text' | 'csv' | 'binary' DELIMITER 'X' — single byte NULL 'X' — null marker HEADER BOOL | MATCH — 1st row treatment QUOTE 'X' — CSV quote char ESCAPE 'X' — CSV escape char (defaults to QUOTE) FORCE_NOT_NULL ( col, ... ) | * FORCE_NULL ( col, ... ) | * FORCE_QUOTE ( col, ... ) | * — TO-only; we accept for COPY FROM as a no-op ENCODING 'UTF8' — UTF-8 aliases accepted; others rejected FREEZE [ BOOL ] — accepted, ignored DEFAULT 'X' — defaults-marker (PG 16+) OIDS [ BOOL ] — legacy, removed in PG 12; rejected

Output shape: {:type :copy-from-stdin :direction :from | :to :target :stdin | :stdout | {:file path} | {:program cmd} :ns string ;; lowercase table namespace :table string ;; original-case table name :columns [string ...] ;; lowercase, or nil if no col-list given :options {:format :text|:csv|:binary :delimiter String :null-marker String :quote String :escape String :header :true|:false|:match :force-not-null #{string ...} | :all :force-null #{string ...} | :all :encoding String :freeze? boolean :default-marker String}}

Defaults (filled by parse-copy-from): text: delimiter "\t", null-marker "\N" csv: delimiter ",", null-marker "", quote """, escape = quote, header :false binary: rejected at this layer (returns :feature-not-supported)

coerce-field clj

(coerce-field raw attr schema)

Convert a raw string from a COPY data row into the typed value expected by attr's :db/valueType. Returns the typed value, or the raw string if no coercion is recognised. Any string→long parse error throws ex-info :invalid-text-representation (the PG SQLSTATE 22P02 we surface to clients via the wire).

This is COPY-specific because all values arrive as strings; INSERT values come through JSqlParser typed and use coerce-insert-value for any further normalisation.

default-sentinel? clj

(default-sentinel? v)

True for the format-specific sentinel emitted by a raw COPY DEFAULT marker.

null-sentinel? clj

(null-sentinel? v)

options->map clj

(options->map opts-vec)

Translate the raw [["key" value] ...] option list (from either paren or legacy parser) into a normalised map with defaults applied. Throws :feature-not-supported for FORMAT 'binary' and :syntax-error for unknown options.

parse-copy clj

(parse-copy toks)

(parse-copy toks sql)

Parse a tokenised COPY statement in either direction.

Returns: {:db-name nil ;; for parity with database.clj parse shape :direction :from | :to :target :stdin | :stdout | {:file path} | {:program cmd} :ns lowercase-string-or-nil :table original-case-string :columns [lowercase-string ...] | nil :options normalised-options-map}

The parse is structural: which direction/target combinations the server can actually execute is decided there, not here, so that an unsupported one reports what it is rather than a syntax error.

Throws ex-info with :error :syntax-error on malformed input, or :feature-not-supported for COPY BINARY / OIDS.

row->entity-map clj

(row->entity-map row columns ns row-marker schema row-idx)

(row->entity-map row columns ns row-marker schema row-idx tempid-prefix)

Build a Datahike entity map from a single COPY data row.

row — vector of (String | text-null-sentinel | csv-null-sentinel) columns — vector of lower-case column-name strings (length must match row) ns — table namespace (string) row-marker — row-existence marker keyword (e.g. :users/db-row-exists) — set true on every entity so SELECT * row-marker filtering finds them, matching the convention pg-datahike's INSERT translator uses (stmt.clj:856). schema — Datahike :schema map row-idx — sequential row index tempid-prefix — statement-unique prefix; two COPY commands buffered in one transaction must never share Datahike tempids

Explicit NULL values remain nil until row validation, so they suppress column defaults. The validator removes nil keys before storage. Empty fields are kept as empty strings; ordered candidate preparation performs not-null and CHECK enforcement before the next source row.

split-copy-query clj

(split-copy-query sql)

For COPY ( query ) TO ..., return [query-text rest-sql] by scanning the raw SQL for the parenthesis that closes the one after COPY.

Tokens cannot answer this. The query has to reach the real parser verbatim, and re-assembling it from this namespace's tokens would hand it back whatever the tokenizer normalised -- so the split is made on the characters, skipping over quoted strings and quoted identifiers so a parenthesis inside one does not count.

Returns nil when the SQL does not have that shape.

tokenize clj