PostgreSQL COPY-IN text-format decoder. Pure data transformation:
bytes / string chunks in, vectors of fields out. Field values are
either Strings or the sentinel ::null (used in place of nil so
downstream tx-data builders can distinguish a missing column from
an explicit NULL).
Spec source: ../postgres/doc/src/sgml/ref/copy.sgml and
../postgres/src/backend/commands/copyfromparse.c.
Salient mechanics:
\n, \r, or \r\n. Once the
first terminator is observed, the EOL type is locked for the
rest of the stream — mid-stream switching is an error per PG.\..
Bytes after the EOD line are silently discarded.\N). Checked against
the raw (pre-de-escape) field text before backslash
processing.\t). Must be backslash-escaped to appear as a data byte.\b \f \n \r \t \v for control chars; \NNN octal
(1-3 digits); \xNN hex (1-2 digits); \<any-other-char>
passes the char through literally.Streaming API:
(def d (make-decoder opts)) (let [[d' rows eod?] (decode-step d chunk)] ;; called per CopyData ...) (let [[rows eod?] (decode-finalize d')] ;; called on CopyDone ...)
opts keys: :delimiter (1-char String, required),
:null-marker (String, required).
Output: rows are vectors of (String | ::null).
PostgreSQL COPY-IN text-format decoder. Pure data transformation:
bytes / string chunks in, vectors of fields out. Field values are
either Strings or the sentinel `::null` (used in place of `nil` so
downstream tx-data builders can distinguish a missing column from
an explicit NULL).
Spec source: `../postgres/doc/src/sgml/ref/copy.sgml` and
`../postgres/src/backend/commands/copyfromparse.c`.
Salient mechanics:
- **Lines** are delimited by `\n`, `\r`, or `\r\n`. Once the
first terminator is observed, the EOL type is locked for the
rest of the stream — mid-stream switching is an error per PG.
- **End-of-data** marker: a line containing exactly `\.`.
Bytes after the EOD line are silently discarded.
- **NULL marker**: configurable (default `\N`). Checked against
the *raw* (pre-de-escape) field text before backslash
processing.
- **Field separator**: configurable single ASCII byte (default
`\t`). Must be backslash-escaped to appear as a data byte.
- **Backslash escapes** (recognised after delimiter/EOD parsing):
`\b \f \n \r \t \v` for control chars; `\NNN` octal
(1-3 digits); `\xNN` hex (1-2 digits); `\<any-other-char>`
passes the char through literally.
Streaming API:
(def d (make-decoder opts))
(let [[d' rows eod?] (decode-step d chunk)] ;; called per CopyData
...)
(let [[rows eod?] (decode-finalize d')] ;; called on CopyDone
...)
`opts` keys: `:delimiter` (1-char String, required),
`:null-marker` (String, required).
Output: rows are vectors of (String | `::null`).(decode-all opts chunks)Convenience: decode an entire seq of chunks and return all rows. Used by tests; production wire-protocol path calls decode-step / decode-finalize directly.
Convenience: decode an entire seq of chunks and return all rows. Used by tests; production wire-protocol path calls decode-step / decode-finalize directly.
(decode-finalize decoder)Called after the wire layer signals CopyDone. Returns [rows eod?]. Any unterminated trailing data is treated as a single final line (pg accepts files without a trailing newline). If eod? was already set, the trailing buffer is ignored.
Called after the wire layer signals CopyDone. Returns [rows eod?]. Any unterminated trailing data is treated as a single final line (pg accepts files without a trailing newline). If eod? was already set, the trailing buffer is ignored.
(decode-step decoder chunk)Process one chunk. Returns [decoder' rows eod?]. After eod? is true, further chunks are silently discarded.
Process one chunk. Returns [decoder' rows eod?]. After eod? is true, further chunks are silently discarded.
(make-decoder {:keys [delimiter null-marker] :as opts})Build a fresh decoder state. opts map keys:
:delimiter — single-character String
:null-marker — String matched against the raw field text
Build a fresh decoder state. `opts` map keys: :delimiter — single-character String :null-marker — String matched against the raw field text
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |