Token grouping stage: collapses opaque regions (reader conditionals, namespaced maps, syntax-quote with brackets) from flat token sequences into single composite tokens.
This replaces the tokenizer's read-balanced-raw by operating on already-tokenized input where bracket matching is trivial — strings, chars, and comments are already individual tokens.
Token grouping stage: collapses opaque regions (reader conditionals, namespaced maps, syntax-quote with brackets) from flat token sequences into single composite tokens. This replaces the tokenizer's read-balanced-raw by operating on already-tokenized input where bracket matching is trivial — strings, chars, and comments are already individual tokens.
(group-tokens tokens source)Process a flat token vector, collapsing opaque regions into single tokens. Marker tokens (:reader-cond-start, :namespaced-map-start, :syntax-quote-start) followed by balanced delimiters are collapsed into the corresponding -raw composite tokens.
source is the original source text, used for reconstructing raw values.
Process a flat token vector, collapsing opaque regions into single tokens. Marker tokens (:reader-cond-start, :namespaced-map-start, :syntax-quote-start) followed by balanced delimiters are collapsed into the corresponding -raw composite tokens. source is the original source text, used for reconstructing raw values.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |