beme tokenizer: character scanning and token production. Transforms beme source text into a flat vector of typed tokens.
beme tokenizer: character scanning and token production. Transforms beme source text into a flat vector of typed tokens.
(attach-whitespace tokens source)Attach leading whitespace/comments to each token as :ws. Computes the gap between consecutive tokens in the source string. Trailing whitespace (after last token) is stored as :trailing-ws metadata on the returned vector.
Attach leading whitespace/comments to each token as :ws. Computes the gap between consecutive tokens in the source string. Trailing whitespace (after last token) is stored as :trailing-ws metadata on the returned vector.
(tokenize s)Tokenize beme source string into a vector of tokens.
Tokenize beme source string into a vector of tokens.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |