ari.lex — ari 0.1.0

General lexing module. Converts a raw text file into a list of tagged words based on defined lexical rules

create-taggers clj

(create-taggers tag-pairs)

Convert regular expressions into predicates

do-tag clj

(do-tag tokens taggers)

Tag a list of tokens using regex-based predicates

find-match clj

(find-match content separators)

Attempt to match a list of separators against a string.

front-match? clj

(front-match? a b)

Check if the leading elements of two sequences match

lex clj

(lex separators special-separators tag-pairs log content)

Convert a string (file) into tagged tokens

match-separator clj

(match-separator content separator c)

Attempt to match a single separator to a string, counting the number of characters that match.

separate clj

(separate content separators special-separators)

Preform lexing of both special and normal separators

separate-normal clj

(separate-normal content separators)

Break a string up by multi-character separators. Assumes that separators are in sorted order.

separate-special clj

(separate-special content specials)

Separate strings, comments, and other specially lexed elements from code

special-endings clj

special-tags clj