cinfix.tokenizer — cinfix 0.3.0

+re-2char-punct-symbol+ clj

+re-bigdecimal-number+ clj

+re-bignum+ clj

+re-double-number+ clj

+re-ident+ clj

+re-int-number+ clj

+re-open-bracket+ clj

+re-open-parent+ clj

+re-punct-symbol+ clj

read-seq clj

(read-seq s startstr endstr)

sをendstrがヒットするまで分解して返す

read-word clj

(read-word s)

tokenize clj

(tokenize s)

入力文字列を単語で分かち書きし、reader-stringで読める文字列を作って返す