strojure.parsesso.core — com.github.strojure/parsesso 1.0.215-beta

Main namespace with parsers and their combinators.

after clj/s

(after q p)

(after q qq p)

(after q qq qqq & more)

This parser tries to apply the parsers in order, until last of them succeeds. Returns the value of the last parser, discards result of all preceding parsers.

• Fails: when any of tried parsers fails.
• Consumes: when any of tried parsers consumes some input.

any-token clj/s

This parser accepts any kind of token. Returns the accepted token.

• Fails: at the end of input.
• Consumes: when succeeds.

between clj/s

(between p around)

(between p open close)

Parses open, followed by p and close. Returns the value returned by p.

• Fails: when any of parses fail.
• Consumes: in all cases except when open fails without consuming any input.

Example:

(defn braces [p]
  (-> p (between (char/is "{")
                 (char/is "}"))))

bind clj/s

(bind p f)

This parser applies parser p and then parser (f x) where x is a return value of the parser p.

• Fails: when any of parsers p or (f x) fails.
• Consumes: when any of parsers p or (f x) consumes some input.

bind-let clj/s macro

(bind-let [& bindings] & body)

Expands into nested bind forms and a function body.

The pattern:

(bind p (fn [x]
          (bind q (fn [y]
                    ...
                    (result (f x y ...))))))

can be more conveniently be written as:

(bind-let [x p
           y q
           ...]
  (result (f x y ...)))

choice clj/s

(choice p q)

(choice p q qq)

(choice p q qq & more)

This parser tries to apply the parsers in order, until one of them succeeds. Returns the value of the succeeding parser.

• Fails:
  • when any of tried parsers fails consuming some input.
  • when all tried parsers fail without consuming any input.
• Consumes:
  • when any of tried parsers consumes some input.

The parser first applies p. If it succeeds, the value of p is returned. If p fails without consuming any input, parser q is tried and so on.

The parser is called predictive since q is only tried when parser p didn't consume any input (i.e. the look ahead is 1). This non-backtracking behaviour allows for both an efficient implementation of the parser combinators and the generation of good error messages.

do-parser clj/s macro

(do-parser & body)

Delays the evaluation of a parser that was forward (declare)d and it has not been defined yet. For use in (def)s of no-arg parsers, since the parser expression evaluates immediately.

each clj/s

(each ps)

This parser tries to apply parsers in order until all of them succeeds. Returns a sequence of values returned by every parser.

• Fails: when any of tried parsers fails.
• Consumes: when any of tried parsers consumes some input.

eof clj/s

(eof)

(eof x)

This parser only succeeds with value x at the end of the input.

• Fails: when input is not completely consumed.
• Consumes: never.

expecting clj/s

(expecting p msg)

This parser behaves as parser p, but whenever the parser p fails without consuming any input, it replaces expect error messages with the expect error message msg.

This is normally used at the end of a set alternatives where we want to return an error message in terms of a higher level construct rather than returning all possible characters. For example, if the expr parser from the maybe example would fail, the error message is: '...: expecting expression'. Without the expecting combinator, the message would be like '...: expecting "let" or alphabetic character', which is less friendly.

The parsers fail, fail-unexpected and expecting are the three parsers used to generate error messages. Of these, only expecting is commonly used.

fail clj/s

(fail)

(fail msg)

This parser always fails with message msg without consuming any input.

• Fails: always.
• Consumes: never.

fail-unexpected clj/s

(fail-unexpected msg)

This parser always fails with an unexpected error message msg without consuming any input.

• Fails: always.
• Consumes: never.

look-ahead clj/s

(look-ahead p)

Parses p without consuming any input. If p fails and consumes some input, so does look-ahead. Combine with maybe if this is undesirable.

• Fails: when p fails.
• Consumes: when p fails and consumes some input.

many-till clj/s

(many-till p end)

This parser applies parser p zero or more times until parser end succeeds. Returns a sequence of values returned by p.

• Fails:
  • when p fails.
  • when end does not succeed before end of input.
• Consumes:
  • when p or end consumes some input.

Example:

(def simple-comment
  (after (word "<!--")
         (many-till any-token (maybe (word "-->")))))

Note the overlapping parsers any-token and (word "-->"), and therefore the use of the maybe combinator.

many0 clj/s

(many0 p)

This parser applies the parser p zero or more times. Returns a sequence of the returned values or p.

• Fails: when p fails and consumes some input.
• Consumes: when p consumes some input.

Example:

(def identifier
  (bind-let [c char/letter?
             cs (many0 (choice char/letter-or-number?
                               (char/is "_")))]
    (result (cons c cs))))

many1 clj/s

(many1 p)

This parser applies the parser p one or more times. Returns a sequence of the returned values of p.

• Fails: when p does not succeed at least once.
• Consumes: when p consumes some input.

Example:

(def word
  (many1 char/letter?)

maybe clj/s

(maybe p)

This parser behaves like parser p, except that it pretends that it hasn't consumed any input when an error occurs.

• Fails: when p fails.
• Consumes: when p succeeds and consumes some input.

This combinator is used whenever arbitrary look ahead is needed. Since it pretends that it hasn't consumed any input when p fails, the choice combinator will try its second alternative even when the first parser failed while consuming input.

The maybe combinator can for example be used to distinguish identifiers and reserved words. Both reserved words and identifiers are a sequence of letters. Whenever we expect a certain reserved word where we can also expect an identifier we have to use the maybe combinator. Suppose we write:

(def identifier
  (many1 char/letter?))

(def let-expr
  (after (word "let")
         ...))

(def expr
  (-> (choice let-expr
              identifier)
      (expecting "expression"))

If the user writes "lexical", the parser fails with: unexpected "x", expecting "t" of (word "let"). Indeed, since the choice combinator only tries alternatives when the first alternative hasn't consumed input, the identifier parser is never tried (because the prefix "le" of the (word "let") parser is already consumed). The right behaviour can be obtained by adding the maybe combinator:

(def let-expr
  (after (maybe (word "let"))
         ...))

not-followed-by clj/s

(not-followed-by q)

(not-followed-by p q)

This parser behaves like parser p, except that it only succeeds when parser q fails. This parser can be used to implement the 'longest match' rule. For example, when recognizing keywords (for example let), we want to make sure that a keyword is not followed by a legal identifier character, in which case the keyword is actually an identifier (for example lets). We can write this behaviour as follows:

(-> (word "let")
    (not-followed-by char/letter-or-number?))

• Fails:
  • when p fails.
  • when q succeeds.
• Consumes:
  • when p consumes some input.

option clj/s

(option p)

(option p x)

This parser tries to apply parser p. If p fails without consuming input, it returns the value x (or nil), otherwise the value returned by p.

• Fails: when p fails and consumes come input.
• Consumes: when p consumes some input.

parse clj/s≠

clj

(parse p input)

(parse p input {:keys [pos tab user-state] :as options})

cljs

(parse p input)

(parse p input opts)

Executes parser p given input sequence of tokens, returns result value or throws exception on parsing error.

Options:

:pos The instance of InputPos or keyword for [[pos/init-pos]] to init parser pos. By default pos is initialized to TextPos for string input or first token of char type, or IndexPos otherwise.

:tab Tab size for TextPos, which is 8 by default.

:user-state Initial value of user state.

parse* clj/s≠

clj

(parse* p input)

(parse* p input {:keys [pos tab user-state] :as options})

cljs

(parse* p input)

(parse* p input opts)

Executes parser p given input sequence of tokens, returns reply record. See parse for available options.

register-word-test clj/s

(register-word-test k f)

Associates keyword k with test-fn of the word parser.

render clj/s

(render obj)

Returns string representation of the obj in parser error messages.

result clj/s

(result x)

This parser always succeeds with value x without consuming any input.

• Fails: never.
• Consumes: never.

sep0 clj/s

(sep0 p sep)

Parses zero or more occurrences of p, separated by sep. Returns a sequence of values returned by p.

(defn comma-sep [p]
  (sep0 p (after (char/is ",")
                 (skip0 char/white?))))

sep0-end clj/s

(sep0-end p sep)

Parses zero or more occurrences of p, separated and ended by sep. Returns a sequence of values returned by p.

sep0-opt clj/s

(sep0-opt p sep)

Parses zero or more occurrences of p, separated and optionally ended by sep. Returns a sequence of values returned by p.

sep1 clj/s

(sep1 p sep)

Parses one or more occurrences of p, separated by sep. Returns a sequence of values returned by p.

sep1-end clj/s

(sep1-end p sep)

Parses one or more occurrences of p, separated and ended by sep. Returns a sequence of values returned by p.

sep1-opt clj/s

(sep1-opt p sep)

Parses one or more occurrences of p, separated and optionally ended by sep. Returns a sequence of values returned by p.

skip0 clj/s

(skip0 p)

This parser applies the parser p zero or more times, skipping its result.

• Fails: when p fails and consumes some input.
• Consumes: when p consumes some input.

Example:

(def spaces
  (skip0 char/white?))

skip1 clj/s

(skip1 p)

This parser applies the parser p one or more times, skipping its result.

• Fails: when p does not succeed at least once.
• Consumes: when p consumes some input.

times clj/s

(times n p)

Parses n occurrences of p. If n is smaller or equal to zero, the parser equals to (return nil). Returns a sequence of n values returned by p.

token clj/s

(token pred)

(token pred msg)

This parser accepts a token when (pred token) returns logical true, and optional expecting msg. 1-arity behaves as pred and can be used in predicate composition.

• Fails: when (pred token) return logical false.
• Consumes: when succeeds.

trace clj/s

(trace label)

(trace label p)

This parser prints the parser state (position, remaining input and user state) at the time it is invoked. When p is provided it then continues to apply parser p, and if p fails will indicate that the label has been backtracked. It is intended to be used for debugging parsers by inspecting their intermediate states.

• Fails: when p fails.
• Consumes: when p consumes some input.

Examples:

(parse (after (char/is "aeiou")
              (trace "test-label"))
       "atest")

> test-label: at line 1, column 2
>  - input: (\t \e \s \t)
>  - user: nil

(parse (after (char/is "aeiou")
              (trace "test-label" (char/is "nope")))
       "atest")

> test-label: at line 1, column 2
>  - input: (\t \e \s \t)
>  - user: nil
> test-label: backtracked

> error at line 1, column 2:
> unexpected "t"
> expecting character of "nope"

tuple clj/s

(tuple p q & ps)

This parser tries to apply argument parsers in order until all of them succeeds. Returns a sequence of values returned by every parser. It is a 2+ arity version of the each parser.

• Fails: when any of tried parsers fails.
• Consumes: when any of tried parsers consumes some input.

using clj/s

(using p f)

(using p f g)

(using p f g h)

(using p f g h & more)

This parser applies series of functions to the result value of the parser p.

• Fails: when p fails.
• Consumes: when p consumes some input.

word clj/s

(word tokens)

(word tokens test-fn)

Parses a sequence of tokens given by ts and returns ts. The optional function (test-fn word-token input-token) is used to match tokens differently than simple equality. The test-fn can be referred by keyword registered using register-word-test. There are two predefined keywords registered: :default for = and :ic for case insensitive char comparison.

• Fails: when any of tokens don't match the input.
• Consumes: when at least first token matches the input.

Example:

(def let-keyword (word "let"))

(def let-keyword-ignorecase (word "let" :ic))