meme.alpha.pipeline — io.github.beme-lang/meme-clj 0.10.0

Explicit pipeline composition: source → step-scan → step-parse → step-expand-syntax-quotes → forms. Each stage is a ctx → ctx function operating on a shared context map.

Context map contract:

Key	Type	Written by	Read by
:source	String	caller	scan, parse
:opts	Map or nil	caller	parse, expand
:raw-tokens	Vector	scan	(tooling)
:tokens	Vector	scan	parse
:forms	Vector	parse, expand	expand, caller

:raw-tokens and :tokens are identical. Both keys are written by scan; :raw-tokens is retained so tooling that reads it continues to work.

Stages are independent functions. Compose them in any order that respects the read/write dependencies above. Guest languages can:

• Replace stages (e.g. a custom parser that reads :tokens, writes :forms)
• Add stages (e.g. a rewrite stage that reads :forms, writes :forms)
• Skip stages (e.g. skip expand for tooling that works with AST nodes)
• Read intermediate state (e.g. :raw-tokens for syntax highlighting)

See meme.alpha.pipeline.contract for formal clojure.spec definitions of the context map at each stage boundary. Enable runtime validation with: (binding [meme.alpha.pipeline.contract/validate true] (pipeline/run source))

run clj/s

(run source)

(run source opts)

Run the reader pipeline: source → tokens → forms. Returns the context map with :source, :raw-tokens, :tokens, :forms. Does NOT expand syntax-quote nodes — call expand separately for eval, or use run-string which includes expansion. The scan stage attaches whitespace metadata (:ws) to tokens.

step-expand-syntax-quotes clj/s

(step-expand-syntax-quotes ctx)

Expand syntax-quote AST nodes and unwrap MemeRaw values in :forms. Produces plain Clojure forms ready for eval. Not needed for tooling that works with AST nodes directly.

step-parse clj/s

(step-parse ctx)

Parse tokens into Clojure forms. If :parser is set in :opts, uses that function instead of the default meme parser. A custom parser has the signature: (fn [tokens opts source] -> forms-vector)

step-rewrite clj/s

(step-rewrite ctx)

Apply rewrite rules to :forms. Rules come from :rewrite-rules in :opts. No-op if no rules are provided. Each form is rewritten independently.

step-scan clj/s

(step-scan ctx)

Tokenize source text into tokens. Attaches leading whitespace/comments to each token as :ws. Writes both :tokens (for parse) and :raw-tokens (backward compat / tooling).