duckling.learn — org.msync/duckling 0.4.25

corpus->dataset clj

(corpus->dataset {context :context tests :tests :as corpus}
                 rules
                 feature-extractor)

Takes a corpus and a feature extractor and builds a dataset (phase 1.a. on duckling.md).

extract-route-features clj

(extract-route-features token)

Extracts names of previous routes used to produce this route token. This is the feature extractor we use.

judge-ml clj

(judge-ml stash classifiers)

Choose the winning token using a classifier. Computes prob of each rule according to their routes.

print-dataset clj

(print-dataset dataset)

Print dataset to STDOUT

route-prob clj

(route-prob route classifiers)

Computes the log prob for a route.

sentence->dataset clj

(sentence->dataset s context check rules feature-extractor dataset)

Takes a sentence, context, check (fn that determines if a winner is valid), feature extractor, and existing dataset. Returns an enriched dataset: [{<rule-name> [features, output]}] Output is true if the rule was contributing successfully, false otherwise

simple-feature-extractor clj

(simple-feature-extractor token)

A very simple one to show if it works. Not used for now. Takes a token, returns a vector of features (can be anything as long as the model understands it).

subtokens clj

(subtokens token)

Get a set of all the tokens in the tree who eventually produced the given token (including token itself)

train-classifiers clj

(train-classifiers corpus rules fextractor)

Given a corpus and a set of rules, train a classifier per rule