metabase.sync.analyze.classify — metabase-core 1.0.0-SNAPSHOT

Analysis sub-step that takes a fingerprint for a Field and infers and saves appropriate information like special type. Each 'classifier' takes the information available to it and decides whether or not to run. We currently have the following classifiers:

1. name: Looks at the name of a Field and infers a special type if possible
2. no-preview-display: Looks at average length of text Field recorded in fingerprint and decides whether or not we should hide this Field
3. category: Looks at the number of distinct values of Field and determines whether it can be a Category
4. text-fingerprint: Looks at percentages recorded in a text Fields' TextFingerprint and infers a special type if possible

All classifier functions take two arguments, a FieldInstance and a possibly nil Fingerprint, and should return the Field with any appropriate changes (such as a new special type). If no changes are appropriate, a classifier may return nil. Error handling is handled by run-classifiers below, so individual classiers do not need to handle errors themselves.

In the future, we plan to add more classifiers, including ML ones that run offline.

classify-fields! clj

(classify-fields! table)

Inputs: [table :- i/TableInstance]

Run various classifiers on the appropriate FIELDS in a TABLE that have not been previously analyzed. These do things like inferring (and setting) the special types and preview display status for Fields belonging to TABLE.

classify-fields-for-db! clj

(classify-fields-for-db! database tables log-progress-fn)

Inputs: [database :- i/DatabaseInstance tables :- [i/TableInstance] log-progress-fn]

Classify all fields found in a given database

classify-table! clj

(classify-table! table)

Inputs: [table :- i/TableInstance]

Run various classifiers on the TABLE. These do things like inferring (and setting) entitiy type of TABLE.

classify-tables-for-db! clj

(classify-tables-for-db! database tables log-progress-fn)

Inputs: [database :- i/DatabaseInstance tables :- [i/TableInstance] log-progress-fn]

Classify all tables found in a given database

run-classifiers clj

(run-classifiers field fingerprint)

Inputs: [field :- i/FieldInstance fingerprint :- (s/maybe i/Fingerprint)] Returns: i/FieldInstance

Run all the available classifiers against FIELD and FINGERPRINT, and return the resulting FIELD with changes decided upon by the classifiers.