metabase.sync.sync-metadata.fields — metabase-core 1.0.0-SNAPSHOT

Logic for updating Metabase Field models from metadata fetched from a physical DB.

The basic idea here is to look at the metadata we get from calling describe-table on a connected database, then construct an identical set of metadata from what we have about that Table in the Metabase DB. Then we iterate over both sets of Metadata and perform whatever steps are needed to make sure the things in the DB match the things that came back from describe-table. These steps are broken out into three main parts:

• Fetch Metadata - logic is in metabase.sync.sync-metadata.fields.fetch-metadata. Construct a map of metadata from the Metabase application database that matches the form of DB metadata about Fields in a Table. This metadata is used to next two steps to determine what sync operations need to be performed by comparing the differences in the two sets of Metadata.

• Sync Field instances -- logic is in metabase.sync.sync-metadata.fields.sync-instances. Make sure the Field instances in the Metabase application database match up with those in the DB metadata, creating new Fields as needed, and marking existing ones as active or inactive as appropriate.

• Update instance metadata -- logic is in metabase.sync.sync-metadata.fields.sync-metadata. Update metadata properties of Field instances in the application database as needed -- this includes the base type, database type, special type, and comment/remark (description) properties. This primarily affects Fields that were not newly created; newly created Fields are given appropriate metadata when first synced (by sync-instances).

A note on terminology used in metabase.sync.sync-metadata.fields.* namespaces:

• db-metadata is a set of field-metadata maps coming back from the DB (e.g. from something like JDBC DatabaseMetaData) describing the columns (or equivalent) currently present in the table (or equivalent) that we're syncing.

• field-metadata is a map of information describing a single columnn currently present in the table being synced

• our-metadata is a set of maps of Field metadata reconstructed from the Metabase application database.

• metabase-field is a single map of Field metadata reconstructed from the Metabase application database; there is a 1:1 correspondance between this metadata and a row in the Field table. Unlike field-metadata, these entries always have an :id associated with them (because they are present in the Metabase application DB).

Other notes:

• In general the methods in these namespaces return the number of rows updated; these numbers are summed and used for logging purposes by higher-level sync logic.

sync-fields! clj

(sync-fields! database)

Inputs: [database :- i/DatabaseInstance] Returns: (s/maybe {:updated-fields su/IntGreaterThanOrEqualToZero, :total-fields su/IntGreaterThanOrEqualToZero})

Sync the Fields in the Metabase application database for all the Tables in a DATABASE.

sync-fields-for-table! clj

(sync-fields-for-table! table)

(sync-fields-for-table! database {old-hash :fields_hash :as table})

Inputs: ([table :- i/TableInstance] [database :- i/DatabaseInstance {old-hash :fields_hash, :as table} :- i/TableInstance]) Returns: {:updated-fields su/IntGreaterThanOrEqualToZero, :total-fields su/IntGreaterThanOrEqualToZero}

Sync the Fields in the Metabase application database for a specific table.