Liking cljdoc? Tell your friends :D

metabase.query-processor.middleware.add-dimension-projections

Middleware for adding remapping and other dimension related projections. This remaps Fields that have a corresponding Dimension object (which defines a remapping) in two different ways, depending on the :type attribute of the Dimension:

external type Dimensions mean the Field's values will be replaced with corresponding values from a column on a different table, joined via a foreign key. A common use-case would be to replace FK IDs with the name of whatever it references, for example replacing a values of venue.category_id with values of category.name. Actual replacement of values happens on the frontend, so this middleware simply adds the column to be used for replacement (e.g. category.name) to the :fields clause in pre-processing, so the Field will be fetched. Recall that Fields referenced via with :fk-> clauses imply that JOINs will take place, which are automatically handled later in the Query Processor pipeline. Additionally, this middleware will swap out and :order-by clauses referencing the original Field with ones referencing the remapped Field (for example, so we would sort by category.name instead of category_id).

internal type Dimensions mean the Field's values are replaced by a user-defined map of values, stored in the human_readable_values column of a corresponding FieldValues object. A common use-case for this scenario would be to replace integer enum values with something more descriptive, for example replacing values of an enum can_type -- 0 becomes Toucan, 1 becomes Pelican, and so forth. This is handled exclusively in post-processing by adding extra columns and values to the results.

In both cases, to accomplish values replacement on the frontend, the post-processing part of this middleware adds appropriate :remapped_from and :remapped_to attributes in the result :cols in post-processing. :remapped_from and :remapped_to are the names of the columns, e.g. category_id is :remapped_to name, and name is :remapped_from :category_id.

Middleware for adding remapping and other dimension related projections. This remaps Fields that have a corresponding
Dimension object (which defines a remapping) in two different ways, depending on the `:type` attribute of the
Dimension:

`external` type Dimensions mean the Field's values will be replaced with corresponding values from a column on a
different table, joined via a foreign key. A common use-case would be to replace FK IDs with the name of whatever it
references, for example replacing a values of `venue.category_id` with values of `category.name`. Actual replacement
of values happens on the frontend, so this middleware simply adds the column to be used for replacement (e.g.
`category.name`) to the `:fields` clause in pre-processing, so the Field will be fetched. Recall that Fields
referenced via with `:fk->` clauses imply that JOINs will take place, which are automatically handled later in the
Query Processor pipeline. Additionally, this middleware will swap out and `:order-by` clauses referencing the
original Field with ones referencing the remapped Field (for example, so we would sort by `category.name` instead of
`category_id`).

`internal` type Dimensions mean the Field's values are replaced by a user-defined map of values, stored in the
`human_readable_values` column of a corresponding `FieldValues` object. A common use-case for this scenario would be
to replace integer enum values with something more descriptive, for example replacing values of an enum `can_type`
-- `0` becomes `Toucan`, `1` becomes `Pelican`, and so forth. This is handled exclusively in post-processing by
adding extra columns and values to the results.

In both cases, to accomplish values replacement on the frontend, the post-processing part of this middleware adds
appropriate `:remapped_from` and `:remapped_to` attributes in the result `:cols` in post-processing.
`:remapped_from` and `:remapped_to` are the names of the columns, e.g. `category_id` is `:remapped_to` `name`, and
`name` is `:remapped_from` `:category_id`.
raw docstring

metabase.query-processor.middleware.add-implicit-clauses

Middlware for adding an implicit :fields and :order-by clauses to certain queries.

Middlware for adding an implicit `:fields` and `:order-by` clauses to certain queries.
raw docstring

metabase.query-processor.middleware.add-implicit-joins

Middleware that creates corresponding :joins for Tables referred to by :fk-> clauses and replaces those clauses with :joined-field clauses.

Middleware that creates corresponding `:joins` for Tables referred to by `:fk->` clauses and replaces those clauses
with `:joined-field` clauses.
raw docstring

metabase.query-processor.middleware.add-row-count-and-status

Middleware for adding :row_count and :status info to QP results.

Middleware for adding `:row_count` and `:status` info to QP results.
raw docstring

metabase.query-processor.middleware.add-settings

Middleware for adding a :settings map to a query before it is processed.

Middleware for adding a `:settings` map to a query before it is processed.
raw docstring

metabase.query-processor.middleware.annotate

Middleware for annotating (adding type information to) the results of a query, under the :cols column.

Middleware for annotating (adding type information to) the results of a query, under the `:cols` column.
raw docstring

metabase.query-processor.middleware.async

Middleware for implementing async QP behavior.

Middleware for implementing async QP behavior.
raw docstring

metabase.query-processor.middleware.async-wait

Middleware that limits the number of concurrent queries for each database.

Each connected database is limited to a maximum of 15 simultaneous queries (configurable) using these methods; any additional queries will park the thread. Super-useful for writing high-performance API endpoints. Prefer these methods to the old-school synchronous versions.

How is this achieved? For each Database, we'll maintain a thread pool executor to limit the number of simultaneous queries.

Middleware that limits the number of concurrent queries for each database.

Each connected database is limited to a maximum of 15 simultaneous queries (configurable) using these methods; any
additional queries will park the thread. Super-useful for writing high-performance API endpoints. Prefer these
methods to the old-school synchronous versions.

How is this achieved? For each Database, we'll maintain a thread pool executor to limit the number of simultaneous
queries.
raw docstring

metabase.query-processor.middleware.auto-bucket-datetimes

Middleware for automatically bucketing unbucketed :type/DateTime (but not :type/Time) Fields with :day bucketing. Applies to any unbucketed Field in a breakout, or fields in a filter clause being compared against yyyy-MM-dd format datetime strings.

Middleware for automatically bucketing unbucketed `:type/DateTime` (but not `:type/Time`) Fields with `:day`
bucketing. Applies to any unbucketed Field in a breakout, or fields in a filter clause being compared against
`yyyy-MM-dd` format datetime strings.
raw docstring

metabase.query-processor.middleware.binning

Middleware that handles binning-strategy Field clauses. This adds a resolved-options map to every binning-strategy clause that contains the information query processors will need in order to perform binning.

Middleware that handles `binning-strategy` Field clauses. This adds a `resolved-options` map to every
`binning-strategy` clause that contains the information query processors will need in order to perform binning.
raw docstring

metabase.query-processor.middleware.cache

Middleware that returns cached results for queries when applicable.

If caching is enabled (enable-query-caching is true) cached results will be returned for Cards if possible. There's a global default TTL defined by the setting query-caching-default-ttl, but individual Cards can override this value with custom TTLs with a value for :cache_ttl.

For all other queries, caching is skipped.

Various caching backends are defined in metabase.query-processor.middleware.cache-backend namespaces. The default backend is db, which uses the application database; this value can be changed by setting the env var MB_QP_CACHE_BACKEND.

Refer to metabase.query-processor.middleware.cache-backend.interface for more details about how the cache backends themselves.

Middleware that returns cached results for queries when applicable.

If caching is enabled (`enable-query-caching` is `true`) cached results will be returned for Cards if possible.
There's a global default TTL defined by the setting `query-caching-default-ttl`, but individual Cards can override
this value with custom TTLs with a value for `:cache_ttl`.

For all other queries, caching is skipped.

Various caching backends are defined in `metabase.query-processor.middleware.cache-backend` namespaces. The default
backend is `db`, which uses the application database; this value can be changed by setting the env var
`MB_QP_CACHE_BACKEND`.

 Refer to `metabase.query-processor.middleware.cache-backend.interface` for more details about how the cache
backends themselves.
raw docstring

metabase.query-processor.middleware.cache-backend.interface

Interface used to define different Query Processor cache backends. Defining a backend is straightforward: define a new namespace with the pattern

metabase.query-processor.middleware.cache-backend.<backend>

Where backend is a key representing the backend, e.g. db, redis, or memcached.

In that namespace, create an object that reifies (or otherwise implements) IQueryProcessorCacheBackend. This object must be stored in a var called instance.

That's it. See metabase.query-processor.middleware.cache-backend.db for a complete example of how this is done.

Interface used to define different Query Processor cache backends.
Defining a backend is straightforward: define a new namespace with the pattern

  metabase.query-processor.middleware.cache-backend.<backend>

Where backend is a key representing the backend, e.g. `db`, `redis`, or `memcached`.

In that namespace, create an object that reifies (or otherwise implements) `IQueryProcessorCacheBackend`.
This object *must* be stored in a var called `instance`.

That's it. See `metabase.query-processor.middleware.cache-backend.db` for a complete example of how this is done.
raw docstring

metabase.query-processor.middleware.catch-exceptions

Middleware for catching exceptions thrown by the query processor and returning them in a friendlier format.

Middleware for catching exceptions thrown by the query processor and returning them in a friendlier format.
raw docstring

metabase.query-processor.middleware.constraints

Middleware that adds default constraints to limit the maximum number of rows returned to queries that specify the :add-default-userland-constraints? :middleware option.

Middleware that adds default constraints to limit the maximum number of rows returned to queries that specify the
`:add-default-userland-constraints?` `:middleware` option.
raw docstring

metabase.query-processor.middleware.cumulative-aggregations

Middlware for handling cumulative count and cumulative sum aggregations.

Middlware for handling cumulative count and cumulative sum aggregations.
raw docstring

metabase.query-processor.middleware.dev

Middleware that's only active in dev and test scenarios. These middleware functions do additional checks of query processor behavior that are undesirable in normal production use.

Middleware that's only active in dev and test scenarios. These middleware functions do additional checks of query
processor behavior that are undesirable in normal production use.
raw docstring

metabase.query-processor.middleware.driver-specific

Middleware that hands off to a driver's implementation of process-query-in-context, if any. If implemented, this effectively lets one inject custom driver-specific middleware for the QP. Drivers can use it to different things like rewrite queries as needed or perform special permissions checks.

Middleware that hands off to a driver's implementation of `process-query-in-context`, if any.
If implemented, this effectively lets one inject custom driver-specific middleware for the QP.
Drivers can use it to different things like rewrite queries as needed or perform special permissions checks.
raw docstring

metabase.query-processor.middleware.expand-macros

Middleware for expanding :metric and :segment 'macros' in unexpanded MBQL queries.

(:metric forms are expanded into aggregations and sometimes filter clauses, while :segment forms are expanded into filter clauses.)

TODO - this namespace is ancient and written with MBQL '95 in mind, e.g. it is case-sensitive. At some point this ought to be reworked to be case-insensitive and cleaned up.

Middleware for expanding `:metric` and `:segment` 'macros' in *unexpanded* MBQL queries.

(`:metric` forms are expanded into aggregations and sometimes filter clauses, while `:segment` forms are expanded
into filter clauses.)

 TODO - this namespace is ancient and written with MBQL '95 in mind, e.g. it is case-sensitive.
 At some point this ought to be reworked to be case-insensitive and cleaned up.
raw docstring

metabase.query-processor.middleware.fetch-source-query

Middleware responsible for 'hydrating' the source query for queries that use another query as their source. This middleware looks for MBQL queries like

{:source-table "card__1" ; Shorthand for using Card 1 as source query ...}

and resolves the referenced source query, transforming the query to look like the following:

{:source-query {...} ; Query for Card 1 :source-metadata [...] ; metadata about columns in Card 1 ...}

This middleware resolves Card ID :source-tables at all levels of the query, but the top-level query often uses the so-called virtual-id, because the frontend client might not know the original Database; this middleware will replace that ID with the approiate ID, e.g.

{:database <virtual-id>, :type :query, :query {:source-table "card__1"}} -> {:database 1, :type :query, :query {:source-query {...}, :source-metadata {...}}}

TODO - consider renaming this namespace to metabase.query-processor.middleware.resolve-card-id-source-tables

Middleware responsible for 'hydrating' the source query for queries that use another query as their source. This
middleware looks for MBQL queries like

  {:source-table "card__1" ; Shorthand for using Card 1 as source query
   ...}

and resolves the referenced source query, transforming the query to look like the following:

  {:source-query    {...} ; Query for Card 1
   :source-metadata [...] ; metadata about columns in Card 1
   ...}

This middleware resolves Card ID `:source-table`s at all levels of the query, but the top-level query often uses the
so-called `virtual-id`, because the frontend client might not know the original Database; this middleware will
replace that ID with the approiate ID, e.g.

  {:database <virtual-id>, :type :query, :query {:source-table "card__1"}}
  ->
  {:database 1, :type :query, :query {:source-query {...}, :source-metadata {...}}}

TODO - consider renaming this namespace to `metabase.query-processor.middleware.resolve-card-id-source-tables`
raw docstring

metabase.query-processor.middleware.format-rows

Middleware that formats the results of a query. Currently, the only thing this does is convert datetime types to ISO-8601 strings in the appropriate timezone.

Middleware that formats the results of a query.
Currently, the only thing this does is convert datetime types to ISO-8601 strings in the appropriate timezone.
raw docstring

metabase.query-processor.middleware.limit

Middleware that handles limiting the maximum number of rows returned by a query.

Middleware that handles limiting the maximum number of rows returned by a query.
raw docstring

metabase.query-processor.middleware.log

Middleware for logging a query before it is processed. (Various other middleware functions log the query as well in different stages.)

Middleware for logging a query before it is processed.
(Various other middleware functions log the query as well in different stages.)
raw docstring

metabase.query-processor.middleware.mbql-to-native

Middleware responsible for converting MBQL queries to native queries (by calling the driver's QP methods) so the query can then be executed.

Middleware responsible for converting MBQL queries to native queries (by calling the driver's QP methods)
so the query can then be executed.
raw docstring

metabase.query-processor.middleware.normalize-query

Middleware that converts a query into a normalized, canonical form.

Middleware that converts a query into a normalized, canonical form.
raw docstring

metabase.query-processor.middleware.parameters

Middleware for substituting parameters in queries.

Middleware for substituting parameters in queries.
raw docstring

metabase.query-processor.middleware.parameters.dates

Shared code for handling datetime parameters, used by both MBQL and native params implementations.

Shared code for handling datetime parameters, used by both MBQL and native params implementations.
raw docstring

metabase.query-processor.middleware.parameters.mbql

Code for handling parameter substitution in MBQL queries.

Code for handling parameter substitution in MBQL queries.
raw docstring

metabase.query-processor.middleware.parameters.sql

Param substitution for SQL queries. This is a new implementation, fondly referred to as 'SQL parameters 2.0', written for v0.23.0. The new implementation uses prepared statement args instead of substituting them directly into the query, and is much better-organized and better-documented.

Param substitution for *SQL* queries.
This is a new implementation, fondly referred to as 'SQL parameters 2.0', written for v0.23.0.
The new implementation uses prepared statement args instead of substituting them directly into the query,
and is much better-organized and better-documented.
raw docstring

metabase.query-processor.middleware.permissions

Middleware for checking that the current user has permissions to run the current query.

Middleware for checking that the current user has permissions to run the current query.
raw docstring

metabase.query-processor.middleware.process-userland-query

Middleware related to doing extra steps for queries that are ran via API endpoints (i.e., most of them -- as opposed to queries ran internally e.g. as part of the sync process). These include things like saving QueryExecutions and formatting the results.

Middleware related to doing extra steps for queries that are ran via API endpoints (i.e., most of them -- as opposed
to queries ran internally e.g. as part of the sync process). These include things like saving QueryExecutions and
formatting the results.
raw docstring

metabase.query-processor.middleware.reconcile-breakout-and-order-by-bucketing

SQL places restrictions when using a GROUP BY clause (MBQL :breakout) in combination with an ORDER BY clause (MBQL :order-by) -- columns that appear in the ORDER BY must appear in the GROUP BY. When we apply datetime or binning bucketing in a breakout, for example cast(x AS DATE) (MBQL :datetime-field clause), we need to apply the same bucketing to instances of that Field in the order-by clause. In other words:

Bad:

SELECT count(*) FROM table GROUP BY CAST(x AS date) ORDER BY x ASC

(MBQL)

{:source-table 1 :breakout [[:datetime-field [:field-id 1] :day]] :order-by [[:asc [:field-id 1]]]}

Good:

SELECT count(*) FROM table GROUP BY CAST(x AS date) ORDER BY CAST(x AS date) ASC

(MBQL)

{:source-table 1 :breakout [[:datetime-field [:field-id 1] :day]] :order-by [[:asc [:datetime-field [:field-id 1] :day]]]}

The frontend, on the rare occasion it generates a query that explicitly specifies an order-by clause, usually will generate one that directly corresponds to the bad example above. This middleware finds these cases and rewrites the query to look like the good example.

SQL places restrictions when using a `GROUP BY` clause (MBQL `:breakout`) in combination with an `ORDER BY`
clause (MBQL `:order-by`) -- columns that appear in the `ORDER BY` must appear in the `GROUP BY`. When we apply
datetime or binning bucketing in a breakout, for example `cast(x AS DATE)` (MBQL `:datetime-field` clause), we need
to apply the same bucketing to instances of that Field in the `order-by` clause. In other words:

Bad:

  SELECT count(*)
  FROM table
  GROUP BY CAST(x AS date)
  ORDER BY x ASC

(MBQL)

   {:source-table 1
    :breakout     [[:datetime-field [:field-id 1] :day]]
    :order-by     [[:asc [:field-id 1]]]}

Good:

  SELECT count(*)
  FROM table
  GROUP BY CAST(x AS date)
  ORDER BY CAST(x AS date) ASC

(MBQL)

  {:source-table 1
   :breakout     [[:datetime-field [:field-id 1] :day]]
   :order-by     [[:asc [:datetime-field [:field-id 1] :day]]]}

The frontend, on the rare occasion it generates a query that explicitly specifies an `order-by` clause, usually will
generate one that directly corresponds to the bad example above. This middleware finds these cases and rewrites the
query to look like the good example.
raw docstring

metabase.query-processor.middleware.resolve-driver

Middleware for resolving the appropriate driver to use for processing a query.

Middleware for resolving the appropriate driver to use for processing a query.
raw docstring

metabase.query-processor.middleware.resolve-fields

Middleware that resolves the Fields referenced by a query.

Middleware that resolves the Fields referenced by a query.
raw docstring

metabase.query-processor.middleware.resolve-joins

Middleware that fetches tables that will need to be joined, referred to by fk-> clauses, and adds information to the query about what joins should be done and how they should be performed.

Middleware that fetches tables that will need to be joined, referred to by `fk->` clauses, and adds information to
the query about what joins should be done and how they should be performed.
raw docstring

metabase.query-processor.middleware.resolve-source-table

Fetches Tables corresponding to any :source-table IDs anywhere in the query.

Fetches Tables corresponding to any `:source-table` IDs anywhere in the query.
raw docstring

metabase.query-processor.middleware.results-metadata

Middleware that stores metadata about results column types after running a query for a Card, and returns that metadata (which can be passed back to the backend when saving a Card) as well as a checksum in the API response.

Middleware that stores metadata about results column types after running a query for a Card,
and returns that metadata (which can be passed *back* to the backend when saving a Card) as well
as a checksum in the API response.
raw docstring

metabase.query-processor.middleware.store

The store middleware is responsible for initializing a fresh QP Store, which caches resolved objects for the duration of a query execution. See metabase.query-processor.store for more details.

The store middleware is responsible for initializing a fresh QP Store, which caches resolved objects for the duration
of a query execution. See `metabase.query-processor.store` for more details.
raw docstring

metabase.query-processor.middleware.validate

Middleware for checking that a normalized query is valid.

Middleware for checking that a normalized query is valid.
raw docstring

metabase.query-processor.middleware.wrap-value-literals

Middleware that wraps value literals in value/absolute-datetime/etc. clauses containing relevant type information; parses datetime string literals when appropriate.

Middleware that wraps value literals in `value`/`absolute-datetime`/etc. clauses containing relevant type
information; parses datetime string literals when appropriate.
raw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close