SQL query compiler
The query compile operates in a multi-step process. Compilation begins with
one of the foo-query->sql functions. The job of these functions is
basically to call compile-term on the first term of the query to get back
the "compiled" form of the query, and then to turn that into a complete SQL
query.
The compiled form of a query consists of a map with two keys: where
and params. The where key contains SQL for querying that
particular predicate, written in such a way as to be suitable for placement
after a WHERE clause in the database. params contains, naturally, the
parameters associated with that SQL expression. For instance, a resource
query for ["=" ["node" "name"] "foo.example.com"] will compile to:
{:where "catalogs.certname = ?"
:params ["foo.example.com"]}
The where key is then inserted into a template query to return
the final result as a string of SQL code.
The compiled query components can be combined by operators such as
AND or OR, which return the same sort of structure. Operators
which accept other terms as their arguments are responsible for
compiling their arguments themselves. To facilitate this, those
functions accept as their first argument a map from operator to
compile function. This allows us to have a different set of
operators for resources and facts, or queries, while still sharing
the implementation of the operators themselves.
Other operators include the subquery operators, in, extract, and
select-resources or select-facts. The select-foo operators implement
subqueries, and are simply implemented by calling their corresponding
foo-query->sql function, which means they return a complete SQL query
rather than the compiled query map. The extract function knows how to
handle that, and is the only place those queries are allowed as arguments.
extract is used to select a particular column from the subquery. The
sibling operator to extract is in, which checks that the value of
a certain column from the table being queried is in the result set returned
by extract. Composed, these three operators provide a complete subquery
facility. For example, consider this fact query:
["and"
["=" ["fact" "name"] "ipaddress"]
["in" "certname"
["extract" "certname"
["select-resources" ["and"
["=" "type" "Class"]
["=" "title" "apache"]]]]]]
This will perform a query (via select-resources) for resources matching
Class[apache]. It will then pick out the certname from each of those,
and match against the certname of fact rows, returning those facts which
have a corresponding entry in the results of select-resources and which
are named ipaddress. Effectively, the semantics of this query are "find
the ipaddress of every node with Class[apache]".
The resulting SQL from the foo-query->sql functions selects all the
columns. Thus consumers of those functions may need to wrap that query with
another SELECT to pull out only the desired columns. Similarly for
applying ordering constraints.
SQL query compiler
The query compile operates in a multi-step process. Compilation begins with
one of the `foo-query->sql` functions. The job of these functions is
basically to call `compile-term` on the first term of the query to get back
the "compiled" form of the query, and then to turn that into a complete SQL
query.
The compiled form of a query consists of a map with two keys: `where`
and `params`. The `where` key contains SQL for querying that
particular predicate, written in such a way as to be suitable for placement
after a `WHERE` clause in the database. `params` contains, naturally, the
parameters associated with that SQL expression. For instance, a resource
query for `["=" ["node" "name"] "foo.example.com"]` will compile to:
{:where "catalogs.certname = ?"
:params ["foo.example.com"]}
The `where` key is then inserted into a template query to return
the final result as a string of SQL code.
The compiled query components can be combined by operators such as
`AND` or `OR`, which return the same sort of structure. Operators
which accept other terms as their arguments are responsible for
compiling their arguments themselves. To facilitate this, those
functions accept as their first argument a map from operator to
compile function. This allows us to have a different set of
operators for resources and facts, or queries, while still sharing
the implementation of the operators themselves.
Other operators include the subquery operators, `in`, `extract`, and
`select-resources` or `select-facts`. The `select-foo` operators implement
subqueries, and are simply implemented by calling their corresponding
`foo-query->sql` function, which means they return a complete SQL query
rather than the compiled query map. The `extract` function knows how to
handle that, and is the only place those queries are allowed as arguments.
`extract` is used to select a particular column from the subquery. The
sibling operator to `extract` is `in`, which checks that the value of
a certain column from the table being queried is in the result set returned
by `extract`. Composed, these three operators provide a complete subquery
facility. For example, consider this fact query:
["and"
["=" ["fact" "name"] "ipaddress"]
["in" "certname"
["extract" "certname"
["select-resources" ["and"
["=" "type" "Class"]
["=" "title" "apache"]]]]]]
This will perform a query (via `select-resources`) for resources matching
`Class[apache]`. It will then pick out the `certname` from each of those,
and match against the `certname` of fact rows, returning those facts which
have a corresponding entry in the results of `select-resources` and which
are named `ipaddress`. Effectively, the semantics of this query are "find
the ipaddress of every node with Class[apache]".
The resulting SQL from the `foo-query->sql` functions selects all the
columns. Thus consumers of those functions may need to wrap that query with
another `SELECT` to pull out only the desired columns. Similarly for
applying ordering constraints.(column-map->sql col-map)Helper function that converts one of our column maps to a SQL string suitable for use in a SELECT
Helper function that converts one of our column maps to a SQL string suitable for use in a SELECT
(compile-boolean-operator* op ops & terms)Compile a term for the boolean operator op (AND or OR) applied to
terms. This is accomplished by compiling each of the terms and then just
joining their where terms with the operator. The params are just
concatenated.
Compile a term for the boolean operator `op` (AND or OR) applied to `terms`. This is accomplished by compiling each of the `terms` and then just joining their `where` terms with the operator. The params are just concatenated.
(compile-event-count-equality fields & [path value :as args])Compile an = predicate for event-count query. The path represents
the field to query against, and value is the value of the field.
Compile an = predicate for event-count query. The `path` represents the field to query against, and `value` is the value of the field.
(compile-event-count-inequality fields & [op path value :as args])Compile an inequality for an event-counts query (> < >= <=). The path
represents the field to query against, and the value is the value of the field.
Compile an inequality for an event-counts query (> < >= <=). The `path` represents the field to query against, and the `value` is the value of the field.
(compile-extract query-api-version ops field subquery)Compile an extract operator, selecting the given field from the compiled
result of subquery, which must be a kind of select operator.
Compile an `extract` operator, selecting the given `field` from the compiled result of `subquery`, which must be a kind of `select` operator.
(compile-fact-equality version)Compile an = predicate for a fact query. path represents the field to
query against, and value is the value.
Compile an = predicate for a fact query. `path` represents the field to query against, and `value` is the value.
(compile-fact-inequality op path value)Compile a numeric inequality for a fact query (> < >= <=). The value for
comparison must be either a number or the string representation of a number.
The value in the database will be cast to a float or an int for comparison,
or will be NULL if it is neither.
Compile a numeric inequality for a fact query (> < >= <=). The `value` for comparison must be either a number or the string representation of a number. The value in the database will be cast to a float or an int for comparison, or will be NULL if it is neither.
(compile-fact-regexp version)Compile an '~' predicate for a fact query, which does regexp matching. This
is done by leveraging the correct database-specific regexp syntax to return
only rows where the supplied path match the given pattern.
Compile an '~' predicate for a fact query, which does regexp matching. This is done by leveraging the correct database-specific regexp syntax to return only rows where the supplied `path` match the given `pattern`.
(compile-in kind query-api-version ops field subquery)Compile an in operator, selecting rows for which the value of
field appears in the result given by subquery, which must be an extract
composed with a select.
Compile an `in` operator, selecting rows for which the value of `field` appears in the result given by `subquery`, which must be an `extract` composed with a `select`.
(compile-not _version ops & terms)Compile a NOT operator, applied to term. This term simply negates the
value of term. Basically this function just serves as error checking for
negate-term*.
Compile a NOT operator, applied to `term`. This term simply negates the value of `term`. Basically this function just serves as error checking for `negate-term*`.
(compile-resource-equality version & [path value :as args])Compile an = operator for a resource query. path represents the field
to query against, and value is the value.
Compile an = operator for a resource query. `path` represents the field to query against, and `value` is the value.
(compile-resource-event-equality version)Compile an = predicate for resource event query. path represents the field to
query against, and value is the value.
Compile an = predicate for resource event query. `path` represents the field to query against, and `value` is the value.
(compile-resource-event-inequality & [op path value :as args])Compile a timestamp inequality for a resource event query (> < >= <=).
The value for comparison must be coercible to a timestamp via
puppetlabs.puppetdb.time/to-timestamp (e.g., an ISO-8601 compatible date-time string).
Compile a timestamp inequality for a resource event query (> < >= <=). The `value` for comparison must be coercible to a timestamp via `puppetlabs.puppetdb.time/to-timestamp` (e.g., an ISO-8601 compatible date-time string).
(compile-resource-event-regexp version)Compile an ~ predicate for resource event query. path represents the field
to query against, and pattern is the regular expression to match.
Compile an ~ predicate for resource event query. `path` represents the field to query against, and `pattern` is the regular expression to match.
(compile-resource-regexp version path value)Compile an '~' predicate for a resource query, which does regexp matching.
This is done by leveraging the correct database-specific regexp syntax to
return only rows where the supplied path match the given pattern.
Compile an '~' predicate for a resource query, which does regexp matching. This is done by leveraging the correct database-specific regexp syntax to return only rows where the supplied `path` match the given `pattern`.
(compile-term ops [op & args :as term])Compile a single query term, using ops as the set of legal operators. This
function basically just checks that the operator is known, and then
dispatches to the function implementing it.
Compile a single query term, using `ops` as the set of legal operators. This function basically just checks that the operator is known, and then dispatches to the function implementing it.
(event-count-ops fields op)Maps resource event count operators to the functions implementing them. Returns nil if the operator is unknown.
Maps resource event count operators to the functions implementing them. Returns nil if the operator is unknown.
(fact-operators version)Maps fact query operators to the functions implementing them. Returns nil if the operator isn't known.
Maps fact query operators to the functions implementing them. Returns nil if the operator isn't known.
(fact-query->sql ops query)Compile a fact query, returning a vector containing the SQL and parameters for the query. All fact columns are selected, and no order is applied.
Compile a fact query, returning a vector containing the SQL and parameters for the query. All fact columns are selected, and no order is applied.
(negate-term* ops term)Compiles term and returns the negated version of the query.
Compiles `term` and returns the negated version of the query.
(qualified-column field columns)given a field and one of the column maps above, produce the fully qualified column name
given a field and one of the column maps above, produce the fully qualified column name
This function takes a query type (:resource, :fact, :node) and a query API version number, and returns a set of strings which are the names the fields that are legal to query
This function takes a query type (:resource, :fact, :node) and a query API version number, and returns a set of strings which are the names the fields that are legal to query
Return the queryable set of fields and corresponding table names where they reside
Return the queryable set of fields and corresponding table names where they reside
(resource-event-ops version)Maps resource event query operators to the functions implementing them. Returns nil if the operator isn't known.
Maps resource event query operators to the functions implementing them. Returns nil if the operator isn't known.
(resource-operators version)Maps resource query operators to the functions implementing them. Returns nil if the operator isn't known.
Maps resource query operators to the functions implementing them. Returns nil if the operator isn't known.
(resource-query->sql ops query)Compile a resource query, returning a vector containing the SQL and parameters for the query. All resource columns are selected, and no order is applied.
Compile a resource query, returning a vector containing the SQL and parameters for the query. All resource columns are selected, and no order is applied.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |