Puppet catalog parsing
Functions that handle conversion of catalogs from wire format to internal PuppetDB format.
The wire format is described in detail in the spec.
There are a number of transformations we apply to wire format catalogs during conversion to our internal format; while wire format catalogs contain complete records of all resources and edges, and most things are properly encoded as lists or maps, there are still a number of places where structure is absent or lacking:
Resource specifiers are represented as opaque strings, like
Class[Foobar], as opposed to something like
{"type" "Class" "title" "Foobar"}
Tags are represented as lists (and may contain duplicates) instead of sets
Resources are represented as a list instead of a map, making operations that need to correlate against specific resources unneccesarily difficult
Keys to all maps are strings (to conform with JSON), instead of more convenient Clojure keywords
Unless otherwise indicated, all terminology for catalog components matches terms listed in the spec.
A map of the form {:type "Class" :title "Foobar"}. This is a
unique identifier for a resource within a catalog.
A map that represents a single resource in a catalog:
{:type "..."
:title "..."
:... "..."
:tags #{"tag1", "tag2", ...}
:parameters {:name1 "value1"
:name2 "value2"
...}}
Certain attributes are treated special:
:type and :title are used to produce a resource-spec for
this resourceA representation of an "edge" (dependency or containment) in the catalog. All edges have the following form:
{:source <resource spec>
:target <resource spec>
:relationship <relationship id>}
A relationship identifier can be one of:
:contains:required-by:notifies:before:subscription-ofA wire-format-neutral representation of a Puppet catalog. It is a map with the following structure:
{:certname "..."
:version "..."
:resources {<resource-spec> <resource>
<resource-spec> <resource>
...}
:edges #(<dependency-spec>,
<dependency-spec>,
...)}
Puppet catalog parsing
Functions that handle conversion of catalogs from wire format to
internal PuppetDB format.
The wire format is described in detail in [the spec](../spec/catalog-wire-format.md).
There are a number of transformations we apply to wire format
catalogs during conversion to our internal format; while wire
format catalogs contain complete records of all resources and
edges, and most things are properly encoded as lists or maps, there
are still a number of places where structure is absent or lacking:
1. Resource specifiers are represented as opaque strings, like
`Class[Foobar]`, as opposed to something like
`{"type" "Class" "title" "Foobar"}`
2. Tags are represented as lists (and may contain duplicates)
instead of sets
3. Resources are represented as a list instead of a map, making
operations that need to correlate against specific resources
unneccesarily difficult
4. Keys to all maps are strings (to conform with JSON), instead of
more convenient Clojure keywords
### Terminology
Unless otherwise indicated, all terminology for catalog components
matches terms listed in [the spec](../spec/catalog-wire-format.md).
### Transformed constructs
### Resource Specifier (resource-spec)
A map of the form `{:type "Class" :title "Foobar"}`. This is a
unique identifier for a resource within a catalog.
### Resource
A map that represents a single resource in a catalog:
{:type "..."
:title "..."
:... "..."
:tags #{"tag1", "tag2", ...}
:parameters {:name1 "value1"
:name2 "value2"
...}}
Certain attributes are treated special:
* `:type` and `:title` are used to produce a `resource-spec` for
this resource
### Edge
A representation of an "edge" (dependency or containment) in the
catalog. All edges have the following form:
{:source <resource spec>
:target <resource spec>
:relationship <relationship id>}
A relationship identifier can be one of:
* `:contains`
* `:required-by`
* `:notifies`
* `:before`
* `:subscription-of`
### Catalog
A wire-format-neutral representation of a Puppet catalog. It is a
map with the following structure:
{:certname "..."
:version "..."
:resources {<resource-spec> <resource>
<resource-spec> <resource>
...}
:edges #(<dependency-spec>,
<dependency-spec>,
...)}(catalog-query->wire-v9 catalog)Inputs: [catalog :- catalog-query-schema] Returns: catalog-wireformat-schema
Inputs: [catalog :- catalog-query-schema] Returns: catalog-wireformat-schema
Full catalog query result schema.
Full catalog query result schema.
Constant representing the version number of the PuppetDB catalog format
Constant representing the version number of the PuppetDB catalog format
This flattened catalog schema is for the superset of catalog information. Use this when in the general case as it can be converted to any of the other (v5-) schemas
This flattened catalog schema is for the superset of catalog information. Use this when in the general case as it can be converted to any of the other (v5-) schemas
(default-missing-keys catalog)Given a catalog (any canonical version) and add any missing keys for it to be the full version
Given a catalog (any canonical version) and add any missing keys for it to be the full version
(edge-query->wire-v9 edge)Inputs: [edge :- edge-query-schema]
Inputs: [edge :- edge-query-schema]
Schema for validating a single edge.
Schema for validating a single edge.
(edges-expanded->wire-v9 edges)Inputs: [edges :- edges-expanded-query-schema]
Inputs: [edges :- edges-expanded-query-schema]
Edges expanded format schema.
Edges expanded format schema.
Parse a wire-format catalog object or string of the specified version,
returning a PuppetDB-suitable representation.
Parse a wire-format `catalog` object or string of the specified `version`, returning a PuppetDB-suitable representation.
(resource-query->wire-v9 resource)Inputs: [resource :- resource-query-schema]
Inputs: [resource :- resource-query-schema]
Schema for validating a single resource.
Schema for validating a single resource.
(resource-spec-to-map str)Convert a textual resource specifier like "Class[foo]" into a map
of the form {:type "Class" :title "foo"}
Convert a textual resource specifier like `"Class[foo]"` into a map
of the form `{:type "Class" :title "foo"}`(resources-expanded->wire-v9 resources)Inputs: [resources :- resources-expanded-query-schema]
Inputs: [resources :- resources-expanded-query-schema]
Resources expanded format schema.
Resources expanded format schema.
Applies every transformation to the catalog, converting it from wire format to our internal structure.
Applies every transformation to the catalog, converting it from wire format to our internal structure.
(transform-edge* edge)Converts the relationship value of edge into a
keyword.
Converts the `relationship` value of `edge` into a keyword.
(transform-edges {:keys [edges] :as catalog})Transforms every edge of the given catalog and converts the edges into a set.
Transforms every edge of the given `catalog` and converts the edges into a set.
(transform-resource* resource expected-keys already-seen-unrecognized-keys)Normalizes the structure of a single resource.
Normalizes the structure of a single `resource`.
(transform-resources {:keys [resources] :as catalog})Turns the list of resources into a mapping of
{resource-spec resource, ...}, as well as transforming each resource.
Turns the list of resources into a mapping of
`{resource-spec resource, ...}`, as well as transforming each resource.(transform-tags {:keys [tags] :as o})Turns a resource's list of tags into a set of strings.
Turns a resource's list of tags into a set of strings.
(validate catalog)Function for validating v9- of the catalogs
Function for validating v9- of the catalogs
(validate-edges {:keys [edges resources] :as catalog})Ensure that all edges have valid sources and targets, and that the relationship types are acceptable.
Ensure that all edges have valid sources and targets, and that the relationship types are acceptable.
(validate-resources {:keys [resources] :as catalog})Ensure that all resource tags conform to the allowed tag pattern.
Ensure that all resource tags conform to the allowed tag pattern.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |