puppetlabs.puppetdb.cli.generate — org.openvoxproject/puppetdb 8.12.0

Data Generation utility

This command-line tool can generate a base sampling of catalog, fact and report files suitable for consumption by the PuppetDB benchmark utility.

Note that it is only necessary to generate a small set of initial sample data since benchmark will permute per node differences. So even if you want to benchmark 1000 nodes, you don't need to generate initial catalog/fact/report json for 1000 nodes.

If you want a representative sample with big differences between catalogs, you will need to run the tool multiple times. For example, if you want a set of 5 large catalogs and 10 small ones, you will need to run the tool twice with the desired parameters to create the two different sets.

Flag Notes

Catalogs

Resource Counts

The num-resources flag is total and includes num-classes. So if you set --num-resources to 100 and --num-classes to 30, you will get a catalog with a hundred resources, thirty of which are classes.

Edges

A containment edge is always generated between the main stage and each class. And non-class resources get a containment edge to a random class. So there will always be a base set of containment edges equal to the resource count. The --additional-edge-percent governs how many non-containment edges are added on top of that to simulate some further catalog structure. There is no guarantee of relationship depth (as far as, for example Stage(main) -> Class(foo) -> Class(bar) -> Resource(biff)), but it does ensure some edges between classes, as well as between class and non-class resources.

Large Resource Parameter Blobs

The --blob-count and --blob-size parameters control inclusion of large text blobs in catalog resources. By default one ~100kb blob is added per catalog.

Set --blob-count to 0 to exclude blobs altogether.

Facts

Baseline Facts

Each fact set begins with a set of baseline facts from: baseline-agent-node.json.

These provide some consistency for a common set of baseline fact paths present on any puppet node. The generator then mutates half of the values to provide variety.

Fact Counts

The --num-facts parameter controls the number of facts to generate per host.

There are 376 leaf facts in the baseline file. Setting num-facts less than this will remove baseline facts to approach the requested number of facts. (Empty maps and arrays are not removed from the factset, so it will never pair down to zero.) Setting num-facts to a larger number will add facts of random depth based on --max-fact-depth until the requested count is reached.

Total Facts Size

The --total-fact-size parameter controls the total weight of the fact values map in kB. Weight is added after count is reached. So if the weight of the adjusted baseline facts already exceeds the total-fact-size, nothing more is done. No attempt is made to pair facts back down the requested size, as this would likely require removing facts.

Max Fact Depth

The --max-fact-depth parameter is the maximum nested depth a fact added to the baseline facts may reach. For example a max depth of 5, would mean that an added fact would at most be a nest of four maps:

{foo: {bar: {baz: {biff: boz}}}}

Since depth is picked randomly for each additional fact, this does not guarantee facts of a given depth. Nor does it directly affect the average depth of facts in the generated factset, although the larger the max-fact-depth and num-facts, the more likely that the average depth will drift higher.

Package Inventory

The --num-packages parameter sets the number of packages to generate for the factset's package_inventory array. Set to 0 to exclude.

Reports

Reports per Catalog

The --num-reports flag governs the number of reports to generate per generated catalog. Since one catalog is generated per host, this means you will end up with num-hosts * num-reports reports.

Variation in Reports

A report details change, or lack there of, during enforcement of the puppet catalog on the host. Since the benchmark tool currently chooses randomly from the given report files, a simple mechanism for determining the likelihood of receiving a report of a particular size (with lots of changes, few changes or no changes) is to produce multiple reports of each type per host to generate a weighted average. (If there are 10 reports, 2 are large and 8 are small, then it's 80% likely any given report submission submitted by benchmark will be of the small variety...)

The knobs to control this with the generate tool are:

• --num-reports, to determine the base number of reports to generate per catalog
• --high-change-reports-percent, percentage of that base to generate as reports with a high number of change events, as determined by:
• --high-change-resource-percent, percentage of resources in a high change report that will experience events (changes)
• --low-change-reports-percent, percentage of the base reports to generate as reports with a low number of change events as determined by:
• --low-change-resource-percent, percentage of resources in a low change report that will experience events (changes)

The left over percentage of reports will be no change reports (generally the most common) indicating the report run was steady-state with no changes.

By default, with a num-reports of 20, a high change percent of 5% and a low change percent of 20%, you will get 1 high change, 4 low change and 15 unchanged reports per host.

Unchanged Resources

In Puppet 8, by default, the agent no longer includes unchanged resources in the report, reducing its size.

The generate tool also does this by default, but you can set --no-exclude-unchanged-resources to instead include unchanged resources in every report (for default Puppet 7 behavior, for example).

Logs

In addition to a few boilerplate log lines, random logs are generated for each change event in the report. However other factors, such as pluginsync, puppet runs with debug lines and additional logging in modules can increase log output (quite dramatically in the case of debug output from the agent).

To simulate this, you can set --num-additional-logs to include in a report. And you can set --percent-add-report-logs to indicate what percentage of reports have this additional number of logs included.

Random Distribution

The default generation produces relatively uniform structures.

• for catalogs it generates equal resource and edge counts and similar byte counts.
• for factsets it generates equal fact counts and similar byte counts.

Example:

jpartlow@jpartlow-dev-2204:~/work/src/puppetdb$ lein run generate --verbose --output-dir generate-test ... :catalogs: 5

| :certname | :resource-count | :resource-weight | :min-resource | :mean-resource | :max-resource | :edge-count | :edge-weight | :catalog-weight | |---------------+-----------------+------------------+---------------+----------------+---------------+-------------+--------------+-----------------| | host-sarasu-0 | 101 | 137117 | 90 | 1357 | 110246 | 150 | 16831 | 154248 | | host-lukoxo-1 | 101 | 132639 | 98 | 1313 | 104921 | 150 | 16565 | 149504 | | host-dykivy-2 | 101 | 120898 | 109 | 1197 | 94013 | 150 | 16909 | 138107 | | host-talyla-3 | 101 | 110328 | 128 | 1092 | 82999 | 150 | 16833 | 127461 | | host-foropy-4 | 101 | 136271 | 106 | 1349 | 109811 | 150 | 16980 | 153551 |

:facts: 5

| :certname | :fact-count | :avg-depth | :max-depth | :fact-weight | :total-weight | |---------------+-------------+------------+------------+--------------+---------------| | host-sarasu-0 | 400 | 2.77 | 7 | 10000 | 10118 | | host-lukoxo-1 | 400 | 2.8 | 7 | 10000 | 10118 | | host-dykivy-2 | 400 | 2.7625 | 7 | 10000 | 10118 | | host-talyla-3 | 400 | 2.7825 | 7 | 10000 | 10118 | | host-foropy-4 | 400 | 2.7925 | 7 | 10000 | 10118 | ...

This mode is best used when generating several different sample sets with distinct weights and counts to provide (when combined) an overall sample set for benchmark that includes some fixed number of fairly well described catalog, fact and report examples.

By setting --random-distribution to true, you can instead generate a more random sample set, where the exact parameter values used per host will be picked from a normal curve based on the set value as mean.

• for catalogs, this will effect the class, resource, edge and total blob counts

Blobs will be distributed randomly through the set, so if you set --blob-count to 2 over --hosts 10, on averge there will be two per catalog, but some may have none, others four, etc...

• for facts, this will effect the fact and package counts, the total weight and the max fact depth.

This has no effect on generated reports at the moment.

Example:

jpartlow@jpartlow-dev-2204:~/work/src/puppetdb$ lein run generate --verbose --random-distribution :catalogs: 5

| :certname | :resource-count | :resource-weight | :min-resource | :mean-resource | :max-resource | :edge-count | :edge-weight | :catalog-weight | |---------------+-----------------+------------------+---------------+----------------+---------------+-------------+--------------+-----------------| | host-cevani-0 | 122 | 33831 | 93 | 277 | 441 | 193 | 22044 | 56175 | | host-firilo-1 | 91 | 115091 | 119 | 1264 | 91478 | 130 | 14466 | 129857 | | host-gujudi-2 | 129 | 36080 | 133 | 279 | 465 | 180 | 20230 | 56610 | | host-xegyxy-3 | 106 | 120603 | 136 | 1137 | 92278 | 153 | 17482 | 138385 | | host-jaqomi-4 | 107 | 211735 | 87 | 1978 | 98354 | 159 | 17792 | 229827 |

:facts: 5

| :certname | :fact-count | :avg-depth | :max-depth | :fact-weight | :total-weight | |---------------+-------------+------------+------------+--------------+---------------| | host-cevani-0 | 533 | 3.4690433 | 9 | 25339 | 25457 | | host-firilo-1 | 355 | 2.7464788 | 7 | 13951 | 14069 | | host-gujudi-2 | 380 | 2.75 | 8 | 16111 | 16229 | | host-xegyxy-3 | 360 | 2.7305555 | 7 | 5962 | 6080 | | host-jaqomi-4 | 269 | 2.7695167 | 7 | 16984 | 17102 | ...

-main clj

(-main & args)

add-blob clj

(add-blob {:keys [resources] :as catalog} blob-size-in-kb)

Add a large parameter string blob to one of the given catalog's resource parameters.

The blob will be of mean blob-size picked from a normal distribution with a standard deviation of one tenth the mean and an upper and lower bound of +/- 50% of the mean.

So given a blob-size of 100kb, a random resource will get an additional content parameter sized roughly between 90-110kb but with an absolute lower bound of 50kb and an upper bound of 150kb.

Returns the updated catalog.

add-logs-to-reports clj

(add-logs-to-reports reports num-additional-logs percent-add-report-logs)

Adds additional logs to some percentage of reports.

analyze-facts clj

(analyze-facts file)

Dev utility function to get some data about a JSON factset file.

build-parameters clj

(build-parameters size)

Build a random map of resource parameter keyword and string values to approximately size number of bytes.

build-to-size clj

(build-to-size size generate-word-fn)

Build up list of strings to approximately size bytes.

builtin-puppet-types clj

List of some built in puppet types for randomized resources.

cli clj

(cli args)

Runs the generate command as directed by the command line args and returns an appropriate exit status.

containment-path clj

(containment-path resource catalog)

Return a vector of the resource names containing the given catalog resource in the given catalog. Throws an error if resource is not in catalog.

create-event clj

(create-event)

(create-event size-class)

Generate a resource event of a particular category of size (of values/message). Typically values are small, but may be large in the case of properties like file content, where content is embedded in the catalog, for example.

create-new-facts clj

(create-new-facts number max-depth)

create-report-resource clj

(create-report-resource catalog catalog-resource changed)

(create-report-resource catalog catalog-resource changed start-time)

Create a report resource map for a report's resources array based on the given catalog resource. Add resource events if changed is true.

create-resource-events clj

(create-resource-events)

(create-resource-events {:keys [num-events start-time]
                         :or {num-events (min (dgen/geometric 0.95) 10)
                              start-time (now)}})

Creates a set of resource events. If the number of resource-events is not specified it is calculated from a geometric distribution with a high mean very likely to return 1 event, but no more than 10.

create-temp-dir clj

(create-temp-dir)

Generate a temp directory and return the Path object pointing to it.

delete-facts clj

(delete-facts fact-values number-to-delete leaf-fact-paths)

Trim facts down to num-facts. NOTE: this does not elliminate empty arrays/maps, so you will still have 50 or so empty facts, even if num-facts is 0.

environment clj

fatten-fact-values clj

(fatten-fact-values fact-values total-fact-size-in-bytes)

Randomly increase size of string facts until we reach total-fact-size-in-bytes.

generate clj

(generate {:keys [output-dir] :as options})

Build up a dataset of sample PuppetDB facts, catalogs and reports based on given options and store them in the given output directory.

generate-catalog clj

(generate-catalog certname
                  {:keys [num-classes num-resources resource-size title-size
                          additional-edge-percent random-distribution]})

generate-catalog-graph clj

(generate-catalog-graph main-stage classes resources additional-edge-percent)

Generate a set of containment edges associating each class with main-stage and randomly distributing remaining resources within the classes. Returns a Catalog instance.

Randomly generates additional required-by, notifies, before and subscription-of between a given percentage of both class and non-class resources.

Makes use of Catalog to ensure we're still building a directed acyclic graph, and to track the relation type between edges.

generate-classes clj

(generate-classes number title-size)

generate-data clj

(generate-data options output-path)

Generate and return a map of facts, catalogs and reports.

generate-edge clj

(generate-edge {:keys [source target relation]})

generate-fact-values clj

(generate-fact-values num-facts max-fact-depth total-fact-size-in-bytes options)

Generates a set of fact values.

• starts by loading the same set of baseline facts from resources
• mutates half the leaves
• then either
  • removes facts down to num-facts if less than the fact count from baseline
  • or uses create-facts to reach num-facts of max-depth
• then fattens up remaining fact strings to reach total-fact-size-in-kb

NOTE: fact removal won't reach 0 as empty arrays and maps in the fact paths are left behind, and won't change the depth of the baseline facts.

NOTE: if total-fact-size is less than the weight of adjusted baseline facts, nothing more is done. So count trumps size.

generate-factset clj

(generate-factset certname
                  {:keys [num-facts max-fact-depth total-fact-size num-packages
                          random-distribution]
                   :as options})

generate-files-from-wireformat-collection clj

(generate-files-from-wireformat-collection {:keys [dir namer col]})

Store each PuppetDB object as a JSON blob. Creates dir and generates filenames using the given namer fn.

generate-log clj

(generate-log {:keys [file line level message source tags]
               :or {level (rand-nth ["info" "notice"])
                    message (rnd/random-sentence-ish)
                    tags #{}
                    source "Puppet"}})

(generate-log level message)

Create one log entry for a report's log array.

generate-package-inventory clj

(generate-package-inventory num-packages)

Build a list of package [name, provider, version] vectors per the wire format.

generate-report clj

(generate-report catalog percent-resource-change exclude-unchanged-resources)

Generate a report based on the given catalog.

Ensure that percent-resource-change resources have events.

Exclude or keep unchanged resources based on exclude-unchanged-resources.

generate-report-logs clj

(generate-report-logs catalog changed-resources)

Generate log of changes for a report based on a set of changed resources and some boilerplate.

generate-report-metrics clj

(generate-report-metrics catalog-resources event-count)

This builds a fake set of metrics that's representative as far as general size. Metrics are just stored as part of the report blob and aren't decomposed or otherwise functional within puppetdb itself, so not trying to make them particularly accurate as far as representing a real report run.

generate-report-resources clj

(generate-report-resources catalog
                           changed-resources
                           exclude-unchanged-resources)

(generate-report-resources catalog
                           changed-resources
                           exclude-unchanged-resources
                           start-time)

Generate the list of resources for the given catalog. Generate resource-events for any resouce that is a member of changed-resources. Include or exclude unchanged resources from the list based on the exclude-unchanged-resources flag.

generate-reports clj

(generate-reports catalog
                  {:keys
                     [num-reports high-change-reports-percent
                      high-change-resources-percent low-change-reports-percent
                      low-change-resources-percent exclude-unchanged-resources
                      num-additional-logs percent-add-report-logs]})

Generate a set of reports for the given catalog based on options.

generate-resources clj

(generate-resources number avg-resource-size title-size)

generate-wrapper clj

(generate-wrapper args)

Generates a set of fact, catalog and report json files based on the given args.

leaf-fact-paths clj

(leaf-fact-paths fact-paths)

Filter for fact-paths that are not collections.

load-baseline-factset clj

(load-baseline-factset)

Loads a default set of baseline facts from the classpath resources.

mutate-fact-values clj

(mutate-fact-values fact-values num-to-change leaf-fact-paths)

Mutates num-to-change leaf facts (string, number, boolean) in the given set of fact-values, using the given leaf-fact-paths to find them.

NamedEdges clj protocol

add clj

(add catalog r1 r2 relation)

Create an edge of relation between resource 1 and 2 in catalog.

new-catalog-graph clj

(new-catalog-graph)

Return a new initialized empty Catalog.

parameter-name clj

(parameter-name)

(parameter-name size)

Generate a fictious but somewhat intelligible snake case parameter name (as used in puppet modules).

print-summary-table clj

(print-summary-table stat-map)

producer-host clj

pseudonym clj

(pseudonym prefix ordinal)

(pseudonym prefix ordinal length)

Generate a fictitious but somewhat intelligible keyword name.

random-type clj

(random-type)

Return a built-in type 80% of the time, or a random defined type name.

resource-fact-path clj

resource-name clj

(resource-name resource)

silent? clj

(silent? {:keys [silent]})

User has requested output silence.

sprinkle-blobs clj

(sprinkle-blobs catalogs {:keys [blob-count blob-size random-distribution]})

Add large text blobs to catalogs.

If random-distribution is false, will sprinkle blob-count blobs of an avg blob-size in kb into each catalog.

If random-distribution is true, will sprinkle a total number of blobs, picked from a random distribution based on a mean of blob-count * catalog count, randomly over the catalogs. The result is likely, but not guaranteed, to be a much more uneven catalog set.

summarize clj

(summarize data)

Print out a verbose summary of the generated data.

system-seconds-str clj

(system-seconds-str)

Epoch seconds as a string. Used by default as a version string in Puppet catalogs and reports.

vary-param clj

(vary-param param-value
            random-distribution
            stnd-deviation-percent
            &
            safe-standard-normal-options)

Return the given param, or if random-distribution is true, pick from a normal distribution with the given param value as mean and a standard deviation of param-value * stnd-deviation-percent.

verbose? clj

(verbose? {:keys [silent verbose]})

User has requested verbose output. But make sure silent isn't set...

weigh clj multimethod

Loosely weigh up the bytes of a structure.