pallet.api — dda/pallet 0.9.1

Pallet API

cluster-spec clj

(cluster-spec cluster-name
              &
              {:keys [extends groups phases node-spec roles] :as options})

Create a cluster-spec.

name is used as a prefix for all groups in the cluster.

• :groups specify a sequence of groups that define the cluster

• :extends specify a server-spec, a group-spec, or sequence thereof for all groups in the cluster

• :phases define phases on all groups.

• :node-spec default node-spec for the nodes in the cluster

• :roles roles for all group-specs in the cluster

compute-service clj

(compute-service service-or-provider-name & options)

Returns a compute service object, used to perform actions on a cloud provider.

converge clj

(converge group-spec->count
          &
          {:keys [compute blobstore user phase all-node-set environment async
                  timeout-ms timeout-val debug plan-state]
           :as options})

Converge the existing compute resources with the counts specified in group-spec->count. New nodes are started, or nodes are destroyed to obtain the specified node counts.

group-spec->count can be a map from group-spec to node count, or can be a sequence of group-specs containing a :count key.

This applies the :bootstrap phase to all new nodes and, by default, the :configure phase to all running nodes whose group-name matches a key in the node map. Phases can also be specified with the :phase option, and will be applied to all matching nodes. The :configure phase is the default phase applied.

Options

:compute : a compute service.

:blobstore : a blobstore service.

:phase : a phase keyword, phase function, or sequence of these.

:user : the admin-user on the nodes.

:consider-groups : a sequence of group-specs, which should have just the :settings phase run on them, so that their configuration is available for other nodes to query. The numbe of nodes in these groups will not be adjusted.

Partitioning

:partition-f : a function that takes a sequence of targets, and returns a sequence of sequences of targets. Used to partition or filter the targets. Defaults to any :partition metadata on the phase, or no partitioning otherwise.

Post phase options

:post-phase-f : specifies an optional function that is run after a phase is applied. It is passed targets, phase and results arguments, and is called before any error checking is done. The return value is ignored, so this is for side affect only.

:post-phase-fsm : specifies an optional fsm returning function that is run after a phase is applied. It is passed targets, phase and results arguments, and is called before any error checking is done. The return value is ignored, so this is for side affect only.

Asynchronous and Timeouts

:async : a flag to control whether the function executes asynchronously. When truthy, the function returns an Operation that can be deref'd like a future. When not truthy, :timeout-ms may be used to specify a timeout. Defaults to nil.

:timeout-ms : an integral number of milliseconds to wait for completion before timeout. Only applies if :async is not truthy (the default).

:timeout-val : a value to be returned should the operation time out.

Algorithm options

:phase-execution-f : specifies the function used to execute a phase on the targets. Defaults to pallet.core.primitives/build-and-execute-phase.

:execution-settings-f : specifies a function that will be called with a node argument, and which should return a map with :user, :executor and :executor-status-fn keys.

OS detection

:os-detect : controls detection of nodes' os (default true).

converge* clj

(converge* group-spec->count
           &
           {:keys [compute blobstore user phase all-node-set consider-groups
                   environment plan-state debug os-detect]
            :or {os-detect true}
            :as options})

Returns a FSM to converge the existing compute resources with the counts specified in group-spec->count. Options are as for converge.

environment-args clj

Arguments that are forwarded to be part of the environment

execute-and-flag-metadata clj

(execute-and-flag-metadata flag-kw)

Returns a metadata map that specifies a phase should be run only if the specified flag-kw is not set on a node. When the phase complete the flag will be set. The map is suitable for use as a value in a map passed to the :phases-meta clause of a server-spec or group-spec.

execute-on-unflagged-metadata clj

(execute-on-unflagged-metadata flag-kw)

Returns a metadata map that specifies a phase should be run only if the specified flag-kw is not set on a node. The map is suitable for use as a value in a map passed to the :phases-meta clause of a server-spec or group-spec.

execute-with-image-credentials-metadata clj

(execute-with-image-credentials-metadata)

Returns a metadata map that specifies a phase should be run using the image credentials (rather than the admin-user). The map is suitable for use as a value in a map passed to the :phases-meta clause of a server-spec or group-spec.

expand-cluster-groups clj

(expand-cluster-groups node-set)

Expand a node-set into its groups

expand-group-spec-with-counts clj

(expand-group-spec-with-counts node-set)

(expand-group-spec-with-counts node-set spec-count)

Expand a converge node spec into its groups

extend-specs clj

(extend-specs spec inherits)

(extend-specs spec inherits algorithms)

Merge in the inherited specs

group-node-maps clj

(group-node-maps compute
                 groups
                 &
                 {:keys [async timeout-ms timeout-val] :as options})

Returns a FSM to converge the existing compute resources with the counts specified in group-spec->count. Options are as for converge.

group-nodes clj

(group-nodes compute groups & {:keys [async timeout-ms timeout-val]})

Return a sequence of node-maps for each node in the specified group-specs.

Options:

:async : a flag to control whether the function executes asynchronously. When truthy, the function returns an Operation that can be deref'd like a future. When not truthy, :timeout-ms may be used to specify a timeout. Defaults to nil.

:timeout-ms : an integral number of milliseconds to wait for completion before timeout. Only applies if :async is not truthy (the default).

:timeout-val : a value to be returned should the operation time out.

group-spec clj

(group-spec name
            &
            {:keys [extends count image phases phases-meta default-phases
                    packager node-spec roles node-filter]
             :as options})

Create a group-spec.

name is used for the group name, which is set on each node and links a node to its node-spec

• :extends specify a server-spec, a group-spec, or sequence thereof and is used to inherit phases, etc.

• :phases used to define phases. Standard phases are:

• :phases-meta metadata to add to the phases

• :default-phases a sequence specifying the default phases

• :bootstrap run on first boot of a new node

• :configure defines the configuration of the node.

• :count specify the target number of nodes for this node-spec

• :packager override the choice of packager to use

• :node-spec default node-spec for this group-spec

• :node-filter a predicate that tests if a node is a member of this group.

lift clj

(lift node-set
      &
      {:keys [compute phase user all-node-set environment async timeout-ms
              timeout-val partition-f post-phase-f post-phase-fsm
              phase-execution-f execution-settings-f debug plan-state]
       :as options})

Lift the running nodes in the specified node-set by applying the specified phases. The compute service may be supplied as an option, otherwise the bound compute-service is used. The configure phase is applied by default unless other phases are specified.

node-set can be a group spec, a sequence of group specs, or a map of group specs to nodes. Examples:

[group-spec1 group-spec2 {group-spec #{node1 node2}}]
group-spec
{group-spec #{node1 node2}}

Options:

:compute : a compute service.

:blobstore : a blobstore service.

:phase : a phase keyword, phase function, or sequence of these.

:user : the admin-user on the nodes.

:consider-groups : a sequence of group-specs, which should have just the :settings phase run on them, so that their configuration is available for other nodes to query.

Partitioning

:partition-f : a function that takes a sequence of targets, and returns a sequence of sequences of targets. Used to partition or filter the targets. Defaults to any :partition metadata on the phase, or no partitioning otherwise.

Post phase options

:post-phase-f : specifies an optional function that is run after a phase is applied. It is passed targets, phase and results arguments, and is called before any error checking is done. The return value is ignored, so this is for side affect only.

:post-phase-fsm : specifies an optional fsm returning function that is run after a phase is applied. It is passed targets, phase and results arguments, and is called before any error checking is done. The return value is ignored, so this is for side affect only.

Asynchronous and Timeouts

:async : a flag to control whether the function executes asynchronously. When truthy, the function returns an Operation that can be deref'd like a future. When not truthy, :timeout-ms may be used to specify a timeout. Defaults to nil.

:timeout-ms : an integral number of milliseconds to wait for completion before timeout. Only applies if :async is not truthy (the default).

:timeout-val : a value to be returned should the operation time out.

Algorithm options

:phase-execution-f : specifies the function used to execute a phase on the targets. Defaults to pallet.core.primitives/build-and-execute-phase.

:execution-settings-f : specifies a function that will be called with a node argument, and which should return a map with :user, :executor and :executor-status-fn keys.

OS detection

:os-detect : controls detection of nodes' os (default true).

lift* clj

(lift* node-set
       &
       {:keys [compute phase all-node-set consider-groups environment debug
               plan-state os-detect]
        :or {os-detect true}
        :as options})

Returns a FSM to lift the running nodes in the specified node-set by applying the specified phases. Options as specified in lift.

lift-nodes clj

(lift-nodes targets
            phases
            &
            {:keys [user environment plan-state async timeout-ms timeout-val]
             :or {environment {} plan-state {}}
             :as options})

Lift targets, a sequence of node-maps, using the specified phases. This provides a way of lifting phases, which doesn't tie you to working with all nodes in a group. Consider using this only if the functionality in lift is insufficient.

phases : a sequence of phase keywords (identifying phases) or plan functions, that should be applied to the target nodes. Note that there are no default phases.

Options:

:user : the admin-user to use for operations on the target nodes.

:environment : an environment map, to be merged into the environment.

:plan-state : an state map, which can be used to passing settings across multiple lift-nodes invocations.

:async : a flag to control whether the function executes asynchronously. When truthy, the function returns an Operation that can be deref'd like a future. When not truthy, :timeout-ms may be used to specify a timeout. Defaults to nil.

:timeout-ms : an integral number of milliseconds to wait for completion before timeout. Only applies if :async is not truthy (the default).

:timeout-val : a value to be returned should the operation time out.

make-user clj

(make-user username
           &
           {:keys [public-key-path private-key-path passphrase password
                   sudo-password no-sudo sudo-user]
            :as options})

Creates a User record with the given username and options. Generally used in conjunction with admin-user and pallet.api/with-admin-user, or passed to lift or converge as the named :user argument.

Options:

• :public-key-path (defaults to ~/.ssh/id_rsa.pub)
• :private-key-path (defaults to ~/.ssh/id_rsa)
• :passphrase
• :password
• :sudo-password (defaults to :password)
• :no-sudo

node-spec clj

(node-spec & {:keys [image hardware location network qos] :as options})

Create a node-spec.

Defines the compute image and hardware selector template.

This is used to filter a cloud provider's image and hardware list to select an image and hardware for nodes created for this node-spec.

:image a map describing a predicate for matching an image: os-family os-name-matches os-version-matches os-description-matches os-64-bit image-version-matches image-name-matches image-description-matches image-id

:location a map describing a predicate for matching location: location-id :hardware a map describing a predicate for matching hardware: min-cores min-ram smallest fastest biggest architecture hardware-id :network a map for network connectivity options: inbound-ports :qos a map for quality of service options: spot-price enable-monitoring

plan-fn clj macro

(plan-fn & body)

Create a plan function from a sequence of plan function invocations.

eg. (plan-fn (file "/some-file") (file "/other-file"))

This generates a new plan function, and adds code to verify the state around each plan function call.

print-nodes clj

(print-nodes nodes)

Print the targets of an operation

print-targets clj

(print-targets op)

Print the targets of an operation

server-spec clj

(server-spec &
             {:keys [phases phases-meta default-phases packager node-spec
                     extends roles]
              :as options})

Create a server-spec.

• :phases a hash-map used to define phases. Phases are inherited by anything that :extends the server-spec. Standard phases are:
  • :bootstrap run on first boot of a new node
  • :configure defines the configuration of the node
• :default-phases a sequence specifying the default phases
• :phases-meta metadata to add to the phases
• :extends takes a server-spec, or sequence thereof, and is used to inherit phases, etc.
• :roles defines a sequence of roles for the server-spec. Inherited by anything that :extends the server-spec.
• :node-spec default node-spec for this server-spec
• :packager override the choice of packager to use

For a given phase, inherited phase functions are run first, in the order specified in the :extends argument.

split-groups-and-targets clj

(split-groups-and-targets node-set)

Split a node-set into groups and targets. Returns a map with :groups and :targets keys

version clj

(version)

Returns the pallet version.

with-admin-user clj macro

(with-admin-user user & exprs)

Specify the admin user for running remote commands. The user is specified either as pallet.utils.User record (see the pallet.utils/make-user convenience fn) or as an argument list that will be passed to make-user.

This is mainly for use at the repl, since the admin user can be specified functionally using the :user key in a lift or converge call, or in the environment.