figurer.core — figurer 0.1.0

define-problem clj

(define-problem {:keys [policy value predict initial-state depth] :as options})

Defines the context for all simulation and optimization. Includes null solution because problem and solution are the same data type in figurer, differing only in degree of refinement. Internally, the problem or solution instance is referred to as a context.

policy: function from state to actuation distribution value: function from state to number predict: function from state and actuation to state initial-state: state depth: integer indicating how many timesteps to consider state: vector of doubles actuation: vector of doubles actuation distribution: vector of incanter distributions (sample from each of these distributions to get one actuation)

expected-value clj

(expected-value context)

(expected-value context state)

Estimates the average value that would be found by sampling many plans from this initial state.

TODO: When state provided, need to search for nearby nodes and interpolate.

figure clj

(figure context {:keys [max-iterations max-seconds] :as options})

Perform optimization, returning a more optimized context.

figure-create-node clj

(figure-create-node context parent-node-id)

Private: Create new node following node-id. Return modified context and created node ID.

figure-next-node clj

(figure-next-node context node-id)

Private: Choose next node (or return nil to create new node).

TODO: Choose promising node (or nil if winner so far is clear) rather than just choosing randomly.

figure-once clj

(figure-once context)

Private: Perform one rollout to collect statistics.

figure-rollout clj

(figure-rollout context node-id remaining-depth)

Private: Reached end of nodes - follow random policy to determine end result. Return average value of remaining states.

sample-plan clj

(sample-plan context)

Returns a list of states and actuations starting from a given state (defaulting to initial state). Result states is longer than actuations because initial state is included.

sample-policy clj

(sample-policy context)

(sample-policy context state)

Samples from optimized policy for a given state (defaulting to initial state). For maximizing policies, the result should be the action that is expected to maximize the value. For random policies, the initial random distribution will be sampled. TODO: Use simulation results to refine rather than just sampling from the initial policy.