Implementation of Hierarchical Temporal Memory.

Encoders mapping arbitrary values to SDRs.


This is the entry point of any HTM system, inputs in various formats have to be translated into SDRs.
It is a (fn encode [sdr input] sdr'). This namespace offers functions for creating encoders and modifying
them.

By default, encoder do not clear the SDR they work with so that they are more flexible. However, clearing
is often needed in HTM systems so the user can refer to the `always-clear`.


Based on :

  [1] Purdy, S. (2016). Encoding data for HTM systems. arXiv preprint arXiv:1602.05925.

Regular cartesian grids.


Glossary

  
  capacity
    How many `element`s fit on a dimension, a portion of a `grid`, or a whole `grid`.

  coordinate, coord
    Integer expression the unique location of an `element` in a dimension.

  coordinates, coords
    Vector of `coordinate`s expressing a unique location in a `grid`.

  dimension, dim
    Always described by its capacity.     

  element
    Member of a grid, always has unique `coordinates`.

  flat-index, f-index
    Any `coordinates` can be translated into a unique and corresponding 'f-index` between 0 (inclusive) and `capacity` (exclusive).
    For instance, in a 2D 32x64 grid, the element with coordinates [4 10] would have a `f-index` of (+ (* 4 64) 10).
    It is like unfolding an N dimensional space into a single dimension.

  grid
    Regular Cartesian grid partitioning a finite N dimensional space into a topology of `element`s with integer `coordinates`.
    Described by a vector of `dimension`s. A 2D 32x64 grid would be [32 64].

  hypercube
    Vector of 2-tuples [`coordinate` `capacity`], one for every dimension, expressing a hypercube in a `grid`. 
    A `coordinate` can be outside of the capacity of the corresponding `grid` dimension. Depending on the context, the hypercube would be truncated
    or wrapped (cf. `wrapping`).

  n-dims
    Number of dimensions.

  normal-coords
    `coordinates` normalized to fit between 0 and 1 (inclusive).

  random-number-generator, rng
    No-arg function returning a random value between 0 (inclusive) and 1 (exclusive).

  relative-coords
    `coordinates` in one `grid` always have relative `coordinates` in another `grid`.
    For instance, `coordinates` [1 1] in a [4 4] `grid` can be translated to `coordinates` [3 3] in a [10 10] `grid` which, proportionally, represents the same location.

  relative-f-index
    Just like `relative-coords`, but for a `flat-index`.

  wrapping
    Dimensions can behave in a cyclic manner. For instance, a `coordinate` of -2 on a dimension of `capacity` 6 would effectively be 4.

Miscellaneous math functions.

Create and act on Sparse Distributed Representations.


Cf. `helins.htm.sdr.props`

Properties of Sparse Distributed Representations (SDRs).


It is a good practise to learn this namespace as SDRs are omnipresent in HTM. It introduces definitions and basic concepts.
The `helins.htm.sdr` namespace is about creating and acting on concrete SDRs whereas this namespace is about the theory.


Based on :

  [1] Ahmad, S., & Hawkins, J. (2015). Properties of sparse distributed representations and their application to hierarchical
        temporal memory. arXiv preprint arXiv:1503.07469.

Spatial pooling.


Based on :

  Byrne, F. (2015). Encoding reality: Prediction-assisted cortical learning algorithm in hierarchical temporal memory. arXiv preprint arXiv:1509.08255.

  Cui, Y., Ahmad, S., & Hawkins, J. (2017). The HTM spatial pooler—a neocortical algorithm for online sparse distributed coding. Frontiers in computational
    neuroscience, 11, 111.

  Hawkins, J. et al. 2016. Biological and Machine Intelligence. Release 0.4. Accessed at http://numenta.com/biological-and-machine-intelligence/.

  Leake, M., Xia, L., Rocki, K., & Imaino, W. (2015, January). Effect of Spatial Pooler Initialization on Column Activity in Hierarchical Temporal Memory.
    In AAAI (pp. 4176-4177).

  Mnatzaganian, J., Fokoué, E., & Kudithipudi, D. (2017). A mathematical formalization of hierarchical temporal memory’s spatial pooler. Frontiers in Robotics
    and AI, 3, 81.
  
  Pietroń, M., Wielgosz, M., & Wiatr, K. (2016, October). Formal analysis of HTM spatial pooler performance under predefined operation conditions. In International
    Joint Conference on Rough Sets (pp. 396-405). Springer, Cham.



Glossary :

  active-inputs
   Sequence of `input`s active as a result of some stimulus.

  active-minicols
   Sequence of `minicol`s active as a result of `active-inputs`.

  center-input
    When relevant, a `minicol` might have a natural center in the input space.

  cnx, connection
    A connection is made when `perm` >= `cnx-threshold``.

  cnxs
    Vector where indices are `minicol`s and item are vectors of `input`s with established connection.

  cnx-threshold, connection-threshold
    Treshold for deciding if a `perm` is strong enough for establishing a `cnx`.

  flat-index
    Cf. `flat-index` in `helins.htm.grid` namespace.

  grid
    Cf. `grid` in `helins.htm.grid` glossary.

  grid-inputs
    `grid` describing the topology of the input space.

  grid-minicols
    `grid describing the topology of `minicols`.

  inhibition-radius
    Radius around a `minicol` within `grid-minicols` where inhibition happens during local inhibition.

  input
    Input bit represented by its `flat-index` within the `grid-inputs`.

  input-pool-mapping
    Vector where indices represent `input`s and items are 2-tuples where the first element is a `minicol` and the second one is the offset of the
    corresponding `input` in that `minicol`'s `pool`.

  minicol
    Modelisation of a neocortical mini-column represented by its `flat-index` within the `grid-minicols`.

  n-pool
    Size of a `pool`.

  overlap-score
    Number of currently `active-inputs` a minicol is currently connected to.

  overlap-scores
    Vector of `overlap-score`s where each index represent `minicol`;

  perm, permanence
    Value between 0 and 1 (inclusive) for deciding, in conjunction with a `cnx-threshold`, if a `cnx` is established.

  perms
    Vector of `perm`s layed out in the same order as their corresponding `pool`.

  perm-table
    Vector of `perms` where each index represents a `minicol`. `perms` are layed out in the same order as corresponding `pool`s.

  pool, potential-pool
    Set of `input`s a `minicol` can potentially connect to.

  pools, potential-pools
    Vector of `pool`s where each index represents a `minicol`.

  receptive-field
    Span of `input`s within `grid-inputs` a `minicol` is currently connected to.

  stimulus-threshold
    During inference, in order for a mini-column to even be considered potentially active, it must have an `overlap-score` of at least that much. During
    initialization, it is important to garantee that at least `stimulus-threshold` connections are randomly established for each mini-column otherwise
    they will never have the chance to compete. This parameter is a measure against noise and should be low. It could even be 0.