tech.compute.tensor — techascent/tech.compute 3.0

Tensor library used to implement the basic math abstraction fairly easily implementable across a wide range of compute devices. This abstraction is meant to provide a language in which to implement some amount of functionalty especially useful in quickly testing out algorithmic updates or moving data to/from external libraries. As such, it has extensive support for reshape/select/transpose type operations but only nominal base math facilities are provided by default.

There is an implicit assumption throughout this file that implementations will loop through smaller entities instead of throwing an exception if sizes don't match. This is referred to as broadcasting in numpy (https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html).

It does mean, however, that certain conditions that would actually be error cases are harder to detect because one has to check for remainders being zero (which potentially could cause a divide by zero error) instead of just checking for equality.

For binary operations there are four forms:

y = ax op by result = ax op by. y[idx] = ax[idx] op by[idx] result[idx] = ax[idx] op by[idx]

Op may be: [:+ :* :/].

In the non-indexed cases the element counts of y or x may differ but they need to be commensurate meaning that the smaller evenly divides the larger. When writing to result it is important that result is as large as the largest. This is a relaxation of the numpy broadcasting rules to allow more forms of broadcasting; the check is that the remainder is zero; not that the smaller dimension is 1.

In general we want as much error checking and analysis done in this file as opposed to at the implementation level (compute stream level) so that different implementations of this duplicate the least number of possible operations and so their edge cases agree to the extent possible.

->tensor clj

(->tensor data
          &
          {:keys [datatype unchecked? shape stream force-copy?] :as options})

Create a tensor from the data by copying the data at least once.

acceptable-tensor-buffer? clj

(acceptable-tensor-buffer? item)

as-2d-matrix clj

(as-2d-matrix tensor)

As a 2d matrix of shape [everything-else most-rapidly-changin-dimension]

as-batch-matrix clj

(as-batch-matrix tensor)

As a 2d matrix of shape [least-rapidly-changing-dimension everything-else]

as-column-vector clj

(as-column-vector tensor)

as-dense clj

(as-dense tensor)

As dense implies that a memcpy call would succeed as one expects. This means actually 2 conditions are checked:

1. dense?
2. dimensions-monotonic-increasing

as-row-vector clj

(as-row-vector tensor)

as-tensor clj

(as-tensor item)

In-place make this a tensor.

as-vector clj

(as-vector tensor)

assign! clj

(assign! dest src)

binary-op! clj

(binary-op! dest alpha x beta y op & [options])

Perform the operation: dest = alpha * x op beta * y. x or y may be a scalar, dest must not be. Datatypes must match.

binary-operations clj

clone clj

(clone src & {:keys [datatype] :as options})

Clone this tensor, keeping as many details as possible identical. This method does type elision, so in most cases it will return something of the same type as passed in.

clone-to-device clj

(clone-to-device src & {:keys [datatype] :as options})

Clone this tensor, creating a new one on the currently bound device. Does not to type elision.

columns clj

(columns tensor)

Returns a vector of matrixes with width of 1 but large column strides.

constrain-inside-hypersphere! clj

(constrain-inside-hypersphere! dest mag-vec radius-length)

Like normalize, but only shorten vectors that are too long. So instead of projecting to the surface of the hypersphere like normalize does, do a <= operation.

construct-tensor clj

(construct-tensor dimensions buffer)

copy-to-java-type clj

(copy-to-java-type dest src & [options])

The options map in this case also contains potentially {:unchecked?} as the dtype/copy method is used.

dense? clj

(dense? item)

ecount clj

(ecount tensor)

enable-cpu-tensors! clj

(enable-cpu-tensors!)

Enables a version of the tensors that run on the cpu and that use netlib blas for operations.

ensure-tensor clj

(ensure-tensor item)

flat-distribution clj

(flat-distribution & {:keys [minimum maximum] :or {minimum 0 maximum 1}})

Create a flat distribution description. Flat (equal) distribution including minimum but excluding maximum [minimum maximum)

from-prototype clj

(from-prototype src & {:keys [datatype shape]})

New tensor just like this one (same device/driver,etc)

gaussian-distribution clj

(gaussian-distribution & {:keys [mean variance] :or {mean 0 variance 1}})

Create a Gaussian distribution description

gemm! clj

(gemm! C trans-a? trans-b? alpha A B beta & [options])

C = alpha * (trans-a? A) * (trans-b? B) + beta * C.

get-datatype clj

(get-datatype tensor)

in-place-reshape clj

(in-place-reshape tensor shape)

make-dense clj

(make-dense tensor)

new-tensor clj

(new-tensor shape
            &
            {:keys [datatype init-value stream] :or {init-value 0} :as options})

normalize! clj

(normalize! dest mag-vec radius-length epsilon & {:as options})

Ensure each vector of the last dimension of dest has length radius-length. Epsilon is used to avoid divide-by-zero conditions. This operation can also be seen as a projection to the surface of a hypersphere of radius radius-length.

rand! clj

(rand! dest distribution & {:as options})

Generate a pool of random numbers. Due to cuda limitations, this function is limited to floating point numbers.

reinterpret-tensor clj

(reinterpret-tensor old-tensor new-dimensions)

Create a new tensor with new dimensions. This is like an in place reinterpretation of the data.

rows clj

(rows tensor)

Returns a vector rows of dense vectors.

scalar? clj

(scalar? item)

select clj

(select tensor & args)

Limited implementation of the core.matrix select function call. Same rules apply Except if you pass in an array of numbers for a dimension then they must be contiguous and monotonically increasing (a proper inclusive range). This is due to limitations of the current gpu implementation and a strong reluctance to add complexity there. There must be an entry for every dimension of the tensor. see: https://cloojure.github.io/doc/core.matrix/clojure.core.matrix.html#var-select

shape clj

(shape tensor)

simple-tensor? clj

(simple-tensor? tensor)

strided? clj

strides clj

(strides tensor)

submatrix clj

(submatrix tensor row-start row-length col-start col-length)

Create a sub matrix of tensor. Tensor will be interpreted as width being n-cols and the rest of the dimensions being squashed into n-rows.

subvector clj

(subvector tensor offset & {:keys [length]})

tensor->2d-shape clj

(tensor->2d-shape tensor)

tensor->batch-shape clj

(tensor->batch-shape tensor)

tensor->batch-size clj

(tensor->batch-size tensor)

tensor->buffer clj

(tensor->buffer tensor)

tensor->dimensions clj

(tensor->dimensions item)

tensor->string clj

(tensor->string tens & {:keys [print-datatype] :or {print-datatype :float64}})

tensor? clj

(tensor? item)

ternary-op! clj

(ternary-op! dest alpha x beta y gamma z op & [options])

Perform the elementwise operation dest = op( alpha * x, beta * y, gamma * z ) dest tensor and must not alias any other arguments. There is no accumulator version of these operations at this time in order to keep kernel permutations low (3 backend permutations).

x, y, z can be constants or tensors.

operations: select: dest = (if (>= x 0) y z)

ternary-operations clj

to-array-of-type clj

(to-array-of-type tensor datatype)

to-core-matrix clj

(to-core-matrix tensor)

to-core-matrix-vector clj

(to-core-matrix-vector tensor)

to-double-array clj

(to-double-array tensor)

to-float-array clj

(to-float-array tensor)

to-jvm clj

(to-jvm item
        &
        {:keys [datatype base-storage]
         :or {datatype :float64 base-storage :persistent-vector}})

Conversion to storage that is efficient for the jvm. Base storage is either jvm-array or persistent-vector.

to-vector clj

(to-vector tensor)

transpose clj

(transpose tensor reorder-vec)

Transpose the tensor returning a new tensor that shares the backing store but indexes into it in a different order. Dimension 0 is the leftmost (greatest) dimension:

(transpose tens (range (count (shape tens))))

is the identity operation.

typed-assign! clj

unary-op! clj

(unary-op! dest alpha x op & [options])

dest[idx] = op(alpha * x)

unary-operations clj

unary-reduce! clj

(unary-reduce! output alpha input op & [options])

Vector operations operate across the last dimension and produce 1 result. output = op((alpha*input)) Output must be a [xyz 1] tensor while input is an [xyz n] tensor; the reduction will occur across the n axis with the results placed in output. The leading dimensions of both vectors must match.

unary-reduction-operations clj