Next generation high performance Clojure toolkit.
dtype-next provides a unified pathway for dealing with contiguous containers of primitive datatypes such as
ints and floats. In addition it defines the basis for
array programming as found in APL or numpy and
a deep Java interface hierarchy with default methods to allow implementing new
arrays painless. This
interface hierarchy integrates with Java Streams, Spliterators, and various members of the java.util.function
package. In addition it extends these concepts to native-heap based containers.
There are namespaces to allow elementwise operations across scalars and arrays, highly optimized reductions
across index spaces, and algorithms that operate in index space for use when multiple buffers share
an index space.
This library forms the numeric basis that underlies the 'tech.ml.dataset'
system. It also defines a language independent ABI which allows zerocopy to C-based systems
such as numpy, OpenCV,
Additional targets of this library:
- Small runtime footprint. This is harder than it looks but the main thing is that
the system needs to produce the right answers with as little type specific code as
- Full native memory support. Malloc, free, memset, memcpy. Just the basics but
guaranteed to be available.
- Graal Native support.
- Support for JDK-8 through JDK-17+ - JDK-16 is no longer supported. For jdk-17 usage, please see
project.clj for required flags.
- Blogpost, example and involved example of using the FFI architecture across JNA, JDK-16 and GraalNative.
- In order to get mmap working on the native test I had to grab the larray .so from the
uberjar and load it manually. Not a big issue at the end of the day but I was having
problems getting graal native to package resources.
Use the scripts to get graal and compile test. The code is located under native-test; so far
reader/writing/copying all work for native and jvm heap datasets. Tensors work.
- Intel® Math Kernel Library - Use your system package manager to install
Copyright © 2020 Chris Nuernberger
This program and the accompanying materials are made available under the
terms of the Eclipse Public License 2.0 which is available at