Find, cache and bind a prebuilt native library — the boring half of every Clojure FFM wrapper, factored out.
A library like com.blockether/imaging ships a Rust cdylib. The cdylib itself
is published as per-platform artifacts (imaging-native-darwin-arm64, …), each
carrying prebuilds/<platform>/<file> on the classpath. At runtime the wrapper
must decide WHICH file to dlopen, and it must work in three very different
situations: a dev pointing at a freshly cargo builded library, a fat uberjar
or GraalVM native image with the library bundled, and a plain deps.edn user
who only declared the pure-Clojure jar.
That resolution order is identical in every such library, so it lives here:
<LIB>_NATIVE_PATH env or <group>.<lib>.native.path
system property. Used verbatim, no existence check, no download.prebuilds/<platform>/<file>. This is the
uberjar / native-image path; a resource inside a jar is copied to a
temp file (a dlopen needs a real path), a directory resource is used
in place.<lib>-native-<platform> artifact resolved
through clojure.tools.deps (so Maven repos, mirrors and
~/.m2/settings.xml are honoured), extracted into
~/.cache/clj-<lib>/<version>/<platform>/ and cached there. Disable with
<LIB>_DISABLE_DOWNLOAD=1.tools.deps is reached through requiring-resolve, so it is only touched on
step 3 and never becomes a hard dependency (it does not fit in a native image).
This library itself has ZERO dependencies.
Everything is driven by a spec map built by spec; every derived name has a
default and can be overridden:
(def ^:private lib (nr/spec {:lib "imaging"})) ;; => env IMAGING_NATIVE_PATH / IMAGING_CACHE_DIR / IMAGING_DISABLE_DOWNLOAD ;; prop com.blockether.imaging.native.path / .cache-dir / .disable-download ;; file libimaging_c.dylib | libimaging_c.so | imaging_c.dll ;; artifact com.blockether/imaging-native-<platform> ;; version from the classpath resource imaging/VERSION ;; cache ~/.cache/clj-imaging
And the binding half:
(defonce ^:private fns (delay (nr/bind! lib {:version [:ptr "imaging_version"] :decode [:ptr "imaging_decode" :ptr :i64] :free [nil "imaging_free_string" :ptr]})))
(nr/invoke @fns :decode seg (count bs))
Run the JVM with --enable-native-access=ALL-UNNAMED so the foreign linker may
load the library without a restricted-method warning. Requires JDK 22+ (the
final FFM API); JDK 25 is what we test on.
Find, cache and bind a prebuilt native library — the boring half of every
Clojure FFM wrapper, factored out.
A library like `com.blockether/imaging` ships a Rust cdylib. The cdylib itself
is published as per-platform artifacts (`imaging-native-darwin-arm64`, …), each
carrying `prebuilds/<platform>/<file>` on the classpath. At runtime the wrapper
must decide WHICH file to `dlopen`, and it must work in three very different
situations: a dev pointing at a freshly `cargo build`ed library, a fat uberjar
or GraalVM native image with the library bundled, and a plain `deps.edn` user
who only declared the pure-Clojure jar.
That resolution order is identical in every such library, so it lives here:
1. An explicit path — `<LIB>_NATIVE_PATH` env or `<group>.<lib>.native.path`
system property. Used verbatim, no existence check, no download.
2. A bundled classpath resource `prebuilds/<platform>/<file>`. This is the
uberjar / native-image path; a resource inside a jar is copied to a
temp file (a `dlopen` needs a real path), a directory resource is used
in place.
3. A runtime download: the `<lib>-native-<platform>` artifact resolved
through `clojure.tools.deps` (so Maven repos, mirrors and
`~/.m2/settings.xml` are honoured), extracted into
`~/.cache/clj-<lib>/<version>/<platform>/` and cached there. Disable with
`<LIB>_DISABLE_DOWNLOAD=1`.
tools.deps is reached through `requiring-resolve`, so it is only touched on
step 3 and never becomes a hard dependency (it does not fit in a native image).
This library itself has ZERO dependencies.
Everything is driven by a spec map built by `spec`; every derived name has a
default and can be overridden:
(def ^:private lib (nr/spec {:lib "imaging"}))
;; => env IMAGING_NATIVE_PATH / IMAGING_CACHE_DIR / IMAGING_DISABLE_DOWNLOAD
;; prop com.blockether.imaging.native.path / .cache-dir / .disable-download
;; file libimaging_c.dylib | libimaging_c.so | imaging_c.dll
;; artifact com.blockether/imaging-native-<platform>
;; version from the classpath resource imaging/VERSION
;; cache ~/.cache/clj-imaging
And the binding half:
(defonce ^:private fns
(delay (nr/bind! lib {:version [:ptr "imaging_version"]
:decode [:ptr "imaging_decode" :ptr :i64]
:free [nil "imaging_free_string" :ptr]})))
(nr/invoke @fns :decode seg (count bs))
Run the JVM with `--enable-native-access=ALL-UNNAMED` so the foreign linker may
load the library without a restricted-method warning. Requires JDK 22+ (the
final FFM API); JDK 25 is what we test on.The small memory chores every cdylib binding repeats: NULL checks, reading a
char* back into a String, pushing bytes/strings down as MemorySegments,
and copying an owned buffer out.
Nothing here knows about a particular library — it is the boring half of an
FFM binding, factored out so clj-ruff, clj-fff and clj-imaging stop
carrying three copies of it.
All allocation takes an explicit Arena: the caller decides the lifetime
(Arena/ofConfined in a with-open for a call's scratch, Arena/ofAuto for
process-lifetime handles).
The small memory chores every cdylib binding repeats: NULL checks, reading a `char*` back into a String, pushing bytes/strings down as `MemorySegment`s, and copying an owned buffer out. Nothing here knows about a particular library — it is the boring half of an FFM binding, factored out so `clj-ruff`, `clj-fff` and `clj-imaging` stop carrying three copies of it. All allocation takes an explicit `Arena`: the caller decides the lifetime (`Arena/ofConfined` in a `with-open` for a call's scratch, `Arena/ofAuto` for process-lifetime handles).
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |