A foreign-function toolkit for binding a prebuilt native library alongside compiled Zig, through the finalized Foreign Function & Memory API (Java 22+).
clj-zig.core/clj-zig.ffm cover the everyday case: a defnz body is
Zig that clj-zig compiles, and the boundary is described by a signature
vector. But a real program also reaches libraries it did NOT compile:
the platform's windowing or input library, a system framework, libc,
the graphics loader. Those expose a flat C ABI with no Zig source and no
signature spec to derive carriers from. Binding them is the same FFM
work every time -- open the library, describe a signature, bind a
downcall, occasionally hand native code a callback -- so this namespace
publishes that work once as a small, data-in/data-out toolkit rather
than leaving each consumer to re-derive it.
This is imperative-shell / native-edge code (ADR 16). It loads
libraries, holds linker handles, and crosses the FFM boundary; it
carries no domain knowledge and the pure core never sees a
MemorySegment. Native pointers returned across the boundary are opaque
handles (ADR 22): they are threaded back into native calls, never
dereferenced into Clojure logic.
PERFORMANCE. A real-time consumer (a 60fps present loop, an audio
callback) calls some of these every frame. downcall therefore binds
the symbol, builds the descriptor, and links the handle AT MOST ONCE per
distinct call and caches the MethodHandle; the per-frame path invokes
the cached handle directly with typed arguments and allocates nothing.
call is the convenience invoker for the cold path (setup, teardown,
once-per-batch reads), where the per-call argument array is fine.
NATIVE ACCESS. Loading a library and calling native code are restricted
operations; a JVM that denies native access throws
IllegalCallerException. Run with --enable-native-access=ALL-UNNAMED
(the :repl and :test aliases in deps.edn set it).
A foreign-function toolkit for binding a prebuilt native library alongside compiled Zig, through the finalized Foreign Function & Memory API (Java 22+). `clj-zig.core`/`clj-zig.ffm` cover the everyday case: a `defnz` body is Zig that clj-zig compiles, and the boundary is described by a signature vector. But a real program also reaches libraries it did NOT compile: the platform's windowing or input library, a system framework, libc, the graphics loader. Those expose a flat C ABI with no Zig source and no signature spec to derive carriers from. Binding them is the same FFM work every time -- open the library, describe a signature, bind a downcall, occasionally hand native code a callback -- so this namespace publishes that work once as a small, data-in/data-out toolkit rather than leaving each consumer to re-derive it. This is imperative-shell / native-edge code (ADR 16). It loads libraries, holds linker handles, and crosses the FFM boundary; it carries no domain knowledge and the pure core never sees a `MemorySegment`. Native pointers returned across the boundary are opaque handles (ADR 22): they are threaded back into native calls, never dereferenced into Clojure logic. PERFORMANCE. A real-time consumer (a 60fps present loop, an audio callback) calls some of these every frame. `downcall` therefore binds the symbol, builds the descriptor, and links the handle AT MOST ONCE per distinct call and caches the `MethodHandle`; the per-frame path invokes the cached handle directly with typed arguments and allocates nothing. `call` is the convenience invoker for the cold path (setup, teardown, once-per-batch reads), where the per-call argument array is fine. NATIVE ACCESS. Loading a library and calling native code are restricted operations; a JVM that denies native access throws `IllegalCallerException`. Run with `--enable-native-access=ALL-UNNAMED` (the `:repl` and `:test` aliases in deps.edn set it).
(async-upcall-stub f desc arena dispatch-map)Build a native upcall stub that ROUTES rather than RUNS. The native thread that fires this stub reads the args, builds an invocation envelope, submits it to the dispatch target, and returns void immediately. The user's fn runs on the dispatch target's thread(s), never on the native thread.
CONTRACT (enforced at build time):
desc MUST be void-returning. A non-void async stub is incoherent:
the native caller cannot consume a value it will never synchronously
receive. Use upcall-stub for value-returning callbacks.arena MUST be the global Arena. A confined arena closes when this
call returns; the stub fires later, from anywhere. Only the global
Arena survives every possible fire.dispatch-map routes the invocation. Build one with
onto-executor or onto-agent.THREADING: your fn runs on the dispatch target's thread(s), not the native thread. The native thread does only read-envelope-submit-return.
SEGMENT SAFETY: pointer args arrive as MemorySegment, but the fn runs on the dispatch target's thread after the routing handle has returned. The native caller may free or reuse the buffer before the fn reads it. The routing handle does not copy segment data, and there is no hook to copy on the native thread. A pointer arg is safe only when the native caller keeps the buffer alive past the dispatch drain. When it does, copy the bytes out at the start of the fn (read-utf8-bounded for strings, read-bytes-bounded for raw bytes) before doing other work.
BACK-PRESSURE: configure the executor's queue bound and rejection policy; they are the back-pressure mechanism. A rejected execution routes to the error handler.
Call release-stub! to stop dispatch after native code has stopped
firing.
Build a native upcall stub that ROUTES rather than RUNS. The native thread that fires this stub reads the args, builds an invocation envelope, submits it to the dispatch target, and returns void immediately. The user's fn runs on the dispatch target's thread(s), never on the native thread. CONTRACT (enforced at build time): - `desc` MUST be void-returning. A non-void async stub is incoherent: the native caller cannot consume a value it will never synchronously receive. Use `upcall-stub` for value-returning callbacks. - `arena` MUST be the global Arena. A confined arena closes when this call returns; the stub fires later, from anywhere. Only the global Arena survives every possible fire. - `dispatch-map` routes the invocation. Build one with `onto-executor` or `onto-agent`. THREADING: your fn runs on the dispatch target's thread(s), not the native thread. The native thread does only read-envelope-submit-return. SEGMENT SAFETY: pointer args arrive as MemorySegment, but the fn runs on the dispatch target's thread after the routing handle has returned. The native caller may free or reuse the buffer before the fn reads it. The routing handle does not copy segment data, and there is no hook to copy on the native thread. A pointer arg is safe only when the native caller keeps the buffer alive past the dispatch drain. When it does, copy the bytes out at the start of the fn (read-utf8-bounded for strings, read-bytes-bounded for raw bytes) before doing other work. BACK-PRESSURE: configure the executor's queue bound and rejection policy; they are the back-pressure mechanism. A rejected execution routes to the error handler. Call `release-stub!` to stop dispatch after native code has stopped firing.
JAVA_BYTE: a C char/int8/bool-as-byte carrier.
JAVA_BYTE: a C `char`/`int8`/`bool`-as-byte carrier.
JAVA_DOUBLE: a C double/f64 carrier.
JAVA_DOUBLE: a C `double`/`f64` carrier.
JAVA_FLOAT: a C float/f32 carrier.
JAVA_FLOAT: a C `float`/`f32` carrier.
JAVA_INT: a C int/int32/DWORD carrier.
JAVA_INT: a C `int`/`int32`/`DWORD` carrier.
JAVA_LONG: a C long long/int64/size_t carrier.
JAVA_LONG: a C `long long`/`int64`/`size_t` carrier.
ADDRESS: a C pointer / opaque handle carrier.
ADDRESS: a C pointer / opaque handle carrier.
JAVA_SHORT: a C short/int16 carrier.
JAVA_SHORT: a C `short`/`int16` carrier.
(call h & args)Invoke a downcall handle h with args. MethodHandle.invoke is
signature-polymorphic and cannot be called reflectively from Clojure, so
this goes through invokeWithArguments, an ordinary varargs method that
builds a per-call argument array. That array is fine for cold-path work
-- setup, teardown, once-per-batch reads -- but NOT for a per-frame hot
path: there, invoke the cached handle directly with typed arguments to
allocate nothing.
Invoke a downcall handle `h` with `args`. `MethodHandle.invoke` is signature-polymorphic and cannot be called reflectively from Clojure, so this goes through `invokeWithArguments`, an ordinary varargs method that builds a per-call argument array. That array is fine for cold-path work -- setup, teardown, once-per-batch reads -- but NOT for a per-frame hot path: there, invoke the cached handle directly with typed arguments to allocate nothing.
(descriptor ret arg-layouts)Build a FunctionDescriptor from ret (a ValueLayout -- use the
c-* shorthands -- or :void) and arg-layouts (a seq of
ValueLayouts). The data shape a caller hands downcall and
upcall-stub to describe a native signature without importing the FFM
classes.
Build a `FunctionDescriptor` from `ret` (a `ValueLayout` -- use the `c-*` shorthands -- or `:void`) and `arg-layouts` (a seq of `ValueLayout`s). The data shape a caller hands `downcall` and `upcall-stub` to describe a native signature without importing the FFM classes.
(downcall lookup nm ret arg-layouts)Bind a cached downcall handle for nm in lookup. ret is a
ValueLayout (a c-* shorthand) or :void; arg-layouts is a seq of
ValueLayouts. Returns a java.lang.invoke.MethodHandle, cached per
distinct [lookup nm ret arg-layouts] so the symbol lookup, descriptor
build, and link happen at most once -- a per-frame call goes through the
cache and does no linker work. Throws (via find-symbol) when the symbol
is absent so the caller degrades rather than faulting on a null segment.
Invoke the returned handle directly with exactly-typed arguments
((.invoke h ...)) on a hot path -- that allocates nothing -- or hand it
to call on the cold path.
Bind a cached downcall handle for `nm` in `lookup`. `ret` is a `ValueLayout` (a `c-*` shorthand) or `:void`; `arg-layouts` is a seq of `ValueLayout`s. Returns a `java.lang.invoke.MethodHandle`, cached per distinct `[lookup nm ret arg-layouts]` so the symbol lookup, descriptor build, and link happen at most once -- a per-frame call goes through the cache and does no linker work. Throws (via `find-symbol`) when the symbol is absent so the caller degrades rather than faulting on a null segment. Invoke the returned handle directly with exactly-typed arguments (`(.invoke h ...)`) on a hot path -- that allocates nothing -- or hand it to `call` on the cold path.
(find-symbol lookup nm)Resolve symbol nm in lookup, returning its address as a
MemorySegment, or throwing an ex-info tagged
:foreign/error :symbol-not-found the caller can catch and degrade (ADR
19) rather than NPE on a null segment.
Resolve symbol `nm` in `lookup`, returning its address as a `MemorySegment`, or throwing an ex-info tagged `:foreign/error :symbol-not-found` the caller can catch and degrade (ADR 19) rather than NPE on a null segment.
(join-then-close-arena worker arena timeout-ms)The teardown tail for a native resource driven on a worker thread: join
worker up to timeout-ms, then close arena once the worker is no
longer alive. The ordering is load-bearing -- closing a shared Arena
while a native frame is still live on the worker faults the VM -- so the
close is gated on the worker no longer being alive. A nil worker (no
thread was started, or it is already gone) still closes arena, so a
caller cannot leak the arena by passing nil. The caller performs any
resource-specific signal step (flip a running flag, close a handle to
unblock a blocking call) BEFORE calling this. Both steps swallow their
exceptions: teardown must not throw.
The teardown tail for a native resource driven on a worker thread: join `worker` up to `timeout-ms`, then close `arena` once the worker is no longer alive. The ordering is load-bearing -- closing a shared `Arena` while a native frame is still live on the worker faults the VM -- so the close is gated on the worker no longer being alive. A nil `worker` (no thread was started, or it is already gone) still closes `arena`, so a caller cannot leak the arena by passing nil. The caller performs any resource-specific signal step (flip a running flag, close a handle to unblock a blocking call) BEFORE calling this. Both steps swallow their exceptions: teardown must not throw.
(library-lookup path)Open the native library at path (a file path string), bound to the
global Arena so the lookup lives for the process -- a library is a
process-lifetime resource (ADR 16). The path loads as an exact file (the
FFM Path overload), not a platform search name, so a content-addressed
cache loads precisely its artifact and a relative name never resolves
through dlopen/LoadLibrary search paths. Returns the SymbolLookup,
or throws an ex-info tagged :foreign/error :library-open-failed (so the
caller can catch and degrade as data, ADR 19) when the library cannot be
opened.
Open the native library at `path` (a file path string), bound to the global Arena so the lookup lives for the process -- a library is a process-lifetime resource (ADR 16). The path loads as an exact file (the FFM `Path` overload), not a platform search name, so a content-addressed cache loads precisely its artifact and a relative name never resolves through `dlopen`/`LoadLibrary` search paths. Returns the `SymbolLookup`, or throws an ex-info tagged `:foreign/error :library-open-failed` (so the caller can catch and degrade as data, ADR 19) when the library cannot be opened.
The process Linker shared by every downcall and upcall stub.
The process `Linker` shared by every downcall and upcall stub.
(onto-agent agnt)(onto-agent agnt opts)Build a dispatch map routing async invocations onto agent, a Clojure
agent. Dispatch uses send, so actions run on Clojure's fixed send pool
(sized to the CPU count). Use this for non-blocking fns; a blocking fn
starves the shared pool and can deadlock unrelated agents. Route onto an
executor instead when the fn blocks. The optional opts map may carry
:error-handler.
Build a dispatch map routing async invocations onto `agent`, a Clojure agent. Dispatch uses `send`, so actions run on Clojure's fixed send pool (sized to the CPU count). Use this for non-blocking fns; a blocking fn starves the shared pool and can deadlock unrelated agents. Route onto an executor instead when the fn blocks. The optional `opts` map may carry :error-handler.
(onto-executor exec)(onto-executor exec opts)Build a dispatch map routing async invocations onto exec, a
java.util.concurrent.Executor. The optional opts map may carry
:error-handler (fn [throwable invocation]); default logs to err.
The executor's own queue bound, thread count, and rejection policy
are the back-pressure mechanism.
Build a dispatch map routing async invocations onto `exec`, a `java.util.concurrent.Executor`. The optional `opts` map may carry :error-handler (fn [throwable invocation]); default logs to *err*. The executor's own queue bound, thread count, and rejection policy are the back-pressure mechanism.
(read-bytes-bounded seg max-bytes)Read up to max-bytes from seg into a fresh byte array. The cap is
the guard: never more than max-bytes cross the boundary. Returns nil
for a NULL segment, and a shorter array when the segment's own size is
below the cap. max-bytes must be non-negative; a negative cap throws
ex-info tagged :foreign/error :invalid-cap. The caller-side counterpart
to read-utf8-bounded for raw byte buffers.
Read up to `max-bytes` from `seg` into a fresh byte array. The cap is the guard: never more than `max-bytes` cross the boundary. Returns nil for a NULL segment, and a shorter array when the segment's own size is below the cap. `max-bytes` must be non-negative; a negative cap throws ex-info tagged :foreign/error :invalid-cap. The caller-side counterpart to `read-utf8-bounded` for raw byte buffers.
(read-utf8-bounded seg max-bytes arena)Read the NUL-terminated UTF-8 C string at seg (a pointer, typically
into memory the OS or another library owns) as a Java String, scanning
no further than max-bytes. Returns nil when seg is NULL, and nil when
no NUL is found within the cap. max-bytes must be non-negative; a
negative cap throws ex-info tagged :foreign/error :invalid-cap, mirroring
read-bytes-bounded.
The cap is the load-bearing guard, not a convenience: the bytes are
untrusted, so the segment is reinterpreted to exactly max-bytes (plus
the terminator slot), NEVER to Long/MAX_VALUE. A missing or corrupt NUL
is then a bounded data outcome (nil), never an unbounded read off the end
of a foreign allocation. arena scopes the reinterpreted view; the
MemorySegment never escapes this fn.
Read the NUL-terminated UTF-8 C string at `seg` (a pointer, typically into memory the OS or another library owns) as a Java `String`, scanning no further than `max-bytes`. Returns nil when `seg` is NULL, and nil when no NUL is found within the cap. `max-bytes` must be non-negative; a negative cap throws ex-info tagged :foreign/error :invalid-cap, mirroring `read-bytes-bounded`. The cap is the load-bearing guard, not a convenience: the bytes are untrusted, so the segment is reinterpreted to exactly `max-bytes` (plus the terminator slot), NEVER to `Long/MAX_VALUE`. A missing or corrupt NUL is then a bounded data outcome (nil), never an unbounded read off the end of a foreign allocation. `arena` scopes the reinterpreted view; the `MemorySegment` never escapes this fn.
(registered-stub-count)The number of currently registered async upcall stubs. An observable for monitoring (a leak detector, a metrics scrape) so a long-lived process can watch for stubs that are registered but never released.
The number of currently registered async upcall stubs. An observable for monitoring (a leak detector, a metrics scrape) so a long-lived process can watch for stubs that are registered but never released.
(release-stub! seg)Mark the stub registered for seg as quiesced and remove it from the
registry. The dispatch target stops receiving invocations after any
in-flight ones drain. The segment itself is not freed (the global
Arena owns it for the process). The caller MUST signal native code to
stop firing BEFORE calling this.
CLOSURE LIFETIME: quiescing stops dispatch, but the user fn f the
stub was built from stays reachable for the JVM's lifetime. The global
Arena owns the upcall segment, and the segment holds the MethodHandle
bound to f (via bindTo), so f and everything it closes over
cannot be GC'd between this call and JVM exit. This is inherent to the
global-Arena + bindTo design (ADR 62 deliberately forbade per-stub
arenas, which would close before a late fire). Bound by how many
distinct callbacks a process registers and releases, not by how often
one fires.
Mark the stub registered for `seg` as quiesced and remove it from the registry. The dispatch target stops receiving invocations after any in-flight ones drain. The segment itself is not freed (the global Arena owns it for the process). The caller MUST signal native code to stop firing BEFORE calling this. CLOSURE LIFETIME: quiescing stops dispatch, but the user fn `f` the stub was built from stays reachable for the JVM's lifetime. The global Arena owns the upcall segment, and the segment holds the MethodHandle bound to `f` (via `bindTo`), so `f` and everything it closes over cannot be GC'd between this call and JVM exit. This is inherent to the global-Arena + bindTo design (ADR 62 deliberately forbade per-stub arenas, which would close before a late fire). Bound by how many distinct callbacks a process registers and releases, not by how often one fires.
(resolve-library {:keys [env candidates default]})Resolve which library path to open, as data, from a config map:
{:env ["MYLIB_PATH" "LIBFOO"] ; env vars to consult, in order
:candidates ["/opt/.../libfoo.dylib" ; concrete paths to probe
"/usr/local/.../libfoo.dylib"]
:default "/opt/.../libfoo.dylib"} ; fallback when none exists
Returns the first set environment variable's value, else the first
candidate path that exists on disk, else :default (which may be nil).
The mechanism is general; the platform-shaped env names, candidate
paths, and .dylib/.dll/.so default belong to the caller. Pair the
result with library-lookup.
Resolve which library path to open, as data, from a config map:
{:env ["MYLIB_PATH" "LIBFOO"] ; env vars to consult, in order
:candidates ["/opt/.../libfoo.dylib" ; concrete paths to probe
"/usr/local/.../libfoo.dylib"]
:default "/opt/.../libfoo.dylib"} ; fallback when none exists
Returns the first set environment variable's value, else the first
candidate path that exists on disk, else `:default` (which may be nil).
The mechanism is general; the platform-shaped env names, candidate
paths, and `.dylib`/`.dll`/`.so` default belong to the caller. Pair the
result with `library-lookup`.(shutdown-async-stubs)Mark all registered async stubs quiesced, drain agent targets, and clear the registry. Installed as a JVM shutdown hook so pending work quiesces before the JVM exits; also callable directly. Idempotent.
Quiesce and removal run per key over a snapshot, so a stub registered during shutdown is not clobbered: it stays registered and active, and a later shutdown will pick it up. A bulk reset! would drop such a stub without quiescing it.
Executor targets are NOT drained here: the caller owns the executor's lifecycle. A blocking native callback that cannot quiesce is the caller's responsibility.
Mark all registered async stubs quiesced, drain agent targets, and clear the registry. Installed as a JVM shutdown hook so pending work quiesces before the JVM exits; also callable directly. Idempotent. Quiesce and removal run per key over a snapshot, so a stub registered during shutdown is not clobbered: it stays registered and active, and a later shutdown will pick it up. A bulk reset! would drop such a stub without quiescing it. Executor targets are NOT drained here: the caller owns the executor's lifecycle. A blocking native callback that cannot quiesce is the caller's responsibility.
(symbol-present? lookup nm)True when nm resolves in lookup. The probe a caller uses before
binding a downcall, so a missing symbol degrades as data (ADR 19) rather
than throwing at bind time.
True when `nm` resolves in `lookup`. The probe a caller uses before binding a downcall, so a missing symbol degrades as data (ADR 19) rather than throwing at bind time.
(upcall-stub f desc arena)Build a native upcall stub for Clojure fn f against FunctionDescriptor
desc, bound to arena. Returns the stub as a MemorySegment -- a C
function pointer native code calls back through. The callback arity is
derived from the descriptor's argument count, so one primitive serves
every callback shape. f receives the native arguments boxed as
Objects (a pointer arrives as a MemorySegment, an integral carrier as
a Long, a float as a Double); guard its body so a single faulty
callback cannot escape into the native run loop.
LIFETIME DISCIPLINE -- load-bearing. arena governs how long the stub's
native pointer stays valid, and freeing it while native code may still
call through it faults the VM. If native code RETAINS the pointer (a
registered window/input callback, a stream callback fired from a run
loop), arena MUST outlive every possible call -- use the
process-lifetime (Arena/global), never a per-frame or confined arena.
Only when the stub is used and discarded entirely within one bounded
scope (a comparator passed to a sort that returns before the scope ends)
may a confined arena own it. This primitive takes arena as a parameter
rather than choosing for you; choosing wrong is a use-after-free.
Build a native upcall stub for Clojure fn `f` against `FunctionDescriptor` `desc`, bound to `arena`. Returns the stub as a `MemorySegment` -- a C function pointer native code calls back through. The callback arity is derived from the descriptor's argument count, so one primitive serves every callback shape. `f` receives the native arguments boxed as `Object`s (a pointer arrives as a `MemorySegment`, an integral carrier as a `Long`, a float as a `Double`); guard its body so a single faulty callback cannot escape into the native run loop. LIFETIME DISCIPLINE -- load-bearing. `arena` governs how long the stub's native pointer stays valid, and freeing it while native code may still call through it faults the VM. If native code RETAINS the pointer (a registered window/input callback, a stream callback fired from a run loop), `arena` MUST outlive every possible call -- use the process-lifetime `(Arena/global)`, never a per-frame or confined arena. Only when the stub is used and discarded entirely within one bounded scope (a comparator passed to a sort that returns before the scope ends) may a confined arena own it. This primitive takes `arena` as a parameter rather than choosing for you; choosing wrong is a use-after-free.
(validate-dispatch-map m)Validate m as an async dispatch map, apply defaults, and return the
normalized map. Throws ex-info tagged :foreign/error :invalid-dispatch-map when malformed.
Required keys: :mode :executor or :agent :target a java.util.concurrent.Executor (:executor) or a Clojure agent (:agent)
Optional: :error-handler (fn [throwable invocation]); default logs to err. The handler may run on the dispatch target's thread (when f throws) or on the native thread (for dispatch errors). It must be thread-safe. clj-zig wraps it so a throwing handler never propagates.
The dispatch target's own queue bound, thread count, and rejection policy are the back-pressure mechanism. Configure them on the target.
Validate `m` as an async dispatch map, apply defaults, and return the
normalized map. Throws ex-info tagged `:foreign/error
:invalid-dispatch-map` when malformed.
Required keys:
:mode :executor or :agent
:target a java.util.concurrent.Executor (:executor)
or a Clojure agent (:agent)
Optional:
:error-handler (fn [throwable invocation]); default logs to *err*.
The handler may run on the dispatch target's thread (when f throws)
or on the native thread (for dispatch errors). It must be thread-safe.
clj-zig wraps it so a throwing handler never propagates.
The dispatch target's own queue bound, thread count, and rejection
policy are the back-pressure mechanism. Configure them on the target.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |