fully-satisfies

Utilities for Clojure.

• fully-satisfies? -- a variant of clojure.core/satisfies? that also checks if a value implements all methods in the protocol (considering direct, extended, and metadata methods).
• partially-satisfies? -- a variant of clojure.core/satisfies? that is compatible with metadata extension.
• somef -- a variant of clojure.core/some-fn that has a simple definitional equivalence, a zero-arity, and consistent return values.
• everyp -- a variant of clojure.core/every-pred that has a simple definitional equivalence, and a zero-arity.
• never? -- a predicate never? that always returns false.
• run-all! -- a variant of clojure.core/run! that does not short-circuit on reduced.
• clearing-future -- a variant of clojure.core/future that clears conveyed bindings after execution, resolving a known memory leak.
• deftest, testing -- drop-in replacements for clojure.test/{deftest,testing} that report the testing context on uncaught exceptions.
• folda -- a variant of clojure.core/areduce that supports naming the array.
• def-shared-protocol -- a variant of clojure.core/defprotocol whose methods can see future extensions.
• non-overflowing vector, vec -- a vector implementation that consistently handles integer overflow cases, resolving a known general attack vector caused by undefined behavior.
• latest protocol ops -- implementations of satisfies?, find-protocol-impl, find-protocol-method, extends?, extenders that look up the latest version of the protocol such that they have the same behavior with partial.
• non-leaky clojure.core, clojure.test macros -- versions of the following macros that don't leak implementation details (eg., recur targets, pre/post, try/catch syntax): locking (upstream report), binding, with-bindings, sync, with-local-vars, with-in-str, dosync, with-precision, with-loading-context, with-redefs, delay, vswap!, lazy-seq, lazy-cat, future, pvalues, clojure.test/{deftest,deftest-,testing,with-test,with-test-out}, clojure.java.shell/with-sh-{dir,env}, clojure.test.tap/with-tap-output, clojure.pprint/with-pprint-dispatch, clojure.core.async/thread, clojure.core.logic.pldb/with-{db,dbs}, clojure.tools.trace/dotrace, clojure.test.check.properties/for-all, clojure.test.check.generators/let, clojure.java.jmx/with-connection, clojure.core.match.debug/with-recur
• ::reify-args, a spec for clojure.core/reify. Use register-reify-spec to add to spec's registry.
• Leaky-seq detection testing framework for detecting memory leaks caused by holding onto the head of sequences. An example test suite is provided exposing issues with clojure.core functions and verifying fixes to them.
• Safer variants of clojure.core functions that improve thread-safety and general robustness when passed mutating collections. Includes: butlast, drop-last, every?, last, not-every?, nthrest, partitionv-all, sort, sort-by, split-at, split-with, splitv-at, take-last. Includes a test suite demonstrating the differences and improvements.
• Lazier variants of clojure.core functions that are slightly lazier when processing and/or returning lazy seqs. Includes: bounded-count, cycle, dedupe, iterator-seq, sequence. Includes a test suite demonstrating the differences and improvements.
• Uniformly generalized variants of clojure.core functions such that they work consistently for all values. Includes: dedupe, halt-when, partition-by. Includes a test suite demonstrating the differences and improvements.
• Head-releasing variants of clojure.core functions that release the head of seqs earlier have thus have improved memory usage characteristics. Includes: every?, keep, keep-indexed, map, map-indexed, mapcat, naive-seq-reduce, not-any?, not-every?, some. Includes a test suite demonstrating the differences and improvements, using the leaky-seq detection testing framework.
• Thread-safe requiring-resolve -- a variant of clojure.core/requiring-resolve that fixes CLJ-2735 and is safe to use concurrently with other calls to itself. Any concurrent calls to require must acquire RT/REQUIRE_LOCK and it is not safe to use concurrently with clojure.core/requiring-resolve, with the same caveats as CLJ-2735.
• Safe locals-clearing variants of delay, lazy-seq -- fixes CLJ-2861 where of delay and lazy-seq do not protect their locals-cleared bodies from recursive calls.

Latest API documentation

Current version API documentation

Dependency

Available on Clojars.

Leiningen:

[io.github.frenchy64/fully-satisfies "1.11.2"]

Clojure CLI (Maven deps):

  :deps {io.github.frenchy64/fully-satisfies 
         {:mvn/version "1.11.2"}}

Clojure CLI (git deps):

  ;; requires `clj -X:deps prep` to compile java
  :deps {io.github.frenchy64/fully-satisfies 
         {:git/tag "1.11.2", :git/sha "227f96f"}}

Try it in a REPL:

# compile
clj -Sdeps '{:deps {io.github.frenchy64/fully-satisfies {:git/tag "1.11.2", :git/sha "227f96f"}}}' -X:deps prep
# start REPL
clj -Sdeps '{:deps {io.github.frenchy64/fully-satisfies {:git/tag "1.11.2", :git/sha "227f96f"}}}'

Usage

fully-satisfies?

Docstring

(require '[io.github.frenchy64.fully-satisfies :refer [fully-satisfies?]])

(defprotocol A
  (a [this])
  (b [this]))

(fully-satisfies? A (reify))
;=> false
(fully-satisfies? A (reify A))
;=> false
(fully-satisfies? A (reify A (a [this])))
;=> false
(fully-satisfies? A (reify A (a [this]) (b [this])))
;=> true

run-all!

(require '[io.github.frenchy64.fully-satisfies.run-all :refer [run-all!]])

(run-all! println [1 (reduced 2) 3])
;1
;2
;3
;=> nil
;; does not short-circuit on reduced
(run-all! println [1 (reduced 2) 3])
;1
;#object[clojure.lang.Reduced 0x3deeac1 {:status :ready, :val 2}]
;3
;=> nil

Releasing

1. Change project.clj version to desired version + SNAPSHOT
   • eg., 1.0.0-SNAPSHOT
2. Commit with message Release {:major,:minor,:patch}
   • this releases the current version then bumps to the next {:major,:minor,:patch} SNAPSHOT
3. Pull

Thanks

Thanks Wanderson Ferreira for the idea of fully-satisfies? and its name. My initial stance that fully-satisfies? was impossible to implement quickly proved to be incorrect after Wanderson's asked the right questions and decompiled some bytecode.

Wanderson and Mark Herman, II also helped improve early iterations of fully-satisfies?.

Related work

• https://clojure.atlassian.net/browse/CLJ-2426
• https://clojure.atlassian.net/browse/CLJ-1814
• https://clojure.atlassian.net/browse/CLJ-2656
  • result of making this library and realizing supers call is suspicious

TODO

• https://clojure.atlassian.net/browse/CLJ-2162
• https://clojure.atlassian.net/browse/CLJ-2069
• agents memory leak via conveyed bindings
• https://ask.clojure.org/index.php/11080/get-find-assoc-vectors-overflows-key-when-passed-large-longs?show=11137#c11137

Clojure 1.10.3
user=> (get (into-array [1 2 42]) 4294967296)
1
user=> (get "123" 4294967296)
\1
user=> (get (into-array [1 2 42]) 4294967296 :not-found)
1
user=> (get "123" 4294967296 :not-found)
\1

• https://clojure.atlassian.net/browse/CLJ-2322
• throwArity(21) in AFn.java should be throwArity(20+args.length)
  • reprod:

Clojure 1.11.1
user=> (apply {} (range 21))
Execution error (ArityException) at user/eval214 (REPL:1).
Wrong number of args (21) passed to: clojure.lang.PersistentArrayMap
user=> (apply {} (range 22))
Execution error (ArityException) at user/eval216 (REPL:1).
Wrong number of args (21) passed to: clojure.lang.PersistentArrayMap
user=> (apply {} (range 24))
Execution error (ArityException) at user/eval218 (REPL:1).
Wrong number of args (21) passed to: clojure.lang.PersistentArrayMap

• same in RestFn.java
  • reprod:

Clojure 1.11.1
user=> (defmacro big-fn [nargs] `(fn ~(conj (mapv #(gensym (do % "arg")) (range nargs)) 'last)))
#'user/big-fn
user=> (apply (big-fn 19) (range 21))
Execution error (ArityException) at user/eval145 (REPL:1).
Wrong number of args (21) passed to: user/eval145/fn--165
user=> (apply (big-fn 19) (range 22))
Execution error (ArityException) at user/eval168 (REPL:1).
Wrong number of args (21) passed to: user/eval168/fn--188
user=> (apply (big-fn 19) (range 24))
Execution error (ArityException) at user/eval191 (REPL:1).
Wrong number of args (21) passed to: user/eval191/fn--211

• direct-linkable protocols

  • apparently a matter of makings protocols non-closures
  • https://ask.clojure.org/index.php/10967/are-protocol-methods-guaranteed-to-not-be-directly-linked?show=10990#a10990
  • approach:
    • add new field to clojure.lang.MethodImplCache$Entry which distinguishes between the (.isInstance c x) test in -cache-protocol-fn
    • update the identical? clause in the with-meta-extension part to grab this entry raw (eg., replace the fnFor with fnEntryFor)
    • move ginterf into the -cache-protocol-fn calls (1 arity at a time)

• defmulti with cache controls https://clojure.atlassian.net/browse/CLJ-2626

• count+last vs count+butlast+last

• thrown?, thrown-with-msg? inherits try syntax https://github.com/clojure/clojure/blob/5ffe3833508495ca7c635d47ad7a1c8b820eab76/src/clj/clojure/test.clj#L504-L535

• • report another some-fn and every-pred short-circuiting inconsistencies

;; # some-fn
;; these are `(some #(some % args) fs)`, or `(some #(some % fs) args)`, or some hybrid
;; should be `(some #(some % args) fs)`
       ([x y z & args] (or (sp2 x y z)
                           (some #(or (p1 %) (p2 %)) args)))))
       ([x y z & args] (or (sp3 x y z)
                           (some #(or (p1 %) (p2 %) (p3 %)) args)))))
         ([x y z & args] (or (spn x y z)
                             (some #(some % args) ps)))))))

;; # every-pred
;; these are `(every? #(every? % args) preds)` or `(every? #(every? % preds) args)`, or some hybrid
;; should be `(every? #(every? % args) preds)`.
       ([x y z & args] (boolean (and (ep2 x y z)
                                     (every? #(and (p1 %) (p2 %)) args))))))
       ([x y z & args] (boolean (and (ep3 x y z)
                                     (every? #(and (p1 %) (p2 %) (p3 %)) args))))))
         ([x y z & args] (boolean (and (epn x y z)
                                       (every? #(every? % args) ps))))))))

• (count (eduction)) => count not supported on this type: Eduction

License

Where noted, contains code from Clojure under license:

Copyright (c) Rich Hickey. All rights reserved.
The use and distribution terms for this software are covered by the
Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
which can be found in the file epl-v10.html at the root of this distribution.
By using this software in any fashion, you are agreeing to be bound by
the terms of this license.
You must not remove this notice, or any other, from this software.

Otherwise:

Copyright © 2021 Ambrose Bonnaire-Sergeant

This program and the accompanying materials are made available under the
terms of the Eclipse Public License 2.0 which is available at
http://www.eclipse.org/legal/epl-2.0.

This Source Code may also be made available under the following Secondary
Licenses when the conditions for such availability set forth in the Eclipse
Public License, v. 2.0 are satisfied: GNU General Public License as published by
the Free Software Foundation, either version 2 of the License, or (at your
option) any later version, with the GNU Classpath Exception which is available
at https://www.gnu.org/software/classpath/license.html.