Liking cljdoc? Tell your friends :D

taoensso.nippy

High-performance serialization library for Clojure

High-performance serialization library for Clojure
raw docstring

*auto-freeze-compressor*clj

(fn [byte-array])->compressor used by `(freeze <x> {:compressor :auto}), nil => default

(fn [byte-array])->compressor used by `(freeze <x> {:compressor :auto}),
nil => default
sourceraw docstring

*custom-readers*clj

{<hash-or-byte-id> (fn [data-input])}

{<hash-or-byte-id> (fn [data-input])}
sourceraw docstring

*final-freeze-fallback*clj

DEPRECATED

DEPRECATED
sourceraw docstring

*freeze-fallback*clj

(fn [data-output x]), nil => default

(fn [data-output x]), nil => default
sourceraw docstring

-cache-proxyclj

{[<x> <meta>] <idx>} for freezing, {<idx> <x-with-meta>} for thawing.

{[<x> <meta>] <idx>} for freezing, {<idx> <x-with-meta>} for thawing.
sourceraw docstring

aes128-cbc-encryptorclj

Default 128bit AES-CBC encryptor with many-round SHA-512 key-gen. See also aes-128-cbc-encryptor.

Default 128bit AES-CBC encryptor with many-round SHA-512 key-gen.
See also `aes-128-cbc-encryptor`.
sourceraw docstring

aes128-encryptorclj

Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.

Password form [:salted "my-password"]

USE CASE: You want more than a small, finite number of passwords (e.g. each item encrypted will use a unique user-provided password).

IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts every key.

PROS: Each key is independent so would need to be attacked independently. CONS: Key caching impossible, so there's an inherent trade-off between encryption/decryption speed and the difficulty of attacking any particular key.

Slower than aes128-cached, and easier to attack any particular key - but keys are independent.

Password form [:cached "my-password"]

USE CASE: You want only a small, finite number of passwords (e.g. a limited number of staff/admins, or you'll be using a single password to encrypt many items).

IMPLEMENTATION: Uses a very expensive (but cached) key hash, and no salt.

PROS: Great amortized encryption/decryption speed. Expensive key hash makes attacking any particular key very difficult. CONS: Using a small number of keys for many encrypted items means that if any key is somehow compromised, all items encrypted with that key are compromised.

Faster than aes128-salted, and harder to attack any particular key - but increased danger if a key is somehow compromised.

Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.

Password form [:salted "my-password"]
---------------------------------------
USE CASE: You want more than a small, finite number of passwords (e.g. each
          item encrypted will use a unique user-provided password).

IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts
                every key.

PROS: Each key is independent so would need to be attacked independently.
CONS: Key caching impossible, so there's an inherent trade-off between
      encryption/decryption speed and the difficulty of attacking any
      particular key.

Slower than `aes128-cached`, and easier to attack any particular key - but
keys are independent.

Password form [:cached "my-password"]
---------------------------------------
USE CASE: You want only a small, finite number of passwords (e.g. a limited
          number of staff/admins, or you'll be using a single password to
          encrypt many items).

IMPLEMENTATION: Uses a _very_ expensive (but cached) key hash, and no salt.

PROS: Great amortized encryption/decryption speed. Expensive key hash makes
      attacking any particular key very difficult.
CONS: Using a small number of keys for many encrypted items means that if any
      key _is_ somehow compromised, _all_ items encrypted with that key are
      compromised.

Faster than `aes128-salted`, and harder to attack any particular key - but
increased danger if a key is somehow compromised.
sourceraw docstring

aes128-gcm-encryptorclj

Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.

Password form [:salted "my-password"]

USE CASE: You want more than a small, finite number of passwords (e.g. each item encrypted will use a unique user-provided password).

IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts every key.

PROS: Each key is independent so would need to be attacked independently. CONS: Key caching impossible, so there's an inherent trade-off between encryption/decryption speed and the difficulty of attacking any particular key.

Slower than aes128-cached, and easier to attack any particular key - but keys are independent.

Password form [:cached "my-password"]

USE CASE: You want only a small, finite number of passwords (e.g. a limited number of staff/admins, or you'll be using a single password to encrypt many items).

IMPLEMENTATION: Uses a very expensive (but cached) key hash, and no salt.

PROS: Great amortized encryption/decryption speed. Expensive key hash makes attacking any particular key very difficult. CONS: Using a small number of keys for many encrypted items means that if any key is somehow compromised, all items encrypted with that key are compromised.

Faster than aes128-salted, and harder to attack any particular key - but increased danger if a key is somehow compromised.

Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.

Password form [:salted "my-password"]
---------------------------------------
USE CASE: You want more than a small, finite number of passwords (e.g. each
          item encrypted will use a unique user-provided password).

IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts
                every key.

PROS: Each key is independent so would need to be attacked independently.
CONS: Key caching impossible, so there's an inherent trade-off between
      encryption/decryption speed and the difficulty of attacking any
      particular key.

Slower than `aes128-cached`, and easier to attack any particular key - but
keys are independent.

Password form [:cached "my-password"]
---------------------------------------
USE CASE: You want only a small, finite number of passwords (e.g. a limited
          number of staff/admins, or you'll be using a single password to
          encrypt many items).

IMPLEMENTATION: Uses a _very_ expensive (but cached) key hash, and no salt.

PROS: Great amortized encryption/decryption speed. Expensive key hash makes
      attacking any particular key very difficult.
CONS: Using a small number of keys for many encrypted items means that if any
      key _is_ somehow compromised, _all_ items encrypted with that key are
      compromised.

Faster than `aes128-salted`, and harder to attack any particular key - but
increased danger if a key is somehow compromised.
sourceraw docstring

cacheclj

(cache x)

Experimental, subject to change.

Wraps value so that future writes of the same wrapped value with same metadata will be efficiently encoded as references to this one.

(freeze [(cache "foo") (cache "foo") (cache "foo")]) will incl. a single "foo", plus 2x single-byte references to "foo".

Experimental, subject to change.

Wraps value so that future writes of the same wrapped value with same
metadata will be efficiently encoded as references to this one.

(freeze [(cache "foo") (cache "foo") (cache "foo")])
  will incl. a single "foo", plus 2x single-byte references to "foo".
sourceraw docstring

compressclj

(compress compressor ba)
source

decompressclj

(decompress compressor ba)
source

decryptclj

(decrypt encryptor pwd ba)
source

encryptclj

(encrypt encryptor pwd ba)
source

extend-freezecljmacro

(extend-freeze type custom-type-id [x out] & body)

Extends Nippy to support freezing of a custom type (ideally concrete) with given id of form:

  • Keyword - 2 byte overhead, resistent to id collisions
  • Integer ∈[1, 128] - no overhead, subject to id collisions

NB: be careful about extending to interfaces, Ref. http://goo.gl/6gGRlU.

(defrecord MyRec [data]) (extend-freeze MyRec :foo/my-type [x data-output] ; Keyword id (.writeUTF [data-output] (:data x))) ;; or (extend-freeze MyRec 1 [x data-output] ; Byte id (.writeUTF [data-output] (:data x)))

Extends Nippy to support freezing of a custom type (ideally concrete) with
given id of form:
  * Keyword           - 2 byte overhead, resistent to id collisions
  * Integer ∈[1, 128] - no overhead, subject to id collisions

NB: be careful about extending to interfaces, Ref. http://goo.gl/6gGRlU.

(defrecord MyRec [data])
(extend-freeze MyRec :foo/my-type [x data-output] ; Keyword id
  (.writeUTF [data-output] (:data x)))
;; or
(extend-freeze MyRec 1 [x data-output] ; Byte id
  (.writeUTF [data-output] (:data x)))
sourceraw docstring

extend-thawcljmacro

(extend-thaw custom-type-id [in] & body)

Extends Nippy to support thawing of a custom type with given id: (extend-thaw :foo/my-type [data-input] ; Keyword id (MyRec. (.readUTF data-input))) ;; or (extend-thaw 1 [data-input] ; Byte id (MyRec. (.readUTF data-input)))

Extends Nippy to support thawing of a custom type with given id:
(extend-thaw :foo/my-type [data-input] ; Keyword id
  (MyRec. (.readUTF data-input)))
;; or
(extend-thaw 1 [data-input] ; Byte id
  (MyRec. (.readUTF data-input)))
sourceraw docstring

fast-freezeclj

(fast-freeze x)

Like freeze but:

  • Writes data without a Nippy header
  • Drops all support for compression and encryption
  • Must be thawed with fast-thaw

Equivalent to (but a little faster than): `(freeze x {:compressor nil :encryptor nil :no-header? true})

Like `freeze` but:
  - Writes data without a Nippy header
  - Drops all support for compression and encryption
  - Must be thawed with `fast-thaw`

Equivalent to (but a little faster than):
  `(freeze x {:compressor nil :encryptor nil :no-header? true})
sourceraw docstring

fast-thawclj

(fast-thaw ba)

Like thaw but:

  • Drops all support for compression and encryption
  • Supports only data frozen with fast-freeze

Equivalent to (but a little faster than): `(thaw x {:compressor nil :encryptor nil :no-header? true})

Like `thaw` but:
  - Drops all support for compression and encryption
  - Supports only data frozen with `fast-freeze`

Equivalent to (but a little faster than):
  `(thaw x {:compressor nil :encryptor nil :no-header? true})
sourceraw docstring

freezable?clj

(freezable? x)
(freezable? x {:keys [allow-clojure-reader? allow-java-serializable?]})

Alpha - subject to change. Returns truthy iff Nippy appears to support freezing the given argument.

:allow-clojure-reader? and :allow-java-serializable? options may be used to enable the relevant roundtrip fallback test(s). These tests are only moderately reliable since they're cached by arg type and don't test for pre/post serialization value equality (there's no good general way of doing so).

Alpha - subject to change.
Returns truthy iff Nippy *appears* to support freezing the given argument.

`:allow-clojure-reader?` and `:allow-java-serializable?` options may be
used to enable the relevant roundtrip fallback test(s). These tests are
only **moderately reliable** since they're cached by arg type and don't
test for pre/post serialization value equality (there's no good general
way of doing so).
sourceraw docstring

freezeclj

(freeze x)
(freeze x
        {:keys [compressor encryptor password]
         :or {compressor :auto encryptor aes128-gcm-encryptor}
         :as opts})

Serializes arg (any Clojure data type) to a byte array. To freeze custom types, extend the Clojure reader or see extend-freeze.

Serializes arg (any Clojure data type) to a byte array. To freeze custom
types, extend the Clojure reader or see `extend-freeze`.
sourceraw docstring

freeze-fallback-as-strclj

DEPRECATED

DEPRECATED
sourceraw docstring

freeze-to-fileclj

(freeze-to-file file x)
(freeze-to-file file x freeze-opts)

Convenience util: writes (freeze x freeze-opts) byte array to (clojure.java.io/file file) and returns the byte array.

(freeze-to-file "my-filename.npy" my-val) => Serialized byte array

Convenience util: writes `(freeze x freeze-opts)` byte array to
`(clojure.java.io/file file)` and returns the byte array.

(freeze-to-file "my-filename.npy" my-val) => Serialized byte array
sourceraw docstring

freeze-to-out!clj

(freeze-to-out! data-output x)

Serializes arg (any Clojure data type) to a DataOutput. This is a low-level util: in most cases you'll want freeze instead.

Serializes arg (any Clojure data type) to a DataOutput.
This is a low-level util: in most cases you'll want `freeze` instead.
sourceraw docstring

IFreezable1cljprotocol

-freeze-without-meta!clj

(-freeze-without-meta! x data-output)
source

IFreezable2cljprotocol

-freeze-with-meta!clj

(-freeze-with-meta! x data-output)
source

inspect-baclj

(inspect-ba ba)
(inspect-ba ba thaw-opts)

Alpha - subject to change

Alpha - subject to change
sourceraw docstring

lz4-compressorclj

Default net.jpountz.lz4 compressor: Ratio: low. Write speed: very high. Read speed: very high.

A good general-purpose compressor, competitive with Snappy.

Thanks to Max Penet (@mpenet) for our first implementation, Ref. https://github.com/mpenet/nippy-lz4

Default net.jpountz.lz4 compressor:
      Ratio: low.
Write speed: very high.
 Read speed: very high.

A good general-purpose compressor, competitive with Snappy.

Thanks to Max Penet (@mpenet) for our first implementation,
Ref. https://github.com/mpenet/nippy-lz4
sourceraw docstring

lz4hc-compressorclj

Like lz4-compressor but trades some write speed for ratio.

Like `lz4-compressor` but trades some write speed for ratio.
sourceraw docstring

lzma2-compressorclj

Default org.tukaani.xz.LZMA2 compressor: Ratio: high. Write speed: very slow (also currently single-threaded). Read speed: slow.

A specialized compressor for large, low-write data in space-sensitive environments.

Default org.tukaani.xz.LZMA2 compressor:
      Ratio: high.
Write speed: _very_ slow (also currently single-threaded).
 Read speed: slow.

A specialized compressor for large, low-write data in space-sensitive
environments.
sourceraw docstring

set-auto-freeze-compressor!clj

(set-auto-freeze-compressor! x)
source

set-freeze-fallback!clj

(set-freeze-fallback! x)
source

snappy-compressorclj

Default org.iq80.snappy.Snappy compressor: Ratio: low. Write speed: very high. Read speed: very high.

A good general-purpose compressor.

Default org.iq80.snappy.Snappy compressor:
      Ratio: low.
Write speed: very high.
 Read speed: very high.

A good general-purpose compressor.
sourceraw docstring

stress-dataclj

Reference data used for tests & benchmarks

Reference data used for tests & benchmarks
sourceraw docstring

stress-data-benchableclj

Reference data with stuff removed that breaks reader or other utils we'll be benching against

Reference data with stuff removed that breaks reader or other utils we'll
be benching against
sourceraw docstring

stress-data-comparableclj

Reference data with stuff removed that breaks roundtrip equality

Reference data with stuff removed that breaks roundtrip equality
sourceraw docstring

swap-custom-readers!clj

(swap-custom-readers! f)
source

thawclj

(thaw ba)
(thaw ba
      {:keys [v1-compatibility? compressor encryptor password]
       :or {compressor :auto encryptor :auto}
       :as opts})

Deserializes a frozen Nippy byte array to its original Clojure data type. To thaw custom types, extend the Clojure reader or see extend-thaw.

** By default, supports data frozen with Nippy v2+ ONLY ** Add {:v1-compatibility? true} option to support thawing of data frozen with legacy versions of Nippy.

Options include: :v1-compatibility? - support data frozen by legacy versions of Nippy? :compressor - :auto (checks header, default) an ICompressor, or nil :encryptor - :auto (checks header, default), an IEncryptor, or nil

Deserializes a frozen Nippy byte array to its original Clojure data type.
To thaw custom types, extend the Clojure reader or see `extend-thaw`.

** By default, supports data frozen with Nippy v2+ ONLY **
Add `{:v1-compatibility? true}` option to support thawing of data frozen with
legacy versions of Nippy.

Options include:
  :v1-compatibility? - support data frozen by legacy versions of Nippy?
  :compressor - :auto (checks header, default)  an ICompressor, or nil
  :encryptor  - :auto (checks header, default), an IEncryptor,  or nil
sourceraw docstring

thaw-from-fileclj

(thaw-from-file file)
(thaw-from-file file thaw-opts)

Convenience util: returns (thaw ba thaw-opts) Clojure value for the byte array read from (clojure.java.io/file file).

(thaw-from-file "my-filename.npy") => Deserialized Clojure value

To thaw from a resource on classpath (e.g in Leiningen resources dir): (thaw-from-file (clojure.java.io/resource "my-resource-name.npy"))

Convenience util: returns `(thaw ba thaw-opts)` Clojure value for the
byte array read from `(clojure.java.io/file file)`.

(thaw-from-file "my-filename.npy") => Deserialized Clojure value

To thaw from a resource on classpath (e.g in Leiningen `resources` dir):
  (thaw-from-file (clojure.java.io/resource "my-resource-name.npy"))
sourceraw docstring

thaw-from-in!clj

(thaw-from-in! data-input)

Deserializes a frozen object from given DataInput to its original Clojure data type.

This is a low-level util: in most cases you'll want thaw instead.

Deserializes a frozen object from given DataInput to its original Clojure
data type.

This is a low-level util: in most cases you'll want `thaw` instead.
sourceraw docstring

throw-unfreezableclj

(throw-unfreezable x)
source

try-write-readableclj

(try-write-readable out x)
source

try-write-serializableclj

(try-write-serializable out x)
source

write-idcljmacro

(write-id out id)
source

write-unfreezableclj

(write-unfreezable out x)
source

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close