Liking cljdoc? Tell your friends :D
Clojure only.
taoensso.carmine.tundra
Semi-automatic archival datastore layer for Carmine. Use multiple Redis instances (recommended) or Redis databases for local key namespacing. Redis keys: * carmine:tundra:evictable - set, ks for which `ensure-ks` fail should throw
dirtyclj
(dirty tstore & ks)Queues given keys for freezing to datastore. Throws an exception if any keys don't exist. Acts as a Redis command: call within a `wcar` context. If TundraStore has a :redis-ttl-ms option, **MARKS GIVEN KEYS FOR EXPIRY**!! ** Worker MUST be running AND FUNCTIONING CORRECTLY or DATA WILL BE LOST! **
ensure-ksclj
(ensure-ks tstore & ks)BLOCKS to ensure given keys are available in Redis, fetching them from datastore as necessary. Throws an exception if any previously evicted keys couldn't be made available. Acts as a Redis command: call within a `wcar` context.
fetch-keys-delayedclj
Used to prevent multiple threads from rushing the datastore to get the same keys, unnecessarily duplicating work.
IDataStorecljprotocol
Extension point for additional datastores.
fetch-keysclj
(fetch-keys dstore ks)(fetch-keys dstore ["key" ...]) => [<#{<frozen-val> <ex>}> ...]
put-keyclj
(put-key dstore k v)(put-key dstore "key" <frozen-val>) => <#{true <ex>}>
IFreezercljprotocol
Extension point for compressors, encryptors, etc.
freezeclj
(freeze freezer x)Returns datastore-ready key val. (comp put-key freeze): dump-ba -> datastore val.
thawclj
(thaw freezer x)Returns Redis-ready key val. (comp thaw fetch-key): datastore val -> dump-ba.
ITundraStorecljprotocol
dirty*clj
(dirty* tstore ks)ensure-ks*clj
(ensure-ks* tstore ks)workerclj
(worker tstore conn-opts wopts)Alpha - subject to change.
Returns a threaded message queue worker to routinely freeze Redis keys
marked as dirty to datastore and mark successfully frozen keys as clean.
Logs any errors. THESE ERRORS ARE **IMPORTANT**: an email or other
appropriate notification mechanism is HIGHLY RECOMMENDED. If a worker shuts
down and your keys are set to expire YOU WILL IRREVOCABLY **LOSE DATA**.
Options:
:nattempts - Number of times worker will attempt to freeze a key to
datastore before failing permanently. >=1.
:retry-backoff-ms - Amount of time (msecs) to backoff before retrying a
failed key freeze. >=0. Can be a (fn [attempt]) -> ms.
:montior, :eoq-backoff-ms, :nthreads, :throttle-ms, :auto-start
- Standard `taoensso.carmine.message-queue/worker` opts.tundra-storeclj
(tundra-store datastore
&
[{:keys [tqname freezer redis-ttl-ms]
:or {tqname "default" freezer nippy-freezer}}])Alpha - subject to change.
Returns a TundraStore with options:
datastore - Storage for frozen key data. Default datastores:
`taoensso.carmine.tundra.faraday/faraday-datastore`
`taoensso.carmine.tundra.s3/s3-datastore`.
:tqname - Optional. Worker message queue name.
:freezer - Optional. Preps key data to/from datastore. May provide
services like compression and encryption, etc. Defaults to
Nippy with default options (Snappy compression and no
encryption).
:redis-ttl-ms - Optional! Time after which frozen, inactive keys will be
EVICTED FROM REDIS (**DELETED!**). Minimum 10 hours. ONLY
use this if you have CONFIRMED that your worker is
successfully freezing the necessary keys to your datastore.
Otherwise YOU WILL IRREVOCABLY **LOSE DATA**.
See `ensure-ks`, `dirty`, `worker` for TundraStore API.cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close