memento.redis.loader — org.clojars.roklenarcic/memento-redis 0.2.20

abandon-load-script clj

bulk-set clj

cached-entries clj

(cached-entries conn ks)

Fetches keys, to retrieve values entered/calculated by a foreign cache (other JVM).

If load marker is still there, the key is not returned. If a value is there, we return a [key value] pair. If the value is not there, then the marker has expired, the foreign loader is dead or stalled. In that case the script returns the special our load marker, and this function returns [key, b/absent].

cached-entries-script clj

cval clj

(cval v)

Carmine saves Numbers and Keywords as Strings, so when you fetch them, they are just left as strings, so we force Nippy serialization. We also force Strings to nippy so we can get the benefit of automatic LZ4 compression on big strings (like people dumping whole JSONs and HTML responses into our DB).

fetch clj

(fetch conn k load-marker load-ms fade-ms)

Fetch a value, returns [present? val]. If load-ms is positive, insert a loader marker with the specified expiry. Also refreshes expiry if fade is used.

load-ms is load marker liveness in ms load-marker must always be provided, even if not setting it, as it's used as type template in an IF

fade-ms is fade setting for entries in ms if any

fetch-script clj

finish-load-script clj

for-cache clj

(for-cache redis-cache)

Return loader for RedisCache

if-cached clj

(if-cached conn k fade-ms)

Retrieves an entry from redis, doing no loading at all, returning b/absent if there is no entry there.

load-marker-fade-sec clj

How long before a load marker fades. This is to prevent JVM exiting or dying from leaving LoadMarkers in Redis indefinitely, causing everyone to block on that key forever. This time is refreshed every second by a daemon thread, however a long GC will cause LoadMarkers to fade when they shouldn't.

Adjust this setting appropriately via memento.redis.load_marker_fade system property.

Loader clj protocol

A loader for the specific Cache

complete clj

(complete this conn k v)

Finishes a load with the given value, giving the entry the desired expire.

put clj

(put this conn k v)

Pushes an entry into cache, ignoring any loading mechanism and locks.

start clj

(start this conn k)

Returns either an IDeref that returns the value of the entry, or nil if the entry is not being cached or calculated. It can be assumed that we claimed a load marker if nil was returned.

maint clj

Map of conn to map of key to promise.

For each connection the submap contains Redis keys to a map of:

• :result (a promise)
• :marker (a load marker)

If marker is present, then it's our load and we must deliver to redis. If not, a foreign JVM is going to deliver to Redis, and we must scan redis for it.

maintain-conn-loads clj

(maintain-conn-loads conn loads-map refresh-load-markers?)

Maintains the list of ongoing loads. It will fetch and fill promises for entries that were completed by other processes. If refresh-load-markers is true, it will also extends load markers this JVM owns.

This should be only called from one thread.

maintenance-step clj

(maintenance-step maint refresh-load-markers?)

Perform maintenance multithreaded, one future per each connection after the first.

new-load-marker clj

(new-load-marker)

refresh-load-markers clj

(refresh-load-markers conn ks markers marker-fade)

Refresh expire of load markers under keys ks, if they are still the same as we expect.

refresh-load-markers-script clj

remove-connections-without-loads clj

(remove-connections-without-loads maint-map)

A function that should be called rarely, which removes connections from loading map which have nothing going on.

remove-load-markers clj

(remove-load-markers maint-map)

Remove all load markers owned by us from Redis. Useful when JVM is shutting down and we want any foreign JVMs on same cache to stop waiting for values that ain't happening.

update-maint clj

(update-maint maint-map conn k f)

Updates maintenance map with f. The return of the function replaces the entry in the maintenance map, but the function also receives a callback which you can call with a value and that value will be the return of the update-maint function. Otherwise the return is nil.

f is a function of (fn [prev-val ret-callback])