Memento Filecache

Extremely limited Memento cache backed by Datalevin KV.

This implementation is intentionally narrow:

• permanent entries only
• single JVM load coalescing only
• invalidation by exact key only
• optional full cache clear
• no segment invalidation
• no tag invalidation
• no TTL, fade, or size eviction
• no retained original keys, segment IDs, segment indexes, or tag indexes

Usage

(ns example
  (:require [memento.core :as m]
            [memento.config :as mc]
            [memento.filecache :as mf]))

(def cached
  (m/memo expensive-fn
          {mc/type mf/cache
           mf/dir "/var/cache/my-app/memento"}))

Caches created with mf/dir share one Datalevin KV connection per canonical filesystem path in the JVM. If the same path is requested with different mf/opts, cache creation throws. Filecache instances are AutoCloseable; close them with with-open or .close when they are no longer used. The last open cache for an mf/dir path closes the shared Datalevin KV connection.

You can also pass a caller-managed Datalevin KV connection with mf/db.

(ns example
  (:require [datalevin.core :as d]
            [memento.config :as mc]
            [memento.filecache :as mf]))

(defonce filecache-db
  (d/open-kv "/var/cache/my-app/memento" {:max-dbs 2}))

(def cache-conf
  {mc/type mf/cache
   mf/db filecache-db})

When using mf/db, the caller owns the Datalevin/LMDB lifecycle. LMDB should not open the same filesystem path multiple times in one JVM. Keep one shared KV connection per path, reuse it for all caches that point at that path, and close it once during application shutdown. Closing a filecache instance does not close a caller-managed mf/db connection.

Tracking And Sweep

memento-filecache can track keys read by one cache instance and later delete entries that were not seen during that tracking window.

(mf/start-tracking! (m/active-cache cached))

;; Run normal application traffic here. `cached` and `if-cached` hits are tracked.

(mf/sweep-untracked! (m/active-cache cached))

Tracking uses an in-memory Bloom filter. False positives are possible, so sweep may keep an unread entry, but it will not delete an entry that was definitely read by this process after start-tracking!.

Tracking sizes the Bloom filter from the current cache entry count when start-tracking! is called, using max(100000, current-entry-count + 1000).

The Bloom filter uses Guava's default 0.03 false-positive probability. memo-add! writes are not tracked as reads. sweep-untracked! deletes in fixed internal batches and disables tracking after it completes.