Liking cljdoc? Tell your friends :D

com.blockether.vis.internal.persistance.sqlite.maintenance

Space reclamation for a file-backed SQLite store.

SQLite never hands the file back on its own: auto_vacuum is off, so pages a delete or a DROP COLUMN frees go on the FREELIST and are only reused by later writes. That is the right default — reuse costs nothing — but a one-off bulk reclaim (retiring a column across a whole transcript) leaves hundreds of megabytes the file keeps until someone runs VACUUM.

maybe-vacuum! is that someone, on the same fortnight window as foundation.housekeeping retention: at most once per vacuum-interval-days per store, and only when the freelist is worth the rewrite — at least vacuum-min-free-bytes AND vacuum-min-free-fraction of the file. A compact store is never rewritten, so the usual answer is three PRAGMAs and no I/O.

VACUUM takes SQLite's exclusive lock and rewrites the whole file (measured: 15 s and 2.4 GB -> 2.0 GB on a real store), so it runs OFF the open path — vacuum-async! gives it to internal.janitor, the one maintenance worker of a process, after a settling delay, never inside a transaction. Readers and writers in this process or another one block for its duration and then continue, which the 30 s busy timeout and the write-retry ladder in core absorb. A process that exits first, or a db-close! that aborts the lease mid-rewrite, simply leaves the store due at the next start: the rewrite is transactional, so an interrupted VACUUM rolls back rather than damaging anything.

The vis.db.vacuum marker beside the store is BOTH the clock (its mtime is the last successful vacuum) and the cross-process mutex (an exclusive FileLock held for the rewrite). It is created only when a vacuum is actually attempted and deleted again when one fails, so a store that never needed reclaiming carries no marker and stays due.

Space reclamation for a file-backed SQLite store.

SQLite never hands the file back on its own: `auto_vacuum` is off, so pages a
delete or a `DROP COLUMN` frees go on the FREELIST and are only reused by
later writes. That is the right default — reuse costs nothing — but a one-off
bulk reclaim (retiring a column across a whole transcript) leaves hundreds of
megabytes the file keeps until someone runs `VACUUM`.

`maybe-vacuum!` is that someone, on the same fortnight window as
`foundation.housekeeping` retention: at most once per `vacuum-interval-days`
per store, and only when the freelist is worth the rewrite — at least
`vacuum-min-free-bytes` AND `vacuum-min-free-fraction` of the file. A compact
store is never rewritten, so the usual answer is three PRAGMAs and no I/O.

VACUUM takes SQLite's exclusive lock and rewrites the whole file (measured:
15 s and 2.4 GB -> 2.0 GB on a real store), so it runs OFF the open path —
`vacuum-async!` gives it to `internal.janitor`, the one maintenance worker of
a process, after a settling delay, never inside a transaction. Readers and
writers in this process or another one block for its duration and then
continue, which the 30 s busy timeout and the write-retry ladder in `core`
absorb. A process that exits first, or a `db-close!` that aborts the lease
mid-rewrite, simply leaves the store due at the next start: the rewrite is
transactional, so an interrupted VACUUM rolls back rather than damaging
anything.

The `vis.db.vacuum` marker beside the store is BOTH the clock (its mtime is
the last successful vacuum) and the cross-process mutex (an exclusive
`FileLock` held for the rewrite). It is created only when a vacuum is
actually attempted and deleted again when one fails, so a store that never
needed reclaiming carries no marker and stays due.
raw docstring

free-spaceclj

(free-space ds)

{:page-size :page-count :freelist-count :file-bytes :free-bytes} as SQLite itself reports it — :file-bytes is the live file, :free-bytes what a VACUUM would give back.

`{:page-size :page-count :freelist-count :file-bytes :free-bytes}` as SQLite
itself reports it — `:file-bytes` is the live file, `:free-bytes` what a
VACUUM would give back.
sourceraw docstring

maybe-vacuum!clj

(maybe-vacuum! ds db-file)
(maybe-vacuum! ds
               db-file
               {:keys [now-ms interval-days min-free-bytes min-free-fraction]})

Reclaim the freelist of the store at db-file when the fortnight has passed AND the freelist is worth a rewrite. Returns a report — :is-vacuumed with a :reason of :vacuumed, :recent (marker inside the window), :compact (nothing worth reclaiming), :locked (another process is doing it) or :failed — and never throws.

Options, all for tests: :now-ms, :interval-days, :min-free-bytes, :min-free-fraction.

Reclaim the freelist of the store at `db-file` when the fortnight has passed
AND the freelist is worth a rewrite. Returns a report — `:is-vacuumed` with a
`:reason` of `:vacuumed`, `:recent` (marker inside the window), `:compact`
(nothing worth reclaiming), `:locked` (another process is doing it) or
`:failed` — and never throws.

Options, all for tests: `:now-ms`, `:interval-days`, `:min-free-bytes`,
`:min-free-fraction`.
sourceraw docstring

vacuum-async!clj

(vacuum-async! ds db-file)
(vacuum-async! ds db-file {:keys [delay-ms before open?] :as opts})

Give maybe-vacuum! for the store at db-file to the janitor, which runs it after :delay-ms, so the rewrite is not what a starting process waits on. A :before function runs first in the same task, so a data rewrite frees its pages before the reclaim checks them. :open? answers whether the store is still open: a store that closed in the delay is due at its next open. Called once per store per process, from the pool that just opened it. A second call for one store replaces a task that still waits.

The janitor writes the report of a vacuum that ran or that another process holds. A store with nothing to reclaim leaves no line, and a failed vacuum is a warning.

Give `maybe-vacuum!` for the store at `db-file` to the janitor, which runs it
after `:delay-ms`, so the rewrite is not what a starting process waits on. A
`:before` function runs first in the same task, so a data rewrite frees its
pages before the reclaim checks them. `:open?` answers whether the store is
still open: a store that closed in the delay is due at its next open. Called
once per store per process, from the pool that just opened it. A second call
for one store replaces a task that still waits.

The janitor writes the report of a vacuum that ran or that another process
holds. A store with nothing to reclaim leaves no line, and a failed vacuum is a
warning.
sourceraw docstring

vacuum-interval-daysclj

Days between two reclaims of one store. The fortnight foundation.housekeeping/default-retention-days already uses for every other self-bounding artifact: long enough that no ordinary week ever pays the rewrite, short enough that space freed by a migration comes back while the operator still remembers the upgrade.

Days between two reclaims of one store. The fortnight
`foundation.housekeeping/default-retention-days` already uses for every other
self-bounding artifact: long enough that no ordinary week ever pays the
rewrite, short enough that space freed by a migration comes back while the
operator still remembers the upgrade.
sourceraw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
←Move to previous article
→Move to next article
Ctrl+/Jump to the search field
× close