Liking cljdoc? Tell your friends :D

dev.kwill.datomic-backup.current-state-restore

Current state restore for Datomic databases.

Restores a database by copying schema and current datom state (no history) from a source database to a destination database.

Restore Strategies

Single-Pass (default, :two-pass? false)

Processes all attributes together, managing ref dependencies with a pending index. Best for databases with few ref attributes or simple dependency graphs.

Two-Pass (:two-pass? true)

Processes non-ref attributes first, then ref attributes second. Benefits:

  • Eliminates most pending overhead (pass 1 has zero pending)
  • Near 100% batch efficiency in pass 1
  • Minimal pending in pass 2 (only circular refs)
  • Expected speedup: 1.5-2.5x for databases with many ref attributes
  • Pass 1 supports parallel transactions (via :tx-parallelism)

Performance Tuning

The restore process has these main performance knobs:

  • :max-batch-size (default 500) - Datoms per transaction Higher = fewer transactions, faster overall

  • :read-parallelism (default 20) - Parallel attribute reads Higher = faster reads

  • :read-chunk (default 5000) - Datoms per read chunk Higher = fewer read calls

  • :tx-parallelism (default 4) - Parallel transaction workers for Pass 1 Only used when :two-pass? true Higher = more concurrent transactions (limited by Datomic transactor throughput)

Performance Metrics

When :debug true, logs every 10 batches with these metrics:

  • :pending-count - Datoms waiting due to ref dependencies High count → increase :max-batch-size or try :two-pass? true

  • :last-tx-ms / :avg-tx-ms - Transaction timing High values → Datomic writes are slow, may be at limit

  • :batch-efficiency - Percentage of batch actually written Low (< 80%) → increase :max-batch-size to resolve more refs

  • :utilization-pct - Channel buffer utilization (every 10s) High (> 80%) → writes are bottleneck (good!) Low (< 20%) → reads are slow, increase :read-parallelism

At completion:

  • :avg-tx-ms - Overall average transaction time
  • :final-pending-count - Should be 0
  • :duration-sec - Total time to read all datoms
Current state restore for Datomic databases.

Restores a database by copying schema and current datom state (no history)
from a source database to a destination database.

## Restore Strategies

### Single-Pass (default, `:two-pass? false`)
Processes all attributes together, managing ref dependencies with a pending index.
Best for databases with few ref attributes or simple dependency graphs.

### Two-Pass (`:two-pass? true`)
Processes non-ref attributes first, then ref attributes second.
Benefits:
- Eliminates most pending overhead (pass 1 has zero pending)
- Near 100% batch efficiency in pass 1
- Minimal pending in pass 2 (only circular refs)
- Expected speedup: 1.5-2.5x for databases with many ref attributes
- Pass 1 supports parallel transactions (via `:tx-parallelism`)

## Performance Tuning

The restore process has these main performance knobs:

- `:max-batch-size` (default 500) - Datoms per transaction
  Higher = fewer transactions, faster overall

- `:read-parallelism` (default 20) - Parallel attribute reads
  Higher = faster reads

- `:read-chunk` (default 5000) - Datoms per read chunk
  Higher = fewer read calls

- `:tx-parallelism` (default 4) - Parallel transaction workers for Pass 1
  Only used when `:two-pass? true`
  Higher = more concurrent transactions (limited by Datomic transactor throughput)

## Performance Metrics

When `:debug true`, logs every 10 batches with these metrics:

- `:pending-count` - Datoms waiting due to ref dependencies
  High count → increase :max-batch-size or try :two-pass? true

- `:last-tx-ms` / `:avg-tx-ms` - Transaction timing
  High values → Datomic writes are slow, may be at limit

- `:batch-efficiency` - Percentage of batch actually written
  Low (< 80%) → increase :max-batch-size to resolve more refs

- `:utilization-pct` - Channel buffer utilization (every 10s)
  High (> 80%) → writes are bottleneck (good!)
  Low (< 20%) → reads are slow, increase :read-parallelism

At completion:
- `:avg-tx-ms` - Overall average transaction time
- `:final-pending-count` - Should be 0
- `:duration-sec` - Total time to read all datoms
raw docstring

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close