Current state restore for Datomic databases.
Restores a database by copying schema and current datom state (no history) from a source database to a destination database.
:two-pass? false
)Processes all attributes together, managing ref dependencies with a pending index. Best for databases with few ref attributes or simple dependency graphs.
:two-pass? true
)Processes non-ref attributes first, then ref attributes second. Benefits:
:tx-parallelism
)The restore process has these main performance knobs:
:max-batch-size
(default 500) - Datoms per transaction
Higher = fewer transactions, faster overall
:read-parallelism
(default 20) - Parallel attribute reads
Higher = faster reads
:read-chunk
(default 5000) - Datoms per read chunk
Higher = fewer read calls
:tx-parallelism
(default 4) - Parallel transaction workers for Pass 1
Only used when :two-pass? true
Higher = more concurrent transactions (limited by Datomic transactor throughput)
When :debug true
, logs every 10 batches with these metrics:
:pending-count
- Datoms waiting due to ref dependencies
High count → increase :max-batch-size or try :two-pass? true
:last-tx-ms
/ :avg-tx-ms
- Transaction timing
High values → Datomic writes are slow, may be at limit
:batch-efficiency
- Percentage of batch actually written
Low (< 80%) → increase :max-batch-size to resolve more refs
:utilization-pct
- Channel buffer utilization (every 10s)
High (> 80%) → writes are bottleneck (good!)
Low (< 20%) → reads are slow, increase :read-parallelism
At completion:
:avg-tx-ms
- Overall average transaction time:final-pending-count
- Should be 0:duration-sec
- Total time to read all datomsCurrent state restore for Datomic databases. Restores a database by copying schema and current datom state (no history) from a source database to a destination database. ## Restore Strategies ### Single-Pass (default, `:two-pass? false`) Processes all attributes together, managing ref dependencies with a pending index. Best for databases with few ref attributes or simple dependency graphs. ### Two-Pass (`:two-pass? true`) Processes non-ref attributes first, then ref attributes second. Benefits: - Eliminates most pending overhead (pass 1 has zero pending) - Near 100% batch efficiency in pass 1 - Minimal pending in pass 2 (only circular refs) - Expected speedup: 1.5-2.5x for databases with many ref attributes - Pass 1 supports parallel transactions (via `:tx-parallelism`) ## Performance Tuning The restore process has these main performance knobs: - `:max-batch-size` (default 500) - Datoms per transaction Higher = fewer transactions, faster overall - `:read-parallelism` (default 20) - Parallel attribute reads Higher = faster reads - `:read-chunk` (default 5000) - Datoms per read chunk Higher = fewer read calls - `:tx-parallelism` (default 4) - Parallel transaction workers for Pass 1 Only used when `:two-pass? true` Higher = more concurrent transactions (limited by Datomic transactor throughput) ## Performance Metrics When `:debug true`, logs every 10 batches with these metrics: - `:pending-count` - Datoms waiting due to ref dependencies High count → increase :max-batch-size or try :two-pass? true - `:last-tx-ms` / `:avg-tx-ms` - Transaction timing High values → Datomic writes are slow, may be at limit - `:batch-efficiency` - Percentage of batch actually written Low (< 80%) → increase :max-batch-size to resolve more refs - `:utilization-pct` - Channel buffer utilization (every 10s) High (> 80%) → writes are bottleneck (good!) Low (< 20%) → reads are slow, increase :read-parallelism At completion: - `:avg-tx-ms` - Overall average transaction time - `:final-pending-count` - Should be 0 - `:duration-sec` - Total time to read all datoms
cljdoc builds & hosts documentation for Clojure/Script libraries
Ctrl+k | Jump to recent docs |
← | Move to previous article |
→ | Move to next article |
Ctrl+/ | Jump to the search field |