Datahike provides pluggable storage through konserve, allowing you to choose the backend that best fits your deployment model and performance requirements.
| Backend | Best For | Distribution | Durability | Write Throughput |
|---|---|---|---|---|
| File | Unix tools, rsync, git-like workflows | Single machine | High | Good |
| LMDB | High-performance single machine | Single filesystem | High | Excellent |
| Memory | Testing, ephemeral data | Single process | None | Excellent |
| JDBC | Existing SQL infrastructure | Multi-machine | High | Good |
| Redis | High write throughput | Multi-machine | Medium | Excellent |
| S3 | Distributed scale-out, cost-effective | Multi-region | Very high | Good |
| GCS | Google Cloud scale-out | Multi-region | Very high | Good |
| DynamoDB | Low latency, AWS-native | Multi-region | Very high | Excellent (expensive) |
| IndexedDB | Browser persistence | Browser | Medium | Good |
Use when: You want to use Unix tools (rsync, git, backup scripts) to manage your database.
Key advantage: Deltas in persistent data structures translate directly into individual file deltas, making incremental backups and synchronization highly efficient.
{:store {:backend :file
:path "/var/lib/myapp/db"}}
Characteristics:
Use when: You need maximum performance on a single machine within a single filesystem.
Key advantage: Lightning-fast memory-mapped database with ACID transactions, optimized for read-heavy workloads.
;; Requires: io.replikativ/datahike-lmdb
{:store {:backend :lmdb
:path "/var/lib/myapp/db"}}
Characteristics:
Note: The LMDB backend is available as a separate library: datahike-lmdb, extending konserve-lmdb.
Use when: Testing, development, or ephemeral data that doesn't need to survive process restarts.
{:store {:backend :memory
:id #uuid "550e8400-e29b-41d4-a716-446655440030"}}
Characteristics:
:idAll distributed backends support Distributed Index Space (DIS): multiple reader processes can directly access shared storage without database connections, enabling massive read scalability.
Important: Datahike uses a single-writer model. Multiple readers can access indices concurrently, but only one writer process should transact at a time. This is the same model used by Datomic, Datalevin, and XTDB.
Use when: You already have PostgreSQL or another JDBC database in your infrastructure.
Key advantage: Leverage existing SQL database skills, backup procedures, and monitoring tools.
;; Requires: io.replikativ/datahike-jdbc
{:store {:backend :jdbc
:dbtype "postgresql"
:host "db.example.com"
:port 5432
:dbname "datahike"
:user "datahike"
:password "..."}}
Characteristics:
Note: Available as separate library: datahike-jdbc
Use when: You need high write throughput and can tolerate weaker durability guarantees.
Key advantage: Excellent write performance with in-memory speed.
;; Requires: io.replikativ/konserve-redis
{:store {:backend :redis
:host "redis.example.com"
:port 6379}}
Characteristics:
Use when: You want cost-effective distributed storage that scales to massive datasets.
Key advantage: Extremely scalable, pay-per-use pricing, natural fit for cloud-native architectures.
;; Requires: io.replikativ/konserve-s3
{:store {:backend :s3
:bucket "my-datahike-bucket"
:region "us-east-1"}}
Characteristics:
Performance note: Higher latency than local storage, but cost-effective for billions of datoms.
Use when: You're on Google Cloud Platform and want distributed storage.
Key advantage: Similar to S3 but optimized for GCP infrastructure.
;; Requires: io.replikativ/konserve-gcs
{:store {:backend :gcs
:bucket "my-datahike-bucket"
:project-id "my-project"}}
Characteristics:
Use when: You need low-latency distributed storage and are willing to pay premium pricing.
Key advantage: Single-digit millisecond latency with strong consistency options.
;; Requires: io.replikativ/konserve-dynamodb
{:store {:backend :dynamodb
:table "datahike"
:region "us-east-1"}}
Characteristics:
Use when: Building offline-capable browser applications with persistent local storage.
Key advantage: Durable browser-local storage with ClojureScript support.
;; ClojureScript only
{:store {:backend :indexeddb
:id "my-app-db"}}
Characteristics:
TieredStore creates memory hierarchies by layering backends, with faster storage in front of slower, more durable storage.
Use cases:
;; Example: Fast memory cache backed by S3
{:store {:backend :tiered
:id #uuid "550e8400-e29b-41d4-a716-446655440031"
:frontend-config {:backend :memory
:id #uuid "550e8400-e29b-41d4-a716-446655440031"}
:backend-config {:backend :s3
:bucket "persistent-store"
:region "us-east-1"
:id #uuid "550e8400-e29b-41d4-a716-446655440031"}
:write-policy :write-through
:read-policy :frontend-first}}
How it works:
Common patterns:
Browser with offline support:
{:store {:backend :tiered
:id #uuid "550e8400-e29b-41d4-a716-446655440032"
:frontend-config {:backend :memory
:id #uuid "550e8400-e29b-41d4-a716-446655440032"}
:backend-config {:backend :indexeddb
:id #uuid "550e8400-e29b-41d4-a716-446655440032"}
:write-policy :write-through}}
AWS Lambda with S3 backing:
{:store {:backend :tiered
:id #uuid "550e8400-e29b-41d4-a716-446655440033"
:frontend-config {:backend :lmdb
:path "/tmp/cache"
:id #uuid "550e8400-e29b-41d4-a716-446655440033"}
:backend-config {:backend :s3
:bucket "lambda-data"
:region "us-east-1"
:id #uuid "550e8400-e29b-41d4-a716-446655440033"}}}
Each backend may have additional configuration options. See the konserve backend documentation for details:
→ Memory or File backend for simplicity
→ LMDB for best performance → File if you need Unix tool integration
→ S3/GCS for cost-effective scale → DynamoDB for low latency (higher cost) → JDBC if you already operate PostgreSQL
→ Redis if you can tolerate some data loss → LMDB for durable local writes → DynamoDB for distributed writes (expensive)
→ IndexedDB for persistence → TieredStore (Memory → IndexedDB) for speed + durability
→ File backend with rsync for cheap backups → S3 for large datasets (pennies per GB) → TieredStore to minimize expensive tier access
To migrate from one backend to another:
(require '[datahike.migrate :refer [export-db import-db]])
(export-db source-conn "/tmp/datoms-export")
(d/create-database new-config)
(def dest-conn (d/connect new-config))
(import-db dest-conn "/tmp/datoms-export")
The export format (CBOR) preserves all data types including binary data.
Datahike can use any konserve backend. To create a custom backend:
See the konserve documentation for details on implementing custom backends.
Can you improve this documentation?Edit on GitHub
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |