A Mandoline store implementation that uses SQLite databases that are persisted on the local file system.
This store persists each dataset as a separate SQLite database on the local filesystem. Each per-dataset SQLite database has 3 tables:
SQLiteChunkStore
type
interacts with this table.SQLiteIndex
type
interacts with this table.SQLiteConnection
type interacts with this table.The mk-schema
function in this namespace instantiates a
SQLiteSchema
type with a root-path
argument that represents a
parent directory. Every dataset that belongs to this schema instance
has a sub-directory under this parent directory. For example, the
dataset "foo" would be stored in the directory
<root-path>/foo/
Each dataset directory contains the SQLite database and journal files for that dataset. Together, these files store the dataset in a durable and portable format. A dataset directory can be safely moved to a new path and opened with a new schema (using the root of the destination path) and a new dataset name (the basename of the destination path).
The SQLite store uses SQLite database transactions to ensure consistency at the expense of performance. Multiple threads and multiple processes can safely read and write the same dataset; however, performance will suffer from exponential backoff and retry upon database contention.
WARNING: Because this store relies on atomic operations of the local filesystem, consistency is not guaranteed when using a network filesystem.
A Mandoline store implementation that uses SQLite databases that are persisted on the local file system. This store persists each dataset as a separate SQLite database on the local filesystem. Each per-dataset SQLite database has 3 tables: - "chunks": This table stores content-addressable binary chunks of the dataset. The columns are "chunk-id" (TEXT), "reference-count" (INTEGER), "data" (BLOB). All versions of the dataset share the same "chunks" table. The `SQLiteChunkStore` type interacts with this table. - "indices": This table stores mappings from coordinates within a versioned dataset to chunk. The columns are "version-id" (TEXT), "coordinates" (TEXT), "chunk-id" (TEXT). The "coordinates" column contains a composite value that includes the variable name and chunk coordinates within the variable. All versions of the dataset share the same "indices" table. The `SQLiteIndex` type interacts with this table. - "versions": This table stores metadata for the version history of the dataset. The columns are "version-id" (TEXT), "timestamp" (TEXT, ISO-8601 encoded), "metadata" (TEXT, JSON encoded). The `SQLiteConnection` type interacts with this table. The `mk-schema` function in this namespace instantiates a `SQLiteSchema` type with a `root-path` argument that represents a parent directory. Every dataset that belongs to this schema instance has a sub-directory under this parent directory. For example, the dataset "foo" would be stored in the directory <root-path>/foo/ Each dataset directory contains the SQLite database and journal files for that dataset. Together, these files store the dataset in a durable and portable format. A dataset directory can be safely moved to a new path and opened with a new schema (using the root of the destination path) and a new dataset name (the basename of the destination path). The SQLite store uses SQLite database transactions to ensure consistency at the expense of performance. Multiple threads and multiple processes can safely read and write the same dataset; however, performance will suffer from exponential backoff and retry upon database contention. WARNING: Because this store relies on atomic operations of the local filesystem, consistency is not guaranteed when using a network filesystem.
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close