Liking cljdoc? Tell your friends :D

io.mandoline.backend.sqlite

A Mandoline store implementation that uses SQLite databases that are persisted on the local file system.

This store persists each dataset as a separate SQLite database on the local filesystem. Each per-dataset SQLite database has 3 tables:

  • "chunks": This table stores content-addressable binary chunks of the dataset. The columns are "chunk-id" (TEXT), "reference-count" (INTEGER), "data" (BLOB). All versions of the dataset share the same "chunks" table. The SQLiteChunkStore type interacts with this table.
  • "indices": This table stores mappings from coordinates within a versioned dataset to chunk. The columns are "version-id" (TEXT), "coordinates" (TEXT), "chunk-id" (TEXT). The "coordinates" column contains a composite value that includes the variable name and chunk coordinates within the variable. All versions of the dataset share the same "indices" table. The SQLiteIndex type interacts with this table.
  • "versions": This table stores metadata for the version history of the dataset. The columns are "version-id" (TEXT), "timestamp" (TEXT, ISO-8601 encoded), "metadata" (TEXT, JSON encoded). The SQLiteConnection type interacts with this table.

The mk-schema function in this namespace instantiates a SQLiteSchema type with a root-path argument that represents a parent directory. Every dataset that belongs to this schema instance has a sub-directory under this parent directory. For example, the dataset "foo" would be stored in the directory

<root-path>/foo/

Each dataset directory contains the SQLite database and journal files for that dataset. Together, these files store the dataset in a durable and portable format. A dataset directory can be safely moved to a new path and opened with a new schema (using the root of the destination path) and a new dataset name (the basename of the destination path).

The SQLite store uses SQLite database transactions to ensure consistency at the expense of performance. Multiple threads and multiple processes can safely read and write the same dataset; however, performance will suffer from exponential backoff and retry upon database contention.

WARNING: Because this store relies on atomic operations of the local filesystem, consistency is not guaranteed when using a network filesystem.

A Mandoline store implementation that uses SQLite databases that are
persisted on the local file system.

This store persists each dataset as a separate SQLite database on the
local filesystem. Each per-dataset SQLite database has 3 tables:

- "chunks": This table stores content-addressable binary chunks of
  the dataset. The columns are "chunk-id" (TEXT),
  "reference-count" (INTEGER), "data" (BLOB). All versions of the
  dataset share the same "chunks" table. The `SQLiteChunkStore` type
  interacts with this table.
- "indices": This table stores mappings from coordinates within a
  versioned dataset to chunk. The columns are "version-id" (TEXT),
  "coordinates" (TEXT), "chunk-id" (TEXT). The "coordinates"
  column contains a composite value that includes the variable name
  and chunk coordinates within the variable. All versions of the
  dataset share the same "indices" table. The `SQLiteIndex` type
  interacts with this table.
- "versions": This table stores metadata for the version history of
  the dataset. The columns are "version-id" (TEXT), "timestamp"
  (TEXT, ISO-8601 encoded), "metadata" (TEXT, JSON encoded). The
  `SQLiteConnection` type interacts with this table.

The `mk-schema` function in this namespace instantiates a
`SQLiteSchema` type with a `root-path` argument that represents a
parent directory. Every dataset that belongs to this schema instance
has a sub-directory under this parent directory. For example, the
dataset "foo" would be stored in the directory

    <root-path>/foo/

Each dataset directory contains the SQLite database and journal files
for that dataset. Together, these files store the dataset in a durable
and portable format. A dataset directory can be safely moved to a new
path and opened with a new schema (using the root of the destination
path) and a new dataset name (the basename of the destination path).

The SQLite store uses SQLite database transactions to ensure
consistency at the expense of performance. Multiple threads and
multiple processes can safely read and write the same dataset;
however, performance will suffer from exponential backoff and retry
upon database contention.

WARNING: Because this store relies on atomic operations of the local
filesystem, consistency is not guaranteed when using a network
filesystem.
raw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close