blocks.core — mvxcvi/blocks 2.0.0

Core block storage API.

Functions which may cause side effects or IO are marked with bangs - for example (read! "foo") doesn't have side-effects, but (read! some-input-stream) will consume bytes from the stream.

->store clj

(->store uri)

Constructs a new block store from a URI by dispatching on the scheme. The store will be returned in an initialized (but not started) state.

default-algorithm clj

The hashing algorithm used if not specified in functions which create blocks.

delete! clj

(delete! store id)

Remove a block from the store. Returns a deferred which yields true if the block was found and removed.

delete-batch! clj

(delete-batch! store ids)

Remove a batch of blocks from the store, identified by a collection of multihashes. Returns a deferred which yields a set of ids for the blocks which were found and deleted.

This is not guaranteed to be atomic; readers may see the store in a partially deleted state.

erase! clj

(erase! store)

Completely remove all data associated with the store. After this call, the store will be empty. Returns a deferred which yields true once the store has been erased.

This is not guaranteed to be atomic; readers may see the store in a partially erased state.

from-file clj

(from-file file)

(from-file file algorithm)

Create a lazy block from a local file. Returns the block, or nil if the file does not exist or is empty.

The file is read once to calculate the identifier.

get clj

(get store id)

Load a block from the store. Returns a deferred which yields the block if the store contains it, or nil if no block is stored for that id.

get-batch clj

(get-batch store ids)

Retrieve a batch of blocks identified by a collection of multihashes. Returns a deferred which yields a collection of the blocks which were found.

The blocks are returned in no particular order, and any missing blocks are omitted from the result.

lazy? clj

(lazy? block)

True if the given block reads its content on-demand.

list clj

(list store & opts)

Enumerate the stored blocks, returning a stream of blocks ordered by their multihash id. The store will continue listing blocks until the stream is closed or there are no more matching blocks to return.

• :algorithm Only return blocks identified by this hash algorithm.
• :after Return blocks whose id (in hex) lexically follows this string. A multihash may also be provided and will be coerced to hex.
• :before Return blocks whose id (in hex) lexically precedes this string. A multihash may also be provided and will be coerced to hex.
• :limit Restrict the maximum number of blocks returned on the stream.

list-seq clj

(list-seq store & opts)

Enumerate the stored blocks, returning a sequence of blocks ordered by their multihash id. This wraps the list method and consumes the stream lazily, terminating when the stream is drained, a timeout is encountered, or a list exception is observed on the stream.

Accepts the same options as list, plus:

• :timeout Millisecond duration to wait for new blocks to arrive on the stream. (default: 10000)

load! clj

(load! block)

Ensure the block's content is loaded into memory. Returns a loaded version of the given block.

If the block is lazy, the stream is read into memory and returned as a new block. If the block is already loaded, it is returned unchanged. The returned block will have the same metadata as the one given.

loaded? clj

(loaded? block)

True if the block's content is already loaded into memory.

open clj

(open block)

(open block opts)

Open an input stream to read the contents of the block.

If :start and :end are given, the input stream will only return content from the starting index byte to the byte before the end index. For example, opening a block with size n with (open block {:start 0, :end n}) would return the full block contents.

put! clj

(put! store block)

Save a block into the store. Returns a deferred which yields the stored block, which may have already been present in the store.

put-batch! clj

(put-batch! store blocks)

Save a collection of blocks into the store. Returns a deferred which yields a collection of stored blocks.

This is not guaranteed to be atomic; readers may see the store in a partially updated state.

read! clj

(read! source)

(read! source algorithm)

Read data into memory from the given source and hash it to identify the block.

scan clj

(scan store & opts)

Scan blocks in the store, building up a summary. Returns a deferred which yields the summary map when the scan is complete.

Accepts the same arguments as list, plus:

• :filter A predicate function which will be used to filter blocks listed by the store. By default, all blocks are included.

stat clj

(stat store id)

Load metadata about a block if the store contains it. Returns a deferred which yields a map with block information but no content, or nil if the store does not contain the identified block.

The block stats include the :id, :size, and :stored-at fields. The returned map may also have additional implementation-specific storage metadata, similar to returned blocks.

store! clj

(store! store source)

(store! store source algorithm)

Store content from a byte source in a block store. Returns a deferred which yields the stored block, or nil if the source was empty.

If the source is a file, it will be streamed into the store, otherwise the content is read into memory.

sync! clj

(sync! source dest & opts)

Synchronize blocks from the source store to the dest store. Returns a deferred which yields a summary of the copied blocks. Options may include:

• :filter A function to run on every block before it is synchronized. The block will only be copied if the filter returns a truthy value.

validate! clj

(validate! block)

Check a block to verify that it has the correct identifier and size for its content. Returns true if the block is valid, or throws an exception on any error.

write! clj

(write! block out)

Write a block's content to an output stream.