blocks.core — mvxcvi/blocks 1.2.0

Block storage API. Functions which may cause IO to occur are marked with bangs.

For example (read! "foo") doesn't have side-effects, but (read! some-input-stream) will consume bytes from the stream.

When blocks are returned from a block store, they may include 'stat' metadata about the blocks, including:

• :source resource location for the block content
• :stored-at time block was added to the store

->store clj

(->store uri)

Constructs a new block store from a URI by dispatching on the scheme. The store will be returned in an initialized (but not started) state.

default-algorithm clj

The hashing algorithm used if not specified in functions which create blocks.

delete! clj

(delete! store id)

Removes a block from the store. Returns true if the block was found and removed.

delete-batch! clj

(delete-batch! store ids)

Removes a batch of blocks from the store, identified by a collection of multihashes. Returns a set of ids for the blocks which were found and deleted.

This is not guaranteed to be an atomic operation; readers may be able to see the store in a partially-deleted state.

erase!! clj

(erase!! store)

Completely removes any data associated with the store. After this call, the store should be empty. This is not guaranteed to be an atomic operation!

from-file clj

(from-file file)

(from-file file algorithm)

Creates a lazy block from a local file. The file is read once to calculate the identifier.

get clj

(get store id)

Loads content for a multihash and returns a block record. Returns nil if no block is stored for that id.

The returned block is checked to make sure the id matches the requested multihash.

get-batch clj

(get-batch store ids)

Retrieves a batch of blocks identified by a collection of multihashes. Returns a sequence of the requested blocks in no particular order. Any blocks which were not found in the store are omitted from the result.

lazy? clj

(lazy? block)

Returns true if the given block loads its content lazily. Returns false if all of the block's content is loaded in memory.

list clj

(list store & opts)

Enumerates the stored blocks, returning a lazy sequence of block stats sorted by id. Iterating over the list may result in additional operations to read from the backing data store.

• :algorithm only return blocks using this hash algorithm
• :after list blocks whose id (in hex) lexically follows this string
• :limit restrict the maximum number of results returned

load! clj

(load! block)

Returns a loaded version of the given block. If the block is lazy, the stream is read into memory and returned as a new block. If the block is already loaded, it is returned unchanged.

The returned block will have the same extra attributes and metadata as the one given.

meta-stats clj

(meta-stats block)

Returns stat information from a block's metadata, if present.

open clj

(open block)

(open block start end)

Opens an input stream to read the contents of the block.

If start and end are given, the input stream will only return content from the starting index byte to the byte before the end index. For example, opening a block with size n with (open block 0 n) would return the full block contents.

put! clj

(put! store block)

Saves a block into the store. Returns the block record, updated with stat metadata.

put-batch! clj

(put-batch! store blocks)

Saves a collection of blocks into the store. Returns a sequence of the stored blocks, in no particular order.

This is not guaranteed to be an atomic operation; readers may be able to see the store in a partially-updated state.

read! clj

(read! source)

(read! source algorithm)

Reads data into memory from the given source and hashes it to identify the block.

scan clj

(scan store)

(scan store p)

Scans all the blocks in the store, building up a store-level summary. If given, the predicate function will be called with each block in the store. By default, all blocks are scanned.

stat clj

(stat store id)

Returns a map with an :id and :size but no content. The returned map may contain additional data like the date stored. Returns nil if the store does not contain the identified block.

store! clj

(store! store source)

(store! store source algorithm)

Stores content from a byte source in a block store and returns the block record.

If the source is a file, it will be streamed into the store. Otherwise, the content is read into memory, so this may not be suitable for large sources.

sync! clj

(sync! source dest & {:as opts})

Synchronize blocks from the source store to the dest store. Returns a summary of the copied blocks. Options may include:

• :filter a function to run on every block stats before it is copied to the dest store. If the function returns a falsey value, the block will not be copied.

validate! clj

(validate! block)

Checks a block to verify that it confirms to the expected schema and has a valid identifier for its content. Returns nil if the block is valid, or throws an exception on any error.

with-stats clj

(with-stats block stats)

Returns the given block with updated stat metadata.

write! clj

(write! block out)

Writes block content to an output stream.