com.eldrix.hermes.impl.lmdb — com.eldrix/hermes 1.0.804

A backing key value store implemented using LMDB.

LMDB has very fast read access, which makes it highly suitable as hermes operates principally in read-only mode. We use netty's direct buffers, and a shared pool because of allocation overhead compared to on-heap buffers.

We use the key value store in one of two ways. The first is to store entities. These are usually keyed by the identifier, except for descriptions, which are keyed by a tuple of concept identifier and description identifier. That optimises the common fetch of all descriptions for a given concept. The second is to store null values as part of an index, with compound keys. This means we can rapidly iterate across a range of keys, which are always sorted, and stored big-endian. The compound key structures are defined below.

It would be possible to create a generic key-value protocol, but this instead contains domain-optimised code.

close clj

(close store)

compact clj

(compact store)

drop-refset-indices clj

(drop-refset-indices store)

Delete all indices relating to reference set items.

drop-relationships-index clj

(drop-relationships-index store)

Deletes all indices relating to relationships.

get-component-refset-ids clj

(get-component-refset-ids store component-id)

Return a set of refset-ids to which this component belongs.

get-component-refset-items clj

(get-component-refset-items store component-id)

(get-component-refset-items store component-id refset-id)

Get the refset items for the given component, optionally limited to the refset specified.

• store
• component-id : id of the component (e.g concept-id or description-id)
• refset-id : (optional) - limit to this refset.

get-concept clj

(get-concept store id)

get-concept-descriptions clj

(get-concept-descriptions store concept-id)

Returns a vector of descriptions for the given concept.

get-description clj

(get-description store description-id)

Return the description with the given description-id. This uses the descriptionId-conceptId index to determine the concept-id, as all descriptions are actually stored by conceptId-descriptionId-concept because that's a more common operation that finding a description by identifier alone.

get-installed-reference-sets clj

(get-installed-reference-sets store)

Returns a set of identifiers representing installed reference sets.

While it is possible to use the SNOMED ontology to find all reference sets:

(get-leaves store (get-all-children store 900000000000455006))

That might return reference sets with no actual members in the installed edition. Instead, we keep track of installed reference sets as we import reference set items, thus ensuring we have a list that contains only reference sets with members.

get-object clj

(get-object env dbi id read-fn)

get-raw-child-relationships clj

(get-raw-child-relationships store concept-id)

(get-raw-child-relationships store concept-id type-id)

Return the child relationships of the given concept. Returns a list of tuples (from--type--group--to).

get-raw-parent-relationships clj

(get-raw-parent-relationships store concept-id)

(get-raw-parent-relationships store concept-id type-id)

Return the parent relationships of the given concept. Returns a list of tuples (from--type--group--to).

get-refset-field-names clj

(get-refset-field-names store refset-id)

Returns the field names for the given reference set.

The reference set descriptors provide a human-readable description and a type for each column in a reference set, but do not include the camel-cased column identifier in the original source file. On import, we store those column names and provide the lookup here.

get-refset-item clj

(get-refset-item store uuid)

(get-refset-item store msb lsb)

Get the specified refset item. Parameters:

• store
• UUID : the UUID of the refset item to fetch
• msb/lsb : the most and least significant 64-bit longs representing the UUID.

get-relationship clj

(get-relationship store relationship-id)

get-source-association-referenced-components clj

(get-source-association-referenced-components store component-id refset-id)

Returns a sequence of component identifiers that reference the specified component in the specified association reference set.

get-source-associations clj

(get-source-associations store component-id)

(get-source-associations store component-id refset-id)

Returns the associations in which this component is the target. targetComponentId -- refsetId -- referencedComponentId - itemId1 - itemId2

index-refsets clj

(index-refsets store)

Iterates all active reference set items and rebuilds indices. Each active item is indexed:

• componentRefsets : referencedComponentId -- refsetId -- msb -- lsb
• associations : targetComponentId -- refsetId -- referencedComponentId - msb - lsb

index-relationships clj

(index-relationships store)

Iterates all active relationships and rebuilds parent and child indices. Each active relationship is referenced in the 'conceptParentRelationships' and 'conceptChildRelationships' indices.

make-dbi-flags clj

(make-dbi-flags read-only? & flags)

map-keys-in-range clj

(map-keys-in-range env dbi start-key end-key f)

Returns a vector consisting of the result of applying f to the keys in the inclusive range between start-key and end-key. 'f' is called with a ByteBuf representing a key within the range. While each key should be a vector of longs, comparison is character-wise. As such,minimum long is 0 and maximum long is -1 (FFFF...) and not Long/MIN_VALUE and Long/MAX_VALUE as might be expected.

open-store clj

(open-store)

(open-store f)

(open-store f opts)

put-flags clj

status clj

(status store)

stream-all clj

(stream-all env dbi ch read-fn)

(stream-all env dbi ch read-fn close?)

Stream all values from the specified dbi to the channel specified.

stream-all-concepts clj

(stream-all-concepts store ch)

(stream-all-concepts store ch close?)

Asynchronously stream all concepts to the channel specified, and, by default, closing the channel when done unless specified. Returns a channel which, by default, will be closed when done.

stream-all-refset-items clj

(stream-all-refset-items store ch)

(stream-all-refset-items store ch close?)

write-concepts clj

(write-concepts store concepts)

Each concept is stored as an entity in the 'concepts' db keyed by identifier.

write-descriptions clj

(write-descriptions store descriptions)

Each description is stored as an entity in the 'descriptions' db, keyed by a tuple of concept-id--description--id.

Each description is referenced in the 'descriptionConcept' index, keyed by description-id--concept-id.

write-refset-items clj

(write-refset-items store headings items)

Each reference set item is stored as an entity in the 'refsetItems' db, keyed by the UUID, a tuple of msb and lsb.

During import, a record of refset field names is created:

• refsetFieldNames : refset-id -- field-names (an array of strings)

write-relationships clj

(write-relationships store relationships)

Each relationship is stored as an entity in the 'relationships' db, keyed by a relationship-id.