A backing key value store implemented using LMDB.
LMDB has very fast read access, which makes it highly suitable as hermes operates principally in read-only mode. We use netty's direct buffers, and a shared pool because of allocation overhead compared to on-heap buffers.
We use the key value store in one of two ways. The first is to store entities. These are usually keyed by the identifier, except for descriptions, which are keyed by a tuple of concept identifier and description identifier. That optimises the common fetch of all descriptions for a given concept. The second is to store null values as part of an index, with compound keys. This means we can rapidly iterate across a range of keys, which are always sorted, and stored big-endian. The compound key structures are defined below.
It would be possible to create a generic key-value protocol, but this instead contains domain-optimised code.
A backing key value store implemented using LMDB. LMDB has very fast read access, which makes it highly suitable as hermes operates principally in read-only mode. We use netty's direct buffers, and a shared pool because of allocation overhead compared to on-heap buffers. We use the key value store in one of two ways. The first is to store entities. These are usually keyed by the identifier, except for descriptions, which are keyed by a tuple of concept identifier and description identifier. That optimises the common fetch of all descriptions for a given concept. The second is to store null values as part of an index, with compound keys. This means we can rapidly iterate across a range of keys, which are always sorted, and stored big-endian. The compound key structures are defined below. It would be possible to create a generic key-value protocol, but this instead contains domain-optimised code.
(drop-refset-indices store)
Delete all indices relating to reference set items.
Delete all indices relating to reference set items.
(drop-relationships-index store)
Deletes all indices relating to relationships.
Deletes all indices relating to relationships.
(get-component-refset-ids store component-id)
Return a set of refset-ids to which this component belongs.
Return a set of refset-ids to which this component belongs.
(get-component-refset-items store component-id)
(get-component-refset-items store component-id refset-id)
Get the refset items for the given component, optionally limited to the refset specified.
Get the refset items for the given component, optionally limited to the refset specified. - store - component-id : id of the component (e.g concept-id or description-id) - refset-id : (optional) - limit to this refset.
(get-concept-descriptions store concept-id)
Returns a vector of descriptions for the given concept.
Returns a vector of descriptions for the given concept.
(get-description store description-id)
Return the description with the given description-id
.
This uses the descriptionId-conceptId index to determine the concept-id,
as all descriptions are actually stored by conceptId-descriptionId-concept because
that's a more common operation that finding a description by identifier alone.
Return the description with the given `description-id`. This uses the descriptionId-conceptId index to determine the concept-id, as all descriptions are actually stored by conceptId-descriptionId-concept because that's a more common operation that finding a description by identifier alone.
(get-installed-reference-sets store)
Returns a set of identifiers representing installed reference sets.
While it is possible to use the SNOMED ontology to find all reference sets:
(get-leaves store (get-all-children store 900000000000455006))
That might return reference sets with no actual members in the installed edition. Instead, we keep track of installed reference sets as we import reference set items, thus ensuring we have a list that contains only reference sets with members.
Returns a set of identifiers representing installed reference sets. While it is possible to use the SNOMED ontology to find all reference sets: ``` (get-leaves store (get-all-children store 900000000000455006)) ``` That might return reference sets with no actual members in the installed edition. Instead, we keep track of installed reference sets as we import reference set items, thus ensuring we have a list that contains only reference sets with members.
(get-raw-child-relationships store concept-id)
(get-raw-child-relationships store concept-id type-id)
Return the child relationships of the given concept. Returns a list of tuples (from--type--group--to).
Return the child relationships of the given concept. Returns a list of tuples (from--type--group--to).
(get-raw-parent-relationships store concept-id)
(get-raw-parent-relationships store concept-id type-id)
Return the parent relationships of the given concept. Returns a list of tuples (from--type--group--to).
Return the parent relationships of the given concept. Returns a list of tuples (from--type--group--to).
(get-refset-field-names store refset-id)
Returns the field names for the given reference set.
The reference set descriptors provide a human-readable description and a type for each column in a reference set, but do not include the camel-cased column identifier in the original source file. On import, we store those column names and provide the lookup here.
Returns the field names for the given reference set. The reference set descriptors provide a human-readable description and a type for each column in a reference set, but do not include the camel-cased column identifier in the original source file. On import, we store those column names and provide the lookup here.
(get-refset-item store uuid)
(get-refset-item store msb lsb)
Get the specified refset item. Parameters:
Get the specified refset item. Parameters: - store - UUID : the UUID of the refset item to fetch - msb/lsb : the most and least significant 64-bit longs representing the UUID.
(get-source-association-referenced-components store component-id refset-id)
Returns a sequence of component identifiers that reference the specified component in the specified association reference set.
Returns a sequence of component identifiers that reference the specified component in the specified association reference set.
(get-source-associations store component-id)
(get-source-associations store component-id refset-id)
Returns the associations in which this component is the target. targetComponentId -- refsetId -- referencedComponentId - itemId1 - itemId2
Returns the associations in which this component is the target. targetComponentId -- refsetId -- referencedComponentId - itemId1 - itemId2
(index-refsets store)
Iterates all active reference set items and rebuilds indices. Each active item is indexed:
Iterates all active reference set items and rebuilds indices. Each *active* item is indexed: - componentRefsets : referencedComponentId -- refsetId -- msb -- lsb - associations : targetComponentId -- refsetId -- referencedComponentId - msb - lsb
(index-relationships store)
Iterates all active relationships and rebuilds parent and child indices. Each active relationship is referenced in the 'conceptParentRelationships' and 'conceptChildRelationships' indices.
Iterates all active relationships and rebuilds parent and child indices. Each *active* relationship is referenced in the 'conceptParentRelationships' and 'conceptChildRelationships' indices.
(map-keys-in-range env dbi start-key end-key f)
Returns a vector consisting of the result of applying f to the keys in the inclusive range between start-key and end-key. 'f' is called with a ByteBuf representing a key within the range. While each key should be a vector of longs, comparison is character-wise. As such,minimum long is 0 and maximum long is -1 (FFFF...) and not Long/MIN_VALUE and Long/MAX_VALUE as might be expected.
Returns a vector consisting of the result of applying f to the keys in the inclusive range between start-key and end-key. 'f' is called with a ByteBuf representing a key within the range. While each key should be a vector of longs, comparison is character-wise. As such,minimum long is 0 and maximum long is -1 (FFFF...) and not Long/MIN_VALUE and Long/MAX_VALUE as might be expected.
(stream-all env dbi ch read-fn)
(stream-all env dbi ch read-fn close?)
Stream all values from the specified dbi to the channel specified.
Stream all values from the specified dbi to the channel specified.
(stream-all-concepts store ch)
(stream-all-concepts store ch close?)
Asynchronously stream all concepts to the channel specified, and, by default, closing the channel when done unless specified. Returns a channel which, by default, will be closed when done.
Asynchronously stream all concepts to the channel specified, and, by default, closing the channel when done unless specified. Returns a channel which, by default, will be closed when done.
(stream-all-refset-items store ch)
(stream-all-refset-items store ch close?)
(write-concepts store concepts)
Each concept is stored as an entity in the 'concepts' db keyed by identifier.
Each concept is stored as an entity in the 'concepts' db keyed by identifier.
(write-descriptions store descriptions)
Each description is stored as an entity in the 'descriptions' db, keyed by a tuple of concept-id--description--id.
Each description is referenced in the 'descriptionConcept' index, keyed by description-id--concept-id.
Each description is stored as an entity in the 'descriptions' db, keyed by a tuple of concept-id--description--id. Each description is referenced in the 'descriptionConcept' index, keyed by description-id--concept-id.
(write-refset-items store headings items)
Each reference set item is stored as an entity in the 'refsetItems' db, keyed by the UUID, a tuple of msb and lsb.
During import, a record of refset field names is created:
Each reference set item is stored as an entity in the 'refsetItems' db, keyed by the UUID, a tuple of msb and lsb. During import, a record of refset field names is created: - refsetFieldNames : refset-id -- field-names (an array of strings)
(write-relationships store relationships)
Each relationship is stored as an entity in the 'relationships' db, keyed by a relationship-id.
Each relationship is stored as an entity in the 'relationships' db, keyed by a relationship-id.
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close