com.eldrix.hermes.importer
Provides import functionality for processing directories of files
Provides import functionality for processing directories of files
all-metadataclj
(all-metadata dir)Returns all release metadata from the directory specified.
Returns all release metadata from the directory specified.
import-fileclj
(import-file f)Import a SNOMED file, returning a map containing :type :headings :parser
and :data as per process-file. This is designed only for testing and
development purposes.
Import a SNOMED file, returning a map containing :type :headings :parser and :data as per [[process-file]]. This is designed only for testing and development purposes.
importable-filesclj
(importable-files dir)Return a list of importable files from the directory specified.
Return a list of importable files from the directory specified.
load-snomedclj
(load-snomed dir & opts)Imports a SNOMED-CT distribution from the specified directory, returning results on the returned channel which will be closed once all files have been sent through. Any exceptions will be passed on the channel.
This streams data in a single pass; in generally usage you will usually want to stream data in multiple passes.
Imports a SNOMED-CT distribution from the specified directory, returning results on the returned channel which will be closed once all files have been sent through. Any exceptions will be passed on the channel. This streams data in a single pass; in generally usage you will usually want to stream data in multiple passes.
load-snomed-filesclj
(load-snomed-files files
&
{:keys [nthreads batch-size]
:or {nthreads 4 batch-size 5000}})Imports a SNOMED-CT distribution from the specified files, returning results on the returned channel which will be closed once all files have been sent through. Any exceptions will be passed on the channel.
Imports a SNOMED-CT distribution from the specified files, returning results on the returned channel which will be closed once all files have been sent through. Any exceptions will be passed on the channel.
metadata-filesclj
(metadata-files dir)Returns a list of release package information files from the directory.
Each entry returned in the list will be a java.io.File.
These files have been issued since the July 2020 International edition release.
Returns a list of release package information files from the directory. Each entry returned in the list will be a `java.io.File`. These files have been issued since the July 2020 International edition release.
process-fileclj
(process-file f out-c & {:keys [batch-size] :or {batch-size 1000}})Process the specified file, streaming batched results to the channel specified, blocking if channel not being drained. Parameters:
- f : anything coercible using clojure.java.io/reader
Each batch is a map with keys
- :type : a type of SNOMED component
- :parser : a parser that can take each row and give you data
- :headings : a sequence of headings from the original file
- :data : a sequence of vectors representing each column.
Process the specified file, streaming batched results to the channel specified, blocking if channel not being drained. Parameters: - f : anything coercible using clojure.java.io/reader Each batch is a map with keys - :type : a type of SNOMED component - :parser : a parser that can take each row and give you data - :headings : a sequence of headings from the original file - :data : a sequence of vectors representing each column.
read-metadataclj
(read-metadata f)Reads the metadata from the file specified.
Unfortunately, some UK releases have invalid JSON in their metadata, so we log an error and avoid throwing an exception. Raised as issue #34057 with NHS Digital.
Unfortunately the name of the release is not included currently, but as the metadata file exists at the root of the release, we can guess the name from the parent directory and use that if a 'name' isn't in the metadata. Raised as issue #32991 with Snomed International.
Reads the metadata from the file specified. Unfortunately, some UK releases have invalid JSON in their metadata, so we log an error and avoid throwing an exception. Raised as issue #34057 with NHS Digital. Unfortunately the *name* of the release is not included currently, but as the metadata file exists at the root of the release, we can guess the name from the parent directory and use that if a 'name' isn't in the metadata. Raised as issue #32991 with Snomed International.
snomed-file-seqclj
(snomed-file-seq dir)A tree sequence for SNOMED CT data files, returning a sequence of maps.
Each result is a map of SNOMED information from the filename as per the release file documentation, with additional keys:
path : the path of the file, component : the canonical name of the SNOMED component (e.g. 'Concept', 'SimpleRefset') component-order : the sort order as defined by component type
A tree sequence for SNOMED CT data files, returning a sequence of maps. Each result is a map of SNOMED information from the filename as per the [release file documentation](https://confluence.ihtsdotools.org/display/DOCRELFMT/3.3.2+Release+File+Naming+Convention), with additional keys: path : the path of the file, component : the canonical name of the SNOMED component (e.g. 'Concept', 'SimpleRefset') component-order : the sort order as defined by component type
SnomedFilecljprotocol
parse-filenameclj
(parse-filename this)Returns structured data about a SNOMED file
Returns structured data about a SNOMED file
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close