nd-db.convert
db->ndnippyclj
(db->ndnippy in-db out-filename)Converts .ndjson and .ndedn files to .ndnippy.
.ndnippy is MUCH faster (~10x) and requires less memory to process.
Also the resulting database size CAN BE much smaller: ~27GB .ndedn becomes ~12GB .ndnippy (~65% reduction!) in a sample with large (>10KB) documents.
For databases with small documents, the resulting size will be bigger than the original.
Note that this function doesn't create an index, if you want that immediately, use ->ndnippy-db instead (which is faster than first generating the file and creating the index afterwards)
But if you only need the index, this function is much faster than ->ndnippy-db, since it runs in parallel on available cores.
Converts .ndjson and .ndedn files to .ndnippy. .ndnippy is MUCH faster (~10x) and requires less memory to process. Also the resulting database size _CAN BE_ much smaller: ~27GB .ndedn becomes ~12GB .ndnippy (~65% reduction!) in a sample with large (>10KB) documents. For databases with small documents, the resulting size will be bigger than the original. Note that this function doesn't create an index, if you want that immediately, use ->ndnippy-db instead (which is faster than first generating the file and creating the index afterwards) But if you only need the index, this function is much faster than ->ndnippy-db, since it runs in parallel on available cores.
db->ndnippy-dbclj
(db->ndnippy-db in-db {:keys [filename id-path id-fn]})Converts .ndjson and .ndedn files to .ndnippy, and returns the corresponding database.
Use this function to convert to .ndnippy, and return a database, if you need to use the database immediately (or just want to pre- create the persistent index for it).
Since the documents from .ndjson or .ndedn will be parsed anyway, you simply supply a path in the document to the ID value via param :id-path.
Alternatively you can use :id-fn for more complex scenarios.
Converts .ndjson and .ndedn files to .ndnippy, and returns the corresponding database. Use this function to convert to .ndnippy, _and_ return a database, if you need to use the database immediately (or just want to pre- create the persistent index for it). Since the documents from .ndjson or .ndedn will be parsed anyway, you simply supply a path in the document to the ID value via param :id-path. Alternatively you can use :id-fn for more complex scenarios.
upgrade-nddbmeta!clj
(upgrade-nddbmeta! {:keys [filename serialized-filename]})Takes nddbmeta filepaths init param.
If :idx-id is not contained of the old nddbmeta, requires additional param for database filepath.
Upgrades the corresponding nddbmeta file. Keeps old file by adding "_old" to filename. Returns nil if file doesn't exist. Logs if already upgraded.
Takes nddbmeta filepaths init param. If :idx-id is not contained of the old nddbmeta, requires additional param for database filepath. Upgrades the corresponding nddbmeta file. Keeps old file by adding "_old" to filename. Returns nil if file doesn't exist. Logs if already upgraded.
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close