Change Log

0.9.21 (2025-03-18)

Added

• [Vector] new-vector-index function creates an on-disk index for equal-length dense numeric vectors to allow similarity search, and related functions close-vector-index, clear-vector-index and vector-index-info. Vector indexing is implemented with usearch.
• [Vector] Corresponding add-vec, remove-vec, and search-vec functions to work with vector index. Similar to full-text search, vector search also support domain semantics to allow grouping of vectors into domains, see doc.
• [Datalog] New data type :db.type/vec, for which a vector index is automatically created for them to allow a query function vec-neighbors to return the datoms with neighboring vector values. #145

Fixed

• [Datalog] Correctly handle certain case of attribute without value. #320

Improved

• [Datalog] Limit the time for counting datoms during planning with a dynamic var datalevin.constants/range-count-time-budget in milliseconds, default is 10; another dynamic var datalevin.constants/range-count-iteration-step determines after how many loop iterations to take a time measure, default is 1000.
• [Datalog] Same for sampling with sample-time-budget and sample-iteratioin-step, defaults are 2 milliseconds and 20000, respectively.
• [Datalog] Reduce the exhaustive plan space to P(n, 2).
• [Datalog] Add :parsing-time for query parsing and :building-time for query graph building to explain results.
• [Search] Dump data to file when re-index to save memory.
• [KV] Periodically flush to disk, with an interval set by dynamic var datalevin.constants/lmdb-sync-interval in seconds, default is 300. This also cleans up dead readers.
• [KV] Remove the semaphore for preventing "Environment maxreaders limit reached" error, as it hurts read performance. User should be careful with functions such as pmap and futurefor parallel read operation, as they use unbounded thread pools. User can use a semaphore to limit the number of read threads in flight, or use a bounded thread pool. If needed, :max-readers KV option can also be set to increase the limit (default is now doubled to 256).
• [Doc] Add built-ins namespace for query functions to cljdoc.

0.9.20 (2025-02-19)

Improved

• [Datalog] Able to disable background sampling.

0.9.19 (2025-02-19)

Added

• [Search] Boolean search expression. #104 #310
• [Search] Phrase search. #311
• [Datalog] analyze function to collect statistics that helps with query planner.
• [Datalog] max-eid function to return the current maximal entity id.
• [Benchmark] write-bench that compares Datalevin with SQLite on transaction performance, and studies Datalevin KV write performance under various conditions.

Fixed

• [Datalog] and in or-join exception #304
• [Datalog] and join exception. #305
• [Datalog] Validation of negative doubles #308
• [Datalog] seek-datom and rseek-datom broken for :eav index

Improved

• [Datalog] fill-db no longer creates a new DB, to reduce chance of user errors. #306
• [Datalog] Added arity to seek-datoms and rseek-datoms to specify the number of datoms desired. [Thx @jeremy302] #312
• [Datalog] datoms function takes an extra n argument as well.
• [Datalog] Faster attribute size count by selecting different counters based on cardinality.
• [Datalog] Sort by cardinality when picking minimal attribute.
• [KV] Prevent "MDB_READERS_FULL: Environment maxreaders limit reached" error when running hundreds of concurrent reading threads.
• [Async] Resepct individual callback of each async call.

0.9.18 (2025-01-20)

Added

• [KV] set-env-flags and get-env-flags, so users may change the env flags after the DB is open. This is useful for adjusting transaction durability settings on the fly.

Fixed

• [Platform] Clojure compiler adds java.util.SequencedCollection automatically when compiled on JDK 21 and above, which does not exist in earlier JDK. Compile library with JDK 17 instead.
• [KV] Type validation and coercion for :bytes.

Improved

• [Feature] Streamline async transactions so it respects env flags.
• [Datalog] Reduce default sample change ratio to 0.05.

0.9.17 (2025-01-17)

Improved

• [Doc] Remove test namespaces from cljdoc.

0.9.16 (2025-01-17)

Changed

• [Platform] Drop Java 8 support. Minimum JVM is now 11.

Improved

• [Doc] Fix cljdoc build.
• [Package] Exclude tests, test data, Java source files from releases.

0.9.15 (2025-01-17)

Added

• [KV] transanct-kv-async function to return a future and transact in batches to enhance write throughput (2.5X or more higher throughput in heavy write condition compared with transact-kv). Batch size is automatically adjusted to the write workload: the higher the load, the larger the batch. Howver, manual batching of transaction data is still helpful, as the effects of automatic batching and manual batching add up. #256

Fixed

• [Datalog] Upsert fails on entity with unique composite tuple #299

Improved

• [Datalog] Both transact and transact-async are also changed to use the same adaptive batching transaction mechanism to enhance write throughout.
• [Datalog] Reduce default sample-processing-interval to 10 seconds, so samples are more up to date. Each invocation will do less work, or no work , based on a change ratio, controlled by a dynamic var sample-change-ratio, default is 0.1, i.e. resample if 10 percent of an attribute's values changed.
• [Datalog] Port applicable fixes from Datascript up to 1.7.4.
• [KV] Consolidated LMDB bindings to a single binding using JavaCPP. Removed both LMDBJava and Graalvm native image specific bindings. This eases maintenance, enhances performance, and makes it easier to add native dependencies. #35.
• [KV] Pushed all LMDB iterators, counters, sampler, and comparator implementation down to C code. #279.
• [KV] Now it is optional to add --add-opens JVM options to open modules for Java 11 and above. If these JVM options are not set, Datalevin will use a safer but slower default option instead of throwing exceptions. It is still recommended to add these JVM options to get optimal performance. For now, native image uses only the safer option.
• [Native] Datalevin library jar can now be used directly to compile GraalVM native image. There's no longer a need for a GraalVM specific Datalevin library, nor GraalVM version restriction.
• [Native] Upgrade to the latest version of GraalVM .
• [Server] Handle rare cases of exceptions in event loop.

0.9.14 (2024-11-25)

Added

• [KV] get-first-n to return first n key-values in a key range.
• [KV] list-range-first-n to return first n key-values in a key-value range of a list DBI. #298

Fixed

• [Datalog] Exception in or-join. #295

Improved

• [Datalog] Faster init-db, fill-db, re-index and load by writing in :nosync mode and syncing only at the end.
• [Datalog] Push down nested predicates as well.
• [Datalog] Remove c/plan-space-reduction-threshold, always use P(n, 3) instead, as only initial 2 joins have accurate size estimation, so larger plan space for later steps is not really beneficial. This change improves JOB.
• [Datalog] Many small code optimizations to improve query speed about 10%.

0.9.13 (2024-11-09)

Fixed

• [Datalog] Repeated cardinality many attributes for the same entity. #284
• [Datalog] Apply type coercion according to scheam to ensure correct storage of values. #285
• [Datalog] Correct query caching when referenced content may have changed. #288 [Thx @andersmurphy]
• [Pod] Typos in search code. #291

Improved

• [Pod] Consistent entity behavior in pod as in JVM. #283
• [Datalog] Allow :offset 0.
• [Datalog] Implement empty on Datom so it can be walked. #286
• [Datalog] Query functions resolve their arguments recursively. #287
• [Native] Remove :aot to avoid potential dependency conflict.

0.9.12 (2024-10-08)

Added

• [Datalog] :offset and :limit support, #126, #117
• [Datalog] :order-by support, #116
• [Datalog] count-datoms function to return the number of datoms of a pattern
• [Datalog] cardinality function to return the number of unique values of an attribute

Improved

• [KV] Added version number to kv-info, in preparation for auto-migration.
• [Datalog] Cache query results by default, can use dynamic var q/*cache?* to turn it off.

0.9.11 (2024-10-04)

Fixed

• [Datalog] Empty results after querying empty database before transact. #269
• [Datalog] Handle multiple variables assigned to the same cardinality many attribute. #272
• [Datalog] Return maps regression. #273
• [Datalog] Regression in dealing with non-existent attributes. #274

Changed

• [KV] Change default write option to be the safest, the same as LMDB defaults, i.e. synchronous flush to disk when commit.

Added

• [KV] Expose sync function to force a synchronous flush to disk, useful when non-default flags for writes are used.
• [Pod] added clear to bb pod.

Improved

• [JVM] Support Java 8 in uberjar, following Clojure supported Java version.
• [Pod] Added missing arity in get-conn [Thx @aldebogdanov]

0.9.10 (2024-08-11)

Fixed

• [Native] compile native image with UTF-8 encoding on Arm64 Linux and Windows.

0.9.9 (2024-08-11)

Added

• [Platform] native image on Arm64 Linux. [Thx @aldebogdanov]
• [Benchmark] ported Join Order Benchmark (JOB) from SQL

Fixed

• [Datalog] Planner: nested logic predicates.
• [Datalog] Planner: multiple predicates turned ranges.
• [Datalog] Planner: missing range turned predicates.
• [Datalog] Planner: need to first try target var to find index for :ref plan.
• [Datalog] Planner: fail to unify with existing vars in certain cases. #263
• [Datalog] Planner: skip initial attribute when it does not have a var.
• [Datalog] Planner: target var may be already bound for link step.
• [Datalog] Planner: missing bound var in merge scan.
• [Datalog] like function failed to match in certain cases.
• [Datalog] clear function also clear the meta DBI

Improved

• [Datalog] Planner: execute initial step if result size is small during planning, controlled by dynamic var init-exec-size-threshold (default 1000), above which, the same number of samples are collected instead. These significantly improved subsequent join size estimation, as these initial steps hugely impact the final plan.
• [Datalog] Planner: search full plan space initially, until the number of plans considered in a step reaches plan-space-reduction-threshold (default 990), then greedy search is performed in later stages, as these later ones have less impact on performance. This provides a good balance between planning time and plan quality, while avoiding potential out of memory issue during planning.
• [Datalog] Planner: do parallel processing whenever appropriate during planning and execution (regular JVM only).
• [LMDB] Lock env when creating a read only txn to have safer concurrent reads.

Changed

• [Datalog] maintain an estimated total size and a representative sample of entity ids for each attribute, processed periodically according to sample-processing-interval(default 3600 seconds).
• [Datalog] reduce default *fill-db-batch-size* to 1 million datoms.
• [KV] throw exception when transacting nil, #267

0.9.8 (2024-06-29)

Fixed

• [Datalog] Planner: column attributes should be a set of equivalent attribute and variable.
• [Datalog] Planner: convert ranges back to correct predicates.
• [Datalog] Handle like, in within complex logic expressions.

Improved

• [Datalog] Optimize not, and and or logic functions that involve only one variable.

0.9.7 (2024-06-23)

Fixed

• [Datalog] Handle bounded entity IDs in reverse reference and value equality scans, #260

Improved

• [Datalog] Added :result to explain result map.

0.9.6 (2024-06-21)

Added

• [Datalog] like function similar to LIKE operator in SQL: (like input pattern) or (like input pattern opts). Match pattern accepts wildcards % and _. opts map has key :escape that takes an escape character, default is \!. Pattern is compiled into a finite state machine that does non-greedy (lazy) matching, as oppose to the default in Clojure/Java regex. This function is further optimized by rewritten into index scan range boundary for patterns that have non-wildcard prefix. Similarly, not-like function is provided.
• [Datalog] in function that is similar to IN operator in SQL: (in input coll) which is optimized as index scan boundaries. Similarly, not-in.
• [Datalog] fill-db function to bulk-load a collection of trusted datoms, *fill-db-batch-size* dynamic var to control the batch size (default 4 million datoms). The same var also control init-db batch size.
• read-csv function, a drop-in replacement for clojure.data.csv/read-csv. This CSV parser is about 1.5X faster and is more robust in handling quoted content.
• Same write-csv for completeness.

Fixed

• [Datalog] Wrong variable may be returned when bounded variables are involved. #259

Changed

• [KV] Change default initial DB size to 1 GiB.

Improved

• [Platform] Use local LMDB library on FreeBSD if available [thx @markusalbertgraf].
• [Datalog] min and max query predicates handle all comparable data.
• [Datalog] Port applicable fixes from Datascript up to 1.7.1.
• update deps.

0.9.5 (2024-04-17)

Fixed

• [Datalog] planner generates incorrect step when bound variable is involved in certain cases.
• [Datalog] explain throws when zero result is determined prior to actual planning. [Thx @aldebogdanov]
• [Datalog] regression in staged entity transactions for refs, #244, [Thx @den1k]

Improved

• [Datalog] added query graph to explain result map.

0.9.4 (2024-04-02)

Added

• [Datalog] explain function to show query plan.
• [Platform] Embedded library for Linux on Aarch64, which is crossed compiled using zig.

Fixed

• [KV] Broken embedded library on Windows.

Improved

• [Datalog] more robust concurrent writes on server.
• [KV] Flags are now sets instead of vectors.
• Update deps

0.9.3 (2024-03-13)

DB Upgrade is required.

Added

• [Datalog] Query optimizer to improve query performance, particularly for complex queries. See details. #11
• [Datalog] More space efficient storage format, leveraging LMDB's dupsort feature, resulting in about 20% space reduction and faster counting of data entries.
• [Datalog] search-datoms function to lookup datoms without having to specify an index.
• [KV] Expose LMDB dupsort feature, i.e. B+ trees of B+ trees, #181, as the following functions that work only for dbi opened with open-list-dbi:
  • put-list-items
  • del-list-items
  • visit-list
  • get-list
  • list-count
  • key-range-list-count
  • in-list?
  • list-range
  • list-range-count
  • list-range-filter
  • list-range-first
  • list-range-some
  • list-range-keep
  • list-range-filter-count
  • visit-list-range
  • operate-list-val-range
• [KV] key-range function that returns a range of keys only.
• [KV] key-range-count function that returns the number of keys in a range.
• [KV] visit-key-range function that visit keys in a range for side effects.
• [KV] range-some function that is similar to some for a given range.
• [KV] range-keep function that is similar to keep for a given range.

Changed

• [Datalog] Change VEA index back to VAE.
• [Datalog] :eavt, :avet and :vaet are no longer accepted as index names, use :eav, :ave and :vae instead. Otherwise, it's misleading, as we don't store tx id.
• [KV] Change default write setting from :mapasync to :nometasync, so that the database is more crash resilient. In case of system crash, only the last transaction might be lost, but the database will not be corrupted. #228
• [KV] Upgrade LMDB to the latest version, now tracking mdb.master branch, as it includes important fixes for dupsort, such as https://bugs.openldap.org/show_bug.cgi?id=9723
• [KV] datalevin/kv-info dbi to keep meta information about the databases, as well as information about each dbi, as flags, key-size, etc. #184
• [KV] Functions that take a predicate have a new argument raw-pred? to indicate whether the predicate takes a raw KV object (default), or a pair of decoded values of k and v (more convenient).

Improved

• [Datalog] Query results is now spillable to disk. #166
• [Search] Functions in search-utils namespace are now compiled instead of being interpreted to improve performance.

0.8.29 (2024-02-23)

Improved

• Support older Clojure version.
• [Server] Recover options after automatic reconnect. #241

0.8.28 (2024-02-23)

Fixed

• [Datalog] Concurrent writes of large data values.

Added

• [Datalog] :closed-schema? option to allow declared attributes only, default is false. [Thx @andersmurphy]

Improved

• [Datalog] ported applicable improvements from Datascript up to 1.6.3

0.8.26 (2024-02-09)

Fixed

• [Datalog] :validate-data? true not working for some data types. [Thx @andersmurphy]
• [Datalog] ported applicable fixes from Datascript up to 1.6.1

Improved

• bump deps

0.8.25 (2023-12-14)

Added

• [Datalog] Add :db.fulltext/autoDomain boolean property to attribute schema, default is false. When true, a search domain specific for this attribute will be created, with a domain name same as attribute name, e.g. "my/attribute". This enables the same fulltext function syntax as Datomic, i.e. (fulltext $ :my/attribute ?search).
• [Search] Add :search-opts option to new-search-engine option argument, specifying default options passed to search function.

0.8.24 (2023-12-12)

Added

• [Datalog] Add :db.fulltext/domains property to attribute schema, #176
• [Datalog] Add :search-domains to connection option map, a map from domain names to search engine option maps.
• [Datalog] Add :domains option to fulltext built-in function option map

Fixed

• [Datalog] Removed problematic caching in pull api implementation

Improved

• [Datalog] Create search engines on-demand. #206

0.8.23 (2023-12-06)

Fixed

• deps conflict

0.8.22 (2023-12-06)

Improved

• [Datalog] <, >, <=, >= built-in functions handle any comparable data, not just numbers.
• [Datalog] Better fix for #224 [Thx @dvingo]
• bump deps

0.8.21 (2023-10-31)

Fixed

• [All] Do not interfere with the default print-methods of regular expression, byte array and big integer. #230

0.8.20 (2023-10-02)

Fixed

• [Datalog] :xform in pull expression not called for :cardinality/one ref attributes, #224. [Thx @dvingo]
• [Datalog] :validate-data? does not recognize homogeneous tuple data type, #227.
• [All] BigDec decoding out of range error for some values in JVM 8. #225.

Improved

• [KV] Add JVM shutdown hook to close DB. per #228
• [Datalog] TxReport prints differently from the actual value, #223.
• [Server] re-open server search engine automatically, #229

0.8.19 (2023-08-16)

Improved

• [Datalog] Handle refs in heterogeneous and homogeneous tuples, #218. [Thx @garret-hopper]
• [All] Remove some clojure.core redefinition warnings. [Thx @vxe]

0.8.18 (2023-07-02)

Improved

• [Test] Fix windows tests.

0.8.17 (2023-07-02)

Added

• [main] Added an --nippy option to dump/load database in nippy binary format, which handles some data anomalies, e.g. keywords with space in them, non-printable data, etc., and produces smaller dump file, #216

Improved

• [KV] More robust bigdec data type encoding on more platforms
• [All] Create a backup db directory dtlv-re-index-<unix-timestamp> inside the system temp directory when re-index, #213
• [Search] Graceful avoidance of proximity scoring when positions are not indexed

0.8.16 (2023-05-10)

Improved

• Remove Clojure 1.11 features to accommodate older Clojure

0.8.15 (2023-05-08)

Added

• [Search] Consider term proximity in relevance when :index-position? search engine option is true. #203
• [Search] :proximity-expansion search option (default 2) can be used to adjust the search quality vs. time trade-off: the bigger the number, the higher is the quality, but the longer is the search time.
• [Search] :proximity-max-dist search option (default 45) can be used to control the maximal distance between terms that would still be considered as belonging to the same span.
• [Search] create-stemming-token-filter function to create stemmers, which uses Snowball stemming library that supports many languages. #209
• [Search] create-stop-words-token-filter function to take a customized stop words predicate.
• [KV, Datalog, Search] re-index function that dump and load data with new settings. Should only be called when no other threads or programs are accessing the database. #179

Fixed

• [KV] More strict type check for transaction data, throw when transacting un-thawable data. #208

Changed

• [Main] Remove *datalevin-data-readers* dynamic var, use Clojure's *data-readers* instead.

0.8.14 (2023-04-28)

Fixed

• [Native] Rollback GraalVM to 22.3.1, as 22.3.2 is missing apple silicon.

0.8.13 (2023-04-28)

Fixed

• [Datalog] Unexpected heap growth due to caching error. #204
• [Datalog] More cases of map size reached errors during transaction. #196
• [Datalog] Existing datoms still appear in :tx-data when unchanged. #207

Improved

• [Datalog] Disable cache during transaction, save memory and avoid disrupting concurrent write processes.
• [Native] upgrade GraalVM to 22.3.2
• [Lib] update deps.

0.8.12 (2023-04-03)

Fixed

• [KV] When open-kv, don't grow :mapsize when it is the same as the current size.
• [Server] automatically reopen DBs for a client that is previously removed from the server.

0.8.11 (2023-04-02)

Added

• [Search] :include-text? option to store original text. #178.
• [Search] :texts and :texts+offsets keys to :display option of search function, to return original text in search results.

Improved

• [Main] more robust dump and load of Datalog DB on Windows.

0.8.10 (2023-04-01)

Added

• [KV] :max-readers option to specify the maximal number of concurrent readers allowed for the db file. Default is 126.
• [KV] max-dbs option to specify the maximal number of sub-databases (DBI) allowed for the db file. Default is 128. It may induce slowness if too big a number of DBIs are created, as a linear scan is used to look up a DBI.

Fixed

• [Datalog] clear after db is resized.

0.8.9 (2023-03-28)

Fixed

• [KV] transacting data more than one order of magnitude larger than the initial map size in one transaction. #196

0.8.8 (2023-03-21)

Improved

• [Pod] serialize TxReport to regular map. #190

Fixed

• [Server] migrate old sessions that do not have :last-active.

0.8.7 (2023-03-21)

Added

• [Datalog] datalog-index-cache-limit function to get/set the limit of Datalog index cache. Helpful to disable cache when bulk transacting data. #195
• [Server] :idle-timeout option when creating the server, in ms, default is 24 hours. #122

Fixed

• [Datalog] error when Clojure collections are used as lookup refs. #194

0.8.6 (2023-03-10)

Fixed

• [Datalog] correctly handle retracting then transacting the same datom in the same transaction. #192
• [Datalog] error deleting entities that were previously transacted as part of some EDN data. #191.

Improved

• [Lib] update deps.

0.8.5 (2023-02-13)

Added

• [KV] added tuple data type that accepts a vector of scalar values. This supports range queries, i.e. having expected ordering by first element, then second element, and so on. This is useful, for example, as path keys for indexing content inside documents. When used in keys, the same 511 bytes limitation applies.
• [Datalog] added heterogeneous tuple :db/tupleTypes and homogeneous tuples :db/tupleType type. Unlike Datomic, the number of elements in a tuple are not limited to 8, as long as they fit inside a 496 bytes buffer. In addition, instead of using nil to indicate minimal value like in Datomic, one can use :db.value/sysMin or :db.value/sysMax to indicate minimal or maximal values, useful for range queries. #167
• [Main] dynamic var *datalevin-data-readers* to support loading custom tag literals. (thx @respatialized)

Fixed

• [Main] dump and load big integers.

Improved

• [Datalog] avoid unnecessary caching, improve transaction speed up to 25% for large transactions.
• [Native] upgrade Graalvm to 22.3.1
• [Native] static build on Linux. #185
• [Lib] update deps.

0.8.4 (2023-01-20)

Fixed

• [Datalog] error when large :db/fulltext value is added then removed in the same transaction.

0.8.2 (2023-01-19)

Fixed

• [Search] search-utils/create-ngram-token-filter now works. #164
• [Datalog] large datom value may throw off full-text indexing. #177

0.8.1 (2023-01-19)

Fixed

• [Datalog] intermittent :db/fulltext values transaction error. #177

0.8.0 (2023-01-19)

DB Upgrade is required.

Changed

• [Search] Breaking search index storage format change. Data re-indexing is necessary.

Improved

• [Search] significant indexing speed and space usage improvement: for default setting, 5X faster bulk load speed; 2 orders of magnitude faster remove-doc and 10X disk space reduction; when term positions and offsets are indexed: 3X faster bulk load and 40 percent space reduction.
• [Search] added caching for term and document index access, resulting in 5 percent query speed improvement on average, 35 percent improvement at median.

Added

• [Search] :index-position? option to indicate whether to record term positions inside documents, default false.
• [Search] :check-exist? argument to add-docindicate whether to check the existence of the document in the index, default true. Set it to false when importing data to improve ingestion speed.

Fixed

• [Datalog] increasing indexing time problem for :db/fulltext values. #151
• [Search] error when indexing huge documents.
• [KV] spillable results exception in certain cases.

Removed

• [Search] doc-refs function.
• [Search] search-index-writer as well as related write and commitfunctions for client/server, as it makes little sense to bulk load documents across network.

0.7.12 (2023-01-11)

Fixed

• [Native] allow native compilation on apple silicon
• [Datalog] db print-method. (thx @den1k)

0.7.11 (2023-01-09)

Fixed

• [Datalog] intermittent concurrent transaction problems

0.7.10 (2023-01-09)

Fixed

• [CI] adjust CI workflow for the latest Graalvm

0.7.9 (2023-01-09)

Improved

• [Datalog] moved entity and transaction ids from 32 bits to 64 bits integers, supporting much larger DB. #144
• [Datalog] wrapped transact! inside with-transaction to ensure ACID and improved performance
• [Native] updated to the latest Graalvm 22.3.0. #174

0.7.8 (2023-01-04)

Fixed

• [KV] get-range regression when results are used in sequence. #172

Improved

• [Datalog] Ported all applicable Datascript improvements since 0.8.13 up to now (1.4.0). Notably, added composite tuples feature, new pull implementation, many bug fixes and performance improvements. #3, #57, #168
• bump deps

0.7.6 (2022-12-16)

Fixed

• [Server] error when granting permission to a db created by create-database instead of being created by opening a connection URI

Improved

• [Datalog] avoid printing all datoms when print a db
• [Doc] clarify that db-name is unique on the server. (thx @dvingo)

0.7.5 (2022-12-16)

Improved

• avoid (random-uuid), since not every one is on Clojure 1.11 yet.

0.7.4 (2022-12-15)

Fixed

• typo prevent build on CI

0.7.3 (2022-12-15)

Fixed

• [KV] spill test that prevents tests on MacOS CI from succeeding.

0.7.2 (2022-12-15)

Fixed

• [KV] broken deleteOnExit for temporary files

0.7.1 (2022-12-15)

Improved

• [KV] clean up spill files

0.7.0

DB Upgrade is required.

Added

• [Platform] embedded library support for Apple Silicon.
• [KV] A new range function range-seq that has similar signature as get-range, but returns a Seqable, which lazily reads data items into memory in batches (controlled by :batch-size option). It should be used inside with-open for proper cleanup. #108
• [KV] The existent eager range functions, get-range and range-filter, now automatically spill to disk when memory pressure is high. The results, though mutable, still implement IPersistentVector, so there is no API level change. The spill-to-disk behavior is controlled by spill-opts option map when opening the db, allowing :spill-threshold and :spill-root options.

Improved

• [KV] write performance improvement

Changed

• [KV] Upgrade LMDB to 0.9.29

0.6.29

Added

• [Client] :client-opts option map that is passed to the client when opening remote databases.

Fixed

• [KV] with-transaction-kv does not drop prior data when DB is resizing.
• [Datalog] with-transaction does not drop prior data when DB is resizing.

0.6.28

Improved

• [Native] Add github action runner image ubuntu-20.04 to avoid using too new a glibc version (2.32) that does not exist on most people's machines.

0.6.27

Fixed

• [KV] with-transaction-kv does not crash when DB is resizing.

0.6.26

Added

• [KV] with-transaction-kv macro to expose explicit transactions for KV database. This allows arbitrary code within a transaction to achieve atomicity, e.g. to implement compare-and-swap semantics, etc, #110
• [Datalog] with-transaction macro, the same as the above for Datalog database
• [KV]abort-transact-kv function to rollback writes from within an explicit KV transaction.
• [Datalog] abort-transact function, same for Datalog transaction.
• [Pod] Missing visit function

Improved

• [Server] Smaller memory footprint
• bump deps

0.6.22

Added

• [Datalog] fulltext-datoms function that return datoms found by full text search query, #157

Fixed

• [Search] Don't throw for blank search query, return nil instead, #158
• [Datalog] Correctly handle transacting empty string as a full text value, #159

Improved

• Datalevin is now usable in deps.edn, #98 (thx @ieugen)

0.6.21

Fixed

• [Datalog] Caching issue introduced in 0.6.20 (thx @cgrand)

0.6.20

Added

• [Pod] entity and touch function to babashka pod, these return regular maps, as the Entity type does not exist in a babashka script. #148 (thx @ngrunwald)
• [Datalog] :timeout option to terminate on deadline for query/pull. #150 (thx @cgrand).

0.6.19

Changed

• [Datalog] Entity equality requires DB identity in order to better support reactive applications, #146 (thx @den1k)

Improved

• bump deps

0.6.18

Fixed

• [Search] corner case of search in document collection containing only one term, #143
• [Datalog] entity IDs has smaller than expected range, now they cover full 32 bit integer range, #140

Added

• [Datalog] Persistent max-tx, #142

0.6.17

Added

• [Datalog] tx-data->simulated-report to obtain a transaction report without actually persisting the changes. (thx @TheExGenesis)
• [KV] Support :bigint and :bigdec data types, corresponding to java.math.BigInteger and java.math.BigDecimal, respectively.
• [Datalog] Support :db.type/bigdec and :db.type/bigint, correspondingly, #138.

Improved

• Better documentation so that cljdoc can build successfully. (thx @lread)

0.6.16

Added

• [Datalog] Additional arity to update-schema to allow renaming attributes. #131
• [Search] clear-docs function to wipe out search index, as it might be faster to rebuild search index than updating individual documents sometimes. #132
• datalevin.constants/*data-serializable-classes* dynamic var, which can be used for binding if additional Java classes are to be serialized as part of the default :data data type. #134

Improved

• [Datalog] Allow passing option map as :kv-opts to underlying KV store when create-conn
• bump deps

Fixed

• [Datalog] clear function on server. #133

Changed

• [Datalog] Changed :search-engine option map key to :search-opts for consistency [Breaking]

0.6.15

Improved

• [Search] Handle empty documents
• [Datalog] Handle safe schema migration, #1, #128
• bump deps

Added

• [KV] visitor function for visit can return a special value :datalevin/terminate-visit to stop the visit.

0.6.14

Fixed

• Fixed adding created-at schema item for upgrading Datalog DB from prior 0.6.4 (thx @jdf-id-au)

Changed

• [breaking] Simplified open-dbi signature to take an option map instead

Added

• :validate-data? option for open-dbi, create-conn etc., #121

0.6.13

Fixed

• Schema update regression. #124

Added

• :domain option to new-search-engine, so multiple search engines can coexist in the same dir, each with its own domain, a string. #112

0.6.12

Fixed

• Server failure to update max-eid regression, #123

Added

• Added an arity to update-schema to allow removal of attributes if they are not associated with any datoms, #99

0.6.11

Fixed

• Search add-doc error when alter existing docs

0.6.10

Improved

• Persistent server session that survives restarts without affecting clients, #119
• More robust server error handling

0.6.9

Fixed

• Query cache memory leak, #118 (thx @panterarocks49)
• Entity retraction not removing :db/updated-at datom, #113

Added

• datalevin.search-utils namespace with some utility functions to customize search, #105 (thx @ngrunwald)

0.6.8

Fixed

• Add visit KV function to core name space
• Handle concurrent add-doc for the same doc ref

Improved

• Bump deps

0.6.7

Fixed

• Handle Datalog float data type, #88

Improved

• Allow to use all classes in Babashka pods

0.6.6

Improved

• Dump and load Datalog option map
• Dump and load inter-fn
• Dump and load regex
• Pass search engine option map to Datalog store

0.6.5

Fixed

• Make configurable analyzer available to client/server

0.6.4

Fixed

• Dot form Java interop regression in query, #103

Added

• Option to pass an analyzer to search engine, #102
• :auto-entity-time? Datalog DB creation option, so entities can optionally have :db/created-at and :db/updated-at values added and maintained automatically by the system during transaction, #86

Improved

• Dependency bump

0.6.3

Added

• doc-count function returns the number of documents in the search index
• doc-refs function returns a seq of doc-ref in the search index

Improved

• datalevin.core/copy function can copy Datalog database directly.

0.6.1

Added

• doc-indexed? function

Improved

• add-doc can update existing doc
• open-kv function allows LMDB flags, #100

0.6.0

DB Upgrade is required.

Added

• Built-in full-text search engine, #27
• Key-value database visit function to do arbitrary things upon seeing a value in a range

Fixed

• [breaking]:instant handles dates before 1970 correctly, #94. The storage format of :instant type has been changed. For existing Datalog DB containing :db.type/instant, dumping as a Datalog DB using the old version of dtlv, then loading the data is required; For existing key-value DB containing :instant type, specify :instant-pre-06 instead to read the data back in, then write them out as :instant to upgrade to the current format.

Improved

• Improve read performance by adding a cursor pool and switch to a more lightweight transaction pool
• Dependency bump

0.5.31

Fixed

• Create pod client side defpodfn so it works in non-JVM.

Added

• load-edn for dtlv, useful for e.g. loading schema from a file, #101

0.5.30

Fixed

• Serialized writes for concurrent transactions, #83

Added

• defpodfn macro to define a query function that can be used in babashka pod, #85

0.5.29

Fixed

• Update max-aid after schema update (thx @den1k)

0.5.28

Improved

• Updated dependencies, particularly fixed sci version (thx @borkdude)

0.5.27

Fixed

• occasional LMDB crash during multiple threads write or abrupt exit

0.5.26

Improved

• Update graalvm version

Fixed

• Exception handling in copy

0.5.24

Fixed

• Handle scalar result in remote queries

Improved

• Server asks client to reconnect if the server is restarted and client reconnects automatically when doing next request

0.5.23

Improved

• Bump versions of all dependency

0.5.22

Improved

• More robust handling of abrupt network disconnections
• Automatically maintain the required number of open connections, #68

Added

• Options to specify the number of connections in the client connection pool and to set the time-out for server requests

0.4.44

Fixed

• Backport the dump/load fix from 0.5.20

0.5.20

Fixed

• Dumping/loading Datalog store handles raw bytes correctly

0.5.19

Improved

• Remove client immediately when disconnect message is received, clean up resources afterwards, so a logically correct number of clients can be obtained in the next API call on slow machines.

0.5.18

Fixed

• Occasional server message write corruptions in busy network traffic on Linux.

0.5.17

Added

• JVM uberjar release for download

Changed

• JVM library is now Java 8 compatible, #69

Improved

• Auto switch to local transaction preparation if something is wrong with remote preparation (e.g. problem with serialization)

0.5.16

Improved

• Do most of transaction data preparation remotely to reduce traffic

Fixed

• Handle entity serialization, fix #66

0.5.15

Changed

• Allow a single client to have multiple open databases at the same time
• Client does not open db implicitly, user needs to open db explicitly

Fixed

• New create-conn should override the old, fix #65

0.5.14

Added

• DTLV_LIB_EXTRACT_DIR environment variable to allow customization of native libraries extraction location.

Improved

• Use clj-easy/graal-build-time, in anticipation of GraalVM 22.

0.5.13

Improved

• Better robust jar layout for org.clojars.huahaiy/datalevin-native

0.5.10

Added

• Release artifact org.clojars.huahaiy/datalevin-native on clojars, for depending on Datalevin while compiling GraalVM native image. User no longer needs to manually compile Datalevin C libraries.

0.5.9

Improved

• Only check to refersh db cache at user facing namespaces, so internal db calls work with a consistent db view
• Replace unnecessary expensive calls such as db/-search or db/-datoms with cheaper calls to improve remote store access speed.
• documentation

0.5.8

Improved

• More robust build

0.5.5

Improved

• Wrap all LMDB flags as keywords

0.5.4

Fixed

• Don't do AOT in library, to avoid deps error due to exclusion of graal

Improved

• Expose all LMDB flags in JVM version of kv store

0.5.3

DB Upgrade is required.

Added

• Transparent networked client/server mode with role based access control. #46 and #61
• dtlv exec takes input from stdin when no argument is given.

Improved

• When open db, throw exception when lacking proper file permission

0.4.40

Added

• Transactable entity [Thanks @den1k, #48]
• clear function to clear Datalog db

0.4.35

Fixed

• Native uses the same version of LMDB as JVM, #58

0.4.32

Improved

• Remove GraalVM and dtlv specific deps from JVM library jar
• Update deps

0.4.31

Improved

• More robust dependency management

Fixed

• Replacing giant values, this requires Java 11 [#56]

0.4.30

Fixed

• Transaction of multiple instances of bytes [#52, Thanks @den1k]
• More reflection config in dtlv
• Benchmark deps

0.4.29

Fixed

• Correct handling of rule clauses in dtlv
• Documentation clarification that we do not support "db as a value"

0.4.28

Added

• Datafy/nav for entity [Thanks @den1k]
• Some datom convenience functions, e.g. datom-eav, datom-e, etc.

Changed

• Talk to Babashka pods client in transit+json

0.4.27

Added

• Exposed more functions to Babashka pod

0.4.26

Added

• Native Datalevin can now work as a Babashka pod

0.4.23

Added

• Compile to native on Windows and handle Windows path correctly
• close-db convenience function to close a Datalog db

0.4.21

Changed

• Compile to Java 8 bytecode instead of 11 to have wider compatibility
• Use UTF-8 throughout for character encoding

0.4.20

Improved

• Improve dtlv REPL (doc f) display

0.4.19

Improved

• Provide Datalevin C source as a zip to help compiling native Datalevin dependency

0.4.17

Improved

• Minor improvement on the command line tool

0.4.16

Changed

• Native image now bundles LMDB

0.4.13

Fixed

• Handle list form in query properly in command line shell [#42]

0.4.4

Changed

• Consolidated all user facing functions to datalevin.core, so users don't have to understand and require different namespaces in order to use all features.

0.4.0

DB Upgrade is required.

Changed

• [Breaking] Removed AEV index, as it is not used in query. This reduces storage and improves write speed.
• [Breaking] Change VAE index to VEA, in preparation for new query engine. Now all indices have the same order, just rotated, so merge join is more likely.
• [Breaking] Change open-lmdb and close-lmdb to open-kv and close-kv, lmdb/transact to lmdb/transact-kv, so they are consistent, easier to remember, and distinct from functions in datalevin.core.

Added

• GraalVM native image specific LMDB wrapper. This wrapper allocates buffer memory in C and uses our own C comparator instead of doing these work in Java, so it is faster.
• Native command line shell, dtlv

0.3.17

Changed

• Improve Java interop call performance

0.3.16

Changed

• Allow Java interop calls in where clauses, e.g. [(.getTime ?date) ?timestamp], [(.after ?date1 ?date2)], where the date variables are :db.type/instance. [#32]

0.3.15

Changed

• Changed default LMDB write behavior to use writable memory map and asynchronous msync, significantly improved write speed for small transactions (240X improvement for writing one datom at a time).

0.3.14

Fixed

• Read :db.type/instant value as java.util.Date, not as long [#30]

0.3.13

Fixed

• Fixed error when transacting different data types for an untyped attribute [#28, thx @den1k]

0.3.12

Fixed

• proper exception handling in lmdb/open-lmdb

0.3.11

Fixed

• Fixed schema update when reloading data from disk

0.3.10

Fixed

• Fixed core/get-conn schema update

0.3.9

Changed

• Remove unnecessary locks in read transaction
• Improved error message and documentation for managing LMDB connection

Added

• core/get-conn and core/with-conn

0.3.8

Fixed

• Correctly handle init-max-eid for large values as well.

0.3.7

Fixed

• Fixed regression introduced by 0.3.6, where :ignore value was not considered [#25]

0.3.6

Fixed

• Add headers to key-value store keys, so that range queries work in mixed data tables

0.3.5

Changed

• Expose all data types to key-value store API [#24]

0.3.4

Fixed

• thaw error for large values of :data type. [#23]

Changed

• portable temporary directory. [#20, thx @joinr]

0.3.3

Fixed

• Properly initialize max-eid in core/empty-db

0.3.2

Changed

• Add value type for :db/ident in implicit schema

0.3.1

Changed

• [Breaking] Change argument order of core/create-conn, db/empty-db etc., and put dir in front, since it is more likely to be specified than schema in real use, so users don't have to put nil for schema.

0.2.19

Fixed

• correct core/update-schema

0.2.18

Fixed

• correctly handle false value as :data
• always clear buffer before putting data in

0.2.17

Fixed

• thaw exception when fetching large values

0.2.16

Changed

• clearer error messages for byte buffer overflow

0.2.15

Fixed

• correct schema update

0.2.14

Added

• core/schema and core/update-schema

0.2.13

Added

• core/closed?

0.2.12

Fixed

• db/entid allows 0 as eid

0.2.11

Fixed

• fix test

0.2.10

Fixed

• correct results when there are more than 8 clauses
• correct query result size

0.2.9

Changed

• automatically re-order simple where clauses according to the sizes of result sets
• change system dbi names to avoid potential collisions

Fixed

• miss function keywords in cache keys

0.2.8

Added

• hash-join optimization [submitted PR #362 to Datascript](https://github.com/tonsky/datascript/pull/362)
• caching DB query results, significant query speed improvement

0.2.7

Fixed

• fix invalid reuse of reader locktable slot #7

Changed

• remove MDB_NOTLS flag to gain significant small writes speed

0.2.6

Fixed

• update existing schema instead of creating new ones

0.2.5

Fixed

• Reset transaction after getting entries
• Only use 24 reader slots

0.2.4

Fixed

• avoid locking primitive #5
• create all parent directories if necessary

0.2.3

Fixed

• long out of range error during native compile

0.2.2

Changed

• apply query/join-tuples optimization
• use array get wherenever we can in query, saw significant improvement in some queries.
• use db/-first instead of (first (db/-datom ..)), db/-populated? instead of (not-empty (db/-datoms ..), as they do not realize the results hence faster.
• storage test improvements

0.2.1

Changed

• use only half of the reader slots, so other processes may read

Added

• add an arity for bits/read-buffer and bits/put-buffer
• add lmdb/closed?, lmdb/clear-dbi, and lmdb/drop-dbi

0.2.0

Added

• code samples
• API doc
• core/close

0.1.0

Added

• Port datascript 0.18.13