add qbits.alia/component module This adds a https://github.com/stuartsierra/component for alia with single/multicluster capabilities and also a query registry that encapsulates query level information/state (prepared queries).
Docs will come for this, but if you feel adventurous you can already try it based on the information you'd find in the related PR:
https://github.com/mpenet/alia/pull/89#issue-287081644
Thanks @pyr for the work on this.
On a side not this is the last release with the alia-all package, in the near future there will be a re-do on the way we manage the modules and alia-all will go away. That simply means you will have to pick and choose what you need/want, which is a better default.
kill deprecated modules: eao.uuid, nippy
bump clojure, java-driver, core.async, test.check
make default seq result-set more lightweight: it's no longer lazy with a cell per row, it's now chunked (default clj behavior)
use clojure 1.9 alpha17
no longer depend on qbits/spex
catch Exception not Throwable
qbits.alia/execute now takes an optional :channel
, kill noop
close! on promise-chan
bump manifold dependency
codec: simplify handing out to map->Record constructors #73 - Thanks to @pyr for the fix
Remove all traces of blocking IO in
qbits.alia-manifold/execute-buffered
. It now pages manually as the
consumer takes values from the stream (as long as the stream is open).
Thanks to @mccraigmccraig this.
Remove all traces of blocking IO in execute-chan-buffered
,
it now pages manually as the consumer takes values from the chan.
You can also now interup streaming/paging by async/close!
ing the
channel returned by execute-chan-buffered
.
** Breaking changes **
The way the internal codec works has changed. Previously we had one
protocol, qbits.alia/PCodec
that included both decoding and
encoding function. Now "Codecs" are first class, they can be passed
a :codec
option to execute/bind functions and can have custom
behavior/state, as a result we are no longer bound to one global
codec registry and can allow multiple strategies.
A codec is now a map/record that contains 2 fields/keys, :decoder
:encoder
, that have functions as values. The
qbits.alia.codec/default
codec for instance, which is the port of
the old one, is a record implementing a protocol with the equivalent
of the previous strategies. You can see it here:
https://github.com/mpenet/alia/blob/feature/codec-alt/modules/alia/src/qbits/alia/codec/default.clj
It can also serve as a base to compose other codecs using the same functions, using clojure.core/extend for instance (more on this later).
So why doing this? This opens the door to new codecs with new behavior with potentially state. That way we can for instance implement specialised codecs that will know how to automaticaly decode/encode keyspace bound UDTs as record instance, or as a deftype, or whatever you'd like really. And mix and match these codecs in your app, in the same jvm without causing clashes or causing performance degradation because you are doing something fancy in your codec.
We will include by default one such custom implementation (other than the default), an UDT aware codec that does just what I described before: translating UDT values to record instance of chosen type at decoding time.
For most users who extended PCodec before, the changes should be minimal: You just need to extend the default codec protocols (with an S, we separated encoding decoding, to avoid creating "identity" functions for nothing when you just care about one or the other) with your functions/types
Here is an example with joda-time https://github.com/mpenet/alia/blob/feature/codec-alt/modules/alia-joda-time/src/qbits/alia/codec/joda_time.clj
Another nice thing this allows us to do is to create high
performance codecs that do the minimum {:encoder identity :decoder identity}
), or create specialized ones where everything is inlined
(ex in defrecord body) but closed.
In terms of performance the default codec is identical as the old one, since it's essentially the same code.
** Non Breaking changes **
** Hayt **
If you're using it, upgrade to the latest version, the performance improved significantly in the latest releases.
Add support for "custom row generators": If you don't care about having rows returned as keywordized maps you can skip what comes next.
All "execute" calls in alia's API can now take an additional option:
row-generator
We now have a qbits.alia.codec/RowGenerator
protocol that opens how
rows are decoded per query.
(defprotocol RowGenerator
(init-row [this] "Constructs a row base")
(conj-row [this r k v] "Adds a entry/col to a row")
(finalize-row [this r] "\"completes\" the row"))
You can this way generate rows as vectors, or records or deftype instances or anything you'd like really. It kind of follows the way transient work, it's a 3 operation definition ("setup" row bag, "add" col to row bag, "close" row bag).
By default there are 3 generators defined:
qbits.alia.codec/row-gen->map
: the defaultqbits.alia.codec/row-gen->vector
: builds a vector of valuesqbits.alia.codec/row-gen->record
: that one is a function that
takes a map->foo like constructor.And feel free to implement your own.
As a result the key-fn
option has been removed, but I don't think
anybody used it (at least in OSS). But that's easy to restore via a
custom generator if you need to (it's a copy of row-gen->map without
a keyword call):
(def row-gen->map
"Row Generator that stringy map instances"
(reify RowGenerator
(init-row [_] (transient {}))
(conj-row [_ row k v] (assoc! row k v))
(finalize-row [_ row] (persistent! row))))
alia
main module: https://github.com/mpenet/alia/pull/55
@blak3mill3r fixed ssl options handling in cluster builderqbits.alia.async
).Breaking changes (package/dependency level, not code)
Alia was split into separate libaries to allow more flexible builds, lighter dependencies, better aot'isability and also make it ok to add more exotic features without adding weight on the core (an upcoming Component, optional schemas for cluster options, direct linking, etc).
If you don't really care and want the whole package like before, you
can just change your dependencies to [cc.qbits/alia-all "3.1.0"]
.
If you were using execute-chan
or execute-chan-buffered
, you
will now find these 2 function in qbits.alia.async
Alia was split like this:
alia
: minimal core of the driver (whithout core.async/manifold/custom codec extensions)alia-async
: core.async interface (qbits.alia/execute-chan
, qbits.alia/execute-chan-buffered
)alia-manifold
: manifold async interaface (qbits.alia.manifold/*
same as before)alia-eaio-uuid
: eaio.uuid codec extensionalia-joda-time
: joda-time codec extensionalia-nippy
: the Nippy codec extensionThey are located in ./modules.
The version numbers will always match the core lib, alia
to avoid confusions.
A simple example, if you only want to use the core and the core.async extenstion you can just add these to your dependencies:
[cc.qbits/alia "3.1.2"] [cc.qbits/alia-async "3.1.2"]
Use java-driver 3.0.0 (final) driver-core/CHANGELOG
Simple statements can now take named placeholders (same as prepared):
(execute session "SELECT * FROM foo WHERE bar = :bar-value" {:values {:bar-value "something"}})`
read-timeout
via execute optionsBreaking change
Hayt
is not included by default anymore. You can include it
manually in your dependencies, then simply loading qbits.hayt
in
the namespace(s) where you use it with alia and will extend the
appropriate protocol and provide total backward compatibility. (I
had to do this to accomodate users which had dependency conflicts
with hayt (and were not using it)).
tldr: add [cc.qbits/hayt "3.0.0"]
in your deps
if/when you use it.
typo translater -> translator (from upstream)
remove hayt query caching, let this to the end user to manage if wanted
More infos here about using 3.x: https://github.com/datastax/java-driver/tree/3.0/upgrade_guide
Use latest core.async: as a result qbits.alia/execute-chan
now
returns a core.async/promise-chan.
Update core.memoize dependency
Respect nullability of row cells: previously a Boolean column would be False when no value was set, a Long would always default to 0 and so on. This differed from cqlsh and what's general expected and was a side effect of the java API available at the time the decoding code was written.
Handle custom decoding in deeply nested datastructures via PCodec protocol https://github.com/mpenet/alia/issues/45. It's on by default but it is possible to turn this off by extending PCodec protocol for Map, List, Set, TupleValue, UDTValue to just be "identity".
ResultSet decoding is now done via a protocol that implements both
seq() and IReduceInit(), the former would return an unchunked lazy seq
(it's the detault, same as previous versions, it's equivalent to
passing {:result-set-fn seq}
to execute), and the later would you to
get a reducible for instance if you pass #(into [] %)
as a
:result-set-fn
, which is eager and cheaper. IReduceInit
is 1.7+
only, hence the new requirement.
Via :result-set-fn
you can also reach ExecutionInfos from the rs
(which was previously always set as metadata to rs) by calling
qbits.codec/execution-info on the argument passed to your function,
from there you could for instance log the information returned.
Breaking: :string-keys?
is removed in favor of :key-fn
drop :exception-info metadata on result-set
prevent calls to ResultSet.one() when not needed (true fire and forget)
All enum taking fns will now reject invalid values and throw if something weird is supplied
row decoding is now truly lazy, skipping clojure chunking and respecting fetch-size more accurately (especially important for execute-chan-buffered).
invalid udt/tuple encoders now throw if supplied an invalid type name
Breaking changes
** New Features **
qbits.alia/tuple-encoder
and qbits.alia/udt-encoder
: add Tuple
and UDT encoder functions to be used with prepared statements. They
both return a function that can be used to encode tuple/udt of the
selected type. ex:
(let [user (qbits.alia/udt-encoder session :user)]
(execute session
user-insert-prepared-stmt
{:values [(user {:id "foo" :age 10})]}))
Internal encoding of values respects the main encoder
(qbits.codec.PCodec
), that means if you extended it for joda time
or your own types for this should work transparently even
if your UDT are deeply nested.
Add support to named placeholders in prepared statements (:value can take a map that will either match placeholders of column name values) - Thanks to @d-t-w
Add qbits.alia/batch
Use shadded Netty dependency (allows to use another netty version in your code if needed) - Thanks to @d-t-w
Remove deprecated Lamina interface
Improve decoding performance
More uniform exception handling in async modes (all exceptions are returned in chan/deferred/function depending on the context)
Use java-driver 2.1.6 driver-core/CHANGELOG
adds cluster options and related utils for: speculative execution, address-translater, cluster-name, netty-options, timestamp-generator, max-schema-agreement-wait-seconds
new execute* options for :paging-state
and idempotent?
Improve joda-time codec for java.util.Date decoding from data returned by cassandra
Adds experimental nippy codec with various modes:
If you need more fine grained control you are be better off calling thaw/freeze manually on a per column basis in your app.
improve resource use and performance in async mode by running the put! in the io thread (since it's async anyway), not using an external thread pool. You can still pass :executor to the execute-* call and override this behavior.
remove now useless qbits.knit dependency
qbits.alia/manifold
:kerberos?
option) & TLS by adding :ssl-options
map argument
support, it now accepts an SSLOption instance (as before) or a map
of :keystore-path
:keystore-password
:cipher-suites
.(execute session "select * from foo where bar = ?;" {:values ["baz"]})
(execute session (select :foo (where [[= :bar ?]])) {:values ["baz"]})
...
Use java-driver 2.0.3 driver-core/CHANGELOG
Use latest core.async
execute-chan-buffered
:
Allows to execute a query and have rows
returned in a clojure.core.async/chan
. Every value in the chan is
a single row. By default the query :fetch-size
inherits from the
cluster setting, unless you specify a different :fetch-size
at
query level and the channel is a regular clojure.core.async/chan
,
unless you pass your own :channel
with its own sizing caracteristics.
:fetch-size
dicts the "chunking" of the rows returned, allowing to
"stream" rows into the channel in a controlled manner.The name of this fn could be subject to change in future versions.
Breaking changes -> API cleanup, performance, better composability
set-*
, with-*
)execute*
functions, as well as prepare
, now require an
explicit session argument (it was optional before), options are now
a map instead of kwargscluster
now takes a map instead of hosts + kwargsThe next release will likely be 2.0 final, and the API will be stable from now on.
Add Statement.setFetchSize - Thanks @coltnz
Add serial consistency support
Use java-driver 2.0.0-rc1 driver-core/CHANGELOG
keyword cql-values are no longer encoded as strings (you have to manually handle this), since keywords will be used as named bind markers in the future.
Breaking changes: Use latest Hayt (2.x) https://github.com/mpenet/hayt/blob/master/CHANGELOG.md
Improved exception handling, query
execution/binding/preparation exception are now ExceptionInfo
instances that hold a map with the original statement and the Query
string used. You can get to this info from the ExceptionInfo
instance using clojure.core/ex-data
.
(try
(execute "slect prout from 1;")
(catch Exception ex
(println (ex-data ex))))
The map looks like this:
{:exception #<SyntaxError com.datastax.driver.core.exceptions.SyntaxError: line 1:0 no viable alternative at input 'slect'>
:query "slect prout from 1;"
:values nil
:statement #<SimpleStatement slect prout from 1;>}
:socket-options
and :defer-initialization
via
cluster
optionsqbits.alia/prepare
prepare
used to be able to convert the query argument when coming
from hayt using ->prepared
and only take the first value of the
returned vector. While this was useful, this was often hiding the
resulting query and limiting in some way (you don't necessarly want to
have every value prepared), forcing the user to compile it in the repl
first to have an idea of what it would look like. prepare
has been
changed so that it never compiles with ->prepare
but now does it
with ->raw
.
Meaning you can now do the following (prepare (select :foo (where {:bar ?})))
You can still use prepare
with queries generated with ->prepared
you just need to do it explicitly(def pq (->prepared (select :foo (where {:bar 1}))))
(prepare (first pq))
...
qbits.alia/connect
Use latest Hayt https://github.com/mpenet/hayt/blob/master/CHANGELOG.md
Make sure namespace keywords are properly encoded in prepared statements
qbits.alia/execute-chan
,
returning a channel usable with go blocks or clojure.core.async/take!
.Use java-driver 1.0.1 driver-core/CHANGELOG
Add SSL support
alia/prepare
bump hayt to 1.0.5 https://github.com/mpenet/hayt/blob/master/CHANGELOG.md#105
bump core.memoize to 0.5.5
core.memoize
to 0.5.4JMXReporting
toggling at the cluster levelqbits.tardis
is now optional
Added wrappers for retry/load-balancing/reconnection policies see qbits.alia.policy.*
Added qbits.alia/lazy-query
: lazy sequences over CQL queries
Depend on hayt 1.0.2 (bugfixes)
:auth-info
option on cluster
becomes :credentials
as SimpleAuthInfoProvider is gone and replaced with .withCredentials.Upgraded to java-driver rc1
Upgraded hayt to 0.5.0
Restored keywordize?
keywordize
option, useless featureUpdate java-driver to beta2, see driver-core/CHANGELOG
Metadata on results updated to follow java-driver's style, it now returns an
:execution-info
key that contains basic info about the query
(hosts queried etc) and allows to retrieve tracing info, in future
versions paging metadata probably, see
JAVA-65.
C* CUSTOM type support see JAVA-61
async-execute
now returns a lamina.core/result-channel
, the only
difference for end-users should be the behavior when an error occurs
and the query is dereferenced: it used to return the exception as a
value, now it throws it, callbacks are unchanged.
Added Lamina as a dependency.
BREAKING CHANGE: Column names are now returned as keywords by
default (way more convenient in real world use), you can change that
globally using set-keywordize!
or per query using the
:keywordize?
option on execute
/execute-async
calls.
Depend on clojure 1.5.1 by default.
keywordize?
options to execute
and execute-async
, make it
settable globally with set-keywordize!
.execute-async
: use distinct functions for sync/async mode
since they don't share return types and some optional
parameters. This means execute
no longer accepts the :async?
option, use execute-async
instead.fixed pooling options only allowing one distance setting, and also make it use hash-map as value
:core-connections-per-host {:remote 10 :local 100}
Can you improve this documentation? These fine people already did:
Max Penet, Andre R & Kirill KondratenkoEdit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close