Changes for Datasplash

v0.7.8 - 2022-05-16

• More complete tests, again.
• Add ability to specify a teardown function for a DoFn
• Invoke start-bundle, finish-bundle and teardown-fn when specified
• Reuse no-op function in dofn
• switch from cheshire to com.cnuernber/charred.

v0.7.7 - 2022-05-13

• Enforce target jdk as 1.8
• Bump clojure to 1.11.1
• more complete tests

v0.7.6 - 2022-04-23

• don't require bq, pubsub, datastore in api.clj - Issue #94
• Bump beam to 2.38.0

v0.7.5 - 2022-04-09

• replace cheshire usage with jsonista by @domparry
• Bump beam to 2.37.0; bump other deps

v0.7.4 - 2021-12-15

• Datastore fix by @domparry for breaking change introduced by nippy bump in 0.7.3
• Bump beam dep (2.34.0)
• Bump tools.logging (1.2.2)
• Clean up

v0.7.3 - 2021-08-14

• Resolve various java compiler warnings
• Bump dependencies including beam (2.31.0)
• Fix probems with Id keys in Google Datastore storage
• resolve various java compiler warnings

v0.7.2 - 2021-01-07

• fix locking problem with requires in safe-exec => Issue #97
• Update Apache Beam to 2.26.0

v0.7.1 - 2020-10-06

• Update Apache Beam to 2.24.0

v0.7.0 - 2020-09-07

• Add Kafka IO support
• Add delete-datastore-raw to datastore features
• add :schema-update-options for BQ write
• Update Apache Beam to 2.22.0
• Performance improvements
• Fixed require bug in partition-by

v0.6.6 - 2019-09-11

Fixes

• Fix cogroup-by for numbers of pcoll > 10 => Issue #83
• Fix :many-files option in read-text-file => Issue #75

v0.6.5 - 2019-08-23

• Add FileIO Read to read a PCollection of files
• Add ElasticSearchIO
• Upgrade to Apache Beam 2.15.0
• Upgrade dependencies

v0.6.4 - 2019-01-16

• Upgrade to Apache Beam 2.9.0
• Upgrade dependencies

v0.6.3 - 2018-11-19

• Add json-schema in bq write
• Combine-fn accepts now map as arguments
• Remove deprecated write-by

v0.6.2 - 2018-11-15

• Upgrade to Apache Beam 2.8.0
• Support dynamic file writes
• Support :deflate and :zip compressions
• bq/->time-partitioning now accepts :field and :require-partition-filter
• Improve docs

v0.6.1 - 2018-03-20

Changes

• Upgrade to Dataflow 2.3.0
• Support time-partitioned Big Query tables (thanks to @neuromantik33)

Fixes

• Fix broken input from BigQuery
• Fix missing safe-exec from filename policy
• Fix broken serializing after group-by in some cases
• Fix rare (hopefully) bug introduced by being lazy. with the vals after a group-by. Now we are greedy.

v0.6.0 - 2018-02-13

Changes

• Compatibility with Dataflow 2.2.0
• Compatibility with Clojure 1.9 and specs (some code had to be ported to Java shims because proxy is not Serializable anymore in Clojure 1.9)

Fixes

• Removed superfluous logging, safe-exec call and type hints

v0.5.3 - 2017-12-18

Fixes

• Fix options on PipelineWithOptions to have it work again on Dataflow (oops, sorry!)

v0.5.2 - 2017-12-14

Changes

• Fixed defoptions again and made it useful for interop with Beam
• Added :checkpoint option, will be made more useful later
• Switched to new PubSub API
• Add support for custom filename-policy

Fixes

• Fixed a possible bug causing StackOverflows on combine

v0.5.1 - 2017-10-05

Changes

• Added wait-pipeline-result

Fixes

• Fix a bug with incorrect call to seq in combine
• Add a missing safe-exec in write-bq-table-raw

[v0.5.0] - 2017-09-18

Changes

• Update to v2.x Dataflow version (Apache Beam)
• Add new streaming features (custom timestamp in PubSub #30 (thanks to @RoIT), BigQuery streaming writes)
• Add compressed TextIO output #31 (thanks to @RoIT)
• Add FileNamePolicy compatibility for TextIO

[v0.4.1] - 2017-03-09

Changes

• Handle recursion in RECORD schema generation for Big Query #9 (thanks to @torbjornvatn)
• Add windows and trigger support #13 (thanks to @RoIT)
• Bump Dataflow SDK to 1.9.0
• Add support for standard SQL syntax for Big Query #11 (thanks to @torbjornvatn)
• Read data from pubsub topics #10 (thanks to @torbjornvatn)

v0.4.0 - 2016-11-12

Breaking Changes

• Update Datastore API to v1, add wrappers for creating Entities from and converting to Clojure maps. Some breakage in the datasplash.datastore namespace ensues.

Changes

• Add pubsub namespace #6 (Thanks to @MartinSahlen)
• Put slf4j deps in :dev profiles #8 (Thanks to @torbjornvatn)