clj-headlights.bigquery-io — logrhythm/clj-headlights 0.2.0

Common bigquery utilites for clojure dataflow jobs.

CreateDispositions clj

read-from-query clj

(read-from-query query)

Inputs: [query :- s/Str] Returns: PTransform

read-from-query-and-extract clj

(read-from-query-and-extract pipeline name query field)

Inputs: [pipeline :- pcollections/PCollectionType name :- s/Str query :- s/Str field :- s/Str] Returns: pcollections/PCollectionType

schema-field->bq-field clj

(schema-field->bq-field schema-field)

table-row-maker clj

(table-row-maker row-as-map schema)

to-tablerow clj

(to-tablerow str)

write clj

(write output pcoll)

Inputs: [output :- (s/cond-pre s/Str SerializableFunction) pcoll :- pcollections/PCollectionType] Returns: pcollections/PCollectionType

Write elements of a pcoll to bigquery in json format

write-with-schema clj

(write-with-schema pcoll name output schema write-options)

Inputs: [pcoll :- pcollections/PCollectionType name :- s/Str output :- (s/cond-pre s/Str SerializableFunction) schema :- [{:name s/Str, :type s/Str, (s/optional-key :mode) s/Str}] write-options :- #:s{Keyword s/Any}] Returns: pcollections/PCollectionType

Write elements of a pcoll to bigquery in json format

WriteDispostions clj