clj-headlights.pubsub — com.zendesk/clj-headlights 0.1.4

Helpers to use Pub/Sub in DataFlow jobs

pubsub-url->type clj

(pubsub-url->type url)

Inputs: [url :- s/Str] Returns: s/Str

Extract the type from a PubSub URL, e.g. topics in projects/my-project/topics/my-topic

read-stream clj

(read-stream url)

Inputs: [url :- (s/maybe s/Str)] Returns: (s/maybe PTransform)

Create a PTransform that reads either a Pub/Sub topic or subscription, based on the given URL.

read-stream-with-ts clj

(read-stream-with-ts url)

(read-stream-with-ts url attr-name)

Inputs: ([url :- (s/maybe s/Str)] [url :- (s/maybe s/Str) attr-name :- s/Str]) Returns: (s/maybe PTransform)

Create a PTransform that reads a Pub/Sub topic or subscription, based on the given URL. Uses .timestampLabel to assign timestamps to each message. Defaults to looking up the 'ts' attribute if no other name is provided.