Optional Google Sheets Integration

Geni will automatically detect whether the following optional dependencies are present:

[com.google.api-client/google-api-client "1.30.9"]
[com.google.apis/google-api-services-drive "v3-rev197-1.25.0"]
[com.google.apis/google-api-services-sheets "v4-rev612-1.25.0"]
[com.google.oauth-client/google-oauth-client-jetty "1.30.6"]
[org.apache.hadoop/hadoop-client "2.7.3"]

If so, the vars g/read-sheets! and g/write-sheets! will be bound to the appropriate functions. Otherwise, the vars will remain unbound.

Note that the additional Hadoop dependency is required to prevent a Guava compatibility conflict between Spark and Google APIs. For a potential fix without downgrading Hadoop client, see here.

In order to read and write from Google Sheets, we need to ensure that we have the following:

1. A spreadsheet ID. The sheet name is optional, and it defaults to "Sheet1".
2. A project with Google Sheets enabled and a service account with a private key stored in JSON.
3. Permissions to read and write for the service account. An easy way to ensure the right permissions is to share an existing spreadsheet to the email of the service account.

With that in mind, the following snippets write and read from a particular sheet on a spreadsheet:

(let [google-props {:credentials    "{path-to-api-key}"
                    :spreadsheet-id "{gsheet-spreadsheet-id}"
                    :sheet-name     "{sheet-name}"}
      options      {:header false}]
  (g/read-sheets! spark google-props options)
  (g/write-sheets! dataframe google-props options))