delta-sharing.client — com.amperity/delta-sharing-client-clj 0.1.0

Main Delta Sharing client namespace. Contains functions for generic client operations, constructing new clients, and using clients as components in a larger system.

get-share clj

(get-share client share)

Returns information about a share that's accessible to the recipient.

list-schema-tables clj

(list-schema-tables client share schema)

(list-schema-tables client share schema opts)

Returns a list of tables in a schema.

Options:

• max-results (int) maximum results per page. If there are more available results, the response will return a :next-page-token that can be used to get the next page.

• next-token (string) a page token to use for the next page of results. See max-results.

list-share-schemas clj

(list-share-schemas client share)

(list-share-schemas client share opts)

Returns a list of all the shemas in a share.

Options:

• max-results (int) maximum results per page. If there are more available results, the response will return a :next-page-token that can be used to get the next page.

• next-token (string) a page token to use for the next page of results. See max-results.

list-share-tables clj

(list-share-tables client share)

(list-share-tables client share opts)

Returns a list of tables within all schemas of a share.

Options:

• max-results (int) maximum results per page. If there are more available results, the response will return a :next-page-token that can be used to get the next page.

• next-token (string) a page token to use for the next page of results. See max-results.

list-shares clj

(list-shares client)

(list-shares client opts)

Returns a list of all shares accessible to the recipient.

Options:

• max-results (int) maximum results per page. If there are more available results, the response will return a :next-page-token that can be used to get the next page.

• next-token (string) a page token to use for the next page of results. See max-results.

new-client clj

(new-client endpoint auth reader-version & {:as http-opts})

Contructs a new delta-sharing client from a URI address endpoint by dispatching on the scheme.

query-table-metadata clj

(query-table-metadata client share schema table)

(query-table-metadata client share schema table opts)

Returns a map of information about a table's metadata and protocol.

See https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#query-table-metadata for more.

Options:

• :min-reader-version (int - default 1) Minimum reader version to support. See https://github.com/delta-io/delta/blob/master/PROTOCOL.md#valid-feature-names-in-table-features for more. Supported reader versions include: 1 {:response-format 'parquet'} 2 {:response-format 'delta' :reader-features 'columnMapping'} 3 {:response-format 'delta' :reader-features 'columnMapping,deletionVectors,timestampNtz,v2Checkpoint,vacuumProtocolCheck'}
• :response-format (string - default 'parquet') The default table format to respond with. Overrides the one from min-reader-version when set.
• :reader-features (string - default none) Advanced table features on the requested table. Requires a response format in delta. Overrides the one from min-reader-version when set.

query-table-version clj

(query-table-version client share schema table)

(query-table-version client share schema table opts)

Returns information about the version of a table.

Optionally accepts a :starting-timestamp from which to return the earliest matching version for.

read-change-data-feed clj

(read-change-data-feed client share schema table)

(read-change-data-feed client share schema table opts)

Returns a map with information about the table (protocol and metadata) along with its change data files since starting-version or starting-timestamp inclusively. The top-level :metadata returned is the first version in the requested range.

One of either the version or timestamp parameters must be passed.

The change data feed represents row-level changes between versions of a Delta table. It records change data for UPDATE, DELETE, and MERGE operations. See https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#read-change-data-feed-from-a-table for more. Only valid for tables with delta.enableChangeDataFeed enabled.

NOTE: Synthetic table columns _change_type, _commit_version, and _commit_timestamp are not added to the response. They can be derived by passing the response to a reader that reads the change file and table actions from the version respectively.

Delta sharing capability options:

• :min-reader-version (int - default 1) Minimum reader version to support. See https://github.com/delta-io/delta/blob/master/PROTOCOL.md#valid-feature-names-in-table-features for more. Supported reader versions include: 1 {:response-format 'parquet'} 2 {:response-format 'delta' :reader-features 'columnMapping'} 3 {:response-format 'delta' :reader-features 'columnMapping,deletionVectors,timestampNtz,v2Checkpoint,vacuumProtocolCheck'}
• :response-format (string - default 'parquet') The default table format to respond with. Overrides the one from min-reader-version when set.
• :reader-features (string - default none) Advanced table features on the requested table. Requires a response format in delta. Overrides the one from min-reader-version when set.

Delta sharing parameters: [:starting-timestamp,:ending-timestamp,:starting-version,:ending-version,:include-historical-metadata?]

read-table-data clj

(read-table-data client share schema table)

(read-table-data client share schema table opts)

Returns a map with information about the latest version of the table including its :protocol, :metadata. Provides the downloadable data files under :files for the latest version (or the requested version when :version is set). Otherwise, a collection of change data files is returned under :change-data-actions along with any historical metadata.

See https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#read-data-from-a-table for more. And https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#request-body for more about the request parameters.

Delta sharing capability options:

• :min-reader-version (int - default 1) Minimum reader version to support. See https://github.com/delta-io/delta/blob/master/PROTOCOL.md#valid-feature-names-in-table-features for more. Supported reader versions include: 1 {:response-format 'parquet'} 2 {:response-format 'delta' :reader-features 'columnMapping'} 3 {:response-format 'delta' :reader-features 'columnMapping,deletionVectors,timestampNtz,v2Checkpoint,vacuumProtocolCheck'}
• :response-format (string - default 'parquet') The default table format to respond with. Overrides the one from min-reader-version when set.
• :reader-features (string - default none) Advanced table features on the requested table. Requires a response format in delta. Overrides the one from min-reader-version when set.

Supported delta sharing options: [:predicate-hints,:json-predicate-hints,:limit-hint,:version,:timestamp,:starting-version,:ending-version]