zero-one.geni.core.column — zero.one/geni 0.4.0

% clj

(% left-expr right-expr)

Params: (other: Any)

Result: Column

Modulo (a.k.a. remainder) expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.822Z

& clj

(& left-expr right-expr)

Params: (other: Any)

Result: Column

Compute bitwise AND of this expression with another expression.

1.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.878Z

&& clj

(&& & exprs)

Params: (other: Any)

Result: Column

Boolean AND.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.824Z

* clj

(* & exprs)

Params: (other: Any)

Result: Column

Multiplication of this expression and another expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.827Z

+ clj

(+ & exprs)

Params: (other: Any)

Result: Column

Sum of this expression and another expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.829Z

- clj

(- & exprs)

Params: (other: Any)

Result: Column

Subtraction. Subtract the other expression from this expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.957Z

->col-array clj

(->col-array args)

Coerce a coll of coerceable values into a coll of columns.

->col-seq clj

(->col-seq arg)

Coerce a value into a coll of columns.

->column clj multimethod

Params: (colName: String)

Result: Column

Returns a Column based on the given column name.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/functions$.html

Timestamp: 2020-10-19T01:56:22.258Z

/ clj

(/ & exprs)

Params: (other: Any)

Result: Column

Division this expression by another expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.832Z

< clj

Params: (other: Any)

Result: Column

Less than.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.834Z

<= clj

Params: (other: Any)

Result: Column

Less than or equal to.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.836Z

<=> clj

Params: (other: Any)

Result: Column

Equality test that is safe for null values.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.838Z

=!= clj

Params: (other: Any)

Result: Column

Inequality test.

2.0.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.840Z

=== clj

Params: (other: Any)

Result: Column

Equality test.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.843Z

> clj

Params: (other: Any)

Result: Column

Greater than.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.845Z

>= clj

Params: (other: Any)

Result: Column

Greater than or equal to an expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.847Z

asc clj

(asc expr)

Params:

Result: Column

Returns a sort expression based on ascending order of the column.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.867Z

asc-nulls-first clj

(asc-nulls-first expr)

Params:

Result: Column

Returns a sort expression based on ascending order of the column, and null values return before non-null values.

2.1.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.869Z

asc-nulls-last clj

(asc-nulls-last expr)

Params:

Result: Column

Returns a sort expression based on ascending order of the column, and null values appear after non-null values.

2.1.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.870Z

between clj

(between expr lower-bound upper-bound)

Params: (lowerBound: Any, upperBound: Any)

Result: Column

True if the current column is between the lower bound and upper bound, inclusive.

1.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.872Z

bitwise-and clj

(bitwise-and left-expr right-expr)

Params: (other: Any)

Result: Column

Compute bitwise AND of this expression with another expression.

1.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.878Z

bitwise-or clj

(bitwise-or left-expr right-expr)

Params: (other: Any)

Result: Column

Compute bitwise OR of this expression with another expression.

1.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.879Z

bitwise-xor clj

(bitwise-xor left-expr right-expr)

Params: (other: Any)

Result: Column

Compute bitwise XOR of this expression with another expression.

1.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.881Z

cast clj

(cast expr new-type)

Params: (to: DataType)

Result: Column

Casts the column to a different data type.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.885Z

col clj multimethod

Params: (colName: String)

Result: Column

Returns a Column based on the given column name.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/functions$.html

Timestamp: 2020-10-19T01:56:22.258Z

contains clj

(contains expr literal)

Params: (other: Any)

Result: Column

Contains the other element. Returns a boolean column based on a string match.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.888Z

desc clj

(desc expr)

Params:

Result: Column

Returns a sort expression based on the descending order of the column.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.890Z

desc-nulls-first clj

(desc-nulls-first expr)

Params:

Result: Column

Returns a sort expression based on the descending order of the column, and null values appear before non-null values.

2.1.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.891Z

desc-nulls-last clj

(desc-nulls-last expr)

Params:

Result: Column

Returns a sort expression based on the descending order of the column, and null values appear after non-null values.

2.1.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.893Z

drop-fields clj

(drop-fields expr & field-names)

Returns the struct column without the fields field-names, each of which can be a path, such as :a.b, into nested structs.

(g/select dataframe {:address (g/drop-fields :address :unit :street)})

ends-with clj

(ends-with expr literal)

Params: (other: Column)

Result: Column

String ends with. Returns a boolean column based on a string match.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.898Z

eq-null-safe clj

Params: (other: Any)

Result: Column

Equality test that is safe for null values.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.901Z

equal-to clj

Params: (other: Any)

Result: Column

Equality test.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.903Z

geq clj

Params: (other: Any)

Result: Column

Greater than or equal to an expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.910Z

get-field clj

(get-field expr field-name)

Params: (fieldName: String)

Result: Column

An expression that gets a field by name in a StructType.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.913Z

get-item clj

(get-item expr k)

Params: (key: Any)

Result: Column

An expression that gets an item at position ordinal out of an array, or gets a value by key key in a MapType.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.915Z

gt clj

Params: (other: Any)

Result: Column

Greater than.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.916Z

hash-code clj

(hash-code expr)

Params: ()

Result: Int

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.918Z

ilike clj

(ilike expr literal)

SQL ILIKE: true where the string column matches the pattern literal, ignoring case. % matches any characters, and _ one.

(g/filter dataframe (g/ilike :suburb "%north%"))

is-in-collection clj

(is-in-collection expr coll)

Params: (values: Iterable[_])

Result: Column

A boolean expression that is evaluated to true if the value of this expression is contained by the provided collection.

Note: Since the type of the elements in the collection are inferred only during the run time, the elements will be "up-casted" to the most common type for comparison. For eg:

1. In the case of "Int vs String", the "Int" will be up-casted to "String" and the comparison will look like "String vs String".
2. In the case of "Float vs Double", the "Float" will be up-casted to "Double" and the comparison will look like "Double vs Double"

2.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.924Z

is-na-n clj

(is-na-n expr)

Params:

Result: Column

True if the current expression is NaN.

1.5.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.927Z

is-nan clj

(is-nan expr)

Params:

Result: Column

True if the current expression is NaN.

1.5.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.927Z

is-not-null clj

(is-not-null expr)

Params:

Result: Column

True if the current expression is NOT null.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.932Z

is-null clj

(is-null expr)

Params:

Result: Column

True if the current expression is null.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.933Z

isin clj

(isin expr coll)

Returns a boolean column that is true where the column's value is in coll, or in the one column of the Dataset coll, as an IN subquery, which needs Spark 4.1.

(g/filter sales (g/isin :region ["north" "south"]))
(g/filter sales (g/isin :customer-id (g/select vip :id)))

leq clj

Params: (other: Any)

Result: Column

Less than or equal to.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.938Z

like clj

(like expr literal)

Params: (literal: String)

Result: Column

SQL like expression. Returns a boolean column based on a SQL LIKE match.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.939Z

lit clj

(lit arg)

Params: (literal: Any)

Result: Column

Creates a Column of literal value.

The passed in object is returned directly if it is already a Column. If the object is a Scala Symbol, it is converted into a Column also. Otherwise, a new Column is created to represent the literal value.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/functions$.html

Timestamp: 2020-10-19T01:56:22.442Z

lt clj

Params: (other: Any)

Result: Column

Less than.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.956Z

minus clj

(minus & exprs)

Params: (other: Any)

Result: Column

Subtraction. Subtract the other expression from this expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.957Z

mod clj

Params: (other: Any)

Result: Column

Modulo (a.k.a. remainder) expression.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.958Z

nan? clj

(nan? expr)

Params:

Result: Column

True if the current expression is NaN.

1.5.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.927Z

not-equal clj

Params: (other: Any)

Result: Column

Inequality test.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.964Z

not-null? clj

(not-null? expr)

Params:

Result: Column

True if the current expression is NOT null.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.932Z

null-count clj

(null-count expr)

Aggregate function: returns the null count of a column.

null-rate clj

(null-rate expr)

Aggregate function: returns the null rate of a column.

null? clj

(null? expr)

Params:

Result: Column

True if the current expression is null.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.933Z

outer clj

(outer expr)

Marks the column as one from the outer query, in a Dataset that becomes a subquery through g/scalar, g/exists or g/isin, or the right side of g/lateral-join. Needs Spark 4.0.

(g/filter orders (g/exists (g/filter refunds (g/=== :order-id (g/outer :id)))))

rlike clj

(rlike expr literal)

Params: (literal: String)

Result: Column

SQL RLIKE expression (LIKE with Regex). Returns a boolean column based on a regex match.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.977Z

starts-with clj

(starts-with expr literal)

Params: (other: Column)

Result: Column

String starts with. Returns a boolean column based on a string match.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.979Z

try-cast clj

(try-cast expr new-type)

Casts the column to new-type, a type name such as "int" or a Spark type, as cast does, but gives null where a value doesn't convert, rather than an error under ANSI mode. Needs Spark 4.0.

(g/select dataframe {:price (g/try-cast :price-text "double")})

with-field clj

(with-field expr field-name value)

Returns the struct column with the field field-name set to value, in its place when the struct has that field, and at the end otherwise. value goes through ->column, so a string names a column; use lit for a string value. field-name can be a path, such as :a.b, into nested structs.

(g/select dataframe {:address (g/with-field :address :postcode (g/lit "3000"))})

| clj

(| left-expr right-expr)

Params: (other: Any)

Result: Column

Compute bitwise OR of this expression with another expression.

1.4.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.879Z

|| clj

(|| & exprs)

Params: (other: Any)

Result: Column

Boolean OR.

1.3.0

Source: https://spark.apache.org/docs/3.0.1/api/scala/org/apache/spark/sql/Column.html

Timestamp: 2020-10-19T01:56:19.994Z