zero-one.geni.core — zero.one/geni 0.0.31

! clj

(! expr)

Params: (e: Column) Result: Column Inversion of boolean expression, i.e. NOT. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.985Z

% clj

(% left-expr right-expr)

& clj

(& left-expr right-expr)

&& clj

(&& & exprs)

* clj

(* & exprs)

** clj

(** base exponent)

Params: (l: Column, r: Column) Result: Column Returns the value of the first argument raised to the power of the second argument.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.009Z

+ clj

(+ & exprs)

- clj

(- & exprs)

->col-array clj

(->col-array args)

->column clj multimethod

->dataset clj multimethod

->date-col clj

(->date-col expr)

(->date-col expr date-format)

Params: (e: Column) Result: Column Converts the column into DateType by casting rules to DateType.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.115Z

->debug-string clj

->kebab-columns clj

(->kebab-columns dataset)

->string clj

->timestamp-col clj

(->timestamp-col expr)

(->timestamp-col expr date-format)

Params: (s: Column) Result: Column Converts to a timestamp by casting rules to TimestampType.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.123Z

->utc-timestamp clj

(->utc-timestamp expr)

Params: (ts: Column, tz: String) Result: Column Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield '2017-07-14 01:40:00.0'.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.125Z

/ clj

(/ & exprs)

< clj

<= clj

<=> clj

= clj

(= l-expr r-expr)

=!= clj

=== clj

> clj

>= clj

abs clj

(abs expr)

Params: (e: Column) Result: Column Computes the absolute value of a numeric value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.577Z

acos clj

(acos expr)

Params: (e: Column) Result: Column inverse cosine of e in radians, as if computed by java.lang.Math.acos Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.580Z

add clj

(add cms item)

(add cms item cnt)

add-months clj

(add-months expr months)

Params: (startDate: Column, numMonths: Int) Result: Column Returns the date that is numMonths after startDate.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.583Z

agg clj

(agg dataframe & args)

agg-all clj

(agg-all dataframe agg-fn)

aggregate clj

(aggregate expr init merge-fn)

(aggregate expr init merge-fn finish-fn)

Params: (expr: Column, initialValue: Column, merge: (Column, Column) ⇒ Column, finish: (Column) ⇒ Column) Result: Column Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state. The final state is converted into the final result by applying a finish function. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.587Z

alias clj multimethod

app-name clj

(app-name)

(app-name spark)

approx-count-distinct clj

(approx-count-distinct expr)

(approx-count-distinct expr rsd)

Params: (e: Column) Result: Column (Since version 2.1.0) Use approx_count_distinct Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.231Z

approx-quantile clj

(approx-quantile dataframe col-or-cols probs rel-error)

array clj

(array & exprs)

Params: (cols: Column*) Result: Column Creates a new array column. The input columns must all have the same data type.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.594Z

array-contains clj

(array-contains expr value)

Params: (column: Column, value: Any) Result: Column Returns null if the array is null, true if the array contains value, and false otherwise. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.595Z

array-distinct clj

(array-distinct expr)

Params: (e: Column) Result: Column Removes duplicate values from the array. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.596Z

array-except clj

(array-except left right)

Params: (col1: Column, col2: Column) Result: Column Returns an array of the elements in the first array but not in the second array, without duplicates. The order of elements in the result is not determined

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.597Z

array-intersect clj

(array-intersect left right)

Params: (col1: Column, col2: Column) Result: Column Returns an array of the elements in the intersection of the given two arrays, without duplicates.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.598Z

array-join clj

(array-join expr delimiter)

(array-join expr delimiter null-replacement)

Params: (column: Column, delimiter: String, nullReplacement: String) Result: Column Concatenates the elements of column using the delimiter. Null values are replaced with nullReplacement. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.601Z

array-max clj

(array-max expr)

Params: (e: Column) Result: Column Returns the maximum value in the array.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.602Z

array-min clj

(array-min expr)

Params: (e: Column) Result: Column Returns the minimum value in the array.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.603Z

array-position clj

(array-position expr value)

Params: (column: Column, value: Any) Result: Column Locates the position of the first occurrence of the value in the given array as long. Returns null if either of the arguments are null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.604Z

array-remove clj

(array-remove expr element)

Params: (column: Column, element: Any) Result: Column Remove all elements that equal to element from the given array.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.606Z

array-repeat clj

(array-repeat left right)

Params: (left: Column, right: Column) Result: Column Creates an array containing the left argument repeated the number of times given by the right argument.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.608Z

array-sort clj

(array-sort expr)

Params: (e: Column) Result: Column Sorts the input array in ascending order. The elements of the input array must be orderable. Null elements will be placed at the end of the returned array.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.609Z

array-type clj

(array-type val-type nullable)

array-union clj

(array-union left right)

Params: (col1: Column, col2: Column) Result: Column Returns an array of the elements in the union of the given two arrays, without duplicates.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.610Z

arrays-overlap clj

(arrays-overlap left right)

Params: (a1: Column, a2: Column) Result: Column Returns true if a1 and a2 have at least one non-null element in common. If not and both the arrays are non-empty and any of them contains a null, it returns null. It returns false otherwise. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.616Z

arrays-zip clj

(arrays-zip & exprs)

Params: (e: Column*) Result: Column Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.617Z

as clj multimethod

asc clj

(asc expr)

asc-nulls-first clj

(asc-nulls-first expr)

asc-nulls-last clj

(asc-nulls-last expr)

ascii clj

(ascii expr)

Params: (e: Column) Result: Column Computes the numeric value of the first character of the string column, and returns the result as an int column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.623Z

asin clj

(asin expr)

Params: (e: Column) Result: Column inverse sine of e in radians, as if computed by java.lang.Math.asin Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.626Z

assoc clj multimethod

atan clj

(atan expr)

Params: (e: Column) Result: Column inverse tangent of e, as if computed by java.lang.Math.atan Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.628Z

atan-2 clj

(atan-2 expr-x expr-y)

Params: (y: Column, x: Column) Result: Column coordinate on y-axis Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.642Z

atan2 clj

(atan2 expr-x expr-y)

Params: (y: Column, x: Column) Result: Column coordinate on y-axis Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.642Z

base-64 clj

(base-64 expr)

Params: (e: Column) Result: Column Computes the BASE64 encoding of a binary column and returns it as a string column. This is the reverse of unbase64.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.646Z

base64 clj

(base64 expr)

Params: (e: Column) Result: Column Computes the BASE64 encoding of a binary column and returns it as a string column. This is the reverse of unbase64.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.646Z

between clj

(between expr lower-bound upper-bound)

bin clj

(bin expr)

Params: (e: Column) Result: Column An expression that returns the string representation of the binary value of the given long column. For example, bin("12") returns "1100".

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.648Z

binary-files clj multimethod

bit-size clj

(bit-size bloom)

bitwise-and clj

bitwise-not clj

(bitwise-not expr)

Params: (e: Column) Result: Column Computes bitwise NOT (~) of a number.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.649Z

bitwise-or clj

bitwise-xor clj

(bitwise-xor left-expr right-expr)

bloom-filter clj

(bloom-filter dataframe expr expected-num-items num-bits-or-fpp)

boolean clj

(boolean expr)

broadcast clj

(broadcast dataframe)

Params: (df: Dataset[T]) Result: Dataset[T] Marks a DataFrame as small enough for use in broadcast joins. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.650Z

bround clj

(bround expr)

Params: (e: Column) Result: Column Returns the value of the column e rounded to 0 decimal places with HALF_EVEN round mode.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.653Z

byte clj

(byte expr)

cache clj

(cache dataframe)

case clj

(case expr & clauses)

cast clj

(cast expr new-type)

cbrt clj

(cbrt expr)

Params: (e: Column) Result: Column Computes the cube-root of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.663Z

ceil clj

(ceil expr)

Params: (e: Column) Result: Column Computes the ceiling of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.665Z

checkpoint clj

(checkpoint dataframe)

(checkpoint dataframe eager)

checkpoint-dir clj

(checkpoint-dir)

(checkpoint-dir spark)

clip clj

(clip expr low high)

coalesce clj multimethod

col clj multimethod

col-regex clj

(col-regex dataframe col-name)

collect clj

(collect dataframe)

collect-col clj

(collect-col dataframe col-name)

collect-list clj

(collect-list expr)

Params: (e: Column) Result: Column Aggregate function: returns a list of objects with duplicates.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.680Z

collect-set clj

(collect-set expr)

Params: (e: Column) Result: Column Aggregate function: returns a set of objects with duplicate elements eliminated.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.682Z

collect-vals clj

(collect-vals dataframe)

column-names clj

(column-names dataframe)

columns clj

(columns dataframe)

compatible? clj

concat clj

(concat & exprs)

Params: (exprs: Column*) Result: Column Concatenates multiple input columns together into a single column. The function works with strings, binary and compatible array columns.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.686Z

concat-ws clj

(concat-ws sep & exprs)

Params: (sep: String, exprs: Column*) Result: Column Concatenates multiple input string columns together into a single string column, using the given separator.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.687Z

cond clj

(cond & clauses)

condp clj

(condp pred expr & clauses)

conf clj

(conf)

(conf spark)

confidence clj

(confidence cms)

contains clj

(contains expr literal)

conv clj

(conv expr from-base to-base)

Params: (num: Column, fromBase: Int, toBase: Int) Result: Column Convert a number in a string column from one base to another.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.688Z

corr clj multimethod

cos clj

(cos expr)

Params: (e: Column) Result: Column angle in radians Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.695Z

cosh clj

(cosh expr)

Params: (e: Column) Result: Column hyperbolic angle Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.699Z

count clj multimethod

count-distinct clj

(count-distinct & exprs)

Params: (expr: Column, exprs: Column*) Result: Column Aggregate function: returns the number of distinct items in a group.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.706Z

count-min-sketch clj

(count-min-sketch dataframe expr eps-or-depth confidence-or-width seed)

cov clj

(cov dataframe col-name1 col-name2)

covar clj

(covar l-expr r-expr)

Params: (column1: Column, column2: Column) Result: Column Aggregate function: returns the sample covariance for two columns.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.714Z

covar-pop clj

(covar-pop l-expr r-expr)

Params: (column1: Column, column2: Column) Result: Column Aggregate function: returns the population covariance for two columns.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.710Z

covar-samp clj

(covar-samp l-expr r-expr)

Params: (column1: Column, column2: Column) Result: Column Aggregate function: returns the sample covariance for two columns.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.714Z

crc-32 clj

(crc-32 expr)

Params: (e: Column) Result: Column Calculates the cyclic redundancy check value (CRC32) of a binary column and returns the value as a bigint.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.717Z

crc32 clj

(crc32 expr)

Params: (e: Column) Result: Column Calculates the cyclic redundancy check value (CRC32) of a binary column and returns the value as a bigint.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.717Z

create-dataframe clj

(create-dataframe rows schema)

(create-dataframe spark rows schema)

create-spark-session clj

(create-spark-session
  {:keys [app-name master configs log-level checkpoint-dir]
   :or {app-name "Geni App" master "local[*]" configs {} log-level "WARN"}})

cross-join clj

(cross-join left right)

crosstab clj

(crosstab dataframe col-name1 col-name2)

cube clj

(cube dataframe & exprs)

cube-root clj

(cube-root expr)

Params: (e: Column) Result: Column Computes the cube-root of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.663Z

cume-dist clj

(cume-dist)

Params: () Result: Column Window function: returns the cumulative distribution of values within a window partition, i.e. the fraction of rows that are below the current row. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.719Z

current-date clj

(current-date)

Params: () Result: Column Returns the current date as a date column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.720Z

current-timestamp clj

(current-timestamp)

Params: () Result: Column Returns the current timestamp as a timestamp column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.722Z

cut clj

(cut expr bins)

date-add clj

(date-add expr days)

Params: (start: Column, days: Int) Result: Column Returns the date that is days days after start

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.735Z

date-diff clj

(date-diff l-expr r-expr)

Params: (end: Column, start: Column) Result: Column Returns the number of days from start to end. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.747Z

date-format clj

(date-format expr date-fmt)

Params: (dateExpr: Column, format: String) Result: Column Converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.739Z

date-sub clj

(date-sub expr days)

Params: (start: Column, days: Int) Result: Column Returns the date that is days days before start

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.742Z

date-trunc clj

(date-trunc fmt expr)

Params: (format: String, timestamp: Column) Result: Column Returns timestamp truncated to the unit specified by the format. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.744Z

datediff clj

(datediff l-expr r-expr)

Params: (end: Column, start: Column) Result: Column Returns the number of days from start to end. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.747Z

day-of-month clj

(day-of-month expr)

Params: (e: Column) Result: Column Extracts the day of the month as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.749Z

day-of-week clj

(day-of-week expr)

Params: (e: Column) Result: Column Extracts the day of the week as an integer from a given date/timestamp/string. Ranges from 1 for a Sunday through to 7 for a Saturday Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.751Z

day-of-year clj

(day-of-year expr)

Params: (e: Column) Result: Column Extracts the day of the year as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.752Z

dayofmonth clj

(dayofmonth expr)

Params: (e: Column) Result: Column Extracts the day of the month as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.749Z

dayofweek clj

(dayofweek expr)

Params: (e: Column) Result: Column Extracts the day of the week as an integer from a given date/timestamp/string. Ranges from 1 for a Sunday through to 7 for a Saturday Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.751Z

dayofyear clj

(dayofyear expr)

Params: (e: Column) Result: Column Extracts the day of the year as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.752Z

dec clj

(dec expr)

decode clj

(decode expr charset)

Params: (value: Column, charset: String) Result: Column Computes the first argument into a string from a binary using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.756Z

default-min-partitions clj

(default-min-partitions)

(default-min-partitions spark)

default-parallelism clj

(default-parallelism)

(default-parallelism spark)

degrees clj

(degrees expr)

Params: (e: Column) Result: Column Converts an angle measured in radians to an approximately equivalent angle measured in degrees.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.759Z

dense clj

(dense & values)

dense-rank clj

(dense-rank)

Params: () Result: Column Window function: returns the rank of rows within a window partition, without any gaps. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.760Z

depth clj

(depth cms)

desc clj

(desc expr)

desc-nulls-first clj

(desc-nulls-first expr)

desc-nulls-last clj

(desc-nulls-last expr)

describe clj

(describe dataframe & col-names)

disk-only clj

disk-only-2 clj

dissoc clj multimethod

distinct clj

(distinct dataframe)

double clj

(double expr)

drop clj

(drop dataframe & col-names)

drop-duplicates clj

(drop-duplicates dataframe & col-names)

drop-na clj

(drop-na dataframe)

(drop-na dataframe min-non-nulls-or-cols)

(drop-na dataframe min-non-nulls cols)

dtypes clj

(dtypes dataframe)

element-at clj

(element-at expr value)

Params: (column: Column, value: Any) Result: Column Returns element of array at given index in value if column is array. Returns value for the given key in value if column is map.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.765Z

empty? clj

encode clj

(encode expr charset)

Params: (value: Column, charset: String) Result: Column Computes the first argument into a binary from a string using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.767Z

ends-with clj

(ends-with expr literal)

estimate-count clj

(estimate-count cms item)

even? clj

(even? expr)

except clj

(except dataframe other)

except-all clj

(except-all dataframe other)

exists clj

(exists expr predicate)

Params: (column: Column, f: (Column) ⇒ Column) Result: Column Returns whether a predicate holds for one or more elements in the array. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.770Z

exp clj

(exp expr)

Params: (e: Column) Result: Column Computes the exponential of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.773Z

expected-fpp clj

(expected-fpp bloom)

explain clj multimethod

explode clj

(explode expr)

Params: (e: Column) Result: Column Creates a new row for each element in the given array or map column. Uses the default column name col for elements in the array and key and value for elements in the map unless specified otherwise.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.774Z

expm-1 clj

(expm-1 expr)

Params: (e: Column) Result: Column Computes the exponential of the given value minus one.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.777Z

expm1 clj

(expm1 expr)

Params: (e: Column) Result: Column Computes the exponential of the given value minus one.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.777Z

expr clj

(expr s)

Params: (expr: String) Result: Column Parses the expression string into the column that it represents, similar to Dataset#selectExpr. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.779Z

factorial clj

(factorial expr)

Params: (e: Column) Result: Column Computes the factorial of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.780Z

fill-na clj

(fill-na dataframe value)

(fill-na dataframe value cols)

filter clj multimethod

first clj multimethod

first-vals clj

(first-vals dataframe)

flatten clj

(flatten expr)

Params: (e: Column) Result: Column Creates a single array from an array of arrays. If a structure of nested arrays is deeper than two levels, only one level of nesting is removed. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.796Z

float clj

(float expr)

floor clj

(floor expr)

Params: (e: Column) Result: Column Computes the floor of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.798Z

forall clj

(forall expr predicate)

Params: (column: Column, f: (Column) ⇒ Column) Result: Column Returns whether a predicate holds for every element in the array. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.800Z

format-number clj

(format-number expr decimal-places)

Params: (x: Column, d: Int) Result: Column Formats numeric column x to a format like '#,###,###.##', rounded to d decimal places with HALF_EVEN round mode, and returns the result as a string column. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.802Z

format-string clj

(format-string fmt & exprs)

Params: (format: String, arguments: Column*) Result: Column Formats the arguments in printf-style and returns the result as a string column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.803Z

freq-items clj

(freq-items dataframe col-names)

(freq-items dataframe col-names support)

from-csv clj

(from-csv expr schema)

(from-csv expr schema options)

Params: (e: Column, schema: StructType, options: Map[String, String]) Result: Column Parses a column containing a CSV string into a StructType with the specified schema. Returns null, in the case of an unparseable string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.807Z

from-json clj

(from-json expr schema)

(from-json expr schema options)

Params: (e: Column, schema: StructType, options: Map[String, String]) Result: Column (Scala-specific) Parses a column containing a JSON string into a StructType with the specified schema. Returns null, in the case of an unparseable string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.826Z

from-unixtime clj

(from-unixtime expr)

(from-unixtime expr fmt)

Params: (ut: Column) Result: Column Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the yyyy-MM-dd HH:mm:ss format.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.830Z

get-field clj

(get-field expr field-name)

get-item clj

(get-item expr k)

greatest clj

(greatest & exprs)

Params: (exprs: Column*) Result: Column Returns the greatest value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.839Z

group-by clj

(group-by dataframe & exprs)

grouping clj

(grouping expr)

Params: (e: Column) Result: Column Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.845Z

grouping-id clj

(grouping-id & exprs)

Params: (cols: Column*) Result: Column Aggregate function: returns the level of grouping, equals to Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.848Z

hash clj

(hash & exprs)

Params: (cols: Column*) Result: Column Calculates the hash code of given columns, and returns the result as an int column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.849Z

hash-code clj

(hash-code expr)

head clj

(head dataframe)

(head dataframe n-rows)

head-vals clj

(head-vals dataframe)

(head-vals dataframe n-rows)

hex clj

(hex expr)

Params: (column: Column) Result: Column Computes hex value of the given column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.851Z

hint clj

(hint dataframe hint-name & args)

hour clj

(hour expr)

Params: (e: Column) Result: Column Extracts the hours as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.852Z

hypot clj

(hypot left-expr right-expr)

Params: (l: Column, r: Column) Result: Column Computes sqrt(a2 + b2) without intermediate overflow or underflow.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.865Z

if clj

inc clj

(inc expr)

initcap clj

(initcap expr)

Params: (e: Column) Result: Column Returns a new string column by converting the first letter of each word to uppercase. Words are delimited by whitespace. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.866Z

input-file-name clj

(input-file-name)

Params: () Result: Column Creates a string column for the file name of the current Spark task.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.867Z

input-files clj

(input-files dataframe)

instr clj

(instr expr substr)

Params: (str: Column, substring: String) Result: Column Locate the position of the first occurrence of substr column in the given string. Returns null if either of the arguments are null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.869Z

int clj

(int expr)

interquartile-range clj multimethod

intersect clj

(intersect dataframe other)

intersect-all clj

(intersect-all dataframe other)

iqr clj multimethod

is-compatible clj

(is-compatible bloom other)

is-empty clj

(is-empty dataframe)

is-in-collection clj

(is-in-collection expr coll)

is-local clj

(is-local dataframe)

is-nan clj

(is-nan expr)

is-not-null clj

(is-not-null expr)

is-null clj

(is-null expr)

is-streaming clj

(is-streaming dataframe)

isin clj

(isin expr coll)

jars clj

(jars)

(jars spark)

java-spark-context clj

(java-spark-context spark)

join clj

(join left right expr)

(join left right expr join-type)

join-with clj

(join-with left right condition)

(join-with left right condition join-type)

keys clj

kurtosis clj

(kurtosis expr)

Params: (e: Column) Result: Column Aggregate function: returns the kurtosis of the values in a group.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.894Z

lag clj

(lag expr offset)

(lag expr offset default)

Params: (e: Column, offset: Int) Result: Column Window function: returns the value that is offset rows before the current row, and null if there is less than offset rows before the current row. For example, an offset of one will return the previous row at any given point in the window partition. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.900Z

last clj multimethod

last-day clj

(last-day expr)

Params: (e: Column) Result: Column Returns the last day of the month which the given date belongs to. For example, input "2015-07-27" returns "2015-07-31" since July 31 is the last day of the month in July 2015.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.918Z

last-vals clj

(last-vals dataframe)

lead clj

(lead expr offset)

(lead expr offset default)

Params: (columnName: String, offset: Int) Result: Column Window function: returns the value that is offset rows after the current row, and null if there is less than offset rows after the current row. For example, an offset of one will return the next row at any given point in the window partition. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.924Z

least clj

(least & exprs)

Params: (exprs: Column*) Result: Column Returns the least value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.927Z

length clj

(length expr)

Params: (e: Column) Result: Column Computes the character length of a given string or number of bytes of a binary string. The length of character strings include the trailing spaces. The length of binary strings includes binary zeros.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.928Z

levenshtein clj

(levenshtein left-expr right-expr)

Params: (l: Column, r: Column) Result: Column Computes the Levenshtein distance of the two given string columns. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.929Z

like clj

(like expr literal)

limit clj

(limit dataframe n-rows)

lit clj

(lit arg)

local? clj

locate clj

(locate substr expr)

Params: (substr: String, str: Column) Result: Column Locate the position of the first occurrence of substr.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.933Z

log clj

(log expr)

Params: (e: Column) Result: Column Computes the natural logarithm of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.937Z

log-10 clj

(log-10 expr)

Params: (e: Column) Result: Column Computes the logarithm of the given value in base 10.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.939Z

log-1p clj

(log-1p expr)

Params: (e: Column) Result: Column Computes the natural logarithm of the given value plus one.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.941Z

log-2 clj

(log-2 expr)

Params: (expr: Column) Result: Column Computes the logarithm of the given column in base 2.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.943Z

log10 clj

(log10 expr)

Params: (e: Column) Result: Column Computes the logarithm of the given value in base 10.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.939Z

log1p clj

(log1p expr)

Params: (e: Column) Result: Column Computes the natural logarithm of the given value plus one.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.941Z

log2 clj

(log2 expr)

Params: (expr: Column) Result: Column Computes the logarithm of the given column in base 2.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.943Z

long clj

(long expr)

lower clj

(lower expr)

Params: (e: Column) Result: Column Converts a string column to lower case.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.944Z

lpad clj

(lpad expr length pad)

Params: (str: Column, len: Int, pad: String) Result: Column Left-pad the string column with pad to a length of len. If the string column is longer than len, the return value is shortened to len characters.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.946Z

ltrim clj

(ltrim expr)

Params: (e: Column) Result: Column Trim the spaces from left end for the specified string value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.948Z

map clj

(map & exprs)

Params: (cols: Column*) Result: Column Creates a new map column. The input columns must be grouped as key-value pairs, e.g. (key1, value1, key2, value2, ...). The key columns must all have the same data type, and can't be null. The value columns must all have the same data type.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.949Z

map->dataset clj

(map->dataset map-of-values)

(map->dataset spark map-of-values)

map-concat clj

(map-concat & exprs)

Params: (cols: Column*) Result: Column Returns the union of all the given maps. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.950Z

map-entries clj

(map-entries expr)

Params: (e: Column) Result: Column Returns an unordered array of all entries in the given map. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.951Z

map-filter clj

(map-filter expr predicate)

Params: (expr: Column, f: (Column, Column) ⇒ Column) Result: Column Returns a map whose key-value pairs satisfy a predicate. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.953Z

map-from-arrays clj

(map-from-arrays key-expr val-expr)

Params: (keys: Column, values: Column) Result: Column Creates a new map column. The array in the first column is used for keys. The array in the second column is used for values. All elements in the array for key should not be null.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.958Z

map-from-entries clj

(map-from-entries expr)

Params: (e: Column) Result: Column Returns a map created from the given array of entries. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.959Z

map-keys clj

(map-keys expr)

Params: (e: Column) Result: Column Returns an unordered array containing the keys of the map. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.960Z

map-type clj

(map-type key-type val-type)

map-values clj

(map-values expr)

Params: (e: Column) Result: Column Returns an unordered array containing the values of the map. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.961Z

map-zip-with clj

(map-zip-with left right merge-fn)

Params: (left: Column, right: Column, f: (Column, Column, Column) ⇒ Column) Result: Column Merge two given maps, key-wise into a single map using a function. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.963Z

master clj

(master)

(master spark)

max clj multimethod

md-5 clj

(md-5 expr)

Params: (e: Column) Result: Column Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.966Z

md5 clj

(md5 expr)

Params: (e: Column) Result: Column Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.966Z

mean clj multimethod

median clj multimethod

memory-and-disk clj

memory-and-disk-2 clj

memory-and-disk-ser clj

memory-and-disk-ser-2 clj

memory-only clj

memory-only-2 clj

memory-only-ser clj

memory-only-ser-2 clj

merge clj

(merge expr & ms)

merge-in-place clj

(merge-in-place bloom-or-cms other)

merge-with clj

might-contain clj

(might-contain bloom item)

min clj multimethod

minute clj

(minute expr)

Params: (e: Column) Result: Column Extracts the minutes as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.971Z

mod clj

monotonically-increasing-id clj

(monotonically-increasing-id)

Params: () Result: Column A column expression that generates monotonically increasing 64-bit integers. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.233Z

month clj

(month expr)

Params: (e: Column) Result: Column Extracts the month as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.974Z

months-between clj

(months-between l-expr r-expr)

Params: (end: Column, start: Column) Result: Column Returns number of months between dates start and end. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.979Z

name-value-seq->dataset clj

nan? clj

nanvl clj

(nanvl left-expr right-expr)

Params: (col1: Column, col2: Column) Result: Column Returns col1 if it is not NaN, or col2 if col1 is NaN. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.980Z

neg? clj

(neg? expr)

negate clj

(negate expr)

Params: (e: Column) Result: Column Unary minus, i.e. negate the expression. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.982Z

next-day clj

(next-day expr day-of-week)

Params: (date: Column, dayOfWeek: String) Result: Column Returns the first date which is later than the value of the date column that is on the specified day of the week. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.984Z

nlargest clj

(nlargest dataframe n-rows expr)

none clj

not clj

(not expr)

Params: (e: Column) Result: Column Inversion of boolean expression, i.e. NOT. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.985Z

not-null? clj

nsmallest clj

(nsmallest dataframe n-rows expr)

ntile clj

(ntile n)

Params: (n: Int) Result: Column Window function: returns the ntile group id (from 1 to n inclusive) in an ordered window partition. For example, if n is 4, the first quarter of the rows will get value 1, the second quarter will get 2, the third quarter will get 3, and the last quarter will get 4. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.988Z

null-count clj

(null-count expr)

null-rate clj

(null-rate expr)

null? clj

nunique clj

(nunique dataframe)

odd? clj

(odd? expr)

off-heap clj

order-by clj

(order-by dataframe & exprs)

over clj

(over column window-spec)

overlay clj

(overlay src rep pos)

(overlay src rep pos len)

Params: (src: Column, replace: Column, pos: Column, len: Column) Result: Column Overlay the specified portion of src with replace, starting from byte position pos of src and proceeding for len bytes.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.991Z

partitions clj

(partitions dataframe)

percent-rank clj

(percent-rank)

Params: () Result: Column Window function: returns the relative rank (i.e. percentile) of rows within a window partition. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.992Z

persist clj

(persist dataframe)

(persist dataframe new-level)

pi clj

The double value that is closer than any other to pi, the ratio of the circumference of a circle to its diameter.

pivot clj

(pivot grouped expr)

(pivot grouped expr values)

pmod clj

(pmod left-expr right-expr)

Params: (dividend: Column, divisor: Column) Result: Column Returns the positive value of dividend mod divisor.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.994Z

pos? clj

(pos? expr)

posexplode clj

(posexplode expr)

Params: (e: Column) Result: Column Creates a new row for each element with position in the given array or map column. Uses the default column name pos for position, and col for elements in the array and key and value for elements in the map unless specified otherwise.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.995Z

posexplode-outer clj

(posexplode-outer expr)

Params: (e: Column) Result: Column Creates a new row for each element with position in the given array or map column. Uses the default column name pos for position, and col for elements in the array and key and value for elements in the map unless specified otherwise.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:40.995Z

pow clj

(pow base exponent)

Params: (l: Column, r: Column) Result: Column Returns the value of the first argument raised to the power of the second argument.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.009Z

print-schema clj

(print-schema dataframe)

put clj

(put bloom item)

qcut clj

(qcut expr num-buckets-or-probs)

quantile clj multimethod

quarter clj

(quarter expr)

Params: (e: Column) Result: Column Extracts the quarter as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.010Z

radians clj

(radians expr)

Params: (e: Column) Result: Column Converts an angle measured in degrees to an approximately equivalent angle measured in radians.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.013Z

rand clj

(rand)

(rand seed)

Params: (seed: Long) Result: Column Generate a random column with independent and identically distributed (i.i.d.) samples uniformly distributed in [0.0, 1.0).

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.015Z

rand-nth clj

(rand-nth dataframe)

randn clj

(randn)

(randn seed)

Params: (seed: Long) Result: Column Generate a column with independent and identically distributed (i.i.d.) samples from the standard normal distribution.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.017Z

random-choice clj

(random-choice choices)

(random-choice choices probs)

(random-choice choices probs seed)

random-exp clj

(random-exp)

(random-exp rate)

(random-exp rate seed)

random-int clj

(random-int)

(random-int low high)

(random-int low high seed)

random-norm clj

(random-norm)

(random-norm mu sigma)

(random-norm mu sigma seed)

random-split clj

(random-split dataframe weights)

(random-split dataframe weights seed)

random-uniform clj

(random-uniform)

(random-uniform low high)

(random-uniform low high seed)

rank clj

(rank)

Params: () Result: Column Window function: returns the rank of rows within a window partition. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.018Z

rchoice clj

rdd clj

(rdd dataframe)

read-avro! clj multimethod

read-csv! clj multimethod

read-edn! clj multimethod

read-jdbc! clj

(read-jdbc! options)

(read-jdbc! spark options)

read-json! clj multimethod

read-libsvm! clj multimethod

read-parquet! clj multimethod

read-text! clj multimethod

read-xlsx! clj multimethod

records->dataset clj

(records->dataset records)

(records->dataset spark records)

regexp-extract clj

(regexp-extract expr regex idx)

Params: (e: Column, exp: String, groupIdx: Int) Result: Column Extract a specific group matched by a Java regex, from the specified string column. If the regex did not match, or the specified group did not match, an empty string is returned.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.019Z

regexp-replace clj

(regexp-replace expr pattern-expr replacement-expr)

Params: (e: Column, pattern: String, replacement: String) Result: Column Replace all substrings of the specified string value that match regexp with rep.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.022Z

relative-error clj

(relative-error cms)

remove clj

(remove dataframe expr)

rename-columns clj

(rename-columns dataframe rename-map)

rename-keys clj

(rename-keys expr kmap)

repartition clj

(repartition dataframe & args)

repartition-by-range clj

(repartition-by-range dataframe & args)

replace-na clj

(replace-na dataframe cols replacement)

resources clj

(resources)

(resources spark)

reverse clj

(reverse expr)

Params: (e: Column) Result: Column Returns a reversed string or an array with reverse order of elements. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.024Z

rexp clj

rint clj

(rint expr)

Params: (e: Column) Result: Column Returns the double value that is closest in value to the argument and is equal to a mathematical integer.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.026Z

rlike clj

(rlike expr literal)

rnorm clj

rollup clj

(rollup dataframe & exprs)

round clj

(round expr)

Params: (e: Column) Result: Column Returns the value of the column e rounded to 0 decimal places with HALF_UP round mode.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.028Z

row clj

(row & values)

row-number clj

(row-number)

Params: () Result: Column Window function: returns a sequential number starting at 1 within a window partition.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.029Z

rpad clj

(rpad expr length pad)

Params: (str: Column, len: Int, pad: String) Result: Column Right-pad the string column with pad to a length of len. If the string column is longer than len, the return value is shortened to len characters.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.030Z

rtrim clj

(rtrim expr)

Params: (e: Column) Result: Column Trim the spaces from right end for the specified string value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.032Z

runif clj

runiform clj

sample clj

(sample dataframe fraction)

(sample dataframe fraction with-replacement)

sample-by clj

(sample-by dataframe expr fractions seed)

sc clj

schema-of-csv clj

(schema-of-csv expr)

(schema-of-csv expr options)

Params: (csv: String) Result: Column Parses a CSV string and infers its schema in DDL format.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.036Z

schema-of-json clj

(schema-of-json expr)

(schema-of-json expr options)

Params: (json: String) Result: Column Parses a JSON string and infers its schema in DDL format.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.043Z

second clj

(second expr)

Params: (e: Column) Result: Column Extracts the seconds as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.045Z

select clj

(select dataframe & exprs)

select-columns clj

select-expr clj

(select-expr dataframe & exprs)

select-keys clj

(select-keys expr ks)

sequence clj

(sequence start stop step)

Params: (start: Column, stop: Column, step: Column) Result: Column Generate a sequence of integers from start to stop, incrementing by step.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.047Z

sha-1 clj

(sha-1 expr)

Params: (e: Column) Result: Column Calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.048Z

sha-2 clj

(sha-2 expr n-bits)

Params: (e: Column, numBits: Int) Result: Column Calculates the SHA-2 family of hash functions of a binary column and returns the value as a hex string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.049Z

sha1 clj

(sha1 expr)

Params: (e: Column) Result: Column Calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.048Z

sha2 clj

(sha2 expr n-bits)

Params: (e: Column, numBits: Int) Result: Column Calculates the SHA-2 family of hash functions of a binary column and returns the value as a hex string.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.049Z

shape clj

(shape dataframe)

shift-left clj

(shift-left expr num-bits)

Params: (e: Column, numBits: Int) Result: Column Shift the given value numBits left. If the given value is a long value, this function will return a long value else it will return an integer value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.050Z

shift-right clj

(shift-right expr num-bits)

Params: (e: Column, numBits: Int) Result: Column (Signed) shift the given value numBits right. If the given value is a long value, it will return a long value else it will return an integer value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.052Z

shift-right-unsigned clj

(shift-right-unsigned expr num-bits)

Params: (e: Column, numBits: Int) Result: Column Unsigned shift the given value numBits right. If the given value is a long value, it will return a long value else it will return an integer value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.053Z

short clj

(short expr)

show clj

(show dataframe)

(show dataframe options)

show-vertical clj

(show-vertical dataframe)

(show-vertical dataframe options)

shuffle clj multimethod

signum clj

(signum expr)

Params: (e: Column) Result: Column Computes the signum of the given value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.056Z

sin clj

(sin expr)

Params: (e: Column) Result: Column angle in radians Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.058Z

sinh clj

(sinh expr)

Params: (e: Column) Result: Column hyperbolic angle Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.060Z

size clj

(size expr)

Params: (e: Column) Result: Column Returns length of array or map. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.062Z

skewness clj

(skewness expr)

Params: (e: Column) Result: Column Aggregate function: returns the skewness of the values in a group.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.064Z

slice clj

(slice expr start length)

Params: (x: Column, start: Int, length: Int) Result: Column Returns an array containing all the elements in x from index start (or starting from the end if start is negative) with the specified length.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.065Z

sort clj

sort-array clj

(sort-array expr)

(sort-array expr asc)

Params: (e: Column) Result: Column Sorts the input array for the given column in ascending order, according to the natural ordering of the array elements. Null elements will be placed at the beginning of the returned array.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.068Z

sort-within-partitions clj

(sort-within-partitions dataframe & exprs)

soundex clj

(soundex expr)

Params: (e: Column) Result: Column Returns the soundex code for the specified expression.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.069Z

spark-conf clj

(spark-conf spark-session)

spark-context clj

(spark-context)

(spark-context spark)

spark-home clj

(spark-home)

(spark-home spark)

spark-partition-id clj

(spark-partition-id)

Params: () Result: Column Partition ID.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.070Z

spark-session clj

(spark-session dataframe)

sparse clj

split clj

(split expr pattern)

Params: (str: Column, pattern: String) Result: Column Splits str around matches of the given pattern.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.073Z

sql-context clj

(sql-context dataframe)

sqr clj

(sqr expr)

Returns the value of the first argument raised to the power of two.

sqrt clj

(sqrt expr)

Params: (e: Column) Result: Column Computes the square root of the specified float value.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.075Z

starts-with clj

(starts-with expr literal)

std clj

(std expr)

Params: (e: Column) Result: Column Aggregate function: alias for stddev_samp.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.077Z

stddev clj

(stddev expr)

Params: (e: Column) Result: Column Aggregate function: alias for stddev_samp.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.077Z

stddev-pop clj

(stddev-pop expr)

Params: (e: Column) Result: Column Aggregate function: returns the population standard deviation of the expression in a group.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.091Z

stddev-samp clj

(stddev-samp expr)

Params: (e: Column) Result: Column Aggregate function: alias for stddev_samp.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.077Z

storage-level clj

(storage-level dataframe)

str clj

(str expr)

streaming? clj

struct clj

(struct & exprs)

Params: (cols: Column*) Result: Column Creates a new struct column. If the input column is a column in a DataFrame, or a derived column expression that is named (i.e. aliased), its name would be retained as the StructField's name, otherwise, the newly generated StructField's name would be auto generated as col with a suffix index + 1, i.e. col1, col2, col3, ...

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.096Z

struct-field clj

(struct-field col-name data-type nullable)

struct-type clj

(struct-type & fields)

substring clj

(substring expr pos len)

Params: (str: Column, pos: Int, len: Int) Result: Column Substring starts at pos and is of length len when str is String type or returns the slice of byte array that starts at pos in byte and is of length len when str is Binary type

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.097Z

substring-index clj

(substring-index expr delim cnt)

Params: (str: Column, delim: String, count: Int) Result: Column Returns the substring from string str before count occurrences of the delimiter delim. If count is positive, everything the left of the final delimiter (counting from left) is returned. If count is negative, every to the right of the final delimiter (counting from the right) is returned. substring_index performs a case-sensitive match when searching for delim.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.098Z

sum clj multimethod

sum-distinct clj

(sum-distinct expr)

Params: (e: Column) Result: Column Aggregate function: returns the sum of distinct values in the expression.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.103Z

summary clj

(summary dataframe & stat-names)

table->dataset clj

(table->dataset table col-names)

(table->dataset spark table col-names)

tail clj

(tail dataframe n-rows)

tail-vals clj

(tail-vals dataframe n-rows)

take clj

(take dataframe n-rows)

take-vals clj

(take-vals dataframe n-rows)

tan clj

(tan expr)

Params: (e: Column) Result: Column angle in radians Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.107Z

tanh clj

(tanh expr)

Params: (e: Column) Result: Column hyperbolic angle Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.109Z

time-window clj

(time-window time-expr duration)

(time-window time-expr duration slide)

(time-window time-expr duration slide start)

Params: (timeColumn: Column, windowDuration: String, slideDuration: String, startTime: String) Result: Column Bucketize rows into one or more time windows given a timestamp specifying column. Window starts are inclusive but the window ends are exclusive, e.g. 12:05 will be in the window [12:05,12:10) but not in [12:00,12:05). Windows can support microsecond precision. Windows in the order of months are not supported. The following example takes the average stock price for a one minute window every 10 seconds starting 5 seconds after the hour: Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.220Z

to-byte-array clj

(to-byte-array cms)

to-csv clj

(to-csv expr)

(to-csv expr options)

Params: (e: Column, options: Map[String, String]) Result: Column (Java-specific) Converts a column containing a StructType into a CSV string with the specified schema. Throws an exception, in the case of an unsupported type.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.112Z

to-date clj

(to-date expr)

(to-date expr date-format)

Params: (e: Column) Result: Column Converts the column into DateType by casting rules to DateType.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.115Z

to-debug-string clj

to-df clj multimethod

to-json clj multimethod

to-string clj

to-timestamp clj

(to-timestamp expr)

(to-timestamp expr date-format)

Params: (s: Column) Result: Column Converts to a timestamp by casting rules to TimestampType.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.123Z

to-utc-timestamp clj

(to-utc-timestamp expr)

Params: (ts: Column, tz: String) Result: Column Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield '2017-07-14 01:40:00.0'.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.125Z

total-count clj

(total-count cms)

transform clj

(transform expr xform-fn)

Params: (column: Column, f: (Column) ⇒ Column) Result: Column Returns an array of elements after applying a transformation to each element in the input array. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.128Z

transform-keys clj

(transform-keys expr key-fn)

Params: (expr: Column, f: (Column, Column) ⇒ Column) Result: Column Applies a function to every key-value pair in a map and returns a map with the results of those applications as the new keys for the pairs. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.130Z

transform-values clj

(transform-values expr key-fn)

Params: (expr: Column, f: (Column, Column) ⇒ Column) Result: Column Applies a function to every key-value pair in a map and returns a map with the results of those applications as the new values for the pairs. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.138Z

translate clj

(translate expr match replacement)

Params: (src: Column, matchingString: String, replaceString: String) Result: Column Translate any character in the src by a character in replaceString. The characters in replaceString correspond to the characters in matchingString. The translate will happen when any character in the string matches the character in the matchingString.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.139Z

trim clj

(trim expr trim-string)

Params: (e: Column) Result: Column Trim the spaces from both ends for the specified string column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.141Z

unbase-64 clj

(unbase-64 expr)

Params: (e: Column) Result: Column Decodes a BASE64 encoded string column and returns it as a binary column. This is the reverse of base64.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.188Z

unbase64 clj

(unbase64 expr)

Params: (e: Column) Result: Column Decodes a BASE64 encoded string column and returns it as a binary column. This is the reverse of base64.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.188Z

unbounded-following clj

unbounded-preceeding clj

unhex clj

(unhex expr)

Params: (column: Column) Result: Column Inverse of hex. Interprets each pair of characters as a hexadecimal number and converts to the byte representation of number.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.189Z

union clj

(union & dataframes)

union-by-name clj

(union-by-name & dataframes)

unix-timestamp clj

(unix-timestamp)

(unix-timestamp expr)

(unix-timestamp expr pattern)

Params: () Result: Column Returns the current Unix timestamp (in seconds) as a long.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.197Z

unpersist clj

(unpersist dataframe)

(unpersist dataframe blocking)

update clj multimethod

upper clj

(upper expr)

Params: (e: Column) Result: Column Converts a string column to upper case.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.198Z

vals clj

value-counts clj

(value-counts dataframe)

var-pop clj

(var-pop expr)

Params: (e: Column) Result: Column Aggregate function: returns the population variance of the values in a group.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.200Z

var-samp clj

(var-samp expr)

Params: (e: Column) Result: Column Aggregate function: alias for var_samp.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.204Z

variance clj

(variance expr)

Params: (e: Column) Result: Column Aggregate function: alias for var_samp.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.204Z

version clj

(version)

(version spark)

week-of-year clj

(week-of-year expr)

Params: (e: Column) Result: Column Extracts the week number as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.209Z

weekofyear clj

(weekofyear expr)

Params: (e: Column) Result: Column Extracts the week number as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.209Z

when clj

(when condition if-expr)

(when condition if-expr else-expr)

Params: (condition: Column, value: Any) Result: Column Evaluates a list of conditions and returns one of multiple possible result expressions. If otherwise is not defined at the end, null is returned for unmatched conditions. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.211Z

where clj multimethod

width clj

(width cms)

window clj

(window {:keys [partition-by order-by range-between rows-between]})

windowed clj

(windowed options)

with-column clj

(with-column dataframe col-name expr)

with-column-renamed clj

(with-column-renamed dataframe old-name new-name)

write-avro! clj

(write-avro! dataframe path)

(write-avro! dataframe path options)

write-csv! clj

(write-csv! dataframe path)

(write-csv! dataframe path options)

write-edn! clj

(write-edn! dataframe path)

(write-edn! dataframe path options)

write-jdbc! clj

(write-jdbc! dataframe options)

write-json! clj

(write-json! dataframe path)

(write-json! dataframe path options)

write-libsvm! clj

(write-libsvm! dataframe path)

(write-libsvm! dataframe path options)

write-parquet! clj

(write-parquet! dataframe path)

(write-parquet! dataframe path options)

write-text! clj

(write-text! dataframe path)

(write-text! dataframe path options)

write-xlsx! clj

(write-xlsx! dataframe path)

(write-xlsx! dataframe path options)

xxhash-64 clj

(xxhash-64 & exprs)

Params: (cols: Column*) Result: Column Calculates the hash code of given columns using the 64-bit variant of the xxHash algorithm, and returns the result as a long column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.222Z

xxhash64 clj

(xxhash64 & exprs)

Params: (cols: Column*) Result: Column Calculates the hash code of given columns using the 64-bit variant of the xxHash algorithm, and returns the result as a long column.

Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.222Z

year clj

(year expr)

Params: (e: Column) Result: Column Extracts the year as an integer from a given date/timestamp/string. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.223Z

zero? clj

(zero? expr)

zip-with clj

(zip-with left right merge-fn)

Params: (left: Column, right: Column, f: (Column, Column) ⇒ Column) Result: Column Merge two given arrays, element-wise, into a single array using a function. If one array is shorter, nulls are appended at the end to match the length of the longer array, before applying the function. Source: https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/functions$.html Timestamp: 2020-10-02T14:21:41.226Z

zipmap clj

| clj

(| left-expr right-expr)

|| clj

(|| & exprs)