Query

Programming relic is mostly about defining queries.

• A query is always a vector
• queries are made up of operations, themselves vectors of the form [operator & args].

;; example query from the tpc-h benchmark suite
[[:from :lineitem]
 [:where [<= :l_shipdate #inst "1998-09-02"]]
 [:agg
   [:l_returnflag
    :l_linestatus]
 [:sum_qty [rel/sum :l_quantity]]
 [:sum_base_price [rel/sum :l_extendedprice]]
 [:sum_disc_price  [rel/sum [* :l_extendedprice [- 1 :l_discount]]]]
 [:sum_charge [rel/sum [* :l_extendedprice [- 1 :l_discount] [+ 1 :l_tax]]]]
 [:avg_qty [rel/avg :l_quantity]]
 [:avg_price [rel/avg :l_extendedprice]]
 [:avg_disc [rel/avg :l_discount]]
 [:count_order count]]]

queries compose by adding to the vector, this is in contrast to other approaches where query optimisers take care of the overall ordering.

In relic, data always flows top-to-bottom.

The super-power is that relic allows you to materialize the query. This will convert the query into a DAG to support incremental re-computation as data in tables changes.

With few exceptions (direct index lookup) you can materialize any query with rel/materialize.

Rationale of form

In the tar pit paper, the language used to express its 'relvars' was a traditional expression tree (e.g union(a, b)).

I wanted a data-first clojure dsl that met two goals, like any good data dsl I wanted to compose using regular clojure functions, and I wanted them to be easy to write and read as literals without ide support.

The vector form I think is close to SQL, with a nice top-to-bottom reading flow. Each operation is self-contained in its own delimited form, and so you can create new queries with conj, split them with split-at and so on.

Operators

• :from a.k.a start here
• :where to select only certain rows
• :join sql style relation inner join
• :left-join sql style relational left join
• :extend compute new columns
• :select project a subset of columns, computations
• :without drop columns
• :expand flatten nested sequences
• :agg apply aggregates over rows in groups
• :sort sort rows
• :sort-limit sort with a limit
• :const provide a constant relation
• :difference set diff
• :intersection set intersection
• :union set union
• :rename rename columns
• :qualify qualify (namespace) columns

Constraints

• :check ensure certain predicates hold
• :req ensure cols exist
• :fk ensure a referenced row exists in some other query/table
• :unique unsure only one row exists for some set of expressions
• :constrain combine multiple constraints on a query/table

Indexes and direct index access

• :hash standard one-to-many hash index
• :btree standard one-to-many sorted index
• :lookup explicit index lookup