Design Thoughts

When I first started this project, I tried to stick with just unqualified keys. Overall that’s worked out ok. But there’s definitely a part of me that wonders is namespaced keys would help create a separation of concerns. As things have matured, I’ve ended up mixing in additional keys to statement maps to give options to the execution engine. I could also see rendering hints getting mixed in too. It lets you pass around one value to functions. Then each function just uses what it needs or recognizes from the map.

For long queries, I like how the DSL formats. It’s just Clojure code and formatting it idiomatically works nicely for SQL statements too.

But designing DSL syntax that’s both flexible yet hopefully intuitive is challenging. Especially for the functions that take a few leading positional arguments followed by mk-map* arguments.

Once you’ve got a nice syntax and designed the spec for each statement map, you still have to write the back-end rendering logic for each new statement and clause. That part may actually be easier.

Anyway, the raw is really easy to write and to render. The main problem is how well it composes.

For statements have a primary helper fn. You tell it how many args are positional and give it a fn to process a sequence of positional args. That function’s job is to return an argument suitable for mk-map* (either a map, a vector, a list, a function, or nil). If the first arg isn’t already a statement, the user fn is called to process the positional args.

There is a similar pattern for rendering functions. One for statements and one for clauses. The functions destructure a map. For each key call a fn of this form:

Basically, the fn body replaces each map value with its rendered result. Then the fn calls either join-by-space or query-clauses to finish up. Could write a helper fn that takes a map from keyword to render-fn and a vector of keys representing the order of rendered elements. To avoid repeating each key, could pass a sequence of pairs where the first item is the map key and the second item is a function to process it. Or first element of pair is either just a keyword or a keyword and the default value if the key is not contained in the map (to let receiving fn distinguish between nil and not present in map). The order of the pairs is the order of the clauses.

This kind of processing is even more general. It’s how you linearize a map. In the case of seaquell, we want to linearize the map into the raw SQL DSL and then transform it to a string. We need a similar transformation to go from something linear (nested vectors) to nested maps. But you need a way to tell when something should be a map key or not.

When using the DSL, it would be nice to override any part of it with raw SQL.

For whole new clauses, it would be nice to have a mechanism to extend the rendering for a given statement.

For existing clauses, it would be nice to replace the entire clause with raw SQL when desired. For example, any property generated by (def-props) should accept raw SQL. Any collection of SQL elements should allow each one to be raw.

Basically, when you’re trying to use the DSL, you want to use as little raw SQL as possible, but you want to have precise very granular control of where that raw SQL goes.

I can also imagine going the other way, where the SQL vector is the primary abstraction and the DSL is the secondary one. For this application, any time the raw encountered a map and didn’t know what to do with it, it could call a user provided function to render it (or remove it from the stream).

In either case, want to tell a consistent story about how to quote strings and identifiers and how to format SQL keywords and such.

You could think of this library as defining three components:

How can I achieve the pure separation I’m looking for between these three components?

I do like the idea of moving all my fragments to their own namespace. If you want them, it’s easy to pull them in. All fragments have upper case names and are just map values that get mixed in. Most fragments are generated by a corresponding function in Clojure case.

At least one of the libraries out there saves your queries in text files and then has some special stuff to templatize those queries. The benefit is that you can write any SQL you want in any dialect. My raw dialect feels like you could achieve much the same thing, but keep all your data in Clojure. You wouldn’t need to slurp and parse files, you just read Clojure forms and traverse them. Perhaps certain placeholders get replaced with your own snippets.

Once things are split out completely, the following properties may be useful, especially from the REPL. Although, from the REPL it would be really nice to not have to supply the extra (db c) property to each statement I want to run. That said, I’ve tried to avoid the use of dynamic vars.