Liking cljdoc? Tell your friends :D

Pathom Developers Guide

Table of Contents

1. Introduction

The pathom library provides a rich set of functionality to build robust parsers to process graph queries for Om.next and Fulcro. The query notation is derived from Datomic pull syntax.

The library includes:

  • A reader abstraction that allows for easy composition.

  • The concept of entity which works as a base framework for reusable sharable readers.

  • A plugin system with some built-in plugins:

    • Error handler: Handles errors at an attribute level.

    • Request cache: For caching the results of parsing repetition that can happen on a single request.

    • Profiler: a plugin to measure the time spent on each attribute during the parser.

  • Connect: a higher level abstraction that can resolve attribute relationships automatically. For example automatic traversal of database joins or resolving data through network requests. This enables exploratory capabilities and much simpler access when the need arises to do extra work to resolve a single conceptual join.

  • GraphQL integration: Use GraphQL endpoints directly from your query system (in development).

Most people will find the most leverage in the "Connect" features, which allow you to quickly build dynamic query processing systems to easily satisfy client data requests.

1.1. Aliases Used in Code Examples

Throughout the book our code examples will use aliases instead of explicit namespaces. The aliases used are as if we had the following namespace requires:

(ns my-namespace
  (:require
    [com.wsscode.pathom.core :as p]
    [com.wsscode.pathom.connect :as pc]
    [com.wsscode.pathom.connect.graphql :as pcg]
    [com.wsscode.pathom.graphql :as pg]
    [com.wsscode.pathom.trace :as pt]
    [com.wsscode.common.async-clj(s) :refer [let-chan <!p go-catch <? <?maybe]]))

So, any time you see a usage of a namespace in a keyword or function like p/parser or ::p/reader you should remember that these are the namespaces involved.

1.2. Presentation

If you like to learn by seeing presentations, I did one at Dutch Clojure Days 2018.

1.3. Contributing

This source for this book is at https://github.com/wilkerlucio/pathom/blob/master/docs/DevelopersGuide.adoc. Feel free to send a PR with edits, corrections, or other contributions. If you’re wanting to make a large edit, please open an issue first.

1.4. Upgrade notes

In pathom we try at most to don’t introduce breaking change, that means in some cases we prefer to introduce a new function or namespace instead of replacing an old one when they might differ in result. This part of the guide will provide information about suggestions to do when upgrading to a certain version. Also in the exceptional cases we introduce breaking changes they will also appear with upgrade notes here.

1.4.1. 2.2.0 - Upgrade guide

Supporting resolver libraries

This is not a breaking change. Pathom 2.2.0 introduces new dispatchers to call resolvers and mutations, the old dispatchers used to rely on a multi-method to invoke the calls, the new dispatchers will just look up for a lamba in the resolver/mutation definition that’s stored in the index. The main advantage is that we reduce the number of places we need to change when adding resolvers and mutations. In the previous case you have 3 places to change, the index, the resolver dispatch and the mutation dispatch, with the new dispatch there is just the index.

This will facilitate the creation of shared resolvers/mutations libraries that you can inject and make part of your parsing system. To give an example shared library I have wrote a demo repo implementing the Youtube API for connect.

To enable this feature you will have to change the dispatch function used in the parser setup, replacing your resolvers fns with new ones provided by connect, example:

; this is the old setup

; setup indexes atom
(def indexes (atom {}))

; setup resolver dispatch and factory
(defmulti resolver-fn pc/resolver-dispatch)
(def defresolver (pc/resolver-factory resolver-fn indexes))

; setup mutation dispatch and factory
(defmulti mutation-fn pc/mutation-dispatch)
(def defmutation (pc/mutation-factory mutation-fn indexes))

(def parser
  (p/parser {::p/env     {::p/reader             [p/map-reader pc/all-readers]
                          ::pc/resolver-dispatch resolver-fn
                          ::pc/mutate-dispatch   mutation-fn
                          ::pc/indexes           @indexes
                          ::db                   (atom {})}
             ::p/mutate  pc/mutate
             ::p/plugins [p/error-handler-plugin
                          p/request-cache-plugin
                          pp/profile-plugin]}))

To do the minimal change is to use this:

; minimal changes to support custom

; setup indexes atom
(def indexes (atom {}))

; setup resolver dispatch and factory
(defmulti resolver-fn pc/resolver-dispatch)
(def defresolver (pc/resolver-factory resolver-fn indexes))

; setup mutation dispatch and factory
(defmulti mutation-fn pc/mutation-dispatch)
(def defmutation (pc/mutation-factory mutation-fn indexes))

(def parser
  (p/parser {::p/env     {::p/reader             [p/map-reader pc/all-readers]
                          ; replace resolver dispatch
                          ::pc/resolver-dispatch pc/resolver-dispatch-embedded
                          ; replace mutation dispatch
                          ::pc/mutate-dispatch   pc/mutation-dispatch-embedded
                          ::pc/indexes           @indexes
                          ::db                   (atom {})}
             ::p/mutate  pc/mutate
             ::p/plugins [; add connect plugin
                          (pc/connect-plugin)
                          p/error-handler-plugin
                          p/request-cache-plugin
                          pp/profile-plugin]}))

The new versions of resolver-factory and mutation-factory will add the lambdas into the definition map, making those compatible with the new *-dispatch-embedded, so you get your old resolvers plus any extra ones from libs.

From now on when I say resolver or resolvers I’m meaning both resolvers and mutations, adding this note here so you don’t have to read all the repetition.

From now on we will be recommending the new way of writing resolvers using the pc/defresolver macro, I see a few advantages that I like to highlight about this approach:

  1. Your resolvers become isolated building blocks on their own, instead of having to spread it’s definition in the index + multi-method, now the map contais everything that resolver needs to be used

  2. You get a fine control of what resolvers you want inject in a given parser, before wasn’t easy to write several parsers using sub sets of resolvers, with each in a symbol you can compose as you please

  3. Simplify the boilerplate, no more need to define the multi-methods for dispatching

This is what the setup looks like by using the new map format:

; setup with map format

; this will generate a def for the symbol `some-resolver` and the def will
; contain a map that is the resolver definition, no external side effects
(pc/defresolver some-resolver [env input]
  {::pc/input  #{::id}
   ::pc/output [::name ::email]}
  (get (::db env) (::id input)))

; define another resolver
(pc/defresolver other-resolver ...)

; now it's a good practice to create a sequence containing the resolvers
(def app-registry [some-resolver other-resolver])

(def parser
  (p/parser {::p/env     {::p/reader             [p/map-reader pc/all-readers]
                          ::pc/resolver-dispatch pc/resolver-dispatch-embedded
                          ::pc/mutate-dispatch   pc/mutation-dispatch-embedded
                          ::db                   (atom {})}
             ::p/mutate  pc/mutate
             ::p/plugins [; you can use the connect plugin to register your resolvers,
                          ; but any plugin with the ::pc/register key will be also
                          ; included in the index
                          (pc/connect-plugin {::pc/register app-registry})
                          p/error-handler-plugin
                          p/request-cache-plugin
                          pp/profile-plugin]}))

The pain point add is in the fact you now have to specify the resolvers to use, but think that before this the only option was all or nothing. If you have resolvers spread across many files, I suggest you create one list at the end of each namespace containing all the resolvers from that file, this way you can combine those in a later index. The resolver list will be flattened out when it’s processed, its ok to send multiple lists inside lists, this facilitates de combination of lists of resolvers.

The multi-method format is still ok to use, there are no plans to remove it and keep using it if you prefer.
Parallel parser

Pathom 2.2.0 also introduces the parallel parser. Before this all the processing of Pathom were done serially, one attribute at a time, the new parser brings the ability to support the attributes to be processed in parallel, the mechanism is described at the parallel parser section.

If you are using the async-parser the change to the parallel is just changing the parser to parallerl-parser and the connect readers. If you are using the regular sync parser, then you may need to adapt some things to support an async enviroment, here are things to watch for:

  1. If you wrote plugins, when wrapping things you must consider that their response will be async (return core.async channels), One of the easiest ways to handle this is using the let-chan macro, which is a let that automatically handles channels and make the process transparent.

  2. If you done recursive parser calls (that includes calls to functions like join, entity with arity 2)

Tracer

Pathom 2.2.0 includes a new tracer feature. I recommend you replace the old profiler with this, you remove pp/profile-plugin and add the p/tracer-plugin (better as the last plugin on your chain).

1.4.2. 2.2.0-beta11 → 2.2.0-RC1 - Breaking changes

In version 2.2.0-beta11 we introduced the pc/connect-plugin and pc/register with the intent to provider an easier to write shared resolvers and also reduce the boilerplate to setup connect.

This strategy failed in be simple to setup a register and more integrations, because it relied on multiple parts, a better strategy emerged by embedding the lamba to run the resolvers and mutations in their own map instead, so they are complete and stand alone.

But to accomodate this the connect plugin and the pc/register had to change, before the pc/connect-plugin was a var, now it’s an fn that you must call. The register used to take the index atom, the multimethod for resolver and the multimethod for mutations, and did a stateful mutation in all three. Now takes the index in a map format and returns another index with the things registered, now it’s a pure function.

2. How to Use This Library

We expect that most of our user base is made up of Om Next or Fulcro users. The purpose of this library is to make it much easier to build code that can process the graph query notation of these libraries on both the client and server side. We expect you to have have one or more of the following needs:

  • You want to fulfill a client UI query from some server-side data source(s).

  • You want to build a client-side parser for directly filling local UI data queries from a local data source.

  • You want to build a parser (client or server) that uses async APIs to fulfill different parts of a query. Perhaps gluing together data sources from various micro-services.

  • You want to use a GraphQL API from the client.

  • You want to provide third-party users a GraphQL API (Future Work)

When building most parsers you’ll want to use Pathom Connect.

To process Fulcro/Om Next queries against GraphQL you’ll use the GraphQL Integration.

3. Pathom Connect

Connect provides is a high-level abstraction layer for building query processing code. It handles a number of the gory details in an automatic way that allows you to focus more on your data model and less on the parser itself. It generates an an index of your graph’s features that can be used for a number of very useful features:

  1. Auto-complete of graph queries in tools for data exploration (see OgE).

  2. Graph edge generation from the index’s connection information.

  3. Multiple ways to reach a given attribute, automatically resolved via known reachable edges and transitive relations.

The Connect index is a rich source of information about how your attributes connect and how they can locate each other.

3.1. The Basics

In order to use connect you need to understand some basics about Pathom’s core features. These are covered in detail in later chapters, but you’ll easily understand the basics we need for connect without going into great detail.

You’re going to be defining a parser that uses an environment and graph query to produce a tree of data from arbitrary data sources.

If you’re unfamiliar with the Om Next/Fulcro graph query notation, you should first read up on that in the Fulcro Developer’s Guide.

Some simple examples of what we’re doing are:

;; query
[:person/name]

;; possible result
{:person/name "Samantha"}

;; query
[:account/id
 {:billing/charges [:charge/amount]}]

;; possible result
{:account/id 1
 :billing/charges [{:charge/amount 11}
                   {:charge/amount 22}]}

To make sure we’re on the same page, here’s a quick bit of vocabulary:

Environment

A map of configuration and context that is used to configure the parser and is also passed to the reader.

Resolver

A Pathom Connect component that you write to indicate ways in which data attributes can be resolved against your data sources. Resolvers are composed together into a connect-based Reader.

Reader

A component of that attempts to resolve elements of the query (one at a time). When using connect you also use a built-in Map Reader that can pull attributes that are already in the environment without having to do further work against your underlying resolvers and data sources.

Connect Indexes

A set of indexes that are filled with resolver data and allow connect to understand how graph queries can be resolved by the resolvers.

3.2. Baseline Boilerplate

Connect is generally set up with the minimal steps every time. Other sections of this book cover the options in more detail, but for the moment take this small bit of code as the general "starting point" for writing a connect-based query processing system:

(ns com.wsscode.pathom.book.connect.getting-started
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))

;; Define one or more resolvers
(pc/defresolver some-symbol [env input] ...)
(pc/defresolver some-other-symbol [env input] ...)
...

;; resolvers are just maps, we can compose many using sequences
(def my-app-registry [some-symbol some-other-symbol])

;; Create a parser that uses the resolvers:
(def parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register my-app-registry}) ; setup connect and use our resolvers
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

; note the parallel parser call will return a channel, you must read the value on it
; to get the parser results

3.3. Resolvers

In Connect you implement the graph by creating resolvers, those resolvers are functions that expose some data on the graph.

A resolver has a few basic elements:

  1. Inputs – A set of attributes that must be in the current parsing context for the resolver to be able to work. Inputs is optional, and missing inputs means that the resolver is always capable of working independent of the current parsing context.

  2. Outputs - A query-like notation that gives the "pattern" of the part of the query the resolver is able to resolve. This is typically a list of attributes/joins, where joins typically include a simple subquery.

  3. A lambda - A (fn [env input-data] tree-of-promised-output) that takes the inputs and turns them into a tree that satisfies the "output query".

So you might define a resolver like this:

(pc/defresolver person-resolver [{:keys [database] :as env} {:keys [person/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:person/first-name :person/age]}
  (let [person (my-database/get-person database id)]
    {:person/age        (:age person)
     :person/first-name (:first-name person)}))
If you use Cursive, you can ask it to resolve the pc/defresolver as a defn and you will get proper symbol resolution

Where the database in the environment would be supplied when running the parser, and the input would have to be found in the current context. Remember that graph queries are contextual…​you have to have a starting node to work from, so in the above example we’re assuming that during our parse we’ll reach a point where the context contains a :person/id. The my-database stuff is just made up for this example, and is intended to show you that your data source need not remotely match the schema of your graph query.

Pathom will scan through the defined resolvers in order to try to satisfy all of the properties in a query. So, technically you can split up your queries as much as makes sense into separate resolvers, and as long as the inputs are in the context Pathom will assemble things back together.

Of course, it doesn’t make sense in this case to do so, because each resolver would end up running a new query:

(pc/defresolver person-age-resolver [{:keys [database] :as env} {:keys [person/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:person/age]}
  (let [person (my-database/get-person database id)]
    {:person/age (:age person)}))

(pc/defresolver person-first-name-resolver [{:keys [database] :as env} {:keys [person/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:person/first-name]}
  (let [person (my-database/get-person database id)]
    {:person/first-name (:first-name person)}))

...

The point is that a single-level query like [:person/id :person/first-name :person/age] can be satisfied and "folded together" by pathom over any number of resolvers.

This fact is the basis of parser (de)composition and extensibility. It can also come in handy for performance refinements when there are computed attributes.

3.3.1. Derived/Computed Attributes

There are times when you’d like to provide an attribute that is computed in some fashion. You can, of course, simply compute it within the resolver along with other properties like so:

(pc/defresolver person-resolver [{:keys [database] :as env} {:keys [person/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:person/first-name :person/last-name :person/full-name :person/age]}
  (let [{:keys [age first-name last-name]} (my-database/get-person database id)]
    {:person/age        age
     :person/first-name first-name
     :person/last-name  last-name
     :person/full-name  (str first-name " " last-name) ; COMPUTED
     ...}))

but this means that you’ll take the overhead of the computation when any query relate to person comes up. You can instead spread such attributes out into other resolvers as we discussed previously, which will only be invoked if the query actually asks for those properties:

(pc/defresolver person-resolver [{:keys [database] :as env} {:keys [person/id]}]
  {::pc/input #{:person/id}
   ::pc/output [:person/first-name :person/last-name :person/age]}
  (let [{:keys [age first-name last-name]} (my-database/get-person database id)]
    {:person/age        age
     :person/first-name first-name
     :person/last-name  last-name}))

(pc/defresolver person-name-resolver [_ {:person/keys [first-name last-name]}]
  {::pc/input #{:person/first-name :person/last-name}
   ::pc/output [:person/full-name]}
  {:person/full-name (str first-name " " last-name)})

This combination of resolvers can still resolve all of the properties in [:person/full-name :person/age] (if :person/id is in the context), but a query for just [:person/age] won’t invoke any of the logic for the person-name-resolver.

3.3.2. Single Inputs — Establishing Context

So far we have seen how to define a resolver that can work as long as the inputs are already in the environment. You’re almost certainly wondering how to do that.

One way is to define global resolvers and start the query from them, but very often you’d just like to be able to say "I’d like the first name of person with id 42."

Om Next/Fulcro use "idents" to specify exactly that sort of query:

[{[:person/id 42] [:person/first-name]}]

The above is a join on an ident, and the expected result is a map with the ident as a key:

{[:person/id 42] {:person/first-name "Joe"}}

The query itself has everything you need to establish the context for running the person-resolver, and in fact that is how Pathom single-input resolvers work.

If you use an ident in a query then Pathom is smart enough to know that it can use that ident to establish the context for finding resolvers. In other words, in the query above the ident [:person/id 42] is turned into the parsing context {:person/id 42}, which satisfies the input of any resolver that needs :person/id to run.

3.3.3. Resolver Without Input — Global Resolver

A resolver that requires no input can output its results at any point in the graph, thus it is really a global resolver. Pay particular note to the qualfication: any point in the graph. Not just root. Thus, a resolver without inputs can "inject" its outputs into any level of the query graph result.

We’re going to start building a parser that can satisfy queries about a music store. So, we’ll start with a global resolver that can resolve the "latest product". The code below shows the entire code needed, boilerplate and all:

link:../src-docs/com/wsscode/pathom/book/connect/getting_started.cljs[role=include]

Our first resolver exposes the attribute ::latest-product, and since it doesn’t require any input it is a global resolver. Also, note that our output description includes the full output details (including nested attributes), this is mostly useful for auto-complete on UI’s and automatic testing. If you return extra data it will still end up in the output context.

Try some of these queries on the demo below:

[::latest-product]
[{::latest-product [:product/title]}]

; ::latest-product can be requested anywhere
[{::latest-product
  [* ::latest-product]}]
[::latest-product]

3.3.4. Resolvers with input

Next, let’s say we want to have a new attribute which is the brand of the product. Of course, we could just throw the data there in our other resolver, but the real power of connect comes out when we start splitting the responsibilities among resolvers, so let’s define a resolver for brand that requires an input of :product/id:

link:../src-docs/com/wsscode/pathom/book/connect/getting_started2.cljs[role=include]
[{::latest-product [:product/title :product/brand]}]

The input is a set containing the keys required on the current entity in the parsing context for the resolver to be able to work. This is where Connect starts to shine because any time your query asks for a bit of data it will try to figure it out how to satisfy that request based on the attributes that the current contextual entity already has.

More importantly: Connect will explore the dependency graph in order to resolve things if it needs to! To illustrate this let’s pretend we have some external ID for the brand, and that we can derive this ID from the brand string, pretty much just another mapping:

;; a silly pretend lookup
(def brand->id {"Taylor" 44151})

(pc/defresolver brand-id-from-name [_ {:keys [product/brand]}]
  {::pc/input #{:product/brand}
   ::pc/output [:product/brand-id]}
  {:product/brand-id (get brand->id brand)})

(comment
  (parser {} [{::latest-product [:product/title :product/brand-id]}])
  ; => #::{:latest-product #:product{:title "Acoustic Guitar", :brand-id 44151}}
)

Note that our query never said anything about the :product/brand. Connect automatically walked the path :product/id → :product/brand → :product/brand-id to obtain the information desired by the query!

When a required attribute is not present in the current entity, Connect will look for resolvers that can fetch it, analyze their inputs, and recursively walk backwards towards the "known data" in the context. When a required attribute is not present in the current entity, Connect will calculate the possibole paths from the data you have to the data you request, then it can use some euristic to decide which path to take and walk this path to reach the data, if there is no possible path connect reader will return ::p/continue to let another reader try to handle that key. You can read more about how this works in the Index page.

Also remember that single-input resolvers can handle ident-based queries. Thus, the following ident-join queries already work without having to define anything else:

(parser {} [{[:product/id 1] [:product/brand]}])
; => {[:product/id 1] #:product{:brand "Taylor"}}

(parser {} [{[:product/brand "Taylor"] [:product/brand-id]}])
; => {[:product/brand "Taylor"] #:product{:brand-id 44151}}

3.3.5. Multiple inputs

The input to a resolver is a set, and as such you can require more than one thing as input to your resolvers. When doing so, of course, your resolver function will receive all of the inputs requested; however, this also means that the parsing context needs to contain them, or there must exist other resolvers that can use what’s in the context to fill them in.

As you have seen before, the only way to provide ad-hoc information to connect is using the ident query, but in the ident itself you can only provide one attribute at a time.

Since version 2.2.0-beta11 the ident readers from connect (ident-reader and open-ident-reader) support adding extra context to the query using parameters, so let’s say you want to load some customer data, but wants to provide some base information that you already have to reduce the number of resolvers called, you can issue a query like this:

[{([:customer/id 123] {:pathom/context {:customer/first-name "Foo" :customer/last-name "Bar"}})
  [:customer/full-name]}]

3.3.6. N+1 Queries and Batch resolvers

When you have a to-many relation that is being resolved by a parser you will typically end up with a single query that finds the "IDs", and then N more queries to fill in the details of each item in the sequence. This is known as the N+1 problem, and can be a source of significant performance problems.

The solution is relatively simple in Pathom: Add the request-cache-plugin to the parser and include a ::pc/batch? true option in your resolver. Pathom then knows that it can send a sub-resolver a sequence of inputs instead of a single one. If your resolver receives a sequence, then it is expected to return a vector of promised outputs instead of a single map.

You must detect if the input is a sequence and also be ready to handle the case when it is not.

Let’s see one example to illustrate the situation:

This example is using Pathom async parsers. The resolvers in async parsers can return channels that (eventually) resolve to the result, which is why you see go blocks in the code. See Async Parsing for more details. We use them in this example so we can "sleep" in a Javascript environment to mimic overhead in processing. In the rest of the book we recommend using the parallel parser, the reason to use the async parser in this example is that it more easely demonstrates the n+1 issue.
link:../src-docs/com/wsscode/pathom/book/connect/batch.cljs[role=include]

Try the example:

[{:items [:number-added]}]

You can note by the tracer that it took one second for each entry, a clear cascade, because it had to call the :number-added resolver once for each item.

We can improving that by turning this into a batch resolver, like this:

link:../src-docs/com/wsscode/pathom/book/connect/batch2.cljs[role=include]

Try the example:

[{:items [:number-added]}]

Note that this time the sleep of one second only happened once, this is because when Pathom is processing a list and the resolver supports batching, the resolver will get all the inputs in a single call, so your batch resolver can get all the items in a single iteration. The results will be cached back for each entry, this will make the other items hit the cache instead of calling the resolver again.

Note the request-cache plugin is required for the batch to work.

3.4. Connect mutations

Using mutations from connect will give you some extra leverage by adding the mutation information to the index, this will enable auto-complete features for explorer interfaces, and also integrates the mutation result with the connect read engine.

3.4.1. Mutations setup

The mutation setup looks very much like the one from resolvers, you define then using pc/defmutation and include on the registry like resolvers.

(ns com.wsscode.pathom.book.connect.mutations
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.connect :as pc]))

(pc/defmutation my-mutation [env params] ...)

(def parser
  (p/parallel-parser
    {::p/env     {::p/reader [p/map-reader
                              pc/parallel-reader
                              pc/open-ident-reader]}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register send-message})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

Now let’s write a mutation with our factory.

3.4.2. Creating mutations

The defmutation have the same interface that we used with defresolver.

link:../src-docs/com/wsscode/pathom/book/connect/mutations.cljs[role=include]
[(send-message {:message/text "Hello Clojurist!"})]

The ::pc/params is currently a non-op, but in the future it can be used to validate the mutation input, it’s format is the same as output (considering the input can have a complex data shape). The ::pc/output is valid and can be used for auto-complete information on explorer tools.

Mutation joins

After doing some operation, you might want to read information about the operation result. With connect you can leverage the resolver engine to expand the information that comes from the mutation. To do that you do a mutation join, and use that to query the information. Here is an example where we create a new user and retrieve some server information with the output.

link:../src-docs/com/wsscode/pathom/book/connect/mutation_join.cljs[role=include]
[{(user/create {:user/name "Rick Sanches" :user/email "rick@morty.com"}) [:user/id :user/name :user/created-at]}]

Note that although we only return the :user/id from the mutation, the resolvers can walk the graph and fetch the other requested attributes.

Mutation join globals

Some attributes need to be in the output, even when they are not asked for. For example, if your parser is driving a Fulcro app, the :tempid part of the mutation will be required for the app to remap the ids correctly. We could ask for the user to add it on every remote query, but instead we can also define some global attributes and they will be read every time. As in this example:

link:../src-docs/com/wsscode/pathom/book/connect/mutation_join_globals.cljs[role=include]
[{(user/create {:user/id "TMP_ID" :user/name "Rick Sanches" :user/email "rick@morty.com"}) [:user/id :user/name :user/created-at]}]

So in case of fulcro apps you can use the :fulcro.client.primitives/tempids as the global and have that pass though.

3.4.3. Async mutations

This section is no longer nescessary, the main recommendation now is to use the parallel-parser which is async, no changes needed to write async mutations, all you gotta know is that you can return channels from your mutations and they will be properly coordinated.

Here is an example of doing some mutation operations using async features.

Example:

link:../src-docs/com/wsscode/pathom/book/connect/mutation_async.cljs[role=include]
[{(user/create {:user/id "TMP_ID" :user/name "Rick Sanches" :user/email "rick@morty.com"}) [:user/id :user/name :user/created-at]}]

Using the same query/mutation interface, we replaced the underlying implementation from an atom to a indexeddb database.

You can do the same to target any type of API you can access.

3.5. Shared resolvers

Since version 2.2.0 Pathom adds support to describe resolvers as pure maps and register those resolvers in your system, making possible to write easy to share resolvers in a library format.

3.5.1. Resolver data format

Here is an example of how you can specific a resolver using the map format:

(def some-resolver
  {::pc/sym     `some-resolver ; this is important! we need to name each resolver, prefer qualified symbols
   ::pc/input   #{:customer/id}
   ::pc/output  [:customer/id :customer/name :customer/email]
   ::pc/resolve (fn [env input] ...)})

As you can see it’s very similar to using defresolver, you just add the key ::pc/resolve to define the runner function of it.

You can also create using the pc/resolver helper function:

(def some-resolver
  (pc/resolver `some-resolver
    {::pc/input #{:customer/id}
     ::pc/output [:customer/id :customer/name :customer/email]}
    (fn [env input] ...)))

This just returns the same map of the previous example.

And using the final macro helper (recommended way):

(pc/defresolver some-resolver
  {::pc/input  #{:customer/id}
   ::pc/output [:customer/id :customer/name :customer/email]}
  (fn [env input] ...))

3.5.2. Mutation data format

Mutations are similar as well:

(def send-message-mutation
  {::pc/sym    `send-message-mutation
   ::pc/params #{:message/body}
   ::pc/output [:message/id :message/body :message/created-at]
   ::pc/mutate (fn [env params] ...)})

As you can see it’s very similar to using defresolver, you just add the key ::pc/resolve to define the runner function of it.

Using the helper:

(def send-message-mutation
  (pc/mutation `send-message-mutation
    {::pc/params #{:message/body}
     ::pc/output [:message/id :message/body :message/created-at]}
    (fn [env params] ...)))

And using the final macro helper (recommended way):

(pc/defmutation send-message-mutation
  {::pc/params #{:message/body}
   ::pc/output [:message/id :message/body :message/created-at]}
  (fn [env params] ...))

3.5.3. Using register

Once you have your maps ready you can register then using the connect register function to the index.

(-> {}
    ; register the resolver we created previously
    (pc/register some-resolver)

    ; same method works for mutations
    (pc/register send-message-mutation)

    ; you can also send collections to register many at once
    (pc/register [some-resolver send-message-mutation])

    ; collections will be recursivelly processed, so this is valid too:
    (pc/register [some-resolver [send-message-mutation]]))

; in the end the index will have the information combined of all the resolvers and mutations

If you are a library author, consider defining each resolver/mutation as its own symbol and then create another symbol that is vector combining your features, this way you make easy for your users to just get the vector, but still allow then to cherry pick which operations he wants to pull if he doesn’t want it all.

3.5.4. Plugins with resolvers

It’s also possible for plugins to declare resolvers and mutations so they get installed when the plugin is used. To do that your plugin must provide the ::pc/register key on the plugin map, and you also need to use the pc/connect-plugin, this plugin will make the instalation, here is an example:

...

(def my-plugin-with-resolvers
 {::pc/register [some-resolver send-message-mutation]})

(def parser
  (p/parser {::p/env     (fn [env]
                           (merge
                             {::p/reader [p/map-reader pc/reader pc/open-ident-reader]}
                             env))
             ::p/mutate  pc/mutate-async
             ::p/plugins [(pc/connect-plugin) ; make sure connect-plugin is here, it's order doesn't matter
                          my-plugin-with-resolvers]}))

And that’s it, the resolvers will be registered right after the parser is defined.

3.6. Advanced Connect Details

3.6.1. Connect readers

pc/parallel-reader

Parallel reader from connect is implemented to work with the paralle-parser. This reader is capable of detecting attribute dependencies, execute multiple in parallel and coordinate the return, including back tracking for secondary paths. Here is how it works:

Getting back to the connect basic idea, that we expand information from a context, to illustrate this case let’s have the following set of resolvers:

(pc/defresolver movie-details [env input]
  {::pc/input  #{:movie/id}
   ::pc/output [:movie/id :movie/title :movie/release-date]}
  ...)

(pc/defresolver movie-rating [env input]
  {::pc/input  #{:movie/id}
   ::pc/output [:movie/rating]}
  ...)

(pc/defresolver movie-title-prefixed [env input]
  {::pc/input  #{:movie/title}
   ::pc/output [:movie/title-prefixed]}
  ...)

Note that we have two resolvers that depend on a :movie/id and one that depends on :movie/title.

Now given the query: [{[:movie/id 42] [:movie/title-prefixed]}]

First we use the ident query to create the context with a :movie/id, for the attribute :movie/title-prefixed the parallel-reader will be invoked. The first thing the reader has to do is compute a plan to reach the attribute considering the data it has now, it does it by recursively iterating over the ::pc/index-oir until it reaches some available dependency or gives up because there is no possible path.

Most cases (specially for small apis) there will be only a single path, and this is the case for our example the result of pc/compute-path is this:

#{[[:movie/title `movie-details] [:movie/title-prefixed `movie-title-prefixed]]}

The format returned by pc/compute-path is a vector of paths, each path is a vector of tuples, the tuple contains the attribute reason (why that resolver is been called) and the resolver symbol that will be used to fetch that attribute, this makes the path from the available data to the attribute requested, this is the plan.

For details on the path selection algorithm in cases of multiple options check the paths selection section.

Ok, now let’s see how it behaves when you have multiple attributes to process, this is the new query, this time let’s try using the interactive parser, run the query and check in the tracing how it goes (I added a 100ms delay to each resolver call so its easier to see):

[{[:movie/id 42] [:movie/id :movie/title :movie/release-date :movie/rating :movie/title-prefixed]}]
Try changing the order of the attributes and see what happens, for example if you put :movie/title-prefixed at start you will this attribute been responsible for the title fetching and itself.

This is what’s happening for each attribute:

:movie/id: this data is already in the entity context, this means it will be read from memory and will not even invoke the parallel reader

:movie/title: this attribute is not on entity, so it will create the plan to call movie-details from this plan we can also compute all the attributes we will incorporate in the call chain (by combining the outs of all the resolvers in the path), we store this information as a waiting list. The waiting list on this case is: [:movie/id :movie/title :movie/releast-date]. The processing of attributes continues in parallel while the resolver is called.

:movie/release-date: this attribute is not on entity, but it is in the waiting list, so the parser will ignore it for now and skip to process the next one.

:movie/rating: this attribute is not in entity, neighter in the waiting list, so we can call the resolver for it immediatly, and the plan output ([:movie/rating]) is appended to the waiting list.

:movie/title-prefixed: like the rating this is not in entity or waiting, so we compute the plan and execute, the plan is again:

#{[[:movie/title `movie-details] [:movie/title-prefixed `movie-title-prefixed]]}

But movie-details is already running because of :movie/title, when the parallel-reader calls a resolver, it actually caches it immediatly as a promise channel in the request cache, so when we hit the same resolver with the same input, it hits the cache, getting a hold of the promise channel, so the process continues normally with only one actual call to the resolver, but two listeners on the promise channel (and any posterior cache hit would get to this same promise channel). This is how the data fetch is coordinated across the attributes, placeholder nodes are also supported and optimized to avoid repeated calls to resolvers.

Another difference is during processing of sequences, the parallel parser uses core.async pipeline to process each sequence with a parallelism concurrency of 10.

Path selection

In case there are multiple possible paths Pathom has to decide which path to take, the current implementation chooses the path with less weight, that calculation is made in this way:

  1. Every resolver starts with weight 1 (this is recorded per instance)

  2. Once a resolver is called, it’s execution time is recorded and updated in the map using the formula: new-value = (old-value + last-time) * 0.5

  3. If a resolver call throws an exception, double it’s weight

  4. Every time we mention some resolver in a path calculation its weight is reduced by one.

If you like to make your own sorting of the plan, you can set the key ::pc/sort-plan in your environment, and Pathom will call this function sort the results, it takes the environment and the plan (which is a set like demonstrate in the previous section).

pc/reader2

This reader leverages some tecniques that were develop during the creationg of the parallel reader, things like path choosing and backtracking.

pc/async-reader2

Like pc/reader2 but knows how to handle async processing inside.

pc/reader [DEPRECATED]

DEPRECATED: use pc/reader2 instead

The main Connect reader. This will look up the attribute in the index and try to resolve it, recursively if necessary.

pc/async-reader [DEPRECATED]

DEPRECATED: use pc/async-reader2 instead

Like pc/reader but knows how to handle async processing inside.

pc/ident-reader

The ident-reader is used to resolve ident-based queries by establishing an initial context from the ident. When an ident query reaches this reader it will check the index to see if the ident key is present on in the indexed idents.

Since version 2.2.0-beta11 this reader also supports extra context provision using the param :pathom/context, here is how to send extra data to it:

[{([:user/id 123] {:pathom/context {:other/data 123}})
  [:user/id :user/name :other/data]}]
pc/open-ident-reader

Like ident-reader, but not constrained to the indexed idents, this will create a context from any ident.

pc/index-reader

This reader exposes the index itself with the name ::pc/indexes.

3.6.2. Understanding the indexes

Connect maintains a few indexes containg information about the resolvers and the relationships on attributes. Connect will look up the index in the environment, on the key :com.wsscode.pathom.connect/indexes, which is a map containing the indexes

In order to explain the different indexes we’ll look at the index generated by our example in the getting started section:

{::pc/index-resolvers
 {get-started/latest-product
  {::pc/sym     get-started/latest-product
   ::pc/input   #{}
   ::pc/output  [{::get-started/latest-product [:product/id
                                                :product/title
                                                :product/price]}]
   ::pc/resolve (fn ...)}

  get-started/product-brand
  {::pc/sym     get-started/product-brand
   ::pc/input   #{:product/id}
   ::pc/output  [:product/brand]
   ::pc/resolve (fn ...)}

  get-started/brand-id-from-name
  {::pc/sym     get-started/brand-id-from-name
   ::pc/input   #{:product/brand}
   ::pc/output  [:product/brand-id]
   ::pc/resolve (fn ...)}}

 ::pc/index-oir
 {:get-started/latest-product {#{} #{get-started/latest-product}}
  :product/brand              {#{:product/id} #{get-started/product-brand}}
  :product/brand-id           {#{:product/brand} #{get-started/brand-id-from-name}}}

 ::pc/index-io
 {#{}               {:get-started/latest-product #:product{:id {} :title {} :price {}}}
  #{:product/id}    {:product/brand {}}
  #{:product/brand} {:product/brand-id {}}}

 ::pc/idents
 #{:product/brand :product/id}}
index-resolvers

This is a raw index of available resolvers, it’s a map resolver-sym → resolver-data. resolver-data is any information relevant that you want to add about that resolver. Any key that you adding during pc/add will end up on this map, also Connect will add the key ::pc/sym automatically, which is the same symbol you added. If you want to access the data for a resolver, Connect provides a helper function for that:

(pc/resolver-data env-or-indexes `product-brand)
; => {::pc/sym     get-started/product-brand
;     ::pc/input   #{:product/id}
;     ::pc/output  [:product/brand]
;     ::pc/resolve (fn ...)}
index-oir

This index stands for output → input → resolver. It’s the index used for the Connect reader to look up attributes. This index is built by looking at the input/output for the resolver when you add it. It will save that resolver as a path to each output attribute, given that input. It basically inverts the order of things: it keys the output attribute to all of the potential "starting points".

Let’s do an exercise and see how connect traverses this index in a practical example:

Given we have this index (oir):

link:../src-docs/com/wsscode/pathom/book/connect/index_oir_example.cljc[role=include]

Now if you try to run the query:

[:name]

So we look in the index for :name, and we get {#{:id} #{thing-by-id}}, now we try to match the current entity attribute keys with the sets to see if we have enough data to call any of them. If we don’t it will fail because we don’t have enough data.

[{[:id 123] [:name]}]

So, if we start with an ident, our initial context is {:id 123}. This time we have the :id, so it will match with the input set #{:id}, and will call the resolver thing-by-id with that input to figure out the name. Connect uses atom entities: when it gets the return value from the resolver it merges it back into the context entities, making all data returned from the resolver available to access new attributes as needed.

index-io

The auto-complete index, input → output. This index accumulates the reach for each single attribute on the index. By walking this information we can know ahead of time all attribute possibilities we can fetch from a given attribute.

If I have a :product/id, what can I reach from it? Looking at the index, the :product/id itself can provide the :product/brand. But if I have access to :product/brand it means I also have access to whatever :product/brand can provide. By doing multiple iterations (until there are no new attributes) we end up knowing that :product/id can provide the attributes :product/brand and :product/brand-id. And this is how autocomplete is done via the index-io.

index-mutations

This index contains the mutation definitions, its similar to the index-resolvers.

idents

The idents index contain information about which single attributes can be used to access some information. This index is used on ident-reader and on OgE to provide auto-complete options for idents. Any time you add a resolver that has a single input, that input attribute is added on the idents index.

autocomplete-ignore

This index is for a more advanced usage. Currently it’s only used by the GraphQL integration. In the GraphQL integration we leverage the fact that types have a fixed set of attributes and add that into the index. The problem is that the types thenselves are not valid entries for the query, then autocomplete-ignore is a way to make those things be ignored in the auto-complete. You probably only need this if you are building the index in some custom way.

4. Getting Started on Pathom core engine

4.1. Query Notation Introduction

A query is a vector that lists the items you want. A keyword requests a scalar (opaque) value, and a map indicates a to-many or to-one join (resolved at runtime using database content).

Queries are always "relative" to some starting context (which is typically supplied via parameters or by a top-level join).

If you want to obtain the name and age of "some" person:

[:person/name :person/age]

If you want to obtain a person’s name and the street of their address you might write this:

[:person/name {:person/address [:address/street]}]

where we imagine that the underlying database has some kind of normalization that needs to be traversed in order to satisfy the address data.

The result of running a query is a map containing the result (in the same recursive shape as the query):

Running [:person/name :person/age] against the person "Sam" might give:

{:person/name "Sam" :person/age 32}

Running [:person/name {:person/address [:address/street]}] against that same person might give:

{:person/name "Sam" :person/address {:address/street "111 Main St."}}

The query establishes the request and expectation. Interpreting and satisfying these queries from some arbitrary data source is the job of a query parser/interpreter. This library gives you tools for quickly building the latter.

4.2. Parsing Context

The elements of a graph query are relative: they have a contextual meaning. If you ask for a person’s name, the implication is that you are querying a "person entity"; however, the other required bit of information is which person. Thus, elements of a query cannot be fulfilled they are rooted in a context. This applies to joins as well (e.g. what is the current person’s address?), but once you’ve resolved the context of the root of some graph query the joins simply describe navigation from that context (the person) to another (their address) via a relation that is either already described in the underlying data source itself, or in code you provide that can figure it out.

As the parser moves through a query like [:person/name {:person/address [:address/street]}] it first starts with some context (e.g. "Sam"). When it finds a join it processes the subquery against a new context (e.g. Sam’s address) to give the result:

{:person/name "Sam" :person/address {:address/street "111 Main St."}}

So, there is always a context at any given point when parsing a query. This context is either established at startup by resolving a specific entity, or is the entity (or entities if to-many) that have been reached by processing the joins of the query.

4.3. Parsing Environment and The Reader

The parsing environment is simply a map that carries along data while parsing (and can be augmented as you go). It establishes the meaning of the "current context", can contain anything you wish (via namespaced keywords), and can be seen in any code that you plug in to process the query.

There are some predefined (namespaced) keys that have special meaning to the parser. In particular :com.wsscode.pathom.core/reader can be used to supply reader(s) for the parser to use. The reader can be a map from attributes to functions, a plain function, or even a vector of functions. It is asked to read the value for the elements of the query using the current environment. We’ll expand on that as we go, or you can read more in the Readers section.

5. Pathom Core Engine

5.1. Parsers

5.1.1. Serial parser

TODO: explain serial parser internals

5.1.2. Async parser

TODO: explain async parser internals

5.1.3. Parallel parser

TODO: explain parallel parser internals

5.2. Readers

A reader is a function that will process a single entry from the query. For example, given the following query: [:name :age]. If you ask an om.next parser to read this the reader function will be called twice; once for :name and another one for :age. Note that in the case of joins, the parser will only be called for the join entry, but not for it’s children (not automatically), for example: given the query [:name :age {:parent [:name :gender]}]. The reader function will be called 3 times now, one for :name, one for :age and one for :parent, when reading :parent, your reader code is responsible for checking that it has a children query, and do a recursive call (or anything else you want to do to handle this join). During this documentation, we are going to see many ways to implement those readers.

Please note the following differences between om.next readers and pathom readers: In om.next a parse read functions has the following signature: (fn [env dispatch-key params]). In pathom we use a smaller version instead, which is: (fn [env]). The env already contains the dispatch-key and params, so there is no loss of information.

(get-in env [:ast :dispatch-key]) ; => dispatch-key
(get-in env [:ast :params]) ; => params

Also, in om.next you need to return the value wrapped in {:value "your-content"}. In pathom this wrapping is done automatically for you: just return the final value.

Readers can be 1-arity function, maps, or vectors. See Map dispatcher and Vector dispacher for information on those respectively.

Here is a formal Clojure Spec definiton for a pathom reader:

(s/def ::reader-map (s/map-of keyword? ::reader))
(s/def ::reader-seq (s/coll-of ::reader :kind vector?))
(s/def ::reader-fn (s/fspec :args (s/cat :env ::env)
                            :ret any?))

(s/def ::reader
  (s/or :fn ::reader-fn
        :map ::reader-map
        :list ::reader-seq))

5.2.1. Functions as Readers

These are quite simply a function that receive the env and resolve the read. More than one reader can exist in a chain, and the special return value ::p/continue allows a reader to indicate it cannot resolve the given property (to continue processing the chain). Returning any value (including nil) you’ve resolved the property to that value.

(ns pathom-docs.fn-dispatch
  (:require [com.wsscode.pathom.core :as p]))

(defn read-value [{:keys [ast]}]
  (let [key (get ast :dispatch-key)]
    (case key
      :name "Saul"
      :family "Goodman"
      ; good pratice: return ::p/continue when your reader is unable
      ; to handle the request
      ::p/continue)))

(def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader read-value})]}))

(parser {} [:name :family])
; => {:name "Saul" :family "Goodman"}

5.2.2. Maps as Readers

Since it is very common to want to resolve queries from a fixed set of possibilities we support defining a map as a reader. This is really just a "dispatch table" to functions that will receive env. We can re-write the previous example as:

(ns pathom-docs.reader-map-dispatch
  (:require [com.wsscode.pathom.core :as p]))

(def user-reader
  {:name   (fn [_] "Saul")
   :family (fn [_] "Goodman")})

(def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader user-reader})]}))

(parser {} [:name :family])
; => {:name "Saul" :family "Goodman"}
The built-in Map Reader will return ::p/continue if the map it is looking in does not contain the key for the attribute being resolved. This allows it to be safely used in a vector of readers.

5.2.3. Vectors of Readers [aka composed readers]

Using a vector for a reader is how you define a chain of readers. This allows you to define readers that serve a particular purpose. For example, some library author might want to supply readers to compose into your parser, or you might have different modules of database-specific readers that you’d like to keep separate.

When pathom is trying to resolve a given attribute (say :person/name) in some context (say against the "Sam" entity) it will start at the beginning of the reader chain. The first reader will be asked to resolve the attribute. If the reader can handle the value then it will be returned and no other readers will be consulted. If it instead returns the special value ::p/continue it is signalling that it could not resolve it (map readers do this if the attribute key is not in their map). When this happens the next reader in the chain will be tried.

(ns pathom-docs.reader-vector-dispatch
  (:require [com.wsscode.pathom.core :as p]))

; a map dispatcher for the :name key
(def name-reader
  {:name   (fn [_] "Saul")})

; a map dispatcher for the :family key
(def family-reader
  {:family (fn [_] "Goodman")})

(def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader [name-reader family-reader]})]}))

(parser {} [:name :family :other])
; => {:name "Saul", :family "Goodman", :other :com.wsscode.pathom.core/not-found}

If no reader in the chain returns a value (all readers reeturn ::p/continue), then ::p/not-found will be returned.

When you write your readers you should always remember to return ::p/continue when you can’t handle a given key. This way your reader will play nice in composition scenarios.

5.2.4. The Map Reader

Not all things need to be computed. Very often the current context will already have attributes that were read during some prior step (for example, a computed attribute might have read an entire entity from the database and made it the current context). The map reader plugin is a plugin that has the following behavior:

  • If the attribute requested exists in the current parsing context (with any value, even nil), it returns it.

  • If the attribute is missing, it returns ::p/continue, which is an indication to move to the next reader in the chain.

  • If the attribute is present, it properly returns the value.

The map reader is also capable of resolving relations (if present in the context). For example, if there is a join in the query and a vector of data at that join key in the context, then it will attempt to fulfill the subquery of the join.

The map reader is almost always inserted into a reader chain because it is so common to read clumps of things from a database into the context and resolve them one by one as the query parsing proceeds.

5.3. Entities

An entity to pathom is the graph node that is tracked as the current context, and from which information (attributes and graph edges to other entities) can be derived. The current entity needs to be "map-like": It should work with all normal map-related functions like get, contains?, etc.

As Pathom parses the query it tracks the current entity in the environment at key ::p/entity. This makes it easier to write more reusable and flexible readers as we’ll see later.

5.3.1. Using p/entity

The p/entity function exists as a convenience for pulling the current entity from the parsing environment:

(ns com.wsscode.pathom-docs.using-entity
  (:require [com.wsscode.pathom.core :as p]))

(defn read-attr [env]
  (let [e (p/entity env)
        k (get-in env [:ast :dispatch-key])]
    (if (contains? e k)
      (get e k)
      ::p/continue)))

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [read-attr]})]}))

; we send the entity using ::p/entity key on environment
(parser {::p/entity #:character{:name "Rick" :age 60}} [:character/name :character/age :character/foobar])
; => #:character{:name "Rick", :age 60, :foobar :com.wsscode.pathom.core/not-found}

Note that the code above is a partial implementation of the map-dispatcher.

The map-reader just has the additional ability to understand how to walk a map that has a tree shape that already "fits" our query:

(ns com.wsscode.pathom-docs.using-entity-map-reader
  (:require [com.wsscode.pathom.core :as p]))

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader p/map-reader})]}))

; we send the entity using ::p/entity key on environment
(parser {::p/entity #:character{:name "Rick" :age 60
                                :family [#:character{:name "Morty" :age 14}
                                         #:character{:name "Summer" :age 17}]
                                :first-episode #:episode{:name "Pilot" :season 1 :number 1}}}
        [:character/name :character/age
         {:character/family [:character/age]}
         {:character/first-episode [:episode/name :episode/number]}])
; =>
; #:character{:name "Rick",
;             :age 60,
;             :family [#:character{:age 14} #:character{:age 17}],
;             :first-episode #:episode{:name "Pilot", :number 1}}

Now that you understand where the entity context is tracked I encourage you to check the p/map-reader implementation. It’s not very long and will give you a better understanding of all of the concepts covered so far.

5.3.2. Understanding Joins

The other significant task when processing a graph query is walking a graph edge to another entity (or entities) when we find a join.

The subquery for a join is in the :query of the environment. Essentially it is a recursive step where we run the parser on the subquery while replacing the "current entity":

(defn join [entity {:keys [parser query] :as env}]
  (parser (assoc env ::p/entity entity) query))

The real pathom implementation handles some additional scenarios: like the empty sub-query case (it returns the full entity), the special * query (so you can combine the whole entity + extra computed attributes), and union queries.

The following example shows how to use p/join to "invent" a relation that can then be queried:

(ns com.wsscode.pathom-docs.using-entity-map-reader
  (:require [com.wsscode.pathom.core :as p]))

(def rick
  #:character{:name          "Rick"
              :age           60
              :family        [#:character{:name "Morty" :age 14}
                              #:character{:name "Summer" :age 17}]
              :first-episode #:episode{:name "Pilot" :season 1 :number 1}})

(def char-name->voice
  "Relational information representing edges from character names to actors"
  {"Rick"   #:actor{:name "Justin Roiland" :nationality "US"}
   "Morty"  #:actor{:name "Justin Roiland" :nationality "US"}
   "Summer" #:actor{:name "Spencer Grammer" :nationality "US"}})

(def computed
  {:character/voice ; support an invented join attribute
   (fn [env]
     (let [{:character/keys [name]} (p/entity env)
           voice (get char-name->voice name)]
       (p/join voice env)))})

(def parser
  ; process with map-reader first, then try with computed
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader computed]})]}))

(parser {::p/entity rick} ; start with rick (as current entity)
        '[:character/name
          {:character/voice [:actor/name]}
          {:character/family [* :character/voice]}])

There are three different scenarios demonstrated in the above query:

  1. Using the invented join property in a normal join. This allows for a subquery that constrains the data returned (from the actor in this case).

  2. Using the * in a query, which returns all "known" attributes of the "current contextual" entity.

  3. Using an additional (non-joined) :character/voice with * "adds in" that additional information. When a property is queried for that is processed via p/join then the entire entity will be returned even though there is no subquery.

5.3.3. Dependent Attributes

When computing attributes it is possible that you might need some other attribute for the current context that is also computed. You could hard-code a solution, but that would create all sorts of static code problems that could be difficult to manage as your code evolves: changes to the readers, for example, could easily break it and lead to difficult bugs.

Instead, it is important that readers be able to resolve attributes they need from the "current context" in an abstract manner (i.e. the same way that they query itself is being resolved). The p/entity function has an additional arity for handling this exact case. You pass it a list of attributes that should be "made available" on the current entity, and it will use the parser to ensure that they are there (if possible):

(let [e (p/entity env [:x])]
   ; e now has :x on it if possible, even if it is computed elsewhere
   ...)

The following example shows this in context:

(ns pathom-docs.entity-attribute-dependency
  (:require [com.wsscode.pathom.core :as p]))

(def computed
  {:greet
   (fn [env]
     (let [{:character/keys [name]} (p/entity env)]
       (str "Hello " name "!")))

   :invite
   (fn [env]
     ; requires the computed property `:greet`, which might not have been computed into the current context yet.
     (let [{:keys [greet]} (p/entity env [:greet])]
       (str greet " Come to visit us in Neverland!")))})

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader
                                                     computed]})]}))

(parser {::p/entity #:character{:name "Mary"}}
        [:invite])
; => {:invite "Hello Mary! Come to visit us in Neverland!"}

There is a variant p/entity! that raises an error if your desired attributes are not found. It’s recommended to use the enforced version if you need the given attributes, as it will give your user a better error message.

(ns pathom-docs.entity-attribute-enforce
  (:require [com.wsscode.pathom.core :as p]))

(def computed
  {:greet
   (fn [env]
     ; enfore the character/name to be present, otherwise raises error, try removing
     ; the attribute from the entity and see what happens
     (let [name (p/entity-attr! env :character/name)]
       (str "Hello " name "!")))

   :invite
   (fn [env]
     ; now we are enforcing the attribute to be available, otherwise raise an error
     ; try changing the :greet to :greete and run the file, you will see the error
     (let [greet (p/entity-attr! env :greet)]
       (str greet " Come to visit us in Neverland!")))})

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader
                                                     computed]})]}))

(parser {::p/entity #:character{:name "Mary"}}
        [:invite])
; => {:invite "Hello Mary! Come to visit us in Neverland!"}

If the parse fails on an enforced attribute you will get an exception. For example, if the current entity were #:character{:nam "Mary"} we’d see:

CompilerException clojure.lang.ExceptionInfo: Entity attributes #{:character/name} could not be realized #:com.wsscode.pathom.core{:entity #:character{:nam "Mary"}, :path [:invite :greet], :missing-attributes #{:character/name}}
If computed attributes require IO or intense computation you should consider adding caching to improve parsing performance. Remember that a given query might traverse the same node more than once! Imagine a query that asks for your friends and co-workers. When there is this kind of overlap the same computational code may run more than once. See Request Caching for more details.

5.3.4. Atom entities

As you move from node to node, you can choose to wrap the new contextual entity in an atom. This can be used as a narrow kind of caching mechanism that allows for a reader to add information into the current entity as it computes it, but which is valid for only the processing of the current entity (is lost as soon as the next join is followed). Therefore, this won’t help with the overhead of re-visiting the same entity more than once when processing different parts of the same query.

The built-in function p/entity always returns a Clojure map, if the entity is an atom it will deref it automatically.

Here is an example using an entity atom:

link:../src-docs/com/wsscode/pathom/book/entities/atom_entities.cljc[role=include]

5.3.5. Union queries

Union queries allow us to handle edges that lead to heterogeneous nodes. For example a to-many relation for media that could result in a book or movie. Following such an edge requires that we have a different subquery depending on what we actually find in the database.

Here is an example where we want to use a query that will search to find a user, a movie or a book:

(ns pathom-docs.entity-union
  (:require [com.wsscode.pathom.core :as p]))

(def search-results
  [{:type :user
    :user/name "Jack Sparrow"}
   {:type :movie
    :movie/title "Ted"
    :movie/year 2012}
   {:type :book
    :book/title "The Joy of Clojure"}])

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader]})]}))

(parser {::p/entity {:search search-results}
         ; here we set where pathom should look on the entity to determine the union path
         ::p/union-path :type}
        [{:search {:user [:user/name]
                   :movie [:movie/title]
                   :book [:book/title]}}])

Of course, unions need to have a way to determine which path to go based on the entity at hand. In the example above we used the :type (a key on the entity) to determine which branch to follow. The value of ::p/union-path can be a keyword (from something inside entity or a computed attribute) or a function (that takes env and returns the correct key (e.g. :book) to use for the union query).

If you want ::p/union-path to be more contextual you can of course set it in the env during the join process, as in the next example:

(ns pathom-docs.entity-union-contextual
  (:require [com.wsscode.pathom.core :as p]))

(def search-results
  [{:type :user
    :user/name "Jack Sparrow"}
   {:type :movie
    :movie/title "Ted"
    :movie/year 2012}
   {:type :book
    :book/title "The Joy of Clojure"}])

(def search
  {:search
   (fn [env]
     ; join-seq is the same as join, but for sequences, note we set the ::p/union-path
     ; here. This is more common since the *method* of determining type will vary for
     ; different queries and data.
     (p/join-seq (assoc env ::p/union-path :type) search-results))})

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [search
                                                     p/map-reader]})]}))

(parser {}
        [{:search {:user [:user/name]
                   :movie [:movie/title]
                   :book [:book/title]}}])

This is something beautiful about having an immutable environment; you can make changes with confidence that it will not affect indirect points of the parsing process.

5.4. Error handling

By default, pathom parser will stop if some exception occurs during the parsing process. This is often undesirable if some node fails you still can return the other ones that succeed. You can use the error-handler-plugin. This plugin will wrap each read call with a try-catch block, and in case an error occurs, a value of ::p/reader-error will be placed in that node, while details of it will go in a separate tree, but at the same path. Better an example to demonstrate:

(ns pathom-docs.error-handling
  (:require [com.wsscode.pathom.core :as p]))

(def computed
  ; create a handle key that will trigger an error when called
  {:trigger-error
   (fn [_]
     (throw (ex-info "Error triggered" {:foo "bar"})))})

; a reader that just flows, until it reaches a leaf
(defn flow-reader [{:keys [query] :as env}]
  (if query
    (p/join env)
    :leaf))

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [computed flow-reader]})
                          ; add the error handler plugin
                          p/error-handler-plugin]}))

(parser {} [{:go [:key {:nest [:trigger-error :other]}
                  :trigger-error]}])
; =>
; {:go {:key :leaf
;       :nest {:trigger-error :com.wsscode.pathom.core/reader-error
;              :other :leaf}
;       :trigger-error :com.wsscode.pathom.core/reader-error}
;  :com.wsscode.pathom.core/errors {[:go :nest :trigger-error] "class clojure.lang.ExceptionInfo: Error triggered - {:foo \"bar\"}"
;                                   [:go :trigger-error] "class clojure.lang.ExceptionInfo: Error triggered - {:foo \"bar\"}"}}

As you can see, when an error occurs, the key ::p/errors will be added to the returned map, containing the detailed error message indexed by the error path. You can customize how the error is exported in this map by setting the key ::p/process-error in your environment:

(ns pathom-docs.error-handling-process
  (:require [com.wsscode.pathom.core :as p]))

(def computed
  ; create a handle key that will trigger an error when called
  {:trigger-error
   (fn [_]
     (throw (ex-info "Error triggered" {:foo "bar"})))})

; a reader that just flows, until it reaches a leaf
(defn flow-reader [{:keys [query] :as env}]
  (if query
    (p/join env)
    :leaf))

; our error processing function
(defn process-error [env err]
  ; if you use some error reporting service, this is a good place
  ; to trigger a call to then, here you have the error and the full
  ; environment of when it ocurred, so you might want to some extra
  ; information like the query and the current path on it so you can
  ; replay it for debugging

  ; we are going to simply return the error message from the error
  ; if you want to return the same thing as the default, use the
  ; function (p/error-str err)
  (.getMessage err))

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [computed flow-reader]
                                         ; add the error processing to the environment
                                         ::p/process-error process-error})
                          ; add the error handler plugin
                          p/error-handler-plugin]}))

(parser {} [{:go [:key {:nest [:trigger-error :other]}
                  :trigger-error]}])
; =>
; {:go {:key :leaf
;       :nest {:trigger-error :com.wsscode.pathom.core/reader-error
;              :other :leaf}
;       :trigger-error :com.wsscode.pathom.core/reader-error}
;  :com.wsscode.pathom.core/errors {[:go :nest :trigger-error] "Error triggered"
;                                   [:go :trigger-error]       "Error triggered"}}

5.4.1. Fail fast

Having each node being caught is great for the UI, but not so much for testing. During testing you probably prefer the parser to blow up as fast as possible so you don’t accumulate a bunch of errors that get impossible to read. Having to create a different parser to remove the error-handler-plugin can be annoying, so there is an option to solve that. Send the key ::p/fail-fast? as true in the environment, and the try/catch will not be done, making it fail as soon as an exception fires, for example, using our previous parser:

(parser {::p/fail-fast? true}
        [{:go [:key {:nest [:trigger-error :other]}
               :trigger-error]}])
; => CompilerException clojure.lang.ExceptionInfo: Error triggered {:foo "bar"}, ...

5.4.2. Raising errors

The default error output format (in a separated tree) is very convenient for direct API calls, because they leave a clean output on the data part. But if you want to expose those errors on the UI, pulling then out of the separated tree can be a bit of a pain. To help with that there is a p/raise-errors helper, this will lift the errors so they are present at the same level of the error entry. Let’s take our last error output example and process it with p/raise-errors

(p/raise-errors {:go {:key :leaf
                      :nest {:trigger-error :com.wsscode.pathom.core/reader-error
                             :other :leaf}
                      :trigger-error :com.wsscode.pathom.core/reader-error}
                 :com.wsscode.pathom.core/errors {[:go :nest :trigger-error] "Error triggered"
                                                  [:go :trigger-error] "Error triggered"}})

; outputs:

{:go {:key :leaf
      :nest {:trigger-error :com.wsscode.pathom.core/reader-error
             :other :leaf
             :com.wsscode.pathom.core/errors {:trigger-error "Error triggered"}}
      :trigger-error :com.wsscode.pathom.core/reader-error
      :com.wsscode.pathom.core/errors {:trigger-error "Error triggered"}}}

Notice that we don’t have the root ::p/errors anymore, instead it is placed at the same level of the error attribute. So the path [::p/errors [:go :nest :trigger-error]] turns into [:go :nest ::p/errors :trigger-error]. This makes very easy to pull the error on the client-side.

5.5. Dispatch helpers

Using multi-methods is a good way to make open readers, pathom provides helpers for two common dispatch strategies: key-dispatch and entity-dispatch. Here is a pattern that I often use on parsers:

(ns pathom-docs.dispatch-helpers
  (:require [com.wsscode.pathom.core :as p]))

(def cities
  {"Recife"    {:city/name "Recife" :city/country "Brazil"}
   "São Paulo" {:city/name "São Paulo" :city/country "Brazil"}})

(def city->neighbors
  {"Recife" [{:neighbor/name "Boa Viagem"}
             {:neighbor/name "Piedade"}
             {:neighbor/name "Casa Amarela"}]})

; this will dispatch according to the ast dispatch-key
(defmulti computed p/key-dispatch)

; use virtual attributes to handle data not present on the maps, like computed attributes, relationships, and globals
(defmethod computed :city/neighbors [env]
  (let [name (p/entity-attr! env :city/name)]
    (p/join-seq env (city->neighbors name))))

; an example of global, same as before but without any dependency on the entity
(defmethod computed :city/all [env]
  (p/join-seq env (vals cities)))

; remember to return ::p/continue by default so non-handled cases can flow
(defmethod computed :default [_] ::p/continue)

; just to make easy to re-use, our base entity reader consists of a map reader + virtual attributes
(def entity-reader [p/map-reader computed])

; dispatch for entity keys, eg: [:user/by-id 123]
(defmulti entity-lookup p/entity-dispatch)

(defmethod entity-lookup :city/by-name [env]
  ; the ident-value helper extracts the value part from the ident, as "Recife" in [:city/by-name "Recife"]
  (let [city (get cities (p/ident-value env))]
    (p/join city env)))

(defmethod entity-lookup :default [_] ::p/continue)

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader computed entity-lookup]})]}))

(parser {} [{:city/all [:city/name]}
            {[:city/by-name "Recife"] [:city/neighbors]}])
; =>
;{:city/all [#:city{:name "Recife"} #:city{:name "São Paulo"}]
; [:city/by-name "Recife"] #:city{:neighbors [#:neighbor{:name "Boa Viagem"}
;                                             #:neighbor{:name "Piedade"}
;                                             #:neighbor{:name "Casa Amarela"}]}}

5.6. Mutations

To handle mutations, you can send the :mutate param to the parser.

(ns com.wsscode.pathom.book.mutation
  (:require [com.wsscode.pathom.core :as p]
            [fulcro.client.primitives :as fp]))

(defmulti my-mutate fp/dispatch)

(defmethod my-mutate `do-operation [{:keys [state]} _ params]
  (swap! state update :history conj {:op :operation :params params}))

(def parser (p/parser {:mutate my-mutate}))

(comment
  (let [state (atom {:history []})]
    (parser {:state state} [`(do-operation {:foo "bar"})
                            `(do-operation {:buz "baz"})])
    @state)
  ; => {:history [{:op :operation, :params {:foo "bar"}}
  ;               {:op :operation, :params {:buz "baz"}}]}
  )

5.7. Request Caching

As your queries grow, there are more and more optimizations that you can do avoid unnecessary IO or heavy computations. Here we are going to talk about a request cache, which is a fancy name for an atom that is initialized on every query and stays on the environment so you can share the cache across nodes. Let’s see how we can use that to speed up our query processing:

(ns pathom-docs.request-cache
  (:require [com.wsscode.pathom.core :as p]))

(defn my-expensive-operation [env]
  ; the cache key can be anything; if we were had an extra
  ; variable here, like some id, a good cache key would be
  ; like: [::my-expensive-operation id]
  (p/cached env :my-key
    ; we are going to send an atom with an int so that we can count
    ; how many times this was called
    (let [counter (:counter env)]
      ; a secondary sign if cache is working, let's make a delay
      (Thread/sleep 1000)
      ; increment and return
      (swap! counter inc))))

(def computed
  {:cached my-expensive-operation})

; a reader that just flows, until it reaches a leaf
(defn flow-reader [{:keys [query] :as env}]
  (if query
    (p/join env)
    :leaf))

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [computed
                                                     flow-reader]})
                          ; add the request cache plugin for cache initialization
                          p/request-cache-plugin]}))

(time
  (parser {:counter (atom 0)}
          [:x :y :cached
           {:z [:foo {:bar [:cached]} :cached]}]))
; "Elapsed time: 1006.760165 msecs"
; =>
; {:x      :leaf
;  :y      :leaf
;  :cached 1
;  :z      {:foo    :leaf
;           :bar    {:cached 1}
;           :cached 1}}

Remember this cache is per request, so after a full query gets finished, the atom is discarded. If you want to make a cache that’s more durable (that retains information across requests), check the [[Plugins|Plugins]] documentation for more information on how to do that.

5.8. Plugins

Pathom allows a parser to have a collection of plugins that modify its behavior. Plugins is a top-level option when creating the parser, and the value is a vector of plugins:

(def parser (p/parser {::p/plugins [...]}))

In this section we’ll be using a few plugins to make our lives easier.

Plugins set code that wraps some of pathom operations, a plugin is a map where you bind keys from event names to functions. They work on wrap fashion, kind like ring wrappers. Here is what a plugin looks like:

(ns pathom-docs.plugin-example
  (:require [com.wsscode.pathom.core :as p]))

(def my-plugin
  ; the ::p/wrap-parser entry point wraps the entire parser,
  ; this means it wraps the operation that runs once on each
  ; query that runs with the parser
  {::p/wrap-parser
   (fn [parser]
     ; here you can initialize stuff that runs only once per
     ; parser, like a durable cache across requests
     (fn [env tx]
       ; here you could initialize per-request items, things
       ; that needs to be set up once per query as we do on
       ; request cache, or the error atom to accumulate errors

       ; in this case, we are doing nothing, just calling the
       ; previous parser, a pass-through wrapper if you may
       (parser env tx)))

   ; this wraps the read function, meaning it will run once for
   ; each recursive parser call that happens during your query
   ::p/wrap-read
   (fn [reader]
     (fn [env]
       ; here you can wrap the parse read, in pathom we use this
       ; on the error handler to do the try/catch per node, also
       ; the profiler use this point to calculate the time spent
       ; on a given node

       ; this is also a good point to inject custom read keys if
       ; you need to, the profile plugin, for example, can capture
       ; the key ::p.profile/profile and export the current profile
       ; information
       (reader env)))})

The plugin engine replaces the old process-reader in a much more powerful way. If you want to check a real example look for the source for the built-in plugins, they are quite small and yet powerful tools (grep for -plugin on the repository to find all of them).

5.8.1. The Environment Plugin

Typically the parsing environment will need to include things you create and inject every time you parse a query (e.g. a database connection) and some parser-related things (e.g. the reader) that might be the same all the time.

In the earlier example we created the parser and then explicitly supplied a reader to the environment every time we called it. In cases where there are specific things that you’d always like included in the environment we can instead use the plugin system to pre-set them for every parse.

So, in our prior example we had:

(def parser (p/parser {}))

and every call to the parser needed an explicit reader: (parser {::p/reader computed} [:hello])

The p/env-plugin is a parser plugin that automatically merges a map of configuration into the parsing environment every time the parser is called. Thus, our earlier example can be converted to:

(def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader computed})]}))

and now each call to the parser needs nothing in the env on each invocation: (parser {} [:hello]).

Providing an environment is such a common operation that there is a shortcut to set it up:

(def parser (p/parser {::p/env {::p/reader computed}}))

The ::p/env option to the parser tells it to install the env-plugin with the given configuration.

5.8.2. Example: Shard switch

For a more practical example, let’s say we are routing in a micro-service architecture and our parser needs to be shard-aware. Let’s write a plugin that anytime it sees a :shard param on a query; and it will update the :shard attribute on the environment and send it down, providing that shard information for any node downstream.

(ns pathom-docs.plugin-shard
  (:require [com.wsscode.pathom.core :as p]))

; a reader that just flows, until it reaches a leaf
(defn flow-reader [{:keys [query] :as env}]
  (if query
    (p/join env)
    :leaf))

(def shard-reader
  ; Clojure neat tricks, let's just fetch the shard
  ; from the environment when :current-shard is asked
  {:current-shard :shard})

(def shard-plugin
  {::p/wrap-read
   (fn [reader]
     (fn [env]
       ; try to get a new shard from the query params
       (let [new-shard (get-in env [:ast :params :shard])]
         (reader (cond-> env new-shard (assoc :shard new-shard))))))})

(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [shard-reader flow-reader]})
                          ; use our shard plugin
                          shard-plugin]}))

(parser {:shard "global"}
        '[:a :b :current-shard
          {(:go-s1 {:shard "s1"})
           ; notice it flows down
           [:x :current-shard {:y [:current-shard]}]}
          :c
          {(:go-s2 {:shard "s2"})
           [:current-shard
            ; we can override at any point
            {(:now-s3 {:shard "s3"})
             [:current-shard]}]}])
; =>
; {:a             :leaf
;  :b             :leaf
;  :current-shard "global"
;  :go-s1         {:x :leaf :current-shard "s1" :y {:current-shard "s1"}}
;  :c             :leaf
;  :go-s2         {:current-shard "s2" :now-s3 {:current-shard "s3"}}}

5.10. Placeholders

There is one issue that some people stumbled upon while using Om.next; the problem happens when you need to display two or more different views of the same item as siblings (regarding query arrangement, not necessarily DOM siblings), how do you make this query?

For example, let’s say you have two different components to display a user profile, one that shows just the username, and another one with its photo.

(om/defui ^:once UserTextView
  static om/IQuery
  (query [_] [:user/name]))

(om/defui ^:once UserImageView
  static om/IQuery
  (query [_] [:user/photo-url]))

(om/defui ^:once UserViewsCompare
  static om/IQuery
  ;; We want to query for both, what we place here?
  (query [_] [{:app/current-user [???]}]))

You might be tempted to concat the queries, and in case you don’t have to nest like we do here, that may even look like it’s working, but let me break this illusion for you; because it’s not. When you use om/get-query it’s not just the query that’s returned; it also contains meta-data telling from which component that query came from.

This information is important, om uses to index your structure and enables incremental updates. When you concat the queries, you lose this, and as a consequence, when you try to run a mutation later that touches those items you will have a “No queries exist at the intersection of component path” thrown in your face.

[This problem is still in discussion on the om repository](https://github.com/omcljs/om/issues/823). So far the best way I know to handle this is to use placeholder nodes, so let’s learn how to manage those cases properly.

What we need is to be able to branch out the different queries, this is my suggestion on how to write the UserViewsCompare query:

(om/defui ^:once UserViewsCompare
  static om/IQuery
  ;; By having extra possible branches we keep the path information working
  (query [_] [{:app/current-user [{:ph/text-view (om/get-query UserTextView)}
                                  {:ph/image-view (om/get-query UserImageView)}]}]))

The trick is to create a convention about placeholder nodes, in this case, we choose the namespace ph to represent “placeholder nodes”, so when the query asks for :ph/something we should just do a recursive call, but staying at the same logical position in terms of parsing, as if we had stayed on the same node.

You can use the p/placeholder-reader to implement this pattern on your parser:

(ns pathom-docs.placeholder
  (:require [com.wsscode.pathom.core :as p]))

(def user
  {:user/name      "Walter White"
   :user/photo-url "http://retalhoclub.com.br/wp-content/uploads/2016/07/1-3.jpg"})

(def computed
  {:app/current-user
   (fn [env]
     (p/join user env))})

(def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader
                                                               computed
                                                               ; placeholder reader
                                                               (p/placeholder-reader "ph")]})]}))

(parser {} [{:app/current-user [{:ph/text-view [:user/name]}
                                {:ph/image-view [:user/photo-url]}]}])
; #:app{:current-user #:ph{:text-view #:user{:name "Walter White"},
;                          :image-view #:user{:photo-url "http://retalhoclub.com.br/wp-content/uploads/2016/07/1-3.jpg"}}}

5.11. Tracing

Pathom 2.2.0 provides a replacement for the old profiler. The old profiler works by wrapping the calls to the Pathom reader and measuring the time around that, this is limiting because then you only have one measure per attribute.

The new tracer works as a event stream, you can inject log events at any time, events might have duration or not (even for events with start and finish, they are recorded as separated events and are combined in a post-processing operation).

This enables detailed logs to understand what happened during the processing of a query, and pathom core already has some system level tracing logs that go automatically, and you can add yours.

To enable the tracing you must add the plugin p/trace-plugin to your parser plugins vector.

5.11.1. Logging custom events

To log custom events you use the function com.wsscode.pathom.trace/trace.

Here is an example parser with some interesting tracing details, run the query to have a look:

link:../src-docs/com/wsscode/pathom/book/tracing/demo.cljs[role=include]
[:com.wsscode.pathom.book.tracing.demo/root-dep :com.wsscode.pathom.book.tracing.demo/root-dep-err]

5.12. Profiling [DEPRECATED, prefer the tracing]

It’s good to know how your queries are performing, and breaking it down by nodes is an excellent level to reason about how your queries are doing. Pathom provides a plugin to make this measurement easy to do:

(ns pathom-docs.profile
  (:require [com.wsscode.pathom.core :as p]
            [com.wsscode.pathom.profile :as p.profile]))

(def computed
  ; to demo delays, this property will take some time
  {:expensive (fn [{:keys [query] :as env}]
                (Thread/sleep 300)
                (if query
                  (p/join env)
                  :done))})

(defn flow-reader [{:keys [query] :as env}]
  (if query
    (p/join env)
    :leaf))

; starting the parser as usual
(def parser
  (p/parser {::p/plugins [(p/env-plugin {::p/reader [computed flow-reader]})
                          ; include the profile plugin
                          p.profile/profile-plugin]}))

(parser {}
        ; run the things
        [:a :b {:expensive [:c :d {:e [:expensive]}]}
         ; profile plugin provide this key, when you ask for it you get the
         ; information, be sure to request this as the last item on your query
         ::p.profile/profile])
; =>
; {:a                  :leaf
;  :b                  :leaf
;  :expensive          {:c :leaf
;                       :d :leaf
;                       :e {:expensive :done}}
;  ::p.profile/profile {:a         0
;                       :b         0
;                       :expensive {:c               1
;                                   :d               0
;                                   :e               {:expensive 304
;                                                     ::p.profile/self 304}
;                                   ::p.profile/self 611}}}

Looking at the profile results, you see the query values, and at the edges is the ms time taken to process that node. When the node has children, a ::p.profile/self indicates the time for the node itself (including children).

If you like to print a flame-graph of this output, you can use some d3 libraries on the web, I recommend the [d3 flame graph from spierman](https://github.com/spiermar/d3-flame-graph). Pathom has a function to convert the profile data to the format accepted by that library:

(-> (parser {}
            ; let's add more things this time
            [:a {:b [:g {:expensive [:f]}]}
             {:expensive [:c :d {:e [:expensive]}]}
             ::p.profile/profile])
    ; get the profile
    ::p.profile/profile
    ; generate the name/value/children format
    p.profile/profile->nvc)
; =>
; {:name     "Root"
;  :value    910
;  :children [{:name ":a" :value 0}
;             {:name     ":b"
;              :value    305
;              :children [{:name ":g" :value 0} {:name ":expensive" :value 304 :children [{:name ":f" :value 1}]}]}
;             {:name     ":expensive"
;              :value    605
;              :children [{:name ":c" :value 0}
;                         {:name ":d" :value 1}
;                         {:name ":e" :value 301 :children [{:name ":expensive" :value 300}]}]}]}

And then use that data to generate the flame graph:

5.13. Path tracking

As you go deep in your parser pathom track record of the current path taken, it’s available at ::p/path at any time. It’s a vector containing the current path from the root, the current main use for it is regarding error reporting and profiling.

(ns pathom-docs.path-tracking
  (:require [com.wsscode.pathom.core :as p]))

(def where-i-am-reader
  {:where-am-i (fn [{::p/keys [path]}] path)})

; a reader that just flows, until it reaches a leaf
(defn flow-reader [{:keys [query] :as env}]
  (if query
    (p/join env)
    :leaf))

(def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader [where-i-am-reader
                                                               flow-reader]})]}))

(parser {} [{:hello [:some {:friend [:place :where-am-i]}]}])
;=>
;{:hello {:some   :leaf
;         :friend {:place      :leaf
;                  :where-am-i [:hello :friend :where-am-i]}}}

5.14. Query Specs

Pathom providers a full specification of the query syntax, in spec format.

5.15. Async parsing

Nowadays the parallel parser is the recommended one to use because of the query strategy, but all the concepts presented here for async parser also applies to the parallel parser, which is async.

If you want to write parsers to run in Javascript environments, then async operations are the norm. The async parser is a version of the parser were you can return core async channels from the readers instead of raw values. This allows for the creation of parsers that do network requests or any other async operation. The async parser is still semantically a serial parser, and it will have the same flow characteristics of the regular parser (the order or resolution is preserved).

To write an async parser we use the p/async-parser function. Here is an example:

link:../src-docs/com/wsscode/pathom/book/async/intro.cljs[role=include]

Try the example:

[:foo :async-info]

The core plugins work normally with the async parser, so error and profiling will work as expected.

5.15.1. Error propagation

When an exception occurs inside a core async channel the error is triggered as part of the channel exception handler. That doesn’t compose very well, and for the parser needs it’s better if we have something more like the async/await pattern used on JS environments. Pathom provides some macros to help making this a simple thing, instead of using go and <!, use the go-catch and <? macros, as in the following example:

link:../src-docs/com/wsscode/pathom/book/async/error_propagation.cljs[role=include]
[:foo :async-info :async-error :com.wsscode.pathom.profile/profile]

Use com.wsscode.common.async-clj for Clojure and com.wsscode.common.async-cljs for ClojureScript. If you writing a cljc file, use the following:

[#?(:clj  com.wsscode.common.async-clj
    :cljs com.wsscode.common.async-cljs)
 :refer [go-catch <?]]

5.15.2. JS Promises

In JS world most of the current async responses comes as promises, you can use the <!p macro to read from promises inside go blocks as if they were channels. Example:

link:../src-docs/com/wsscode/pathom/book/async/js_promises.cljs[role=include]
[:dog.ceo/random-dog-url]

6. GraphQL Integration

Pathom provides a collection of utilities to integrate with GraphQL. There are two main ways in which to use GraphQL:

  1. Simple: Use utilities to convert queries/mutations to GraphQL, and parse the responses. This gives you a quick and easy interface to existing GraphQL APIs, but is not extensible.

  2. Advanced: Integrate with Connect. This method pulls the GraphQL schema into Connect indexes with various benefits: Tools give better support (e.g. query autocompletion within Fulcro Inspect), and you can add your own client-side resolvers that can derive new shapes/data for the API, making it possible to shape the external API to your local UI whims.

In both cases Pathom includes implementations of Fulcro Remotes, so you can easily drop GraphQL support into a Fulcro application as a remote!

This chapter assumes you’re familiar with Pathom’s async support.

The namespaces concerned are:

    [com.wsscode.pathom.graphql :as pg]
    [com.wsscode.pathom.connect.graphql :as pcg]
    [com.wsscode.pathom.fulcro.network :as pfn]

6.1. Simple GraphQL

There is a Fulcro Remote in pfn/graphql-network that allows you to easily add plain GraphQL support to a Fulcro client like so:

(fulcro/new-fulcro-client
    :networking
    {:remote
     (pfn/graphql-network
       {::pfn/url (str "https://api.github.com/graphql?access_token=" token)})})

The queries from components have the following rules:

  1. You can use any namespace on the query keywords.

  2. The name portion of a keyword can be hyphenated instead of camel-case

    1. Hyphenated names will automatically be converted to the standard GraphQL camel case.

Mutations on a Simple GraphQL remote have the following rules:

  1. Mutations can have any namespace. The GraphQL conversion will elide the namespace.

6.1.1. Simple GraphQL Example

To demonstrate how easy it is to get a simple application going against an external GraphQL API we’ll build a simple TODO app. We’ve already gone to graph.cool, and created a GraphQL schema at https://www.graph.cool/ (a back-end as a service provider). You can play with the API by entering queries and mutations via their interface to our endpoint at https://api.graph.cool/simple/v1/cjjkw3slu0ui40186ml4jocgk.

For example, entering this query into the left pane:

query {
  allTodoItems {id, title, completed}
}

should give you something like this (people play with this, so yours will be different):

{
  "data": {
    "allTodoItems": [
      {
        "id": "cjjkw7yws06el0135q5sf372s",
        "title": "Write docs on workspaces",
        "completed": true
      }]
  }
}

So, you can see we have a root query that we can run to get all todo items, and each one has an id and title. So, we can write a simple Fulcro tree of components for that query:

(defsc TodoItem
  [this props]
  {:ident         [:todo/id :todo/id]
   :query         [:todo/id :todo/title :todo/completed]}
  ...)

(defsc TodoSimpleDemo [this props]
  {:ident         (fn [] [::root "singleton"])
   :query         [{:all-todo-items (fp/get-query TodoItem)}]}
  ...)

Notice that on TodoItem we namespaced the keys. This is fine, as the integration code will strip these from the query. If TodoSimpleDemo were your root component, the query for it is already compatible with our defined API when using our GraphQL network:

(fulcro/new-fulcro-client
  :started-callback
  (fn [app]
    (df/load app :all-todo-items todo/TodoItem {:target [::root "singleton" :all-todo-items]}))

  :networking
  {:remote (pfn/graphql-network "https://api.graph.cool/simple/v1/cjjkw3slu0ui40186ml4jocgk")})

Mutations are similarly easy. The network component translates them as discussed earlier, so doing something like adding a new todo item likes like this:

(fm/defmutation create-todo-item [todo]
  (action [env] ...local optimistic stuff...)
  (remote [{:keys [ast]}]
    ;; Don't send the UI-specific params to the server...just the id and title
    (update ast :params select-keys [:todo/id :todo/title])))

The full source is shown below, but hopefully you can see how simple it is to get something going pretty quickly.

link:../src-docs/com/wsscode/pathom/book/graphql/fulcro_network/graphql_todo.cljs[role=include]

6.2. GraphQL and Connect

The more powerful way to use GraphQL from Pathom is to use it with Connect. This gives you the basic features you saw in the simple version, but also gives you a lot more power and extensibility.

The integration has a bit of boilerplate, but it’s all relatively simple. Please make sure you already understand Connect before reading this.

6.2.1. Keywords and GraphQL – Prefixes

In order to properly generate indexes Connect needs to know how you will prefix them for a given GraphQL endpoint. From there, the keyword also gives an indication of the "type" and attribute name.

Say we are interfacting with GitHub: we might choose the prefix github. Then our keywords would need to be things like :github.user/name. The name portion of the keyword can be hyphenated, and Pathom will turn that into a camel-case name, since that is the standard for GraphQL. Name munging is customizable via Pathom settings.

You will have to formally declare the prefix you’ve decided on in order to Connect to work.

6.2.2. GraphQL Entry Points and Connect Ident Maps

In GraphQL the schema designer indicates what entry points are possible. In GitHub’s public API you can, for example, access a User if you know their login. You can access a Repository if you know both the owner and the repository name.

You might wish to take a moment, log into GitHub, and play with these at https://developer.github.com/v4/explorer.

To look at a user, you need something like this:

query {
   user(login:"wilkerlucio") {
    createdAt
  }
}

To look at a repository, you need something like this:

query {
  repository(owner:"wilkerlucio" name:"pathom") {
    createdAt
  }
}

Our EDN queries use idents to stand for these kind of entry points. So, we’d like to be able to translate an EDN query like this:

[{[:github.user/login "wilkerlucio"] [:github.user/created-at]}]

into the GraphQL query above. This is the purpose of the "Ident Map". It is a map whose top-level keys are GraphQL entry point names, and whose value is a map of the attributes required at that entry point associated with EDN keywords:

{ENTRY-POINT-NAME {ATTR connect-keyword
                   ...}
 ...}

So, an ident map for the above two GraphQL entry points is:

{"user"       {"login" :github.user/login}
 "repository" {"owner" :github.user/login
               "name"  :github.repository/name}}

Installing such an ident map (covered shortly) will enable this feature.

If an entry point requires more than one input (as repository does), then there is no standard EDN ident that can directly use it. We’ll cover how to handle that in Multiple Input Entry Points

Interestingly, this feature of Pathom gives you an ability on GraphQL that GraphQL itself doesn’t have: the ability to nest an entry point anywhere in the query. GraphQL only understands entry points at the root of the query, but our EDN notation allows you to use an ident on a join at any level. Pathom Connect will correctly interpret such a join, process it against the GraphQL system, and properly nest the result.

6.2.3. Setting Up Connect with GraphQL

Now that you understand entry points we can explain the rest of the setup. A lot of it is just the standard Connect stuff, but of course there are additions for GraphQL.

First, you need to declare a place to store the indexes, that’s because the GraphQL schema will be loaded asynchronosly later and we need the index reference to add the GraphQL connection.

(defonce indexes (atom {}))

We need to define the configuration for the GraphQL connection:

(def github-gql
  {::pcg/url       (str "https://api.github.com/graphql?access_token=" (ls/get :github-token))
   ::pcg/prefix    "github"
   ::pcg/ident-map {"user"       {"login" ["User" "login"]}
                    "repository" {"owner" ["User" "login"]
                                  "name"  ["Repository" "name"]}}
   ::p.http/driver p.http.fetch/request-async})
::pcg/url

The GraphQL API endpoint

::pcg/prefix

The prefix you’ll use in your EDN queries and mutations.

::pcg/ident-map

The definition of GraphQL entry points, as discussed previously.

::p.http/driver

A driver that can run HTTP requests. Used to issue requests (e.g. fetch schema).

We’re using ls/get to pull our github access token from browser local storage so we don’t have to check it into code, and so anyone can use the example unedited. In Chrome, you can set this via the developer tools "Application" tab (once at the page for your app). Click on local storage, then add a key value pair. The key should be the keyword (typed out), and the value must be a QUOTED token (e.g. "987398ahbckjhbas"). The quotes are required!

Next, we need to create a parser. This will essentially be basically this:

(def parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}
                  ::p.http/driver          p.http.fetch/request-async}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {; we can specify the index for the connect plugin to use
                                      ; instead of creating a new one internally
                                      ::pc/indexes  indexes})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

6.2.4. Loading the GraphQL Schema and Creating a Remote

The final setup step is to make sure that you load the GraphQL schema into the Connect indexes. If you’re using Fulcro it looks like this:

(new-fulcro-client
  :started-callback
  (fn [app]
    (go-catch
      (try
        (let [idx (<? (pcg/load-index github-gql))]
          (swap! indexes pc/merge-indexes idx))
        (catch :default e (js/console.error "Error making index" e)))))

  :networking
  {:remote (-> (create-parser)
               (pfn/pathom-remote)
               ;; OPTIONAL: Automatically adds profile queries to all outgoing queries, so you see profiling from the parser
               (pfn/profile-remote))}}

6.2.5. Adding Resolvers

Of course we’ve done all of this setup so we can make use of (and extend the capabilities of) some GraphQL API.

The normal stuff is trivial: Make EDN queries that ask for the proper attributes in the proper context.

In our example, we might want to list some information about some repositories. If you remember, repositories take two pieces of information, and idents can supply only one.

That’s ok, we can define a resolver for a root-level Connect property that can pre-establish some repositories into our context!

(pc/defresolver repositories [_ _]
  {::pc/output [{:demo-repos [:github.user/login :github.repository/name]}]}
  {:demo-repos
   [{:github.user/login "wilkerlucio" :github.repository/name "pathom"}
    {:github.user/login "fulcrologic" :github.repository/name "fulcro"}
    {:github.user/login "fulcrologic" :github.repository/name "fulcro-inspect"}
    {:github.user/login "fulcrologic" :github.repository/name "fulcro-css"}
    {:github.user/login "fulcrologic" :github.repository/name "fulcro-spec"}
    {:github.user/login "thheller" :github.repository/name "shadow-cljs"}]})

Remember, once Connect has enough info in a context, it can fill in the remaining details. Our Ident Map indicates that if we have "user login" and "repository name", then we can get a repository. Thus, a resolver that outputs values for the keywords associated with those requirements is sufficient!

Remember to add this mutation definition before the parser, then we have to add this resolver to our connect system, do that by updating the call to the connect-plugin, here is the updated parser:

(def parser
  (p/parallel-parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}
                  ::p.http/driver          p.http.fetch/request-async}
     ::p/mutate  pc/mutate-async
     ::p/plugins [(pc/connect-plugin {::pc/register repositories ; registering the resolver
                                      ::pc/indexes  indexes})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

Now we can run a query on :demo-repos like [{:demo-repos [:github.repository/created-at]}], and walk the graph from there to anywhere allowed!

6.2.6. Queries

The queries that are supported "out of the box" are those queries that follow the allowed shape of the documented GraphQL schema for your API. The EDN queries in Fulcro might look like this:

(fp/defsc Repository
  [this {:github.repository/keys [id name-with-owner viewer-has-starred]}]
  {:ident         [:github.repository/id :github.repository/id]
   :query         [:github.repository/id :github.repository/name-with-owner :github.repository/viewer-has-starred]}
  ...)

(fp/defsc GraphqlDemo
  [this {:keys [demo-repos]}]
  {:query         [{:demo-repos (fp/get-query Repository)}]}
  (dom/div
    (mapv repository demo-repos)))

All of Connect’s additinal features (placeholder nodes, augmenting the graph, reshaping) are now also easily accessible.

6.2.7. Fulcro Mutations and Remote

If you’re using Fulcro, then the normal method of defining mutations will work if you use the remote shown earlier. You simply prefix the mutation name with your GraphQL prefix and it’ll work:

(fm/defmutation github/add-star [_]
  (action [{:keys [state ref]}] ...)
  (remote [_] true))
This is not the defmutation we showed earlier in the setup. This is Fulcro’s defmutation.

You can, of course, modify the parameters, do mutation joins, etc.

6.2.8. Connect-Based Mutations

It is possible that you might want to define a mutation that is not on the GraphQL API, but which does some alternative remote operation.

The notation is the same as for resolvers:

(pc/defmutation custom-mutation [_ params]
  {::pc/sym 'custom-mutation         ;; (optional) if provided will be used as mutation symbol, otherwise it will use the def symbol (including namespace)
   ::pc/params [:id {:boo [:y]}]     ;; future autocomplete...noop now
   ::pc/output [:x]}                 ;; future autocomplete...
  ;; can be async or sync.
  (async/go ...))

Note: The params and output are currently meant as documentation. In an upcoming version they’ll also be leveraged for tool autocomplete.

The body of the mutation can return a value (sync) or a channel (async). This means that the custom mutation could do something like hit an alternate REST API. This allows you to put in mutations that the async parser understands and allows to be integrated into a single expression (and API), even though they are not part of the GraphQL API you’re interacting with.

Of course, if you’re using Fulcro, then you’ll also have to make sure they’re OK with the mutation symbolically (e.g. define a fm/defmutation as well).

6.2.9. Multiple Input Entry Points

Earlier we talked about how the Ident Map might specify GraphQL endpoints the required more than one parameter, and the fact that EDN idents only really have a spot for one bit of data beyond the keyword: [keyword value].

Sometimes we have cases like GitHub’s repository entry point where more than one parameter is required.

This can be gracefully handled with EDN query parameters if you modify how Connect processes the query.

Since version 2.2.0 the connect readers ident-reader and open-ident-reader support the provision of extra context information using the query parameter :pathom/context.

Now, remember that this query:

[{[:github.repository/name "n"] [...]}]

cannot work because there is only one of the required two bits of info (we also need owner).

What we’re going to do is allow parameters to make up the difference. If you unfamiliar with them, you just surround the element of the query in a list and add a map of params, like this:

'[{([:github.repository/name "n"] {:x v}) [...]}]

Here is how you can use it to query for a pathom in the Github GraphQL API:

[{([:github.repository/name "pathom"] {:pathom/context {:github.repository/owner "wilkerlucio"}}) [...]}]

The problem, of course, is that this is really hard on the eyes. A bit too much nesting soup, and you need the quote ' in order to prevent an attempt to run a function! But this is what we need to allow us to add in more information. We can clean up the notation by defining a helper function:

(defn repository-ident
  "Returns a parameterized ident that can be used as a join key to directly query a repository."
  [owner name]
  (list [:github.repository/name name] {:pathom/context {:github.user/login owner}}))

Now we can write a reasonable query that contains everything we need:

[{(repository-ident "joe" "boo") [:github.repository/created-at]}]

and we’re good to go!

6.3. Complete GraphQL Connect Example

A complete working example (for workspaces) is shown below:

link:../workspaces/src/com/wsscode/pathom/workspaces/graphql/github_demo.cljs[role=include]

6.4. EDN→GraphQL

Here you can try an interactive convertor. Type your EDN graph query on the left side and see the GraphQL equivalent been generated on the right.

Can you improve this documentation?Edit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close