Pathom Developers Guide

In version 2.2.0-beta11 we introduced the pc/connect-plugin and pc/register with the intent to provider an easier to write shared resolvers and also reduce the boilerplate to setup connect.

This strategy failed in be simple to setup a register and more integrations, because it relied on multiple parts, a better strategy emerged by embedding the lamba to run the resolvers and mutations in their own map instead, so they are complete and stand alone.

But to accomodate this the connect plugin and the pc/register had to change, before the pc/connect-plugin was a var, now it’s an fn that you must call. The register used to take the index atom, the multimethod for resolver and the multimethod for mutations, and did a stateful mutation in all three. Now takes the index in a map format and returns another index with the things registered, now it’s a pure function.

There are times when you’d like to provide an attribute that is computed in some fashion. You can, of course, simply compute it within the resolver along with other properties like so:

but this means that you’ll take the overhead of the computation when any query relate to person comes up. You can instead spread such attributes out into other resolvers as we discussed previously, which will only be invoked if the query actually asks for those properties:

This combination of resolvers can still resolve all of the properties in [:person/full-name :person/age] (if :person/id is in the context), but a query for just [:person/age] won’t invoke any of the logic for the person-name-resolver.

So far we have seen how to define a resolver that can work as long as the inputs are already in the environment. You’re almost certainly wondering how to do that.

One way is to define global resolvers and start the query from them, but very often you’d just like to be able to say "I’d like the first name of person with id 42."

Om Next/Fulcro use "idents" to specify exactly that sort of query:

The above is a join on an ident, and the expected result is a map with the ident as a key:

The query itself has everything you need to establish the context for running the person-resolver, and in fact that is how Pathom single-input resolvers work.

If you use an ident in a query then Pathom is smart enough to know that it can use that ident to establish the context for finding resolvers. In other words, in the query above the ident [:person/id 42] is turned into the parsing context {:person/id 42}, which satisfies the input of any resolver that needs :person/id to run.

A resolver that requires no input can output its results at any point in the graph, thus it is really a global resolver. Pay particular note to the qualfication: any point in the graph. Not just root. Thus, a resolver without inputs can "inject" its outputs into any level of the query graph result.

We’re going to start building a parser that can satisfy queries about a music store. So, we’ll start with a global resolver that can resolve the "latest product". The code below shows the entire code needed, boilerplate and all:

Our first resolver exposes the attribute ::latest-product, and since it doesn’t require any input it is a global resolver. Also, note that our output description includes the full output details (including nested attributes), this is mostly useful for auto-complete on UI’s and automatic testing. If you return extra data it will still end up in the output context.

Try some of these queries on the demo below:

Next, let’s say we want to have a new attribute which is the brand of the product. Of course, we could just throw the data there in our other resolver, but the real power of connect comes out when we start splitting the responsibilities among resolvers, so let’s define a resolver for brand that requires an input of :product/id:

The input is a set containing the keys required on the current entity in the parsing context for the resolver to be able to work. This is where Connect starts to shine because any time your query asks for a bit of data it will try to figure it out how to satisfy that request based on the attributes that the current contextual entity already has.

More importantly: Connect will explore the dependency graph in order to resolve things if it needs to! To illustrate this let’s pretend we have some external ID for the brand, and that we can derive this ID from the brand string, pretty much just another mapping:

Note that our query never said anything about the :product/brand. Connect automatically walked the path :product/id → :product/brand → :product/brand-id to obtain the information desired by the query!

When a required attribute is not present in the current entity, Connect will look for resolvers that can fetch it, analyze their inputs, and recursively walk backwards towards the "known data" in the context. When a required attribute is not present in the current entity, Connect will calculate the possibole paths from the data you have to the data you request, then it can use some euristic to decide which path to take and walk this path to reach the data, if there is no possible path connect reader will return ::p/continue to let another reader try to handle that key. You can read more about how this works in the Index page.

Also remember that single-input resolvers can handle ident-based queries. Thus, the following ident-join queries already work without having to define anything else:

The input to a resolver is a set, and as such you can require more than one thing as input to your resolvers. When doing so, of course, your resolver function will receive all of the inputs requested; however, this also means that the parsing context needs to contain them, or there must exist other resolvers that can use what’s in the context to fill them in.

As you have seen before, the only way to provide ad-hoc information to connect is using the ident query, but in the ident itself you can only provide one attribute at a time.

Since version 2.2.0-beta11 the ident readers from connect (ident-reader and open-ident-reader) support adding extra context to the query using parameters, so let’s say you want to load some customer data, but wants to provide some base information that you already have to reduce the number of resolvers called, you can issue a query like this:

When you have a to-many relation that is being resolved by a parser you will typically end up with a single query that finds the "IDs", and then N more queries to fill in the details of each item in the sequence. This is known as the N+1 problem, and can be a source of significant performance problems.

The solution is relatively simple in Pathom: Add the request-cache-plugin to the parser and include a ::pc/batch? true option in your resolver. Pathom then knows that it can send a sub-resolver a sequence of inputs instead of a single one. If your resolver receives a sequence, then it is expected to return a vector of promised outputs instead of a single map.

Let’s see one example to illustrate the situation:

Try the example:

You can note by the tracer that it took one second for each entry, a clear cascade, because it had to call the :number-added resolver once for each item.

We can improving that by turning this into a batch resolver, like this:

Try the example:

Note that this time the sleep of one second only happened once, this is because when Pathom is processing a list and the resolver supports batching, the resolver will get all the inputs in a single call, so your batch resolver can get all the items in a single iteration. The results will be cached back for each entry, this will make the other items hit the cache instead of calling the resolver again.

Note the request-cache plugin is required for the batch to work.

The mutation setup looks very much like the one from resolvers, you define then using pc/defmutation and include on the registry like resolvers.

Now let’s write a mutation with our factory.

The defmutation have the same interface that we used with defresolver.

The ::pc/params is currently a non-op, but in the future it can be used to validate the mutation input, it’s format is the same as output (considering the input can have a complex data shape). The ::pc/output is valid and can be used for auto-complete information on explorer tools.

This section is no longer nescessary, the main recommendation now is to use the parallel-parser which is async, no changes needed to write async mutations, all you gotta know is that you can return channels from your mutations and they will be properly coordinated.

Here is an example of doing some mutation operations using async features.

Example:

Using the same query/mutation interface, we replaced the underlying implementation from an atom to a indexeddb database.

You can do the same to target any type of API you can access.

Here is an example of how you can specific a resolver using the map format:

As you can see it’s very similar to using defresolver, you just add the key ::pc/resolve to define the runner function of it.

You can also create using the pc/resolver helper function:

This just returns the same map of the previous example.

And using the final macro helper (recommended way):

Mutations are similar as well:

As you can see it’s very similar to using defresolver, you just add the key ::pc/resolve to define the runner function of it.

Using the helper:

And using the final macro helper (recommended way):

Once you have your maps ready you can register then using the connect register function to the index.

If you are a library author, consider defining each resolver/mutation as its own symbol and then create another symbol that is vector combining your features, this way you make easy for your users to just get the vector, but still allow then to cherry pick which operations he wants to pull if he doesn’t want it all.

It’s also possible for plugins to declare resolvers and mutations so they get installed when the plugin is used. To do that your plugin must provide the ::pc/register key on the plugin map, and you also need to use the pc/connect-plugin, this plugin will make the instalation, here is an example:

And that’s it, the resolvers will be registered right after the parser is defined.

Connect maintains a few indexes containg information about the resolvers and the relationships on attributes. Connect will look up the index in the environment, on the key :com.wsscode.pathom.connect/indexes, which is a map containing the indexes

In order to explain the different indexes we’ll look at the index generated by our example in the getting started section:

TODO: explain serial parser internals

TODO: explain async parser internals

TODO: explain parallel parser internals

These are quite simply a function that receive the env and resolve the read. More than one reader can exist in a chain, and the special return value ::p/continue allows a reader to indicate it cannot resolve the given property (to continue processing the chain). Returning any value (including nil) you’ve resolved the property to that value.

Since it is very common to want to resolve queries from a fixed set of possibilities we support defining a map as a reader. This is really just a "dispatch table" to functions that will receive env. We can re-write the previous example as:

Using a vector for a reader is how you define a chain of readers. This allows you to define readers that serve a particular purpose. For example, some library author might want to supply readers to compose into your parser, or you might have different modules of database-specific readers that you’d like to keep separate.

When pathom is trying to resolve a given attribute (say :person/name) in some context (say against the "Sam" entity) it will start at the beginning of the reader chain. The first reader will be asked to resolve the attribute. If the reader can handle the value then it will be returned and no other readers will be consulted. If it instead returns the special value ::p/continue it is signalling that it could not resolve it (map readers do this if the attribute key is not in their map). When this happens the next reader in the chain will be tried.

If no reader in the chain returns a value (all readers reeturn ::p/continue), then ::p/not-found will be returned.

Not all things need to be computed. Very often the current context will already have attributes that were read during some prior step (for example, a computed attribute might have read an entire entity from the database and made it the current context). The map reader plugin is a plugin that has the following behavior:

The map reader is also capable of resolving relations (if present in the context). For example, if there is a join in the query and a vector of data at that join key in the context, then it will attempt to fulfill the subquery of the join.

The map reader is almost always inserted into a reader chain because it is so common to read clumps of things from a database into the context and resolve them one by one as the query parsing proceeds.

The p/entity function exists as a convenience for pulling the current entity from the parsing environment:

Note that the code above is a partial implementation of the map-dispatcher.

The map-reader just has the additional ability to understand how to walk a map that has a tree shape that already "fits" our query:

Now that you understand where the entity context is tracked I encourage you to check the p/map-reader implementation. It’s not very long and will give you a better understanding of all of the concepts covered so far.

The other significant task when processing a graph query is walking a graph edge to another entity (or entities) when we find a join.

The subquery for a join is in the :query of the environment. Essentially it is a recursive step where we run the parser on the subquery while replacing the "current entity":

The real pathom implementation handles some additional scenarios: like the empty sub-query case (it returns the full entity), the special * query (so you can combine the whole entity + extra computed attributes), and union queries.

The following example shows how to use p/join to "invent" a relation that can then be queried:

There are three different scenarios demonstrated in the above query:

When computing attributes it is possible that you might need some other attribute for the current context that is also computed. You could hard-code a solution, but that would create all sorts of static code problems that could be difficult to manage as your code evolves: changes to the readers, for example, could easily break it and lead to difficult bugs.

Instead, it is important that readers be able to resolve attributes they need from the "current context" in an abstract manner (i.e. the same way that they query itself is being resolved). The p/entity function has an additional arity for handling this exact case. You pass it a list of attributes that should be "made available" on the current entity, and it will use the parser to ensure that they are there (if possible):

The following example shows this in context:

There is a variant p/entity! that raises an error if your desired attributes are not found. It’s recommended to use the enforced version if you need the given attributes, as it will give your user a better error message.

If the parse fails on an enforced attribute you will get an exception. For example, if the current entity were #:character{:nam "Mary"} we’d see:

As you move from node to node, you can choose to wrap the new contextual entity in an atom. This can be used as a narrow kind of caching mechanism that allows for a reader to add information into the current entity as it computes it, but which is valid for only the processing of the current entity (is lost as soon as the next join is followed). Therefore, this won’t help with the overhead of re-visiting the same entity more than once when processing different parts of the same query.

Here is an example using an entity atom:

Union queries allow us to handle edges that lead to heterogeneous nodes. For example a to-many relation for media that could result in a book or movie. Following such an edge requires that we have a different subquery depending on what we actually find in the database.

Here is an example where we want to use a query that will search to find a user, a movie or a book:

Of course, unions need to have a way to determine which path to go based on the entity at hand. In the example above we used the :type (a key on the entity) to determine which branch to follow. The value of ::p/union-path can be a keyword (from something inside entity or a computed attribute) or a function (that takes env and returns the correct key (e.g. :book) to use for the union query).

If you want ::p/union-path to be more contextual you can of course set it in the env during the join process, as in the next example:

This is something beautiful about having an immutable environment; you can make changes with confidence that it will not affect indirect points of the parsing process.

Having each node being caught is great for the UI, but not so much for testing. During testing you probably prefer the parser to blow up as fast as possible so you don’t accumulate a bunch of errors that get impossible to read. Having to create a different parser to remove the error-handler-plugin can be annoying, so there is an option to solve that. Send the key ::p/fail-fast? as true in the environment, and the try/catch will not be done, making it fail as soon as an exception fires, for example, using our previous parser:

The default error output format (in a separated tree) is very convenient for direct API calls, because they leave a clean output on the data part. But if you want to expose those errors on the UI, pulling then out of the separated tree can be a bit of a pain. To help with that there is a p/raise-errors helper, this will lift the errors so they are present at the same level of the error entry. Let’s take our last error output example and process it with p/raise-errors

Notice that we don’t have the root ::p/errors anymore, instead it is placed at the same level of the error attribute. So the path [::p/errors [:go :nest :trigger-error]] turns into [:go :nest ::p/errors :trigger-error]. This makes very easy to pull the error on the client-side.

Typically the parsing environment will need to include things you create and inject every time you parse a query (e.g. a database connection) and some parser-related things (e.g. the reader) that might be the same all the time.

In the earlier example we created the parser and then explicitly supplied a reader to the environment every time we called it. In cases where there are specific things that you’d always like included in the environment we can instead use the plugin system to pre-set them for every parse.

So, in our prior example we had:

and every call to the parser needed an explicit reader: (parser {::p/reader computed} [:hello])

The p/env-plugin is a parser plugin that automatically merges a map of configuration into the parsing environment every time the parser is called. Thus, our earlier example can be converted to:

and now each call to the parser needs nothing in the env on each invocation: (parser {} [:hello]).

Providing an environment is such a common operation that there is a shortcut to set it up:

The ::p/env option to the parser tells it to install the env-plugin with the given configuration.

For a more practical example, let’s say we are routing in a micro-service architecture and our parser needs to be shard-aware. Let’s write a plugin that anytime it sees a :shard param on a query; and it will update the :shard attribute on the environment and send it down, providing that shard information for any node downstream.

To log custom events you use the function com.wsscode.pathom.trace/trace.

Here is an example parser with some interesting tracing details, run the query to have a look:

When an exception occurs inside a core async channel the error is triggered as part of the channel exception handler. That doesn’t compose very well, and for the parser needs it’s better if we have something more like the async/await pattern used on JS environments. Pathom provides some macros to help making this a simple thing, instead of using go and <!, use the go-catch and <? macros, as in the following example:

Use com.wsscode.common.async-clj for Clojure and com.wsscode.common.async-cljs for ClojureScript. If you writing a cljc file, use the following:

In JS world most of the current async responses comes as promises, you can use the <!p macro to read from promises inside go blocks as if they were channels. Example:

To demonstrate how easy it is to get a simple application going against an external GraphQL API we’ll build a simple TODO app. We’ve already gone to graph.cool, and created a GraphQL schema at https://www.graph.cool/ (a back-end as a service provider). You can play with the API by entering queries and mutations via their interface to our endpoint at https://api.graph.cool/simple/v1/cjjkw3slu0ui40186ml4jocgk.

For example, entering this query into the left pane:

should give you something like this (people play with this, so yours will be different):

So, you can see we have a root query that we can run to get all todo items, and each one has an id and title. So, we can write a simple Fulcro tree of components for that query:

Notice that on TodoItem we namespaced the keys. This is fine, as the integration code will strip these from the query. If TodoSimpleDemo were your root component, the query for it is already compatible with our defined API when using our GraphQL network:

Mutations are similarly easy. The network component translates them as discussed earlier, so doing something like adding a new todo item likes like this:

The full source is shown below, but hopefully you can see how simple it is to get something going pretty quickly.

In order to properly generate indexes Connect needs to know how you will prefix them for a given GraphQL endpoint. From there, the keyword also gives an indication of the "type" and attribute name.

Say we are interfacting with GitHub: we might choose the prefix github. Then our keywords would need to be things like :github.user/name. The name portion of the keyword can be hyphenated, and Pathom will turn that into a camel-case name, since that is the standard for GraphQL. Name munging is customizable via Pathom settings.

You will have to formally declare the prefix you’ve decided on in order to Connect to work.

In GraphQL the schema designer indicates what entry points are possible. In GitHub’s public API you can, for example, access a User if you know their login. You can access a Repository if you know both the owner and the repository name.

You might wish to take a moment, log into GitHub, and play with these at https://developer.github.com/v4/explorer.

To look at a user, you need something like this:

To look at a repository, you need something like this:

Our EDN queries use idents to stand for these kind of entry points. So, we’d like to be able to translate an EDN query like this:

into the GraphQL query above. This is the purpose of the "Ident Map". It is a map whose top-level keys are GraphQL entry point names, and whose value is a map of the attributes required at that entry point associated with EDN keywords:

So, an ident map for the above two GraphQL entry points is:

Installing such an ident map (covered shortly) will enable this feature.

If an entry point requires more than one input (as repository does), then there is no standard EDN ident that can directly use it. We’ll cover how to handle that in Multiple Input Entry Points

Now that you understand entry points we can explain the rest of the setup. A lot of it is just the standard Connect stuff, but of course there are additions for GraphQL.

First, you need to declare a place to store the indexes, that’s because the GraphQL schema will be loaded asynchronosly later and we need the index reference to add the GraphQL connection.

We need to define the configuration for the GraphQL connection:

Next, we need to create a parser. This will essentially be basically this:

The final setup step is to make sure that you load the GraphQL schema into the Connect indexes. If you’re using Fulcro it looks like this:

Of course we’ve done all of this setup so we can make use of (and extend the capabilities of) some GraphQL API.

The normal stuff is trivial: Make EDN queries that ask for the proper attributes in the proper context.

In our example, we might want to list some information about some repositories. If you remember, repositories take two pieces of information, and idents can supply only one.

That’s ok, we can define a resolver for a root-level Connect property that can pre-establish some repositories into our context!

Remember, once Connect has enough info in a context, it can fill in the remaining details. Our Ident Map indicates that if we have "user login" and "repository name", then we can get a repository. Thus, a resolver that outputs values for the keywords associated with those requirements is sufficient!

Remember to add this mutation definition before the parser, then we have to add this resolver to our connect system, do that by updating the call to the connect-plugin, here is the updated parser:

Now we can run a query on :demo-repos like [{:demo-repos [:github.repository/created-at]}], and walk the graph from there to anywhere allowed!

The queries that are supported "out of the box" are those queries that follow the allowed shape of the documented GraphQL schema for your API. The EDN queries in Fulcro might look like this:

All of Connect’s additinal features (placeholder nodes, augmenting the graph, reshaping) are now also easily accessible.

If you’re using Fulcro, then the normal method of defining mutations will work if you use the remote shown earlier. You simply prefix the mutation name with your GraphQL prefix and it’ll work:

You can, of course, modify the parameters, do mutation joins, etc.

It is possible that you might want to define a mutation that is not on the GraphQL API, but which does some alternative remote operation.

The notation is the same as for resolvers:

Note: The params and output are currently meant as documentation. In an upcoming version they’ll also be leveraged for tool autocomplete.

The body of the mutation can return a value (sync) or a channel (async). This means that the custom mutation could do something like hit an alternate REST API. This allows you to put in mutations that the async parser understands and allows to be integrated into a single expression (and API), even though they are not part of the GraphQL API you’re interacting with.

Of course, if you’re using Fulcro, then you’ll also have to make sure they’re OK with the mutation symbolically (e.g. define a fm/defmutation as well).

Earlier we talked about how the Ident Map might specify GraphQL endpoints the required more than one parameter, and the fact that EDN idents only really have a spot for one bit of data beyond the keyword: [keyword value].

Sometimes we have cases like GitHub’s repository entry point where more than one parameter is required.

This can be gracefully handled with EDN query parameters if you modify how Connect processes the query.

Since version 2.2.0 the connect readers ident-reader and open-ident-reader support the provision of extra context information using the query parameter :pathom/context.

Now, remember that this query:

cannot work because there is only one of the required two bits of info (we also need owner).

What we’re going to do is allow parameters to make up the difference. If you unfamiliar with them, you just surround the element of the query in a list and add a map of params, like this:

Here is how you can use it to query for a pathom in the Github GraphQL API:

The problem, of course, is that this is really hard on the eyes. A bit too much nesting soup, and you need the quote ' in order to prevent an attempt to run a function! But this is what we need to allow us to add in more information. We can clean up the notation by defining a helper function:

Now we can write a reasonable query that contains everything we need:

and we’re good to go!