Pathom Developers Guide

In version 2.2.0-beta11 we introduced the pc/connect-plugin and pc/register with the intent to provider an easier to write shared resolvers and also reduce the boilerplate to setup connect.

This strategy failed in be simple to setup a register and more integrations, because it relied on multiple parts, a better strategy emerged by embedding the lamba to run the resolvers and mutations in their own map instead, so they are complete and stand alone.

But to accomodate this the connect plugin and the pc/register had to change, before the pc/connect-plugin was a var, now it’s an fn that you must call. The register used to take the index atom, the multimethod for resolver and the multimethod for mutations, and did a stateful mutation in all three. Now takes the index in a map format and returns another index with the things registered, now it’s a pure function.

There are times when you’d like to provide an attribute that is computed in some fashion. You can, of course, simply compute it within the resolver along with other properties like so:

but this means that you’ll take the overhead of the computation when any query relate to person comes up. You can instead spread such attributes out into other resolvers as we discussed previously, which will only be invoked if the query actually asks for those properties:

This combination of resolvers can still resolve all of the properties in [:person/full-name :person/age] (if :person/id is in the context), but a query for just [:person/age] won’t invoke any of the logic for the person-name-resolver.

So far we have seen how to define a resolver that can work as long as the inputs are already in the environment. You’re almost certainly wondering how to do that.

One way is to define global resolvers and start the query from them, but very often you’d just like to be able to say "I’d like the first name of person with id 42."

EQL uses "idents" to specify exactly that sort of query:

The above is a join on an ident, and the expected result is a map with the ident as a key:

The query itself has everything you need to establish the context for running the person-resolver, and in fact that is how Pathom single-input resolvers work.

If you use an ident in a query then Pathom is smart enough to know that it can use that ident to establish the context for finding resolvers. In other words, in the query above the ident [:person/id 42] is turned into the parsing context {:person/id 42}, which satisfies the input of any resolver that needs :person/id to run.

A resolver that requires no input can output its results at any point in the graph, thus it is really a global resolver. Pay particular note to the qualification: any point in the graph. Not just root. Thus, a resolver without inputs can "inject" its outputs into any level of the query graph result.

We’re going to start building a parser that can satisfy queries about a music store. So, we’ll start with a global resolver that can resolve the "latest product". The code below shows the entire code needed, boilerplate and all:

Our first resolver exposes the attribute ::latest-product, and since it doesn’t require any input it is a global resolver. Also, note that our output description includes the full output details (including nested attributes), this is mostly useful for auto-complete on UI’s and automatic testing. If you return extra data it will still end up in the output context.

Try some of these queries on the demo below:

Next, let’s say we want to have a new attribute which is the brand of the product. Of course, we could just throw the data there in our other resolver, but the real power of connect comes out when we start splitting the responsibilities among resolvers, so let’s define a resolver for brand that requires an input of :product/id:

The input is a set containing the keys required on the current entity in the parsing context for the resolver to be able to work. This is where Connect starts to shine because any time your query asks for a bit of data it will try to figure it out how to satisfy that request based on the attributes that the current contextual entity already has.

More importantly: Connect will explore the dependency graph in order to resolve things if it needs to! To illustrate this let’s pretend we have some external ID for the brand, and that we can derive this ID from the brand string, pretty much just another mapping:

Note that our query never said anything about the :product/brand. Connect automatically walked the path :product/id → :product/brand → :product/brand-id to obtain the information desired by the query!

When a required attribute is not present in the current entity, Connect will look for resolvers that can fetch it, analyze their inputs, and recursively walk backwards towards the "known data" in the context. When a required attribute is not present in the current entity, Connect will calculate the possibole paths from the data you have to the data you request, then it can use some euristic to decide which path to take and walk this path to reach the data, if there is no possible path connect reader will return ::p/continue to let another reader try to handle that key. You can read more about how this works in the Index page.

Also remember that single-input resolvers can handle ident-based queries. Thus, the following ident-join queries already work without having to define anything else:

The input to a resolver is a set, and as such you can require more than one thing as input to your resolvers. When doing so, of course, your resolver function will receive all of the inputs requested; however, this also means that the parsing context needs to contain them, or there must exist other resolvers that can use what’s in the context to fill them in.

As you have seen before, the only way to provide ad-hoc information to connect is using the ident query, but in the ident itself you can only provide one attribute at a time.

Since version 2.2.0-beta11 the ident readers from connect (ident-reader and open-ident-reader) support adding extra context to the query using parameters, so let’s say you want to load some customer data, but wants to provide some base information that you already have to reduce the number of resolvers called, you can issue a query like this:

Parameters enable another dimension of information to be add to the request. Params have different semantics from inputs, inputs are more a dependency thing, while params are more like options. In practice the main difference is that inputs are something Pathom will try to look up and make available, while parameters must always be provided at query time, no auto resolution. Common cases to use parameters are: pagination, sorting, filtering…​

Let’s write a resolver that outputs a key with a sequence that can take a parameter to sort the resulting list via user provided attribute.

Then we can run queries like:

Try it out:

When you have a to-many relation that is being resolved by a parser you will typically end up with a single query that finds the "IDs", and then N more queries to fill in the details of each item in the sequence. This is known as the N+1 problem, and can be a source of significant performance problems.

To solve this problem, the idea is to instead of running a resolver once for each item on the list, we can send all the inputs as a sequence, so the resolver can do some optimal implementation to handle multiple items, when this happens we call it a batch resolver. For example, let’s take a look at the following demo:

Try the demo:

You can note by the tracer that it took one second for each entry, a clear cascade, because it had to call the :number-added resolver once for each item.

We can improving that by turning this into a batch resolver, like this:

Try the demo:

Note that this time the sleep of one second only happened once, this is because when Pathom is processing a list and the resolver supports batching, the resolver will get all the inputs in a single call, so your batch resolver can get all the items in a single iteration. The results will be cached back for each entry, this will make the other items hit the cache instead of calling the resolver again.

The mutation setup looks very much like the one from resolvers, you define then using pc/defmutation and include on the registry like resolvers.

Now let’s write a mutation with our factory.

The defmutation have the same interface that we used with defresolver.

The ::pc/params is currently a non-op, but in the future it can be used to validate the mutation input, it’s format is the same as output (considering the input can have a complex data shape). The ::pc/output is valid and can be used for auto-complete information on explorer tools.

This section is no longer necessary, the main recommendation now is to use the parallel-parser which is async, no changes needed to write async mutations, all you gotta know is that you can return channels from your mutations and they will be properly coordinated.

Here is an example of doing some mutation operations using async features.

Example:

Using the same query/mutation interface, we replaced the underlying implementation from an atom to a indexeddb database.

You can do the same to target any type of API you can access.

The map format contains all the information needed for a resolver to run, this means a symbol to name it, the input, the output and the lamba to run the computation. This is considered an open map, any extra keys will end up in the index and can be read later.

Here is an example of how you can specific a resolver using the map format:

It’s very similar to using defresolver, you just add the key ::pc/resolve to define the runner function of it. Note that using this helper you don’t have to provide the ::pc/sym key, its added automatically for you.

You can also create using the pc/resolver helper function:

This just returns the same map of the previous example.

And using the final macro helper (recommended way):

Mutations are similar as well:

As you can see it’s very similar to using defresolver, you just add the key ::pc/resolve to define the runner function of it.

Using the helper:

And using the final macro helper (recommended way):

Mutations must be included in the register to be available, like resolvers.

Sometimes it can be interesting to wrap the resolver/mutation function with some generic operation to augment its data or operations. For example, imagine you want some mutations to run in a transaction context:

We could use a transform to clean this up:

Note that ::pc/transform receives the full resolver/mutation map and returns the final version, it can modify anything about the entry.

For another example check the built-in batch transformations.

Once you have your maps ready you can register then using the connect register function to the index.

If you are a library author, consider defining each resolver/mutation as its own symbol and then create another symbol that is vector combining your features, this way you make easy for your users to just get the vector, but still allow then to cherry pick which operations he wants to pull if he doesn’t want it all.

It’s also possible for plugins to declare resolvers and mutations so they get installed when the plugin is used. To do that your plugin must provide the ::pc/register key on the plugin map, and you also need to use the pc/connect-plugin, this plugin will make the instalation, here is an example:

And that’s it, the resolvers will be registered right after the parser is defined.

Connect maintains a few indexes containg information about the resolvers and the relationships on attributes. Connect will look up the index in the environment, on the key :com.wsscode.pathom.connect/indexes, which is a map containing the indexes

In order to explain the different indexes we’ll look at the index generated by our example in the getting started section, each piece of the index will be listed with it’s explanation.

The menu is always visible in the left bar, you can use this to find attributes, resolvers and mutations. By default it shows a complete index of things, you can click on the grey headers to collapse a group. Try it out in the demo above.

There is a search input on top of the menu, it will do a fuzzy search on everything.

The first screen you see in the index contains some main stats about the index.

In the counters section here is explanation for some non obvious counters:

The most connected attributes section will give you a top list of attributes with most connections, a few attributes in a system tend to raise up on this list and can point to effective "hubs" in the center of your data.

When you navigate to an attribute you will be at the attribute view. This view can tell you details about a single attribute.

In the resolver view the left column will give you details about the resolver input and output. Mouse over items to highlight it in the graph.

The right side will have the graph will all attributes that participate in this resolver, the center of the graph will be the resolver input.

The mutation view lists the mutation parameters and the mutation output.

If you click in the Full Graph button it will display a complete graph of the attributes connection in the system. Use this view to get a general feeling of the system, you can understand the main clusters and how they organize.

To expose the index for the index explorer you need to a write a resolver that gets your index out.

Using this you can control what gets out to the explorer.

Here you will find some ways to visualize your index.

One common issue with the index explorer is the fact that resolvers include fns and may include other things that are not possible to encode with transit by default. We suggest you setup a default write handler on Transit so it doesn’t break when it encounter a value that it doesn’t know how to encode.

If you are running Pathom in Clojure, then you also need to know there is a bug in the current Clojure writer, it doesn’t support default handlers (although the docs say it does).

To fix this, here is a code snippet example on how to get around the bug:

And this is how to do in Clojurescript:

During the process of joins it’s possible to modify the environment map that will be used when processing the join. To do that you must return the key ::p/env with the new full environment. As in this example:

TODO: explain serial parser internals

TODO: explain async parser internals

TODO: explain parallel parser internals

These are quite simply a function that receive the env and resolve the read. More than one reader can exist in a chain, and the special return value ::p/continue allows a reader to indicate it cannot resolve the given property (to continue processing the chain). Returning any value (including nil) you’ve resolved the property to that value.

Since it is very common to want to resolve queries from a fixed set of possibilities we support defining a map as a reader. This is really just a "dispatch table" to functions that will receive env. We can re-write the previous example as:

Using a vector for a reader is how you define a chain of readers. This allows you to define readers that serve a particular purpose. For example, some library author might want to supply readers to compose into your parser, or you might have different modules of database-specific readers that you’d like to keep separate.

When pathom is trying to resolve a given attribute (say :person/name) in some context (say against the "Sam" entity) it will start at the beginning of the reader chain. The first reader will be asked to resolve the attribute. If the reader can handle the value then it will be returned and no other readers will be consulted. If it instead returns the special value ::p/continue it is signalling that it could not resolve it (map readers do this if the attribute key is not in their map). When this happens the next reader in the chain will be tried.

If no reader in the chain returns a value (all readers reeturn ::p/continue), then ::p/not-found will be returned.

Not all things need to be computed. Very often the current context will already have attributes that were read during some prior step (for example, a computed attribute might have read an entire entity from the database and made it the current context). The map reader plugin is a plugin that has the following behavior:

The map reader is also capable of resolving relations (if present in the context). For example, if there is a join in the query and a vector of data at that join key in the context, then it will attempt to fulfill the subquery of the join.

The map reader is almost always inserted into a reader chain because it is so common to read clumps of things from a database into the context and resolve them one by one as the query parsing proceeds.

The p/entity function exists as a convenience for pulling the current entity from the parsing environment:

Note that the code above is a partial implementation of the map-dispatcher.

The map-reader just has the additional ability to understand how to walk a map that has a tree shape that already "fits" our query:

Now that you understand where the entity context is tracked I encourage you to check the p/map-reader implementation. It’s not very long and will give you a better understanding of all of the concepts covered so far.

The other significant task when processing a graph query is walking a graph edge to another entity (or entities) when we find a join.

The subquery for a join is in the :query of the environment. Essentially it is a recursive step where we run the parser on the subquery while replacing the "current entity":

The real pathom implementation handles some additional scenarios: like the empty sub-query case (it returns the full entity), the special * query (so you can combine the whole entity + extra computed attributes), and union queries.

The following example shows how to use p/join to "invent" a relation that can then be queried:

There are three different scenarios demonstrated in the above query:

When computing attributes it is possible that you might need some other attribute for the current context that is also computed. You could hard-code a solution, but that would create all sorts of static code problems that could be difficult to manage as your code evolves: changes to the readers, for example, could easily break it and lead to difficult bugs.

Instead, it is important that readers be able to resolve attributes they need from the "current context" in an abstract manner (i.e. the same way that they query itself is being resolved). The p/entity function has an additional arity for handling this exact case. You pass it a list of attributes that should be "made available" on the current entity, and it will use the parser to ensure that they are there (if possible):

The following example shows this in context:

There is a variant p/entity! that raises an error if your desired attributes are not found. It’s recommended to use the enforced version if you need the given attributes, as it will give your user a better error message.

If the parse fails on an enforced attribute you will get an exception. For example, if the current entity were #:character{:nam "Mary"} we’d see:

As you move from node to node, you can choose to wrap the new contextual entity in an atom. This can be used as a narrow kind of caching mechanism that allows for a reader to add information into the current entity as it computes it, but which is valid for only the processing of the current entity (is lost as soon as the next join is followed). Therefore, this won’t help with the overhead of re-visiting the same entity more than once when processing different parts of the same query.

Here is an example using an entity atom:

Union queries allow us to handle edges that lead to heterogeneous nodes. For example a to-many relation for media that could result in a book or movie. Following such an edge requires that we have a different subquery depending on what we actually find in the database.

Here is an example where we want to use a query that will search to find a user, a movie or a book:

Of course, unions need to have a way to determine which path to go based on the entity at hand. In the example above we used the :type (a key on the entity) to determine which branch to follow. The value of ::p/union-path can be a keyword (from something inside entity or a computed attribute) or a function (that takes env and returns the correct key (e.g. :book) to use for the union query).

If you want ::p/union-path to be more contextual you can of course set it in the env during the join process, as in the next example:

This is something beautiful about having an immutable environment; you can make changes with confidence that it will not affect indirect points of the parsing process.

By default Pathom error handler will just return a short error message about the exception, but to debug you will want the stack trace. To view the stack trace you can use a custom process-error, this is an example to do in Clojure:

In ClojureScript:

Having each node being caught is great for the UI, but not so much for testing. During testing you probably prefer the parser to blow up as fast as possible so you don’t accumulate a bunch of errors that get impossible to read. Having to create a different parser to remove the error-handler-plugin can be annoying, so there is an option to solve that. Send the key ::p/fail-fast? as true in the environment, and the try/catch will not be done, making it fail as soon as an exception fires, for example, using our previous parser:

The default error output format (in a separated tree) is very convenient for direct API calls, because they leave a clean output on the data part. But if you want to expose those errors on the UI, pulling then out of the separated tree can be a bit of a pain. To help with that there is a p/raise-errors helper, this will lift the errors so they are present at the same level of the error entry. Let’s take our last error output example and process it with p/raise-errors

Notice that we don’t have the root ::p/errors anymore, instead it is placed at the same level of the error attribute. So the path [::p/errors [:go :nest :trigger-error]] turns into [:go :nest ::p/errors :trigger-error]. This makes very easy to pull the error on the client-side.

Typically the parsing environment will need to include things you create and inject every time you parse a query (e.g. a database connection) and some parser-related things (e.g. the reader) that might be the same all the time.

In the earlier example we created the parser and then explicitly supplied a reader to the environment every time we called it. In cases where there are specific things that you’d always like included in the environment we can instead use the plugin system to pre-set them for every parse.

So, in our prior example we had:

and every call to the parser needed an explicit reader: (parser {::p/reader computed} [:hello])

The p/env-plugin is a parser plugin that automatically merges a map of configuration into the parsing environment every time the parser is called. Thus, our earlier example can be converted to:

and now each call to the parser needs nothing in the env on each invocation: (parser {} [:hello]).

Providing an environment is such a common operation that there is a shortcut to set it up:

The ::p/env option to the parser tells it to install the env-plugin with the given configuration.

For a more practical example, let’s say we are routing in a micro-service architecture and our parser needs to be shard-aware. Let’s write a plugin that anytime it sees a :shard param on a query; and it will update the :shard attribute on the environment and send it down, providing that shard information for any node downstream.

To log custom events you use the function com.wsscode.pathom.trace/trace.

Here is an example parser with some interesting tracing details, run the query to have a look:

When an exception occurs inside a core async channel the error is triggered as part of the channel exception handler. That doesn’t compose very well, and for the parser needs it’s better if we have something more like the async/await pattern used on JS environments. Pathom provides some macros to help making this a simple thing, instead of using go and <!, use the go-catch and <? macros, as in the following example:

Use com.wsscode.common.async-clj for Clojure and com.wsscode.common.async-cljs for ClojureScript. If you writing a cljc file, use the following:

In JS world most of the current async responses comes as promises, you can use the <!p macro to read from promises inside go blocks as if they were channels. Example:

In GraphQL, when you want to load some entity, the path usually goes from some root accessor that determines which type of entity is gonna be returned, plus a parameter (usually some id) to specifify the entry. For example, to load some user you might write a query like:

You can write it similar in EQL as:

When you need to load multiple things at once, GraphQL provides an aliasing feature:

To generate that kind of query with EQL you can use special parameters:

The parameter will be removed from the list, and the alias will be created. If you don’t like the syntax, since this is just Clojure data we can make some helper around:

Not that much different, but you can be more creative and design your favorite API for it.

So far we just used the params to express an entity request, but in EQL we also have another way to express identity, via idents.

If you are not familiar with the ident concept, you can imagine it as a way to point to some specific entity, an ident is takes the form of a vector with two elements, the first element will determine the "identity type", which is always a keyword, and on the second element you have the "identity value", which can be any EDN value.

Some applied example of EQL idents:

The ident concept doesn’t exist in GraphQL, but still can we try to translate it.

What Pathom does then is get the most common case, that is a single entry point with a single parameter, and extract that from a qualified ident using the following syntax:

So the previous ident turns in:

Note that it also created an alias automatically, this way you can write queries using multiple similar idents and they will end up in different names:

Turns in:

In some cases one param is not enough, there is a supported And notation:

But to be honest, that just looks bad, if you need more than one param is better just to don’t use an ident for direct translation, still this can be useful if you need a quick and dirty way to access some multi-param in an ident fashion.

You can customize this ident translation behavior by providing ::pg/ident-transform to the pg/query→graphql call, here is the code of the default implementation so you can understand what a translation mean:

There is a Fulcro Remote in pfn/graphql-network that allows you to easily add plain GraphQL support to a Fulcro client like so:

The queries from components have the following rules:

Mutations on a Simple GraphQL remote have the following rules:

The more powerful way to use GraphQL from Pathom is to use it with Connect. This gives you the basic features you saw in the simple version, but also gives you a lot more power and extensibility.

The integration has a bit of boilerplate, but it’s all relatively simple. Please make sure you already understand Pathom Connect before reading this.

A complete working example (for workspaces) is shown below: