Testing

The suite is layered, and each layer answers a different question. A green result at one layer says nothing about the layer below it — which is the whole argument of the last section on this page.

Layer	What it holds	Command
Contract spine	every function in src/ declares an m/=>, and it is enforced while the suite runs	bb test
Load guard	every namespace under src/ compiles and loads, including the one nothing requires	bb test
Schema-driven properties	the generator, the oracle and the mutants come from the schemas the contracts are written in	bb test
Schema mutation	a corrupted Settlement schema is caught, so the schema is a specification	bb test
Browser scenarios	what a payer sees, and what the page believes	code cljs e2e run
Scenario mutation	whether those scenarios would notice a broken storefront	code cljs e2e mutate

bb test shells clojure -M:test, which is the cognitect test-runner over test/. CI runs clojure -M:test directly, before it will mint a version. bb check is that plus bb tokens:check.

The current measure is 101 tests containing 525 assertions, 0 failures and 0 errors; nine e2e scenarios over 73 steps, mutation score 1.0.

The contract spine, instrumented

monero-store.contract-test calls malli.instrument/instrument! at load. Loading that namespace instruments the whole store, and the runner loads it alongside every other test namespace — so every other test in the suite runs against enforced contracts, not declared ones.

That distinction is the point. An m/=> that is only declared is a comment with a parser. An instrumented one fails at the call site. Four schemas in this repository lied about their own functions when instrumentation was first switched on, and in all four cases the schema was the thing that was wrong.

Two tests state it rather than assuming it:

(deftest the-contracts-are-instrumented-not-merely-declared
  (testing "a wrong argument type is caught at the call, not deep inside"
    (is (thrown? clojure.lang.ExceptionInfo (currency/->display :xmr "1000")))
    (is (thrown? clojure.lang.ExceptionInfo (currency/known? "xmr"))))

  (testing "and the right one still passes straight through"
    (is (= "0.001000000000" (currency/->display :xmr 1000000000)))
    (is (true? (currency/known? :xmr)))))

The wrong argument never reaches the function body. ->display does not see the string; known? does not see "xmr".

Every function declares one

every-function-in-the-store-declares-its-contract compares the keys of (m/function-schemas) against ns-interns for every store namespace. A defn with no contract lands in a failure map, named, per namespace.

Three things are excluded, and the reasons differ:

Excluded	Reason
monero-store.adt	its functions are generated by defadt from the sum declaration — there is no form to attach a contract to, and the generated malli schema already describes the values
monero-store.support	the faked store the tests drive, not the store
macros and protocol methods	a protocol method dispatches to whatever implements it, so it carries no contract of its own

The guard carries its own non-vacuity check, because "nothing is missing" is also true of an empty spine:

(testing "and the spine is not empty, which is the failure this guard could hide"
  (is (< 100 (reduce + (map (comp count first contracted-fns) store-namespaces)))))

Every namespace loads

The coverage guard above derives its namespace list from (all-ns). That set contains only namespaces something has already required, which is exactly why it cannot be the source of truth: a namespace nothing requires is not loaded, is not in (all-ns), and is therefore checked by nothing. It can fail to compile outright and the suite stays green.

So the walk is over the filesystem, and the namespace requires every file it finds:

(defn- source-namespaces
  "Every namespace under src/, read off disk rather than listed by hand."
  []
  (->> (file-seq (io/file "src"))
       (filter #(and (.isFile ^java.io.File %)
                     (str/ends-with? (.getName ^java.io.File %) ".clj")))
       (map #(-> (.getPath ^java.io.File %)
                 (str/replace #"^src/" "")
                 (str/replace #"\.clj$" "")
                 (str/replace "_" "-")
                 (str/replace "/" ".")
                 symbol))
       sort))

(run! require (source-namespaces))

monero-store.system is the case that motivated it. It is the entry namespace — the one nothing else requires — and it could not load, and nothing noticed. The test now names it:

(testing "the entry namespace included — it is the one nothing else requires"
  (is (contains? (set found) 'monero-store.system)))

The same test guards against a vacuous walk, because file-seq over a missing src returns nothing and "every namespace found is loaded" holds trivially of no namespaces:

(testing "the walk finds the store, not an empty directory"
  (is (< 20 (count found))))

There are 29 .clj files under src/.

Properties synthesized from the schemas

hive-schemas.test/deftrifecta-from-schema takes a subject and an :in/:out schema pair and emits one test var per facet: <name>-conformance (generated input, output conforms to :out), <name>-relation (a stated relation between input and output holds), and — when :mutation is on — <name>-mutants-present and <name>-mutations.

Nothing in this section is a hand-written case. The generator, the oracle and the mutants all come from the same schema vocabulary the contracts are written in, so a property cannot drift away from the contract it is testing.

Property	Subject	What the relation states	:num-tests
display-is-total	currency/->display	a scaled currency renders with a decimal point and an unscaled one without; a zero amount renders starting with 0	200
median-is-a-middle	promote.quote/median	the result lies between the smallest and the largest input	200
scrubbing-only-ever-removes	collect.analytics/scrub	the output is never larger than the input, and none of :email :address :tx-hash :reference :secret :token survives	200
settlement-never-credits-a-double-spend	payments.chain/settlement-of	the provider and the expected amount are echoed unchanged, and the paid amount never exceeds the sum of the transfers that are not double spends	100
consensus-never-invents-a-price	promote.quote/consensus	the price lies between the smallest and the largest quoted price, at least two sources are named, and every named source is one that was supplied	200
money-is-never-applied-to-a-closed-invoice	promote.invoice/resolution	the result is :applied exactly when the invoice is open and not lapsed, and :late otherwise	200

Conformance alone is a weak oracle — it checks the shape of the output and nothing else — which is why every one of the six states a :rel.

Two of them need a generator the contract does not supply, for the same underlying reason both times: a contract and a generator are different obligations on the same schema.

PlausibleObservation bounds the transfer amounts. The production schema says [:int {:min 0}] and is right to — an amount is a non-negative integer of atomic units and the wallet is the authority on it. But a generator reading that literally mints transfers of 9.2e18 piconero each and sums them past Long/MAX. The overflow is real; no wallet on a chain with a 1.8e19 supply cap can produce it. The bound belongs to the generator, not to the contract.

GeneratedInvoice exists because schema/Invoice validates an invoice but cannot generate one: its instant is [:fn inst?], and a predicate has no generator. So the generator is a tuple of the two things the property actually varies — the status, and how far the expiry sits either side of the clock — :gen/fmap'd into a real invoice. The value handed to the subject still satisfies schema/Invoice, which the live instrumentation checks on every call.

Mutation on the schemas themselves

A permissive :out schema makes conformance vacuous: anything passes. The :mutation facet closes that. schema-mutants derives corruptions of the :out schema — a required key dropped, or set to a value that key's schema provably rejects — and requires that each one be caught by the seeded cases. A mutant that survives is a schema that does not constrain what it claims to.

Subject	Schema mutated
payments.chain/settlement-of	schema/Settlement
promote.quote/consensus	its closed output map

The Settlement case is the one that matters most, because Settlement is what the whole money path hands to pipeline/checkout. Every corruption of it must be caught, and that is what makes it a specification rather than a description of whatever the function happens to return.

<name>-mutants-present fails loudly when the :out schema yields no mutants at all, which is the failure the mutation facet could otherwise hide:

(is (pos? (count (schema-mutants ~subj ~out)))
    "no schema-derived mutants — :out too permissive; tighten :out or pass :mutation false")

The four properties marked :mutation false have :out schemas the engine cannot corrupt at all, because it corrupts required map keys and none of the four returns a map with any:

Property	:out
display-is-total	schema/NonBlank
median-is-a-middle	[:maybe number?]
scrubbing-only-ever-removes	[:map-of :keyword :any]
money-is-never-applied-to-a-closed-invoice	[:enum :applied :late]

The first three are also loose. The fourth is not — a two-value enum is as closed as an output gets — but it is still not a map, and both corruptions the engine makes are corruptions of a required key. All four would trip <name>-mutants-present, so all four say :mutation false rather than emit a facet that would always pass.

The storefront, in a browser

Nine scenarios in test/e2e/storefront.edn, run by hive-cljs over Playwright against the real store on :8080. Seventy-three steps.

Each scenario mixes two assertion channels:

Channel	Example step	What it holds
DOM	[:expect-text ".pay .status" "Waiting"]	what a payer sees
:eval-cljs	[:eval-cljs "(assert (= \"pending\" (get-in @monero-store-ui.state/app [:invoice :invoice :status])))"]	what the app believes, evaluated inside the page being driven

Having both in one scenario is the point. A red DOM assertion beside a green state assertion localises the fault to rendering; both red points at the money path underneath.

The scenarios need two processes. The store serves resources/public/js itself, so shadow-cljs and the store share one origin and there is no proxy:

clojure -M:nrepl        # then (go) — a store on :8080, every rail faked
bb ui:watch             # npx shadow-cljs watch app
code cljs e2e list      # the scenarios declared
code cljs e2e run       # every scenario
code cljs e2e mutate    # inject each fault, report what survived

Both e2e run and e2e mutate narrow by :scenario or :tags; the tags in use are :smoke, :runtime, :checkout, :xmr and :mutation.

The store under test must trust the x-customer-ref header, because the scenarios type a payer reference into .bar input and the storefront sends it as that header. It must also offer a chain rail, because two scenarios click button.rail:has-text("monero"): one asserts on the address and the monero: URI, the other that nothing is paid and the whole amount is still owing. The REPL store is already wired that way — dev/user.clj passes identity/header-identity and a wallet/fake-wallet directly. A store started from the environment instead gets the same wiring from IDENTITY=header and MONERO_BACKEND=fake.

A pass count and a mutation score are different claims

code cljs e2e mutate neutralizes one var at a time and re-runs the scenarios. A fault the scenarios notice is killed; one they do not notice survives. The score is killed over injected. Five faults are declared in .hive-project.edn under :hive.cljs :e2e :faults:

Fault	Target	Replacement
:money-blank	monero-store-ui.format/money	(constantly "")
:status-blank	monero-store-ui.format/status	(constantly "")
:payment-uri-nil	monero-store-ui.format/payment-uri	(constantly nil)
:catalog-never-loads	monero-store-ui.state/load-catalog!	(constantly nil)
:invoices-never-load	monero-store-ui.state/load-invoices!	(constantly nil)

The first catalog of scenarios passed every step and scored 0.2. Four of five faults survived a suite with no failures in it.

The reason is mechanical rather than subtle. A fault is spliced in after a scenario's opening navigation and waits — it cannot be injected before the page exists. A scenario that goes to /, waits for .card, and then asserts on .card is asserting against DOM that was rendered before the fault arrived. Blanking format/money afterwards changes nothing that assertion will ever look at. The scenario passes, honestly, and reports nothing.

The second reason was configuration. payment-uri is reached only when a chain rail renders an address, and the store under test offered only the manual rail. The path never executed, so no fault on it could be observed either way.

Two changes took the score to 1.0:

• Scenarios that re-render after the fault is in place. :catalog-redraws-from-the-store clicks the reload button and only then asserts on .card. :reload-re-reads-rather-than-keeping-what-it-had clears :catalog in the page's own state, asserts the cards are gone, reloads, and asserts they came back — so both the read and the draw happen under the mutant.
• MONERO_BACKEND=fake, so the chain rail exists and the address path renders at all.

The constraint is written at the head of test/e2e/storefront.edn, so that the next scenario added does not quietly reintroduce it:

;; A fault from `cljs e2e mutate` is spliced in after the opening navigation and
;; waits. A scenario whose assertions all run against DOM rendered before that
;; point cannot kill a rendering fault, so the ones below re-render first.

The general point outlives this storefront. A pass count says the assertions that were written held. A mutation score says the assertions that were written would have noticed something wrong. Only the second is a claim about coverage. A suite can be green, look complete, and be blind — 0.2 was exactly that, and no amount of staring at nine passing scenarios would have said so.