![Query benchmarks dashboard example](https://github.com/zackteo/xtdb/raw/efb1c473262ce9ce88eccef9508f3e7e107cfd12/docs/resources/modules/ROOT/pages/bench-dash-query.png)
Rigorous benchmarking supports the team’s development activities by providing rapid feedback on complex changes and a regular benchmark process allows the team to discover and address performance regressions during day-to-day development, well in advance of any releases.
The bench
sub-project contains a number of benchmarking suites that are run nightly on standard AWS infrastructure (m5.xlarge
instances via CloudFormation) to support regression testing of new development and validate all performance improvement changes.
These suites cover a wide range of analysis at a variety of scale factors, including: graph query performance, bitemporal query performance, ingestion performance (throughput & latency), and disk usage.
Officially published benchmark results are forthcoming, but in the meantime please feel welcome to explore bench
independently and get in touch if you need any assistance.
For an example of independent benchmarking by the community, see the "XTDB Connector" performance analysis by the Linux Foundation’s Egeria project, which compares XTDB against an existing JanusGraph Connector.
See the TPC-H test fixture for details.
WatDiv has been developed to measure how an RDF data management system performs across a wide spectrum of SPARQL queries with varying structural characteristics and selectivity classes.
Benchmarking is performed nightly against the WatDiv test suite. These tests demonstrate comprehensive RDF subgraph matching. Note that XTDB does not natively implement the RDF specification and only a simplified subset of the RDF tests have been translated for use in XTDB. XTDB generates Datalog, which shares many of the same properties as SPARQL, directly from the SPARQL definitions.
Subgraph matching is a fundamental kind of graph query and is a common mechanism for computing query results in SPARQL: RDF triples in both the queried RDF data and the query pattern are interpreted as nodes and edges of directed graphs, and the resulting query graph is matched to the data graph using variables as wildcards.
The diversity and scale of subgraph matching in the WatDiv tests provides a particularly helpful measure of end-to-end performance across many dimensions.
SELECT ?v0 ?v3 ?v4 ?v8 WHERE {
?v0 sorg:legalName ?v1 .
?v0 gr:offers ?v2 .
?v2 sorg:eligibleRegion wsdbm:Country5 .
?v2 gr:includes ?v3 .
?v4 sorg:jobTitle ?v5 .
?v4 foaf:homepage ?v6 .
?v4 wsdbm:makesPurchase ?v7 .
?v7 wsdbm:purchaseFor ?v3 .
?v3 rev:hasReview ?v8 .
?v8 rev:totalVotes ?v9 .
}
XTDB’s test suite incorporates a large number of DataScript's query tests, demonstrating extensive functional coverage of DataScript’s Datalog implementation.
The Lehigh University Benchmark is developed to facilitate the evaluation of Semantic Web repositories in a standard and systematic way. The benchmark is intended to evaluate the performance of those repositories with respect to extensional queries over a large data set that commits to a single realistic ontology. It consists of a university domain ontology, customizable and repeatable synthetic data, a set of test queries, and several performance metrics.
Several Datalog tests from the Racket Datalog examples have been translated and re-used within XTDB’s query tests.
Specifically, from https://github.com/racket/datalog/tree/master/tests/examples
tutorial.rkt
path.rkt
revpath.rkt
bidipath.rkt
sym.rkt
Several Datalog examples from a classic Datalog paper have been translated and re-used within XTDB’s query tests.
Specifically:
"sgc"
3 examples of "stratified Datalog"
Can you improve this documentation? These fine people already did:
The Alchemist & Steven DeobaldEdit on GitHub
cljdoc is a website building & hosting documentation for Clojure/Script libraries
× close