data-scope

A Clojure library to provide tools for interactively visualizing data. This version was forked from the original data-scope, which was inspired by Spyscope.

Contributors

• Trevor Brandt (@tervorz, maintainer)
• James Sofra (@sofra) (prior to the fork)
• Denis Korzunov https://github.com/disya2 (prior to the fork)
• https://github.com/sundbp (prior to the fork)

Installation

Leiningen

Add [tervorz/data-scope "0.1.3"] to your project.clj's :dependencies.

If you want data-scope to be automatically loaded and available in every project's user namespace, add the following to the :user profile in ~/.lein/profiles.clj:

:dependencies [[tervorz/data-scope "0.1.3"]]
:injections [(require 'ds)]

Boot

After requiring the namespace, you must also run (boot.core/load-data-readers!) to get the reader tags working. Using a ~/.boot/profile.boot file:

(set-env! :dependencies #(conj % '[tervorz/data-scope "0.1.3"]))
(boot.core/load-data-readers!)
(require 'ds)

Usage

Data-scope includes a number of reader tools for visualizing Clojure data.

To use in any namespace, you'll need to require it:

(ns my.namespace
  (:require [ds]))
  
(defn my-function
  [input]
  (->> #pp input
       #pp->> (map inc)
       (apply +)))

Currently there are readers tags for visualizing data as both charts, graphs, tables and trees. There is also support for pretty printing data. See all the usage examples below for the details.

Inspectors

The inspector tags are:

• #ds/i - inspect the Clojure data structure in a tree inspector

• #ds/i-> - thread first version that can be used inside a thread first pipe line.

• #ds/i->> - thread last version that can be used inside a thread first pipe line.

• #ds/it inspect the data in a table inspector

#ds/i

user> (let [data [{:a 4 :b [4 5] :c [{:b 4 :n {:v {:d {:f [4 4]}}}}]}]]
        #ds/i data)
[{:a 4 :b [4 5] :c [{:b 4 :n {:v {:d {:f [4 4]}}}}]}]

#ds/it

user> (let [data [{:first-name "James" :last-name "Sofra" :age 36} {:first-name "Ada" :last-name "Lovelace":age 201 }]]
        #ds/it data)
[{:first-name "James" :last-name "Sofra" :age 36} {:first-name "Ada" :last-name "Lovelace":age 201 }]

user> (let [data [(range 9 19) (range 4 14) (range 2 12) (range 20 30)]]
        #ds/it data)
[(9 10 11 12 13 14 15 16 17 18) (4 5 6 7 8 9 10 11 12 13) (2 3 4 5 6 7 8 9 10 11) (20 21 22 23 24 25 26 27 28 29)]

Pretty Print

The pretty print tags are:

• #ds/pp (or just #pp) - Pretty print a form

• #ds/pp-> (or just #pp->) - thread first version that can be used inside a thread first pipe line.

• #ds/pp->> (or just #pp->>) - thread last version that can be used inside a thread first pipe line.

• #ds/pt - Print data as a table

• #ds/pt-> - thread first version that can be used inside a thread first pipe line.

• #ds/pt->> - thread last version that can be used inside a thread first pipe line.

#pp

user> (let [data [{:a 4 :b (range 20) :c [{:b 4 :n {:v {:d {:f [4 4]}}}}]}]]
        #pp data)

[{:a 4,
  :b (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19),
  :c [{:b 4, :n {:v {:d {:f [4 4]}}}}]}]

[{:a 4, :b (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19), :c [{:b 4, :n {:v {:d {:f [4 4]}}}}]}]

user> (->> [1 2 3]
        #pp->> (map (fn [x] (inc x)))
        (map (fn [x] (* x x))))

(2 3 4)

(4 9 16)

#ds/pt

user> (let [data [{:first-name "James" :last-name "Sofra" :age 36} {:first-name "Ada" :last-name "Lovelace":age 201 }]]
        #ds/pt data)

| :first-name | :last-name | :age |
|-------------+------------+------|
|       James |      Sofra |   36 |
|         Ada |   Lovelace |  201 |

[{:first-name "James" :last-name "Sofra" :age 36} {:first-name "Ada" :last-name "Lovelace":age 201 }]

user> (let [data [(range 9 19) (range 4 14) (range 2 12) (range 20 30)]]
        #ds/pt data)

|  0 |  1 |  2 |  3 |  4 |  5 |  6 |  7 |  8 |  9 |
|----+----+----+----+----+----+----+----+----+----|
|  9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 |
|  4 |  5 |  6 |  7 |  8 |  9 | 10 | 11 | 12 | 13 |
|  2 |  3 |  4 |  5 |  6 |  7 |  8 |  9 | 10 | 11 |
| 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 |

[(9 10 11 12 13 14 15 16 17 18) (4 5 6 7 8 9 10 11 12 13) (2 3 4 5 6 7 8 9 10 11) (20 21 22 23 24 25 26 27 28 29)]

Graphs

The graph tags are:

• #ds/graph - graph viz
• #ds/tree - tree viz
• #ds/trie - trie viz
• #ds/dot - dot graph viz

Graph Examples

#ds/graph

user> (let [data {:a [:b :c]
                  :b [:c]
                  :c [:a]}]
        #ds/graph data)
{:a [:b :c], :b [:c], :c [:a]}

#ds/tree

user> (let [data [[1 [2 3]] [4 [5]]]]
        #ds/tree data)
[[1 [2 3]] [4 [5]]]

#ds/trie

user> (let [data '([1 2] ([3 4] ([5 6 7])))]
        #ds/trie data)
([1 2] ([3 4] ([5 6 7])))

Charts

Each of the chart tags may visualize:

• Numeric sequences
  • (range 10)
• Sequences of numeric sequences
  • [(range 10) (reverse (range 10))]
• Maps with numeric sequences as values
  • {:a (range 10) :b (reverse (range 10))}
  • except #ds/p which can visualize a map with numeric values as values
    • {:a 1 :b 2}
• Reference types (atom, agent, var, ref) containing any of the above data types.
• A watch will be added and the chart will update on state transition of the reference.
• See an example in the #ds/b examples below.
• The is no reference type support at this time for histograms.

The tags are:

• #ds/b - bar chart
• #ds/b-sum - row wise summed bar chart
• #ds/b-max - row wise maximum bar chart
• #ds/b-min - row wise minimum bar chart
• #ds/b-sum* - column wise summed bar chart
• #ds/b-max* - column wise maximum bar chart
• #ds/b-min* - column wise minimum bar chart
• #ds/l - line chart
• #ds/l-sum - row wise summed line chart
• #ds/l-max - row wise maximum line chart
• #ds/l-min - row wise minimum line chart
• #ds/l-sum* - column wise summed line chart
• #ds/l-max* - column wise maximum line chart
• #ds/l-min* - column wise minimum line chart
• #ds/a - area chart
• #ds/a-sum - row wise summed area chart
• #ds/a-max - row wise maximum area chart
• #ds/a-min - row wise minimum area chart
• #ds/a-sum* - column wise summed area chart
• #ds/a-max* - column wise maximum area chart
• #ds/a-min* - column wise minimum area chart
• #ds/sa - stacked area chart
• #ds/sa-sum - row wise summed stacked area chart
• #ds/sa-max - row wise maximum stacked area chart
• #ds/sa-min - row wise minimum stacked area chart
• #ds/sa-sum* - column wise summed stacked area chart
• #ds/sa-max* - column wise maximum stacked area chart
• #ds/sa-min* - column wise minimum stacked area chart
• #ds/p - pie chart
• #ds/p-sum - row wise summed pie chart
• #ds/p-max - row wise maximum pie chart
• #ds/p-min - row wise minimum pie chart
• #ds/p-sum* - column wise summed pie chart
• #ds/p-max* - column wise maximum pie chart
• #ds/p-min* - column wise minimum pie chart
• #ds/hf - histogram frequency chart
• #ds/hd - histogram density chart

Print Limits

Data-scope provides print limits that mirror the Clojure core *print-length* and *print-level*. The Data-scope limits are provided as atoms ds-print-length and ds-print-level, these will limit the printing of titles and legends within the carts. They can be set just as other atoms are set, e.g. (reset! ds-print-length 10)

Chart Examples

#ds/b

user> #ds/b (range 10)

user> #ds/b [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

user> (def foo (atom (range 10)))
#'user/foo
user> #ds/b foo
Watching chart with watch - :ds-chart-watcher-d60f0bdc-418e-4f55-8d98-5d449b3dcac1
#atom[(0 1 2 3 4 5 6 7 8 9) 0x39b56714]
user> (swap! foo reverse)
#atom[(9 8 7 6 5 4 3 2 1 0) 0x39b56714]

#ds/b-sum - row wise sum

user> #ds/b-sum [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b-sum {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/b-sum* - column wise sum

user> #ds/b-sum* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b-sum* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/b-max - row wise max

user> #ds/b-max [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b-max {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/b-max* - column wise max

user> #ds/b-max* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b-max* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/b-min - row wise min

user> #ds/b-min [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b-min {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/b-min* - column wise min

user> #ds/b-min* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/b-min* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l

user> #ds/l (range 10)

user> #ds/l [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l-sum - row wise sum

user> #ds/l-sum [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l-sum {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l-sum* - column wise sum

user> #ds/l-sum* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l-sum* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l-max - row wise max

user> #ds/l-max [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l-max {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l-max* - column wise max

user> #ds/l-max* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l-max* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l-min - row wise min

user> #ds/l-min [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l-min {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/l-min* - column wise min

user> #ds/l-min* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/l-min* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a

user> #ds/a (range 10)

user> #ds/a [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a-sum - row wise sum

user> #ds/a-sum [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a-sum {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a-sum* - column wise sum

user> #ds/a-sum* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a-sum* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a-max - row wise max

user> #ds/a-max [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a-max {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a-max* - column wise max

user> #ds/a-max* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a-max* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a-min - row wise min

user> #ds/a-min [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a-min {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/a-min* - column wise min

user> #ds/a-min* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/a-min* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa

user> #ds/sa (range 10)

user> #ds/sa [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa-sum - row wise sum

user> #ds/sa-sum [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa-sum {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa-sum* - column wise sum

user> #ds/sa-sum* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa-sum* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa-max - row wise max

user> #ds/sa-max [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa-max {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa-max* - column wise max

user> #ds/sa-max* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa-max* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa-min - row wise min

user> #ds/sa-min [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa-min {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/sa-min* - column wise min

user> #ds/sa-min* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/sa-min* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/p

user> #ds/p (range 10)

user> #ds/p [(range 2) [8 1 2] [2 4]]

user> #ds/p {:a 10, :b 2, :c 6}

#ds/p-sum - row wise sum

user> #ds/p-sum [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/p-sum {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/p-sum* - column wise sum

user> #ds/p-sum* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/p-sum* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/p-max - row wise max

user> #ds/p-max [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/p-max {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/p-max* - column wise max

user> #ds/p-max* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/p-max* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/p-min - row wise min

user> #ds/p-min [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/p-min {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/p-min* - column wise min

user> #ds/p-min* [(range 10) [20 1 2 23 8 3 7 4 6 5] (reverse (range 4 14))]

user> #ds/p-min* {:a (range 10), :b [20 1 2 23 8 3 7 4 6 5], :c (reverse (range 4 14))}

#ds/hf

user> #ds/hf [8 8 8 8 2 2 2 3 4 7 7 7 1 1 1]

user> #ds/hf [(range 10) [8 8 8 8 2 2 2 3 4 7 7 7 1 1 1]]

user> #ds/hf {:a (range 10), :b [8 8 8 8 2 2 2 3 4 7 7 7 1 1 1]}

#ds/hd

user> #ds/hd [8 8 8 8 2 2 2 3 4 7 7 7 1 1 1]

user> #ds/hd [(range 10) [8 8 8 8 2 2 2 3 4 7 7 7 1 1 1]]

user> #ds/hd {:a (range 10), :b [8 8 8 8 2 2 2 3 4 7 7 7 1 1 1]}

License

Copyright © 2016 James Sofra

Distributed under the Eclipse Public License, the same as Clojure.