Blah lets you stream babbling itself!

A ClojureScript library for reading audio input (i.e microphone input) as a stream of data. Record it! Transform it! Send it some place for transcription! The possibilities are endless!

Table of contents

• Usage
  • Inputs
  • Transducers
  • Caveats
• Data
• Development
• Advanced Compilation

Usage

Blah is built on top of core.async. It produces a channel that yields audio data as a stream of Float32 samples.

(require '[blah.core :as blah])

;;; Start a session using the default input
(let [session (blah/listen)]
  (go-loop []
    (let [audio-data (<! session)]
      (when audio-data
        (process-audio-data audio-data)
        (recur)))))

Inputs

If a specific input is desired, they can be obtained by creating an input channel or querying via query-inputs

(require '[blah.core :as blah])

(blah/query-inputs
  (fn [inputs]
    (let [session (blah/listen (first inputs))]
      (go-loop []
        (let [audio-data (<! session)]
          (when audio-data
            (process-audio-data audio-data)
            (recur)))))))

An input channel can be useful for keeping tabs on audio inputs as they become available/unavailable

(let [input-ch (blah/input-ch)
      ui-state (atom)]
  (go-loop []
    (let [audio-inputs (a/<! input-ch)]
      (when audio-inputs
        (swap! ui-state assoc :inputs audio-inputs)
        (recur)))))

Transducers

A transducer can be supplied, and this works exactly as it does with a core.async channel:

;;; We give nil as the input to use the default audio input
(let [session (blah/listen nil blah.transforms/float32)]
  (go-loop []å
    ,,,))

If a transducer is omitted it will use blah.transforms/float32.

Caveats

Many browsers require a user action to initiate audio contexts and enumerate devices. This means a robust application of blah will start a session as part of a click handler or some other user interaction. See dev/cljs/user.cljs for an example of developing with blah.

Data

The magic behind blah is that data is streamed in realtime via an AudioWorkletProcessor.

This processor sends data to blah as JS types, and transforms are intended to make this data more accessible to a Clojure developer.

The implementation of the processor looks like the following:

class WorkletProcessor extends AudioWorkletProcessor {
  process (inputs, outputs, params) {
    const input = inputs[0];

    let frames = [];
    for (let ch = 0; ch < input.length; ch++) {
        const samples = input[ch];
        frames[ch] = samples;
    }

    this.port.postMessage(frames);
    
    return true;
  }
}

This sends data as a JS array of channel samples, such that data[0] would be an array of audio samples for the first channel of the input, and data[1] would be an array of audio samples for the second channel, and so on, and so on. All samples are Float32 types from JavaScript.

Transforms work with this data structure.

See src/blah/transforms.cljs for an example of transforming audio data.

Development

Blah is developed using a vanilla ClojureScript repl. This plays well with development environments like Calva, or a plain repl.

See the ClojureScript quick start for info on running a repl.

See dev/cljs/user.cljs. This is where development takes place.

Advanced Compilation

Blah is developed using ClojureScript 1.11.4+

Versions prior to this one should work fine, but they may have issues with advanced compilation. The reason for this is that externs for the native AudioContext and audioWorklet were not fully in place.

As of 1.11.4, advanced compilation is no problem.