Implementing an operation in Koi-clj (in Clojure)

In this document, we'll discuss using koi-clj as a template to implement a new operation in Clojure.

In order to implement a new operation, a developer must

• Implement the protocols declared protocols.clj
• Create the metadata, which is discussed in detail here

The protocol declares three methods

• invoke-sync This method is called for the synchronous invocation of an operation.
• invoke-async This method is called for the asynchronous invocation of an operation. It must return immediately, and return a job id.
• get-params This method returns the metadata (input and output) for the operation.

Lets look at the implementation of an iris predictor. This is an example of a machine learning classifier which is trained on the Iris dataset. The implementation is dummy predictor which takes in an instance from the iris dataset and always predicts setosa as the predicted class.

Creating a deftype

First, we need to create a deftype that implements the protocol abstractions.

Note that:

• the invoke-* methods call the process function (explained later)
• get-params is a clojure.spec compliant definition of the inputs

(deftype PredictIrisClass [jobs jobids]

  prot/PSyncInvoke
  (invoke-sync [_ args]
    (process args predict-class))

  prot/PAsyncInvoke
  (invoke-async [_ args]
    (async-handler jobids jobs #(process args predict-class)))

  prot/PParams
  (get-params [_]
    ::params))

Function that runs the operation.

(defn predict-class
  [agent {:keys [dataset]}]
  ;;get the content of the input dataset.
 ;;this would be a test dataset with iris instances

  (let [ast (get-asset agent dataset)
        cont (-> ast s/content s/to-string)]
    ;;this should return a function
    (fn []
      (let [isp (clojure.string/split cont #"\n" )
      ;;create predictions, which adds the value for predicted class
            predictions (->> (into [(str (first isp) ",predclass")]
                       (mapv #(str % ",setosa") (rest isp)))
                 (clojure.string/join "\n"))
                 ;;;return a map with the parametername (predictions) and the content
            res {:results [{:param-name :predictions
                            :type :asset
                            :content predictions}]}]
        ;(info " result of predict-class " res)
        res))))

Adding it to the list of operations.

op_handler.clj maintains a list of operations. On Koi startup, the operation handler

• registers the metadata for each operation using an Agent.
• registers a rest endpoint corresponding to the ID of the operation.

The complete source can be found here