Helper namespace related to configuration,
all configuration options should have a default and a specific key that can override it

e.g :base-dir
default is :base-dir, but for key :a doing :a.base-dir can override it.

entry point namespace, for usage see https://github.com/gerritjvv/fileape

(require '[fileape.core :refer :all])
(import '[java.io File DataOutputStream])
(defn callback-f [{:keys [file]}]
   (prn "File rolled " file))
   (def ape2 (ape {:codec :gzip
                   :base-dir "testdir"
                   :check-freq 5000
                   :rollover-size 134217728
                   :rollover-timeout 60000
                   :roll-callbacks [callback-f]}))
                   (write ape2 "abc-123" (fn [{:keys [^DataOutputStream out]}]
                                               (.writeInt out (int 1))))
    (close ape2)

Basic api for doing async file IO using refs and agents

Provide protocols to allow extension for different storage and compression architectures
e.g OutputStream gzip, lzo and partquet

Implement java write support for parquet
the different types and cases that can be written is
done via multimethods while the actual WriterSupport is reified

Support writing parquet files
Usage:
  (def pf (open-parquet-file! k (parse-schema "message test{ required binary name; repeated int32 age;}") fname))
  (write! pf {"name" "hi" "age" [1 2 3]})
  (close! pf)

  

Random io util functions for the io and io-plugin namespaces