fileape.parquet.writer — fileape 2.0.3

Support writing parquet files Usage: (def pf (open-parquet-file! k (parse-schema "message test{ required binary name; repeated int32 age;}") fname)) (write! pf {"name" "hi" "age" [1 2 3]}) (close! pf)

as-path clj

(as-path file)

close! clj

(close! {:keys [record-writer]})

hadoop-conf clj

(hadoop-conf)

open-parquet-file! clj

(open-parquet-file! k type file & conf)

parquet-metadata clj

(parquet-metadata file)

Returns the ParquetMetadata from a file file can be a string, File, or Hadoop Path. Can be used to validate files for correctness also.

parquet-ok? clj

(parquet-ok? file)

Return nil when the parquet file is corrupt or cannot be read

parse-schema clj

(parse-schema schema)

Takes a schema string and parses it into a MessageType

record-writer clj

(record-writer k conf schema file codec)

Create a Parquet Record write that will accept Maps as groups and messages and primitive Java/Clojure types as values

write! clj

(write! {:keys [record-writer]} msg)