ductile.document — threatgrid/ductile 0.5.0

bulk clj

(bulk conn actions opts)

(bulk {:keys [version] :as conn} actions opts max-size)

Inputs: ([conn :- ESConn actions :- BulkActions opts :- CRUDOptions] [{:keys [version], :as conn} :- ESConn actions :- BulkActions opts :- CRUDOptions max-size :- (s/maybe s/Int)])

Bulk actions on ES

bulk-create-docs clj

(bulk-create-docs conn docs opts)

(bulk-create-docs conn docs opts max-size)

Inputs: ([conn :- ESConn docs :- [(s/pred map?)] opts :- CRUDOptions] [conn :- ESConn docs :- [(s/pred map?)] opts :- CRUDOptions max-size :- (s/maybe s/Int)])

create multiple documents on ES

bulk-delete-docs clj

(bulk-delete-docs conn docs opts)

(bulk-delete-docs conn docs opts max-size)

Inputs: ([conn :- ESConn docs :- [(st/open-schema {:_id s/Str, :_index s/Str})] opts :- CRUDOptions] [conn :- ESConn docs :- [s/Any] opts :- CRUDOptions max-size :- (s/maybe s/Int)])

delete multiple documents on ES

bulk-index-docs clj

(bulk-index-docs conn docs opts)

(bulk-index-docs conn docs opts max-size)

Inputs: ([conn :- ESConn docs :- [(s/pred map?)] opts :- CRUDOptions] [conn :- ESConn docs :- [s/Any] opts :- CRUDOptions max-size :- (s/maybe s/Int)])

index multiple documents on ES

bulk-update-docs clj

(bulk-update-docs conn docs opts)

(bulk-update-docs conn docs opts max-size)

Inputs: ([conn :- ESConn docs :- [(st/open-schema {:_id s/Str, :_index s/Str})] opts :- CRUDOptions] [conn :- ESConn docs :- [s/Any] opts :- CRUDOptions max-size :- (s/maybe s/Int)])

update multiple documents on ES

bulk-uri clj

(bulk-uri uri)

make an uri for bulk action

BulkActions clj

BulkOps clj

byte-size clj

(byte-size s)

Count the size of the given string in bytes.

count-docs clj

(count-docs es-conn index-name)

(count-docs {:keys [uri request-fn] :as conn} index-name query)

Inputs: ([{:keys [uri request-fn], :as conn} :- ESConn index-name :- s/Str query :- (s/maybe ESQuery)] [es-conn :- ESConn index-name :- s/Str])

Count documents on ES matching given query.

count-uri clj

(count-uri uri index-name)

make an uri for search action

create-doc clj

(create-doc conn index-name doc opts)

(create-doc conn index-name doc-type doc opts)

Inputs: ([conn :- ESConn index-name :- s/Str doc :- s/Any opts :- CRUDOptions] [conn :- ESConn index-name :- s/Str doc-type :- (s/maybe s/Str) doc :- s/Any opts :- CRUDOptions])

create a document on es return the created document

default-limit clj

default-retry-on-conflict clj

delete-by-query clj

(delete-by-query {:keys [uri request-fn] :as conn} index-names q opts)

Inputs: [{:keys [uri request-fn], :as conn} :- ESConn index-names :- [s/Str] q :- ESQuery opts :- CRUDOptions]

delete all documents that match a query in an index

delete-by-query-uri clj

(delete-by-query-uri uri index-names)

Inputs: [uri index-names]

delete-doc clj

(delete-doc conn index-name id opts)

(delete-doc {:keys [uri request-fn version] :as conn}
            index-name
            doc-type
            id
            opts)

Inputs: ([conn :- ESConn index-name :- s/Str id :- s/Str opts :- CRUDOptions] [{:keys [uri request-fn version], :as conn} :- ESConn index-name :- s/Str doc-type :- (s/maybe s/Str) id :- s/Str opts :- CRUDOptions])

delete a document on es, returns boolean

delete-doc-uri clj

make an uri for doc deletion

format-bulk-doc-fn clj

(format-bulk-doc-fn op-type)

helper to prepare a bulk operation

format-bulk-docs clj

(format-bulk-docs op-type documents)

generates the content for a bulk operation

format-bulk-res clj

(format-bulk-res bulk-res-list)

generate-search-params clj

(generate-search-params query aggs params)

get-doc clj

(get-doc conn index-name id opts)

(get-doc {:keys [uri request-fn version] :as conn} index-name doc-type id opts)

Inputs: ([{:keys [uri request-fn version], :as conn} :- ESConn index-name :- s/Str doc-type :- (s/maybe s/Str) id :- s/Str opts :- CRUDOptions] [conn :- ESConn index-name :- s/Str id :- s/Str opts :- CRUDOptions])

get a document on es and return only the source

get-doc-uri clj

make an uri for doc retrieval

index-doc clj

(index-doc conn index-name doc opts)

(index-doc conn index-name doc-type doc opts)

Inputs: ([conn :- ESConn index-name :- s/Str doc-type :- (s/maybe s/Str) doc :- s/Any opts :- CRUDOptions] [conn :- ESConn index-name :- s/Str doc :- s/Any opts :- CRUDOptions])

index a document on es return the indexed document

index-doc-uri clj

(index-doc-uri uri index-name doc-type id)

make an uri for document index

params->pagination clj

(params->pagination {:keys [sort_by sort_order offset limit search_after]
                     :or {sort_order :asc limit pagination/default-limit}})

partition-json-ops clj

(partition-json-ops json-ops max-size)

Return a lazy sequence of lists of ops whose size is less than max-size. If a json-op exceeds the max size, it is included in a list of one element.

query clj

(query conn index-name q opts)

(query {:keys [uri request-fn] :as conn}
       index-name
       q
       aggs
       {:keys [full-hits?] :as opts})

Inputs: ([{:keys [uri request-fn], :as conn} :- ESConn index-name :- (s/maybe s/Str) q :- (s/maybe ESQuery) aggs :- (s/maybe ESAggs) {:keys [full-hits?], :as opts} :- s/Any] [conn index-name q opts])

Search for documents on ES using any query. Performs aggregations when specified.

search-docs clj

(search-docs conn index-name es-query all-of opts)

Inputs: [conn :- ESConn index-name :- (s/maybe s/Str) es-query :- (s/maybe ESQuery) all-of :- (s/maybe #:s{Any s/Any}) opts :- s/Any]

Search for documents on ES using a query string search. Also applies a filter map, converting the values in the all-of into must match terms.

search-uri clj

(search-uri uri index-name)

make an uri for search action

sort-params clj

(sort-params sort_by sort_order)

string->input-stream clj

(string->input-stream s)

update-by-query clj

(update-by-query {:keys [uri request-fn] :as conn} index-names form-params opts)

Inputs: [{:keys [uri request-fn], :as conn} :- ESConn index-names :- [s/Str] form-params :- (s/maybe UpdateByQueryParams) opts :- CRUDOptions]

Performs an update on every document in the data stream or index without modifying the source, which is useful for picking up mapping changes.

update-by-query-uri clj

(update-by-query-uri uri index-names)

Inputs: [uri index-names]

update-doc clj

(update-doc conn index-name id doc opts)

(update-doc {:keys [uri request-fn version] :as conn}
            index-name
            doc-type
            id
            doc
            opts)

Inputs: ([{:keys [uri request-fn version], :as conn} :- ESConn index-name :- s/Str doc-type :- (s/maybe s/Str) id :- s/Str doc :- s/Any opts :- CRUDOptions] [conn :- ESConn index-name :- s/Str id :- s/Str doc :- s/Any opts :- CRUDOptions])

update a document on es return the updated document

update-doc-uri clj

(update-doc-uri uri index-name id)

(update-doc-uri uri index-name doc-type id)

make an uri for document update