bosquet.llm.generator — io.github.zmedelis/bosquet 2025.03.28

->chatml clj

(->chatml messages)

Convert messages in tuple format ot ChatML. There is a caviat. When content might not be a string (a likely case when one LLM call result in say JSON feeds into another call.

append-generation-instruction clj

(append-generation-instruction string-template)

If template does not specify generation function append the default one.

call-llm clj

(call-llm llm-config
          {llm-impl wkk/service
           model-params wkk/model-params
           use-cache wkk/cache
           :as properties}
          messages)

Make a call to the LLM service.

• llm-config provides a map containing LLM service configurations, the LLM to call is specified in
• properties providing model parameters and other details of LLM invocation
• messages contains the context/prompt to be supplied to LLM.

chat clj

(chat llm-config messages inputs)

Chat completion using

• llm-config holding LLM service configuration
• messages chat message tuples [[:system "You are ..."] [:user "Please, ..."] [:assistant {...}]]
• inputs data map to fill the tempalte slots

complete-graph clj

(complete-graph llm-config graph vars-map)

Completion case when we are processing prompt graph. Main work here is on constructing the output format with usage and completions sections.

complete-template clj

(complete-template llm-config template vars-map)

Completion for a case when we have simple string prompt

completions clj

Result map key holding LLM generated parts

conversation clj

Result map key holding full chat conversation including generated parts

default-template-completion clj

Simple string template generation case does not create var names for the completions, compared to Map generation where map keys are the var names.

This is a key for completion entry

default-template-prompt clj

Simple string template generation case does not create var names for the completions, compared to Map generation where map keys are the var names.

This is a key for prompt entry

find-refering-templates clj

(find-refering-templates var-name context-map)

Given all the templates in a context-map find out which ones have references to var-name

fun clj

(fun impl args)

gen-environment clj

(gen-environment llm-config context vars-map)

generate clj

(generate messages)

(generate messages inputs)

(generate env messages inputs)

Generate completions for various modes. Generation mode is determined by the type of the messages:

• Vector of tuples, triggers chat mode completion

   [[:system "You are ..."]
    [:user "Please, ..."]
    [:assistant {...}]]
  

• A map, triggers graph mode completion

  {:question-answer "Question: {{question}}"
   :answer          {...}}
  

• A string results in a template completion mode

env/config holds configuration to make LLM calls and inputs has a data map for template slot filling.

llm clj

(llm service-or-model & args)

A helper function to create LLM spec for calls during the generation process. It comes back with a map constructed from service-or-model and args. service-or-model can be one of:

• serivice (like openai, mistral, ...), in this case args need to specify {:llm/model-params {:model :x}}
• model in this case service will be determined from env/model-providers, no need to specify {:llm/model-params {:model :x}}

{:llm/service      service
 :llm/cache        true
 :llm/model-params params}

run-node-function clj

(run-node-function node available-data)

Run a function definition in the prompt tree.

• node is the function defining node in the tree
• available-data already resolved data, must contain function params

top-level-template clj

(top-level-template index context)

usage clj

Result map key holding LLM token usage data