Liking cljdoc? Tell your friends :D

hive-compose

docker compose sections as a hive IAddon. Name what you want to work on, a service or a native compose profile, and exactly that runs, closed over the compose file's depends_on. Take it down later and only that section goes; anything another running section needs stays up. An idle reaper stops sections nobody touched within their TTL, so no stack keeps loading the machine.

Your compose YAML is the source of truth. The addon never parses it: compose itself resolves every closure (config --format json <svc>), lists services and profiles (config --services, config --profiles) and reports what runs (ps --format json).

Setup: point it at your compose projects

config.edn → :addons {"hive.compose" {...}}:

{:compose/projects
 [{:project/id "sisf"
   :project/dir "~/PP/funeraria/dc"
   :project/files ["docker-compose.yml" "docker-compose.override.yml"]  ; optional
   :project/ttl-minutes 45}]}                                            ; optional

Use: name a target

compose targets                      # always-on services + native profiles per project
compose up target=sisf-web           # section sisf/sisf-web: sisf-web + its depends_on closure
compose up target=crm                # a native profile: its own services + closure,
                                     #   NOT the always-on base a bare --profile adds
compose up target=sisf/sisf-web,sisf-crm   # several targets, one section
compose switch target=sisf-crm       # stop what only the current section needed
compose down target=sisf-web         # remove that section's containers (volumes never)
compose status                       # active sections, idle time, reaper countdown

Declare runtime needs in the YAML

A section is exactly compose's closure, so what an app calls at runtime belongs in its depends_on. For a soft dependency (start it with the app, but don't fail without it) use compose's required: false:

sisf-web:
  profiles: [frontend]
  depends_on:
    envoy:
      condition: service_started
      required: false

With that, up target=sisf-web brings envoy and envoy's own closure (auth, keycloak, postgres, redis...) and nothing else.

Host programs: what runs beside the containers

A dev stack is rarely containers alone: a shadow-cljs watch or a JVM runs on the host and the containers proxy to it. Declare those as programs of the project, and a section starts them once its containers are up:

{:compose/projects
 [{:project/id "sisf"
   :project/dir "~/PP/funeraria/dc"
   :project/programs
   [{:program/id "sisf-web"
     :program/dir "../sisf-web"          ; relative to :project/dir, or absolute, or ~
     :program/with ["envoy"]}            ; starts with a section that runs envoy
    {:program/id "inventory"
     :program/dir "../inventory"
     :program/kind :deps                 ; the directory also has a shadow-cljs.edn
     :program/with ["inventory-frontend-dev"]}]}]}

A directory with a Clojure build file needs no command. The build file decides how it starts, and it always starts with an nREPL:

build filestarts asnREPL port
shadow-cljs.ednnpx shadow-cljs watch <every build>its :nrepl {:port N}
deps.edn with an :nrepl aliasclojure -M:nreplthe alias's --port
deps.edn without oneclojure -Sdeps <nrepl + cider-nrepl> -M -m nrepl.cmdline:program/nrepl-port, else the .nrepl-port it writes
project.cljlein repl :headless:program/nrepl-port
bb.ednbb nrepl-server:program/nrepl-port (1667)

:program/command (an argv or a shell string) replaces the detected command, and the port is still read from the build file; any other directory needs one. :program/kind picks the build file when a directory has several, :program/env adds environment, :program/with names the services the program pairs with (none: every section of the project).

up answers each program's pid, log and nREPL port; status and programs say whether the process lives and whether the port answers yet. A program two sections want runs once and stops with the last of them, on down, stop, switch or the reaper. It runs detached in its own process group, with output appended to ~/.local/state/hive-compose/logs/<project>-<program>.log, and is tracked by pid, so it survives a restart of the host and is still stopped afterwards. A program that fails to start is reported and never fails the section. A program whose nREPL port already answers is somebody else's: it is reported as external?, not started a second time and never stopped.

Environment and secrets

:project/env, :profile/env (presets), :program/env and the env param of up/switch/restart take {VAR value}. A value is a string, or a secret reference read from pass:

{:project/id "dc"
 :project/dir "~/PP/funeraria/dc"
 :project/env {"CRM_WA_ACCESS_TOKEN" {:pass "Funeraria/Whatsapp/token"}
               "SISF_CRM_SRC" "../../funeraria-wt/dc-staging-view-sisf-crm"}}
compose up target=sisf-crm env={"EXTRA_TOKEN": {"pass": "Funeraria/other"}}

A reference is read (pass <path>, first line, trimmed) only when containers are created (up, switch, restart recreate=true) or a host program starts, and the value goes to that process alone. Every other compose call (config, ps, stop, down, logs, an in-place restart) is given the rendering pass:<path>, so interpolation still works and no store is read. The value is never kept in the state file, never shown by status, ps, targets, programs or logs, and what docker prints has it replaced by the rendering. An entry that cannot be read (missing, gpg locked, pass absent) fails that up before docker runs, naming the variable, the reference and the reason; a program whose env cannot be read is reported and not started. :compose/resolver injects the stores ({:pass ISecretBackend}) in place of pass.

Tool compose

commandeffect
targetswhat each configured project offers
programsthe host programs each project declares, how each starts, and whether its nREPL answers
up target=… \| profile=…start a section alongside whatever runs
switch target=…make it current; stop what only the previous section needed
down / stopremove / stop a section's containers, sparing services other active sections need
touchreset a section's idle clock (ps and logs touch too)
ps / logs tail=Ninspect
statusactive sections, idle time, seconds until reaped
reaprun the reaper now
adopttake charge of running containers nobody owns
projectsevery compose project on the host, managed ones marked
reloadre-read config
claim target=… note=…an agent declares it works on a section (bringing it up if needed); answers the shared nREPL per module
release target=… [stop=true]give the claim up; with none left the section is orphaned, or stopped now with stop
whoper section: claims, what is using it, when it is reaped; per agent: the sections it works in
nrepl path=…which shared nREPL drives that file or directory (no path: every route)

Hooks for other addons: :compose/touch!, :compose/status, :compose/tick!, :compose/claim!, :compose/release!, :compose/nrepl-for, :compose/who.

Swarm work

Several agents working on one app should share its running JVMs, not each boot their own, and an app nobody works on should not keep holding memory.

  • Claims. compose claim target=shop/api note="fix login" brings the section up if needed and records the claim (the agent is the call's agent, else its agent_id). The answer lists the section's nREPL routes: for each program directory, the port to drive it through (clojure_eval port=…) and one line of advice. A claim must be renewed (claim again) within :compose/claim-ttl-minutes (30); a claim made by a swarm agent also ends when that agent leaves the swarm.
  • Routes. compose nrepl path=/w/shop/api/src/core.clj answers the program whose directory holds the file most closely, whether its nREPL answers, and whether somebody outside the addon runs it (:external?, shared all the same).
  • Usage. Every reaper pass counts as use of a section: claims that hold, swarm agents whose cwd lies in the project's directories, and connections newly opened to its nREPL ports (/proc/net/tcp; a connection left open counts once). :compose/activity #{:nrepl :published} also counts new connections to the ports its containers publish (opt-in: an app's own connection pool would otherwise look like use). What is in use has its idle clock reset; who shows why.
  • Notices. When an agent starts working in a project whose section serves a ready nREPL, the swarm is told once which port drives which module. up and switch announce the routes, and the reaper announces what it freed.
  • Enforcement. Nothing in use for the TTL is stopped, as before. A section whose last claim ended (released, stale or its agent gone) is orphaned and reaped after :compose/orphan-ttl-minutes (10) of disuse instead. release stop=true frees it at once, sparing what other sections need.

With no hive host (published standalone), the swarm adapter answers absent and the addon runs on claims and connections alone.

Presets (optional)

Named presets in ~/.config/hive-mcp/compose-profiles.edn, used with profile=ID:

[{:profile/id "shop-api"
  :profile/dir "~/code/shop"
  :profile/services ["api"]
  :profile/ttl-minutes 30
  :profile/idle-action :stop}]      ; :stop (default) | :down | :none

A preset carries its own :profile/files and :profile/env ({:pass ..} values included), so an overlay compose file and the secrets it needs apply only when that preset runs, never to the project's targets. To run the overlay against the same containers, give it the project's compose name:

{:profile/id "crm-whatsapp"
 :profile/dir "~/PP/funeraria/dc"
 :profile/project "dc"
 :profile/files ["docker-compose.yml" "docker-compose.override.yml"
                 "docker-compose.crm-whatsapp.yml"]
 :profile/services ["sisf-crm"]
 :profile/env {"CRM_WA_ACCESS_TOKEN" {:pass "Funeraria/Whatsapp/token"}}}

compose up profile=crm-whatsapp then runs docker compose -p dc -f docker-compose.yml -f docker-compose.override.yml -f docker-compose.crm-whatsapp.yml up -d sisf-crm with the token read from pass at that moment. compose down profile=crm-whatsapp releases it.

:profile/project only names the compose project. A preset does not inherit that project's :project/env from the addon config, so repeat in :profile/env every value the preset's services need.

Safety

  • Only sections the addon started or adopted are ever reaped. Adoption covers presets that are running and the running containers of configured projects that no section accounts for (section <project>/adopted); unconfigured projects are listed, never touched.
  • down never passes -v: volumes survive every operation.
  • An empty service set never reaches docker (a bare compose stop would stop the whole project).
  • The active map lives in ~/.local/state/hive-compose/state.edn, so a restart still reaps what it started.

Config

:compose/projects, :compose/default-ttl-minutes (60), :compose/default-idle-action (:stop), :compose/tick-seconds (60), :compose/profiles, :compose/profiles-file, :compose/state-file, :compose/timeout-ms (60000), :compose/up-timeout-ms (600000), :compose/adopt? (true), :compose/programs? (true), :compose/log-dir, :compose/stop-grace-ms (5000, before a program's group is killed), :compose/program-grace-ms (3000, how long a started program is watched; one that exits within it is reported :status :exited with :exit-code, :log-tail and :advice, and is not held; status reports a held program that died since the same way), :compose/swarm? (true), :compose/notify? (true), :compose/activity (#{:nrepl}), :compose/claim-ttl-minutes (30), :compose/orphan-ttl-minutes (10).

:compose/activity says which new connections count as use; a project overrides it for its sections with :project/activity (a preset with :profile/activity). #{:nrepl :published} also counts new peers on the host ports the section's containers publish, so a browser or any host-side client keeps it alive. Peers whose address belongs to one of the project's own running containers (docker inspect) are left out, so pools do not count, and a keep-alive connection counts once, when first seen. who and status name the source that counted (:in-use-via, e.g. [:published]) and each section's :activity.

Development

clojure -M:test          # unit suite (stub engine)
clojure -M:integration   # against the local docker daemon

MIT. Depends on hive-addon, hive-dsl and hive-system (MIT).

Can you improve this documentation?Edit on GitHub

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
←Move to previous article
→Move to next article
Ctrl+/Jump to the search field
× close