Liking cljdoc? Tell your friends :D

Sizing & Scaling

Wagoe’s architecture (FC/IS + hexagonal ports) is meant to let you scale vertically, horizontally, or a mix — mostly by configuration rather than rewrites. This page is an honest map of how far that holds today: which knobs exist, which components are already safe to run as many replicas, and which still hold in-process state that you must account for.

Three axes

AxisMeaning

Vertical

One process, more resources. Bigger heap, larger connection/thread pools, more CPU. Pure configuration in Wagoe.

Horizontal

Many processes (replicas) behind a load balancer, sharing backing services (Postgres, Redis). Requires that no request-handling state lives in a single process.

Functional decomposition

Slice modules into separate deployables (microservices-style) that scale independently. A cross-module call that was in-process becomes a network call. Highest leverage, highest effort.

Vertical scaling — by configuration today

All sizing knobs live in resources/conf/{dev,test,prod,acc}/config.edn plus environment variables. Change the value, restart the process. No code.

KnobWhereNotes

DB connection pool

:wagoe/postgresql :poolminimum-idle, maximum-pool-size, connection-timeout-ms, idle-timeout-ms, max-lifetime-ms

HikariCP. prod default min 10 / max 50, dev 2 / 10, test (H2) 1 / 5. DB_POOL_SIZE env override. These five only. The pool map is a :closed Malli schema and the pool builder applies exactly this set, so adding keepalive-time-ms, validation-timeout-ms or leak-detection-threshold-ms does not tune anything — it fails the boot at :wagoe/db-context with "Invalid database configuration". Restoring one means adding the matching .setKeepaliveTime / .setValidationTimeout / .setLeakDetectionThreshold call in adapters/database/common/connection.clj first.

HTTP server

:wagoe/http:port, :host, :port-range

Jetty. HTTP_PORT / HTTP_HOST env. Jetty manages its own thread pool.

JVM heap / GC

JAVA_OPTS (see docker-compose.yml)

Default -Xmx512m -Xms128m -XX:+UseG1GC -XX:+UseContainerSupport. Raise -Xmx for vertical scale.

Cache (Redis) pool

:wagoe/cache:max-total, :max-idle, :min-idle, :timeout, :default-ttl

Jedis pool. prod default max-total 50 / max-idle 20 / min-idle 5.

Watch the multiplication: with N replicas each holding a pool of maximum-pool-size, total Postgres connections = N × max. Keep N × max under the server’s max_connections.

How the architecture enables horizontal scaling

Cross-module calls go through protocols defined in each module’s ports.clj (enforced by bb check:ports). Core logic depends on a protocol, never on a concrete adapter. That seam is the scaling lever: swap an in-process adapter for a distributed one — Redis, a queue, a remote service — without touching the functional core.

Several libraries already ship both adapters. The provider keyword is per library, not a convention — check it rather than assume:

:wagoe/cache    {:provider :redis ...}          ; | :in-memory
:wagoe/realtime {:provider :redis ...}          ; | :in-memory (the default)
:wagoe/events   {:provider :redis-streams ...}  ; | :in-memory — no default, throws otherwise
;; libs/jobs has :in-memory, :redis and :db adapters, but no :provider key:
;; the application constructs :wagoe/job-queue / :wagoe/job-store directly.

This is the template every other seam follows: the protocol is the contract, the distributed adapter is "just configuration" once it exists.

With one condition that is easy to leave implicit. Swapping adapters is only safe if both answer the protocol the same way, and a protocol does not enforce that — a test across both adapters does. libs/cache had thirteen divergences between its two adapters, each invisible to per-adapter suites that only ever asked one of them, until a contract sweep enumerated the surface (BOU-288). Any new seam here inherits that risk, so a second adapter comes with a sweep that runs every method against both.

Horizontal readiness matrix

ComponentN replicasDetail

Cache

Redis adapter (libs/cache/…​/adapters/redis.clj) — Nippy serialization, atomic ops. In-memory adapter is dev/test only. adapter_surface_test.clj holds both to one contract, so the swap is behaviour-preserving rather than merely type-compatible (BOU-288) — which matters here because both the rate limiter and the RPC circuit breaker keep their cross-replica state in this component. (Realtime does not — it opens its own Jedis pools.)

Jobs

Redis queue (libs/jobs/…​/adapters/redis.clj) — priority lists, worker heartbeat, retry/backoff, dead-letter. Dequeue is reliable: RPOPLPUSH atomically moves a job into a per-worker in-flight (processing) list, the worker ack`s it only once it is completed / re-enqueued / dead-lettered, and `reclaim-abandoned-jobs! (run periodically by every worker) returns jobs stranded by a crashed worker — one whose heartbeat has expired — back to the ready queue. So a kill -9 mid-job is re-run rather than lost (at-least-once; handlers must be idempotent). Durability across a Redis restart requires Redis persistence (AOF appendonly yes). Scheduled-job promotion is an atomic claim (Redis ZREM / in-memory swap-vals!), so a due job is moved to an execution queue exactly once across concurrent workers. The handler registry is per-process, but a dequeued job with no local handler is re-enqueued for another worker and dead-lettered only after it has gone unhandled for :max-requeue-age-ms (default 5 min) — no silent DLQ drop. The re-enqueue is delayed (parked in the scheduled set :requeue-delay-ms ahead) so a handlerless worker can’t reacquire it in a tight loop. Give-up is age-based, not attempt-based, so wrong-worker misses under load (or more handlerless workers than any attempt budget) can’t drop a job a slow handler-owning worker simply hadn’t polled yet. A worker started with an empty registry warns loudly at startup (BOU-88).

Auth / sessions

DB-backed, pure core (libs/user/…​/core/session.clj). No server-side sticky state — any replica can serve any request.

Multi-tenancy

schema-per-tenant (libs/tenant/…​/provisioning.clj). Instance-agnostic; routing is per-request.

Email / external

Async via the jobs queue / stateless IO adapters.

Event bus

Redis Streams adapter (libs/events/…​/adapters/redis_streams.clj) — at-least-once, consumer groups, so one logical subscriber gets an event once across replicas rather than once per replica. Selected with :provider :redis-streams, spelled differently from the cache’s and realtime’s :redis; an unrecognised provider throws at boot rather than falling back. The in-memory adapter (in_memory.clj) is single-process only. :wagoe/events is not in the prod profile yet; activate it there to use it (BOU-93).

Readiness checks

/health/ready (readiness-handler, wired in wiring.clj) probes DB + cache and returns 503 when any is down — correct for load balancers and k8s.

Rate limiting

The config-driven http-rate-limit-protection interceptor is wired into the default route pipeline (wiring.clj injects :rate-limit config + the :wagoe/cache into the interceptor system map). Configure under :wagoe/http :rate-limit {:enabled? :limit :window-ms} (per-env; HTTP_RATE_LIMIT_ENABLED/HTTP_RATE_LIMIT/HTTP_RATE_LIMIT_WINDOW_MS). Enforcement is opt-in (default off — the bundled prod/acc configs ship it disabled; enable it together with an active Redis cache) so an upgrade cannot start 429-ing existing consumers, nor silently run a per-process limiter in production. With an active Redis cache the fixed-window limit is shared across replicas; with no cache it falls back to a per-process counter — single-node only, effective global limit = limit × N. Enabling it without a reachable cache is handled per profile: in :prod the wiring refuses to boot (BOU-173 — a limit counted per replica is not a limit, and a false sense of protection is worse than none), in dev/test/acc it logs a warning and falls back. Note :wagoe/cache ships under :inactive in resources/conf/prod/config.edn, so enabling the limiter in production is two edits, not one. The in-process fallback is heap-bounded by a hard cap — before a new client is recorded at the cap, stale clients are swept and, if the map is still full, the least-recently-active client is evicted — so high-cardinality client ids can’t leak memory on a long-running node even when every client is in-window. BOU-87.

Graceful shutdown

Integrant halt! runs on a JVM shutdown hook (src/wagoe/main.clj), closing pools and stopping Jetty. Jetty is configured for graceful connection draining (configure-graceful-shutdown! in wiring.clj): on stop it stops accepting new connections, rejects new requests with 503, and lets in-flight requests finish within :wagoe/http :drain-timeout-ms (default 30000 ms in prod/acc, env HTTP_DRAIN_TIMEOUT_MS; 0 disables). Set the window above the load balancer’s deregistration delay for zero-downtime rollouts (BOU-86).

Realtime / WebSocket

✅ *

Replica-safe via :provider :redis (BOU-85, ADR-035). Live sockets remain node-local; routing envelopes fan out over a Redis pub/sub channel so a broadcast reaches clients on any replica. Topic subscriptions are stored in Redis sets and are cluster-wide. *The default :in-memory provider is single-node only — use :redis for multi-replica deployments.

Topologies

ShapeWhen

Single fat node

Vertical only. One process, large heap and pools. Everything works, including realtime and the in-memory adapters. Rate limiting is the one exception: the :prod profile refuses to boot with it enabled and no cache, single node or not. Simplest; capped by one machine.

N stateless web replicas

The main horizontal mode. N copies of the uberjar behind a load balancer, sharing Postgres + Redis. Cache, jobs, auth, tenancy all scale. Caveats: activate :wagoe/cache and wire Redis rate limiting; WebSocket scales horizontally via :provider :redis (sticky sessions are only required when running the :in-memory provider). Reference: deploy/compose/multi-instance.yml.

Web / worker split

Dedicated job-worker processes separate from web. java -jar wagoe.jar worker boots the full system minus the HTTP surface (:wagoe/http-server, :wagoe/http-handler, :wagoe/dashboard), so it binds no port and runs only background components. Reference: the two Deployments in deploy/k8s/wagoe.yaml.

Module as its own service

java -jar wagoe.jar service <module>…​ — see Running a module as a service. Reference: deploy/compose/per-service.yml.

The same image runs all four; the container argument selects the mode. Compose files, manifests and what has actually been brought up are in Deployment Topologies.

Production checklist

  • Use the Redis cache and jobs adapters, never :in-memory, for more than one replica.

  • Register all job handlers on every instance. A dequeued job with no local handler is re-enqueued so another instance can run it, and is dead-lettered only after :max-requeue-age-ms (default 5 min) of going continuously unhandled — so a job-type that no worker registers costs five minutes of re-enqueue churn before it fails. (:max-requeues, default 10000, is a runaway backstop, not the budget: give-up is age-based so a slow handler-owning worker cannot lose a job to attempt exhaustion.)

  • Enable rate limiting (:wagoe/http :rate-limit :enabled? true) with an active Redis cache for a global limit across replicas. In :prod this is not optional — the boot fails without a reachable cache. :wagoe/cache is :inactive in the shipped prod profile; move it to :active first.

  • Keep replicas × maximum-pool-size under Postgres max_connections.

  • Confirm your load balancer points health probes at /health/ready (503-aware), not /health/live.

  • For WebSocket: use :provider :redis on :wagoe/realtime to scale across replicas (ADR-035). Sticky sessions / single-node are only required with the default :in-memory provider.

  • Deploy Redis, and a load balancer for N replicas. The root docker-compose.yml is a single-instance dev stack (db, app, docs, dev-tools — no Redis); the replica and per-service references are deploy/compose/multi-instance.yml, deploy/compose/per-service.yml and deploy/k8s/wagoe.yaml.

Functional decomposition (slicing services out)

The third axis: run a module (or a few) as its own process, scaled and deployed independently of the rest. This is where the ports.clj seam pays off most — and where the most net-new infrastructure is needed. It is not free "by config" today, but the architecture is positioned for it.

What already enables it

AssetHow it helps

Per-module activation

Modules are gated by :enabled? / :active in config.edn. A process can boot a subset — the http-handler concats only present routes ((or routes []) in wiring.clj), so a "user-only" process is a config, not a fork.

The protocol seam

Consumers depend on the protocol (e.g. IUserService), never the concrete record. Swapping an in-process record for a remote HTTP client implementing the same protocol leaves the caller untouched.

Wire format ready

Muuntaja (JSON / EDN / Transit) is already in the HTTP stack (reitit_router.clj); Malli schema.clj per module gives ready contracts.

Remote-adapter template

libs/external (SMTP, Twilio) is a gold-standard outbound adapter: record + extend-protocol + clj-http + error envelope + logging. Copy it for a service client.

Clean data boundaries

bb check:ports already forbids one module’s shell from touching another’s shell.persistence/shell.service — the only cross-module path is the service port. No cross-module SQL joins to untangle.

An acyclic dependency graph

allowed-cycle-edges in check_deps.clj is now empty — every source-level cycle was dissolved (BOU-171/192/193/194/198). A new one fails bb check:deps. What is left between modules is one-directional coupling, which is a porting job; a cycle is not.

Context plumbing

correlation-id, tenant, and auth already flow through the interceptor pipeline and can ride request headers across a network hop.

What must be built

  • Generic remote-port adapter — shipped in BOU-90. wagoe.platform.shell.rpc.client/remote-adapter returns a value implementing any protocol by calling a service over HTTP, built from the protocol’s own :sigs so every module’s port works without a bespoke client. The counterpart …rpc.server/rpc-handler serves a protocol from the process that owns it. correlation-id, tenant and auth ride the same headers the interceptor pipeline already uses. See The remote-port adapter.

  • Network resilience — timeouts, retries and a shared circuit breaker are in the remote-port adapter (BOU-90, BOU-285). Service discovery is not: URLs are configuration, hardcoded or env-supplied. The libs/external adapters still use :throw-exceptions false with no retry/breaker of their own.

  • Break the allowlisted dependency cycles — done. allowed-cycle-edges in check_deps.clj is #{}; admin↔user and platform↔{user,tenant,admin,workflow,search} were dissolved rather than allowlisted, so no module pair is blocked from separation by a cycle.

  • Async option — shipped in BOU-93 as libs/events: IEventPublisher / IEventSubscriber / IEventHistory, with a Redis Streams adapter (at-least-once, consumer groups) and an in-memory one. A publisher does not know who is listening and is unaffected if a consumer is down — the complement to the synchronous remote-port adapter.

  • Data ownership decision — schema-per-tenant assumes co-located modules in one Postgres. Across services either share the DB (pragmatic) or give each service its own; there are no distributed transactions, so split writes become eventual-consistency.

  • Service launch mode — shipped in BOU-91: java -jar wagoe.jar service <module>…​ boots a named subset plus the platform, and starts that module’s RPC endpoint when one is configured. See Running a module as a service.

Running a module as a service

Running configuration — compose files, Kubernetes manifests, the environment variables and what has actually been brought up — is in Deployment Topologies.

service boots only the modules named and the platform they need:

java -jar wagoe.jar service payments        # one module
java -jar wagoe.jar service user tenant     # several in one process

In the test profile service user runs 20 of the application’s 34 components. /health answers, /web/login answers, /api/v1/tenants is a 404 — the tenant module is not there to serve it.

Which keys belong to which module is declared, not guessed:

;; config.edn, under :active — merged over
;; wagoe.config/default-service-catalogue, replacing an entry of the same name
;; rather than merging into it
:wagoe/services
{:payments {:keys [:wagoe/payment-provider]
            :rpc  {:protocol  'my.app.payments.ports/IPaymentProvider
                   :component :wagoe/payment-provider}}}

The keys must be the ones the config actually emits. Naming a component that does not exist does not fail — the real component is then claimed by nobody, counts as platform, and runs inside every service. wagoe.service-launch-test asserts every emitted key is either claimed by a module or listed as platform, which is the only thing that catches it.

A key no entry claims counts as platform and runs everywhere. That is the deliberate failure direction: a catalogue that has fallen behind boots a service larger than it needs to be, rather than one missing a component it depended on.

With :wagoe/rpc in config, a service also starts the remote-port endpoint on its own port, so the rest of the deployment can call it. Without it the boot says so explicitly — a module running alone that nothing can reach is a process doing no work, and it otherwise reports itself healthy.

The mechanics are in wagoe.platform.core.system-selection, and the one thing worth knowing about them: refs to unselected modules are removed before the dependency closure is taken. :wagoe/http-handler refers to every module, so following its refs first pulls the whole system back in — and the result still boots and still passes a health check, which is what makes the mistake worth naming.

The remote-port adapter

A caller depends on a protocol, never on a concrete record. So a module can move into its own process without its callers changing — provided something implements that protocol by making a network call. That is this:

;; In the process that owns payments — serve the protocol on its own listener
(jetty/run-jetty (rpc-server/rpc-app pay-ports/IPaymentProvider provider
                                    {:service-key service-key})
                 {:port 3001 :join? false})

;; In the process that consumes it — a value that satisfies IPaymentProvider
(def payments
  (rpc-client/remote-adapter pay-ports/IPaymentProvider "http://payments:3001"
                             {:timeout-ms  5000
                              :service-key service-key
                              :cache       cache            ; enables the breaker
                              :context     {:correlation-id id :tenant-id t}}))

(ports/create-checkout-session payments {...})   ; unchanged at the call site

rpc-app is a standalone Ring handler, not a route map for a module’s :api or :web slot, and that is deliberate. The router rewrites both: :api paths gain the version prefix and :web paths gain /web, so a client on the default :path gets a 404 either way. :web is worse than merely wrong — a POST there is CSRF-validated when CSRF is enabled, so the call is rejected 403 by a check meant for browser forms, for which a service-to-service caller has no token.

Underneath the mechanics: this endpoint invokes port methods, and the public listener is not where it belongs. A sliced-out service serves it on a listener reachable only from inside the deployment — which is what the service launch mode (BOU-91) starts. A service that serves it elsewhere tells the client with :path.

The adapter is built from the protocol’s :sigs, so adding a method to a port carries it across the hop with no client change. Each adapter is its own object implementing that protocol’s interface — not a shared type extended in place, which would make every adapter already built satisfy each newly adapted protocol, so a payments adapter would answer satisfies? for ICache and send cache calls to the payments URL.

What it does, and what it deliberately does not:

Context

correlation-id, tenant-id and auth-token ride x-correlation-id, x-tenant-id and Authorization. A request keeps one correlation id across the hop instead of starting a fresh trace on the far side.

Wire format

application/transit+json, negotiated by the muuntaja stack already in the pipeline — so the server side needs no special decoding. Not JSON: both ends are Clojure, and JSON has no keywords. Payments returns {:status :paid}, which over JSON arrives as {:status "paid"}; (= :paid (:status r)) would hold in-process and silently stop holding across the hop, which is the exact transparency the adapter exists to provide. Transit also preserves sets, UUIDs and instants.

Transport errors

Returned as data, in the {:error {:type … :message …}} shape adapters already use, so a caller needs no new branch. The types distinguish cases that need different responses: :rpc/unavailable, :rpc/timeout, :rpc/remote-error, :rpc/protocol, :rpc/unknown-operation. An error envelope the server built on purpose survives its own status code — a refused operation reaches the caller as :rpc/unknown-operation, not as a generic "status 500". These are not raised: they have no in-process equivalent, so no caller has a catch for them.

Thrown exceptions

Raised again on the near side, keeping the :type from the original ex-data. Payment providers throw typed ex-info and nothing catches it — the HTTP boundary reads that :type and maps it to a status (ADR-022). If a throw became a returned map across the hop, the caller’s try/catch would stop firing and the error would flow on as though it were a result. Entries in ex-data that a wire format cannot carry (a connection, a response object) are dropped rather than allowed to fail the encoding; :rpc/remote true on the re-raised exception records that the stack trace describes the calling process, not the one that failed.

HTTP status

A typed error keeps the status it has in-process, read from the same default-error-mappings the HTTP boundary uses — a :not-found answers 404 across the hop as well. The status is what proxies, dashboards and alerting read, and counting a missing record as a server error moves an error budget for something that is not an error. A request the service could not understand is 400; everything else is 500.

Malformed requests

Answered, not thrown. The endpoint is reachable by anything that can post to it, so a body carrying no operation — or an operation that is not a name, or the wrong number of arguments for one — comes back as :rpc/protocol with 400. A request the service could not understand is the caller’s error; reserving 500 for the service actually failing keeps the two apart in an operator’s logs.

Argument count is checked against the protocol’s own :sigs before the method is invoked. Letting apply raise an ArityException would be indistinguishable at that point from the implementation failing, so a client built against a different version of the protocol — the case this is most likely to arise from — would be reported as the service breaking, and re-raised as an exception on the near side.

Retries

Only failures where the call may not have completed, and only :rpc/unavailable among those by default. A timeout is not retried: a call that timed out may well have executed, and re-sending it could charge twice. Override with :retry-on.

Which failures count as never-executed is decided conservatively. A SocketException reading Network is unreachable was never sent and is retryable; one reading Connection reset happened with the request already on the wire and is not, though both are the same class. An unrecognised failure is treated as possibly-executed — being wrong that way costs a retry that would have helped, the other way costs a duplicate payment.

The HTTP client’s own retry handler is disabled. Apache HttpClient resends low-level I/O failures before this code sees an outcome, so :retries 0 and keeping :rpc/timeout out of :retry-on would not have prevented a second checkout submission — whether to resend is a decision this adapter makes, and it cannot make it if something underneath has already decided.

An answer is never retried, whatever :retry-on says — a result, or an error envelope the service built on purpose. That distinction is structural rather than read off the shape of the value, because adapters here return {:error {:type …}} as an ordinary result: a protocol method’s legitimate return can look exactly like a transport failure, and re-sending a call that already ran is a second charge.

Exposure

The server resolves an operation against the protocol’s own :sigs. An operation the protocol does not declare is refused, so the endpoint cannot become a general-purpose remote eval.

Authentication

A service key in x-rpc-service-key, required, minimum 32 characters, compared in constant time. Validated when the handler is built, so a service configured without one fails to start rather than coming up serving port methods to anything that can reach the port. :sigs bounds what may be called; this bounds who. Deliberately not the Authorization header, which carries the end user’s token through the hop — sharing them would let any caller with a valid user token invoke ports directly. {:auth :none} opts out explicitly, for a listener that is not externally reachable; it is spelled out so it can be grepped for and never happens by omission.

This is not a substitute for keeping the endpoint off a public listener. It is the part that can be enforced in code.

Service discovery

Not included — the URL is configuration. Hardcoded or env-supplied for now, as the MVP in this document assumes.

Circuit breaker

Opt-in: pass a :cache component and the breaker keeps its state there, so replicas share one rather than each discovering the outage separately — and when the window elapses a set-if-absent! lease lets exactly one of them probe, instead of all N at once. Without a :cache there is no breaker and the client behaves as it did before.

The failure count is an atomic increment, not a read-modify-write: many callers hitting one outage at the same moment is what a shared breaker is for, and it is exactly when a read-modify-write loses increments — each reads the same value and writes back the same successor, so a burst advances the counter by one and the breaker never trips.

A probe that fails reopens the window from that moment and releases its lease, so the outage is not forgotten on the original window’s schedule and the next window can still be probed.

An invalid :circuit-breaker:failure-threshold 0, or a :trip-on naming an error the client never produces — throws rather than being accepted, since an inert breaker and a working one look identical from outside.

It protects the traffic after the burst, not the burst. Calls issued at the same instant all pass the check before any of them has failed.

Trips on consecutive :rpc/unavailable and :rpc/timeout by default — failures where the call did not reach the service. A remote error does not count: the service answered, so it is up.

:trip-on overrides that, and the client counts whatever it names. Adding :rpc/protocol is a reasonable policy — a service returning bodies that are not envelopes is broken, and continuing to call it achieves nothing. Whether the call reached the far side governs retries, where re-sending something that already ran is the danger; tripping re-runs nothing, so it is not the same question. Note that :rpc/timeout trips the breaker although it is never retried; retrying a timeout risks running a non-idempotent call twice, while declining to make a new call risks nothing.

A refused call returns :rpc/circuit-open with :retry-after-ms, which is deliberately distinct from :rpc/unavailable: one means we tried and could not reach it, the other that we declined to try.

If the cache itself is unreachable the breaker fails open — the worst case is the behaviour of no breaker, which is better than a cache outage taking every remote call with it.

Sliceability by module

ModuleEffortWhy

payments

Easy

Zero internal Wagoe deps (only Maven). Already a self-contained provider. The natural pilot for the remote-adapter pattern.

core, observability

Easy

Leaf / infra; no sibling deps. (Usually shared libs, not standalone services.)

user, tenant, external

With work

Depend on platform + the in-process service assumption in middleware. The remote adapter exists and the cycles are gone; what remains is deciding data ownership and pointing consumers at a remote adapter instead of the local component — a wiring change in the application’s config, which is why deploy/compose/per-service.yml demonstrates the mechanism without making it for you.

admin, search, workflow

Entangled

search/workflow depend on admin’s schema provider — a one-directional dep, but a heavy one: slicing either means the schema provider goes over the wire or gets extracted behind its own port. `admin→user is now one-directional too, so this is coupling to untangle, no longer a cycle to break.

Recommended path: prove the remote-port adapter by extracting payments as a standalone service (zero internal deps, and the one the reference topology uses), then user. admin/search/workflow are last — not because of cycles any more, but because of the shared schema provider.

Known gaps & roadmap

The architecture delivers the promise; these are the concrete pieces that make "scale by configuration" fully true. Tracked under the BOU-84 spike:

  1. Realtime Redis pub/sub adapter — shipped in BOU-85 (ADR-035). WebSocket is now replica-safe via :provider :redis on :wagoe/realtime.

  2. Graceful connection draining — shipped in BOU-86. Configurable shutdown grace (:wagoe/http :drain-timeout-ms) lets rollouts finish in-flight requests; Jetty GracefulHandler + setStopTimeout wired in wiring.clj.

  3. Default rate-limit wiring — shipped in BOU-87. Config-driven http-rate-limit-protection is in the default pipeline; enable via :wagoe/http :rate-limit and it uses the Redis cache for a cross-replica limit (per-process fallback documented).

  4. Jobs hardening — shipped in BOU-88. Missing-handler jobs are re-enqueued (bounded) instead of silently dead-lettered, an empty-registry worker warns at startup, and scheduled-job promotion is an atomic claim (ZREM / swap-vals!) so a due job runs exactly once across workers.

  5. Deploy topology reference — done (BOU-89). Compose + k8s with N replicas, Redis, a load balancer, the web/worker split and a module-as-a-service, in Deployment Topologies. No instance-id variable: nothing in the codebase reads one, so adding it would have been decoration.

For functional decomposition (the bigger bet):

  1. Generic remote-port adapter + RPC envelope — done (BOU-90), with the circuit breaker in BOU-285. clj-http client implementing a module protocol over transit, context propagation, typed errors, retry bounded to calls that never executed.

  2. Service launch mode — done (BOU-91). wagoe.main boots a named module subset as an independent service, with its RPC endpoint on its own port.

  3. Break allowlisted dependency cycles — done (BOU-171/192/193/194/198). admin↔user and platform↔{user,tenant,admin,workflow,search} are gone; check_deps.clj allowlists no cycle edges, so the gate now fails on any cycle at all.

  4. Event-bus port + adapter — done (BOU-93). libs/events, Redis Streams and in-memory. Kafka would be another adapter behind the same ports, not a change to them.

What is left, in order:

  1. Service discovery — remote-port URLs are configuration today. Fine for a fixed deployment, insufficient once services move.

  2. Data ownership across services — schema-per-tenant assumes co-located modules in one Postgres (see the entry above).

  3. Activate :wagoe/events in the prod profile — it is configured in test only, so the event bus is currently a test-profile capability in practice.

Can you improve this documentation?Edit on GitHub

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
Move to previous article
Move to next article
Ctrl+/Jump to the search field
× close