The change feed with a cursor where the in-process one has a callback — the daemon-side state a remote caller holds a feed open against.
core/watch takes a function, and a function does not cross an EDN wire (the same
wall :export's :on-progress hits). So the wire's half of the feed is not the
callback marshalled somehow; it is the one thing a request/response protocol can
carry, which is state with a cursor: the daemon registers an ordinary listener of
its own, that listener files each event into a bounded ring, and a caller reads the
ring forward from where it left off. Three ops — register, read, drop — every one of
them EDN in and EDN out, so the guards, the client and the error taxonomy that already
exist carry it unchanged (docs/operations.md).
A cursor counts events, not handles. It starts at 0 when the subscription is
registered and advances by one per delivered event, so a caller compares nothing and
stores one integer. poll answers the events past the cursor it was handed and the
cursor to send next time.
The ring is bounded, and falling off it is said out loud. A subscriber that stops
reading must not grow the daemon's heap, so the ring keeps max-events and drops the
oldest past it — and the count of what it dropped is reported as :lagged on the
next poll. That number is the whole reason this is usable: a feed with a silent gap
is strictly worse than polling, because the caller believes it is current and is not.
:lagged is present on every reply, zero and all, so a client that forgets to read it
is a client that cannot have one.
A token that names no subscription is refused, never answered empty. The same
argument: a reaped, dropped or invented token answering {:events []} is a feed that
has silently stopped. :unknown-subscription says so.
What a subscription costs the daemon, and what bounds it. One listener on the
KB's feed and one ring of at most max-events events; max-subscriptions of those at
once, and one that nobody has polled inside idle-ms is reaped at the next call.
Nothing here authenticates the caller — that is the bearer token's job, one layer out
(vaelii.impl.serve) — but heap a stranger can allocate wants a ceiling whether or
not it is authenticated, and the reap is what keeps an abandoned subscription from
holding a slot against a live one.
The wait happens here, outside the daemon's monitor. A long poll parks on the
subscription's own signal object, so a writer serialized behind serve's one monitor
runs to completion while a poll is parked — the feature is about liveness, and a
parked poll that blocked every writer would be a global stall wearing its name. The
writing thread's only cost is the swap that files the event and a notifyAll on a
monitor no poller holds for longer than a compare.
The three entry points are spelled without ! for core/watch's reason: nothing here
destroys stored knowledge (docs/api.md). See docs/feed.md, "Across the wire".
The change feed with a **cursor** where the in-process one has a callback — the
daemon-side state a remote caller holds a feed open against.
`core/watch` takes a function, and a function does not cross an EDN wire (the same
wall `:export`'s `:on-progress` hits). So the wire's half of the feed is not the
callback marshalled somehow; it is the one thing a request/response protocol can
carry, which is **state with a cursor**: the daemon registers an ordinary listener of
its own, that listener files each event into a bounded ring, and a caller reads the
ring forward from where it left off. Three ops — register, read, drop — every one of
them EDN in and EDN out, so the guards, the client and the error taxonomy that already
exist carry it unchanged (docs/operations.md).
**A cursor counts events, not handles.** It starts at 0 when the subscription is
registered and advances by one per delivered event, so a caller compares nothing and
stores one integer. `poll` answers the events past the cursor it was handed and the
cursor to send next time.
**The ring is bounded, and falling off it is said out loud.** A subscriber that stops
reading must not grow the daemon's heap, so the ring keeps `max-events` and drops the
oldest past it — and the *count* of what it dropped is reported as `:lagged` on the
next poll. That number is the whole reason this is usable: a feed with a silent gap
is strictly worse than polling, because the caller believes it is current and is not.
`:lagged` is present on every reply, zero and all, so a client that forgets to read it
is a client that cannot have one.
**A token that names no subscription is refused, never answered empty.** The same
argument: a reaped, dropped or invented token answering `{:events []}` is a feed that
has silently stopped. `:unknown-subscription` says so.
**What a subscription costs the daemon, and what bounds it.** One listener on the
KB's feed and one ring of at most `max-events` events; `max-subscriptions` of those at
once, and one that nobody has polled inside `idle-ms` is reaped at the next call.
Nothing here authenticates the caller — that is the bearer token's job, one layer out
(`vaelii.impl.serve`) — but heap a stranger can allocate wants a ceiling whether or
not it is authenticated, and the reap is what keeps an abandoned subscription from
holding a slot against a live one.
**The wait happens here, outside the daemon's monitor.** A long poll parks on the
subscription's own signal object, so a writer serialized behind `serve`'s one monitor
runs to completion while a poll is parked — the feature is about liveness, and a
parked poll that blocked every writer would be a global stall wearing its name. The
writing thread's only cost is the swap that files the event and a `notifyAll` on a
monitor no poller holds for longer than a compare.
The three entry points are spelled without `!` for `core/watch`'s reason: nothing here
destroys stored knowledge (docs/api.md). See docs/feed.md, "Across the wire".How long a subscription survives with nobody polling it. A client that goes away
without saying so leaves a listener and a ring behind; without this they hold a slot
against a live caller until the daemon restarts. Reaped lazily, at the next watch
or poll, so nothing here needs a thread of its own.
How long a subscription survives with nobody polling it. A client that goes away without saying so leaves a listener and a ring behind; without this they hold a slot against a live caller until the daemon restarts. Reaped lazily, at the next `watch` or `poll`, so nothing here needs a thread of its own.
How many events one subscription's ring retains. Past it the oldest goes and the
drop is counted, so the depth is the slack a reader has between polls rather than a
promise it cannot fall behind. Reported by watch, since it is the number a poll
interval is chosen against.
How many events one subscription's ring retains. Past it the oldest goes and the drop is counted, so the depth is the slack a reader has between polls rather than a promise it cannot fall behind. Reported by `watch`, since it is the number a poll interval is chosen against.
How many long polls may be parked at once on one daemon.
A parked poll holds an HTTP worker thread for the length of its wait, so the ceiling
that matters here is the server's thread pool rather than anything this namespace owns:
park more polls than the pool has threads and the daemon answers nothing at all —
/health, a write, another caller's read — until one of them times out. Moving the
wait outside serve's monitor is what keeps a parked poll from blocking the writer;
it does nothing about the threads, and the two are separate ceilings.
So this one is deliberately well under vaelii.impl.serve/http-threads, and
serve_test pins that relationship rather than trusting the two numbers to stay
related. Over it, a poll asking to wait is refused (:too-many-waiters) rather
than parked: the caller is told to poll on a timer instead, which costs the daemon one
request and no held thread, and is the same feed at a worse latency. Answering it an
immediate empty page would be the cheaper lie — the caller would see a long poll that
never waits and a latency it has no way to explain.
How many long polls may be **parked at once** on one daemon. A parked poll holds an HTTP worker thread for the length of its wait, so the ceiling that matters here is the server's thread pool rather than anything this namespace owns: park more polls than the pool has threads and the daemon answers nothing at all — `/health`, a write, another caller's read — until one of them times out. Moving the wait outside `serve`'s monitor is what keeps a parked poll from blocking the *writer*; it does nothing about the *threads*, and the two are separate ceilings. So this one is deliberately well under `vaelii.impl.serve/http-threads`, and `serve_test` pins that relationship rather than trusting the two numbers to stay related. Over it, a poll asking to wait is **refused** (`:too-many-waiters`) rather than parked: the caller is told to poll on a timer instead, which costs the daemon one request and no held thread, and is the same feed at a worse latency. Answering it an immediate empty page would be the cheaper lie — the caller would see a long poll that never waits and a latency it has no way to explain.
How many live subscriptions one daemon holds at once. Reached, a further watch is
refused (:too-many-subscriptions) rather than evicting somebody else's: a
subscription silently dropped is the silent gap this whole namespace exists to
refuse, and the caller that gets the refusal is the one that can still do something
about it.
How many live subscriptions one daemon holds at once. Reached, a further `watch` is refused (`:too-many-subscriptions`) rather than evicting somebody else's: a subscription silently dropped is the silent gap this whole namespace exists to refuse, and the caller that gets the refusal is the one that can still do something about it.
The longest a long poll parks before answering, whatever :wait-ms asks for. A
parked poll holds a server thread, so the ceiling is on the daemon's threads rather
than on the caller's patience — a caller wanting to wait longer polls again, which
costs it one round trip and costs the daemon nothing it was not already paying.
The longest a long poll parks before answering, whatever `:wait-ms` asks for. A parked poll holds a server thread, so the ceiling is on the daemon's threads rather than on the caller's patience — a caller wanting to wait longer polls again, which costs it one round trip and costs the daemon nothing it was not already paying.
(poll reg kb token cursor)(poll reg kb token cursor opts)Read a subscription forward: the events past cursor, the cursor to send next time,
and the number the ring dropped before this call could see them.
(poll reg kb 3 17) => {:events [{…}] :cursor 19 :lagged 0} (poll reg kb 3 17 {:wait-ms 20000}) => the same, waiting for the first one
:wait-ms is the long poll: park until an event arrives or the wait runs out, capped
at max-wait-ms. It buys the latency a feed is for while keeping one wire format and
one content type; nothing about the reply changes, so a caller that does not want it
omits the key and polls on a timer.
Refused rather than answered empty: a token naming no live subscription
(:unknown-subscription — dropped, timed out, or from another daemon), and a cursor
that is not a whole number or that runs ahead of what the subscription has delivered
(:bad-cursor). Either of those answered {:events []} would be a feed that has
stopped without saying so.
Read a subscription forward: the events past `cursor`, the cursor to send next time,
and the number the ring dropped before this call could see them.
(poll reg kb 3 17) => {:events [{…}] :cursor 19 :lagged 0}
(poll reg kb 3 17 {:wait-ms 20000}) => the same, waiting for the first one
`:wait-ms` is the long poll: park until an event arrives or the wait runs out, capped
at `max-wait-ms`. It buys the latency a feed is for while keeping one wire format and
one content type; nothing about the reply changes, so a caller that does not want it
omits the key and polls on a timer.
Refused rather than answered empty: a `token` naming no live subscription
(`:unknown-subscription` — dropped, timed out, or from another daemon), and a `cursor`
that is not a whole number or that runs ahead of what the subscription has delivered
(`:bad-cursor`). Either of those answered `{:events []}` would be a feed that has
stopped without saying so.(registry)The subscription state one daemon holds — serve/app builds one per handler, beside
the monitor, because a token names a subscription on this daemon and means nothing
anywhere else.
:next is the token counter, monotone and never reissued, so a token from a dropped
subscription is refused rather than landing on somebody else's — feed/register!'s
reasoning one layer up.
:parked is how many long polls are holding a thread right now — a property of the
handler rather than of any one subscription, and the thing max-parked bounds.
The subscription state one daemon holds — `serve/app` builds one per handler, beside the monitor, because a token names a subscription *on this daemon* and means nothing anywhere else. `:next` is the token counter, monotone and never reissued, so a token from a dropped subscription is refused rather than landing on somebody else's — `feed/register!`'s reasoning one layer up. `:parked` is how many long polls are holding a thread right now — a property of the handler rather than of any one subscription, and the thing `max-parked` bounds.
(subscriptions reg kb)What this daemon is holding open, in token order: the goal and context each
subscription watches, how far it has been delivered, and how many events are waiting
on its ring. The listener functions are left out for core/watchers' reason — a
token is what the ops take, and a listener is not a value to compare.
:delivered is how many events the subscription has been handed and :pending how
many its ring still holds — neither is the reader's position, which lives on the
client and is a thing this daemon has no way to know. Together they are the read that
answers "is the caller keeping up": a :pending sitting at max-events is a
subscriber already dropping events.
Reaps first, like the other three, so it answers what the daemon is holding rather than what it has not got round to letting go of — a listing naming a subscription the very next call would drop is a listing nobody can act on.
What this daemon is holding open, in token order: the goal and context each subscription watches, how far it has been delivered, and how many events are waiting on its ring. The listener functions are left out for `core/watchers`' reason — a token is what the ops take, and a listener is not a value to compare. `:delivered` is how many events the subscription has been handed and `:pending` how many its ring still holds — neither is the *reader's* position, which lives on the client and is a thing this daemon has no way to know. Together they are the read that answers "is the caller keeping up": a `:pending` sitting at `max-events` is a subscriber already dropping events. Reaps first, like the other three, so it answers what the daemon is holding rather than what it has not got round to letting go of — a listing naming a subscription the very next call would drop is a listing nobody can act on.
(unwatch reg kb token)Drop the subscription token names and unregister its listener; true if there was
one. Idempotent, like core/unwatch — a token already dropped removes nothing and
says so, and never lands on the next subscription because tokens are not reissued.
A poll parked on it is woken rather than left to time out: it finds the subscription
gone and answers :unknown-subscription, which is the true thing to tell a reader
whose feed no longer exists.
Drop the subscription `token` names and unregister its listener; true if there was one. Idempotent, like `core/unwatch` — a token already dropped removes nothing and says so, and never lands on the next subscription because tokens are not reissued. A poll parked on it is woken rather than left to time out: it finds the subscription gone and answers `:unknown-subscription`, which is the true thing to tell a reader whose feed no longer exists.
(watch reg kb goal context)Register a subscription over kb and answer {:token :cursor :max-events}.
goal and context are core/watch's, and nil for the whole feed — so a goal this
engine cannot answer from a moved region is refused here by exactly the code that
refuses it in process (:not-watchable), and the two cannot drift because there is
only one check.
The entry lands in the registry before the listener is registered, so an event fired between the two has somewhere to go; a refused goal takes the entry back out again. The token is read inside the swap that allocates it, so two concurrent registrations cannot be handed one.
Register a subscription over `kb` and answer `{:token :cursor :max-events}`.
`goal` and `context` are `core/watch`'s, and nil for the whole feed — so a goal this
engine cannot answer from a moved region is refused here by exactly the code that
refuses it in process (`:not-watchable`), and the two cannot drift because there is
only one check.
The entry lands in the registry **before** the listener is registered, so an event
fired between the two has somewhere to go; a refused goal takes the entry back out
again. The token is read inside the swap that allocates it, so two concurrent
registrations cannot be handed one.cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |