Database-backed job queue (IJobQueue) on next.jdbc.
Lets an app run durable background jobs on its existing SQL database (H2 or PostgreSQL) without standing up Redis, and survive a Redis outage. Implements the same reliable-dequeue contract as the Redis adapter:
dequeue-job! atomically claims the next ready row (sets status=processing,
locked_by/locked_at) so a worker that crashes mid-job does not lose it.ack-job! removes the row once the worker is done with it.reclaim-abandoned-jobs! returns rows whose lease has expired
(locked_at older than lease-ms) back to ready, i.e. at-least-once.Portable SQL only (no SELECT ... FOR UPDATE SKIP LOCKED): the claim is a
SELECT of the best candidate followed by a conditional UPDATE ... WHERE status='ready'. Concurrent workers that pick the same row have their UPDATEs
serialized by the row lock — exactly one wins (update count 1); the losers
retry the next candidate. This runs identically on H2 and PostgreSQL, so the
reliability tests run on the default in-memory H2 test database.
The authoritative job value is the JSON payload column (same wire format as
the Redis adapter — keyword fields restored, instants as epoch-millis). The
indexed columns (queue, priority_rank, status, execute_at, locked_at) exist
only for ordering, claiming, and reclaim.
Caveats:
lease-ms is reclaimed and may run again concurrently. Set :lease-ms
comfortably above your longest job, and/or keep handlers idempotent.
(Lease renewal is a possible future addition.)payload is VARCHAR(1000000); keep serialized jobs under ~1 MB.Database-backed job queue (IJobQueue) on next.jdbc. Lets an app run durable background jobs on its existing SQL database (H2 or PostgreSQL) without standing up Redis, and survive a Redis outage. Implements the same reliable-dequeue contract as the Redis adapter: - `dequeue-job!` atomically claims the next ready row (sets status=processing, locked_by/locked_at) so a worker that crashes mid-job does not lose it. - `ack-job!` removes the row once the worker is done with it. - `reclaim-abandoned-jobs!` returns rows whose lease has expired (locked_at older than `lease-ms`) back to `ready`, i.e. at-least-once. Portable SQL only (no `SELECT ... FOR UPDATE SKIP LOCKED`): the claim is a SELECT of the best candidate followed by a conditional `UPDATE ... WHERE status='ready'`. Concurrent workers that pick the same row have their UPDATEs serialized by the row lock — exactly one wins (update count 1); the losers retry the next candidate. This runs identically on H2 and PostgreSQL, so the reliability tests run on the default in-memory H2 test database. The authoritative job value is the JSON `payload` column (same wire format as the Redis adapter — keyword fields restored, instants as epoch-millis). The indexed columns (queue, priority_rank, status, execute_at, locked_at) exist only for ordering, claiming, and reclaim. Caveats: - The lease is fixed, not renewed (no heartbeat): a job still running after `lease-ms` is reclaimed and may run again concurrently. Set `:lease-ms` comfortably above your longest job, and/or keep handlers idempotent. (Lease renewal is a possible future addition.) - `payload` is `VARCHAR(1000000)`; keep serialized jobs under ~1 MB.
(create-db-job-queue ds & {:keys [lease-ms] :or {lease-ms default-lease-ms}})Create a DB-backed job queue over the given next.jdbc datasource ds.
:lease-ms (default 60s) is the in-flight lease after which a claimed job is
reclaimable. Call create-jobs-table! once before use (or run the migration).
Create a DB-backed job queue over the given `next.jdbc` datasource `ds`. `:lease-ms` (default 60s) is the in-flight lease after which a claimed job is reclaimable. Call `create-jobs-table!` once before use (or run the migration).
(create-jobs-table! ds)Create the job_queue table + ready-scan index if absent. Portable DDL
(H2 + PostgreSQL). Call once at startup, or ship as a migration.
Create the `job_queue` table + ready-scan index if absent. Portable DDL (H2 + PostgreSQL). Call once at startup, or ship as a migration.
How long a claimed job may stay in-flight before reclaim-abandoned-jobs!
considers its worker dead and returns it to the ready queue.
How long a claimed job may stay in-flight before `reclaim-abandoned-jobs!` considers its worker dead and returns it to the ready queue.
(enqueue-in-tx! tx queue-name job)DEPRECATED — use wagoe.jobs.ports/enqueue-in-tx! on the queue component.
Kept so existing callers keep working, but reaching into this adapter
namespace couples application code to one implementation. The capability now
lives on ports/ITransactionalJobQueue, so callers can stay on the port and
ask ports/transactional-queue? instead of knowing which adapter they have.
DEPRECATED — use `wagoe.jobs.ports/enqueue-in-tx!` on the queue component. Kept so existing callers keep working, but reaching into this adapter namespace couples application code to one implementation. The capability now lives on `ports/ITransactionalJobQueue`, so callers can stay on the port and ask `ports/transactional-queue?` instead of knowing which adapter they have.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |