Workers and instances
SDK workers
An SDK worker is your own process. Give it the server's origin (JOOBQ_GATEWAY_URL in the examples) and the SDK token, register its jobs and workflows, and start it: see SDKs: install and connect. It long-polls the gateway over HTTPS and never holds the Redis credentials or the payload sealing keys. It registers itself every 10 seconds, which is its heartbeat on the Workers page. Run more processes for more throughput; each runs up to concurrency tasks at once.
On SIGTERM or SIGINT a worker stops polling, runs what it holds (also the tasks it had prefetched), sends the outcomes and deregisters, within 30 seconds by default (drain_timeout); what is still unsent then is left to its lease. A lease that is not renewed expires: the task is queued again with one retry used, and a job with none left goes to the dead letter queue as LeaseExpired. Delivery is at least once.
Crystal workers
A Crystal application runs JoobQ's queue workers (JoobQ.forge) and JoobQ Pro's workflow Worker against the same Redis. joobqui-selfhosted --worker runs the queues of the Crystal job classes the server itself defines and the schedule runner; it serves no HTTP. The first worker to claim a schedule's tick in Redis enqueues it, so more workers never double a job.
For JoobQ Pro run each --worker process with the same /data volume and environment as the web process: it reads the saved settings, licence included. The web process keeps the deployment's current Pro key in Redis and each worker reads it from there: within a minute while it has no key yet, then every 15 minutes, so a first key, renewals and revocations reach running workers without a restart. A worker without the saved licence, or with another deployment's, runs Core only; a Pro key set by hand in JOOBQ_PRO_LICENSE is used only if it belongs to the licence (see Licence and Pro key).
Every process that conducts joobq-pro workflows on the same Redis must run joobq-pro 0.4.1 or later, Crystal workers included: an older engine ignores the task queue and would send SDK steps to Crystal workers.
Several server instances
Run more joobqui-selfhosted instances behind a load balancer (JOOBQUI_SELF_HOSTED_REPLICAS in the Compose file); SDK workers can poll any of them and no sticky sessions are needed (leases, waits and delayed jobs are in Redis). Work that must run once is done by one instance at a time, which holds a lease in Redis:
| Work | Done by |
|---|---|
| Sending alert notifications | one instance (every instance evaluates the rules) |
| Purging the audit log | one instance, once a day |
| Purging dead letters and pruning finished workflows | one instance per pass, one pass at a time |
| Firing workflow schedules | one process: a workflow Worker, or a web process with JOOBQUI_WORKFLOW_SCHEDULER=true |
A holder that dies is replaced within 30 seconds; an orderly shutdown hands the lease over at once. What stays per instance: alert state (what is firing, who acknowledged it: an acknowledgement sent to another instance answers 404), the console's telemetry history and the request metrics; if that matters, send console traffic to one instance. Each instance opens up to JOOBQUI_DB_POOL_SIZE PostgreSQL connections (20 by default): keep instances x pool size under max_connections.