Understanding JoobQ
JoobQ is three things that work together:
| Part | What it is | Licence |
|---|---|---|
JoobQ Core (azutoolkit/joobq, a Crystal shard) | Redis-backed queues and jobs: retries with backoff, a dead letter queue, delayed and scheduled jobs, cron schedules, throttling, middleware, field encryption | MIT |
JoobQ Pro (azutoolkit/joobq-pro, a Crystal shard) | A durable workflow engine (event-sourced histories in Redis), workflow schedules, rate limiting, unique jobs, batches, timeouts, metrics and alerting | Needs a licence key |
| JoobQUI | The operator console and its API (/api/v1), and the SDK gateway (/api/v1/sdk) that the eight non-Crystal SDKs talk to | Self-hosted with a deployment licence |
How a job runs
A Crystal application uses the shards directly: its jobs and workflows run on JoobQ and the JoobQ Pro engine against Redis. An application in Python, TypeScript, Ruby, Go, Rust, Java, PHP or .NET uses a thin SDK that talks only to the JoobQUI server's SDK gateway over HTTP(S) with one shared token (JOOBQUI_SDK_TOKEN):
- A producer enqueues a job (
POST /api/v1/sdk/jobs) or starts a workflow (POST /api/v1/sdk/workflows). - A worker long-polls
POST /api/v1/sdk/tasks/pollfor the queues it serves. A poll waiting on an empty queue is woken through Redis pub/sub the moment a task lands, so the worker needs no inbound endpoint. - The worker runs the handler and reports the outcome; the gateway retries a failure with backoff, sends an exhausted or non-retryable one to the dead letter queue, and re-queues a task whose lease expired (at least once delivery).
The gateway keeps the job and workflow rules in one implementation, tested against a real server in every language. SDK workers never hold the Redis credentials or the sealing keys.
Workflows
A workflow is code: a blueprint of steps, parallel groups, signal waits, timers and queries. The SDK worker registers the blueprint when it starts (POST /api/v1/sdk/blueprints); with a Pro key the joobq-pro engine conducts each run: it decides every step (dependencies, retries, compensation, waits, timeouts) and sends each step to the workers of the blueprint's task queue as a task of its own. See Durable workflows.
Where to go next
- Architecture and deployment: the processes, what lives in Redis and PostgreSQL, several instances.
- Licensing: Core and Pro: what needs a key, and what happens without one.
- Feature matrix: every feature, its edition and its languages.