Skip to content

Backups and retention ​

Back up ​

  • PostgreSQL holds accounts, tokens, the audit trail and notification channels: back it up with pg_dump (or your provider's snapshots) while the application runs. After a restore, start the application: it runs pending migrations. An old dump brings back accounts, roles and tokens as they were then: re-check the admins and revoke tokens revoked since.
  • Redis holds queues, dead letters and workflow histories. JoobQUI does not configure its persistence: enable AOF (appendonly yes) or RDB snapshots, or use a managed Redis with persistence, and back up its files.
  • The workflow archive (JOOBQUI_WORKFLOW_ARCHIVE_DIR) is a directory of files on the instance that runs retention. Put it on a persistent volume and back it up: it is the only copy of a run once retention has deleted it from Redis.

Retention ​

DataSettingDefault
Audit eventsJOOBQUI_AUDIT_RETENTION_DAYS (whole days from 1)365 days; purged at start and then daily, recorded as audit.purge
Dead lettersJOOBQUI_DLQ_RETENTION_DAYS (1 to 36500; unset or 0 keeps them)kept for ever
Finished workflows (Pro)JOOBQUI_WORKFLOW_RETENTION_DAYS (1 to 36500; unset or 0 keeps them)kept for ever. Deleted runs, their history and search indexes cannot be restored; open runs are never deleted
PacingJOOBQUI_RETENTION_BATCH (1 to 5000), JOOBQUI_RETENTION_INTERVAL_MINUTES (1 to 1440)500, 60
SessionsJOOBQUI_SESSION_TTL_MINUTES (5 to 1440, from sign-in)60

Audit and dead-letter retention run without a Pro key; pruning workflows needs the engine. GET /api/v1/workflow-retention shows whether finished workflows are pruned and what the last pass removed.

The workflow archive Pro ​

With JOOBQUI_WORKFLOW_ARCHIVE_DIR set, retention writes each finished run to a gzip JSON file there before deleting it; a run that cannot be archived stays in Redis and is reported (GET /api/v1/workflow-retention, joobqui_workflow_archive_failures). The start stops if the directory cannot be created or written.

  • JOOBQUI_WORKFLOW_ARCHIVE_MAX_RUNS (default 50,000, at most 200,000) drops the oldest files beyond it; JOOBQUI_WORKFLOW_ARCHIVE_MAX_BYTES (default 16 MiB, 64 KiB to 64 MiB) is the largest run archived.
  • JOOBQUI_WORKFLOW_ARCHIVE_REDACT (default true) masks secret-looking keys and error texts and keeps no payloads in the stored state. A redacted archive cannot be replayed (409 archive_redacted). It is masking, not the removal of personal data.
  • The archive is per instance: any instance can win the retention lease, so for a complete archive run retention on one instance or mount shared storage. Once an instance has run a pass with an archive, an instance without one skips the pass rather than delete runs unarchived.

The console reads it through GET /api/v1/workflow-archive and GET /api/v1/workflow-archive/:id, and POST /api/v1/workflow-archive/:id/replay replays an unredacted run against the definition registered now.

Erasure ​

./joobqui accounts:erase EMAIL --yes (or POST /api/v1/users/:id/erase, admin) deactivates the account, ends its sessions, revokes its tokens and replaces its email and username with erased-<id>@invalid; audit rows naming the old email are rewritten. It cannot be undone and is refused for the last active admin. Backups still hold the old data: erase again after a restore.

JoobQ Core is MIT licensed. JoobQ Pro needs a licence key.