Graphile Worker

Graphile Worker is a high-performance job queue built specifically for PostgreSQL and Node.js applications. It enables developers to process background jobs with exceptional speed—up to 10,000 jobs per second—while maintaining low latency through PostgreSQL's LISTEN/NOTIFY mechanism, typically starting jobs within milliseconds. The tool leverages PostgreSQL's reliable transactions to ensure jobs are never lost, making it ideal for production workloads that require dependable background processing.

Key Features:

• Automatic Retries with Exponential Backoff - Jobs that fail are automatically retried with intelligent backoff timing, built on PostgreSQL's reliable transaction system to ensure no job is ever lost.

• High Performance Processing - Capable of handling up to 10,000 jobs per second, allowing teams to keep infrastructure simple with just PostgreSQL and Node.js.

• Low Latency Execution - Jobs start in milliseconds thanks to PostgreSQL's LISTEN/NOTIFY feature, enabling near real-time background processing for customer-facing applications.

• Cron Job Scheduling - Set up repeating tasks with minute-by-minute granularity, with optional backfill capability to cover times when servers weren't running.

• Job Control Features - Schedule jobs to run in the future, debounce jobs to avoid redundant work, and cancel or update scheduled jobs as needed.

• Task De-duplication - Prevent duplicate job execution with built-in de-duplication mechanisms.

• Standalone and Embedded Modes - Run Worker as a standalone process or embed it directly within your Node.js application.

• Easy Migration Path - Examples included for exporting jobs to other queues if you decide to migrate to a dedicated job queue solution later.

To get started, install Graphile Worker via npm and configure it to connect to your PostgreSQL database. Define task handlers as JavaScript functions, then schedule jobs using the provided API. The CLI quickstart guide provides step-by-step instructions for setting up your first job queue. Worker Pro, available to sponsors, adds advanced features like crashed worker recovery and live migration to new Worker versions.