Postgres as a message queue (and when not to)
There is a recurring debate in our codebase reviews: should this team add SQS, Kafka, or RabbitMQ, or can they get away with Postgres? The answer, more often than people expect, is that Postgres will do — and dropping a broker from your architecture is one of the highest-leverage simplifications available to a small team.
When Postgres is the right call
Postgres works well as a queue when your throughput is moderate (low thousands of jobs per minute), your jobs are idempotent or transactional with the rest of your database writes, and you already operate Postgres. You get exactly-once processing, atomic enqueue alongside your business writes, and zero new operational surface area.
The pattern that holds up
The combination that consistently works in production is a jobs table with a small set of columns: id, queue, payload, run_at, attempts, locked_at, locked_by. Workers select with FOR UPDATE SKIP LOCKED, which avoids the contention that plagued earlier patterns. Backoff and retries live in application code.
A partial index on (queue, run_at) WHERE locked_at IS NULL keeps the hot read path fast even on a large table. A scheduled job archives or deletes completed rows so the table does not grow without bound.
When to graduate
There are signals that you have outgrown the pattern. Sustained throughput above a few thousand jobs per minute. Need for fan-out to multiple consumers. Strict ordering across partitions. Cross-region distribution. Any of these justify the operational cost of a real broker.
The mistake we see most often is adopting Kafka or RabbitMQ on day 1 "because we will need it eventually". You probably won't, and the operational complexity will tax you every week until you do.
The migration is not as scary as you think
When the day comes, migrating from Postgres-backed jobs to a real broker is straightforward if your worker code is well-abstracted. The job payload becomes a message. The dispatch interface stays the same. The database table becomes an outbox.
Start with Postgres. Graduate when the data tells you to.