At some point almost every backend hits the same wall: a request triggers slow work — sending email, resizing an image, charging a card — and making the user wait for it feels wrong. So what is a message queue? It is a buffer that lets one piece of code hand a task to another and immediately move on, while a separate worker picks up that task and does it later.
What changed in 2026
Message queues are not new, but the defaults have shifted:
- Managed is now the norm. AWS SQS, Google Pub/Sub, and Azure Service Bus have made self-hosting a broker a deliberate choice rather than the obvious one, removing the burden of keeping a cluster alive.
- Kafka and streaming blurred together. Kafka, Redpanda, and cloud log services are increasingly pitched for both event streaming and plain queuing, which makes picking the right tool harder, not easier.
- Postgres-as-a-queue got respectable. Libraries built on
SELECT ... FOR UPDATE SKIP LOCKED matured enough that many teams skip a dedicated broker for low-to-moderate volume.
- Exactly-once claims are everywhere. Treat them with skepticism, and verify current pricing and throughput limits yourself — vendor numbers drift constantly.
How a message queue actually works
Three roles do all the work. A producer writes a message. The queue (or broker) stores it durably. A consumer reads the message, processes it, and then acknowledges it so the broker can delete it.
The important detail is the acknowledgement. A message is not gone when it is read — it is gone when the consumer confirms success. If the worker crashes mid-task, the message becomes visible again and another worker retries it. That is why queues are reliable, and why you must plan for duplicates.
Queue vs stream vs pub/sub
These three get lumped together but are not the same thing.
| Pattern |
Who reads a message |
Typical use |
Watch out for |
| Work queue |
One consumer per message |
Background jobs |
Ordering is best-effort |
| Pub/sub |
Every subscriber gets a copy |
Fan-out, notifications |
Per-subscriber retry needs config |
| Log/stream |
Many readers track a position |
Analytics, event sourcing |
Heavier ops; you manage offsets |
A work queue is what most people mean by "message queue": one task, one worker. Pub/sub broadcasts an event to many subscribers. A log/stream (Kafka-style) keeps messages so consumers can replay history. Reach for the simplest one that fits.
The delivery guarantees that trip people up
There are three delivery models, and only two are real.
- At-most-once: fire and forget. Fast, but messages can vanish. Fine for disposable metrics, dangerous for payments.
- At-least-once: the default for most brokers. Messages are never lost but can arrive twice. This is what you want.
- Exactly-once: the marketing favorite. True end-to-end exactly-once is extremely hard across a network; what vendors deliver is at-least-once plus deduplication inside their own boundary.
The honest answer: pick at-least-once and make your consumers idempotent — processing the same message twice produces the same result. Use a unique message ID and a "have I already handled this?" check. That habit removes most correctness bugs.
When you need one, and when to skip it
A message queue earns its keep when you want to:
- Return a response fast and do slow work afterward (email, PDFs, webhooks).
- Absorb traffic spikes so a burst does not knock over a downstream service.
- Decouple services so a failing consumer does not break the producer.
- Retry flaky operations without hand-rolling retry loops everywhere.
Skip it when a queue adds more complexity than it removes. If a call finishes in milliseconds and never fails, wrapping it in async messaging just adds moving parts, harder debugging, and eventual-consistency surprises. And do not start with Kafka for a job a single managed queue — or a Postgres table with SKIP LOCKED — handles fine.
Common mistakes and caveats
No dead-letter queue. Without one, a poison message that always fails will retry forever and clog your workers. Route repeated failures aside for inspection.
Assuming order. Most queues do not guarantee strict ordering across many consumers. If order matters, you need partitioning or a single-consumer setup — and you pay for it in throughput.
Ignoring backpressure. If producers outrun consumers indefinitely, the queue grows without bound. Monitor queue depth and consumer lag as first-class alerts.
Forgetting idempotency. It is the mistake that hurts most in production, because duplicates will happen.
FAQ
Is a message queue the same as Kafka?
No. Kafka is a distributed log that can act like a queue but is built for streaming and replay. A plain queue like SQS or RabbitMQ is simpler and often the better default.
Do I need a message queue for a small app?
Usually not at first. A background job library backed by your existing database handles low volume well. Add a broker when scale or decoupling demands it.
What happens if a consumer crashes mid-message?
The message is not acknowledged, so after a visibility timeout the broker makes it available again and another worker retries it. Design for that retry.
RabbitMQ or SQS in 2026?
SQS if you are on AWS and want zero ops. RabbitMQ if you need rich routing or run your own infrastructure. Pick for operational fit, not feature checklists.
Where to go next
Wiring a queue into a service? Pair it with API rate limiting in 2026 to keep producers from overwhelming downstream systems, and lean on the best AI coding assistants of 2026 to scaffold consumer and retry logic. Choosing a stack for the app around it? Start with Astro vs Next.js in 2026.