A service under more load than it can handle has two options. It can accept the work and get slower, or it can refuse some work and stay fast for the rest. Systems that pick the first option do not degrade gracefully — they collapse, because accepting work you cannot complete consumes memory, lengthens queues, and raises latency until timeouts fire everywhere at once.
Backpressure is the mechanism for picking the second option deliberately.
What changed in 2026
- Bounded queues became the default in framework guidance. Documentation across major frameworks moved from mentioning queue bounds as an option to presenting unbounded queues as a bug.
- Load shedding matured in service meshes. Adaptive concurrency limits that shed load automatically became configurable infrastructure rather than application code.
- Retry budgets spread. Limiting total retries as a fraction of traffic, rather than per-request retry counts, became recognized as the fix for retry amplification.
- Queue depth overtook CPU in dashboards. Teams shifted toward saturation signals that lead rather than lag.
Where things go wrong
| Design |
Behavior under overload |
| Unbounded in-memory queue |
Memory grows, latency climbs, process dies |
| Bounded queue, block on full |
Backpressure propagates upstream; acceptable |
| Bounded queue, reject on full |
Fast failure; caller can react |
| No queue, synchronous processing |
Natural backpressure via connection limits |
| Retries with no budget |
Amplifies load exactly when load is the problem |
| Timeouts longer than upstream's |
Work continues on requests nobody is waiting for |
The last row is subtle and common. If your service has a thirty-second timeout and its caller has a ten-second timeout, then for twenty seconds you are doing work whose result nobody will read, while holding resources that could serve live requests. Timeouts should decrease as you go deeper into the call chain, not increase.
Signalling overload
The mechanism depends on the transport. In synchronous request-response, reject with a status code indicating overload and, where possible, a retry-after hint — that gives the caller information to back off rather than retrying immediately.
In streaming protocols, flow control is usually built in: consumers advertise how much they can accept, and producers respect it. The failure mode there is a consumer that reads eagerly into an unbounded internal buffer, which defeats the protocol-level control that was working correctly.
In message queues, the queue itself is the buffer and depth is your signal. A growing queue means consumers cannot keep up, and the answer is either more consumers or shedding at the producer. Letting it grow indefinitely just moves the failure later and makes it worse.
Whatever the transport, retries need a budget. Per-request retry counts feel safe and are not — under broad overload, every client retrying three times triples the load precisely when the system is already failing. A global retry budget, capping retries as a percentage of total traffic, prevents that amplification. Pair it with the failure isolation of a circuit breaker, which stops sending to a dependency that is clearly down.
Common mistakes
- Unbounded queues anywhere. They convert overload into outage.
- Retries without a budget. Amplification at the worst moment.
- Timeouts that grow with depth. Work continues for callers who have given up.
- Monitoring CPU instead of queue depth. CPU saturates after latency has already degraded.
- Treating rejection as failure. A fast rejection under load is correct behavior, not a bug.
FAQ
Is rejecting requests bad for users?
Less bad than accepting all of them and serving everyone slowly or not at all. A fast rejection lets clients retry later, fail over, or show a degraded experience.
How do I pick a queue bound?
From your latency target and processing rate. If you serve a hundred per second and target one second, a queue much deeper than a hundred guarantees you miss the target for anything at the back.
Does backpressure apply to background jobs?
Yes. An unbounded job queue is the same failure with a slower fuse, and it eventually exhausts either memory or the broker.
What about autoscaling?
Helpful and slower than overload arrives. Backpressure protects you during the minutes scaling takes, and covers the case where scaling cannot help because a downstream dependency is the bottleneck.
Where to go next
For failure isolation, read circuit breaker pattern. For handling messages that repeatedly fail, dead letter queues explained.