Rate limiting looks like one problem and is several. Protecting a fragile downstream service, enforcing a pricing tier, and stopping abuse have different requirements, and the algorithm that suits one may be wrong for another. The differences show up mainly in how each handles bursts.
What changed in 2026
- Standard response headers converged. Consistent conventions for communicating limits and retry timing became widely adopted, which made client-side handling more reliable.
- Agent traffic changed the profile. Automated clients making rapid sequential calls shifted typical traffic patterns toward bursts that simple limits handle badly.
- Edge-based limiting spread. Enforcing limits at the network edge rather than in application code became standard for public APIs.
- Approximate distributed limiting became accepted. Recognition that exact global counting is rarely worth its coordination cost pushed practice toward local counters with periodic reconciliation.
The algorithms
| Algorithm |
Burst behaviour |
Memory |
Complexity |
| Fixed window |
Allows double the limit across a boundary |
One counter per key |
Lowest |
| Sliding window log |
Exact; no boundary artifact |
One timestamp per request |
Highest memory |
| Sliding window counter |
Approximates the log cheaply |
Two counters per key |
Low |
| Token bucket |
Allows a burst up to bucket size, then steady rate |
Two values per key |
Low |
| Leaky bucket |
No burst; constant output rate |
Queue per key |
Moderate |
The fixed-window boundary problem is worth understanding because it surprises people. With a limit of one hundred per minute, a client can send one hundred at the end of one minute and one hundred at the start of the next — two hundred requests in a short span, technically compliant. For abuse prevention that gap matters.
Token bucket is the usual right answer for user-facing APIs. Tokens accumulate at a fixed rate up to a cap; each request consumes one. A client that has been idle can burst up to the cap, which matches how real clients behave, and sustained throughput is limited to the refill rate. It handles the agent traffic pattern — a flurry of calls then a pause — without either blocking legitimate use or allowing sustained overload.
Leaky bucket is right when the downstream genuinely cannot absorb bursts. It queues and releases at a constant rate, smoothing output completely, at the cost of added latency for queued requests. Use it in front of something fragile, not in front of a normal API.
Distributed enforcement
The exact approach requires a shared counter every instance consults, which adds a network round trip to every request and makes the counter store a dependency of your entire API.
The pragmatic approach gives each instance a share of the limit locally and reconciles periodically. Enforcement is approximate — a client hitting several instances may briefly exceed the nominal limit — and it costs nothing per request. For nearly all rate limiting, that is the right trade.
Whatever you choose, communicate it. Returning the limit, the remaining allowance, and a retry-after value lets clients back off intelligently rather than hammering, which is the difference between rate limiting that protects you and rate limiting that provokes a retry storm — the amplification described in backpressure explained.
Common mistakes
- Fixed window for abuse prevention. The boundary allows double the intended rate.
- No retry-after in the response. Clients retry immediately and make it worse.
- Exact distributed counting by default. Expensive coordination for little benefit.
- One global limit for all endpoints. An expensive endpoint and a cheap one should not share a budget.
- Limiting by address alone. Shared addresses and proxies make this both unfair and evadable.
- No burst allowance. Rejects legitimate clients whose traffic is naturally bursty.
FAQ
What should I rate limit by?
Authenticated identity where available, since it is stable and fair. Network address as a fallback, understanding its limitations.
Should limits differ per endpoint?
Yes, where cost differs substantially. A search endpoint and a health check should not share a budget.
What status code should I return?
The standard too-many-requests code, with a retry-after header. Consistency here lets generic clients handle it correctly.
Does rate limiting replace backpressure?
No. Rate limiting enforces a policy per client; backpressure protects the system when aggregate load exceeds capacity. Both are needed.
Where to go next
For overload handling, read backpressure explained. For failure isolation, circuit breaker pattern, and for API conventions, API error design.