Rate limiting is one of those API features teams add after their first incident, not before. A single runaway client, a retry storm, or a scraper can exhaust your database connections in seconds. The right rate-limiting architecture stops that while giving legitimate users headroom to burst — if you pick the right algorithm and store state correctly.
What changed in 2026
- Redis 8 shipped a native sliding-window rate-limit module (
RL.THROTTLE), making correct sliding-window counters trivial to deploy without Lua scripts.
- HTTP RFC 9110 companion drafts formalised
RateLimit headers — RateLimit-Policy, RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset — and major API gateways (Kong, Envoy, AWS API Gateway) now emit them by default.
- Edge rate limiting at CDN/WAF layer (Cloudflare, Fastly) became the default for public-facing APIs, pushing the first line of defence to the network edge.
- AI API providers established metered token-per-minute limits as the dominant quota unit, making token-bucket semantics the most widely understood model across engineering teams.
The four main algorithms
| Algorithm |
Burst handling |
Reset spike |
State size |
Best for |
| Fixed window counter |
Allows 2× burst at boundary |
Yes |
Tiny |
Internal metrics, coarse limits |
| Sliding window log |
Exact, no spikes |
No |
O(requests) |
High-precision auditing |
| Sliding window counter |
Near-exact, no spikes |
No |
Tiny |
Production default |
| Token bucket |
Smooth bursts up to bucket size |
No |
Tiny |
User-facing APIs |
| Leaky bucket |
Forces constant output rate |
No |
Tiny |
Egress shaping |
Token bucket is the recommended default. The bucket holds up to capacity tokens; tokens refill at rate per second; each request consumes one (or N for weighted calls). Bursts are allowed up to the capacity ceiling, then the rate is enforced.
Token bucket: reference implementation
import time
import redis
def is_allowed(r: redis.Redis, key: str, capacity: int, rate: float) -> bool:
"""
Returns True if the request is allowed.
Uses a Redis hash to store (tokens, last_refill_time).
"""
now = time.time()
pipe = r.pipeline()
pipe.hgetall(key)
result, = pipe.execute()
tokens = float(result.get(b'tokens', capacity))
last = float(result.get(b'last', now))
# Refill tokens since last check
elapsed = now - last
tokens = min(capacity, tokens + elapsed * rate)
if tokens < 1:
return False # Emit 429
tokens -= 1
r.hset(key, mapping={'tokens': tokens, 'last': now})
r.expire(key, int(capacity / rate) + 10)
return True
For production, replace the non-atomic read-modify-write with a Lua script or use Redis 8's RL.THROTTLE to avoid race conditions under concurrent load.
Sliding window counter
The sliding window counter approximates a true sliding window using two adjacent fixed windows and a weighted interpolation — near-exact precision with O(1) state.
current_count = window_count[current] +
window_count[previous] * (1 - elapsed_fraction_of_current_window)
If current_count >= limit, reject. This is what Redis's RL.THROTTLE implements natively.
Correct HTTP headers
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
RateLimit-Policy: "default";l=100;w=60
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 37
Retry-After: 37
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1748908937
{"error": "rate_limit_exceeded", "retry_after_seconds": 37}
Always emit Retry-After (seconds). Emit both the draft RFC headers and the legacy X-RateLimit-* headers until client adoption of the new standard is complete.
How to pick your limits
- Baseline your actual P99 usage per client tier before setting limits — pulling numbers from thin air leads to limits that block legitimate users.
- Set burst capacity at 2–5× the per-second rate to absorb legitimate spikes (page load fans out to 5 API calls).
- Differentiate tiers: anonymous (strictest), authenticated (moderate), paying customer (generous), service-to-service (highest or unlimited).
- Limit by IP for anonymous, by API key for authenticated — IP-only limits break shared corporate NAT gateways.
- Add a global limit per endpoint separate from per-client limits to cap total load.
Common mistakes
In-process state with multiple pods. If you run 10 app servers and track limits locally, each server sees 1/10 of actual traffic. Clients get 10× the intended limit. Use Redis (or Memcached) for shared state.
Not handling Redis failure gracefully. If Redis is unavailable, don't crash or block all requests — fail open with a circuit breaker, log the bypass, and alert.
Silently dropping requests. Always return 429, never silently discard. Clients need the signal to back off.
Counting retries against the limit without Retry-After. A client that retries immediately on 429 can generate more load than the original request. The Retry-After header is the only reliable mechanism to break that loop.
Single global counter across all endpoints. A bulk export endpoint burning the same quota as lightweight reads is unfair. Weight heavy endpoints or give them separate buckets.
What to skip
- Rolling your own distributed counter in a relational database — the write throughput required at API scale will saturate your DB. Redis is purpose-built for this.
- Fixed-window counters for end-user quotas — the burst-at-boundary problem is real and users will notice the cliff.
- Blocking at the application layer for DDoS — CDN/WAF edge rate limiting should absorb floods before packets reach your app servers.
FAQ
Should I rate limit by IP or by API key?
Both, at different layers. IP limits at the edge/WAF for anonymous traffic; API key limits at the API gateway for authenticated traffic. IP-only limits break behind NATs.
What bucket size and refill rate should I start with?
Start with a burst size of 60 tokens and refill of 10/second for a typical authenticated API endpoint — 600 requests/minute sustained with bursts up to 60 in a second. Tune from real traffic data.
How do I test rate limiting in CI?
Use a fake Redis (Fakeredis in Python, ioredis-mock in Node) and write parameterized tests that send N+1 requests and assert the (N+1)th returns 429 with correct headers.
Can I use API Gateway (AWS/GCP) instead of building my own?
Yes, and you should for the outer layer. AWS API Gateway, Google Cloud Endpoints, and Kong all provide built-in rate limiting. Roll your own only for business-logic-aware limits (e.g., per-resource quotas) the gateway can't express.
Where to go next
See Caching strategies in 2026 for reducing load so your rate limits are rarely hit, and Feature flags guide in 2026 for gradually rolling out limit changes without a full redeployment.