Rate limiting is the mechanism by which a service caps how many requests a client can make within a given time period. Without it, a single misbehaving client — or a traffic spike — can exhaust your server resources, starve other users, and turn a minor issue into a full outage. Every public API and most internal services in 2026 implement some form of rate limiting. The question is which algorithm to use and where to enforce it.
What changed in 2026
- Edge-native rate limiting is the default. Cloudflare Workers, AWS WAF, and Vercel Edge Middleware all support rate limiting at the edge, before requests hit your origin — reducing load by catching abusers early.
- Redis 8.x with modules provides native rate-limiting primitives (
RATELIMIT in Redis Stack) that avoid manual Lua script management.
- AI API providers publish sophisticated rate limits. OpenAI, Anthropic, and Google all enforce token-per-minute (TPM) and request-per-minute (RPM) limits independently, requiring clients to handle both dimensions.
- Rate limiting as observability. Modern APM tools (Datadog, Grafana) visualize rate limit hit rates alongside latency, making tuning data-driven.
The main algorithms
Fixed window
Divide time into fixed buckets (e.g., 0–60s, 60–120s). Count requests per bucket; reject if over the limit.
Problem: the boundary attack — a client can make N requests at 11:59 and N more at 12:00, effectively doubling the rate at the window edge.
Sliding window log
Store a timestamp for each request. Count timestamps within the last N seconds. Accurate but memory-proportional to request volume per client.
Sliding window counter
Approximate sliding window using two fixed buckets and a weighted calculation — memory-efficient and boundary-attack resistant. Used by Cloudflare.
Token bucket
Each client has a bucket that fills at a constant rate up to a maximum capacity. Each request consumes one token; requests are rejected when the bucket is empty.
# Token bucket with Redis
import redis, time
def allow_request(client_id: str, rate: float, capacity: int) -> bool:
r = redis.Redis()
key = f"ratelimit:{client_id}"
now = time.time()
pipe = r.pipeline()
pipe.get(key)
tokens, last_refill = pipe.execute()[0] or (capacity, now)
# Refill tokens based on elapsed time
elapsed = now - float(last_refill)
tokens = min(capacity, float(tokens) + elapsed * rate)
if tokens >= 1:
pipe.set(key, f"{tokens-1}:{now}", ex=int(capacity/rate) + 1)
pipe.execute()
return True
return False
Leaky bucket
Requests enter a queue (the bucket) and are processed at a fixed output rate. Excess requests overflow and are rejected. Smooths traffic but adds queue latency.
Algorithm comparison
| Algorithm |
Burst handling |
Memory |
Boundary attack |
Complexity |
| Fixed window |
Allows at boundary |
Low |
Vulnerable |
Simple |
| Sliding window log |
None |
High |
None |
Moderate |
| Sliding window counter |
Approximate |
Low |
Approximate |
Moderate |
| Token bucket |
Allows bursts |
Low |
None |
Moderate |
| Leaky bucket |
Queues bursts |
Medium |
None |
Moderate |
Token bucket is the right default for most APIs — it handles bursts gracefully while enforcing average rates.
Redis implementation pattern
-- Atomic Lua script for token bucket in Redis
-- KEYS[1] = rate limit key
-- ARGV[1] = rate (tokens/sec), ARGV[2] = capacity, ARGV[3] = now (unix float)
local key = KEYS[1]
local rate = tonumber(ARGV[1])
local capacity = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local bucket = redis.call("HMGET", key, "tokens", "ts")
local tokens = tonumber(bucket[1]) or capacity
local ts = tonumber(bucket[2]) or now
local elapsed = now - ts
tokens = math.min(capacity, tokens + elapsed * rate)
if tokens >= 1 then
redis.call("HMSET", key, "tokens", tokens - 1, "ts", now)
redis.call("EXPIRE", key, math.ceil(capacity / rate) + 1)
return 1
else
return 0
end
How to pick
- Simple internal service? Fixed window with Redis
INCR/EXPIRE is fast and easy.
- Public API with bursts expected? Token bucket — allows short spikes, enforces long-term rate.
- Billing-grade precision required? Sliding window log — exact, but budget the memory cost.
- Multi-server deployment? Redis-backed distributed rate limiter, or use edge-layer limiting (Cloudflare, AWS WAF).
- LLM API client? Track both request count and token count separately; LLM providers rate-limit on both dimensions.
Common mistakes
In-process rate limiters on multi-instance deployments. If you have 4 API servers and each allows 100 req/min, clients get 400 req/min. Rate limiters must be backed by a shared store.
Not returning Retry-After headers. Clients that get a 429 without a Retry-After header tend to immediately retry, worsening the load.
HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1748736000
Retry-After: 45
Rate limiting by IP alone. Shared NATs and corporate proxies can make thousands of users appear as one IP. Prefer per-authenticated-user limits, falling back to IP.
No rate limit bypass for health checks. /health and /metrics endpoints should be exempt from per-client rate limits or they trip during monitoring spikes.
What to skip
- Custom rate limiting frameworks when your infrastructure already provides it — Nginx (
limit_req_zone), Kong gateway, or Cloudflare Workers KV handle it without app code.
- Synchronous distributed rate limit checks on every request in the critical path — add async background counters or use approximate algorithms to avoid adding latency.
- Global rate limits without per-user limits — a global limit doesn't prevent one user from consuming all capacity.
FAQ
What status code should a rate-limited response return?
429 Too Many Requests per RFC 6585. Include Retry-After (seconds to wait) and X-RateLimit-* headers to help clients back off gracefully.
How do I rate limit WebSocket connections?
Rate limit at connection establishment (HTTP Upgrade), and apply message-level rate limiting inside the connection handler. See WebSockets deep dive in 2026.
Can I rate limit GraphQL queries differently?
Yes — normalize query complexity (count fields, depth) and rate limit on complexity units rather than request count. Libraries like graphql-query-complexity provide scoring.
How does DDoS protection differ from rate limiting?
Rate limiting is per-client or per-user enforcement of fair-use quotas. DDoS protection operates at the network layer (volumetric, protocol) and typically runs at the edge before requests reach your rate limiter.
Where to go next