{"post":{"slug":"rate-limit-algorithms-2026","title":"Rate Limit Algorithms: Picking the Right One","excerpt":"Fixed windows allow double bursts at the boundary; token buckets allow bursts by design. Here is how each algorithm behaves and which to use where.","category":"code","date":"July 24, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">R</span>ate limiting looks like one problem and is several. Protecting a fragile downstream service, enforcing a pricing tier, and stopping abuse have different requirements, and the algorithm that suits one may be wrong for another. The differences show up mainly in how each handles bursts.\n\n## What changed in 2026\n\n- **Standard response headers converged.** Consistent conventions for communicating limits and retry timing became widely adopted, which made client-side handling more reliable.\n- **Agent traffic changed the profile.** Automated clients making rapid sequential calls shifted typical traffic patterns toward bursts that simple limits handle badly.\n- **Edge-based limiting spread.** Enforcing limits at the network edge rather than in application code became standard for public APIs.\n- **Approximate distributed limiting became accepted.** Recognition that exact global counting is rarely worth its coordination cost pushed practice toward local counters with periodic reconciliation.\n\n## The algorithms\n\n| Algorithm | Burst behaviour | Memory | Complexity |\n| --- | --- | --- | --- |\n| Fixed window | Allows double the limit across a boundary | One counter per key | Lowest |\n| Sliding window log | Exact; no boundary artifact | One timestamp per request | Highest memory |\n| Sliding window counter | Approximates the log cheaply | Two counters per key | Low |\n| Token bucket | Allows a burst up to bucket size, then steady rate | Two values per key | Low |\n| Leaky bucket | No burst; constant output rate | Queue per key | Moderate |\n\nThe fixed-window boundary problem is worth understanding because it surprises people. With a limit of one hundred per minute, a client can send one hundred at the end of one minute and one hundred at the start of the next — two hundred requests in a short span, technically compliant. For abuse prevention that gap matters.\n\nToken bucket is the usual right answer for user-facing APIs. Tokens accumulate at a fixed rate up to a cap; each request consumes one. A client that has been idle can burst up to the cap, which matches how real clients behave, and sustained throughput is limited to the refill rate. It handles the agent traffic pattern — a flurry of calls then a pause — without either blocking legitimate use or allowing sustained overload.\n\nLeaky bucket is right when the downstream genuinely cannot absorb bursts. It queues and releases at a constant rate, smoothing output completely, at the cost of added latency for queued requests. Use it in front of something fragile, not in front of a normal API.\n\n## Distributed enforcement\n\nThe exact approach requires a shared counter every instance consults, which adds a network round trip to every request and makes the counter store a dependency of your entire API.\n\nThe pragmatic approach gives each instance a share of the limit locally and reconciles periodically. Enforcement is approximate — a client hitting several instances may briefly exceed the nominal limit — and it costs nothing per request. For nearly all rate limiting, that is the right trade.\n\nWhatever you choose, communicate it. Returning the limit, the remaining allowance, and a retry-after value lets clients back off intelligently rather than hammering, which is the difference between rate limiting that protects you and rate limiting that provokes a retry storm — the amplification described in [backpressure explained](/blog/backpressure-explained-2026).\n\n## Common mistakes\n\n- **Fixed window for abuse prevention.** The boundary allows double the intended rate.\n- **No retry-after in the response.** Clients retry immediately and make it worse.\n- **Exact distributed counting by default.** Expensive coordination for little benefit.\n- **One global limit for all endpoints.** An expensive endpoint and a cheap one should not share a budget.\n- **Limiting by address alone.** Shared addresses and proxies make this both unfair and evadable.\n- **No burst allowance.** Rejects legitimate clients whose traffic is naturally bursty.\n\n## FAQ\n\n**What should I rate limit by?**\nAuthenticated identity where available, since it is stable and fair. Network address as a fallback, understanding its limitations.\n\n**Should limits differ per endpoint?**\nYes, where cost differs substantially. A search endpoint and a health check should not share a budget.\n\n**What status code should I return?**\nThe standard too-many-requests code, with a retry-after header. Consistency here lets generic clients handle it correctly.\n\n**Does rate limiting replace backpressure?**\nNo. Rate limiting enforces a policy per client; backpressure protects the system when aggregate load exceeds capacity. Both are needed.\n\n## Where to go next\n\nFor overload handling, read [backpressure explained](/blog/backpressure-explained-2026). For failure isolation, [circuit breaker pattern](/blog/circuit-breaker-pattern-2026), and for API conventions, [API error design](/blog/api-error-design-2026).","hue":132,"metaTitle":"Rate Limit Algorithms Compared for 2026","metaDescription":"How fixed window, sliding window, token bucket, and leaky bucket rate limiting differ in burst behaviour, memory cost, and distributed implementation.","keywords":["rate limit algorithms","token bucket vs leaky bucket","sliding window rate limiting","fixed window boundary burst","distributed rate limiting","api throttling design","retry after header","abuse prevention","api engineering 2026"],"tldr":["**Fixed window is simplest and allows a double burst at the boundary** — twice the limit in a short span.","**Sliding window log is exact and costs memory proportional to requests** — precise, expensive at scale.","**Token bucket allows controlled bursts** — usually the best fit for user-facing APIs.","**Leaky bucket smooths output to a constant rate** — right when the downstream cannot absorb bursts.","**Skip:** distributed exact limiting unless you need it. Approximate limits with local state are far cheaper and usually sufficient."],"readTime":"3 min read","wordCount":764},"related":[{"slug":"api-rate-limiting-2026","title":"API rate limiting in 2026: algorithms, patterns, and implementation","excerpt":"Rate limiting protects your API from abuse and keeps costs predictable, but the wrong algorithm causes thundering-herd failures. Here is the 2026 guide.","category":"code","date":"March 24, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":306},{"slug":"rate-limiters-explained-2026","title":"Rate limiters explained in 2026: algorithms, tradeoffs, and implementation","excerpt":"Rate limiting protects your API from abuse and overload, but choosing the wrong algorithm causes user pain. Here is the practical 2026 guide.","category":"code","date":"March 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":306},{"slug":"how-to-rate-limit-an-api-2026","title":"How to rate limit an API in 2026","excerpt":"Rate limiting protects your API from abuse and runaway clients. This guide covers token bucket, sliding window, and Redis-backed strategies for 2026.","category":"code","date":"April 2, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":311}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"rag-evaluation-metrics-2026","title":"RAG Evaluation Metrics: Measuring Retrieval and Generation","excerpt":"A wrong RAG answer has two possible causes and one combined score cannot tell them apart. Here is how to measure each stage separately and act on the result.","category":"ai","date":"July 24, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":206},"older":{"slug":"slo-error-budget-guide-2026","title":"SLOs and Error Budgets: Making Reliability a Decision","excerpt":"An SLO turns reliability from an argument into arithmetic. Here is how to set targets that mean something and use the error budget as an actual control.","category":"code","date":"July 24, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":260}}