{"post":{"slug":"thread-pools-explained-2026","title":"Thread Pools: Sizing the Thing Everyone Gets Wrong","excerpt":"A thread pool sized to expected concurrency is nearly always wrong. The right size depends on whether the work computes or waits, and those want opposite answers.","category":"code","date":"February 10, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> thread pool exists so you do not create a thread per task. It has a size, and choosing that size is one of the more consistently mishandled configuration decisions in server software.\n\nThe common approach is to size it to expected concurrency: a thousand concurrent requests, so a large pool. That is the wrong basis, and the right one depends on what the threads actually do.\n\n## What changed in 2026\n\n- **Lightweight runtime threads changed the calculus.** Where available, they removed much of the pooling motivation for I/O-bound work.\n- **Separate pools per workload type became standard advice.** Mixing CPU and I/O work in one pool got recognised as a common cause of unexplained latency.\n- **Bounded queues with explicit rejection became the default.** Unbounded queuing was widely recognised as converting overload into memory exhaustion.\n- **Observability improved.** Queue depth and rejection metrics became standard rather than something you added.\n\n## Two opposite answers\n\n**CPU-bound work** — computation with no waiting. The threads compete for cores. Beyond roughly the core count, additional threads add context switching without adding throughput; each thread gets a smaller slice and the total work done stays the same or falls.\n\nA pool near the number of available cores is correct here. More is actively worse.\n\n**I/O-bound work** — threads spend most of their time waiting on network or disk. A waiting thread uses no CPU, so a pool can be much larger than the core count and still be useful. The constraint becomes memory per thread and whatever the downstream resource can handle.\n\n| Workload | Pool size basis |\n| --- | --- |\n| Pure CPU | Around the core count |\n| Mostly waiting | Much larger; bounded by the downstream limit |\n| Mixed | Separate pools |\n| Database calls | Bounded by the connection pool, not the core count |\n\nThat last row is important. A thread pool larger than your database connection pool means threads queue for connections rather than doing work — you have moved the bottleneck without relieving it, per [connection limits](/blog/connection-limits-explained-2026).\n\n## Separate the workloads\n\nMixing CPU-heavy and I/O-heavy tasks in one pool produces the worst of both.\n\nSize it for CPU work and I/O tasks queue behind computation, wasting time that could have been spent waiting concurrently. Size it for I/O work and CPU tasks oversubscribe the cores, making everything slower.\n\nSeparate pools let each be sized correctly, and they provide isolation: a burst of expensive computation cannot starve the pool serving requests.\n\nThe same reasoning applies to isolating by importance — a pool for user-facing requests and another for background work means background load cannot consume the capacity users need.\n\n## Bound the queue\n\nA thread pool has a queue for tasks waiting for a thread. Whether that queue is bounded is a more consequential decision than the pool size.\n\nAn **unbounded queue** never rejects. Under sustained overload it grows until memory is exhausted, and long before that, tasks sit in the queue so long that their results are useless — the client timed out ten seconds ago.\n\nA **bounded queue** rejects when full. That is unpleasant and it is honest: the system is telling you it cannot accept more work, promptly, rather than accepting it and failing later.\n\nRejection needs a policy. Fail the request with a clear error is usually right, since the caller can retry or degrade. Executing the task on the calling thread is another option that applies natural backpressure to whoever is submitting.\n\nThe metrics to watch are queue depth and rejection rate. Consistently deep queues mean the pool is undersized or the downstream is the constraint; rising rejections mean genuine overload — see [backpressure explained](/blog/backpressure-explained-2026).\n\n## Common mistakes\n\n- **Sizing to expected concurrency.** Wrong basis entirely.\n- **Large pools for CPU-bound work.** Context switching without throughput.\n- **One pool for everything.** Wrong size for both workloads.\n- **Unbounded queues.** Overload becomes memory exhaustion.\n- **A thread pool larger than the connection pool.** Moves the bottleneck.\n- **No queue depth or rejection metrics.** Invisible saturation.\n- **Ignoring per-thread memory.** Large pools consume real memory.\n\n## FAQ\n\n**How do I size an I/O-bound pool?**\nStart from what the downstream resource supports — connection pool size, external API rate limit — and size to that rather than to a formula. The thread pool should not be able to overwhelm what it calls.\n\n**Should I use one global pool?**\nRarely. Separate pools by workload type and by importance give better sizing and isolation, and the overhead of having several is negligible.\n\n**What about lightweight threads?**\nWhere a runtime provides them, pooling for I/O-bound work becomes largely unnecessary — you spawn per task. CPU-bound work still benefits from bounding concurrency to core count.\n\n**How do I choose a queue size?**\nSmall enough that a queued task still has a chance of being useful when it runs. Longer than a client's timeout is pointless — the result will be discarded.\n\n## Where to go next\n\nFor the concurrency model choice, read [async vs threads](/blog/async-vs-threads-2026). For shedding load rather than queueing it, [backpressure explained](/blog/backpressure-explained-2026), and for the downstream constraint that usually caps pool size, [connection limits](/blog/connection-limits-explained-2026).","hue":316,"metaTitle":"Thread Pools: Sizing the Thing Everyone Gets Wrong","metaDescription":"Why CPU-bound and I/O-bound thread pools need opposite sizing, the danger of unbounded queues, and why mixed workloads need separate pools.","keywords":["thread pool sizing","cpu bound threads","io bound concurrency","bounded queue","rejection policy","executor configuration","concurrency tuning","performance 2026"],"tldr":["**CPU-bound pools should be near the core count** — more threads means more switching, not more work.","**I/O-bound pools can be much larger** — threads are waiting, not competing for CPU.","**Mixing both in one pool produces the worst of each** — separate them.","**An unbounded queue converts a capacity problem into a memory problem** — bound it and reject.","**Skip:** sizing to expected concurrency. Size to what the resource can actually do."],"readTime":"4 min read","wordCount":865},"related":[{"slug":"what-is-a-thread-2026","title":"What is a thread in 2026? Concurrency explained simply","excerpt":"A thread is the smallest unit of work a CPU can run, letting one program do several things at once. This guide explains threads, how they differ from processes, and their trade-offs in 2026.","category":"code","date":"May 25, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":28},{"slug":"connection-timeout-tuning-2026","title":"Connection Timeout Tuning: The Settings Nobody Sets","excerpt":"Default timeouts are usually far too long or absent entirely, which turns a slow dependency into an outage. Here is how to set them deliberately.","category":"code","date":"July 26, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":205},{"slug":"connection-pooling-explained-2026","title":"Connection Pooling Explained in 2026","excerpt":"Opening a database connection is not free. Connection pooling reuses open connections instead of paying that cost every request - here is how pools work and how to size one.","category":"code","date":"July 1, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":320}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"tesla-model-y-juniper-review-2026","title":"Tesla Model Y Juniper review in 2026: a year with the refresh","excerpt":"A year after the Juniper refresh, the Model Y is meaningfully better in the ways that mattered. Here is the honest review after 12,000 miles.","category":"tech","date":"February 10, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":235},"older":{"slug":"v0-vs-bolt-vs-lovable-2026","title":"v0 vs Bolt vs Lovable in 2026: which AI app generator builds what?","excerpt":"Three \"prompt-to-app\" tools dominate in 2026. Each builds a different shape of app well. Here is the head-to-head with real outputs and which to pick by use case.","category":"ai","date":"February 10, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":198}}