In a partitioned datastore, the partition key does not just organize data — it determines your throughput ceiling. Each partition has limits, and a key that sends most traffic to one partition means you hit those limits while the rest of the cluster sits idle. Total provisioned capacity is irrelevant if the work is not spread.
This is the design decision that determines whether the system scales, and it must be made before the data exists.
What changed in 2026
- Adaptive capacity reduced some pain. Managed stores got better at handling temporary skew, without removing the need for a well-distributed key.
- Access-pattern-first modelling became the standard teaching. The relational habit of modelling entities and querying later remained the main source of failed partition designs.
- Write sharding patterns spread. Adding a computed prefix to distribute writes across partitions became a well-understood technique rather than folklore.
- Observability improved. Per-partition metrics became more accessible, making hot partitions diagnosable rather than mysterious throttling.
The design method
| Step |
Action |
| 1 |
List every query the application will make |
| 2 |
Note frequency and latency requirement for each |
| 3 |
Identify what each query filters on |
| 4 |
Choose a key that serves the highest-volume queries from one partition |
| 5 |
Check distribution — does any key value carry disproportionate traffic |
| 6 |
Add secondary access paths for the remaining queries |
Step one is the one relational habits skip. In a relational database you model entities and write whatever queries you need later, because the query planner and indexes absorb the mismatch. In a partitioned store, a query that does not align with the key is a scatter across every partition, and no amount of capacity fixes that.
Step five is where designs fail quietly. A key with high cardinality can still be badly distributed if traffic concentrates on a few values — a tenant identifier in a system where one tenant is enormous, a status field where most rows are in one state.
Fixing skew
Composite keys with a distributing component. Prefixing or suffixing the natural key with a computed value spreads writes across partitions. The cost is that reading requires querying each variant and merging, which is acceptable when writes dominate.
Bucketing time-series data. A timestamp as the partition key means every current write hits the same partition — all of today's traffic on one node. Combining a coarse time bucket with an entity identifier distributes it while keeping time-range queries tractable within an entity.
Separating hot entities. Where one tenant or entity genuinely dominates, giving it dedicated partitions rather than trying to distribute it evenly is sometimes the honest answer.
Secondary indexes for other access paths. Most partitioned stores support alternative keys, at the cost of storage and write amplification. Use them for lower-volume queries rather than trying to make one key serve everything.
The relationship to database-level sharding is close — the same distribution and query-alignment concerns appear at both levels, as covered in sharding strategy guide.
Common mistakes
- Modelling entities first, queries later. The relational habit that fails here.
- Timestamp as partition key. Moving hot spot on every write.
- High cardinality assumed to mean good distribution. Traffic concentration is what matters.
- One key expected to serve all queries. Use secondary access paths.
- No per-partition monitoring. Throttling looks like general slowness.
- Unbounded partition growth. Some stores limit partition size; plan for it.
FAQ
How do I know if I have a hot partition?
Per-partition throughput metrics, where the store exposes them. Symptoms are throttling despite low aggregate utilization.
Can I change the partition key later?
Only by migrating to a new table and backfilling. Treat it as effectively permanent.
What about small tables?
Below a size where partitioning matters, key choice is less consequential. Design for the scale you expect, not the scale you have.
Does adaptive capacity solve skew?
It absorbs temporary imbalance and does not fix a structurally bad key. Persistent skew still throttles.
Where to go next
For database-level splitting, read sharding strategy guide. For query patterns over large result sets, cursor pagination explained.