Sharding key
The column(s) used to split a table across multiple database nodes. Choosing it determines how evenly data and load distribute and which queries can be served by a single shard. The hardest decision in horizontal scaling.
- When one node can't handle writes/storage.
- Multi-tenant data naturally keyed by tenant.
- When vertical scaling suffices.
- When most queries are cross-shard (pick a different model).
Definition
A sharding (partition) key hashes rows to a shard. Rows with the same key live together, enabling locality for key-scoped queries. Cross-shard queries and joins are expensive or impossible, so the key must match your access patterns.
Why it exists
One node can't hold or write all data. Sharding distributes it; the key decides the distribution. A bad key causes hot shards or cross-shard fan-out, defeating the purpose.
How it works
Hash the key for even distribution, or range-partition for ordered scans (risk of hot spots). Co-locate data accessed together by choosing a key like tenant_id or user_id. Resharding is painful, so pick a key with enough cardinality and even distribution.
Scaling limits
Queries not on the shard key fan out to all shards. Hot keys (a celebrity user) overload one shard. Resharding/rebalancing is operationally expensive and often requires downtime or careful dual-writes.
Cost shape
More shards = more nodes = more cost, but better per-shard performance. Blow-ups: over-sharding (too many small shards) and resharding events.
Alternatives
- Read replicas — Scale reads without sharding, but writes still on one node.
- Citus/Vitess — Managed sharding that hides some complexity.
Failure modes
- Hot shard from a skewed key.
- Cross-shard joins and transactions.
- Resharding downtime.