Why this matters
- Partition key choice is effectively permanent — resharding is expensive and risky.
- Cross-shard operations need sagas, denormalization, or careful API design — avoid when possible.
- Hot partitions break horizontal scale — even perfect hash partitioning fails on skewed keys.
- Interviewers ask you to pick a partition strategy for a payment ledger and defend trade-offs.
Partition key design
ShardPay uses hash(accountId) mod 256 logical shards, mapped to physical nodes via consistent hashing. Range partitioning was rejected early: celebrity merchant accounts and platform settlement accounts would saturate range boundaries. Composite keys tenantId + accountId isolate SaaS merchants so one tenant's growth does not reshuffle others.
The partition key determines colocation — transfers between two accounts hit one shard only when both accounts hash to the same partition.
Strategies
- Hash partitioning —
shard = hash(key) % Nor consistent hash ring. Even spread for random keys; range scans across shards expensive. ShardPay hashesaccountIdfor even merchant distribution across 256 logical shards. - Range partitioning — Contiguous key ranges per shard (e.g., A–M, N–Z). Efficient range queries and time-series scans. ShardPay uses range partitioning only for immutable audit archives partitioned by
settledDatemonth. - Composite / compound key — Multi-column partition key for tenant isolation. ShardPay's enterprise tier uses
(tenantId, accountId)so a large tenant can be migrated to dedicated physical shards without rehashing the global keyspace. - Directory-based partitioning — Lookup table maps key → shard. Flexible migration at cost of directory availability. ShardPay's control plane maintains a shard directory in Raft — hot tenants get directory overrides pointing to dedicated hardware.
- Cross-shard query — Scatter-gather across all shards or avoided by denormalization. ShardPay's
GET /merchants/{id}/total-volumereads a pre-aggregated counter updated by CDC — no live cross-shard sum.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 2 sections
- Colocation for single-shard transfers
- Avoiding cross-shard pain
Test yourself
Answer these before moving on — recall is what makes it stick.