There's a point where a single machine can't cope: the table is too big, the writes are too many. When you get there, you need to split the data. Partitioning and sharding are the two ways to do that — and confusing them, or picking the wrong key, is a mistake that haunts production systems for years.

Vertical vs horizontal partitioning

Splitting a table can be done in two directions:

• Vertical partitioning — separates columns. Rarely used or heavy columns (a blob, a long text) go to a separate table, leaving the main one lean and fast to read.
• Horizontal partitioning — separates rows. You split the rows into groups by some criterion, for example 2024 orders in one partition and 2025 in another.

Horizontal partitioning is the most relevant for scale, because it's the basis of sharding.

Sharding: horizontal partitioning across machines

Sharding is horizontal partitioning taken beyond a single machine: each slice (shard) lives on a different server. Instead of one giant database, you have several smaller ones, each responsible for a piece of the data. That way, storage and processing capacity grow by adding machines.

          router (picks the shard by key)
         /            |             \
   +---------+   +---------+   +---------+
   | Shard A |   | Shard B |   | Shard C |
   | ids     |   | ids     |   | ids     |
   | 0–999   |   |1000–1999|   |2000–2999|
   +---------+   +---------+   +---------+

The partition key and the hotspot problem

The most critical decision is the partition key: the column that decides which shard each row lives on. A good key spreads rows and load evenly. A bad key creates hotspots — one overloaded shard while the others sit idle.

Classic example: sharding by date makes the "current month" shard receive all the writes while the old months sleep. A key like customer id usually distributes better, as long as no customer is disproportionately large.

The cost: cross-shard queries and joins

Sharding isn't free. If the query filters by the partition key, the router goes straight to the right shard — great. But a query that has to scan all shards (a global COUNT, or a filter on another column) has to ask every machine and merge the results — slow and complex. Even worse are JOINs between tables that live on different shards: often unworkable, forcing you to denormalize or join in the application.

Rebalancing

Data grows unevenly, and sooner or later you need to move slices between shards or add machines — rebalancing. Moving data without stopping the system is hard; that's why techniques like consistent hashing exist, to minimize how much data moves when a shard joins or leaves.

Sharding is not replication

They're often confused, but they solve different problems:

In practice they're combined: each shard is replicated for fault tolerance.

Sharding swaps a scale problem for a design problem: from the moment you split data across machines, the choice of partition key becomes the decision that defines the performance of everything. Getting it wrong is very expensive to fix later.

In interviews they often ask you to pick a partition key for a scenario and explain how you'd avoid hotspots — and to tell sharding apart from replication. Day to day, this shows up when a monolithic database hits its limit and the team has to decide how to scale without locking themselves into a bad key.

Conclusion

Partitioning divides data — vertically by columns, horizontally by rows — and sharding extends the horizontal split across machines to scale volume and writes. The partition key governs load balance and query cost; replication is a different thing, aimed at availability. Understanding these trade-offs is what lets you scale a database without creating new bottlenecks.