If your data lives in one place today, there are three ways to spread it out. They cost very different amounts, and they are not equally fast.
Most systems start with their data in one place: one database, one region. That works until your users are a long way from it, and every request pays for the trip.
The fix is to put copies of the data closer to people, and there are three ways to do it. You can keep a full copy everywhere. You can shard, so each node holds one slice. Or you can shard and let each node keep hot replicas, local copies of the records people read most. This post walks through all three in that order, with a simple model you can adjust, so you can see what each one costs, what it buys, and which one fits the system you have.
A note on the baseline. This analysis started with Harper, where a node typically answers a read from memory or local disk in about a millisecond. That makes it a clean starting point: with server time that small, nearly everything left is network, so the model isolates what distance costs and what a multi-region, sharded cluster can do about it. The reasoning carries over to any data platform. If yours takes longer to answer a read, add that time on top.
Latency is distance
Once a request reaches a node that has the record in memory or on local disk, serving it is the quick part. Almost everything else a user waits for is the network path between them and that node.
A useful rule of thumb is about 0.014 ms of round trip per kilometer, plus a small fixed cost per hop. New York to London works out to roughly 70 ms. No query optimizer gets that back. The only way to shorten the path is to put a node closer to the user.
So for read-heavy applications, the question "how do I get lower latency" mostly reduces to "how many regions can I afford to be in."
With everything in one region, the average read in this model takes 106 ms. Four regions bring it down to 38 ms, and eleven to 17 ms. The first few regions do most of the work, but they do not finish it. The problem is paying for it.
Cost is node size times node count
The simplest way to add a region is to put a full copy of the data there. Every node can answer every read, and there is nothing to tune. If every node holds a full copy of the dataset, every node has to be sized for all of it: disk for the data, memory for the working set, and enough write capacity to receive every write in the cluster.
That ties together two things you would rather keep separate. Say you have 200 GB of data and can pay for 800 GB of node capacity in total. That is four full copies, in four regions, and that is the end of it. A fifth region means paying for another 200 GB. Wider distribution costs proportionally more, and a node can never be smaller than the whole dataset.
Sharding alone makes reads worse
Sharding breaks that link. Each node owns a slice of the data, so it can be smaller, and the same budget buys more nodes in more regions.
The catch is obvious once you say it out loud. A nearby node does not help if it does not have the record you asked for. Split the data three ways and only a third of reads are served by the node that received them. The rest are relayed to an owner somewhere else, which adds a second network round trip on top of the first.
In the model below, a node that owns a third of the data needs about 73 GB, so the same 800 GB of capacity buys ten nodes instead of four. The average read goes from 38 ms to 62 ms. Same money, more than twice as many regions, slower application.
Hot replicas recover the locality
Real read traffic is not uniform. A small share of records gets most of the reads: the popular products, the trending content, the active sessions. This is usually modeled with a Zipf distribution, and for typical web and catalog traffic the skew factor is close to 1.0.
That skew is what makes the whole approach work. If every node keeps its owned slice plus local copies of the most-read records from the other shards, a small replica budget covers a large share of reads. At a skew of 1.0, replicas of the hottest 2% of records cover about 78% of reads.
Two properties keep these replicas cheap:
- Nobody tracks them. Ownership is deterministic (a hash of the primary key picks the shard), so every node knows where the authoritative copy lives. A replica is a cached copy on a non-owner. The cluster does not record where replicas exist, so creating one never turns a read into a cluster-wide write.
- They follow access. A replica is created when a non-owner fetches a record, is invalidated when the record changes, and expires when it stops being read. Cold records leave on their own.
Run the same ten-node cluster with hot replicas and 85% of reads are served locally. The average read drops to 28 ms, about a quarter lower than the four-node cluster, for the same spend.
The tool below runs this for your own situation. Tell it how much data you have, how much hardware you can run, how concentrated your reads are, and where your users are, and it recommends a layout:
Everything depends on read skew
Here is the part I did not expect to be so sharp. The benefit is extremely sensitive to how skewed your reads are.
With the default settings, the four-node baseline averages 38 ms. At a skew of 1.0, where the most-read 2% of records get 78% of reads, the sharded cluster averages 28 ms. At 0.8, where they get 44%, it averages 43 ms. Somewhere in between, the sharded cluster crosses over and becomes worse than the cluster it replaced. A small change in the access pattern moves a three-shard design from a clear win to a net loss.
The practical advice: measure your read distribution before you choose a shard count. Take a day of access logs, rank records by read count, and check what share of reads the top 1% and 5% receive. That one number tells you more about whether sharding will help than any amount of capacity planning.
Where it helps less
This design favors applications where the latency of the first response is the product: e-commerce, catalogs, content, API caching. It helps less in a few cases.
- Real-time workloads. Long-lived connections and streams of small messages gain little from hot replicas. They are also easier to scale on demand, because there is less stationary data to move.
- Uniform access. If every record is equally likely to be read, no replica budget short of a full copy keeps reads local.
- Too few owners. Smaller nodes do not have to mean less availability, but each shard needs at least two owners in different places. A budget that buys fewer than two owners per shard is buying an outage.
A quick way to decide
Four questions, in order. The first "no" is usually your answer.
- Are your users spread across continents? If they sit in one region, a few full copies already put a node next to almost everyone. There is little distance left to remove, and sharding for latency will not pay.
- Do a few records get most of your reads? Check what share of reads goes to your most-read 2% of records. In this model, with hardware for four full copies, sharding starts to pay at around 70%. Well below that, stay with full copies.
- Can you afford two owners for every shard? If the hardware only stretches to one, you are trading availability for latency. Use fewer shards or do not shard.
- Is the speed of the first response what your users notice? Catalogs, content and cached API responses benefit. Streams and long-lived connections mostly do not.
Adding capacity without moving data
The classic objection to auto-scaling a database is that compute scales easily and data does not. Adding a node means copying data to it, and copying data at the moment you are short on capacity makes things worse.
A sharded cluster with replicas has another option: a node that owns nothing. It joins with index entries only, takes requests immediately, and fills its replica set as records are asked for. For the first few minutes it relays most reads. As it warms up, it absorbs more and more of the read traffic without a bulk copy and without a load spike on the owners.
That does not make a database scale in seconds. It does shrink the lead time from hours to minutes, which is enough for a planned sale, a holiday, or a forecast that says the next hour will be busy.
It is also worth being precise about what people mean by "auto-scaling." In most conversations I have had, the real requirement is assurance that the system survives peak traffic at a price closer to average traffic. Scaling on demand is one way to get that. Shared headroom is another. When several workloads share hosts, their peaks rarely line up. Across Harper's own fleet, the combined peak is about 63% of what you get by adding up every customer's individual peak. That gap is headroom nobody has to pay for separately. Many small nodes pack onto shared hosts far better than a few large ones, and spare capacity that is already running is available instantly.
One more lever: who sends each write
Every node that stores a write has to receive it once, so a cluster of N nodes makes N − 1 sends per write however it is wired. What the wiring, or network topology, decides is which links carry those sends.
In a full mesh, the node that takes a write sends it directly to every other node, including one transfer across the ocean for every node on the other side. Full mesh is the common default. An alternative design is a grid: each node connects to the other nodes in its region and to the nodes that own the same shard in other regions. A write then crosses between regions once per remote region and fans out locally from there.
Whether that saves money depends on your provider's pricing for transfer between regions versus inside one. Where the two cost the same, the gain is fewer connections and less CPU. Where cross-region transfer is priced several times higher, it shows up on the bill.
This analysis came out of building sharding and hot replicas into Harper. If you want to see how that looks in practice, the Harper 5.4 announcement covers what shipped.
What to do next
If your data sits in one place today, here is a sensible order to work in.
- Find out where your users are. Break your traffic down by region. If nearly everyone is close to your current system, distance is not your problem and none of this is urgent.
- Start with full copies. Put a complete copy in the two or three regions with the most users. It is the simplest layout to run, every node can answer every read, and it gives you redundancy you probably want anyway.
- Measure how concentrated your reads are. Take a day of access logs, rank records by read count, and work out what share of all reads goes to the most-read 2%. You will need this number before you consider sharding.
- Watch your capacity ratio. Add up the capacity of every node and divide by the size of your data. As your data grows, or as you want more regions than you can afford full copies for, that ratio falls. That is the point to open the tool above and see what it recommends.
- If you shard, take your time choosing the layout. Use the fewest shards that give you the gain, keep two owners for every shard, and turn on hot replicas from the start. Shard layouts are hard to change once data is in them, so this is the decision worth a second look.
Want a second opinion on your numbers?
The model in this post is simple on purpose. Real systems have write-heavy tables, uneven traffic between regions, and data that has to stay in one place. If you are planning the move from one central system to many regions and would like to work through it with someone who builds this for a living, tell us what you are building and an expert at Harper will go through it with you.






.webp)
