Capacity estimation in system design: numbers before boxes

Arjun Nair has spent the whole morning on the wrong problem. His notebook is full of disk and backup plans for the final’s bookings.

Meera Kulkarni asks him how much space those bookings need. Ten minutes later he is red in the face. One booking is about 1 KB, roughly a thousand letters of text. 40,000 seats at 1 KB each is 40,000 KB, which is 40 MB (1,000 KB make 1 MB). That fits on a phone.

“And how many taps a second at noon?” Meera asks. Her guess is 5 lakh people in the first 10 minutes, about 20 requests each: 1 crore requests. Arjun spreads them over a day: about 116 a second. “Our ₹1,200 server handles 200 a second,” he says. “We are fine.” Meera frowns. Fans do not come spread over a day.

Capacity estimation is quick, rough arithmetic that tells you how much load a system must carry and store, before you draw any box. It gives four numbers: users, requests a second, storage and data sent.

The number to design for is the busiest short stretch, called the peak. For SeatJaldi it is the first 10 minutes after noon.

• Rough numbers decide the design, and the peak, not the average, is the design.
• Round hard: 116 becomes 100, and 86,400 becomes 1 lakh. A little off is fine; ten times off is not.
• Say every guess out loud, so someone can check it.
• Find the biggest number. That is the hard part.

You have seen this when a ticket site is fine all week, then shows a waiting page at noon on sale day. It was sized for a normal day.

An exam-results site is quick all year, then falls over a minute after results come out. Nearly every visitor arrived in the same few minutes.

A cab app in the 6 pm rain shows “no cabs nearby”. Far more people wanted a cab that hour than the fleet was planned for.

To calculate QPS, the number from lesson 6, divide the total requests by the seconds they arrive in. The hard part is picking the right seconds.

Spread 1 crore requests over a day and you get about 116 a second. Squeeze them into the 600 seconds after noon and you get about 16,700. Nandini’s 2,900 rotis are a trickle over a day, but about 48 a minute in a one-hour rush.

Divide the peak by what one server can take, then round up: 16,700 divided by 200 is about 84 servers. Round up, never down: you cannot rent 83.5 servers, and 83 would leave some requests with nowhere to go.

Nandini does the same with cooks. Say one cook rolls 20 rotis a minute and the rush needs 48. That is 2.4 cooks, so she books 3.

Engineers add headroom, split reads from writes, and use Little's law to count the workers needed.

Headroom means building for more than the estimate. A common habit is 2x: 16,700 x 2 = 33,400 a second, and 33,400 / 200 = 167 servers. You gain room for a bad guess or a dead machine, at twice the cost. Nandini’s extra 20% of rotis is her headroom.

Google’s guide to running large systems works an example where a system still carries its peak with two groups of servers down. It also says to test each part until it breaks, which turns a guess like 200 a second into a measurement.

Little's law counts the workers, treating the 10-minute rush as steady. Assume each request takes 0.2 seconds and each worker holds one request at a time. A server at 200 a second has 200 x 0.2 = 40 requests in progress, so it needs about 40 workers. Check: each worker finishes 1 / 0.2 = 5 a second, and 40 x 5 = 200.

For the whole sale, 16,700 x 0.2 = 3,340 requests are in progress, and 3,340 / 40 = 83.5, about 84 servers. A slowdown eats capacity: at 0.4 seconds each, the same peak needs 167 servers, double, just as the doubling in lesson 6 showed.

A classic worked example: a link shortener

A link shortener turns a long web address into a short one.

What just happened? Writes were tiny, reads 100 times bigger, and 3 TB fits on a few disks. Raw capacity was simple. The real design is availability: keeping it answering all day.

Try it yourself

1. 1

   Write the guesses

   A concert has 10,000 seats. Assume 1 lakh people in the first 10 minutes, about 20 requests each.

2. 2

   Find the peak

   Work out the total requests, then divide by 600 seconds.

3. 3

   Find the storage

   Work out the space for 10,000 bookings at about 1 KB each.

4. 4

   Count servers

   Divide the peak by 200 and round up. Then add 2x headroom the same way as the 167 above.

5. 5

   Name the hard part

   Say which number is the hard part, and which guess to check first.

6. 6

   Check your answer

   Total: 100,000 x 20 = 2,000,000. Peak: 2,000,000 / 600 = about 3,300 a second. Storage: 10,000 x 1 KB = about 10 MB. Servers: 3,300 / 200 = 16.5, so 17, or about 33 to 34 with 2x headroom. The hard part is the peak; check the 20 requests each first. You got it right if you have about 3,300 a second, about 10 MB and about 17 servers.

Arjun tears up his disk plans. Storage was never the hard part. The peak was: about 16,700 requests a second, against 200 on one server.

“Will it hold at noon?” Meera asks. “Not on one server,” Arjun says. “We need about 84 like it.” Meera shakes her head. “I cannot rent 84 of anything.” Arjun has an idea: buy one huge server instead. Meera does not look sure.

Like Nandini, they have their plate count, far more than one cook can make. One server cannot take 16,700 requests a second. A bigger machine, or more machines?