beginnerintermediateadvanced

System Design · Section 13

Horizontal vs Vertical Scaling

Level: intermediate
Read: 18 min
Concepts: 3

A system can scale in one of two directions — vertically, by moving to a bigger machine, or horizontally, by adding more machines and distributing load across them. Vertical scaling is operationally simpler with a non-linear cost curve and a hard ceiling; horizontal scaling has, in principle, no ceiling, at the cost of real operational complexity — load balancing and, for stateful systems, replication or sharding.

This section

Horizontal and Vertical Scaling (How to Scale Your Application) - System DesignCaleb Curry

What is true here

Vertical scaling means a bigger machine; horizontal scaling means more machines.
Vertical scaling has a hard ceiling — the largest machine available; horizontal scaling does not, in principle.
Vertical scaling's cost curve is often non-linear; horizontal scaling's is closer to linear per instance.
Horizontal scaling requires a distributable workload and adds real operational complexity that vertical scaling does not.

What you will be able to do

Distinguish vertical from horizontal scaling and what each actually changes
Weigh cost curve and operational complexity, not just raw capacity, when choosing a scaling direction
Recognize when a workload is approaching vertical scaling's ceiling and needs a horizontal plan

Two directions, one trade-off

What each direction actually changes, the cost/complexity trade-off behind choosing between them, and the statelessness check that decides whether horizontal scaling will actually help.

Horizontal vs vertical scaling

corebeginner

Vertical scaling means moving to a bigger machine — more CPU, more RAM, on the same single instance. Horizontal scaling means adding more machines and spreading the work across them. They solve the same problem — not enough capacity — in structurally different ways.

Think of it as

Vertical scaling is replacing a small delivery van with a bigger truck — the same one vehicle now carries more, but there is still only one, and eventually no truck is big enough. Horizontal scaling is adding more delivery vans — each one stays the same size, but the fleet as a whole carries more, and adding another van has no upper limit the way "make the truck bigger" eventually does.

text

vertical:   1 instance, size ↑  (CPU, RAM, disk)
horizontal: N instances, N ↑    (load-balanced)

What we're doing: Show a database hitting the vertical-scaling ceiling and needing horizontal scaling instead.

scaling-ceiling.txttext

A single Postgres instance on the largest available
machine (e.g. 128 vCPU / 1TB RAM) is still CPU-bound
under peak read traffic.

Vertical scaling is exhausted — there is no bigger
machine to move to.

Horizontal options from here:
  - read replicas: route reads across several
    read-only copies (see Replication, section 22)
  - sharding: split the data itself across multiple
    primary instances (see Sharding, section 23)

Both are horizontal — more machines sharing the load
— because vertical scaling ran out of room first.

5: This is the ceiling itself: at some size, there is no bigger machine to buy.
8: Both options that follow are horizontal — the only direction left once vertical is exhausted.

Why this works: Almost every system eventually hits vertical scaling's ceiling — a design that only ever plans to "get a bigger box" has no answer for what happens after the biggest box is already in use.

Treating vertical scaling as a permanent solution rather than a stopgap

Wrong

text

"Traffic doubled — just resize the database to
the next instance size."

Better

text

"Traffic doubled — resize now if there's room,
but also plan for read replicas or sharding
before we're on the largest available instance
with nowhere left to go."

What you see: A team repeatedly resizes to the next instance tier until, without warning, they are already on the provider's largest SKU and traffic keeps growing — with no horizontal plan in place and a design that assumed vertical scaling would always have room.

Why: Vertical scaling is the easy, no-code-change lever, which makes it tempting to lean on indefinitely — but it has a real ceiling, and a design should have a horizontal plan ready before that ceiling is reached, not after.

Vertical vs. horizontal scaling

Vertical (scale up)

+One instance, bigger CPU/RAM/disk
+Hard ceiling — the largest machine available
+Usually no code change needed

Horizontal (scale out)

−More instances, load balanced
−No hard ceiling, in principle
−Needs a distributable workload, often statelessness

Vertical vs horizontal scaling

Vertical vs horizontal scaling
Property	Vertical (scale up)	Horizontal (scale out)
Mechanism	Bigger single machine	More machines
Ceiling	Largest machine available	No hard ceiling, in principle
Code changes needed	Usually none	Load balancing, often statelessness
Downtime to scale	Often requires a restart/resize	New instances join without downtime
Cost curve	Non-linear — big machines cost disproportionately more	Roughly linear per added instance

Together

text

Vertical: 4 vCPU / 16GB instance handling 500 req/s
  → resize to 16 vCPU / 64GB → handles ~2,000 req/s
  → next size up is the cloud provider's largest SKU;
    after that, there is nowhere left to scale up to.

Horizontal: 4 instances of 4 vCPU / 16GB, 500 req/s each
  → add 4 more identical instances → ~4,000 req/s
  → add 4 more again → ~8,000 req/s, same pattern,
    no ceiling from machine size.

Remember: Vertical scaling means a bigger machine and has a hard ceiling; horizontal scaling means more machines and has (in principle) none, at the cost of needing a distributable workload.

Scaling trade-offs: limits, cost curves and operational complexity

standardintermediate

Choosing vertical vs horizontal scaling is not just "which one is possible" — it is a trade-off between cost curve (does price rise linearly or steeply), operational complexity (does it need load balancing, replication, coordination), and how close the workload is to its scaling limit either way.

Think of it as

A single very large machine is like renting a private jumbo jet: simple to reason about (one thing, one route), but the price per extra seat rises steeply past a certain size, and there is a largest jet that exists. Many smaller planes flying the same route cost roughly linearly per extra seat and have no upper limit on total seats — but now you are coordinating multiple flights, crews and gates instead of one.

text

vertical:   simpler ops, non-linear cost, hard ceiling
horizontal: more ops complexity, ~linear cost, no hard ceiling

What we're doing: Compare the real cost and complexity trade-off for a workload that could go either way.

cost-curve-comparison.txttext

Workload needs roughly 32 vCPU total.

Vertical: one 32 vCPU instance
  - Simplest possible architecture, no load balancer.
  - Cloud pricing is often non-linear at the top end —
    a 32 vCPU instance can cost noticeably more than
    4x a 8 vCPU instance's price.
  - Single point of failure: this one instance dying
    takes the whole service down.

Horizontal: four 8 vCPU instances behind a load balancer
  - Cost scales roughly linearly — 4x the 8 vCPU price.
  - Needs a load balancer and (if stateful) some way
    to share state across instances.
  - Losing one instance loses 25% of capacity, not 100%.

8: This non-linear pricing at the high end is a real, common cloud-provider pattern — bigger instance tiers are frequently priced at a premium per unit of capacity.
13: The single-instance failure mode is a genuine cost of the "simpler" vertical option, not just an aside.

Why this works: The decision is rarely "which one works" — both often work at a given scale. It is "which trade-off (cost curve, blast radius, operational complexity) is worth paying for this workload."

Choosing horizontal scaling by default, without weighing the added operational cost

Wrong

text

"Horizontal scaling is best practice — split
everything into many small instances."

Better

text

"This workload fits comfortably on one mid-size
instance today, with room to grow vertically —
add the load-balancing/replication complexity
of horizontal scaling once we're actually
approaching that ceiling, not before."

What you see: A small service ends up with a load balancer, service discovery and multi-instance coordination for a workload that would have run comfortably, and much more simply, on a single reasonably-sized machine.

Why: Horizontal scaling's complexity (load balancing, and for stateful systems, replication/sharding) is a real, ongoing operational cost — paying it before the workload is anywhere near vertical scaling's ceiling is complexity with no corresponding benefit yet.

Three axes the scaling decision actually turns on

Cost curve

linear vs non-linear pricing

Blast radius

one instance dying vs 25%

Ops complexity

none vs load balancing/replication

Remember: Vertical scaling is operationally simpler with a non-linear cost curve and a hard ceiling; horizontal scaling is closer to linear cost with no hard ceiling, at the price of real operational complexity — choose based on where the workload sits relative to each limit, not by default.

Check statelessness before committing to horizontal scaling

standardintermediate

Before choosing horizontal scaling for a component, check whether it can actually be made stateless — because a stateful component does not automatically get easier to scale out just because more machines were added; it needs its state relocated first, or scaling out will not help.

Think of it as

Deciding to scale a service horizontally without checking statelessness first is like deciding to add more delivery trucks to speed up deliveries — without first checking that any truck can carry any package. If packages are pre-assigned to specific trucks (state tied to one instance), adding more trucks does not spread the existing backlog; it only helps with brand-new packages.

text

before choosing horizontal scaling, ask:
  can any instance handle any request right now?
  yes → add instances, done
  no  → relocate the state first, THEN add instances

What we're doing: Show a scaling decision that fails because statelessness was assumed rather than checked.

unchecked-statelessness.txttext

Report-generation service is slow under load.
Decision: "let's scale it horizontally — add 3 more
instances."

3 new instances are added. Load barely improves.

Investigation: each instance caches a large,
expensive-to-build report index in local memory,
built once at startup. New instances start with a
cold cache and are far slower per request until
each independently rebuilds the same index — and
requests are not even routed to prefer a "warm"
instance.

The fix was not "add instances" — it was moving the
report index to a shared cache (Redis) FIRST, which
is what actually would have made horizontal scaling
work as expected.

3: This decision assumed statelessness without checking it — the actual blocker (local cache state) was invisible until investigated.
9: This is the concrete cost: new instances add capacity in name only, because their state has to be rebuilt independently.

Why this works: Horizontal scaling only delivers the expected capacity increase when the thing being scaled was actually stateless to begin with — checking that assumption before scaling avoids paying for instances that do not help.

Adding instances to "fix" slowness without checking what state each instance holds

Wrong

text

"Traffic is up and requests are slow — add
more instances of the service."

Better

text

"Traffic is up — first check whether this
service holds any local state (cache, session,
in-memory queue). If it does, relocate that
state to a shared store before assuming more
instances will help proportionally."

What you see: New instances are added and infrastructure cost rises, but measured throughput improves far less than the instance count would predict — because much of each new instance's early capacity goes to independently rebuilding state a stateless design would have shared.

Why: The proportional capacity gain horizontal scaling is supposed to deliver depends entirely on the assumption that any instance can immediately do useful work — an assumption that silently fails for a component with unrelocated local state.

Check statelessness before scaling out

Can any instance handle any request?

Add instances

done

Relocate state first

e.g. move cache to Redis

Remember: Check whether a component is actually stateless before choosing horizontal scaling as the fix — a stateful component needs its state relocated first, or the new instances will not deliver the expected capacity.