Load Balancing cheat sheet · System Design

1.L4 vs L7 load balancing

An L4 (transport-layer) load balancer routes by IP and port alone, without looking inside the request — fast, but blind to content. An L7 (application-layer) load balancer reads the actual HTTP request — path, headers, cookies — and can route on that, at the cost of more work per request.

Key features

L4 (transport layer): routes by IP address and port, does not inspect request content
L7 (application layer): routes by HTTP path, headers, cookies, or the request body
L4 is faster and simpler — no parsing of the actual protocol payload
L7 enables content-based routing: /api/* to one service, /static/* to another
L7 can also terminate TLS, inspect headers for auth, and do sticky sessions by cookie

What each layer can see and decide on

What each layer can see and decide on
Layer	Sees	Can route on	Cost per request
L4 (transport)	IP + port	Source/destination IP, port	Very low
L7 (application)	Full HTTP request	Path, host header, cookie, body	Higher (parses the request)

Same traffic, two routing decisions

L4 (transport layer)

Sees only IP + port
Cannot tell /api/orders from /static/logo.png
Forwards any packet on port 443 to any backend IP
Very low cost per request

L7 (application layer)

Reads the full HTTP request
Routes /api/* to the orders-service pool
Routes /static/* to the CDN origin pool
Higher cost — parses the request

Command

text

L4: route by (source IP, dest IP, port) — no payload inspection
L7: route by (HTTP method, path, host, header, cookie)

L4 routes by IP/port only, fast and content-blind; L7 reads the actual HTTP request and can route by path, header or cookie, at higher cost per request.

2.Load balancing algorithms

A load balancer needs a rule for picking which backend instance handles the next request. Round robin cycles through instances evenly; weighted round robin favors bigger instances; least connections sends traffic to whichever instance is least busy right now; hashing routes the same client (or key) to the same instance consistently.

Key features

Round robin: cycles through instances in order, ignoring current load
Weighted round robin: like round robin, but bigger/faster instances get proportionally more requests
Least connections: routes to whichever instance currently has the fewest open connections
Hashing: routes by a key (client IP, session ID) so the same key always reaches the same instance
Consistent hashing: a hashing scheme that minimizes reshuffling when instances are added or removed

Choosing an algorithm

Choosing an algorithm
Algorithm	Decision basis	Good fit when
Round robin	Fixed rotation order	Instances are equal-sized, requests are similar cost
Weighted round robin	Rotation + a weight per instance	Instances have different capacity
Least connections	Current open-connection count	Requests vary a lot in how long they take
Hashing	Hash of a key (IP, session)	The same client should keep hitting the same instance
Consistent hashing	Hash placed on a ring of instances	Instances scale up/down often, minimize reshuffling

Adding a 4th instance: plain hashing vs consistent hashing

Plain hashing (% N)

hash(key) % 3 -> hash(key) % 4
Almost every client's assignment changes
Caches and sticky sessions go cold at once

Consistent hashing

Instances placed on a ring
Only keys between old and new neighbor move
Most assignments stay untouched

Command

text

round-robin:        instance[i % N]
weighted round-robin: instance chosen proportional to weight
least-connections:   instance with min(open_connections)
hashing:             instance[hash(key) % N]
consistent hashing:  instance nearest to hash(key) on a ring

Round robin/weighted round robin ignore current load; least connections adapts to it; hashing pins a key to an instance; consistent hashing does that without reshuffling everyone on scale events.

3.Health checks, connection draining, sticky sessions and failover

A load balancer needs more than an algorithm — it needs to know which instances are actually healthy (health checks), how to retire one without dropping in-flight requests (connection draining), how to keep a client on the same instance when needed (sticky sessions), and what to do when an instance dies mid-request (failover).

Key features

Health check: a periodic probe (HTTP ping, TCP connect) confirming an instance can serve traffic
An instance failing health checks is removed from rotation until it passes again
Connection draining: stop sending new requests to a retiring instance, let in-flight ones finish first
Sticky sessions: pin a client to the same instance, usually via a cookie or source IP hash
Failover: when an instance dies, in-flight requests to it fail or retry against a healthy instance

One backend instance's life in the pool

scheduled forretirementin-flightrequests finishK consecutivehealth checks failtaken outof rotationhealth checkpasses again

In rotation (healthy)

start

Draining (retiring)

Removed from pool

Failed health check

Command

text

health check:       GET /healthz every N seconds
                     fail K times → remove from pool
connection draining: stop new traffic, wait up to T
                     seconds for in-flight requests
sticky session:      cookie or IP hash → same instance
failover:            instance dies → route future
                     requests to remaining healthy ones

Health checks decide who is in the pool; draining retires an instance without dropping in-flight work; sticky sessions pin a client to an instance for locality, not as the source of truth; failover reroutes when one dies.

4.Why stateless servers make horizontal scaling easier

If an application instance holds no local state (no data that only it has), a load balancer can route any request to any instance interchangeably. That is what makes horizontal scaling — adding more instances — simple: a new instance is immediately as capable as any existing one.

Key features

Stateless instance: holds no data that only it has — any instance can serve any request
A load balancer can route to any healthy instance without caring which one served the last request
Adding a new instance instantly adds capacity — no state to replicate onto it first
A stateful instance (local session data, local file writes) cannot be freely substituted
Statelessness is what makes an instance disposable — it can be killed and replaced with no data loss

Adding capacity: stateless vs stateful instances

Stateless pool

Session data lives in shared Redis, not on any one instance
A new instance is immediately as capable as any existing one
Load balancer routes to any healthy instance, no exceptions
Traffic doubles → add instances → done

Stateful pool

Each instance holds its own local session data
A request must reach the specific instance holding it
A new instance starts with no data — cannot serve existing sessions
Scaling requires sticky routing or a state-migration plan

Command

text

stateless: any_instance.handle(request) — result is
           the same regardless of which instance runs it
stateful:  request must reach the specific instance
           holding the relevant local state

A stateless instance can serve any request, which is what lets horizontal scaling add capacity instantly instead of needing to replicate state onto every new instance first.