beginnerintermediateadvanced

System Design · Section 50

WebSockets at Scale

Level: intermediate
Read: 20 min
Concepts: 3

A WebSocket connection is pinned to the single server process that accepted its handshake — connection affinity, which means any other part of the system needs a shared registry (typically Redis, with a short TTL refreshed by heartbeats) to find out which instance currently holds a given user's connection. A registry lookup alone does not deliver anything, though — pub/sub or a message broker is the actual relay that gets a message onto the right instance, and the same mechanism handles fan-out to many recipients at once. Running a connection reliably also needs reconnect logic with backoff and jitter (to avoid a thundering-herd reconnect), a heartbeat/ping cycle (since TCP alone will not detect a connection that is silently dead), and active re-validation of authentication rather than trusting it forever — the whole point being that no amount of in-process cleverness substitutes for this shared coordination layer once there is more than one server instance.

System Design overview

System Design for Beginners CoursefreeCodeCamp.org

What is true here

A WebSocket is pinned to one server process — a shared registry is the only way other instances learn where a connection actually lives.
A registry hit proves a connection existed recently, not that a message was delivered — pub/sub or a broker is the relay that actually gets it there.
Registry entries need a short TTL refreshed by heartbeats so a crashed instance's stale entries expire rather than silently misdirecting messages.
Reconnect logic needs backoff and jitter, and authentication must be actively re-checked — a WebSocket authenticates once, at connect time, with no per-message equivalent of an HTTP header.

What you will be able to do

Explain why a WebSocket connection cannot be handled by just any server instance the way a stateless request can
Design a connection registry with a TTL that survives an ungraceful instance crash
Use pub/sub or a broker to relay a message to the specific instance holding a target connection
Avoid the mistake of assuming local process memory can coordinate connections across a multi-instance fleet

Finding and reaching a connection

Why one instance holds a connection, how other instances learn which one, and the relay mechanism that actually delivers a message there.

Connection affinity and connection registries

coreintermediate

A WebSocket is a long-lived, stateful TCP connection — once a client's handshake completes, that specific connection is held open in the memory of exactly one server process, on exactly one machine. This is connection affinity: unlike a stateless HTTP request, which any server behind a load balancer can answer, a WebSocket message can only be pushed to a client through the one process that is holding its open socket. A connection registry is the piece of shared infrastructure — usually Redis or a database — that records which server instance currently holds which user's connection, so any other part of the system can look up where to send a message.

Think of it as

Connection affinity is like a phone call versus a letter. A letter (HTTP request) can be handed to any postal worker who happens to be free — none of them need to remember you. A phone call (WebSocket) is answered by one specific operator, and it stays connected to that operator for the whole conversation; if someone else in the building wants to relay a message to you, they cannot just pick up any phone — they have to find out which operator's line you are on. The connection registry is the building's switchboard log that records "caller 482 is on operator 7's line right now."

text

// Connection registry as a Redis hash, updated on connect/disconnect
HSET conn:registry user:482 instance:server-7
EXPIRE conn:registry:user:482 90   -- guards against stale entries

// On disconnect (graceful or crash-detected via heartbeat)
DEL conn:registry:user:482

What we're doing: Route a server-initiated push to a user whose WebSocket lives on a different instance.

connection-registry.txttext

1. Client opens a WebSocket; the handshake lands on
   server-7 (chosen by the load balancer).
2. server-7 writes to the shared registry:
     user:482 -> server-7
3. Later, server-3 needs to push a notification to
   user 482 (e.g. triggered by an API call it received).
4. server-3 looks up the registry: user:482 -> server-7.
5. server-3 cannot write to that socket directly — it
   asks the pub/sub layer to relay the message to
   server-7, which holds the actual open connection.
6. server-7 receives the relayed message and writes it
   to user 482's live socket.

2: The instance that actually accepted the handshake is the only one that can write to this specific socket — this is connection affinity in effect.
4: The registry lookup is what makes step 5 possible at all — without it, server-3 has no way to know user 482 is even connected, let alone to which instance.
10: server-3 never touches the socket itself; it hands the message to a relay mechanism (pub/sub or a broker) that reaches the correct instance.

Why this works: This is the concrete two-hop pattern every WebSocket fan-out system uses: a registry to find the right instance, then a relay to actually deliver the message through it — the registry alone cannot deliver anything, and the relay alone has no idea who to deliver to.

Letting a crashed instance leave a stale registry entry behind

Wrong

text

// Registry entry is written on connect,
// but nothing ever removes it except a
// clean disconnect handler.
HSET conn:registry user:482 instance:server-7
// server-7 crashes hard — the disconnect
// handler never runs.

Better

text

// Give the entry a TTL and refresh it on every
// heartbeat; a crashed instance's entries expire
// on their own within one missed-heartbeat window.
HSET conn:registry user:482 instance:server-7
EXPIRE conn:registry:user:482 90
// refreshed every heartbeat (e.g. every 30s)

What you see: Messages meant for a user are silently dropped, or errors accumulate trying to reach a dead instance, minutes or hours after that instance actually crashed — the registry still confidently reports a location that no longer holds any connection.

Why: A registry entry written only on connect and removed only on graceful disconnect has no way to reflect an ungraceful failure (crash, network partition, OOM kill) — a short TTL refreshed by the heartbeat mechanism bounds how long a stale entry can mislead the rest of the system.

Routing a push to a socket on a different instance

Client

server-7

Registry

server-3

1. WebSocket handshake
2. user:482 → server-7
3. lookup user:482
4. server-7
5. relay via pub/sub
6. write to live socket

Stateless HTTP vs stateful WebSocket routing

Stateless HTTP vs stateful WebSocket routing
Property	Stateless HTTP request	WebSocket connection
Which server can handle it	Any instance behind the load balancer	Only the one instance holding the open socket
State needed to route	None — request carries everything	Registry lookup: user/connection ID → instance
Effect of instance restart	Next request just goes elsewhere	Every connection on that instance is dropped and must reconnect
Scaling model	Add instances, load balancer spreads requests	Add instances, but each holds a disjoint slice of live connections

Remember: A WebSocket is pinned to one server process (connection affinity) — a registry (typically Redis, with a short TTL refreshed by heartbeats) is the only way other instances learn where a given user's connection actually lives.

Pub/sub or brokers for cross-instance fan-out

coreintermediate

Knowing which instance holds a connection (the registry) is only half the problem — something still has to actually get the message onto that instance. Pub/sub (e.g. Redis Pub/Sub) or a message broker is that relay: every server instance subscribes to a shared channel or topic, and when any instance needs to deliver a message to a user, it publishes the message once; every instance receives it, and the one instance that actually holds that user's socket writes it to the connection. This is also how fan-out to many recipients at once works — a single publish (e.g. "user X started typing") reaches every subscribed instance, each of which forwards it only to its own locally connected recipients.

Think of it as

Think of an apartment building's intercom system instead of knocking directly on a resident's door. A visitor at the front desk (one server instance) does not know which floor or unit a resident is on. Instead of running through the building checking every door, they press the intercom button, which broadcasts to every floor's speaker (every server instance subscribed to the channel). Only the speaker on the correct floor near the correct unit (the instance actually holding that user's connection) does anything useful with the announcement — everyone else's speaker just plays it and it goes nowhere.

text

// Every instance, on startup:
SUBSCRIBE ws:fanout

// Any instance, to deliver to one user:
PUBLISH ws:fanout '{"userId":482,"payload":{...}}'

// Each subscribing instance, on receipt:
if registry.ownsLocally(msg.userId):
    localSocket(msg.userId).send(msg.payload)
// else: message is silently discarded on this instance

What we're doing: Fan a single chat message out to every recipient, wherever their connections happen to live.

fanout.txttext

1. User 482 (connected to server-7) sends a chat
   message to room "general".
2. server-7 looks up room "general"'s member list
   (not a per-user registry lookup this time — a
   whole room needs the message).
3. server-7 publishes once:
     PUBLISH room:general '{"from":482,"text":"hi"}'
4. server-3, server-7, server-9 — every instance
   subscribed to room:general — receives the message.
5. Each instance checks its own local connection map
   for members of "general" it happens to be holding,
   and writes the message to just those sockets.
6. A member connected to server-3 gets the message;
   an instance with no members of "general" connected
   discards it after receiving it.

8: One publish call is enough — the sender does not enumerate which instances hold which recipients; that is the whole point of routing through pub/sub.
12: Every subscribed instance receives every message on the channel, even ones with zero relevant local connections — this fan-in-then-filter pattern is intentional, not a bug.

Why this works: This is the mechanism that makes connection affinity workable at scale: instances never need to know about each other directly or maintain a map of every other instance's connections — they only need to agree on a shared channel and filter locally.

Relying on Redis Pub/Sub for messages that must not be lost

Wrong

text

// Critical order-status update, fanned out
// only via Redis Pub/Sub
PUBLISH order-updates '{"orderId":9,"status":"shipped"}'
// if the owning instance is mid-restart and
// briefly unsubscribed, this update is gone
// forever -- no retry, no persistence.

Better

text

// Persist the state change first; Pub/Sub is
// only the "wake up and check" signal.
db.update(order_id=9, status='shipped')
PUBLISH order-updates '{"orderId":9}'
// a client that reconnects after missing the
// publish can still fetch the current status
// from the database -- Pub/Sub is a nudge,
// not the source of truth.

What you see: A user misses a real-time update entirely — no error is logged anywhere, because Redis Pub/Sub delivered the message exactly as designed to whichever instances were subscribed at that instant; an instance mid-restart or a brief network blip simply never received it, and Pub/Sub does not retry or persist.

Why: Redis Pub/Sub is documented as at-most-once delivery with no persistence — a message published while a subscriber is not actively connected is gone, not queued. Anything the application cannot tolerate silently losing needs either a durable broker or a fallback path (poll the database on reconnect) alongside Pub/Sub.

One publish, every instance receives it, only relevant ones deliver

server-7

publishes once

room:general

shared pub/sub channel

server-3

has a member — delivers

server-9

no members — discards

Redis Pub/Sub vs a message broker for WebSocket fan-out

Redis Pub/Sub vs a message broker for WebSocket fan-out
Property	Redis Pub/Sub	Message broker (Kafka/RabbitMQ)
Delivery guarantee	At-most-once — lost if no subscriber is listening	Durable; can offer at-least-once and replay
Operational complexity	Low — reuses a Redis instance already in the stack	Higher — separate cluster, consumer group management
Fits	Ephemeral, real-time-only messages (typing indicators, presence)	Fan-out that must survive a consumer being briefly down
Message ordering	Per-channel, best-effort	Configurable, stronger guarantees per partition/queue

Remember: Pub/sub or a broker is the actual relay that solves connection affinity: publish once, every instance receives it, only the instance holding the target connection acts on it. Redis Pub/Sub is fast but at-most-once with no persistence — use a durable broker when losing a fan-out message is unacceptable.

Keeping a connection honest over time

Reconnect logic, heartbeat/ping, and authentication — and the explicit mistake this section warns against by name.

Connection lifecycle: reconnect, heartbeat/ping and authentication

coreintermediate

A WebSocket connection is not "up" just because the handshake once succeeded — three mechanics keep it meaningfully alive and correct over time. Heartbeat/ping is a small periodic control frame (WebSocket defines Ping and Pong control frames, opcodes 0x9 and 0xA) each side sends so the other can detect a dead connection that TCP itself did not notice — network devices and proxies silently drop idle connections, and without a ping/pong cycle a "connected" socket can actually be dead for minutes before anyone finds out. Reconnect logic is the client-side behavior for when a connection does drop — normally with exponential backoff and jitter, so a mass disconnect (e.g. one server instance restarting) does not cause every affected client to reconnect in the same instant and overwhelm the fleet. Authentication on a WebSocket happens once, at connection time (since there is no per-message equivalent of an HTTP header), so the server must revalidate or expire that identity itself rather than trusting it forever.

Think of it as

Think of heartbeat/ping like a phone call where both people occasionally say "you there?" — without it, a dropped call due to a bad signal can go unnoticed by one side for a long time, with them talking into silence. Reconnect logic with backoff and jitter is like a crowd leaving a stadium through one door: if everyone rushes at once, the door jams; staggering people out over a few minutes gets everyone through faster overall. Authentication on connect-only is like checking ID once at the door of a members-only club — nobody re-checks your ID every time you speak inside, so the club needs its own way to notice if your membership expired while you were still standing at the bar.

text

// Server-side heartbeat loop (per connection)
every 30s: send PING
if no PONG received within 10s: close connection

// Client-side reconnect with backoff + jitter
delay = min(max_delay, base_delay * 2 ** attempt)
delay += random(0, delay * 0.3)   // jitter
sleep(delay); reconnect(); attempt += 1

What we're doing: Trace one connection through a disconnect, detection, and recovery.

connection-lifecycle.txttext

1. Client connects with ?token=<jwt> in the handshake
   URL; server validates it once and stores the user
   ID against this connection.
2. Server sends PING every 30s; client replies PONG.
3. Client's WiFi drops. TCP does not immediately
   notice -- the socket looks "open" to the OS.
4. Server sends PING at t=30s; no PONG arrives.
5. At t=40s (10s timeout), server marks the
   connection dead, removes it from the registry,
   closes the socket.
6. Client's network returns. Its reconnect logic
   has been backing off: attempt 1 at 1s, attempt 2
   at ~2s, attempt 3 at ~4s (each with jitter).
7. Reconnect succeeds; server re-validates the token
   (now possibly closer to its expiry) and writes a
   fresh registry entry.

4: The server never re-checks the token after this point — authentication was a connect-time event, not a per-message one.
12: The 10-second timeout after a missed Pong is what turns a silent TCP-level failure into a detected, actionable disconnect.
17: Backoff with jitter means this client is not retrying at the exact same instant as every other client who dropped at the same moment.

Why this works: Each of the three mechanics fires at a different point in this single trace — authentication only at step 1, heartbeat detecting the failure at steps 2-5, reconnect logic governing recovery at step 6 — showing they are separate concerns that happen to compose around one connection's life.

Using a fixed reconnect interval instead of backoff with jitter

Wrong

text

// Every client retries exactly 2 seconds
// after any disconnect
def on_disconnect():
    sleep(2)
    reconnect()

Better

text

def on_disconnect(attempt):
    delay = min(30, 1 * 2 ** attempt)
    delay += random(0, delay * 0.3)
    sleep(delay)
    reconnect()
    # attempt increments on failure, resets on success

What you see: A single server restart or brief network blip that disconnects thousands of clients simultaneously is immediately followed by a synchronized reconnect spike exactly 2 seconds later — the resulting load can be worse than the outage itself, sometimes causing a second wave of failures on the servers now absorbing every client's retry at once.

Why: A fixed delay preserves whatever synchronization caused the mass disconnect in the first place — every client counts down the same interval and retries in the same instant. Exponential backoff spreads retries out over time as failures repeat, and jitter breaks the remaining synchronization within each retry round.

A connection's lifecycle from handshake to reconnect

Connecting

start

Open

Detected dead

end

Reconnecting

end

The three connection-lifecycle mechanics and what each one is actually for

The three connection-lifecycle mechanics and what each one is actually for
Mechanic	Problem it solves	Typical implementation
Heartbeat/ping	TCP alone does not reliably detect a silently-dead connection (idle timeouts, NAT/proxy drops)	Ping frame every N seconds; disconnect if no Pong within a timeout
Reconnect logic	A client must recover from a drop without hammering the server the instant it happens	Exponential backoff with random jitter, capped at a max interval
Authentication	No per-message header exists to re-prove identity on every frame	Token validated once at handshake; expiry handled by forced disconnect or explicit re-auth

Remember: Heartbeat/ping (protocol-level control frames) detects a connection TCP thinks is fine but actually is not; reconnect logic needs exponential backoff with jitter to avoid a synchronized thundering herd; authentication happens once at connect time and must be actively re-checked, not assumed to hold forever. None of the three make local process memory a valid coordinator across nodes — that requires the registry and pub/sub layer instead.