beginnerintermediateadvanced

System Design · WebSockets at Scale

WebSockets at Scale

Why one connection lives on one instance, how a message finds it, and the lifecycle mechanics that keep it honest

Connection affinityone socket, one instance, no exceptions
Registryuser/connection ID → holding instance
Pub/sub relaythe mechanism that actually delivers
Heartbeat + backoffTCP won't tell you; don't reconnect all at once

1.Connection affinity and connection registries

A WebSocket is a long-lived, stateful TCP connection — once a client's handshake completes, that specific connection is held open in the memory of exactly one server process, on exactly one machine. This is connection affinity: unlike a stateless HTTP request, which any server behind a load balancer can answer, a WebSocket message can only be pushed to a client through the one process that is holding its open socket. A connection registry is the piece of shared infrastructure — usually Redis or a database — that records which server instance currently holds which user's connection, so any other part of the system can look up where to send a message.

Key features

A WebSocket connection is pinned to the single server process that accepted its handshake — it is not interchangeable the way a stateless HTTP request is
Connection affinity is usually enforced at the load balancer with sticky sessions (routing reconnects back to the same backend) or at the protocol level via a connection ID
A connection registry maps identity (user ID, connection ID) to location (which server instance, which process) and must be shared and fast — typically Redis hashes or sets, keyed by user ID
The registry must be updated on connect, on disconnect, and on any server crash — a stale entry pointing at a dead instance causes messages to silently go nowhere
Without a registry, one instance has no way to know that a given user is even connected, let alone which of N instances holds that connection

Stateless HTTP vs stateful WebSocket routing

Stateless HTTP vs stateful WebSocket routing
Property	Stateless HTTP request	WebSocket connection
Which server can handle it	Any instance behind the load balancer	Only the one instance holding the open socket
State needed to route	None — request carries everything	Registry lookup: user/connection ID → instance
Effect of instance restart	Next request just goes elsewhere	Every connection on that instance is dropped and must reconnect
Scaling model	Add instances, load balancer spreads requests	Add instances, but each holds a disjoint slice of live connections

Routing a push to a socket on a different instance

Client

server-7

Registry

server-3

1WebSocket handshake
2user:482 → server-7
3lookup user:482
4server-7
5relay via pub/sub
6write to live socket

Command

text

// Connection registry as a Redis hash, updated on connect/disconnect
HSET conn:registry user:482 instance:server-7
EXPIRE conn:registry:user:482 90   -- guards against stale entries

// On disconnect (graceful or crash-detected via heartbeat)
DEL conn:registry:user:482

A WebSocket is pinned to one server process (connection affinity) — a registry (typically Redis, with a short TTL refreshed by heartbeats) is the only way other instances learn where a given user's connection actually lives.

2.Pub/sub or brokers for cross-instance fan-out

Knowing which instance holds a connection (the registry) is only half the problem — something still has to actually get the message onto that instance. Pub/sub (e.g. Redis Pub/Sub) or a message broker is that relay: every server instance subscribes to a shared channel or topic, and when any instance needs to deliver a message to a user, it publishes the message once; every instance receives it, and the one instance that actually holds that user's socket writes it to the connection. This is also how fan-out to many recipients at once works — a single publish (e.g. "user X started typing") reaches every subscribed instance, each of which forwards it only to its own locally connected recipients.

Key features

Every server instance subscribes to the same shared channel(s); a publish from any one instance is delivered to all of them
Redis Pub/Sub is "fire-and-forget": at-most-once delivery, no persistence — a message published while a subscriber is disconnected is lost forever for that subscriber
A message broker (Kafka, RabbitMQ) trades Pub/Sub's simplicity for durability, replay and consumer groups — worth it when losing a fan-out message is unacceptable
Fan-out to many recipients (a chat room, a live feed) uses the same mechanism: publish once to a topic; every instance holding a subscribed connection forwards locally
Every instance receiving an irrelevant message and discarding it is the accepted cost of this design — it trades some wasted delivery for not needing every instance to know every other instance's connection map

Redis Pub/Sub vs a message broker for WebSocket fan-out

Redis Pub/Sub vs a message broker for WebSocket fan-out
Property	Redis Pub/Sub	Message broker (Kafka/RabbitMQ)
Delivery guarantee	At-most-once — lost if no subscriber is listening	Durable; can offer at-least-once and replay
Operational complexity	Low — reuses a Redis instance already in the stack	Higher — separate cluster, consumer group management
Fits	Ephemeral, real-time-only messages (typing indicators, presence)	Fan-out that must survive a consumer being briefly down
Message ordering	Per-channel, best-effort	Configurable, stronger guarantees per partition/queue

One publish, every instance receives it, only relevant ones deliver

publishreceivesreceives

server-7

publishes once

room:general

shared pub/sub channel

server-3

has a member — delivers

server-9

no members — discards

Command

text

// Every instance, on startup:
SUBSCRIBE ws:fanout

// Any instance, to deliver to one user:
PUBLISH ws:fanout '{"userId":482,"payload":{...}}'

// Each subscribing instance, on receipt:
if registry.ownsLocally(msg.userId):
    localSocket(msg.userId).send(msg.payload)
// else: message is silently discarded on this instance

Pub/sub or a broker is the actual relay that solves connection affinity: publish once, every instance receives it, only the instance holding the target connection acts on it. Redis Pub/Sub is fast but at-most-once with no persistence — use a durable broker when losing a fan-out message is unacceptable.

3.Connection lifecycle: reconnect, heartbeat/ping and authentication

A WebSocket connection is not "up" just because the handshake once succeeded — three mechanics keep it meaningfully alive and correct over time. Heartbeat/ping is a small periodic control frame (WebSocket defines Ping and Pong control frames, opcodes 0x9 and 0xA) each side sends so the other can detect a dead connection that TCP itself did not notice — network devices and proxies silently drop idle connections, and without a ping/pong cycle a "connected" socket can actually be dead for minutes before anyone finds out. Reconnect logic is the client-side behavior for when a connection does drop — normally with exponential backoff and jitter, so a mass disconnect (e.g. one server instance restarting) does not cause every affected client to reconnect in the same instant and overwhelm the fleet. Authentication on a WebSocket happens once, at connection time (since there is no per-message equivalent of an HTTP header), so the server must revalidate or expire that identity itself rather than trusting it forever.

Key features

WebSocket Ping and Pong are protocol-level control frames (RFC 6455, opcodes 0x9 and 0xA), not application messages — a missed Pong within a timeout window is the standard signal a connection is dead
Reconnect logic should use exponential backoff with jitter — a fixed retry interval causes synchronized "thundering herd" reconnects after any server-side disruption
Authentication happens once at connect time (often a token in the handshake URL or first message) — there is no per-frame equivalent of an HTTP Authorization header, so token expiry mid-connection must be handled explicitly (server-side forced disconnect, or a re-auth message)
On reconnect, the client typically needs a resume/replay strategy (a last-seen message ID or sequence number) since messages published while it was disconnected are not automatically redelivered
None of these three mechanics fix the core scaling mistake by themselves — they manage one connection's health, not how many connections a fleet of servers can coordinate

The three connection-lifecycle mechanics and what each one is actually for

The three connection-lifecycle mechanics and what each one is actually for
Mechanic	Problem it solves	Typical implementation
Heartbeat/ping	TCP alone does not reliably detect a silently-dead connection (idle timeouts, NAT/proxy drops)	Ping frame every N seconds; disconnect if no Pong within a timeout
Reconnect logic	A client must recover from a drop without hammering the server the instant it happens	Exponential backoff with random jitter, capped at a max interval
Authentication	No per-message header exists to re-prove identity on every frame	Token validated once at handshake; expiry handled by forced disconnect or explicit re-auth

A connection's lifecycle from handshake to reconnect

handshakesucceedsmissed pongclientretries

Connecting

start

Open

Detected dead

end

Reconnecting

end

Command

text

// Server-side heartbeat loop (per connection)
every 30s: send PING
if no PONG received within 10s: close connection

// Client-side reconnect with backoff + jitter
delay = min(max_delay, base_delay * 2 ** attempt)
delay += random(0, delay * 0.3)   // jitter
sleep(delay); reconnect(); attempt += 1

Heartbeat/ping (protocol-level control frames) detects a connection TCP thinks is fine but actually is not; reconnect logic needs exponential backoff with jitter to avoid a synchronized thundering herd; authentication happens once at connect time and must be actively re-checked, not assumed to hold forever. None of the three make local process memory a valid coordinator across nodes — that requires the registry and pub/sub layer instead.

Spotted a mistake on this page? Tell us.

Read Sheet