beginnerintermediateadvanced

System Design · Section 49

Real-Time Systems

Level: intermediate
Read: 20 min
Concepts: 3

Polling, long polling, Server-Sent Events and WebSockets are the four standard ways a client gets near-real-time data, trading implementation simplicity and infrastructure compatibility for immediacy and bidirectionality. Choosing between them comes down to six concrete factors — directionality, connection count, message frequency, ordering, infrastructure support, and client support — run through in order rather than picked by preference. Once a transport is chosen, a long-lived connection has its own lifecycle (handshake, open, heartbeat, close, reconnect) and its own backpressure problem distinct from queue-depth backpressure: one slow client's own send buffer, which needs an explicit bound and policy rather than growing without limit.

This section

System Design Interview: Designing a Real Time Price FeedJustin

What is true here

Polling is simplest but wasteful; long polling lowers latency at the cost of tying up a connection per open request.
SSE gives one-directional server push with automatic reconnect; WebSockets give full-duplex communication at the highest infrastructure sensitivity.
Six factors — directionality, connection count, message frequency, ordering, infrastructure, client support — narrow the choice systematically.
A long-lived connection needs a heartbeat (TCP alone will not detect a silently dead peer) and a bounded per-connection buffer with an explicit backpressure policy.

What you will be able to do

Compare the four real-time transports on their actual trade-offs, not just familiarity
Run a real scenario through the six-factor framework to choose a transport
Design a heartbeat and a bounded per-connection buffer policy for a long-lived connection

The options, and how to choose between them

The four transports compared directly, and the six-factor framework for picking one.

Polling, long polling, SSE and WebSockets

coreintermediate

These are the four standard ways a client gets near-real-time data from a server. Polling repeats a plain request every few seconds regardless of whether anything changed. Long polling holds each request open until the server actually has something to say, then the client immediately reopens it. Server-Sent Events (SSE) is a single long-lived HTTP connection the server keeps pushing text events down, one direction only, with the browser reconnecting automatically if it drops. WebSockets upgrade one HTTP handshake into a persistent, full-duplex socket where either side can send a message at any time. Each option trades implementation simplicity and infrastructure compatibility for how immediate and how bidirectional the communication can be.

Think of it as

Think of four ways to find out if a friend has replied to your message. Polling is calling them every five minutes to ask "anything new?" — simple, but wasteful and never quite up to date. Long polling is calling once and asking them to stay on the line until they actually have something to tell you, then you hang up and immediately call back. SSE is like leaving a walkie-talkie on their desk that only they can talk into — they push updates to you whenever they want, but you can't talk back on it. A WebSocket is a real phone call left open — either of you can speak at any moment, and the line stays live until someone hangs up.

text

// SSE — server-to-client only, browser auto-reconnects
const es = new EventSource('/stream');
es.onmessage = (e) => console.log(e.data);

// WebSocket — full-duplex, single connection
const ws = new WebSocket('wss://example.com/ws');
ws.onmessage = (e) => console.log(e.data);
ws.send('client can push too');

What we're doing: Compare what actually happens on the wire for long polling vs SSE when a chat message arrives.

long-polling-vs-sse.txttext

Long polling:
1. Client -> GET /messages/poll        (request opens)
2. ...server waits, nothing new yet...
3. New message arrives server-side
4. Server -> 200 OK [new message]      (request closes)
5. Client -> GET /messages/poll        (client reopens immediately)
6. ...cycle repeats...

SSE:
1. Client -> GET /messages/stream      (connection opens once)
2. ...server waits, nothing new yet... (connection stays open)
3. New message arrives server-side
4. Server -> data: {new message}\n\n  (pushed on the SAME connection)
5. ...connection stays open, waiting for the next event...
6. New message arrives again
7. Server -> data: {another message}\n\n  (still the same connection)

4: Long polling closes the HTTP request after every single message — step 5 has to open a brand new one before the next message can arrive.
12: SSE never closes the connection between messages — step 7 reuses the exact same open connection from step 1, which is why SSE avoids the reconnect overhead long polling pays per message.

Why this works: The difference is not just "who talks first" — it's that long polling pays a full request/response round trip per message, while SSE pays that cost once per connection no matter how many messages flow afterward.

Reaching for WebSockets by default because they seem the most "modern" option

Wrong

text

// dashboard that only ever displays server
// metrics pushed down once every few seconds
const ws = new WebSocket('wss://api.example.com/metrics');
ws.onmessage = (e) => render(JSON.parse(e.data));
// client never calls ws.send() at all

Better

text

// same use case, server-to-client only:
const es = new EventSource('/metrics/stream');
es.onmessage = (e) => render(JSON.parse(e.data));
// plain HTTP, automatic reconnect for free,
// no custom heartbeat/reconnect code to write

What you see: A team ships a WebSocket server, plus custom reconnect and heartbeat logic, plus load-balancer configuration for sticky sessions and upgrade support — for a feed that only ever flows one direction and never needed the client to send anything back.

Why: WebSockets add real operational cost: connection-aware load balancing, manual reconnect/heartbeat logic, and infrastructure that must support the HTTP Upgrade handshake end-to-end. If the data only ever flows server-to-client, SSE gets the same low latency with a simpler, plain-HTTP transport and a browser-managed reconnect built in.

Four transports, increasing immediacy and cost

Polling

fixed interval, always wasteful

Long polling

held open until data arrives

SSE

server push, one direction

WebSockets

full-duplex, most infra cost

The four transports compared

The four transports compared
Transport	Direction	Latency	Connections held open	Infra fit
Polling	Client-initiated only	Up to one poll interval	None (short-lived requests)	Works everywhere — plain HTTP, any proxy/load balancer
Long polling	Client-initiated, server delays reply	Near-immediate once server has data	One per client, held until data or timeout	Works almost everywhere, but ties up server threads/workers per waiting client
SSE	Server → client only	Immediate (server pushes as events occur)	One long-lived HTTP connection per client	Plain HTTP/HTTPS — works through most proxies; needs HTTP/2 to escape the 6-connections-per-domain browser cap
WebSockets	Full-duplex, either side	Immediate, both directions	One persistent TCP connection per client	Needs upgrade support end-to-end — some older proxies/load balancers and corporate firewalls block or mishandle it

Remember: Polling: simplest, wastes requests, laggy. Long polling: lower latency, one request per message, ties up a server slot while waiting. SSE: server push over plain HTTP, one direction, auto-reconnect built in. WebSockets: full-duplex, needs upgrade support everywhere in the path, most infrastructure cost.

Six factors for choosing a real-time transport

coreintermediate

Picking between polling, long polling, SSE and WebSockets is not a matter of taste — six concrete factors narrow it down. Directionality asks whether the client ever needs to push data, not just receive it. Connection count asks how many simultaneous clients the server has to hold state for. Message frequency asks whether updates are rare (minutes apart) or constant (many per second). Ordering asks whether messages must arrive in the order they were sent. Infrastructure asks whether every hop between client and server — load balancers, proxies, corporate firewalls — actually supports the transport. Client support asks whether the client (browser, mobile app, IoT device, another backend service) has a usable library for it. Running a real scenario through all six, in order, usually eliminates all but one option.

Think of it as

This is like choosing a vehicle for a delivery route, not just picking the fastest one. A motorcycle (WebSocket) is fastest and can carry things both ways, but it can't use every road (infrastructure) and can't be driven by someone without a motorcycle license (client support). A cargo van (long polling) is slower to dispatch each time but works on every road. You would not send a motorcycle to deliver one package a day (message frequency too low to justify the overhead), and you would not use a bicycle courier who has to physically return and be re-dispatched for every single delivery (long polling) if you need continuous, instant, two-way radio contact with the driver instead (WebSockets).

What we're doing: Walk the six factors for a second scenario — a collaborative document editor — to show how a different answer to just one factor flips the conclusion.

collaborative-editor-scenario.txttext

1. Directionality: BOTH ways — every keystroke from
   any editor must reach every other editor. Rules out
   SSE and polling immediately (no client-to-server push).

2. Connection count: dozens of concurrent editors per
   document, not hundreds of thousands site-wide.
   A persistent per-client connection is affordable.

3. Message frequency: potentially many edits per second
   while someone is typing. Per-message overhead matters;
   a fresh HTTP request per keystroke would be too slow.

4. Ordering: edits MUST apply in a consistent order across
   all clients, or the document diverges.

5. Infrastructure: internal product behind infrastructure
   the team controls end-to-end — Upgrade handshake support
   is not a blocker here.

6. Client support: modern browsers only.

Conclusion: WebSockets — this is the one scenario in this
file where directionality alone already forces the answer,
before the other five factors are even considered.

1: Directionality is checked first because it is the strongest filter — needing client-to-server push eliminates SSE and polling in one step, before connection count or frequency are even weighed.
25: The conclusion calls out that one factor (directionality) was already decisive here, unlike the sports-score scenario where all six factors had to agree.

Why this works: Two scenarios that both plausibly sound "real-time" land on opposite transports — SSE for the score widget, WebSockets for the editor — because the factors are not independent votes, they are a funnel where an early hard requirement (needing two-way communication) can settle the choice by itself.

Weighing message frequency before directionality

Wrong

text

"Updates happen several times a second, so we
need WebSockets" — decided from frequency alone,
before checking whether the client ever sends
anything back.

Better

text

Check directionality first: if it's server-to-
client only, SSE handles several updates per
second over one open connection just fine —
frequency alone doesn't require a full-duplex
socket.

What you see: A team builds WebSocket infrastructure — connection-aware load balancing, custom reconnect logic — for a feed that never needed two-way communication, when SSE would have delivered the same update frequency with less operational surface.

Why: High frequency is often the reason people reach for WebSockets, but frequency alone does not require bidirectionality — SSE's single open connection delivers frequent server-to-client events with no per-message reconnect cost. Directionality, not frequency, is the factor that actually requires a full-duplex transport.

Same six-factor funnel, opposite conclusions

Sports-score widget → SSE

+Server pushes only; client never sends
+Hundreds of thousands of viewers
+A few updates per minute

Collaborative editor → WebSockets

−Every keystroke must reach every other editor
−Dozens of concurrent editors per document
−Many edits per second while typing

Worked scenario: a live sports-score widget on a public website

Worked scenario: a live sports-score widget on a public website
Factor	What the scenario needs	What it rules out
Directionality	Server pushes score updates; client never sends anything	WebSockets (unnecessary two-way channel)
Connection count	Potentially hundreds of thousands of simultaneous viewers	Anything that needs a server worker thread blocked per client
Message frequency	A handful of updates per minute (goals, score changes)	Nothing yet — low frequency fits several options
Ordering	Score updates must apply in the order they happened	Polling/long polling with overlapping requests, if a naive client fired more than one at once
Infrastructure	Public website behind a standard CDN/load balancer, plain HTTPS	Nothing — SSE and polling both run over plain HTTP
Client support	Modern browsers only, no legacy or embedded clients	Nothing — EventSource is broadly supported
Conclusion	SSE fits every factor: one-directional, cheap per connection relative to WebSockets, plain HTTP, ordered per connection, broad support	—

Remember: Six factors, roughly in filter strength: directionality (two-way instantly rules out SSE/polling), connection count (state cost per client), message frequency (per-message overhead vs setup cost), ordering (needed within one connection, not across separate requests), infrastructure (does Upgrade survive every hop), client support (does the target client have a usable library).

Running a live connection

The lifecycle a long-lived connection goes through, and the per-connection backpressure problem it introduces.

Connection lifecycle and backpressure

standardintermediate

A long-lived connection — long polling, SSE or a WebSocket — goes through a lifecycle a plain request/response never has to think about: connect (handshake), an open period that can last minutes or hours, detecting that the other side silently died (a dropped WiFi connection does not politely close the socket), and reconnecting without losing or duplicating messages. Backpressure, in this specific setting, is what happens when a server produces messages faster than one particular client's connection can absorb them — a slow mobile connection, a paused browser tab, a client stuck in a slow render loop. Without a plan for it, the server's per-connection send buffer grows without bound until that one slow client exhausts server memory or gets messages arbitrarily delayed.

Think of it as

A long-lived connection is like a phone call left open on a video conference, not a quick text message. The call needs a way to detect the other person's connection silently dropped (a heartbeat — "can you hear me?" — because the call software will not always tell you they disconnected). And if you're narrating a fast-moving video to someone on a slow connection, you can't just keep talking faster and faster hoping they catch up — eventually you have to decide: do I keep buffering everything I've said until they're caught up (unbounded buffer, risky), do I let them miss parts and just continue live (drop), or do I hang up and let them redial once their connection improves (disconnect)?

text

// WebSocket ping/pong heartbeat — server side (concept)
setInterval(() => {
  for (const socket of openSockets) {
    if (!socket.gotPongSinceLastPing) {
      socket.terminate();   // assume dead, free the slot
      continue;
    }
    socket.gotPongSinceLastPing = false;
    socket.ping();
  }
}, HEARTBEAT_INTERVAL_MS);

What we're doing: Show a bounded per-connection buffer with a drop-oldest policy protecting the server from one slow client.

bounded-send-buffer.txttext

const MAX_BUFFERED = 100;

function sendToClient(connection, message) {
  if (connection.bufferedAmount > MAX_BUFFERED) {
    // client isn't draining fast enough — drop the
    // oldest queued message instead of growing forever
    connection.pendingQueue.shift();
  }
  connection.pendingQueue.push(message);
  connection.flush();
}

// A slow client now loses its oldest unsent updates
// under sustained load, but the server's memory use
// per connection stays bounded no matter how far
// behind that one client falls.

4: The check happens before adding the new message — the server measures how far behind this specific connection already is, not global server load.
6: Dropping the oldest queued message (not the newest) keeps the client eventually current rather than permanently stuck replaying an ever-growing backlog.

Why this works: This is the concrete difference between per-connection backpressure and queue-depth backpressure: the decision is made per socket, based on that one client's drain rate, not on a shared queue's overall depth.

Buffering every message for every client with no cap, assuming clients will "catch up eventually"

Wrong

text

function sendToClient(connection, message) {
  connection.pendingQueue.push(message);
  connection.flush();
  // no size check — queue grows without limit
  // for any client slower than the message rate
}

Better

text

function sendToClient(connection, message) {
  if (connection.pendingQueue.length >= MAX_BUFFERED) {
    connection.pendingQueue.shift();
  }
  connection.pendingQueue.push(message);
  connection.flush();
}

What you see: Server memory climbs steadily under normal load, correlated with the number of slow or backgrounded client tabs rather than total traffic — a handful of clients on poor connections can consume more server memory than thousands of healthy ones, and the failure is invisible until it becomes an out-of-memory crash.

Why: Without a cap, a per-connection buffer's size is bounded only by how long the client stays connected and slow — a phone that goes into a pocket with the tab still open can accumulate an unbounded backlog for hours before the OS or browser eventually kills the connection.

A long-lived connection's lifecycle

Handshake

start

Open

Detected dead

end

Closed

end

Three ways to handle a slow client's connection filling up

Three ways to handle a slow client's connection filling up
Strategy	What happens	Good fit
Bounded buffer + drop oldest	Buffer caps at N messages; oldest is discarded to make room for new	Live dashboards, ticker feeds — staleness matters more than completeness
Disconnect and resync	Server closes the connection once the buffer is full; client reconnects and fetches current state fresh	State that can be fully resynced cheaply — e.g. "give me the current board," not a message log
Coalesce updates	Multiple pending updates to the same value collapse into one before sending	High-frequency updates to a small set of keys, e.g. a stock price or a live counter

Remember: Lifecycle: handshake, open, heartbeat (TCP alone won't tell you a peer silently vanished), close, reconnect. Backpressure here means one slow client's own buffer, not queue depth — bound it and pick one policy: drop-oldest, disconnect-and-resync, or coalesce.