Chat System Design

Level: advanced
Read: 15 min
Concepts: 1

A chat system is a durable, per-conversation sequenced log with a fast delivery channel attached, and the most common design error is building it the other way round. The transport is a WebSocket because chat needs server-initiated delivery and a request per message would carry more overhead than payload — but the socket is a channel, not a record: a message becomes durable when it is persisted and sequenced, so a dead connection costs a retry rather than a message. Ordering is per conversation using a server-assigned sequence number, because client clocks disagree and arrival order is not send order. What products show as one delivery signal is really three separate facts — the server accepted it, the recipient's device received it, the recipient read it — and each needs its own record, because inferring the second from a successful socket write produces ticks that lie: a socket write returns when the data enters a local buffer, which can happen well after the peer is unreachable. Presence is deliberately approximate, a short-TTL key refreshed by heartbeats rather than durable state, since storing it in the database makes the green dot the highest-write table in the system and leaves users shown online for hours after an unclean shutdown. Offline sync is a pull rather than a push: a reconnecting client sends the highest sequence number it holds per conversation and receives everything after it, which handles a two-second blip and a two-week absence through the same code path. Group fan-out routes through a pub/sub layer, because members' connections are spread across servers. And a push notification is the fallback when no live connection exists, driven off the same delivery state rather than a parallel path.

This section

WHATSAPP System Design: Chat Messaging Systems for InterviewsGaurav Sen

What is true here

Persist and sequence before delivering — a message is durable when it is written to the log, not when it is written to a socket.
Order per conversation with a server-assigned sequence number; client clocks and arrival order both disagree with send order.
Accepted, delivered and read are three separate acknowledgements; a successful socket write proves none of them.
Presence is a short-TTL heartbeat key, not durable state — otherwise the green dot becomes the busiest write in the system.
Offline sync is a position-based pull, so one mechanism repairs a brief blip and a long absence alike.

What you will be able to do

Order the send path correctly — persist, sequence, publish, deliver, acknowledge — and explain what each step protects
Design a reconnect that repairs any gap using only the client's last sequence number per conversation
Model the three delivery acknowledgements separately and explain why a socket write cannot stand in for the second
Choose an appropriate presence mechanism and justify why it is not stored durably

The whole design, as a log with a channel attached

Transport, persistence, ordering, acknowledgements, presence, offline sync, fan-out and push fallback.

Chat: transport, persistence, ordering, acknowledgements, presence, offline sync

coreadvanced

A chat system is a real-time delivery problem wrapped around a durable log, and the single most common design error is treating it as the reverse. The transport is a WebSocket, because chat needs server-initiated messages and a per-message HTTP request would carry more overhead than payload — but the connection is a delivery channel, never the record: a message is durable when it is written to storage, not when it is written to a socket. Ordering is per conversation, not global, so each message gets a sequence number assigned by the server within its conversation, and clients order by that rather than by their own clocks or by arrival. Delivery acknowledgements are three distinct facts people mistake for one — the server accepted it, the recipient's device received it, the recipient read it — and each needs its own record because each is visible in the product as a different tick. Presence is deliberately approximate: it is high-write, low-value data with a short TTL refreshed by heartbeats, and storing it durably is a common way to make a chat system expensive for no product gain. Offline sync is what makes the whole thing usable, and it is a pull, not a push: a reconnecting client sends the last sequence number it holds per conversation and receives everything after it, which is also exactly what a client that missed messages during a network blip needs. Fan-out to a group means resolving members to their live connections, which live on different servers, so a pub/sub layer routes each message to the servers holding those connections. And a push notification is the fallback for a recipient with no live connection — driven by the same delivery state, not by a separate code path.

Think of it as

Think of the conversation as an append-only log that happens to have a fast notification channel attached. The log is the truth: it has an order, it survives disconnection, and any client can catch up by asking for everything after position N. The WebSocket is a courtesy that saves clients from polling the log. When you are unsure how some feature should behave — a missed message, a reconnect, a device that was offline for a week, a group of 500 people — ask what the log says and how the client catches up to it. Almost every chat feature has a clean answer in those terms and a messy one in terms of sockets.

javascript

// send path: persist, sequence, then deliver
async function onMessage(conversationId, body, senderId) {
  const seq = await nextSequence(conversationId);
  await messages.insert({ conversationId, seq, body, senderId });
  await pubsub.publish(`conv:${conversationId}`, { seq, body });
  return { status: 'accepted', seq };   // ack #1
}

// reconnect path: catch up by position, not by time
socket.on('open', () => {
  socket.send({ type: 'sync', cursors: lastSeqPerConversation });
});

What we're doing: Follow one recipient through a disconnection and see how the log, not the socket, repairs it.

offline-sync-trace.txttext

Conversation 9812, recipient device holds seq 4468

10:00  socket drops (train tunnel). No error is
       visible to the sender; delivery simply
       stops.

10:00  msg seq 4469 persisted, published.
       No live connection for the recipient ->
       delivery state stays 'accepted', and a
       push notification is queued.
10:02  msg seq 4470 persisted, same.
10:05  msg seq 4471 persisted, same.

10:07  socket reconnects. Client sends:
         sync { 9812: 4468 }
       Server returns 4469, 4470, 4471 in order.
       Client marks delivered(4469..4471).

Nothing was lost, nothing was duplicated, and
the recovery needed no knowledge of what
happened during the gap -- only a position.

7: Because the message was persisted before delivery was attempted, a dead socket costs nothing. Had the socket write been the only record, these three messages would exist only in the sender's client.
15: The sync request carries a position, not a timestamp. A timestamp-based catch-up has to choose between re-sending messages the client already has and missing ones written slightly out of clock order — a sequence number has neither problem.
19: The same request handles a two-second blip and a two-week absence. Designing catch-up as a position-based pull means there is no separate "long offline" code path to get wrong.

Why this works: Every hard part of chat — reconnection, multiple devices, ordering, read receipts — reduces to "what is the client's position in this log, and what comes after it". A design that treats the socket as the delivery record has to invent a separate mechanism for each of those cases; a design built on a sequenced, persisted log answers all of them with one request.

Treating a successful socket write as delivery

Wrong

javascript

socket.send(JSON.stringify(message));
markDelivered(message.id);   // the write returned;
                             // that says nothing
                             // about the client

Better

javascript

socket.send(JSON.stringify(message));
// stays 'accepted' until the recipient's client
// sends back an explicit acknowledgement:
socket.on('ack', ({ seq }) => markDelivered(seq));

What you see: Messages show a delivered tick that the recipient never received, most often when their connection dropped moments earlier — the TCP buffer accepted the write, the socket had not yet noticed it was dead, and nothing ever corrected the state.

Why: A socket write succeeds when the data enters the local send buffer, which can happen well after the peer is unreachable. Delivery is a statement about the recipient, so only the recipient can make it — which is why the acknowledgement travels back from the device rather than being inferred at the sender.

One message: persist, fan out, acknowledge, fall back to push

Sender

Chat server A

Message store

Pub/sub

Chat server B

Recipient

1. send(conversation, body)
2. assign seq, persistdurable here, not before
3. accepted (seq 4471)
4. publish to conv:9812
5. route to servers holding member connections
6. deliver over live socket
7. delivered(4471), later read(4471)
8. no live socket → push notification instead

Three acknowledgements that are usually collapsed into one

Three acknowledgements that are usually collapsed into one
Acknowledgement	Sent by	Means	Product signal
Accepted	The server, to the sender	Persisted and sequenced	The message stopped spinning
Delivered	The recipient's device	Received over a live connection	Second tick
Read	The recipient's client	Displayed to the user	Blue tick / read receipt

Each requirement and where it is handled

Each requirement and where it is handled
Requirement	Mechanism	Failure it prevents
Transport	WebSocket, with reconnect and backoff	Polling overhead; missing server-initiated delivery
Persistence	Write to the conversation log before delivering	A message lost when a socket dies mid-send
Ordering	Server-assigned per-conversation sequence number	Messages displayed in the wrong order across devices
Acknowledgements	Three separate delivery-state records	Ticks that lie about whether anyone received anything
Presence	Short-TTL key refreshed by heartbeat	Users shown online days after they closed the app
Offline sync	Pull everything after the client's last sequence number	Gaps after a reconnect that nothing ever repairs
Fan-out	Pub/sub routing to the servers holding member connections	Delivery only to members on the same server as the sender
Notification	Push when no live connection exists	Silence for anyone whose app is closed

Remember: Chat is a durable, per-conversation sequenced log with a fast delivery channel attached — not a socket with storage bolted on. Persist and sequence before delivering; order by server-assigned sequence number, never by client clocks; keep accepted, delivered and read as three separate acknowledgements; make presence a short-TTL heartbeat key rather than durable state; sync offline clients by pulling everything after their last sequence number; fan out to groups through pub/sub because connections live on different servers; and fall back to push when no live connection exists.