Consensus

Why distributed nodes need to agree, and how Raft/Paxos-style protocols make that agreement durable

Agreement under failuresone value/order despite crashes, lost messages
Leader + quorumleader proposes, majority must acknowledge
Split brainthe failure consensus exists to prevent
In practiceetcd/ZooKeeper power election, locks, config

1.Why nodes need agreement under failures

When the same data or decision is replicated across several nodes, those nodes must all end up agreeing on the same sequence of operations — otherwise they silently drift into different states while each one still believes it is correct. This is easy on a single machine (there is only one copy of the truth) and hard across a network, because nodes can crash, restart, or simply fail to hear a message in time, and the rest of the system cannot always tell the difference between "that node is dead" and "that node is just slow." Consensus is the general problem of getting a group of nodes to agree on one value or one ordered log of operations despite some of them crashing or messages being lost or delayed.

Key features

The core requirement is agreement on one value or one ordered sequence of operations, not just "everyone eventually has similar-looking data".
Failures come in two flavors that look identical from the outside: a crashed node and a slow/network-partitioned node — this is why timeouts, not certainty, drive most real decisions.
Without consensus, two nodes can each believe they are authoritative at the same time and accept conflicting writes — a split-brain scenario.
A system that just lets any node accept writes independently trades consensus for availability, and must resolve conflicting histories later (or accept inconsistency) instead of preventing them up front.
Consensus is provably impossible to guarantee in bounded time on a fully asynchronous network with even one faulty node (the FLP impossibility result) — real systems sidestep this with timeouts and randomization rather than a perfect guarantee.

No agreement rule → two leaders, same time

Node A

Node B

Node C

1network partitionA isolated from B and C
2still accepts writesbelieves it is still leader
3elect B (no quorum rule)B and C proceed without A
4also accepts writessplit brain — no way to tell whose writes win

Signature

agreement on ONE value/order, despite crashes + lost messages

Consensus is the general problem of getting nodes to agree on one value or ordered log despite crashes and lost/delayed messages — the danger without it is split brain, where two nodes both believe they are authoritative at once.

2.The high-level purpose of Raft/Paxos-style consensus

Raft and Paxos are algorithms that let a cluster of nodes agree on an ordered log of operations even while nodes crash or messages are lost. At a conceptual level (not the implementation detail) they share the same two ideas: one node is elected leader and is the only one allowed to propose the next entry in the log, and an entry only counts as committed once a majority (quorum) of nodes have stored it. Because a majority is required, any two majorities out of the same cluster must overlap by at least one node — so a committed decision can never be silently forgotten or contradicted by a later majority, even if some nodes are down. Raft was explicitly designed to be easier to understand and implement than Paxos while providing the same guarantees; Paxos came first and is more general but notoriously hard to reason about correctly.

Key features

Leader-based: one node is elected leader and is the sole node allowed to propose new entries for the replicated log during its term.
Majority quorum: an entry is only considered committed once it is stored by more than half the nodes — any two majorities from the same cluster must share at least one node.
That overlap guarantee is what prevents two conflicting decisions from both being "committed" — a future majority always includes at least one node that already knows the prior decision.
A cluster of N nodes tolerates the failure of up to floor((N-1)/2) nodes and keeps operating; e.g. a 5-node cluster keeps making progress with 2 nodes down, per Raft's own documentation.
Raft was designed specifically for understandability, splitting the problem into leader election, log replication, and safety as separate, more approachable subproblems — Paxos solves the same underlying problem but is widely regarded as harder to implement correctly.

Raft vs Paxos, conceptually

Raft vs Paxos, conceptually
Property	Raft	Paxos (classic)
Roles	Explicit leader, followers, candidates	Proposers, acceptors, learners (roles can overlap)
Design goal	Understandability, decomposed into clear subproblems	Generality and provable correctness first
Log model	Strong leader appends entries in order	Value-per-instance agreement, leader is an optimization (Multi-Paxos)
Common real-world use	etcd, Consul, CockroachDB	Chubby (Google), classic distributed databases

5-node cluster: majority (3/5) commits the entry

Majority — committed

Not yet responded

Signature

leader proposes → majority acks → committed (majorities always overlap)

Raft/Paxos, conceptually: one leader proposes the next log entry, a majority (quorum) must acknowledge before it counts as committed, and any two majorities out of the same cluster always overlap — so a committed decision can't be silently lost or contradicted. Raft optimizes for understandability over Paxos's generality.

3.Where consensus actually gets used in practice

Application code almost never implements Raft or Paxos itself — instead, teams run a small, dedicated cluster of a consensus-backed coordination service (etcd, ZooKeeper, or a cloud-managed equivalent) and build features like leader election, distributed locks, and shared configuration on top of it. The consensus algorithm's job is confined to keeping that small coordination cluster's own data consistent; everything else in the larger system talks to it over a normal client API rather than running consensus itself.

Key features

etcd and ZooKeeper are themselves small clusters (typically 3 or 5 nodes) that run a consensus protocol internally (Raft for etcd) to keep their own key-value store consistent.
Leader election in a larger application (covered in the coordination/leader-election material) is commonly built as a thin client pattern on top of one of these services, not implemented from scratch.
A distributed lock ("only one worker may process this job") is implemented as a key or lease in the coordination service that only one client can hold at a time, backed by the same consensus guarantee.
Centralized configuration ("what feature flags are on right now, what is the current primary database host") is stored in the coordination service so every service reads a single consistent answer instead of drifting local copies.
etcd's own documentation describes this as limiting consensus to one small replication group to get distributed consistency with low latency, rather than running consensus across an entire large fleet of application servers.

Three uses of the same consensus guarantee

Leader election

only one holds "current-leader"

Distributed lock

only one holds the job key

Config store

one consistent value, many readers

Signature

app services → client of a small etcd/ZooKeeper consensus cluster

Applications rarely run consensus themselves — they run a small etcd/ZooKeeper-style cluster that runs Raft/Paxos-family consensus internally, then build leader election, distributed locks, and shared config as client features on top of that one consistent source.

Spotted a mistake on this page? Tell us.

Read Sheet