Filter concepts by levelShowing all levels.

System Design · Section 108

Recommended Learning Order

Level
beginner
Read
13 min
Concepts
1

Twenty-nine steps in five phases, where each phase supplies something the next one needs. Phase one (steps 1–5) is saying what the system must do: requirements, non-functional requirements, capacity estimation, latency and throughput, availability and reliability. It is first because it produces the numbers every later choice gets checked against. Phase two (6–12) builds a working single-region request path: APIs, load balancing, databases, indexes, transactions and concurrency, replication, caching. Phase three (13–17) makes work asynchronous and survivable: queues, Kafka, idempotency, retries and timeouts, circuit breakers. Phase four (18–22) distributes state and is the hardest: consistency, distributed transactions, sagas and outbox, distributed locks, search and object storage. Phase five (23–29) is operating and communicating: security, observability, disaster recovery, cloud, high- and low-level design, case studies, interview practice. Four edges genuinely cannot be swapped, and each is the same shape — the earlier topic is the condition under which the later mechanism is correct. Capacity before caching, or a cache goes in with an 8% hit rate and makes p99 worse. Local transactions before distributed consistency. Idempotency before retries, or a timeout on a payment that actually succeeded gets charged twice by the retry that was added to improve reliability. And everything before case studies, because a case study tests what you know rather than teaching it. Everything else is a default: the order is for first exposure, not a single sweep to mastery, and when a real system is failing in front of you the right move is to jump to that topic, check its preconditions, and go deep where the failure is.

System Design overview

What is true here

  1. Five phases, each supplying the conditions under which the next phase's tools are correct.
  2. Four edges cannot be swapped: capacity→caching, transactions→consistency, idempotency→retries, everything→case studies.
  3. A mechanism learned without its precondition gets applied everywhere, including where it makes things worse.
  4. The order is for first exposure; depth comes from a topic failing on a system you own.
  5. When a real system is failing, jump to that topic, check what it assumes, and learn that instead.

What you will be able to do

  • Place any system-design topic in one of the five phases and say what it depends on
  • Explain what breaks when caching is learned before capacity, or retries before idempotency
  • Use the sequence for first exposure and a real system for depth, rather than expecting one pass to do both
  • Depart from the order deliberately when the system in front of you is teaching a later topic

The order

Twenty-nine steps as five phases, and the four edges that are not negotiable.

Twenty-nine steps, in five phases

corebeginner

Twenty-nine steps, and they group into five phases where each phase supplies something the next one needs. Phase one, steps 1 to 5, is saying what the system must do: requirements, non-functional requirements, capacity estimation, latency and throughput, availability and reliability. It comes first because it produces the numbers every later choice gets checked against — without them, caching is a habit rather than a decision. Phase two, steps 6 to 12, builds a working single-region request path: APIs, load balancing, databases, indexes, transactions and concurrency, replication, caching. At the end of it you can build a real system. Phase three, steps 13 to 17, makes work asynchronous and survivable: queues, Kafka, idempotency, retries and timeouts, circuit breakers. Phase four, steps 18 to 22, distributes state and is the hardest: consistency, distributed transactions, sagas and outbox, distributed locks, search and object storage. Phase five, steps 23 to 29, is operating and communicating: security, observability, disaster recovery, cloud, high- and low-level design, case studies, interview practice. Four edges in this order genuinely cannot be swapped — capacity before caching, local transactions before distributed consistency, idempotency before retries, and everything before case studies. The rest is a sensible default you can rearrange around whatever you are currently building.

Think of it as

Learning a language: sounds, then words, then sentences, then argument, then writing well. You can skip ahead and produce something that sounds right, and it collapses under the first real question — not because the later material is hard, but because it was resting on something that was never there.

text
Five phases, twenty-nine steps

  1. SAY WHAT IT MUST DO          01-05
     requirements · NFRs · capacity
     latency/throughput · availability

  2. BUILD THE REQUEST PATH       06-12
     APIs · load balancing · databases
     indexes · transactions · replication
     caching

  3. ASYNC AND SURVIVABLE         13-17
     queues · Kafka · idempotency
     retries/timeouts · circuit breakers

  4. DISTRIBUTE STATE             18-22
     consistency · distributed transactions
     sagas/outbox · locks · search/objects

  5. OPERATE AND COMMUNICATE      23-29
     security · observability · DR · cloud
     HLD/LLD · case studies · practice

Four edges are hard. The rest is a default.

What we're doing: See what actually goes wrong when two of the four hard edges are reversed.

reversed-edges.txttext
REVERSAL 1 — caching before capacity

  What it sounds like:
    "Reads are slow. Add Redis."

  What is missing:
    - How many reads per second?
    - What is the key distribution? Ten hot
      keys or a million cold ones?
    - What hit rate would make this worth
      the operational cost?
    - What is actually slow -- the query, or
      the number of queries?

  What happens:
    A cache goes in. Hit rate is 8%, because
    the access pattern has no hot set. The
    p99 gets worse, because every miss now
    pays a Redis round trip on top of the
    database. Nobody can say whether it
    helped, because nothing was measured
    first.
    The real problem was an N+1 query.

  With capacity first:
    "3,000 reads/sec, and 40 product ids
     are 70% of them. A cache on those 40
     removes ~2,100 reads/sec." Now it is
     a decision, with a number to check
     afterwards.

REVERSAL 2 — retries before idempotency

  What it sounds like:
    "Calls to the payment service fail
     sometimes. Add a retry."

  What happens:
    The call times out at 3 seconds. The
    provider took 3.4 seconds and succeeded.
    The retry charges the customer again.
    The retry was added to improve
    reliability and it created a
    double-charge that did not exist before.

  With idempotency first:
    The call already carries a key. The
    retry sends the same key, the provider
    returns the original charge, and the
    retry is now safe -- which is the only
    condition under which a retry is a
    resilience feature rather than a
    duplication feature.

WHAT BOTH HAVE IN COMMON

  The later topic is a mechanism, and the
  earlier one is the condition under which
  the mechanism is correct. Learn the
  mechanism first and you get a system that
  applies it everywhere, including where it
  makes things worse.
17
A cache with a low hit rate is strictly worse than no cache on the miss path, because every miss pays the lookup before the query. That is only visible if the key distribution was measured.
24
The N+1 detail is the common real ending: caching was applied to a symptom whose cause was a query-count problem the cache cannot fix.
39
A timeout is not a failure — it is an unknown outcome. Treating the two as the same is exactly what makes an un-keyed retry dangerous.

Why this works: Both reversals produce the same shape of damage: a mechanism applied without the condition that makes it correct. That is the argument for the order in general — each phase is largely the set of conditions under which the next phase's tools are the right ones.

Treating the order as a syllabus to complete once

Wrong

text
# Week 1-2:  steps 01-05, ticked
# Week 3-6:  steps 06-12, ticked
# ...
# Week 20:   step 29. Finished.
# Nothing was built. Every topic was met
# once, at reading depth, and none of it
# was tested against a real failure.

Better

text
# One pass for first exposure, shallow.
# Then build something that needs phases
# 1-3 (project 1 or 8), and revisit those
# topics with a system to break.
# Then a second pass, deeper, on the parts
# that failed.
# The order is for first exposure, not for
# a single sweep to mastery.

What you see: A complete-feeling coverage of every topic and no ability to answer a follow-up on any of them, because nothing in the sequence ever produced a failure to learn from.

Why: Depth comes from a topic failing on a system you own, not from reading it in the right sequence. The order exists to stop you meeting a mechanism before its precondition; it was never a claim that one pass gets you to mastery.

The five phases, and what each one unlocks
  1. Steps 01–05

    Say what it must do

    Requirements, NFRs, capacity estimation, latency and throughput, availability and reliability. Produces the numbers every later decision is checked against.

  2. Steps 06–12

    Build the request path

    APIs, load balancing, databases, indexes, transactions and concurrency, replication, caching. At the end of this phase you can build a correct single-region system.

  3. Steps 13–17

    Make it asynchronous and survivable

    Queues, Kafka, idempotency, retries and timeouts, circuit breakers. Work outlives the request, and a failing dependency stops being an outage.

  4. Steps 18–22

    Distribute state

    Consistency, distributed transactions, sagas and outbox, distributed locks, search and object storage. The hardest phase, and the one that needs phase two to be solid.

  5. Steps 23–29

    Operate and communicate

    Security, observability, disaster recovery, cloud, HLD/LLD, case studies, interview practice. The system can now be run, defended, explained and reviewed.

  1. Steps 01–05: Say what it must do — Requirements, NFRs, capacity estimation, latency and throughput, availability and reliability. Produces the numbers every later decision is checked against.
  2. Steps 06–12: Build the request path — APIs, load balancing, databases, indexes, transactions and concurrency, replication, caching. At the end of this phase you can build a correct single-region system.
  3. Steps 13–17: Make it asynchronous and survivable — Queues, Kafka, idempotency, retries and timeouts, circuit breakers. Work outlives the request, and a failing dependency stops being an outage.
  4. Steps 18–22: Distribute state — Consistency, distributed transactions, sagas and outbox, distributed locks, search and object storage. The hardest phase, and the one that needs phase two to be solid.
  5. Steps 23–29: Operate and communicate — Security, observability, disaster recovery, cloud, HLD/LLD, case studies, interview practice. The system can now be run, defended, explained and reviewed.

Five phases, and what each one makes possible

Five phases, and what each one makes possible
PhaseStepsTopicsWhat it makes possible
1. Say what it must do01–05Requirements · NFRs · capacity estimation · latency and throughput · availability and reliabilityEvery later choice has a number to be checked against
2. Build the request path06–12APIs · load balancing · databases · indexes · transactions and concurrency · replication · cachingYou can build a correct single-region system
3. Make it asynchronous and survivable13–17Queues · Kafka · idempotency · retries and timeouts · circuit breakersWork outlives a request, and a failing dependency stops being an outage
4. Distribute state18–22Consistency · distributed transactions · sagas and outbox · distributed locks · search and object storageState can live in more than one place and still be reasoned about
5. Operate and communicate23–29Security · observability · disaster recovery · cloud · HLD/LLD · case studies · interview practiceThe system can be run, defended, explained and reviewed

The four edges that cannot be swapped

The four edges that cannot be swapped
Must come firstBeforeWhat breaks if you reverse it
03 Capacity estimation12 CachingA cache is added by reflex with no hit-rate target, no size, and no idea what it is protecting
10 Transactions and concurrency18 ConsistencyDistributed consistency is being reasoned about by someone who cannot say what a local transaction guarantees
15 Idempotency16 Retries and timeoutsRetries are added to a system that duplicates on every one — resilience machinery generating the bug it hides
01–27 Everything28 Case studiesA case study is answered by recalling a diagram rather than by deriving it, which teaches the diagram and not the method

Remember: Five phases: say what it must do (01–05), build the request path (06–12), make it asynchronous and survivable (13–17), distribute state (18–22), operate and communicate (23–29). Each phase is largely the set of conditions under which the next phase's tools are correct, which is why four edges cannot be swapped — capacity before caching, local transactions before distributed consistency, idempotency before retries, and everything before case studies. The rest is a default: when a real system is failing in front of you, jump to that topic, check its preconditions, and go deep there instead.

See also: tier 1 master · url shortener · estimation categories · why caching works · non idempotent retry danger · the ten step sequence

Advertisement