System Design quick reference

requirement → architecture + data flow + APIs + storage + failure handling

The five things a system design must make explicit that a plain-English requirement leaves implicit.

functional / non-functional / constraint / assumption / capacity / latency / ...

The twelve terms every later system-design section assumes you already know.

"10M req/day, p95 < 300ms, 99.95% uptime" -> capacity + 2 non-functional requirements

functional (what) + non-functional (how well, with a number)

Every feature needs both — the second is what makes the first testable.

"users can upload files" + "p99 < 2s" = one testable requirement

architecture → detailed design → implementation → operations

Four altitudes of the same system, coarsest to finest — keep design conversations at one altitude at a time.

feature / workflow / business rule / user action / data produced / external integration

The six categories a functional requirement falls into.

latency / throughput / availability / durability / consistency / scalability / security / observability / recoverability / compliance / cost

The eleven non-functional categories, grouped by what they measure.

"fast" → p95/p99 ms · "reliable" → % uptime · "scalable" → peak RPS by when

The standard question that turns a vague requirement word into a measurable target.

10M req/day · p95 < 300ms · 99.95% availability · RPO 5min · RTO 30min

A realistic, fully measurable requirements block, and what each figure implies for design.

users · clients · use cases · read/write · volume · retention · region · traffic · peak · SLA · security · regulatory

The twelve questions to answer before drawing a single component.

read-heavy / write-heavy / bursty / interactive / batch-oriented / real-time / event-driven

The seven traffic shapes, each pointing toward a different architecture — a system is usually several at once.

strong: a stale read breaks something · eventual: a stale read is barely noticeable

The test for whether one piece of data needs strong or eventual consistency.

account balance -> strong · like count -> eventual

sync: the caller needs the result · async: the caller only needs "accepted"

The test for whether one workflow step must complete inline or can move off the request path.

charge payment -> sync · send confirmation email -> async

RPS · storage growth · bandwidth · memory · queue volume

The five numbers a capacity estimate needs before any component is chosen.

avg RPS = req/day ÷ 86,400 · peak RPS = avg × factor · storage = records × size × multiplier

The three formulas behind most capacity estimates.

DB reads/sec = reads/sec × (1 − cache hit ratio) · growth = writes × size × replication

Second-layer capacity math: read/write split, cache hit ratio, and replicated storage growth.

quick avg RPS ≈ requests/day ÷ 10^5 (86,400 seconds/day rounds up to ~10^5)

Fast mental-math shortcut for a daily-total-to-RPS estimate, refined afterward with ÷86,400.

server count = ceil(peak RPS ÷ RPS-per-server × (1 + headroom))

Turning a capacity estimate into an actual infrastructure count, not a guessed server number.

bandwidth · storage · memory · cache size · queue depth · CPU needs · database IOPS

The seven quantities a back-of-the-envelope pass covers, fast and rounded.

1 byte = 8 bits · 1 KB/MB/GB/TB step = ×1,024 · 1 day = 86,400 sec

The bits-vs-bytes and unit-ladder conversions every capacity estimate depends on.

peak = avg × factor (once) · growth: size(N) = today × (1 + rate)^N (compounds)

A same-day peak multiplier vs a compounding growth rate over time — different math, both needed.

the widest-range assumption dominates the result — state a range, not a bare number

Finding which guessed input actually swings a back-of-the-envelope estimate.

latency = time/operation · throughput = operations/time

The two axes every performance conversation needs to separate.

p50 · p90 · p95 · p99 · p99.9

The five latency percentiles worth naming, from median to deep tail.

mean = sum(latencies) / count — dilutes outliers; use p99/p99.9

A mean blends a few very slow requests into a much larger fast group; percentiles report an actual value instead.

concurrency ≈ throughput × latency

Little's Law — connects concurrency, throughput and latency.

serial: sum(latencies) · parallel: max(latencies)

Independent steps only cost the slowest one when run in parallel — identify real dependencies, not code order.

availability = reachable · reliability = correct · durability = data preserved

The three distinct guarantees a system can be measured on.

99% ≈ 3.65d/yr · 99.9% ≈ 8.76h/yr · 99.99% ≈ 52.6min/yr · 99.999% ≈ 5.26min/yr

Allowed downtime per year at each availability "nine".

redundancy + health checks + failover; backups + replication; DR

Seven techniques, four guarantees — redundancy needs health checks and failover to matter; backups and replication protect data differently.

availability(system) ≈ availability(least redundant component)

Walk the full request path and name every SPOF explicitly before claiming high availability.

SLI = measured · SLO = internal target · SLA = external contract

The three layers of a service level commitment.

success rate · p95 latency · queue age · data freshness

Four common SLIs, each catching a failure mode the others miss — a service usually needs more than one.

error budget = (1 − SLO) × time window

The allowed room for failure under an SLO, and what spending it triggers.

client → application → database

The most common starting architecture for a web system.

presentation → application → domain → data access → storage

Each layer only calls the layer directly below it; domain logic stays free of I/O.

one deployable + enforced module boundaries + module-owned data

A single deployable with real internal boundaries, distinct from a plain monolith or microservices.

independent services + own data + network communication

Multiple independently deployable services with explicit network boundaries.

producer --publish--> broker --route--> consumer(s)

Producers publish events without knowing or waiting on consumers; new consumers subscribe independently.

ingestion → processing → storage/output

Each pipeline stage has one job and feeds the next stage's input format.

every pattern buys a property and costs a price — name both

No architecture pattern is universally superior; choose based on trade-offs, not reputation.

simpler deploy · local calls · easier transactions · lower ops overhead

The four advantages a plain monolith has over a distributed architecture.

modular monolith = real boundaries + no network cost

Enforced module boundaries and data ownership, still one deployable — the coupling fix without the microservices network bill.

independent scaling · independent deployment · independent ownership

The three advantages microservices provide over any monolith shape.

network calls · distributed transactions · observability · ops overhead · eventual consistency · new failure modes

The six costs microservices pay for their independence.

split only for a named, specific pain — not speculatively

The discipline check before splitting any component into its own microservice.

client → load balancer → API gateway → backend service(s)

The layers a request passes through before business logic runs.

gateway: routing + auth + rate limit + aggregation + observability, never business logic

What an API gateway should and should not own.

gateway: routing + cross-cutting concerns · service: business rules — never swap the two

The one thing an API gateway must never own.

L4 = IP/port only · L7 = full HTTP request

The two layers a load balancer can operate at.

round robin · weighted round robin · least connections · hashing · consistent hashing

The five algorithms a load balancer can pick an instance with.

health check → pool membership · drain → graceful retirement · sticky session → locality only

The operational mechanics a load balancer needs beyond an algorithm.

stateless instance → any instance serves any request → horizontal scaling is simple

Why statelessness is the precondition for easy horizontal scaling.

vertical = bigger machine, hard ceiling · horizontal = more machines, needs distributable workload

The two directions a system can scale in.

vertical: simple ops, non-linear cost, hard ceiling · horizontal: complex ops, ~linear cost, no ceiling

The real trade-off behind choosing a scaling direction.

check statelessness first — a stateful component needs relocation, not just more instances

The design-time check that decides whether horizontal scaling will actually help.

disposable instance = no unique state + safe to kill and replace anytime

What makes an instance safe to auto-scale, redeploy, or recover automatically.

shared state → database | cache | object storage | queue, never one instance alone

Where state that must outlive a single request or instance actually belongs.

shared store: lookup + instant revoke · signed token: no lookup, no easy revoke

The two dominant approaches to sessions across stateless instances.

relational: consistency + joins, harder scale-out · non-relational: scale-out + flexible schema, weaker joins

The system-design-level trade-off between a relational and a non-relational data model.

each store in the mix needs a stated, specific reason — not habit or reputation

Polyglot persistence done deliberately versus accumulated by accident.

one DB → ACID transaction · multiple services → needs an explicit saga/outbox strategy

Recognizing when a logical transaction crosses a service boundary a single database cannot cover.

object model = what exists · access pattern = how it is read/written — design for the second

Why the dominant query pattern, not the object model alone, decides a schema.

auto-increment: 1 authority, sortable · UUID: no coordination, not sortable · ULID: both

The three dominant ID strategies and what each trades away.

soft delete · audit history · temporal data · status + transitions · versioning

Five recurring patterns for handling change and deletion in a data model.

Read Uncommitted < Read Committed < Repeatable Read < Serializable

The four standard SQL isolation levels, from weakest to strongest.

dirty read · non-repeatable read · phantom read · lost update

The four named concurrency anomalies isolation levels defend against.

pessimistic = lock first · optimistic = verify at write time

The two families of strategy for handling concurrent writes to the same data.

version column · compare-and-set · unique constraint · row lock

The four concrete tools that implement optimistic/pessimistic concurrency control.

sync = wait for ack, no loss, slower · async = confirm now, faster, can lose recent writes

Primary/replica architecture and the sync-vs-async replication trade-off.

replicas scale reads, not writes · every replica read trades freshness for capacity

What routing reads to replicas gains and what it can silently cost in consistency.

lag = write volume vs replica replay throughput, queued

Why a replica can fall behind the primary, and why that gap is not fixed.

partitioning = within one instance · sharding = across many instances

The scope difference between partitioning and sharding, and why only sharding adds capacity.

good shard key = even distribution + matches the dominant query pattern

What makes a shard key choice good or bad, and the sequential-key hotspot trap.

sharding costs: hot shards, cross-shard queries, expensive resharding

The recurring operational costs sharding introduces in exchange for added capacity.

caching pays off when: expensive to produce + read >> written + skewed access

Why caching works — temporal/spatial locality and avoiding repeated expensive work.

cache-aside · read-through · write-through · write-back

The four cache patterns, differing in who loads on a miss and when a write reaches the store.

TTL = time-based · eviction = space-based (e.g. LRU) · invalidation = write-triggered

The three ways an entry leaves a cache, and why invalidation alone is risky without a TTL backstop.

warming = pre-load before traffic · negative caching = cache the "not found" too

Two specialized cache techniques beyond TTL/eviction/invalidation.

Redis fits: caching, counters, sessions, rate limiting, lightweight locks, simple queues

What Redis is used for beyond caching, and where its speed comes from being in-memory.

string · hash · list · set · sorted set — plus pipelines, Pub/Sub, Streams

Redis's core data structures and the mechanisms for atomicity, batching and messaging.

stampede = synchronized misses · hot key = one key overloads one node · pressure = watch evictions

Three distinct ways a distributed cache breaks down under real load.

CDN = edge cache near users → cuts latency + origin load, for explicitly cacheable content

What a CDN caches and why it reduces both latency and origin load.

max-age/s-maxage set TTL · purge/versioning forces refresh · signed URLs restrict access

The Cache-Control directives, invalidation mechanisms and signed URLs a CDN relies on.

safe to cache = identical for every requester, with no embedded personalized fields

How to tell cacheable, shared content apart from user-specific content that must not be cached publicly.

sync = wait for a result you need · async = hand off work you don't need to wait for

The core choice between blocking on a result and decoupling from how long work takes.

producer → broker (partition/offset) → consumer → ack / retry / DLQ

The nine core terms describing how a message moves from send to successful processing.

queues: decouple producer/consumer, absorb bursts, move slow work off the request path

The three problems a queue solves between two components, and the capacity trade-off it doesn't remove.

topic = partitions (ordered logs) · consumer group = one consumer per partition at a time

Kafka's core vocabulary and the scope of its ordering guarantee.

at-most-once: can lose · at-least-once: can duplicate · exactly-once: Kafka-internal only

The three delivery guarantees and the real boundary of Kafka's exactly-once semantics.

consumption does not delete — a topic is a durable, replayable log

Why Kafka's retention model enables replay and event-driven architecture that a transient queue cannot.

partition count = parallelism ceiling · partition key = ordering guarantee

Why partitioning is a scaling and ordering decision, not just a storage detail.

producer → exchange (direct/fanout/topic) → binding → queue → consumer

RabbitMQ's core AMQP routing model — always through an exchange, never directly to a queue.

manual ack (safe) vs auto-ack (fast, risky) · nack → requeue or dead-letter exchange

How RabbitMQ tracks delivery success and recovers from repeated processing failure.

task queue: one worker, no replay, simpler · event log: many consumers, replay, more complex

When a task-queue broker is the simpler, sufficient choice over an event log.

at-most-once: may lose · at-least-once: may duplicate · exactly-once: needs idempotency, not just delivery

The general delivery-semantics trade-off and why exactly-once is fundamentally hard across a network boundary.

idempotent = same end result no matter how many times applied — needs an atomic check, not just a unique ID

What makes a consumer safe under at-least-once delivery's expected duplicates.

idempotency key · unique constraint · dedup table · state check · outbox/inbox

The five concrete mechanisms that implement idempotent processing, and what each fits best.

retry storm = many clients retrying together against an already-overloaded service

Why uncontrolled retries can turn a brief failure into a sustained outage.

wait = random(0, min(cap, base * 2^attempt))

Exponential backoff with full jitter — the standard formula for spacing out retries.

transient (503, 429, timeout) -> retry; permanent (400, 401/403, 404) -> fail fast

Which errors are worth retrying, and which never will succeed no matter how many attempts.

attempt >= max_attempts -> dead_letter_queue.send(message) + alert

Bound retries with a max attempt count; route exhausted messages to a DLQ instead of dropping them.

GET/PUT/DELETE -> safe to retry; POST/relative updates -> only retry if idempotency-guarded

When a retry can safely repeat a request versus when it risks duplicating a side effect.

connect timeout · read timeout · request (total) timeout — three distinct deadlines

The three deadlines a network call needs, and the specific failure mode each one alone catches.

queue depth · body/upload size · concurrency · DB connections · memory — six limits, six explicit caps

The resource-limit checklist beyond timeouts — every one of these defaults to unbounded unless explicitly capped.

sequential timeouts sum · parallel timeouts take the max · always fit under the caller's own deadline

Why downstream timeouts must be derived from the caller's remaining budget, not chosen independently.

circuit breaker: fail fast on a known-failing dependency instead of failing slow, repeatedly

Why circuit breakers exist — protecting both the caller's own resources and a struggling dependency's ability to recover.

Closed (normal) → Open (fail fast) → Half-Open (trial) → Closed or back to Open

The three-state machine every circuit breaker implementation follows.

bulkhead: dedicated resource slice per dependency — isolation, not maximum efficiency

Partitioning a shared resource so one failing dependency cannot consume capacity that belongs to another.

bulkhead (resource) + circuit breaker (attempt at all?) + bounded retry (how many) + timeout (how long each)

Four resilience techniques that answer different questions and layer together around the same call.

disagreement is temporary and structural — propagation takes real time, but the system provably converges

Why replicas, queues and caches legitimately disagree at any given moment, and what "eventually" actually means.

ask: what is the real cost if THIS read is 5 seconds stale? — decide per workflow, not system-wide

How to choose between eventual and strong consistency for a specific workflow, not the whole system.

model async completion as an explicit status (incl. failure states), not an inferred timing guess

Why a genuinely asynchronous workflow needs a real, visible status field rather than a hidden or assumed delay.

each database is only ACID with itself — no shared commit boundary exists across services by default

The structural reason cross-service atomicity is hard, not a solvable gap in any single database's design.

2PC: prepare/vote → commit or abort, all-or-nothing — real atomicity, real blocking cost

The two-phase commit protocol and the specific coordinator-failure scenario that makes it fragile.

saga: local transactions commit independently + explicit compensating actions on failure — no cross-service locks

Why sagas are generally preferred over 2PC for cross-service workflows, and the real cost (visible intermediate states, non-symmetric compensation) that trade-off carries.

choreography: services react to each other's published events — no central coordinator

Decentralized saga coordination where the workflow emerges from a chain of event reactions.

orchestration: one coordinator explicitly directs every step and every failure decision

Centralized saga coordination — easier to trace and debug, at the cost of a more tightly-coupled, single point of workflow control.

compensating action per step + idempotency + bounded retries + monitoring for stuck sagas

What makes a saga's compensation logic actually reliable in production, not just correct in the happy path.

outbox row: event_id, aggregate type/id, event type, payload, created_at, published_at — transient, not permanent

What an outbox table needs to store, and why it needs its own cleanup policy unlike a general event log.

poll outbox_events WHERE published_at IS NULL | or tail the DB's replication log (CDC)

How a separate relay process actually publishes outbox rows: polling vs CDC, marking published, cleanup.

INSERT event_id INTO inbox (unique) → then apply effect, same transaction

The inbox pattern: recording processed event IDs atomically with the effect they cause, so redelivery is a safe no-op.

distributed lock needed ⇔ invariant spans systems no single DB transaction covers

The test for whether a problem genuinely needs distributed coordination.

lease TTL bounds crash recovery; fencing tokens (not TTL alone) prevent overlap

The four risk categories in lease-based distributed locks, and why fencing tokens are the real fix.

UPDATE ... WHERE <condition> RETURNING ... — atomic check-and-act in one round trip

Prefer a unique constraint or atomic UPDATE over a distributed lock whenever the invariant fits in one database operation.

owner(key) = next node clockwise from hash(key) on the ring

The core consistent-hashing lookup: hash both nodes and keys into one circular space.

ring[hash(node_id + "-" + i)] = node, for i in 0..virtual_nodes_per_node

Hashing each physical node to many ring positions to even out load.

keys moved on membership change ~= K / N (vs. ~K for hash(key) % N)

The key-movement bound that makes ring-based consistent hashing worth using over modulo hashing.

strong > causal > read-your-writes > eventual — each relaxes a different guarantee for performance/availability

The four named consistency models and their relative strength.

stronger consistency = more coordination = higher latency, lower availability under failure — a real trade-off, not a flaw

Why distributed stores make genuinely different consistency choices, and what each one costs.

ask "what does THIS operation do during an actual partition?" — not "what's our one-word CAP label?"

Using CAP as a concrete design-reasoning tool rather than a shallow system-wide label.

partition + conflicting operation -> choose Consistency or Availability

The precise moment the CAP theorem describes, and the two options at that moment.

"pick 2 of 3" (myth) vs. "choose C or A, only during a partition" (accurate)

The corrected reading of CAP: a temporary, partition-scoped trade-off, not a permanent global choice.

CP: minority partition rejects writes · AP: minority partition accepts writes, reconciles later

How a Raft-based CP system (etcd) and a Dynamo-style AP system each resolve the same partition, and the concrete consequence of each.

if Partition: A or C; else: L or C

PACELC extends CAP theorem by naming the latency/consistency trade-off that exists even without a partition.

w:1 + nearest read (fast) vs w:"majority" + primary read (consistent)

A concrete, partition-free trade between read/write latency and read-your-writes consistency.

Ask "PA or PC?" then "EL or EC?" — independently, per operation

A two-question checklist for classifying any replicated system's PACELC behavior.

lease = "leader until time T" — expires unless renewed before T

How a leader proves it is still alive and in charge: heartbeats signal liveness, leases bound it with an expiry.

split-brain: two nodes, one partition, both believing "I am leader"

Why failover after a network partition (not a crash) can produce two simultaneous leaders, and what actually prevents it.

ZooKeeper (ZAB) / etcd (Raft) — a consensus-backed store for "who is the leader"

What coordination systems solve conceptually: a consistent, fault-tolerant place to store shared state like leadership, so applications don't implement split-brain-safe election themselves.

agreement on ONE value/order, despite crashes + lost messages

The general distributed-systems problem that consensus protocols solve.

leader proposes → majority acks → committed (majorities always overlap)

The conceptual core shared by Raft and Paxos-style consensus algorithms.

app services → client of a small etcd/ZooKeeper consensus cluster

How Raft/Paxos-style consensus shows up in real systems: leader election, distributed locks, config stores.

bucket → key (object: data + metadata), versioned by write/delete

The core S3-style object storage model — a flat namespace of immutable, versioned objects addressed by key, with lifecycle rules automating storage-class transitions and expiration.

CreateMultipartUpload → UploadPart × N → CompleteMultipartUpload | presign(key, expiresIn)

Multipart upload splits a large object into independently-retriable parts (recommended at 100 MB+); a signed URL grants one caller's time-limited access to a single object action without sharing credentials.

client --(signed URL)--> object storage, app server records metadata only

Route large durable files directly to object storage instead of relaying them through an application server, keeping the app server's job to metadata only.

LIKE '%term%' → full scan, no ranking — B-trees can't do relevance

A leading-wildcard LIKE defeats a B-tree index; reach for a search engine once ranking, fuzzy matching, or faceting outgrows built-in full-text search.

term -> [document IDs] (built via tokenize -> analyze -> index; queried via tokenize -> analyze -> lookup -> filter -> rank)

The inverted index and the analysis pipeline that builds and queries it.

primary store (authoritative) → indexing pipeline → search engine (derived)

Elastic/OpenSearch are specialized for ranked, faceted search, not ACID transactions or being the source of truth.

fixed window · sliding window · token bucket · leaky bucket

The four standard algorithms for deciding whether to allow or reject a request against a rate limit.

in-process counter = broken at N>1 instances · fix: shared state or edge-level enforcement

Why a rate limit needs a single source of truth across every instance handling traffic, not a per-instance counter.

limit = (key, count, window) — key ∈ {user, IP, API key, endpoint, or a combination}

Choosing what a rate limit is counted per, matched to the specific abuse scenario it needs to catch.

quota: total ≤ limit per period · burst: rate ≤ limit per second

A hard quota bounds total usage over time; a burst limit bounds instantaneous rate — distinct controls, often applied together.

one tenant heavy → other tenants (flat traffic) degrade too

The diagnostic signature and fix for one tenant monopolizing a resource shared across tenants.

weighted fairness · per-tenant queues · resource isolation

The three concrete mechanisms for sharing a resource across tenants without one tenant starving the others.

Polling → Long polling → SSE → WebSockets

The four real-time transports, ordered by increasing immediacy and infrastructure cost.

directionality → connection count → frequency → ordering → infrastructure → client support

The six-factor funnel for choosing between polling, long polling, SSE and WebSockets.

connect → open → heartbeat → close → reconnect; bounded buffer + drop/disconnect/coalesce

The stages a long-lived connection moves through, and the three standard responses to one client falling behind.

registry: user/connection ID → holding instance

How WebSocket connection affinity forces a shared registry so other instances can find where a connection lives.

PUBLISH channel msg → every subscribed instance → local-only delivery

Redis Pub/Sub or a broker relays a message to every instance so the one holding the target connection can deliver it.

ping/pong (liveness) + backoff+jitter (reconnect) + connect-time auth (identity)

The three mechanics that keep one WebSocket connection healthy and correctly identified over its lifetime.

batch: bounded input, scheduled, exits | stream: unbounded input, continuous, never exits

The two processing paradigms and the freshness-vs-simplicity trade-off between them.

window (bounds the stream) → state (running aggregate) → checkpoint (state + offset saved) → replay (resume from there)

The vocabulary a stream processor needs because its input has no end and a crash cannot mean starting over.

throughput ↑ + latency ↑ together via batching | backpressure = input rate > processing ceiling

The throughput/latency trade-off a stream processor tunes, and backpressure as the failure mode when input outruns it.

producer rate > consumer rate, unbounded → memory growth → OOM

The producer/consumer rate mismatch backpressure exists to control, and what happens without it.

bounded queue + rate limit + flow control + batch size + consumer scaling

The five concrete mechanisms that apply backpressure across a pipeline, usually combined rather than used alone.

queue length (volume) vs. queue age / consumer lag (health)

Why raw queue length is a weaker backpressure signal than the oldest item's age or the consumer's offset lag.

peak memory: load fully = O(file size), stream = O(chunk size)

Streaming processes a file in small fixed-size chunks instead of buffering the whole thing, keeping memory use constant regardless of file size.

POST -> 202 { job_id } ... GET /status -> { done, total }

Chunk large transfers, hand processing off to a background job instead of blocking the request, and track progress by writing incremental state as chunks complete.

DB row: { object_key, size, content_type, owner } — blob bytes live in object storage

Keep a file's metadata in a small database row that references the blob by key, instead of storing the blob's bytes directly in the database.

GET/POST/PUT/PATCH/DELETE + 2xx/4xx/5xx

REST resource modeling, method semantics (safe/idempotent) and the status code families a client branches on.

?status=x&sort=-field&q=text&limit=20&cursor=abc

The four query mechanics a collection endpoint combines: filter, sort, search, paginate.

/v2/orders or Api-Version: 2 header

When a change needs a version bump, and the three common ways to carry the version.

?offset=40 (skip N) vs ?cursor=abc123 (after this row)

Offset pagination is simple but unsafe under writes; cursor pagination is safe and fast but cannot jump to an arbitrary page.

{ "error": { "code", "message", "details", "requestId" } }

A stable id never changes for the resource's life; a consistent error contract gives every failure a machine-readable code.

API gateway: one contract, every client · BFF: one backend, one client experience

When to share one gateway contract versus tailoring a backend per client.

authenticate -> throttle -> aggregate (parallel) -> shape

The order and purpose of the four core gateway/BFF request mechanics.

thin router + fixed cross-cutting concerns, forever — not "just add it here" one more time

The anti-pattern where a gateway accretes logic until it is a single point of failure nobody can test.

Idempotency-Key: <client-generated> — store the outcome, replay it on retry

How POST/command requests get the same retry-safety GET/PUT/DELETE have for free by HTTP semantics.

authentication = who; authorization = what they may do

The foundational distinction every other concept in this section builds on.

session+cookie = stateful, revocable | JWT = stateless, expiry-only

The core trade-off between server-side session state and self-contained signed tokens.

OAuth 2.0 = authorization (access_token) | OIDC = + authentication (id_token)

Delegated access via OAuth 2.0, and the identity layer OpenID Connect adds on top of it.

API key (static secret) | mTLS (mutual certs) | workload identity (short-lived, auto-rotated)

Authentication mechanisms for service-to-service calls, where there is no human or login screen.

RBAC (role) + ABAC (policy) + object-level check (this caller vs this resource)

The layered models that decide what an authenticated caller is actually allowed to do.

threat model -> shrink attack surface -> least privilege -> layer controls -> secure by default

The design-time mindset behind a defensible system architecture.

secrets manager (never in source) -> encrypt -> rotate on a schedule -> audit log every access

The operational controls that keep a secure design secure over time.

injection (input-as-code) vs access (reaches what it shouldn't) — 9 named OWASP-style risks

The common vulnerability classes behind most real-world application breaches, and their fixes.

TLS (in transit) + AES-256 or similar (at rest)

Two separate protections for two separate windows in data's life — neither substitutes for the other.

bcrypt.hashpw() / argon2.hash() — never encrypt(password)

Passwords are hashed one-way with a slow, purpose-built algorithm — never encrypted with a reversible cipher, and never hashed with a fast general-purpose hash.

KMS.GenerateDataKey() / KMS.Decrypt() — envelope encryption

Keys live in a KMS/HSM, not in app code; rotate on a schedule; keep decrypt access and audit-log control on separate roles.

shared schema < separate schema < separate database (isolation strength and cost both increase)

The three ways to partition tenant data, and the isolation-vs-cost trade-off between them.

API + query + cache key + file path + queue message + background job — each scoped independently

The full set of access paths tenant isolation must be enforced on, not just the database query.

shared schema: quota mandatory · separate schema: quota still needed, smaller blast radius · separate DB: quota moves to the shared layer above it

How the three tenancy models change where noisy-neighbor risk and per-tenant quotas actually apply.

logs (detail) + metrics (aggregate trends) + traces (cross-service request path)

The three complementary data shapes that make up observability — none of the three substitutes for the others.

IDs (correlate) + service name (attribute) + latency/errors (health) + saturation (early warning) + dependency metrics (whose fault)

The seven concrete signals to attach to every request for a system to actually be debuggable.

sync: context rides HTTP headers · async: context serialized into the message and extracted by the consumer

How a single trace stays connected across both synchronous service calls and asynchronous, queue-based workflows.

errors/requests, p95/p99, resource/limit, oldest-item-age, staleness, black-box reachability

The six symptom signals to alert on, each tied directly to something a user or downstream system feels.

resource% alone → false pages + missed incidents; symptom + resource-as-context → both fixed

Why alerting only on raw infrastructure metrics fails in both directions, and what to alert on instead.

liveness → restart, readiness → rotation, synthetic → scheduled external real-action probe

The three operational health checks, what question each answers, and what should happen when each fails.

incident response + on-call + postmortem + capacity planning + change management

The operational-maturity vocabulary beyond SLOs/error budgets: what happens once you decide to act, and what prevents the next incident.

canary/blue-green + backward-compatible rollback + graceful degradation + named ownership

Operability as concrete design properties a system either has or lacks, not a process checklist added after the fact.

backups + PITR (logical) · replication (§22, hardware) · multi-zone (DC) · multi-region (region)

The four DR mechanisms and the distinct failure class each one actually protects against.

RPO = max data loss (backward) · RTO = max downtime (forward)

The two independent numbers that turn a DR strategy into a testable, specific guarantee.

restore + verify != "job status: success"

Why an actually-restored, schema-checked backup is the only real evidence of recoverability.

audit all 7 layers: LB, app, DB, cache, queue, storage, network

Each layer on the request path needs its own explicit redundancy decision -- one un-redundant layer caps the whole system's availability.

active-active = all nodes serve now; active-passive = standby promoted on failure

Active-active trades consistency complexity for failover speed; active-passive trades failover speed for a single-writer simplicity.

health check (detect) -> automatic failover (react) -> graceful degradation (fallback)

The three mechanics that turn redundant capacity into an actual recovery during a real failure, in sequence.

latency | disaster recovery | data residency | global availability

The four independent business requirements that justify the cost of a multi-region architecture.

replication + conflicts + routing + consistency + clock/order + failover + cost + ops complexity

The eight compounding challenges that are the price of any multi-region architecture.

name the reason -> attach a number/citation -> rule out single-region -> only then build

The gate that catches prestige-driven multi-region adoption before it pays the eight-challenge cost for nothing.

metric polled → compared to target → replica count changed

What decides when horizontal autoscaling fires — CPU, memory, request rate, queue depth, or a custom metric matched to the workload's real bottleneck.

poll interval + decision time + cold start = scaling lag

Why autoscaling is reactive, not instant — cold starts, the cost of hiding them with warm capacity, and the danger of reacting to a noisy metric instead of sustained load.

orchestration layer (Kubernetes) != architecture (service boundaries, data ownership, failure behavior)

Operating Kubernetes fluently is a distinct skill from being able to explain the system design decisions underneath it.

LB + compute + managed DB + object storage + queue + cache + IAM + networking + observability = a cloud's menu

The nine generic system-design components as they map onto any major cloud provider's provisionable services.

prefer managed IF (ops burden saved) > (lock-in severity + cost at scale), else self-host

A workload-by-workload trade-off between operational burden saved, lock-in severity, and cost at actual scale — not a blanket rule.

compute + storage + bandwidth + DB IOPS + cache memory + cross-region + observability

The seven independent surfaces that make up a real cloud bill.

cost_per_unit = total_spend / units_of_completed_work

The four standard unit-cost lenses — per request, per GB stored, per active user, per job — and what each predicts.

size for realistic traffic + headroom + a documented migration trigger, not for an unquantified maximum

Why over-engineering for hypothetical scale is a continuous cost, and how to size for the traffic a product actually has.

hot (fast, expensive) → warm (medium) → cold (cheap, slow) — independent of retention length and legal-hold status

Storage temperature tiers by access pattern, and how retention and legal holds are separate axes on top of them.

TTL (per-record) · lifecycle rule (blob storage) · partition pruning (drop, not scan) · archival pipeline (batch to cold)

The four automated mechanisms that enforce a retention policy without manual per-record deletion.

minimize → least privilege → audit trail → retention/deletion → encryption → access review

The six controls a privacy-conscious data architecture applies together, each closing a different part of the exposure risk.

erasure request → primary + replicas + index + cache + crypto-shred backups + confirm no raw export — within the legal deadline

What a genuine GDPR-style deletion has to reach, and how regional residency and retention minimums layer on top.

critical = primary function cannot complete without it, no fallback · optional = degrades gracefully, has a fallback

How to classify each edge in a dependency map, and why the code itself does not make this decision for you.

critical → timeout + bulkhead + circuit breaker (contain, fail visibly) · optional → fallback or async decoupling (hide the failure)

Which of the five standard defensive mechanisms to apply to a dependency, based on whether it is critical or optional.

times out · returns invalid data · goes down · duplicates work · becomes slow · loses data · becomes unreachable

The seven distinct failure-mode questions to ask about every component in a design.

total failure = all fail the same way · partial failure = some succeed, some do not — needs its own explicit design

Why partial failure (some replicas/items/steps failing while others succeed) needs different handling than total failure.

retry storm · shared resource exhaustion · overloaded database · queue backlog · synchronized clients

The five most common amplification mechanisms behind a cascading failure.

bounded retries + jitter + circuit breakers + admission control + backpressure + load shedding + bulkheads

The seven standard mitigations against cascading failure, each targeting one or more of its common causes.

optional dependency fails → a pre-built, tested fallback engages, not an unhandled error

Why a degraded mode has to be deliberately designed, coded, and tested — classification alone changes nothing.

stale cache · disable the feature · queue the work · partial results

Four recurring patterns for graceful degradation, and the situation each one actually fits.

reject the excess fast and cheaply → the accepted fraction stays healthy; accept everything → throughput often collapses toward zero

Why deliberately rejecting work under overload usually produces more total successful throughput than accepting everything.

priorities (what to shed) + quotas (per-caller cap) + admission control (accept/reject point) + bounded queues (cheap overload signal)

The four composing mechanisms that turn "shed load" into a concrete, fair, and cheap-to-enforce policy.

additive change → no version bump · breaking change → new version + deprecation window before retiring the old one

When a version bump is needed, and why a deprecation window is what actually makes a breaking change safe.

expand (add new, keep old) → migrate (dual-write, backfill, roll out) → contract (remove old, only after verified zero-use)

The expand-contract pattern for making a breaking database change safe during a rolling deployment.

old + new instances coexist for the whole rollout → every change must work in all four old/new interaction combinations

Why a rolling deployment requires designing for old and new versions actively interacting, not just for the final all-new state.

backward compat = new schema reads old data · forward compat = old schema reads new data — additive changes with defaults preserve both

The two independent compatibility properties event schema evolution has to preserve, and why replay makes backward compatibility strict.

global ordering = one sequence, expensive coordination · partition-level ordering = strict order per key, no cross-key guarantee

Why most systems guarantee ordering only within a partition/key, and why that matches most real per-entity ordering needs.

dedupe by unique event ID (not content) + compare an explicit version/timestamp (not arrival order)

The two defenses every consumer needs against normal, expected duplicate and out-of-order delivery.

timestamp (clock-dependent) · sequence number (clock-independent counter) · version check (compare before applying)

Three mechanisms for determining true event order, and why sequence numbers and version checks are usually preferred over raw timestamps.

wall clock can step backward (NTP) · two machines' clocks drift apart continuously — neither is a rare fault

Why real clocks are neither monotonic nor identical across machines, and what that breaks when code assumes otherwise.

local timestamps across machines are unreliable for close-together events — use a sequence number, version check, or logical clock instead

Why cross-machine timestamp comparison fails silently for events close together in time, and what to use instead.

UTC (no timezone ambiguity) · monotonic clock (duration) · logical clock (causal order) · timeout (bound an indefinite wait)

Four distinct tools for four distinct time-and-ordering problems in a distributed system.

invariant = a rule true before, after, and after every partial/retried/concurrent attempt

How to name the correctness rules a multi-step user workflow owns, before choosing anything to enforce them with.

constraint > transaction > lock > idempotency key > workflow — take the strongest rung the invariant fits

The five enforcement mechanisms, what each guarantees, and the rule for choosing between them.

idempotency keys · immutable records · explicit states · webhooks · reconciliation · retry safety

The six building blocks of a correct payment flow, and the specific failure each one closes.

redirect = what to show · signed webhook / server-side lookup = what actually happened

Why a browser redirect cannot prove a payment completed, and what to hang fulfilment off instead.

payment_state (provider's machine) + fulfil_state (yours) + an enforced link rule

Why one order status column cannot hold two independent lifecycles, and how to split them.

intent = channel-independent fact, written in the business transaction · delivery = one channel attempt, retried

Why the decision to notify and the attempt to reach someone are two records with two lifecycles.

queue · per-attempt state (sent ≠ delivered) · backoff on transient only · dedupe on intent+channel · prefs and provider limits at send time

The five things the delivery half of a notification system has to do, and the order to do them in.

~100:1 read:write · base62/random/hash codes · unique constraint + retry · 302 for metered links · cache the immutable mapping · async analytics

The full URL shortener design, with every decision traced back to the read-to-write ratio.

write-time join = fast reads, O(followers) writes · read-time join = fast writes, O(following) reads · hybrid at a follower threshold

The two fan-out strategies, the celebrity skew that breaks both pure forms, and the hybrid that resolves it.

cache per-user pages + hot timelines · retrieve then rank a bounded candidate set · cursor pagination always · materialization is a dial

The four read-path decisions in a feed, and the two signals — skew and read/write ratio — that set them.

persist → sequence → publish → deliver → ack · presence = TTL key · offline sync = pull after last seq

The full chat design: transport, persistence, per-conversation ordering, three acknowledgements, presence, offline sync, group fan-out and push fallback.

metadata service (queryable) + object storage (bytes) · pending → signed URL → storage event → scan → available · lifecycle rules for cost

How a file-storage product composes a metadata service, direct signed-URL uploads and an asynchronous post-upload pipeline.

ingestion → indexing → index ← query service → ranking → cache · the index is the only join

The five stages of a search system, their independent scaling and failure profiles, and what belongs in a result cache key.

lag is inherent · reindex = build alongside + alias flip · shard count is fixed for life · hot terms = hot partitions

The six recurring operational facts of running a search index, all following from the index being a rebuildable derived copy.

identity · scope · algorithm · storage · coordination · failure mode (fail open vs fail closed, per limit)

The six decisions a rate limiter design has to make, with the weight on the failure mode that is usually left implicit.

one atomic op (INCR or script, expiry in the same step) · edge for capacity, app for identity-aware rules

Why a limiter check must be indivisible, and why the layer it runs at decides whether it protects capacity or only fairness.

durable job + run_at · lease (claim with expiry) + heartbeat renewal · backoff → dead-letter · alert on oldest-job age

The whole job-scheduling design, built around what happens when a worker dies holding a job.

auto-increment (db-assigned, guessable) · UUIDv4 (free, unordered) · ULID/UUIDv7 (free, time-ordered) · Snowflake (8 bytes, machine id required)

The four id schemes and the coordination cost that distinguishes them.

randomness ↔ index locality, resolved by a timestamp prefix · sortability free with time-ordered ids · coordination: none / once / per id

The five properties that distinguish id schemes, and why index locality is the one that only shows up at scale.

entry = {post id, score, author id} · bounded length + query fallback · hot keys replicated · edit/delete/unfollow/block invalidate differently

The storage layer under a feed: what a timeline entry holds, why it is bounded, and what each invalidation event costs.

unique constraint on the unit · atomic hold + expires_at checked at read time · idempotency key · convert inside a re-checking transaction

The reservation design that keeps "sold at most once" true under contention, abandonment, retries and slow payments.

ceiling = 1 / lock hold time · unaffected by more servers · non-linear approach · blocked transactions consume connections

Why a single counter row has a hard write ceiling, and why the failure spreads beyond the counter.

shard (exact, costlier reads) · batch (stale, lossy on crash) · approximate (bounded error) · aggregate (derived, recomputable)

The four ways to remove counter contention, and the different thing each one gives up.

few rows × all columns (operational) vs many rows × few columns (analytical) — same memory, same disk, opposite patterns

The structural reason large analytical scans do not belong on the database serving your application.

immutable events → buffered collection → streaming and/or batch → columnar lake + warehouse → rollups (keep the raw)

The four parts of an analytics pipeline, and why pre-aggregation and raw retention are complements rather than alternatives.

many small transactions · few rows each · milliseconds · integrity enforced by the database

What the OLTP workload is, and which storage-engine choices exist to serve it.

few long queries · many rows × few columns · aggregates · columnar, bulk-append, parallel, denormalised

What the OLAP workload is, and which engineering choices follow from its shape.

estimate first · primary (small only) → replica (fixes contention, not scan cost) → warehouse · always: read-only role, own pool, statement timeout

Why a read-only report can degrade a healthy primary for longer than it runs, and the ladder of places to move it.

structured filtering + transactional reads → database · B-tree, composite, inverted, trigram, partial, built-in full-text

The positive case for a database index, and the index types that cover more than people assume.

relevance · fuzziness · faceting · blended ranking — against a second system, a sync pipeline, derived-data discipline and reindex-only schema changes

The four capabilities a search engine buys and the four costs it charges, stated so the decision is explicit.

one system of record · no dual writes — outbox or CDC · derived stores answer "which items", the source answers "what is true"

How copies of the same fact stay in step, and which reads may never be served from a derived store.

requirements → estimates → APIs → data model → architecture → bottlenecks → deep dive → failures → observability/security → summary

The ten-step sequence as a dependency chain, where each step supplies an input the next one needs.

need (with numbers) → component → why it fits — never component first

Why naming a technology before stating the requirement removes the only thing that makes the choice reviewable.

components · owned stores · queues · network and trust boundaries · 2–3 major flows · sync vs async edges

The vocabulary of a high-level design, and the test for whether you have stayed at the right level.

one component: modules · classes and interfaces · schema (keys, indexes, constraints) · state machines · algorithms · internal APIs

The vocabulary of a low-level design, and the implementability test that marks it finished.

descend deliberately and announce it · one level per artefact · check high→low completeness and low→high dependency visibility

How to move between system context and detailed design without producing a document that fails at both.

requirements · constraints · capacity | clients · APIs | services · stores · cache · queues | external deps · deployment | observability · security · failure · scaling

The fifteen-item high-level checklist, grouped, with the three answers each item may receive.

entities · interfaces · responsibilities | state machine · schema · indexes | errors · concurrency · retries | module boundaries · testability

The eleven-item low-level checklist, grouped, run against one component after its design exists.

BENEFIT (with a number) · COST (concrete) · ALTERNATIVE (rejected) · REASON (specific to this system)

The four-part form every significant design decision is written in.

Postgres↔Dynamo · local↔Redis · sync↔async · monolith↔services · strong↔eventual · fan-out read↔write

The six recurring design axes, each with its default side and the condition that justifies leaving it.

skipped the thinking · wrong default shape · no failure model · blind in production

The twelve common system-design anti-patterns, grouped into the four questions a design failed to ask.

does it do the job · does it survive · does it grow · can we run it

The eleven-question design review, in four passes, asked by someone who did not write the design.

create path (rate limited) · id scheme by property · Redis read-through · 302 + click events on a queue

Project 1: a URL shortener, built as two paths with an explicit ID-scheme decision.

intent row + delivery row per attempt · pool per channel · dedupe key · 429 = backpressure · preferences at send time

Project 2: a notification platform, built around the intent/delivery split and per-attempt state.

signed PUT → object storage · metadata-only API · lease + extend · stream, never load · deterministic output key

Project 3: a file processing platform where bytes bypass the application tier entirely.

TX reserve → charge (outside, idem key) → TX record + outbox → async fulfil · conditional update, never read-then-write

Project 4: an order system where money, inventory and a third party must agree across three transaction boundaries.

persist → publish · per-room sequence · pub/sub per room · client cursor ?after=seq · presence = TTL key

Project 5: a chat platform, and the one project whose connections are stateful.

RLS + scoped repo · tenant in every cache key · tenant in every job payload · isolation then RBAC · one usage counter for quotas and billing

Project 6: a multi-tenant SaaS platform where isolation is enforced structurally on every access path.

index is derived · ingest from outbox/CDC · reindex behind an alias · evaluation set for ranking · cache queries not docs

Project 7: a search platform whose index can always be thrown away and rebuilt.

atomic check+update (INCR or script) · token bucket for APIs · identity key ≠ IP · decide fail-open/closed · enforce at the edge too

Project 8: a distributed rate limiter — the smallest project that teaches atomicity properly.

sizing and choosing · making reads fast · doing it exactly once · surviving failure

The twelve questions a five-year engineer answers without preparation, grouped, with mechanism-level answers.

requirements · capacity · APIs · data modeling · SQL · caching · queues · concurrency · transactions · consistency · availability · reliability · security · observability · scaling · trade-offs

Tier 1: the sixteen topics to master — usable without reference, and correct about the failure mode.

recognise it applies → design with it → know its failure mode · look up config and API

Tier 2: thirteen topics needing strong working knowledge, and the four usually adopted too early.

choose a system that implements it · reason about the guarantee · do not build it

Tier 3: six specialized topics where conceptual depth is the correct target, and why.

say what it must do → build the request path → async and survivable → distribute state → operate and communicate

The twenty-nine-step learning order as five phases, with the four edges that cannot be reordered.

"How do I implement this?" — correct when the decisions exist; dangerous when it makes them silently

The implementation question: its scope, and the decisions that escape into code when nobody names them.

"What architecture and technology?" — correct as the last question, misleading as the first

The technology question: why its answers are plausible without being derived, and how to tell the direction.

requirements · invariants · scale assumptions · failure modes · consistency needs · cost · trade-offs → the simplest design that satisfies them

The senior question: seven inputs, one output, and the filter that keeps the analysis from becoming an elaborate design.