beginnerintermediateadvanced

System Design · Section 103

Common System Design Anti-Patterns

Level: intermediate
Read: 14 min
Concepts: 1

Twelve recurring mistakes, grouped into four families by what the design skipped. Skipping the thinking: starting from technologies instead of requirements, having no capacity estimates, and designing for global scale nobody asked for. Choosing the wrong default shape: microservices on day one, a shared database with no owning service, and synchronous calls for work that takes minutes. Having no failure model: unbounded retries, no idempotency, and no written answer to what happens when a dependency is down. Being blind in production: ignoring tail latency, treating a cache as the source of truth, and having no observability. The grouping is the point — twelve loose items get memorised and forgotten, four questions get asked of every page of a design document. They also compound rather than sitting side by side: no capacity estimate means no tail-latency target, which means no timeout, which means callers hold connections through a slow dependency and retry without a cap, which turns one degraded service into a full outage. That chain is one root cause wearing four names. Every item is more accurately described as "we never decided" than "we decided wrong", which is exactly why they survive review — nothing in the document is visibly false. The fix always names a mechanism with a number in it: an attempt cap, a deadline, a percentile target, an owning service. And inverting an item into a rule — never cache, never split, never think about scale — reproduces the original error with the sign flipped, because it is another default adopted without a requirement.

System Design overview

System Design for Beginners CoursefreeCodeCamp.org

What is true here

Four families, one unasked question each: what is required, what shape, what happens on failure, how would we know.
They compound: no capacity estimate → no latency target → no timeout → unbounded retries → cascading outage.
Every item is "we never decided" rather than "we decided wrong", which is why reviews miss them.
A finding without a named mechanism produces agreement and no diff — name the cap, the store, the percentile.
Inverting an anti-pattern into a prohibition is the same mistake with the sign flipped.

What you will be able to do

Read a design document and place each gap in one of the four families rather than recalling twelve separate items
Trace how a missing capacity estimate becomes an outage four steps later
Rewrite an anti-pattern finding into a review comment with a definition of done
Explain why "we removed the cache" and "we never split anything" are not fixes

The four families

Twelve anti-patterns sorted by the question the design failed to ask, with a mechanism-level fix for each.

Twelve anti-patterns, in four families

coreintermediate

Twelve recurring mistakes, and they group into four families by what each one skips. Skipping the thinking: starting from technologies instead of requirements, having no capacity estimates, and designing for global scale nobody asked for. Choosing the wrong default shape: microservices by default, one shared database with no owner, and synchronous calls for work that takes minutes. Having no failure model: unbounded retries, no idempotency, and no written answer to "what happens when this dependency is down". Being blind in production: ignoring tail latency, treating a cache as the source of truth, and having no observability. Grouping them matters because twelve loose items get memorised and four families get recognised, and recognition during a live review is the only place the list is worth anything. They also compound: no capacity estimate leads to no tail-latency target, which leads to no timeout, which turns a slow dependency into unbounded retries, which turns one slow service into a system-wide outage. That chain is a single root cause wearing four names. The fix for each is a mechanism rather than a slogan — a number, a bound, a deadline, an owner — and the honest form of every one of these is not "we did it wrong" but "we never decided".

Think of it as

Four questions a design can fail to ask, not twelve rules it can break. What is actually required? What shape does that require? What happens when part of it fails? How would we know? Each unasked question produces its own family of symptoms, which is why fixing one member of a family without asking the question usually surfaces the next member a month later.

text

Four questions. Each unasked one is a family.

  What is actually required?
      -> technologies first · no capacity
         estimates · global scale, no reason

  What shape does that require?
      -> microservices by default · one DB
         with no owner · sync calls for long work

  What happens when part of it fails?
      -> unbounded retries · no idempotency
         · no failure model

  How would we know?
      -> ignoring tail latency · cache as
         source of truth · no observability

Twelve items are memorised.
Four questions are recognised.

What we're doing: Read one real design document and name which family each gap belongs to.

review-notes.txttext

Document: "Media Processing Service v1"

Page 1
  "We will use Kubernetes, Kafka and S3."
  -> Skipped the thinking. Three products
     named before a single requirement.
     Ask: what must this do, for how many
     users, at what latency?

Page 2
  "Users upload a video and receive the
   processed result in the response."
  -> Wrong default shape. Transcoding is
     minutes of work on an HTTP connection.
     Fix: accept, enqueue, return a job id;
     the client polls /jobs/{id} or gets a
     webhook.

Page 3
  "Workers retry failed jobs until they
   succeed."
  -> No failure model. Unbounded. A poison
     job retries forever and a provider
     outage multiplies load exactly when
     the provider is weakest.
     Fix: 5 attempts, exponential backoff
     with jitter, then dead-letter.

Page 4
  "Job status is stored in Redis for fast
   reads."
  -> Blind in production, and worse. Redis
     here IS the record: a restart loses
     every in-flight job with no way to
     rebuild it.
     Fix: status in Postgres, Redis as a
     read-through cache in front of it.

Page 5
  "Average processing time: 45 seconds."
  -> Blind in production. An average over a
     workload whose input size varies by
     1000x. The 4-hour job is invisible.
     Fix: report p50/p95/p99 by input size
     bucket, and set a target on p99.

Five findings, four families, one document.
None of them is a wrong decision. All five
are decisions nobody made.

4: The tell for this family is ordering: a design that names products before requirements has usually chosen the products first and will bend the requirements to fit them.
20: Unbounded retries are the most reliably damaging item on the list, because they convert a partial failure into a load multiplier aimed at the exact component that is already struggling.
30: The cache-as-record failure is not "the cache might be stale" — it is that there is no store to rebuild from, so an eviction is data loss rather than a slow request.
41: An average over a 1000x-variable workload is not a weak metric, it is an actively misleading one: it can improve while the slowest jobs get worse.

Why this works: The value of the four families is that they turn a review from twelve recall attempts into four questions asked of every page. Each finding here also names a fix with a mechanism in it — a job id, an attempt cap, a store behind the cache, a percentile target — so the review produces work rather than disapproval.

Reporting anti-patterns without naming the mechanism that fixes them

Wrong

text

Review comment:
  "This has no failure model and relies on
   the cache as a source of truth. Please
   address before merge."
# Correct, unactionable, and impossible to
# tell when it has been satisfied.

Better

text

Review comment:
  "Two things. (1) Job status needs a store
   behind Redis — Postgres jobs table,
   Redis read-through. (2) Retries need a
   cap: 5 attempts, exponential backoff
   with jitter, then dead-letter with an
   alert on DLQ depth > 0."
# Two changes, both checkable.

What you see: A design review that generates agreement and no diff: the author accepts every point, changes nothing structural, and the same findings reappear in the next review of the same system.

Why: An anti-pattern name describes a shape, not a change. Without a named mechanism there is no definition of done, so the fix defaults to whatever the author already intended — which is the design that produced the finding.

Twelve anti-patterns in four families

Skipped the thinking

Technologies first

requirements arrive on slide six

No capacity estimates

Global scale, no requirement

Wrong default shape

Microservices by default

One database, no owner

a rename breaks four teams

Sync calls for long work

No failure model

Unbounded retries

turns slow into down

No idempotency

No answer for a dead dependency

Blind in production

Ignoring tail latency

the mean hides the p99

Cache as source of truth

No observability

The twelve, grouped by what the design skipped

The twelve, grouped by what the design skipped
Family	Anti-pattern	What it looks like	The fix — a mechanism, not a slogan
Skipped the thinking	Starting with technologies	The design opens with Kafka and Kubernetes; requirements appear on slide six	Write functional and non-functional requirements before naming any product
Skipped the thinking	No capacity estimates	No RPS, no data volume, no growth rate anywhere in the document	One page of arithmetic: peak RPS, bytes/day, storage/year, connections
Skipped the thinking	Global scale with no requirement	Multi-region active-active for a product with users in one country	Name the business requirement (latency, residency, DR) or run one region
Wrong default shape	Microservices by default	Eight services on day one, one team, one deploy pipeline in practice	Modular monolith until deploy cadence, scaling or on-call genuinely diverge
Wrong default shape	One database, no ownership	Every service reads every table; a column rename breaks four teams	One writer per table; everyone else goes through an API or a replicated copy
Wrong default shape	Synchronous calls for long work	An HTTP request that transcodes a video and holds the connection for 90 s	Accept, enqueue, return an id; the caller polls or gets a callback
No failure model	Unbounded retries	Retry until success, no cap, no backoff, no jitter	Max attempts, exponential backoff with jitter, then a dead-letter queue
No failure model	No idempotency	A retried POST creates a second order; a redelivered event charges twice	Idempotency key on every unsafe write; the second call returns the first result
No failure model	No failure model at all	Nobody can say what happens when the payment provider is down	Per-dependency: what fails, what degrades, what the user sees
Blind in production	Ignoring tail latency	The dashboard shows a 40 ms average; p99 is 3 s and nobody is looking	Set and alert on p95/p99 targets, not means
Blind in production	Cache as the source of truth	A cache flush loses data, or a cache miss returns a wrong answer rather than a slow one	The cache is derived; every entry must be reconstructible from the store
Blind in production	No observability	Failures are discovered from customer emails	A symptom-based alert per user-visible promise, plus traces across service hops

How four of them chain into one outage

How four of them chain into one outage
Step	What was missing	What it caused
1	No capacity estimate	Nobody knew the dependency saturates at 400 rps
2	No tail-latency target	A p99 of 4 s looked acceptable because the mean stayed at 60 ms
3	No timeout (a consequence of 2)	Callers held connections for the full 4 s and exhausted the pool
4	Unbounded retries	Every timed-out call became three more, tripling load on an already-saturated dependency
—	Result	A slow dependency became a total outage; four names, one root cause

Remember: Twelve anti-patterns, four families, one question each. What is actually required? (technologies first, no capacity estimates, global scale with no reason.) What shape does that require? (microservices by default, one database with no owner, synchronous calls for long work.) What happens when it fails? (unbounded retries, no idempotency, no failure model.) How would we know? (ignoring tail latency, cache as source of truth, no observability.) They compound into single outages, every one is really "we never decided", and the fix always names a mechanism with a number in it.