beginnerintermediateadvanced

System Design · Section 24

Caching Fundamentals

Level: intermediate
Read: 22 min
Concepts: 4

Caching works because real access patterns are skewed, not random — the same or related data tends to be requested again soon, which is exactly what a cache exploits to avoid repeating expensive work. Four patterns — cache-aside, read-through, write-through, write-back — differ in who loads a cache miss and when a write reaches the underlying store. Entries leave a cache one of three ways: a TTL expiring, eviction under space pressure, or explicit invalidation on a known write — with cache warming and negative caching as two further techniques covering the cold-start and repeated-miss cases those three don't.

System Design overview

System Design for Beginners CoursefreeCodeCamp.org

What is true here

Caching pays off when data is expensive to produce, read much more than written, and accessed with real temporal or spatial locality.
Cache-aside, read-through, write-through and write-back differ in who populates the cache and when a write reaches the store.
TTL, eviction and invalidation are the three ways an entry leaves a cache — invalidation is the most precise but easiest to miss a path for.
Cache warming avoids a cold-start miss storm; negative caching protects the backend from repeated requests for data that doesn't exist.

What you will be able to do

Judge whether a given piece of data is actually a good caching candidate
Choose the right cache pattern for a given consistency/latency trade-off
Explain why a TTL backstop matters even alongside explicit invalidation
Recognize the cold-start and repeated-miss gaps warming and negative caching close

Why caching works, and the four patterns

The locality principle that makes caching pay off, and the four ways an application and cache can divide responsibility for loading and writing data.

Why caching works: locality and avoiding repeated work

corebeginner

Caching works because real access patterns are not random — the same data tends to be requested again soon (temporal locality), and related data tends to be requested together (spatial locality). A cache stores the result of expensive work once and serves it repeatedly from somewhere far cheaper to read, exploiting exactly those patterns.

Think of it as

A cache is like keeping today's most-asked-for library books on a cart by the front desk instead of walking to the stacks every time someone requests one. It works because a small number of books actually account for most requests (temporal locality) and because someone asking for one popular book is likely to ask for a related one next (spatial locality) — the cart pays off precisely because requests are not evenly spread across every book in the building.

What we're doing: Show the actual cost difference a cache exploits, quantified.

caching-cost-model.txttext

Without a cache:
  Every request for a product page runs a query
  joining 4 tables — measured at ~40ms average.
  10,000 requests/minute → 400,000ms = ~6.7 minutes
  of cumulative database time per minute of traffic.

With a cache (95% hit rate, 1ms cache read):
  9,500 requests/min served from cache: 9,500 × 1ms
    = 9.5 seconds of cache time.
  500 requests/min still hit the database: 500 × 40ms
    = 20 seconds of database time.
  Total database load drops by ~95% for the same
  traffic — this is the concrete payoff, not an
  abstract "caching is good" claim.

5: This is the cost caching is trying to avoid paying repeatedly — the expensive join, run on nearly every request.
13: A 95% hit rate turns nearly all of that repeated cost into a single-millisecond cache read instead.

Why this works: The benefit of caching is not free or automatic — it is exactly proportional to how expensive the original work was and how often the same result would otherwise be recomputed, which is why locality (the same or related data being requested repeatedly) is the precondition that makes the trade worthwhile.

Caching data with a low hit rate or near-uniform access pattern

Wrong

text

"Let's cache every database query result —
caching always helps."

Better

text

"Cache queries with a skewed access pattern —
a small set of keys gets most requests. For
queries where access is close to uniformly
random across a huge key space, a cache adds
memory cost and invalidation complexity for a
hit rate too low to pay for itself."

What you see: A cache layer is added, but its hit rate stays low (well under 50%) and the system still shows the same database load as before, plus the added memory and invalidation-logic cost of the cache itself — a sign the underlying access pattern didn't actually have the locality caching depends on.

Why: Caching's entire value proposition depends on the same or related data being requested again — without real temporal or spatial locality in the access pattern, a cache is paying memory and complexity cost for a hit rate too low to recoup it.

Database load, with and without a cache

Without a cache

+Every request runs the expensive 4-table join (~40ms)
+10,000 requests/min → ~6.7 minutes of cumulative DB time
+Load scales linearly with traffic — no ceiling

With a cache (95% hit rate)

−9,500 requests/min served from cache in ~1ms each
−Only 500 requests/min still reach the database
−Database load drops by ~95% for the same traffic

What makes data a good caching candidate

What makes data a good caching candidate
Property	Good fit for caching	Poor fit for caching
Read/write ratio	Read far more often than written	Written as often as read
Access skew	A small hot subset gets most requests	Access is close to uniformly random
Cost to produce	Expensive query, computation, or external call	Already cheap to read directly
Staleness tolerance	Slightly stale data is acceptable	Must always reflect the very latest write

Remember: Caching works because real access is skewed, not random — the same or related data gets requested again soon. It only pays off when the underlying work is genuinely expensive and read far more often than it's written.

Cache-aside, read-through, write-through and write-back

coreintermediate

These four patterns answer two questions: who is responsible for loading data into the cache, and when does a write update the cache relative to the underlying store. Cache-aside puts the application in charge of both; read-through and write-through hand that responsibility to the cache itself; write-back further delays writing to the underlying store until later, trading durability risk for write speed.

Think of it as

Cache-aside is a personal assistant who checks their own notebook first, and only calls the archive (the store) and writes the answer down themselves if the notebook doesn't have it. Read-through is an assistant backed by a smarter notebook that fetches from the archive automatically the moment it's asked something it doesn't know. Write-through is updating both the notebook and the archive together, right now, before moving on. Write-back is jotting the update in the notebook and promising to update the archive "later" — faster, but risky if the notebook is lost before that promise is kept.

text

cache-aside:  app -> cache (miss) -> store -> app fills cache
read-through: app -> cache -> (miss) store, cache fills itself
write-through: app -> cache + store (both, synchronously)
write-back:    app -> cache (immediately) -> store (later, async)

What we're doing: Walk through cache-aside's read and write paths for a user profile.

cache-aside-flow.txttext

READ (cache-aside):
1. App checks cache for key "user:42".
2. Miss — app queries the database directly.
3. App writes the result into the cache with a TTL.
4. App returns the result to the caller.
   Next read for "user:42" is a cache hit — skips
   steps 2 and 3 entirely until the TTL expires.

WRITE (cache-aside, invalidate-on-write):
1. App writes the updated profile to the database.
2. App deletes (does not update) the "user:42"
   cache entry.
3. Next read is a miss, which repopulates the cache
   with the fresh value from step 1.

3: The application, not the cache, is responsible for populating the cache — this is the defining trait of cache-aside.
11: Deleting rather than updating the cache entry on write is the safer default — see the mistake below for why.

Why this works: Cache-aside is the most common pattern precisely because it requires no special cache infrastructure — any key-value store works, and the application fully controls both the read-miss and write paths.

Updating the cache directly on write instead of invalidating it

Wrong

text

-- on write:
db.update(user)
cache.set("user:42", user)  -- write the new value
                             -- directly into the cache

Better

text

-- on write:
db.update(user)
cache.delete("user:42")  -- invalidate; the next
                          -- read repopulates it
                          -- from the database

What you see: Two concurrent writes to the same key can race such that the cache ends up holding the OLDER of the two values even though the database correctly holds the newer one — the cache silently disagrees with the source of truth until its TTL expires.

Why: If two writers both update the database and then both set the cache, the cache's final value depends on which writer's cache.set() executes last — not which database write happened last — so the cache can end up stale in a way that a simple delete-and-let-the-next-read-repopulate approach avoids.

Cache-aside read path, on a miss

Application

checks cache first

Cache miss

not found

Database

source of truth

Cache

app writes the result in

The four cache patterns

The four cache patterns
Pattern	Who loads on a miss	Write path	Risk
Cache-aside	Application	App writes to store; cache is invalidated or updated separately	Momentary inconsistency between store and cache
Read-through	Cache itself	Same as cache-aside, or paired with write-through	Cache library/proxy needs store-loading logic
Write-through	Cache or application	Synchronous: cache + store both updated before write returns	Higher write latency
Write-back	Cache or application	Asynchronous: cache updated now, store updated later	Data loss if the cache fails before flushing

Remember: Cache-aside: app manages both the miss and the write path — most common. Read-through: the cache loads itself. Write-through: cache and store updated together, synchronously. Write-back: cache updated now, store later — fastest, riskiest.

How an entry leaves the cache

TTL, eviction and invalidation as the three exit paths, plus warming and negative caching for the cold-start and repeated-miss cases.

TTL, eviction and invalidation

coreintermediate

These are the three ways an entry leaves a cache. A TTL (time-to-live) expires an entry automatically after a set duration. Eviction removes entries when the cache is full, following a policy like least-recently-used (LRU). Invalidation is an explicit, immediate removal triggered by the application when it knows the underlying data changed — the only one of the three that reacts to an actual write rather than time or space pressure.

Think of it as

A TTL is food with a printed expiration date — it gets thrown out at a fixed time no matter what. Eviction is a fridge that's run out of room, so the oldest untouched item gets tossed to make space for something new. Invalidation is someone actively noticing the milk went bad early and throwing it out right then, regardless of its printed date — the most accurate signal, but only as good as someone remembering to check.

text

SET user:42 "..." EX 300   -- TTL: expires in 300s
-- eviction: cache-internal, policy-driven (e.g. LRU)
DEL user:42                -- invalidation: explicit

What we're doing: Show a missed invalidation path leaving stale data behind despite a TTL safety net.

missed-invalidation.txttext

Two code paths update a user's email:
  1. The profile settings page — correctly calls
     cache.delete("user:42") after the database write.
  2. An admin "merge duplicate accounts" tool, added
     later — updates the database directly, but nobody
     added the same cache.delete() call there.

Result: after an admin merge, "user:42" in the cache
still holds the pre-merge email for up to the TTL
window (say, 10 minutes) — a real, if bounded, window
of stale data that only the TTL eventually corrects,
not the invalidation logic that was supposed to.

5: This is the missed path — a second writer to the same data that the invalidation logic was never extended to cover.
9: The TTL is the safety net here — it bounds the staleness window, but only because it happened to be set at all.

Why this works: This is the exact failure mode that makes cache invalidation famously hard — the bug isn't in the invalidation code that exists, it's in the write path nobody remembered to add invalidation to, and a TTL is often the only thing that bounds the resulting staleness.

Relying on invalidation alone with no TTL as a safety net

Wrong

text

SET user:42 "..."   -- no expiration set at all
-- relies entirely on every write path
-- remembering to call DEL user:42

Better

text

SET user:42 "..." EX 3600   -- TTL as a backstop
-- invalidation still fires on known writes,
-- but a missed path self-heals within an hour
-- instead of staying stale indefinitely

What you see: A cache entry keeps returning the exact same stale value indefinitely, discovered only when a user reports data that "never updates" — because with no TTL, a missed invalidation path has literally no mechanism to ever self-correct.

Why: Invalidation depends on every write path remembering to trigger it — a new feature or refactor can easily add a new way to write the underlying data without updating the invalidation logic; a TTL bounds the damage from that near-inevitable gap instead of leaving it unbounded.

Three ways an entry leaves the cache

TTL

expires after a fixed duration

Eviction

removed under space pressure (LRU)

Invalidation

explicit removal on a known write

The three ways an entry leaves a cache

The three ways an entry leaves a cache
Mechanism	Trigger	Guarantees freshness?
TTL	Time elapses	No — stale for up to the TTL window
Eviction (e.g. LRU)	Cache is full, needs space	No — evicted for space, not staleness
Invalidation	Application detects a write	Yes, if every write path correctly invalidates

Remember: TTL expires by time, eviction removes by space pressure, invalidation removes by an explicit write signal — invalidation is the most precise but the easiest to miss a path for, which is why a TTL backstop is usually kept even alongside it.

Cache warming and negative caching

standardintermediate

Cache warming pre-loads likely-needed entries before real traffic arrives, avoiding a wave of cache misses right after a cold start or deploy. Negative caching stores the fact that something does NOT exist (a 404, a not-found lookup) so repeated requests for the same missing item don't hit the expensive backend every time — an easily overlooked case, since it feels odd to "cache" an absence.

Think of it as

Warming is stocking a new store's shelves with the popular items before opening day, instead of making the very first customers wait for a truck to arrive. Negative caching is a receptionist keeping a short list of "asked for and not on file" names, so the tenth person asking about the same nonexistent tenant gets an instant "no" instead of the receptionist searching the entire building records again.

text

-- warming: populate before serving traffic
for key in top_1000_products: cache.set(key, load(key))

-- negative caching: cache the miss itself
if not found_in_db(id):
    cache.set(f"user:{id}", NOT_FOUND, ttl=60)

What we're doing: Show a deploy causing a cache-miss storm without warming, and the negative-caching fix for a repeated-miss attack.

warming-and-negative-caching.txttext

Without warming:
  Cache redeployed empty at 09:00. Traffic resumes
  immediately. First 30 seconds: 90% of requests are
  cache misses, all hitting the database at once —
  a load spike the database wasn't sized for during
  steady state.

Without negative caching:
  A client (or bot) repeatedly requests
  GET /users/999999999 (an ID that doesn't exist).
  Every request misses the cache (nothing to cache —
  there's no such user) and hits the database's
  lookup path directly, every single time.

Fix for the second case: cache "user 999999999: not
found" for 60 seconds. Repeated requests for the same
missing ID now hit the cache, not the database.

3: This is the cold-start problem warming exists to prevent — an empty cache and full traffic arriving at the same moment.
12: This is the case negative caching covers — a cache that only ever stores hits has no defense against repeated misses for the same key.

Why this works: Both techniques address gaps in the "just cache what exists" mental model — warming covers the moment before anything has been cached yet, negative caching covers requests for things that will never successfully populate the cache through the normal hit path.

Assuming a cache protects the backend from requests for nonexistent data

Wrong

text

if key in cache: return cache[key]
result = db.lookup(key)
if result: cache.set(key, result)
return result
-- misses are never cached, only hits

Better

text

if key in cache: return cache[key]
result = db.lookup(key)
cache.set(key, result if result else NOT_FOUND,
          ttl=60 if result else 30)
return result

What you see: A cache with a very high hit rate for real data still shows heavy backend load — traced to a small number of IDs being requested repeatedly that never resolve to anything, none of which the cache is protecting against because only successful lookups were ever being cached.

Why: A cache-aside implementation that only writes on a successful lookup provides zero protection against repeated requests for data that doesn't exist — every one of those requests still falls through to the backend exactly as if there were no cache at all.

Two holes in "just cache what exists"

One hole opens before the cache has anything in it. The other never closes, because the key it is asked for will never resolve.

Cache warming vs negative caching

Cache warming vs negative caching
Technique	Problem it solves	Typical trigger
Cache warming	Cold-start miss storm right after deploy/restart	Pre-load top-N known-hot keys on startup
Negative caching	Repeated lookups for something that doesn't exist	Cache the miss result itself, with a short TTL

Remember: Warming pre-loads a cache before real traffic hits it, avoiding a cold-start miss storm. Negative caching stores "not found" results too, so repeated misses for the same missing key don't bypass the cache entirely.