beginnerintermediateadvanced

System Design · Section 25

Redis and Distributed Caching

Level: advanced
Read: 22 min
Concepts: 3

Redis's speed and atomic server-side operations make it useful for far more than caching — counters, session storage, rate limiting, lightweight coordination locks, and simple queues, all sharing the trait of fast, shared, short-lived state. Its core data structures (string, hash, list, set, sorted set) each fit a different shape of problem, with pipelining for batched round trips and Pub/Sub or Streams for messaging. At scale, three specific failure modes recur: a cache stampede when a popular key's expiry causes synchronized concurrent misses, a hot key overloading a single node despite spare cluster capacity, and memory pressure causing silent eviction under load.

This section

All About Caching | Distributed Caching | Redis vs Memcached | System Design Interview in 10 MinutesBangla Coding Tutor

What is true here

Redis fits fast, shared, short-lived state: caching, counters, sessions, rate limiting, lightweight locks, simple queues.
Five core data structures — string, hash, list, set, sorted set — each with atomic, server-side commands.
Pipelining batches many commands into one round trip; Pub/Sub broadcasts live with no storage; Streams persist and support replay.
Cache stampedes, hot keys and memory-pressure eviction are three distinct real-world failure modes, each with its own specific fix.

What you will be able to do

Recognize which Redis use cases fit and which need a durable store or full message broker instead
Choose the right Redis data structure for a given access pattern
Guard against a cache stampede on a popular key's expiry
Diagnose a hot key or memory-pressure eviction problem in a Redis cluster

What Redis is for, and its data structures

The range of problems Redis fits beyond caching, and the five core data structures plus atomicity, pipelining and messaging mechanisms.

What Redis is actually used for beyond caching

coreintermediate

Redis is best known as a cache, but its speed and atomic operations make it a fit for anything that needs fast, shared, short-lived state — counters, session storage, rate limiting, lightweight coordination locks, and simple queues. The common thread across all of these is: state that many instances need to share, updated frequently, where losing it is tolerable or backed up elsewhere.

Think of it as

Think of Redis as a shared whiteboard in a team's office instead of everyone keeping sticky notes on their own desk (in-process memory). Anyone can read or update it instantly, and because everyone reads the same board, counters and locks written there mean the same thing to everyone — a sticky note on one person's desk means nothing to a coworker in another room.

text

INCR page_views:42          -- atomic counter
SET session:abc "{...}" EX 1800  -- session with TTL
SET lock:job-1 "worker-a" NX EX 30  -- lightweight lock
LPUSH queue:emails "{...}"  -- simple queue push

What we're doing: Implement a fixed-window rate limiter with two atomic Redis commands.

rate-limiter.txttext

def is_allowed(user_id, limit=100, window_seconds=60):
    key = f"ratelimit:{user_id}:{current_minute()}"
    count = redis.incr(key)      # atomic increment
    if count == 1:
        redis.expire(key, window_seconds)  # first hit
                                            # in this window
                                            # sets the TTL
    return count <= limit

# 101st request in the same minute for this user
# returns False; the key expires on its own at the
# next minute boundary, resetting the count to 0.

3: INCR is atomic — many concurrent requests incrementing the same key never lose an increment to a race.
4: Only the request that creates the key sets its TTL, so the counter naturally resets at the window boundary.

Why this works: This is the whole rate limiter in two Redis commands — no application-level locking needed, because INCR and EXPIRE are each atomic on the server side.

Using Redis as the sole source of truth for data that must never be lost

Wrong

text

"Store the order total in Redis only — it's
fast and that's all we need."

Better

text

"Store the order total in the database as the
source of truth; use Redis to cache it or hold
short-lived state (a rate-limit counter, a
session) where losing it on a restart or
eviction is acceptable — Redis is not a durable
primary store by default."

What you see: A Redis restart, eviction under memory pressure, or replica failover silently loses data a team assumed was safe — because Redis's in-memory nature and (by default) periodic-not-continuous persistence make it a poor fit as the only copy of anything that must survive a crash.

Why: Redis is excellent for fast, shared, short-lived or reconstructible state, but its default persistence (RDB snapshots or AOF) is not the same durability guarantee a primary transactional database gives — using it as the only copy of critical data trades that durability for speed without deciding to.

What Redis is used for, beyond caching

Cache

string/hash + TTL

Counter

atomic INCR/DECR

Session store

shared across instances

Rate limiter

INCR + EXPIRE

Lightweight lock

SET NX EX

Simple queue

list or stream

Redis use cases and the primitive behind each

Redis use cases and the primitive behind each
Use case	Redis primitive	Why Redis fits
Cache	String/hash + TTL	Sub-millisecond reads, built-in expiration
Counter	INCR/DECR (atomic)	No race condition on concurrent increments
Session store	Hash or string, keyed by session ID	Shared across stateless app instances
Rate limiter	INCR + EXPIRE, or sorted set for sliding window	Atomic check-and-increment in one round trip
Lightweight lock	SET key val NX EX ttl	Atomic "set if not exists" with automatic expiry
Simple queue	List (LPUSH/RPOP) or Stream	Fast, ordered, no separate broker needed for simple cases

Remember: Redis fits anything that needs fast, shared, short-lived state: caching, atomic counters, session storage, rate limiting, lightweight locks, simple queues — not as a substitute for a durable primary store or a full-featured message broker when the workload genuinely needs one.

Redis data structures, atomicity and pipelines

coreintermediate

Redis is not just a key-value string store — it offers several distinct data structures (hashes, lists, sets, sorted sets, streams) natively, each with commands that operate atomically on the server. Pipelining batches multiple commands into one round trip for throughput; Pub/Sub broadcasts messages to subscribers with no storage; Streams add an append-only, replayable log on top.

Think of it as

Redis's data structures are like different specialized containers in a toolbox rather than one generic bucket: a hash is a labeled drawer of related items, a list is a stack of trays you can only add or remove from the top or bottom, a set is a bag where duplicates just bounce off, and a sorted set is that same bag but with every item pinned to a specific spot on a ruler. Pipelining is handing the whole toolbox to an assistant with a list of tasks instead of walking back and forth for each one.

text

HSET user:42 name "Ada" email "ada@example.com"
LPUSH recent:42 "viewed:product-99"
SADD tags:99 "sale" "featured"
ZADD leaderboard 1500 "player-7"
XADD events:orders '*' order_id 42 status "placed"

What we're doing: Build a leaderboard with a sorted set and show why it beats a plain sorted list.

leaderboard-zset.txttext

ZADD leaderboard 1500 "player-7"
ZADD leaderboard 2200 "player-3"
ZADD leaderboard 1800 "player-9"

ZREVRANGE leaderboard 0 2 WITHSCORES
-- "player-3" 2200
-- "player-9" 1800
-- "player-7" 1500
-- top 3, in score order, in O(log N) per insert

ZRANK leaderboard "player-9"
-- returns this player's rank directly, without
-- scanning or re-sorting the whole set

5: The sorted set stays ordered automatically on every insert — no separate sort step is ever needed.
10: Getting one member's rank is a direct O(log N) lookup, not a linear scan through a plain list.

Why this works: A sorted set gives ranking, top-N and single-member-rank operations all in logarithmic time, which is exactly the operation profile a leaderboard needs and a plain list or set cannot provide efficiently.

Using Pub/Sub for messages that must not be lost

Wrong

text

PUBLISH order_events '{"order_id": 42, ...}'
-- no subscriber connected at publish time

Better

text

XADD order_events '*' order_id 42 status "placed"
-- a Stream persists the entry; a consumer that
-- connects later (or reconnects after a drop)
-- can still read it from where it left off

What you see: A subscriber that briefly disconnects (a deploy, a network blip) silently misses every message published during that gap, with no error and no way to retrieve what was lost — because Pub/Sub delivers only to currently connected subscribers and stores nothing.

Why: Pub/Sub is fire-and-forget by design — it was built for live broadcast, not durable delivery; Streams exist specifically for the case where messages need to be persisted and consumers need to be able to catch up after a gap.

The five core data types

String

single value per key

Hash

field-value map

List

ordered sequence

Set

unordered, unique members

Sorted set

unique + scored, ordered

The five core data types and what each is good for

The five core data types and what each is good for
Type	Shape	Good for
String	Single value per key	Cache values, counters, flags
Hash	Field-value map per key	Object-like records (a user, a product)
List	Ordered sequence	Simple queues, recent-activity feeds
Set	Unordered unique members	Tags, unique-visitor tracking, set operations
Sorted set	Unique members with a score, ordered	Leaderboards, priority queues, range queries

Remember: Five core data structures — string, hash, list, set, sorted set — each with atomic commands. Pipelining batches commands into one round trip. Pub/Sub broadcasts live with no storage; Streams add a persistent, replayable log.

How a distributed cache breaks down at scale

Three specific, recurring failure modes and the fix for each.

Cache stampede, hot keys and memory pressure

coreadvanced

These are three specific ways a distributed cache breaks down under real load. A cache stampede (or dogpile) happens when a popular key expires and many concurrent requests all miss at once, hammering the backend simultaneously. A hot key is a single key receiving disproportionate traffic, which can overload the one Redis node/shard holding it even if the cluster overall has capacity. Memory pressure is Redis running low on memory and having to evict data, sometimes unpredictably.

Think of it as

A cache stampede is a popular restaurant's doors opening the instant a "closed for cleaning" sign comes down — everyone who was waiting rushes in at once, overwhelming a kitchen sized for steady, spread-out traffic. A hot key is when everyone wants the same one table, no matter how many other tables are free. Memory pressure is the restaurant running out of table space and having to turn people away or clear a table mid-meal to make room.

text

-- single-flight refill lock (prevents a stampede)
if SET lock:key1 1 NX EX 5:
    value = recompute_expensive_thing()
    redis.set("key1", value, ex=300)
    redis.delete("lock:key1")
else:
    return stale_or_wait()

What we're doing: Show a stampede happening on a popular key's expiry, and the single-flight fix.

cache-stampede.txttext

"homepage:featured" is read by 2,000 requests/sec
and cached with a 60-second TTL.

At second 60, the key expires. In the next ~10ms,
several hundred of those 2,000 req/sec all miss the
cache at the same instant, and all several hundred
independently query the (expensive) backend to
rebuild the same value — a stampede.

Fix: the FIRST request to miss takes a short lock
(SET lock:featured 1 NX EX 5) and rebuilds the cache;
every other concurrent request sees the lock is held
and either serves the just-expired stale value for a
few more seconds, or waits briefly for the rebuild —
only one backend query happens instead of hundreds.

7: This is the stampede itself — every concurrent miss independently pays the full backend cost, at the same moment.
11: The lock reduces "hundreds of backend queries at once" to exactly one, with everyone else waiting on or reusing that one result.

Why this works: A cache existing does not automatically protect a popular key's expiry moment — without a stampede guard, the backend load spike at that exact instant can be worse than having no cache at all, because it is now concentrated into a single burst.

Setting the same fixed TTL on many related keys, causing them to expire together

Wrong

text

for product in top_100_products:
    cache.set(f"product:{product.id}", data, ex=300)
-- all 100 keys expire at almost exactly
-- the same moment, 300 seconds later

Better

text

import random
for product in top_100_products:
    jitter = random.randint(-30, 30)
    cache.set(f"product:{product.id}", data,
              ex=300 + jitter)
-- expirations spread across a ~60s window
-- instead of one synchronized instant

What you see: The backend sees a recurring, sharp traffic spike at a regular interval — every 5 minutes, say — instead of steady load, and the spike lines up exactly with a batch of cache keys that were all set with the same TTL at the same time.

Why: Setting many keys with an identical TTL synchronizes their expiry, turning what should be spread-out cache refills into a recurring coordinated stampede — adding a small random jitter to each TTL breaks that synchronization cheaply.

Three ways a distributed cache breaks under load

Stampede

concurrent misses hit the backend at once

Hot key

one key overloads one node

Memory pressure

eviction under a memory limit

Three distinct failure modes and their fixes

Three distinct failure modes and their fixes
Problem	Cause	Common fix
Cache stampede	Many concurrent misses on the same expired key	Refill lock (single-flight), staggered/jittered TTLs
Hot key	Traffic concentrated on one key, one shard	Replicate the value across multiple keys, client-side load spreading
Memory pressure	Cache nears its memory limit	Right-size memory, choose an eviction policy deliberately, monitor evictions

Remember: Cache stampede: many concurrent misses on one expired key hit the backend at once — fix with a single-flight refill lock or jittered TTLs. Hot key: one key overloads one node — fix by spreading it across several keys. Memory pressure: watch eviction counts, not just memory usage.