Filter concepts by levelShowing all levels.

Python · Section 26

Redis

Level
intermediate
Read
130 min
Concepts
9

Redis as a key-value store: the five data structures (strings, lists, sets, sorted sets, hashes), TTL-based expiration, atomic operations and transactions, Pub/Sub and Streams for messaging, distributed locks and their single-instance caveats, and the cache-aside pattern that ties storage and TTL together.

This section

What is true here

  1. Every Redis value lives under a unique string key; incr/incrby/hincrby update a number atomically with no read-modify-write race.
  2. Sets give O(1) membership checks; sorted sets add a per-member score and stay readable in score order — the structure behind any leaderboard.
  3. A TTL (r.expire, or ex= on set) is a self-deleting countdown on a key — a plain set() silently clears it unless keepttl=True is passed.
  4. A distributed lock acquires with SET NX PX, but must release by checking a unique token and deleting atomically, or it can delete a lock it no longer owns.
  5. Pub/Sub broadcasts live with zero history; Streams persist an ID-ordered log consumer groups can read and replay — pick Streams when a missed message is unacceptable.

What you will be able to do

  • Read and write strings, lists, sets, sorted sets, and hashes with the correct redis-py command for each
  • Attach, read, and clear a TTL, and explain why a plain SET silently drops one
  • Use INCR/pipeline+MULTI/EXEC/WATCH to keep an operation atomic instead of racing on a client-side read-modify-write
  • Explain the difference between Pub/Sub and Streams and choose correctly between them
  • Implement a safe single-instance distributed lock, including its release pattern, and state why it is not sufficient alone for a lock that must never double-acquire
  • Implement the cache-aside pattern with an appropriate TTL

Redis

Redis as a key-value store — the five data structures it can hold, how a key stops existing on its own via TTL, what "atomic" actually guarantees, its two messaging primitives, and the two patterns (distributed locks, caching) built out of everything above.

Key-value storage and strings

corebeginner

Redis stores every value under a unique text key, like a giant dictionary that lives outside your process. r.set("user:1001:name", "Priya Shah") writes it; r.get("user:1001:name") reads it back — a string is the simplest value a key can hold.

Think of it as

Think of Redis as one enormous Python dict shared by every process that connects to it, kept in memory for speed. r.set(key, value) and r.get(key) are that dict's [key] = value and [key] — except the value always travels over the network as bytes (or str, once decode_responses=True asks the client to decode it), and the key has to be unique across your whole application, not just one script.

python
import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)
r.set("session:42", "active")
r.get("session:42")          # 'active'
r.get("no_such_key")         # None

What we're doing: Write and read a string value, use SETNX to avoid overwriting an existing key, and INCR to update a counter without a race condition.

redis_strings.pypython
r.set("user:1001:name", "Priya Shah")
print(r.get("user:1001:name"))

r.set("page_views", 1)
r.incrby("page_views", 5)      # 1 from set, +5 here = 6
print(r.get("page_views"))

r.setnx("user:1001:name", "Someone Else")   # key exists — no-op
print(r.get("user:1001:name"))

r.append("user:1001:name", " (verified)")
print(r.get("user:1001:name"))
2
r.get returns the string exactly as stored — decode_responses=True on the client turns the raw bytes into a Python str.
7
setnx ("SET if Not eXists") only writes when the key is absent — since user:1001:name already exists, this call does nothing.
10
append adds to the end of the existing string value and creates the key if it was missing — it does not error.
Output
Priya Shah
6
Priya Shah
Priya Shah (verified)

Why this works: set/get map directly onto Redis storing one value per key. incrby is atomic on the server — Redis, not your Python process, does the read-add-write, so two clients calling it concurrently never lose an update the way a plain get-then-set would. setnx makes "create if missing" a single round trip instead of a check-then-act race.

Reading, modifying in Python, then writing back instead of using INCR

Wrong

python
count = r.get("hits")
count = int(count) + 1 if count else 1
r.set("hits", count)

Better

python
r.incr("hits")   # atomic on the server, no read-modify-write gap

What you see: Under concurrent requests, some increments silently disappear — the counter ends up lower than the number of increments actually issued.

Why: get-then-set is two separate round trips with a gap in between. If two processes both read the same value before either writes, both compute old_value + 1 and the second write clobbers the first — one increment is lost. r.incr runs entirely on the Redis server as a single operation, so there is no gap for another client's write to land in.

Redis as a key-value store

r.set("user:1001:name", "Priya Shah")

key → value, written in memory

Redis server

holds every key across every connected client

r.get("user:1001:name")

returns 'Priya Shah'

  1. r.set("user:1001:name", "Priya Shah") — key → value, written in memory
  2. Redis server — holds every key across every connected client
  3. r.get("user:1001:name") — returns 'Priya Shah'

Core string commands

Core string commands
Command (redis-py)EffectReturns
r.set(key, value)Writes value, overwriting any existing value at keyTrue
r.get(key)Reads the value at keythe value, or None if absent
r.mset({...}) / r.mget(*keys)Sets or gets several keys in one round tripTrue / list, with None for missing keys
r.incr(key) / r.incrby(key, n)Atomically increments an integer-valued key by 1 or nthe new integer value
r.append(key, value)Appends value to the end of an existing string, or creates itthe new string length
r.strlen(key)Returns the length of the string in bytesan int

Together

python
r.incr("retry_count")           # key did not exist: created at 1
r.incrby("retry_count", 4)      # now 5
r.mset({"a": "1", "b": "2"})
print(r.mget("a", "b", "missing"))   # ['1', '2', None]
print(r.strlen("retry_count"))       # 1  (the string "5" is 1 byte)

Remember: Every Redis value lives under a unique string key; r.set/r.get read and write it, and r.incr/r.incrby update a numeric one atomically without a read-modify-write race.

See also: ttl and expiration · hashes · atomic operations

Lists

standardbeginner

A Redis list holds an ordered sequence of strings under one key. r.rpush appends to the right end, r.lpush prepends to the left, and r.lrange(key, 0, -1) reads the whole thing back in order.

Think of it as

Think of a Redis list as a Python deque that lives on the server: appendleft is r.lpush, append is r.rpush, and popping from either end (r.lpop, r.rpop) is O(1) regardless of how long the list has grown. Reading a range with r.lrange is the one operation that gets slower the more of the list you ask for — indexing into the middle is not instant like a Python list.

python
r.rpush("recent_orders", "order:1", "order:2")   # append
r.lpush("recent_orders", "order:0")               # prepend
r.lrange("recent_orders", 0, -1)   # ['order:0', 'order:1', 'order:2']
r.lpop("recent_orders")            # 'order:0', removed
r.llen("recent_orders")            # remaining length

What we're doing: Build a small job queue: push work onto a list and pop it with a blocking read so a worker does not have to poll.

redis_lists.pypython
r.rpush("job_queue", "task:a", "task:b")

print(r.llen("job_queue"))            # 2
print(r.blpop("job_queue", timeout=1))   # ('job_queue', 'task:a')
print(r.lrange("job_queue", 0, -1))   # remaining: ['task:b']
1
rpush enqueues work at the tail — first task pushed is first to come out via lpop/blpop.
5
blpop blocks up to timeout seconds waiting for an element rather than returning None immediately on an empty queue — the standard pattern for a worker that should sleep, not poll.
Output
2
('job_queue', 'task:a')
['task:b']

Why this works: rpush/lpop make the list behave as a FIFO queue: work goes in at the tail and comes out at the head in the order it arrived. blpop turns "wait for work" into one blocking call instead of a client-side sleep-and-retry loop, so a worker sits idle at zero CPU cost until something is pushed.

Remember: r.rpush/r.lpush push onto either end in O(1); r.lrange(key, 0, -1) reads the whole list; r.blpop blocks for work instead of polling.

See also: key value and strings · sets and sorted sets

Sets and sorted sets

coreintermediate

A Redis set (r.sadd) holds unique unordered members with fast O(1) membership checks via r.sismember. A sorted set (r.zadd) is the same, but each member also carries a float score, and members always come back ordered by score.

Think of it as

A Redis set is a Python set() living on the server — sadd is add(), sismember is the in check, and sinter/sunion/sdiff are & / | / - between two sets, computed server-side so you never pull both sets into Python just to combine them. A sorted set (zset) is the same set, but every member is also a key in a dict of scores — it stays sorted by score at all times, which is what makes 'top 10 by score' a single O(log N) range read instead of a client-side sort.

python
r.sadd("tags:post42", "python", "redis", "backend")
r.sismember("tags:post42", "redis")   # True

r.zadd("leaderboard", {"alice": 100, "bob": 250})
r.zrevrange("leaderboard", 0, 0, withscores=True)  # [('bob', 250.0)]

What we're doing: Tag a post with a set, find overlapping tags between two posts, then rank users on a sorted-set leaderboard.

redis_sets.pypython
r.sadd("tags:post42", "python", "redis", "backend")
r.sadd("tags:post7", "python", "frontend")

print(r.sinter("tags:post42", "tags:post7"))   # shared tags

r.zadd("leaderboard", {"alice": 100, "bob": 250, "carol": 175})
r.zincrby("leaderboard", 50, "alice")          # alice now 150

print(r.zrevrange("leaderboard", 0, 0, withscores=True))  # current leader
print(r.zrank("leaderboard", "carol"))                     # ascending rank
1
sadd builds the tag set for post 42 — order does not matter and re-adding "python" later would be a no-op.
6
zadd sets three members with scores in one call; zincrby then updates one member's score atomically without reading it first.
9
zrevrange(key, 0, 0, withscores=True) reads just the single highest-scoring member — the current leaderboard leader.
Output
{'python'}
[('bob', 250.0)]
1

Why this works: sinter runs the set intersection on the server and returns only the result, rather than pulling both full sets into Python to compare — the same reason sunion/sdiff exist. zincrby is atomic for the same reason incr is: no read-modify-write gap for a concurrent update to land in, which matters for a shared leaderboard.

Set vs. sorted set

Set — r.sadd

  • +Unique, unordered members
  • +O(1) r.sismember membership check
  • +sinter/sunion/sdiff between sets
  • +Use for: tags, "seen" tracking, dedup

Sorted set — r.zadd

  • Unique members, each with a float score
  • Always readable in score order
  • zrevrange for "top N", zincrby to bump a score
  • Use for: leaderboards, ranked queues, rate windows
  • Set — r.sadd
    • Unique, unordered members
    • O(1) r.sismember membership check
    • sinter/sunion/sdiff between sets
    • Use for: tags, "seen" tracking, dedup
  • Sorted set — r.zadd
    • Unique members, each with a float score
    • Always readable in score order
    • zrevrange for "top N", zincrby to bump a score
    • Use for: leaderboards, ranked queues, rate windows

Using a list for membership checks instead of a set

Wrong

python
r.rpush("seen_ids_list", "id1", "id2", "id3")
"id2" in r.lrange("seen_ids_list", 0, -1)   # pulls the whole list to check one value

Better

python
r.sadd("seen_ids_set", "id1", "id2", "id3")
r.sismember("seen_ids_set", "id2")   # O(1), no data transferred back

What you see: Membership checks get slower as the collection grows, and every check transfers the entire list over the network first — a list has no server-side "is this in here" command.

Why: lrange has to fetch every element back to Python before you can test membership with in — that's O(N) data transfer and O(N) comparison, every single check. A set's sismember runs entirely on the server in O(1) and returns just a boolean, because sets are stored as a hash table internally, not a sequence.

Set and sorted-set commands

Set and sorted-set commands
Command (redis-py)StructureEffect
r.sadd(key, *members)setAdds members; duplicates are ignored
r.smembers(key)setReturns all members as a Python set
r.sismember(key, m)setO(1) check: is m in the set
r.sinter(k1, k2) / r.sunion / r.sdiffsetServer-side intersection / union / difference
r.zadd(key, {m: score})zsetAdds/updates member m with a float score
r.zrange(key, 0, -1, withscores=True)zsetMembers in ascending score order, with scores
r.zrevrange(key, 0, n)zsetTop n+1 members, highest score first
r.zscore(key, m) / r.zrank(key, m)zsetMember m's score, or its 0-based rank (ascending)

Together

python
r.sadd("active_users", "u1", "u2", "u3")
r.zadd("scores", {"u1": 10, "u2": 30, "u3": 20})
print(r.scard("active_users"))                       # 3
print(r.zrevrange("scores", 0, 1, withscores=True))   # top 2, highest first
print(r.zrangebyscore("scores", 15, 100))             # members scored 15-100

Remember: r.sadd/r.sismember give O(1) unique-membership; r.zadd/r.zrevrange keep members ordered by a float score — reach for a zset the moment "top N" or "rank" enters the requirement.

See also: lists · hashes · atomic operations

Hashes

standardbeginner

A Redis hash stores field-value pairs under one key, like a dict nested under a single top-level key. r.hset("user:1001", mapping={"name": "Priya", "logins": 4}) writes it; r.hget("user:1001", "name") reads one field back.

Think of it as

Think of a hash as one Python dict living under a single Redis key, instead of one Redis key per field. r.hset(key, mapping={...}) is dict.update(); r.hget(key, field) is dict[field]; r.hgetall(key) is dict(d) — the whole thing back at once. The advantage over separate keys (user:1001:name, user:1001:logins) is that hgetall/hdel/hincrby operate on the whole object as a unit, and Redis stores small hashes more compactly than the same fields as separate top-level keys.

python
r.hset("user:1001", mapping={"name": "Priya Shah", "logins": 4})
r.hget("user:1001", "name")     # 'Priya Shah'
r.hgetall("user:1001")          # {'name': 'Priya Shah', 'logins': '4'}
r.hincrby("user:1001", "logins", 1)

What we're doing: Store a user profile as a hash, read a subset of fields, increment a counter field, and remove one field.

redis_hashes.pypython
r.hset("user:1001", mapping={
    "name": "Priya Shah", "email": "priya@example.com", "logins": 4,
})

print(r.hget("user:1001", "name"))
r.hincrby("user:1001", "logins", 1)
print(r.hget("user:1001", "logins"))
r.hdel("user:1001", "email")
print(r.hexists("user:1001", "email"))
1
A single hset with mapping= writes all three fields in one round trip, instead of three separate string keys.
6
hincrby treats the "logins" field as an integer and adds to it atomically, the same guarantee incr gives a top-level string key.
8
hdel removes just the email field — name and logins are untouched, and the key itself still exists.
Output
Priya Shah
5
False

Why this works: A hash groups related fields under one key, so hgetall/hdel/hincrby operate on the whole object without needing to know every field's name up front, and without the client stitching together several separate GETs into one object.

Remember: r.hset(key, mapping={...}) writes several fields under one key; r.hget/r.hgetall read one field or all of them; r.hincrby updates a numeric field atomically.

See also: key value and strings · sets and sorted sets

TTL and expiration

corebeginner

A TTL (time to live) is a countdown Redis attaches to a key; when it hits zero, Redis deletes the key by itself. r.expire("otp:42", 300) gives a key 300 seconds to live; r.ttl("otp:42") checks how many seconds remain.

Think of it as

Think of a TTL as an alarm clock attached to a key, not a property you have to check yourself. You set it once with expire (or ex= on set), and Redis deletes the key the moment it fires — no polling, no cron job, no code of yours has to run. r.ttl(key) just asks 'how much time is left on the alarm' at any point; -1 means the key exists with no alarm set, -2 means the key does not exist at all (already fired, or never existed).

python
r.set("otp:user42", "914213")
r.expire("otp:user42", 300)     # expires in 300 seconds
r.ttl("otp:user42")             # 300

r.set("cache:homepage", "<html>...</html>", ex=60)   # value + TTL together
r.ttl("cache:homepage")         # 60

What we're doing: Set a TTL two ways, read it back, remove it with persist, and check a millisecond TTL actually expires the key.

redis_ttl.pypython
r.set("cache:homepage", "<html>...</html>", ex=60)
print(r.ttl("cache:homepage"))          # 60

print(r.persist("cache:homepage"))      # True: TTL removed
print(r.ttl("cache:homepage"))          # -1: exists, no TTL

r.set("short_lived", "x", px=50)        # 50ms TTL
time.sleep(0.1)
print(r.get("short_lived"))             # None: already expired
print(r.exists("short_lived"))          # 0
1
ex=60 sets the value and a 60-second TTL in the same round trip as set — the usual way to write a cache entry.
5
persist strips the TTL entirely; the key and its value are unaffected, but it will not auto-expire anymore.
9
After sleeping past the 50ms px= TTL, the key is gone — get returns None exactly as it would for a key that never existed.
Output
60
True
-1
None
0

Why this works: Redis tracks each key's expiry internally and removes it lazily on access or via a background sweep — your code never has to check the time itself. ex=/px= on set() is the same expire call folded into the write so there is no window where the key exists without its intended TTL.

A plain SET silently clearing an existing TTL

Wrong

python
r.expire("cache:report", 120)
r.set("cache:report", "overwritten-again")   # TTL is gone now
print(r.ttl("cache:report"))                 # -1, not 120

Better

python
r.expire("cache:report", 120)
r.set("cache:report", "overwritten-with-keepttl", keepttl=True)
print(r.ttl("cache:report"))                 # 120, preserved

What you see: A cache key that used to expire on schedule starts living forever after any code path updates it with a plain set() — stale data never gets cleared.

Why: set() without ex=/px=/exat=/keepttl= replaces the key entirely, including dropping any TTL it had — this is documented SET behavior, not a bug. keepttl=True is the explicit opt-in to keep the existing expiry when you only mean to update the value.

A key with a 300-second TTL
  1. t=0s

    r.set("otp:42", "914213", ex=300)

    value written, 300s countdown starts

  2. t=299s

    r.ttl("otp:42") → 1

    still readable, one second left

  3. t=300s

    key expires

    Redis deletes it automatically

  4. t=301s

    r.get("otp:42") → None

    r.ttl("otp:42") → -2 (does not exist)

  1. t=0s: r.set("otp:42", "914213", ex=300) — value written, 300s countdown starts
  2. t=299s: r.ttl("otp:42") → 1 — still readable, one second left
  3. t=300s: key expires — Redis deletes it automatically
  4. t=301s: r.get("otp:42") → None — r.ttl("otp:42") → -2 (does not exist)

Setting and reading TTLs

Setting and reading TTLs
Command (redis-py)UnitEffect
r.expire(key, n)secondsSet/replace TTL to n seconds from now
r.pexpire(key, n)millisecondsSame, with millisecond precision
r.expireat(key, unix_ts)secondsSet TTL to expire at an absolute Unix timestamp
r.ttl(key) / r.pttl(key)seconds / msRemaining time; -1 no TTL, -2 key missing
r.persist(key)Removes the TTL — key lives forever until deleted
r.set(key, value, ex=n, keepttl=True)secondsex= sets a new TTL; keepttl=True keeps the key's existing TTL instead

Together

python
r.set("session:abc", "data")
r.pexpire("session:abc", 120000)      # 120000ms = 120s
print(r.pttl("session:abc"))          # 120000
r.expireat("session:abc", int(time.time()) + 300)
print(r.ttl("session:abc") > 0)       # True

Remember: A TTL is a self-deleting countdown on a key (r.expire/r.ttl); a plain r.set() clears it unless you pass ex=/keepttl=True — -1 means no TTL, -2 means the key is gone.

See also: caching · key value and strings · distributed locks

Atomic operations

coreintermediate

A single Redis command like r.incr always runs atomically — no other client's command can interleave in the middle of it. r.pipeline(transaction=True) groups several commands into one all-or-nothing MULTI/EXEC block when you need more than one command to be atomic together.

Think of it as

Redis is single-threaded for command execution, so any one command — incr, hset, sadd, whatever — already runs start-to-finish with nothing else able to interleave. The question is only what happens when your logic needs MORE than one command to be atomic together. A pipeline batches commands into one network round trip (a performance win, not automatically atomic); wrapping it with transaction=True (MULTI/EXEC) makes the whole batch atomic; adding watch(key) before that makes it 'run this transaction, but abort if key changed since I looked at it' — optimistic locking instead of a lock.

python
pipe = r.pipeline(transaction=True)
pipe.set("balance:acct1", 100)
pipe.decrby("balance:acct1", 30)
pipe.get("balance:acct1")
results = pipe.execute()   # [True, 70, '70'] — one result per queued command

What we're doing: Decrement stock safely under concurrency using WATCH/MULTI/EXEC, retrying if another client changes the key first.

redis_atomic.pypython
import redis

r.set("stock:sku9", 5)
with r.pipeline() as pipe:
    while True:
        try:
            pipe.watch("stock:sku9")
            current = int(pipe.get("stock:sku9"))
            pipe.multi()
            pipe.set("stock:sku9", current - 1)
            pipe.execute()
            break
        except redis.WatchError:
            continue   # someone else changed it first — recompute and retry

print(r.get("stock:sku9"))
4
watch tells Redis "abort my transaction if this key changes before I EXEC" — read happens after watch, so nothing is missed.
8
pipe.multi() switches the pipeline into queuing mode — subsequent commands are queued, not sent, until execute().
10
execute() sends MULTI + the queued commands + EXEC as one atomic block, but only if stock:sku9 has not changed since watch().
13
A WatchError means another client wrote to stock:sku9 first — the loop retries with a freshly read current value instead of overwriting blind.
Output
4

Why this works: watch/multi/execute is optimistic concurrency control: instead of blocking other clients out with a lock, Redis lets everyone proceed and only refuses to commit if the watched key actually changed underneath you — cheaper than locking when conflicts are rare, at the cost of a retry loop when they are not.

WATCH / MULTI / EXEC — optimistic locking
if keychanged

pipe.watch("stock:sku9")

start monitoring the key

read current value

decide the new value in Python

pipe.multi() + queue writes

pipe.execute()

runs atomically — unless key changed

WatchError → retry the loop

  • pipe.watch("stock:sku9") — start monitoring the key
    • leads to read current value
  • read current value — decide the new value in Python
    • leads to pipe.multi() + queue writes
  • pipe.multi() + queue writes
    • leads to pipe.execute()
  • pipe.execute() — runs atomically — unless key changed
    • on error, leads to WatchError → retry the loop (if key changed)
  • WatchError → retry the loop

Read-modify-write without WATCH, assuming no one else writes concurrently

Wrong

python
val = int(r.get("inventory:sku7"))   # read
r.set("inventory:sku7", val - 1)      # write — gap between read and write

Better

python
r.decr("inventory:sku7")   # atomic for the single-key case — no gap at all

What you see: Under concurrent decrements, the final count is higher than it should be — some decrements are overwritten and lost, the same failure as the r.incr mistake but in the other direction.

Why: Between the get and the set, another client's own get-modify-set can run and be overwritten by yours. For a single key with a simple increment/decrement, Redis's atomic incr/decr/incrby/decrby remove the gap entirely — reach for WATCH/MULTI/EXEC only when the logic between read and write needs more than one key or is too complex for a single atomic command.

Remember: Every single Redis command is already atomic; use r.pipeline(transaction=True) for MULTI/EXEC, and pipe.watch(key) for optimistic locking when the write depends on a value you just read.

See also: key value and strings · distributed locks

Pub/Sub and Streams

standardintermediate

Pub/Sub broadcasts a message to whoever is subscribed right now — r.publish(channel, msg) — with no history kept. A Stream (r.xadd) is a persisted, ordered log a consumer can read from any point, including messages sent before it started listening.

Think of it as

Pub/Sub is a live radio broadcast: r.publish sends to every currently-tuned-in subscriber (pubsub.subscribe) and nobody who was not listening at that moment gets it — there is no rewind. A Stream is a shared, append-only logbook: r.xadd writes an entry with an auto-generated ID, and any reader can start from the beginning, the end, or any ID in between with r.xrange or r.xread — including replaying history a Pub/Sub message could never give back. Consumer groups (r.xreadgroup) split a stream's entries across multiple workers, each entry going to exactly one consumer in the group.

python
# Pub/Sub
pubsub = r.pubsub()
pubsub.subscribe("notifications")
r.publish("notifications", "order:1001 shipped")
pubsub.get_message(timeout=1)   # {'type': 'message', 'channel': 'notifications', 'data': '...'}

# Streams
r.xadd("events:orders", {"order_id": "1001", "status": "created"})   # '<ts>-0'
r.xrange("events:orders", "-", "+")   # every entry, in ID order

What we're doing: Publish and receive one Pub/Sub message, then append two entries to a stream and read them back with a consumer group.

redis_pubsub_streams.pypython
pubsub = r.pubsub()
pubsub.subscribe("notifications")
print(pubsub.get_message(timeout=1))   # subscribe confirmation, not a real message

r.publish("notifications", "order:1001 shipped")
print(pubsub.get_message(timeout=1))   # the actual message

msg_id = r.xadd("events:orders", {"order_id": "1001", "status": "created"})
r.xadd("events:orders", {"order_id": "1002", "status": "created"})
print(r.xlen("events:orders"))

r.xgroup_create("events:orders", "workers", id="0")
read = r.xreadgroup("workers", "consumer-1", {"events:orders": ">"}, count=1)
print(len(read[0][1]))   # entries delivered to consumer-1 in this read
3
The first get_message() after subscribe() is always the subscription confirmation itself, not application data — a common first-message surprise.
8
xadd returns an ID combining the millisecond timestamp and a sequence number, guaranteeing IDs are strictly increasing even for entries added in the same millisecond.
13
xreadgroup with ">" delivers only entries not yet claimed by any consumer in the "workers" group — a second consumer reading the same group would get the next unclaimed entry, not a repeat.
Output
{'type': 'subscribe', 'pattern': None, 'channel': 'notifications', 'data': 1}
{'type': 'message', 'pattern': None, 'channel': 'notifications', 'data': 'order:1001 shipped'}
2
1

Why this works: Pub/Sub delivers only to whoever is subscribed at publish time and keeps nothing — fine for "notify anyone currently listening," wrong for anything that must not be missed. Streams persist every entry with a durable ID, so xrange/xreadgroup can always retrieve history a late-joining or restarted consumer would otherwise have missed under Pub/Sub.

Remember: Pub/Sub (r.publish) is fire-and-forget with zero history; Streams (r.xadd/r.xreadgroup) persist every entry and let a consumer group split and replay them — pick Streams whenever a missed message is not acceptable.

See also: atomic operations · caching

Distributed locks

coreadvanced

A distributed lock uses r.set(key, token, nx=True, px=ttl_ms) so only one process can hold the key at a time. Releasing it must check the stored token matches yours before deleting — otherwise you can delete a lock someone else already acquired after yours expired.

Think of it as

A single-instance Redis lock is a claim ticket: SET key token NX PX ttl writes the key only if it is empty (NX) and gives it an expiry (PX) so a crashed holder cannot lock the resource forever. The token is your proof of ownership — releasing the lock means "delete this key, but only if the value still matches the token I set," checked and deleted in one atomic step, never as two separate commands.

python
import uuid
token = str(uuid.uuid4())
acquired = r.set("lock:invoice:99", token, nx=True, px=5000)   # True or None

release_script = """
if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end
"""
release = r.register_script(release_script)
release(keys=["lock:invoice:99"], args=[token])   # 1 if released, 0 if not yours

What we're doing: Acquire a lock, prove a second acquire attempt is refused while it is held, then release it safely with the token-checking Lua script.

redis_lock.pypython
import uuid

token = str(uuid.uuid4())
acquired = r.set("lock:invoice:99", token, nx=True, px=5000)
print(acquired)                 # True — lock acquired

blocked = r.set("lock:invoice:99", "other-token", nx=True, px=5000)
print(blocked)                  # None — someone else already holds it

release_script = """
if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end
"""
release = r.register_script(release_script)
result = release(keys=["lock:invoice:99"], args=[token])
print(result)                   # 1 — released, token matched
print(r.get("lock:invoice:99")) # None — key is gone
3
nx=True makes this a "set only if absent" — the return value True confirms client A won the race and now holds the lock.
6
A second nx=True attempt on the same key while it exists returns None (falsy) — this is how another process learns it did NOT get the lock, without blocking.
17
The Lua script runs GET and DEL as one atomic server-side step — no other client can slip a write in between the check and the delete.
Output
True
None
1
None

Why this works: nx=True turns "claim this resource" into a single atomic command instead of a check-then-set race. The release script closes the matching gap on the other end: without it, a compare done in Python (GET, compare, then DEL as two separate commands) has a window where another client could acquire the lock in between your GET and your DEL — and your DEL would then delete a lock that is no longer yours.

Releasing with GET then DEL as two separate commands instead of one atomic script

Wrong

python
if r.get("lock:job:5") == token:
    r.delete("lock:job:5")   # gap between GET and DEL — another client can slip in here

Better

python
release(keys=["lock:job:5"], args=[token])   # GET+DEL run as one atomic Lua script

What you see: No exception — the bug is silent. Under the right timing, client A's DEL removes a lock that Redis had already auto-expired and client B had since legitimately acquired, so B loses mutual exclusion without either side seeing an error.

Why: GET and DEL as two separate commands are each atomic individually, but nothing is atomic about the two together — Redis can run another client's command in between them. If A's lock happens to expire (or A's own release runs late) right as B acquires the same key, A's delayed DEL can remove B's active lock. Wrapping the check-and-delete in one Lua script closes that gap because Redis runs the whole script as a single atomic unit.

Safe acquire and release
Client A
Redis
Client B
  1. 1. SET lock:invoice:99 token-A NX PX 5000succeeds — key was absent
  2. 2. SET lock:invoice:99 token-B NX PX 5000fails — key already set, returns nil
  3. 3. EVAL release_script KEYS[1]=lock:invoice:99 ARGV[1]=token-Achecks value == token-A before DEL
  4. 4. DEL succeeds — token matched
  1. Client A → Redis: SET lock:invoice:99 token-A NX PX 5000 (succeeds — key was absent)
  2. Client B → Redis: SET lock:invoice:99 token-B NX PX 5000 (fails — key already set, returns nil)
  3. Client A → Redis: EVAL release_script KEYS[1]=lock:invoice:99 ARGV[1]=token-A (checks value == token-A before DEL)
  4. Redis → Client A: DEL succeeds — token matched

Remember: Acquire with r.set(key, token, nx=True, px=ttl_ms); release by checking the token and deleting in one atomic script — and treat a single Redis instance as fault-tolerant only for low-stakes locks, not ones where a double-acquire is unacceptable.

See also: atomic operations · ttl and expiration

Caching

standardintermediate

Cache-aside means checking Redis first, and only doing the expensive lookup (a database query, an API call) if the value is not there — then writing the result into Redis with a TTL so it expires instead of going stale forever.

Think of it as

Think of Redis as a sticky note you check before doing expensive work: look at the note first (r.get); if it already has the answer, use it and skip the work entirely — a cache hit. If it is blank (a cache miss), do the real work, write the answer onto the note, and set an expiry on the note so it does not still say the same thing a week from now. The TTL is what keeps a cache from becoming a second, silently-wrong source of truth.

python
def get_user_profile(user_id, db_lookup):
    cache_key = f"cache:user_profile:{user_id}"
    cached = r.get(cache_key)
    if cached is not None:
        return cached, "HIT"
    value = db_lookup(user_id)      # the slow path
    r.set(cache_key, value, ex=300)  # cache for 5 minutes
    return value, "MISS"

What we're doing: Implement cache-aside for a slow lookup and confirm the first call is a miss (does the real work) while the second is a hit (skips it).

redis_cache_aside.pypython
def get_user_profile(user_id, db_lookup):
    cache_key = f"cache:user_profile:{user_id}"
    cached = r.get(cache_key)
    if cached is not None:
        return cached, "HIT"
    value = db_lookup(user_id)
    r.set(cache_key, value, ex=300)
    return value, "MISS"

def fake_db_lookup(user_id):
    return f"profile-data-for-{user_id}"

print(get_user_profile("1001", fake_db_lookup))
print(get_user_profile("1001", fake_db_lookup))   # same call, now cached
1
db_lookup is passed in rather than hardcoded so the same cache-aside shape works for a database query, an API call, or any other expensive function.
4
A cache hit returns immediately — fake_db_lookup (the 'slow' path) never runs a second time.
7
ex=300 caps how long a stale profile can be served after the underlying data changes — 5 minutes here, tuned to how fresh the data needs to be.
Output
('profile-data-for-1001', 'MISS')
('profile-data-for-1001', 'HIT')

Why this works: The first call finds nothing under cache:user_profile:1001, so it pays the full db_lookup cost and populates the cache. The second call finds the cached value and returns immediately — the pattern's entire benefit is that a repeat lookup for the same key skips the expensive path, at the cost of possibly serving data up to `ex` seconds old.

Remember: Cache-aside: check Redis, do the expensive work only on a miss, then r.set(key, value, ex=ttl) — the TTL is what stops a cache from quietly going stale forever.

See also: ttl and expiration · key value and strings

Advertisement