ElastiCache / Managed Caching

What we're doing: Pick the right engine for two different caching needs on the same application.

1. Cache rendered HTML fragments, purely by key, across 12 large nodes
   → Memcached: simple values, multithreaded, scales out easily

2. Real-time gaming leaderboard with live rank updates and a
   "player joined" notification stream
   → Valkey or Redis OSS: sorted sets (ZADD/ZREVRANGEBYSCORE) for
     ranking, pub/sub for the notification stream

1

Rendered HTML fragments are opaque blobs looked up by key — Memcached's simple string values and multithreading are the better fit, and there is no data structure or failover requirement to justify Redis/Valkey.

4

A leaderboard needs a data structure the application would otherwise reimplement in every server, and the notification stream needs pub/sub — both are native to Valkey/Redis OSS and absent from Memcached.

Why this works: The two workloads look similar on the surface ("cache some data"), but one needs nothing beyond a key-value store while the other needs server-side data structures and messaging — the engine choice follows directly from which capabilities the workload actually uses, not from which engine is "better" in general.

What we're doing: See why a cache-aside read on a cold cache costs three round trips, and how a TTL bounds staleness afterward.

get_customer(id):
    record = cache.get(id)
    if record is None:                 # cache miss
        record = db.query(id)           # trip 2: database
        cache.set(id, record, ttl=300)  # trip 3: write back, expires in 5 min
    return record                       # trip 1 was the cache.get above

1

Every read goes through the cache first — this function is called exactly the same way whether the data is cached or not.

5

The 300-second TTL means this key self-expires — the next read after 5 minutes is guaranteed to be a miss and re-fetch current data, bounding how stale it can ever get.

Why this works: A cache miss on cache-aside is not free — it is strictly slower than a cache hit because it does the database query anyway, plus a cache write. TTL is what keeps a cache-aside cache from serving arbitrarily old data forever between misses.

What we're doing: See the SET NX PX lock pattern used to let only one process rebuild an expired hot key.

get_or_rebuild(key):
    value = cache.get(key)
    if value is not None:
        return value

    token = random_string()
    if cache.set("lock:" + key, token, nx=True, px=5000):   # acquired
        value = db.query(key)
        cache.set(key, value, ttl=300)
        release_lock("lock:" + key, token)
        return value
    else:                                                    # someone else holds it
        sleep(0.05)
        return get_or_rebuild(key)   # retry — usually finds the rebuilt key

2

A hot key's expiry means many concurrent callers reach this exact point at once — without a lock, every one of them would fall through to the database query below.

8

Only the first caller to win the NX SET rebuilds the key; every other concurrent caller gets a false return and retries shortly, usually finding the key already rebuilt by then.

Why this works: The lock turns "every concurrent miss queries the database" into "one miss queries the database, the rest wait a few milliseconds and re-read the cache" — this is the entire value of stampede prevention, and it is a database-load optimization, not a correctness mechanism.

What we're doing: Implement a fixed-window rate limiter with an atomic counter and a TTL.

allow_request(user_id, limit=100, window_seconds=60):
    key = "ratelimit:" + user_id + ":" + current_minute()
    count = cache.incr(key)          # atomic increment, starts at 1 if new
    if count == 1:
        cache.expire(key, window_seconds)   # set TTL only on the first hit
    return count <= limit

2

Bucketing the key by the current minute creates a new counter for each fixed window automatically — no cleanup logic needed once the TTL below expires it.

3

INCR is atomic, so concurrent requests from the same user in the same window never race each other into an incorrect count — the increment and the read happen as one operation.

Why this works: Redis/Valkey's INCR is a single atomic operation on the server, which is what makes it safe for many concurrent requests to share one counter without a separate lock — the TTL on the same key makes the counter self-cleaning per window.

What we're doing: See what actually happens, step by step, when only the primary node fails in a Multi-AZ-enabled replication group.

1. Primary node fails (health check / connectivity loss)
2. Replica with the LEAST replication lag is promoted to primary
   -> writes can resume once promotion completes, typically a few seconds
3. ElastiCache propagates the promoted replica's DNS to the primary endpoint
   -> application keeps writing to the same endpoint, no code change
4. A replacement read replica is launched in the failed primary's AZ
5. The replacement syncs from the new primary

2

AWS documentation is explicit that the replica with the least replication lag is chosen, specifically to minimize how much data was not yet replicated when the primary failed.

3

The primary endpoint's DNS name is what gets re-pointed — this is why an application writing to the primary endpoint (not an individual node's address) needs no reconfiguration during failover.

Why this works: The whole design goal of Multi-AZ is to replace "recreate and reprovision a new primary from scratch" (slow, and requires an application endpoint change) with "promote an already-running, already-synced replica" (a few seconds, same endpoint) — understanding which replica gets chosen and why the endpoint does not change is what "failover implications" actually means in practice.

What we're doing: Contrast a design that treats the cache as expendable with one that accidentally makes it the only copy of data.

# Safe: cache is a derived, rebuildable copy
save_order(order_id, data):
    db.insert(order_id, data)          # the real, durable copy
    cache.set(order_id, data, ttl=600)  # a disposable accelerator

# Unsafe: cache is the only copy
record_cart_item(session_id, item):
    cache.append(session_id, item)     # nothing else stores this

1

The database write happens first and unconditionally — losing the cache entry afterward loses nothing, because the order still exists in the real system of record.

5

Nothing durable ever receives this write — if the node holding this key is replaced, restarted, or evicts the key under memory pressure, the cart item is gone permanently with no way to recover it.

Why this works: The difference is not about which API calls were used — it is about whether a second, durable copy of the data exists anywhere else. The safe version can lose the cache at any moment with zero data loss; the unsafe version cannot.