beginnerintermediateadvanced

System Design · Section 14

Stateless Architecture

Level: intermediate
Read: 18 min
Concepts: 3

A stateless application instance holds no unique local data, which makes it disposable — safe to kill, replace, auto-scale and redeploy without data loss. Whatever must survive past one request goes to a database, cache, object storage or durable queue instead, and a distributed deployment specifically needs a plan for session management: a shared store that revokes instantly, or a signed token that skips the lookup at the cost of easy revocation.

This section

Stateful vs Stateless Architecture - System Design BasicsCloud Advocate

What is true here

A disposable instance holds no unique state and can be killed/replaced at any time with no data loss.
Shared state belongs in a database, cache, object storage or durable messaging system — never only in one instance.
Session data across a distributed deployment goes in a shared store (instant revocation, needs a lookup) or a signed token (no lookup, hard to revoke early).
Never treat local filesystem or in-memory state as the sole source of truth once requests can hit multiple instances.

What you will be able to do

Explain what makes an application instance disposable and why that matters for auto-scaling and deploys
Route each kind of state (session, file, queued work, application data) to the right shared store
Choose between a shared session store and a signed token based on the revocation requirement

Staying disposable

What disposability requires, and where state goes once it can no longer live on the instance itself.

Disposable application instances

coreintermediate

A disposable instance can be killed and replaced at any moment without losing anything important — because nothing important lives only on that instance. This is what makes auto-scaling, rolling deploys and crash recovery all simple: killing an instance is a routine event, not an emergency.

Think of it as

Think of a disposable instance like a rental car rather than a personal vehicle. You do not keep anything irreplaceable in a rental car, because you might swap it for a different one tomorrow. An application instance built the same way — no irreplaceable data living only there — can be swapped, restarted, or terminated without anyone needing to plan around it.

text

disposable instance = no unique local state
                     + config/state from shared sources
                     + safe to kill and replace anytime

What we're doing: Show an auto-scaling group only working safely because its instances are disposable.

autoscaling-disposability.txttext

Traffic spikes at 9am. Auto-scaler adds 5 new
instances to the pool.

Traffic drops at 6pm. Auto-scaler terminates those
5 instances to save cost.

This only works safely because:
  - the new instances needed no manual setup — they
    boot from the same image and read the same
    shared config
  - terminating an instance loses nothing, because
    session data, uploads and cache all live in
    shared stores, not on the instance itself

8: Both directions of auto-scaling — adding and removing — assume disposability.
11: This is the actual mechanism: identical boot, no unique setup, so scaling up needs no manual step.

Why this works: Auto-scaling terminates instances routinely as part of normal operation — a design that is not disposability-safe will lose data every time the scaler shrinks the pool.

Writing uploaded files to an instance's local disk

Wrong

text

fs.writeFileSync('/var/app/uploads/' + filename, data);
// only exists on this one instance's disk

Better

text

await s3.putObject({ Bucket: 'uploads', Key: filename, Body: data });
// durable, and any instance can read it back

What you see: An uploaded file "disappears" the next time a user's request happens to land on a different instance, or entirely when the instance that received the upload is later terminated by the auto-scaler.

Why: Local disk is exactly the kind of unique, unreplicated state that makes an instance non-disposable. Moving it to shared, durable storage (object storage, a database) is what restores disposability.

Disposable ("cattle") vs. "pet" instances

Disposable

+No data lives only on this instance
+Crash → auto-restart, no incident
+Enables auto-scaling and rolling deploys

"Pet"

−Has unique, hand-configured local state
−Crash → paged, manual recovery
−Cannot be freely killed and replaced

Disposable vs "pet" instances

Disposable vs "pet" instances
Property	Disposable ("cattle")	Non-disposable ("pet")
Loses data on kill?	No — nothing important lives only there	Yes — has unique local state
Replaced by	An identical new instance, automatically	Manual recovery, often paged
Crash response	Auto-restart, no incident	Incident — this specific instance is gone
Enables	Auto-scaling, rolling deploys	Neither, safely

Together

text

Disposable instance crashes:
  orchestrator notices the health check fails
  → kills the instance, starts a fresh one
  → fresh instance reads config + connects to the
    same shared database/cache everyone else uses
  → back to full capacity in seconds, no data lost

"Pet" instance crashes:
  on-call is paged
  → someone manually restores it from a backup
  → any data that only lived on that instance since
    the last backup is gone

Remember: A disposable instance holds no unique state and can be killed and replaced at any time — the precondition for auto-scaling, rolling deploys and self-healing crash recovery.

Moving shared state out of the instance

coreintermediate

To keep instances disposable, anything that needs to survive past a single request or be visible to every instance has to live somewhere shared: a database, a cache like Redis, object storage like S3, or a durable message queue — never only in one instance's memory or local disk.

Think of it as

Think of a shared filing cabinet in a central office versus a note kept in one employee's desk drawer. Anyone in the office can retrieve something from the shared cabinet, and it survives even if that particular employee goes home. A note in someone's personal drawer is invisible to everyone else and gone the moment that employee is unavailable.

text

shared state → database | cache | object storage | queue
                (never: this instance's memory or local disk)

What we're doing: Show a session lookup working correctly across instances only because the session lives in a shared cache.

shared-session-store.txttext

Login request hits instance A:
  A generates a session ID, writes
  { userId: 42, role: 'admin' } to Redis under that ID,
  returns the session ID as a cookie.

Next request from the same browser hits instance B
(load balancer routed it differently this time):
  B reads the same session ID's cookie, looks it up
  in the SAME shared Redis instance A wrote to,
  gets { userId: 42, role: 'admin' } back.

Instance B never talked to instance A directly —
they only share the same external store.

6: The load balancer is free to route this request anywhere — that's the point.
9: Instance B finds the session because it looked in the shared store, not because it asked instance A.

Why this works: This is what "any instance can serve any request" actually depends on in practice — a shared store both instances read and write, instead of one instance holding the only copy.

Using local filesystem or in-memory state as the sole source of truth

Wrong

text

const cache = new Map();  // in-process only
cache.set(userId, profile);

Better

text

await redis.set(`profile:${userId}`, JSON.stringify(profile));
// visible to every instance behind the load balancer

What you see: Data written by one request is invisible to a later request that happens to land on a different instance — intermittent, load-balancer-routing-dependent bugs that don't reproduce consistently.

Why: Local filesystem or in-process memory is only visible to the one instance that wrote it. The moment requests can land on more than one instance — which is true of almost any load-balanced system — that local state stops being a reliable source of truth.

Both instances share the same session store

Instance A

writes the session

Redis

shared session store

Instance B

reads the same session

Where state goes, by kind

Where state goes, by kind
Kind of state	Wrong place	Right place
User session	One instance's memory	Shared cache (Redis) or signed token
Uploaded file	One instance's local disk	Object storage (S3-style)
Work queued for later	One instance's in-memory list	A durable queue (SQS, Kafka, RabbitMQ)
Application data	One instance's memory/disk	A database

Together

text

Request arrives at instance A, uploads a file and
queues a resize job.

Wrong: file saved to instance A's disk, job pushed to
  an in-memory array on instance A.
  → if the resize worker request lands on instance B,
    it has no file and no job to process.

Right: file saved to S3, job pushed to a durable queue.
  → instance B (or any instance) can read the file
    from S3 and pop the job from the queue — which
    instance handles which request no longer matters.

Remember: Anything that needs to survive past one request or be visible to every instance goes in a database, cache, object storage or durable queue — never only in one instance's memory or local disk.

Sessions across a fleet

The specific problem of keeping a login session valid no matter which instance handles the next request.

Session management in distributed deployments

standardintermediate

With multiple stateless instances behind a load balancer, a user's session has to be readable by whichever instance handles their next request. Two dominant approaches: a shared session store (Redis, a database) every instance reads from, or a self-contained signed token (a JWT) the client holds, needing no shared store lookup at all.

Think of it as

A shared session store is like a coat check ticket: the ticket itself just has a number, and any employee at the counter can look up your coat using it, because the coat is kept in one shared room. A signed token is like a wristband stamped with your seat number and a tamper-evident seal: any usher can read it directly and knows it is genuine without checking any shared list at all.

text

shared store: client holds opaque ID → server looks up data
signed token: client holds signed data  → server verifies, no lookup

What we're doing: Show the revocation trade-off concretely — the actual reason to pick one approach over the other.

revocation-tradeoff.txttext

User's account is compromised; security needs to
force-logout that user immediately.

Shared session store:
  DELETE session:abc123 from Redis.
  Next request with that session ID fails the lookup
  → user is logged out on their very next request.

Signed token (JWT), no server-side store:
  There is nothing to delete — the token is
  self-contained and cryptographically valid until
  its own expiry, even if issued an hour ago.
  Forcing immediate logout requires either a short
  expiry + refresh flow, or a separate revocation
  list — extra machinery the shared-store approach
  gets for free.

6: Deleting one entry is enough — this is the shared store's main structural advantage.
10: A pure signed token has no revocation hook at all; anything added here is extra infrastructure on top of the basic token approach.

Why this works: This is the concrete trade-off that decides between the two approaches in practice — how urgently a session might ever need to be revoked before its natural expiry.

Choosing pure signed tokens for a system that needs instant revocation

Wrong

text

// long-lived JWT, no revocation mechanism,
// used for a banking app's session

Better

text

// short-lived JWT + refresh token checked
// against a revocation list, OR a shared
// session store for anything security-sensitive

What you see: A compromised or logged-out session remains valid and usable for as long as the token's expiry window, with no way to cut it off sooner — discovered only after an incident where "force logout everyone" turned out to be impossible.

Why: A pure signed token trades instant revocability for avoiding a server-side lookup. That trade only makes sense where the session's blast radius from staying valid a bit longer is acceptable — not for anything where an immediate cutoff might be needed.

Shared session store vs. signed token

Shared store

+Client holds an opaque session ID
+One lookup per request
+Instant revocation — delete the entry

Signed token (JWT)

−Session data lives in the token itself
−No lookup — just verify the signature
−Hard to revoke before its own expiry

Shared store vs signed token

Shared store vs signed token
Property	Shared session store	Signed token (JWT)
Session data lives	Server-side store (Redis, DB)	Client-side, inside the token
Per-request cost	One lookup against the store	Verify a signature, no lookup
Instant revocation	Easy — delete the store entry	Hard — token is valid until it expires
Token size	Small (just an opaque ID)	Larger — carries the actual claims

Together

text

Shared store:
  cookie: session_id=abc123
  server: redis.get("session:abc123") → { userId: 42 }

Signed token:
  cookie: token=eyJhbGciOiJIUzI1NiJ9...
  server: verify signature, decode claims directly
          → { userId: 42, exp: 1699999999 }
          (no store lookup needed at all)

Remember: A shared session store needs a per-request lookup but revokes instantly; a signed token skips the lookup but cannot be revoked before its own expiry without extra machinery — the application instance stays stateless either way.