beginnerintermediateadvanced

System Design · Section 59

Encryption

Level: intermediate
Read: 20 min
Concepts: 3

Encryption in transit (TLS) and encryption at rest are two separate protections for two separate windows in data's life — TLS guards a network hop and does nothing once data is written to disk, so a system needs both or a stolen backup bypasses TLS entirely. Passwords are hashed, never encrypted, because hashing is one-way (nothing, including the system itself, can recover the original) while encryption is deliberately reversible by whoever holds the key — the wrong primitive for a value that must never be recoverable. A purpose-built password hash (bcrypt, scrypt, Argon2) is also deliberately slow, unlike a fast general-purpose hash (MD5, SHA-256) that makes brute-forcing a stolen hash cheap. Underneath both protections sits key management: a KMS or HSM keeps raw key material out of application code, keys get rotated on a schedule to bound how much damage a leaked key can do, and separation of duties keeps decrypt access and audit-log control on different roles so no single compromised credential can both act and erase the record of acting.

System Design overview

System Design for Beginners CoursefreeCodeCamp.org

What is true here

TLS protects data in motion; encryption at rest protects data at standstill — neither implies the other, and both are required.
A password hash is one-way by design; encryption is two-way by design — hashing is correct for passwords, encryption is not, regardless of cipher strength.
A slow, purpose-built password hash (bcrypt, scrypt, Argon2) makes brute force expensive; a fast general-purpose hash (MD5, SHA-256) makes it cheap.
A KMS/HSM keeps raw key material out of application code and enforces who can use a key; rotation and separation of duties bound what a single leak or single compromised role can do.

What you will be able to do

Explain why TLS coverage does not mean stored data is protected, and vice versa
Identify why password hashing must be one-way and why a reversible cipher is the wrong tool for passwords
Choose a purpose-built password hash over a fast general-purpose one, and justify why the speed difference matters
Describe how a KMS/HSM, key rotation, and separation of duties limit the damage a single leaked key or single compromised role can do

Protecting data in motion and at rest

Two separate protections for two separate windows in data's life, and why enabling one is often mistaken for covering both.

Encryption in transit (TLS) vs encryption at rest

coreintermediate

Encryption in transit (TLS) protects data while it moves across a network; encryption at rest protects data while it sits on a disk, in a backup, or in a database file. You need both — TLS does nothing once a request lands and the data is written to storage.

Think of it as

Think of TLS as an armored truck and encryption at rest as a bank vault. The armored truck protects cash while it travels between two buildings — it does nothing once the cash is inside a building. A vault protects the cash while it sits still — it does nothing while the cash is on the road. A bank needs both; leaving out either one means the cash is exposed for part of its life.

text

// In transit: negotiated per connection
client --TLS 1.3 handshake--> server
       <---- encrypted channel ---->

// At rest: applied to the storage layer, not the request
disk / database volume: AES-256 encryption
backup snapshot: AES-256 encryption
(application reads/writes plaintext; storage layer
 handles encrypt/decrypt transparently)

What we're doing: Trace a password-change request through both protections to see where each one actually applies.

request-lifecycle.txttext

1. User submits a new password over HTTPS.
   -- TLS encrypts the request on the wire --
2. Load balancer terminates TLS, forwards to
   the app server (often over an internal
   network, sometimes also TLS -- service mesh).
3. App server hashes the password, writes the
   hash to the database.
4. Database volume is encrypted at rest.
   -- AES-256 protects the stored hash --
5. Nightly backup job copies the database to
   an object store.
6. Backup is encrypted at rest, same as the
   live volume.

1: TLS protects the password only while it crosses the network — it says nothing about what happens after step 2.
4: Encryption at rest protects the stored hash from anyone who gets the raw disk or volume snapshot, without needing to go through the app.
6: A backup is a full copy of the data outside the live system — if it is not separately encrypted at rest, it is a second, easier target.

Why this works: Each protection covers a different window in the data's life — TLS covers steps 1 and 2, at-rest encryption covers steps 4 and 6 — and neither substitutes for the other during the window it does not cover.

Enabling TLS and assuming stored data is now "encrypted"

Wrong

text

# Team enables HTTPS everywhere, calls the
# database "encrypted" because all traffic
# to it is TLS-protected. Database volume
# and backups are left unencrypted.

Better

text

# TLS for every network hop, PLUS:
# - database volume encryption enabled
#   (e.g. AES-256 at the storage layer)
# - backup snapshots encrypted separately
# - object storage buckets have
#   encryption-at-rest enabled by default

What you see: A leaked database backup or a misconfigured storage bucket exposes every record in plaintext, even though every request to the live system was served over HTTPS the whole time — the incident report reads "TLS was enabled everywhere" right next to "the backup was not encrypted."

Why: TLS only ever covers the network hop. Once a request lands and data is written to disk, TLS has nothing left to protect — a stolen disk, an exposed snapshot, or a misconfigured bucket is read directly, with no network step for TLS to have guarded.

Two protections, two different attack scenarios

TLS in transit

+Stops an attacker sniffing traffic on the network path
+Active only while a request is moving between endpoints
+Useless against a stolen disk or leaked backup file

Encryption at rest

−Stops an attacker who obtains the raw storage medium
−Active only while data is sitting still, not moving
−Useless against traffic sniffed off an unencrypted connection

What each protection actually covers

What each protection actually covers
Protection	Covers	Does not cover
TLS (in transit)	Data on the network between client and server, or between two services	Data already written to disk, a backup file, or a database snapshot
Encryption at rest	Data on disk, in backups, in snapshots, in log files	Data currently being read or written by an authorized, already-connected process

Remember: TLS protects data in motion; encryption at rest protects data at standstill. A system needs both, because a stolen disk or leaked backup never touches the network path TLS covers.

Passwords and the keys behind everything else

Why a password hash must never be reversible, and who is allowed to hold and use the keys that protect everything else.

Password hashing: why it is not encryption

coreintermediate

A password hash is one-way — there is no key that turns it back into the original password, even for the system that created it. Encryption is two-way by design — whoever holds the key can reverse it. Passwords must be hashed, never encrypted, because nothing should ever be able to recover the original.

Think of it as

Encryption is like a locked box with a key — if you have the key, you open the box and the contents are exactly what went in. Hashing a password is like putting it through a paper shredder and keeping only a fingerprint of the shredded result — there is no key that un-shreds it. When a user logs in, you do not unlock anything; you shred their attempt the same way and compare fingerprints. If your system can ever produce the original password from what it stored, it used the wrong tool.

python

import bcrypt

# hash once, at signup -- salt is generated
# and stored inside the returned hash itself
hashed = bcrypt.hashpw(password.encode(), bcrypt.gensalt())
# store `hashed` (bytes) in the database

# verify at login -- never decrypt, always re-hash and compare
bcrypt.checkpw(login_attempt.encode(), hashed)  # -> True / False

What we're doing: Compare what an attacker recovers from a stolen SHA-256 hash versus a stolen bcrypt hash of the same weak password.

brute-force-comparison.txttext

Password: "Summer2024!" (a common, guessable pattern)

SHA-256 (fast, general-purpose):
  ~10 billion hashes/sec on a consumer GPU
  -> guessed within seconds against a
     dictionary + pattern-mutation attack

bcrypt, cost factor 12 (slow, purpose-built):
  ~50 hashes/sec on the same GPU
  -> the same guess attempt now takes
     hours instead of seconds, and scales
     to days/weeks for less common passwords

3: SHA-256 is designed to be fast — that is exactly right for checksums and file integrity, exactly wrong for password verification.
8: bcrypt's cost factor is tunable specifically to keep pace with faster hardware over time — raise it as GPUs get cheaper.

Why this works: The gap is not about the hash "leaking" the password — both algorithms are one-way. It is about how many guesses per second an attacker gets once they have the stolen hash offline; a purpose-built password hash makes that number small on purpose.

Storing passwords with reversible encryption instead of hashing

Wrong

python

from cryptography.fernet import Fernet

# reversible -- whoever holds `key` can
# recover every user's actual password
key = Fernet.generate_key()
cipher = Fernet(key)
encrypted_password = cipher.encrypt(password.encode())
db.save(user_id, encrypted_password)

# "verification" decrypts back to plaintext
stored = db.get(user_id)
if cipher.decrypt(stored).decode() == login_attempt:
    grant_access()

Better

python

import bcrypt

hashed = bcrypt.hashpw(password.encode(), bcrypt.gensalt())
db.save(user_id, hashed)

# verification re-hashes the attempt, never decrypts
stored = db.get(user_id)
if bcrypt.checkpw(login_attempt.encode(), stored):
    grant_access()

What you see: A database or key-management breach recovers every user's actual password in plaintext, not just password-shaped hashes an attacker still has to crack one at a time — because decryption with the stolen key is instant and exact, unlike brute-forcing a one-way hash.

Why: Encryption is built to be reversed by a key holder — that is the entire point of encryption. A password must never be recoverable by anyone, including the system storing it, so encryption is the wrong primitive regardless of which cipher or key size is used. Hashing removes the original value entirely; only a match can be confirmed, never a recovery.

Hashing (one-way) vs encryption (two-way) for password storage

Hashing — correct for passwords

+No key exists that reverses a hash back to the password
+Verification re-hashes the login attempt and compares
+A stolen hash still requires brute force to recover anything

Encryption — wrong for passwords

−A key can decrypt ciphertext back to the exact original
−Verification would mean decrypting and comparing plaintext
−A stolen key (or the system itself) recovers every password instantly

Password hashing algorithms and what to set

Password hashing algorithms and what to set
Algorithm	Tunable cost factor	Notes
bcrypt	work factor (rounds), commonly 10-12	Widest support; capped at 72-byte input, pre-hash longer inputs if needed
scrypt	CPU/memory cost + block size	Memory-hard — also expensive to brute-force on custom ASIC/GPU hardware
Argon2id	memory, iterations, parallelism	Winner of the 2015 Password Hashing Competition; OWASP's current default recommendation

Remember: Hashing is one-way and correct for passwords; encryption is two-way and wrong for passwords, no matter how strong the cipher. Use a slow, purpose-built hash (bcrypt, scrypt, Argon2) — a fast general-purpose hash like SHA-256 makes brute force cheap.

Encryption keys: KMS/HSM, rotation and separation of duties

coreintermediate

A Key Management Service (KMS) stores and controls access to encryption keys so application code never touches raw key material. Keys get rotated on a schedule to limit how much damage a leaked key can do, and separation of duties means no single person can both decrypt data and hide that they did it.

Think of it as

A KMS is like a hotel safe bolted to the floor of your room — you can ask it to lock or unlock your valuables, but you never get to take the safe's internal mechanism apart to see the combination. A Hardware Security Module (HSM) is the physical, tamper-resistant version of that safe — dedicated hardware that generates and holds keys so they never exist anywhere as plain, exportable material. Rotation is like changing that combination on a schedule, so a combination written down and lost six months ago is already useless. Separation of duties is the rule that the person who opens the safe is never the only person who reviews the security camera footage of who opened it.

text

// Envelope encryption with a KMS
1. app calls KMS.GenerateDataKey(masterKeyId)
   -> KMS returns { plaintextKey, encryptedKey }
2. app encrypts the actual data with plaintextKey
   (locally, never sends bulk data to the KMS)
3. app discards plaintextKey from memory,
   stores encryptedKey alongside the data
4. to decrypt later: app calls
   KMS.Decrypt(encryptedKey) -> plaintextKey
   (only succeeds if the caller's IAM role
   is authorized to use masterKeyId)

What we're doing: Trace how envelope encryption and separation of duties combine so no single role can both read data and hide having read it.

decrypt-request-flow.txttext

1. Support engineer's role requests decrypt
   access to a customer record via the KMS.
2. KMS checks the engineer's IAM policy:
   allowed to call Decrypt on this key.
3. KMS performs the decrypt, returns plaintext
   to the app -- the engineer's role can NOT
   read or export the master key itself.
4. KMS emits an audit log entry (key ID,
   caller identity, timestamp) to a separate
   logging system the engineer's role has
   no write or delete access to.
5. Security team, a different role, reviews
   decrypt-audit logs weekly for anomalies.

3: The engineer's role can use the key to decrypt but never extract the raw key material — that separation is what a KMS/HSM enforces structurally, not just by policy.
4: The audit log write path is a separate permission from the decrypt permission — the same role cannot do both.
9: A different team reviewing the logs is what makes separation of duties real — if the same person could decrypt and also edit the audit trail, the control is theater.

Why this works: Separation of duties only works if decrypt access and audit-log control are enforced as genuinely separate permissions, not just separate job titles — a role that can do both makes the audit log unable to catch that role's own misuse.

Letting the same role decrypt data and manage the audit log of who decrypted it

Wrong

text

# IAM policy: "DataAdmin" role
- kms:Decrypt on customer-data-key
- logs:DeleteLogGroup on audit-logs
- logs:PutLogEvents on audit-logs
# One role can decrypt sensitive records
# AND delete or rewrite the evidence

Better

text

# IAM policy: "DataAdmin" role
- kms:Decrypt on customer-data-key
# (no permissions on audit-logs at all)

# IAM policy: "SecurityAudit" role
- logs:GetLogEvents on audit-logs
- logs:PutLogEvents on audit-logs
# (no kms:Decrypt permission)
# Two roles, two different people/teams

What you see: An insider threat investigation finds that a compromised or malicious "DataAdmin" credential decrypted sensitive records and then deleted the corresponding audit log entries — nothing in the system structurally prevented the same credential from doing both, so the incident is undetectable from logs alone.

Why: Separation of duties exists specifically to bound the damage a single compromised credential or single bad actor can do. If decrypt access and audit-log control sit on the same role, that role is a single point of failure for both the action and its own evidence trail — the control provides no real assurance.

Key rotation without breaking already-encrypted data

Key v1

marked rotated, kept for decrypt

Data encrypted under v1

still decryptable via v1

KMS vs HSM — where each one fits

KMS vs HSM — where each one fits
Concept	What it is	When you reach for it
KMS	Managed service API for creating, rotating, and using encryption keys	Default choice for application-level encryption — S3/database encryption keys, envelope encryption, secrets
HSM	Dedicated tamper-resistant hardware holding key material	Regulatory requirements (e.g. PCI-DSS, FIPS 140-2) or the highest-sensitivity keys, often as the layer backing a KMS
Key rotation	Scheduled or event-triggered key replacement	Every long-lived key — limits the blast radius of a leak that has not yet been discovered
Separation of duties	No single role can both decrypt and control the audit trail of decryption	Any system where insider risk or a single compromised credential must not go undetected

Remember: A KMS/HSM keeps raw key material out of application code and enforces who can use a key, not just who can see it. Rotate keys on a schedule to bound a leak's usefulness, and keep decrypt access separate from audit-log control so no single role can act and erase the evidence.