beginnerintermediateadvanced

System Design · Section 69

Cloud Architecture

Level: intermediate
Read: 16 min
Concepts: 2

Every system-design component covered elsewhere in this roadmap — load balancer, compute, managed database, object storage, queue, cache, IAM, networking, observability — reappears on a cloud provider's console as a provisionable menu item rather than software built from scratch; learning "a cloud" for system-design purposes means internalizing that generic mapping on at least one provider, not memorizing one vendor's specific console. Provider-specific depth belongs to a dedicated cloud topic (this project's AWS content), not here. Whether to actually use a given managed offering is a second, separate judgment: prefer managed services when they reduce operational burden, but only when that savings is not outweighed by unacceptable lock-in (a proprietary interface with no compatible exit) or cost (a managed pricing premium that loses to self-hosting at large, steady scale) — a genuine per-workload trade-off, not a blanket rule in either direction.

This section

Cloud-Native Architecture Explained: 5 Key Principles | Cloud-Native Concepts | Architecture DesignKodeKloud

What is true here

The nine system-design components (LB, compute, managed DB, object storage, queue, cache, IAM, networking, observability) all reappear as a cloud provider's menu items — the transferable skill is the mapping, not one vendor's naming.
"Managed" means the provider absorbs undifferentiated operational work — patching, replication, failover, capacity planning — in exchange for less control over the underlying host.
Provider-specific service mechanics are this project's AWS topic's job, not this section's — this section stays at the mapping and the trade-off, one level above any single vendor.
"Prefer managed" is a default, not a rule: it holds when operational-burden savings outweigh lock-in severity and cost at actual scale, and flips at large, steady scale or against a proprietary interface with no compatible exit.
Lock-in is a severity spectrum (open standard underneath vs. proprietary API with no alternative), not a binary property — judge it per service, not as a reason to blanket-avoid or blanket-accept managed offerings.

What you will be able to do

Map a system design expressed in generic vocabulary (LB, compute, DB, storage, queue, cache, IAM, network, observability) onto any cloud provider's menu of managed services
Explain what operational burden a managed service actually removes, versus what still remains the caller's responsibility (retries, backoff, multi-region posture)
Evaluate a managed-vs-self-hosted decision for a specific workload using operational burden saved, lock-in severity, and cost at actual and projected scale
Avoid treating a vendor's specific service name as if it were the system-design concept, and avoid treating "prefer managed" as an unconditional rule

The nine generic system-design components, mapped onto what a cloud provider actually sells as provisionable services.

A cloud provider's menu as system-design building blocks

coreintermediate

Every system-design component covered elsewhere in this roadmap — a load balancer, a compute layer, a database, an object store, a queue, a cache, an identity/access layer, a network boundary, an observability stack — shows up again on a cloud provider's console, except now it is a menu item you provision rather than a piece of software you build and operate from scratch. Learning "a cloud" for system-design purposes does not mean memorizing one vendor's console; it means recognizing that these nine categories of building block exist on essentially every major provider under different brand names, with the provider handling the undifferentiated operational work (patching, replication, failover, capacity) so a design can reference "a managed queue" or "a managed database" as a dependable primitive instead of a from-scratch subsystem. This System Design topic stops at that mapping — the deep, provider-specific mechanics of any one of these categories (e.g. how a specific managed database replicates, or how a specific load balancer's health checks work) is the job of a dedicated cloud-provider topic such as this project's AWS content, not this section.

Think of it as

Think of a cloud provider's console like a well-stocked hardware store versus a lumber yard where you mill your own boards. The lumber yard (self-hosting every layer) gives total control over the exact grain and cut, but every project starts from raw material. The hardware store (a cloud's managed menu) sells pre-cut, pre-finished pieces — a load balancer, a managed database, a queue — in standard sizes that cover most jobs; you still design the house, but you are assembling from stock parts instead of milling lumber for every wall. Different hardware stores (different cloud providers) stock the same nine categories of part under different brand names, and knowing "system design" well means recognizing the part you need on any store's shelf, not just one particular store's aisle layout.

text

// The mapping to internalize, generically (not any one vendor's names):
system-design component  -->  cloud menu category
------------------------------------------------------
load balancer             -->  managed L4/L7 balancer
compute                    -->  VM / managed container / serverless
managed database           -->  managed relational or NoSQL engine
object storage             -->  managed blob store
queue                      -->  managed message broker
cache                      -->  managed in-memory store
IAM                        -->  provider identity & policy service
networking                 -->  virtual network + subnets + private link
observability               -->  provider logs/metrics/traces per resource

// Provider-specific service names (e.g. AWS's ELB, EC2, RDS, S3,
// SQS, ElastiCache, IAM, VPC, CloudWatch) are covered in the AWS topic,
// not here.

What we're doing: Trace one request path through a design expressed purely in system-design vocabulary, then in cloud-menu vocabulary, to see the mapping hold end to end.

request-path-mapping.txttext

System-design vocabulary:
1. Client request hits a load balancer.
2. Load balancer forwards to a compute instance.
3. Compute instance reads/writes a managed database.
4. Compute instance uploads a large file to object storage.
5. Compute instance pushes a job onto a queue for async work.
6. A cache sits in front of the database for hot reads.
7. IAM policy governs which compute identity may reach
   the database and the queue.
8. Networking rules restrict the database to only be
   reachable from inside the private network.
9. Observability captures logs/metrics/traces for every
   hop above.

Cloud-menu vocabulary (same nine steps, same order):
1. A managed L4/L7 load balancer.
2. A VM, managed container, or serverless compute unit.
3. A managed relational or NoSQL database service.
4. A managed object storage bucket.
5. A managed message queue.
6. A managed in-memory cache.
7. The provider's IAM policy attached to the compute
   identity.
8. A virtual private network with subnet-level rules.
9. The provider's centralized logs/metrics/traces.

3: Every step in the first list has a same-numbered counterpart in the second — the design does not change, only who operates each piece.
7: IAM and networking are not optional extras bolted on afterward — they gate every other component in the same list.
16: None of the second list names a specific vendor's service — that vendor-specific naming is the AWS topic's job, not this one's.

Why this works: Walking the identical path twice, once generically and once as "a cloud's menu," makes the mapping concrete: nothing about the design itself changes, only which layer the team must build versus provision.

Treating a specific provider's service name as if it were the system-design concept itself

Wrong

text

// Design doc says:
"We'll put an ALB in front of EC2, backed by RDS,
with objects in S3 and jobs in SQS."
// Nothing here explains WHY a load balancer, WHY a
// managed database, or WHY a queue is the right shape
// for this problem -- it's a vendor shopping list,
// not a design.

Better

text

// Design doc says:
"Traffic needs load balancing across stateless compute
because instances are disposable; state lives in a
managed database because we don't want to operate
replication ourselves; large binary payloads go to
object storage because they don't belong in the
database; async work is decoupled via a queue because
producers shouldn't block on consumers."
// THEN: "On AWS this is ALB + EC2/ECS + RDS + S3 + SQS."

What you see: A design review can name every AWS (or GCP, or Azure) service involved but cannot answer "why a queue here" or "why managed vs self-hosted" when asked — the vendor names substitute for design reasoning, and swapping to a different cloud (or presenting the same design in an interview that does not name a vendor) leaves the candidate unable to explain the architecture at all.

Why: The system-design reasoning (why this component category, why this shape) has to exist independently of any vendor's branding — the vendor-specific name is a provisioning detail chosen last, not the thing being designed. Confusing the two means the reasoning was never actually done.

Generic system-design components mapped onto a cloud's managed menu

System-design component

LB, compute, DB, storage, queue, cache, IAM, network, observability

Cloud provider menu

the same nine categories, provisioned rather than built

Operational burden

patching, replication, failover, capacity — absorbed by the provider

The nine system-design categories, generically, as a cloud menu maps them

The nine system-design categories, generically, as a cloud menu maps them
System-design component	What the cloud sells as this	What "managed" removes from you
Load balancer	A managed L4/L7 balancing service with health checks	Running and scaling the balancer software itself
Compute	VMs, managed containers, or serverless functions	OS patching (VMs less so), and all of it for serverless
Managed database	A relational or NoSQL engine as a service	Replication, failover, backup, and patching
Object storage	Durable, replicated blob storage	Disk provisioning, replication, durability engineering
Queues	A managed message broker/queue	Running and scaling broker software, durability guarantees
Caches	A managed in-memory data store	Running cache software, failover, and cluster rebalancing
IAM	A provider-wide identity and policy service	Building your own auth/authz system for infrastructure access
Networking	Virtual networks, subnets, private connectivity	Physical network hardware and topology
Observability	Centralized logs, metrics, traces per resource	Building your own telemetry pipeline from nothing

Remember: A cloud provider's console sells the same nine system-design categories — load balancer, compute, managed database, object storage, queue, cache, IAM, networking, observability — as provisionable menu items rather than software built from scratch; the transferable skill is the generic mapping, not one vendor's specific naming. Provider-specific mechanics live in a dedicated cloud topic (this project's AWS content); this concept is the higher-altitude view of which category each menu item corresponds to and what "managed" removes from the operator.

Managed vs. self-hosted, as a real trade-off

When "prefer managed" holds, and the two forces — lock-in and cost at scale — that can flip the decision the other way.

The managed-vs-self-hosted trade-off

coreintermediate

Choosing a managed service over self-hosting the same capability is a genuine trade-off, not a default answer — the roadmap's own phrasing names the two costs that can outweigh the operational-burden savings: lock-in (how hard it becomes to leave the provider or the specific service) and cost (a managed service's pricing model can cost far more than self-hosting at a given scale, especially for steady, predictable, high-volume workloads). "Prefer managed" is the right default posture because most teams underinvest in undifferentiated operational work and overestimate their own ability to run infrastructure as reliably as a provider whose entire business is running it — but it is a posture that gets overridden whenever a workload is large and stable enough that self-hosting is cheaper, or whenever a managed service's proprietary interface would make a future migration prohibitively expensive relative to the operational burden it saves.

Think of it as

Think of it like choosing between renting a fully-furnished apartment and buying a house. Renting furnished (managed service) removes nearly all maintenance burden — no fixing the water heater, no buying furniture — and is the right call for anyone who might move in a year, or who does not want maintenance to be their problem. But renting furnished long-term, at scale (many years, a family's worth of furniture-equivalent scale), often costs more in total than owning would have, and the landlord's lease terms (lock-in) may make it disproportionately painful to leave even after the math stops favoring renting. Owning (self-hosting) is the right call when you are confident you are staying long enough, and using enough space, that the fixed cost of ownership pays for itself — but only if you are also prepared to be your own plumber when the water heater breaks.

text

// A short decision checklist, generically:
if (operational_burden_saved > 0
    and lock_in_cost < acceptable_threshold
    and managed_pricing_premium < ops_cost_it_replaces):
    prefer managed
else:
    self-host is the better trade, even though it is
    more operational work

// Note: "prefer managed" is the *default* because most
// teams underestimate ops_cost_it_replaces -- but the
// inequality above can and does flip at scale.

What we're doing: Walk through the trade-off analysis for one concrete decision — whether to use a managed queue or self-host a message broker — showing the reasoning that leads to a choice either way depending on the workload.

managed-queue-decision.txttext

Workload A: a five-person startup's background job
queue, volume unpredictable, no dedicated ops team.
  - Operational burden saved: high (no one on the team
    wants to run and patch broker software)
  - Lock-in: low if using the queue's standard protocol
    equivalent; moderate if using provider-only APIs
  - Cost: managed pricing premium is small in absolute
    dollars at this volume
  -> Decision: use the managed queue.

Workload B: a large, steady-state event pipeline
processing billions of messages/day at a company with
an existing platform-engineering team running other
message infrastructure already.
  - Operational burden saved: lower marginal value --
    the team already carries this expertise for other
    systems
  - Lock-in: the managed service's proprietary consumer
    model doesn't match the team's existing tooling
  - Cost: at this volume, the managed premium is a large
    absolute dollar figure, and self-hosting reuses
    already-paid-for operational capacity
  -> Decision: self-host, using the team's existing
     broker expertise.

6: Same category of decision (queue: managed vs self-hosted), opposite conclusion — the trade-off, not the component, determines the answer.
15: Existing team expertise changes the "operational burden saved" term dramatically — it is not a fixed number across organizations.
20: The dollar cost of the managed premium scales with volume; at low volume it is often negligible, which is why "prefer managed" is a reasonable default at small scale specifically.

Why this works: The same component (a queue) and the same generic trade-off variables produce opposite recommendations once the workload's actual scale, team expertise, and lock-in exposure are filled in — showing the roadmap's "prefer managed... without unacceptable lock-in or cost" is a real conditional, not a euphemism for "always managed."

Defaulting to a managed service for a workload whose scale makes it a poor cost trade, without ever running the numbers

Wrong

text

// Company policy: "always use the managed version,
// it's less ops work" -- applied uniformly regardless
// of workload size, with no cost comparison ever done
// against self-hosting at the company's actual, very
// large and steady scale.

Better

text

// Before committing to a managed service at scale:
// 1. Estimate the managed pricing at current AND
//    projected volume (12-24 months out).
// 2. Estimate the self-hosted cost: infra + the
//    fully-loaded cost of the ops time it requires.
// 3. Estimate migration cost if lock-in makes leaving
//    expensive later, and weigh it against #1 vs #2.
// Only then decide -- the "prefer managed" default is
// a starting point for the analysis, not a substitute
// for it.

What you see: A cost review months or years later reveals the managed service's bill has grown to many times what an equivalent self-hosted deployment would cost at the same volume, and by then the team's code is written directly against the provider's proprietary APIs, making migration a multi-quarter project rather than a straightforward swap — the "operational burden saved" argument that justified the original choice was real at launch but was never re-evaluated as volume grew past the point where it still made sense.

Why: The trade-off is scale-dependent and time-dependent, not a one-time decision that stays correct forever — a policy that never revisits it, and a policy that never even ran the numbers in the first place, both skip the actual judgment the roadmap is asking for and substitute a blanket rule instead.

The trade-off as three competing forces

Operational burden saved

patching, failover, capacity — provider absorbs it

Lock-in created

proprietary interface, migration cost

Cost at scale

managed premium vs self-hosted fixed cost

Decision

managed only if burden savings outweigh lock-in + cost

Forces that push the decision toward managed vs toward self-hosted

Forces that push the decision toward managed vs toward self-hosted
Force	Pushes toward managed	Pushes toward self-hosted
Team size / expertise	Small team, no deep ops specialization for this system	Large team with dedicated expertise already in place
Scale and stability	Unpredictable or moderate, elastic load	Large, steady, well-forecast load where fixed cost wins
Lock-in surface	Open/portable API or standard interface underneath	Proprietary interface with no compatible alternative
Cost sensitivity	Operational risk cost exceeds the managed pricing premium	Managed pricing premium exceeds the ops cost it replaces
Time to market	Need it running now, not in a quarter	Long-lived system where build time amortizes over years

Remember: "Prefer managed" is a default, not a rule — it holds when operational-burden savings outweigh the lock-in severity and the cost at actual (and projected) scale, and it flips at large, steady scale or against a proprietary interface with no compatible exit. Run the trade-off per workload and re-run it as scale changes; lock-in itself is a spectrum of severity, not a yes/no property, and the same organization can correctly land on opposite answers for two different workloads.

The cloud as a menu of building blocks

Managed vs. self-hosted, as a real trade-off