Cloud Architecture Patterns

What we're doing: Follow one page request through every hop, and name what each hop is for.

1. Browser asks for app.example.com
   Route 53 alias record answers with the CloudFront distribution.

2. CloudFront edge location receives the request
   /static/* is a cache hit, served from the edge, origin never touched.
   /api/*    is not cacheable, forwarded to the ALB origin over HTTPS.

3. ALB receives the forwarded request
   Listener rule on host + path picks the target group.
   Health check has already removed any unhealthy task.

4. An ECS task in a private subnet handles it
   Reads session from ElastiCache. Cache miss on the product
   record, so it queries RDS and writes the result back to the cache.

5. RDS writer answers the query
   The task never knows which physical instance it reached — the
   Multi-AZ endpoint is a DNS name that moves on failover.

6. Response travels back up the same chain
   ALB → CloudFront → browser. Nothing new is opened downward.

1

One domain covers both static and dynamic traffic, because CloudFront routes by path to two different origins.

7

The ALB is the only component in a public subnet, and the only thing the edge can reach.

13

Cache-aside: the application, not the cache, decides what to store and when. See the ElastiCache section.

18

The application connects to an endpoint name, never an IP — this is what makes failover survivable without a deploy.

Why this works: The value of the pattern is not the box diagram, it is that every hop has one job and one caller. When a request fails you can name which hop it died at, and when you scale you can scale exactly one tier. A design where the browser talks to two of these tiers directly loses both properties at once.

What we're doing: Design "submit an order" so the caller waits for a decision, not for the work.

SYNCHRONOUS — must finish in a few hundred milliseconds
  POST /orders  →  API Gateway (JWT authorizer, throttle)
                →  Lambda create_order
                →  DynamoDB PutItem with
                     ConditionExpression: attribute_not_exists(order_id)

  Returns 201 with the order id. That is the whole contract.
  A retry of the same request hits the condition and is rejected,
  so a client that retries on a timeout cannot create two orders.

ASYNCHRONOUS — the caller has already gone
  create_order also puts an OrderPlaced event on EventBridge.

  EventBridge rule  →  SQS queue  →  Lambda charge_payment
  EventBridge rule  →  SQS queue  →  Lambda send_confirmation
  EventBridge rule  →  SQS queue  →  Lambda update_inventory

  Each worker is idempotent, each has its own dead-letter queue,
  and each can fail and be redriven without touching the others.

1

Deciding what belongs on this side is the entire design decision — everything else follows from it.

6

The condition expression is what makes the endpoint safe to retry; without it a network timeout becomes a duplicate order.

12

One event, three independent consumers. Adding a fourth does not touch create_order.

17

Separate queues mean a failing payment processor does not stall confirmation emails.

Why this works: Serverless pushes you toward this split whether you plan for it or not, because the timeout is a hard ceiling rather than a slow degradation. Deciding the split deliberately gives you a fast, retry-safe API and a set of workers that can each fail on their own; deciding it accidentally gives you one function doing everything, timing out at 15 minutes with no record of how far it got.

What we're doing: Add a fourth consumer to a live platform without touching the producer.

Today: OrderPlaced → billing, search index, notifications.
Ask: "finance wants a ledger entry for every order."

In a request/response design this is a change to the orders
service: a new outbound call, a new failure mode in the
checkout path, a new deploy, a new rollback risk.

In this design it is three additions, none of them upstream:
  1. new SQS queue  ledger-work  + its dead-letter queue
  2. new EventBridge rule, same pattern, new target
  3. new consumer that reads ledger-work

The orders service is not redeployed, not restarted, and does
not know the ledger exists. If the ledger consumer breaks on
day one, ledger-work grows and the other three keep running.

What you have taken on: the ledger is eventually consistent.
"Order placed" and "ledger entry written" are two moments, and
any report that assumes they are one moment will be wrong.

1

The starting point matters less than what changes when a requirement arrives — that is the property being bought.

8

All three changes are additive and reversible. Deleting the rule removes the consumer with no upstream coordination.

14

The honest cost, stated up front: this is what you traded the coupling for, and it is not free.

Why this works: Event-driven architecture is usually sold on decoupling, which is real but vague. The concrete test is this one: when a new consumer appears, how many existing services get redeployed? A design where the answer is zero absorbs new requirements cheaply. The bill comes due as eventual consistency and as harder debugging, both of which have to be paid deliberately.

What we're doing: Recover from a transform bug discovered three weeks after it shipped.

The bug: currency amounts parsed as integers, so every order
from the three markets that use decimal minor units is 100×
too large in the curated table. Shipped 21 days ago.

What is damaged: curated/orders for 21 days, and every
dashboard and extract built on top of it.

What is not damaged: raw/orders. It holds exactly the bytes
that arrived, with no parsing applied, for every one of those
days — because the ingestion step never transformed anything.

The recovery:
  1. fix the parser, deploy the transform
  2. rerun it over raw/orders/ingest_date=2026-08-14..2026-09-04
  3. the job overwrites the affected curated partitions
  4. downstream aggregates rebuild from curated

No source system is asked to resend. Nothing is reconstructed
by hand. The whole incident costs one backfill run.

1

A parsing bug that survives weeks is the normal case, not the exceptional one — pipelines are rarely watched closely enough to catch it on day one.

6

Everything derived is damaged together, which sounds bad and is actually the good outcome: one rerun fixes all of it.

13

Reprocessing is a range of partitions, which is only possible because raw is partitioned by arrival time.

Why this works: The raw zone earns its storage cost exactly once, on the day you need it, and then it pays for itself completely. A pipeline that transforms on the way in has no such day: the only copy of the data is the wrong one, and recovery means asking every upstream system to resend — which some of them cannot do.

What we're doing: Handle the same S3 event arriving twice without processing the file twice.

Event arrives: s3://acme-uploads/uploads/t-14/j-8821/report.csv

1. Derive the job id from the key: j-8821.
   Nothing in the message body is trusted for this — the key is
   the identity, and S3 guarantees it.

2. Claim the job with a conditional write:
     UpdateItem(pk=j-8821,
                SET status='processing',
                ConditionExpression: status = 'pending')

   Condition fails  →  another delivery already claimed it.
                       Delete the message, return. Not an error.
   Condition passes →  this delivery owns the work.

3. Process. Write processed/t-14/j-8821/report.parquet.
   Same input always produces the same output key, so even a
   worker that dies after writing but before step 4 is safe.

4. UpdateItem status='done', publish JobCompleted, delete message.

1

S3 event notifications are delivered at least once — this is documented behaviour and the handler is written for it.

5

Deriving identity from the object key rather than from a message id means a replay of an old message is also handled correctly.

11

The claim is one atomic conditional write. Two concurrent deliveries race, one wins, and the loser exits cleanly.

16

A deterministic output key turns a crash between steps 3 and 4 into a harmless overwrite rather than a duplicate.

Why this works: Everything in this pipeline is at-least-once: the event notification, the queue, the retry after a visibility timeout. Rather than trying to make delivery exact, the worker is written so a second delivery is a no-op — a conditional claim at the start and a deterministic output key at the end. Those two lines are what make the pipeline safe to operate.