Filter concepts by levelShowing all levels.

AWS · Architecture Patterns

Cloud Architecture Patterns

The five shapes that cover most workloads, and the trade-off each one is making

  • Traffic moves one wayeach tier accepts only the tier above it
  • Split on "is anyone waiting?"the queue is where slow work goes
  • One queue per consumerfan-out without it re-couples them
  • Land raw, transform aftera bug then costs a rerun, not the data

1.Three-Tier Web Application

The three-tier pattern splits a web application into a presentation tier a browser talks to, an application tier that runs your code, and a data tier that stores state. On AWS that is Route 53 and CloudFront at the edge, an Application Load Balancer in public subnets, containers or instances in private subnets, and RDS plus ElastiCache plus S3 behind them. Every tier sits in at least two Availability Zones.

Key features

  • Only the load balancer sits in public subnets; application and data tiers sit in private subnets with no route to an internet gateway
  • Security groups chain by reference: the app tier allows the ALB's security group, the database allows the app tier's — never CIDR ranges
  • CloudFront serves static assets from S3 and forwards dynamic paths to the ALB, so one domain covers both
  • An ACM certificate used by CloudFront must be requested or imported in US East (N. Virginia) (us-east-1)
  • The application tier must be stateless: sessions live in ElastiCache or a cookie, uploads live in S3, so any instance can serve any request
  • ElastiCache is a read accelerator in front of RDS, never the system of record — a cache flush must only cost latency

Each tier: what it owns, where it lives, what reaches it

Each tier: what it owns, where it lives, what reaches it
TierAWS servicesSubnetInbound from
EdgeRoute 53, CloudFront, AWS WAF, ACMNone — globalThe internet
PresentationApplication Load BalancerPublic, one per AZCloudFront (and only CloudFront, if locked down)
ApplicationECS/Fargate tasks or EC2 in an Auto Scaling groupPrivate, one per AZThe ALB security group
DataRDS or Aurora, Multi-AZPrivate data subnet, no default routeThe application security group
CacheElastiCachePrivate data subnetThe application security group
ObjectsS3None — regional, reached via a gateway endpointThe application role, and CloudFront via OAC
Every tier, every boundary, one direction of traffic

Traffic only ever moves downward. Each tier spans two Availability Zones, and only the public subnets have a route to the internet gateway.

  • A five-band diagram, read top to bottom, with an arrow between each band.
  • Band 1, Client: users on the internet.
  • Band 2, Edge (global): Route 53 resolves the domain, CloudFront and AWS WAF terminate TLS and filter requests, and S3 holds static assets served through CloudFront.
  • Band 3, Public subnets across two Availability Zones: the Application Load Balancer, and a NAT gateway for outbound-only traffic from private subnets.
  • Band 4, Private application subnets across two Availability Zones: ECS tasks or EC2 instances in an Auto Scaling group, one set per zone.
  • Band 5, Private data subnets across two Availability Zones: RDS with a Multi-AZ standby, and ElastiCache.
  • Footnote: only band 3 has a route to an internet gateway, and each tier accepts traffic only from the tier directly above it.

Signature

Route 53 → CloudFront/WAF → ALB (public) → ECS/EC2 (private) → RDS + ElastiCache + S3

Route 53 and CloudFront at the edge, ALB in public subnets, application in private subnets, data in private subnets with no default route — traffic only ever moves downward, security groups reference the group above rather than a CIDR, and the application tier holds no state.

2.Serverless API

A serverless API replaces the load balancer with API Gateway and the always-on application tier with Lambda functions. Route 53 points a custom domain at API Gateway, each route invokes a function, and the function reads and writes DynamoDB or S3. Anything slow is not done in the request — it is published to EventBridge or SQS and handled by a second function.

Key features

  • API Gateway has a default throttle limit of 10,000 requests per second while Lambda has a default concurrency limit of 1,000, so a burst can pass the front door and be throttled behind it
  • A Lambda function can run for at most 900 seconds (15 minutes) and can be allocated 128 MB to 10,240 MB of memory
  • Synchronous invocation payloads are capped at 6 MB each way; asynchronous payloads at 1 MB — so large objects travel by S3 reference, not in the event
  • DynamoDB conditional writes (`attribute_not_exists`) make a create idempotent: the second attempt is rejected rather than overwriting
  • Asynchronous Lambda invocations are retried by the service, so every handler on that path has to be idempotent
  • SQS or EventBridge between the API and the slow work also gives you a buffer: a traffic spike becomes queue depth instead of throttled requests

Which limit bites first, and what it turns into

Which limit bites first, and what it turns into
LimitDefaultWhat the caller sees
API Gateway throttle10,000 requests per second429 Too Many Requests at the front door
Lambda concurrent executions1,000 per Region, adjustable429 from Lambda — the request reached the function and was refused
Lambda timeout900 seconds (15 minutes) maximumFunction killed mid-work; the invocation is retried if asynchronous
Lambda synchronous payload6 MB request and 6 MB responseRequest rejected — pass an S3 key instead of the object
Lambda asynchronous payload1 MBEvent rejected at publish time, not at handling time
Lambda memory128 MB to 10,240 MB, CPU scales with itA CPU-bound function is slow until memory is raised
The synchronous path, and everything deliberately pushed off it

Left of the dashed line is the request the caller waits on. Right of it is work that must not be allowed to affect the caller's latency or the API's timeout.

  • A diagram split by a vertical dashed line into a synchronous path on the left and asynchronous work on the right.
  • Synchronous path, left to right: a client calls Route 53, which resolves a custom domain to API Gateway; API Gateway invokes a Lambda function; the function reads and writes DynamoDB and S3 and returns a response.
  • The function also publishes an event across the dashed line.
  • Asynchronous side: the event goes to EventBridge or SQS, which invokes a worker Lambda, which does the slow work and writes back to DynamoDB and S3; failed messages land in a dead-letter queue.
  • Note: the caller never waits for anything on the right-hand side.

Signature

Route 53 → API Gateway → Lambda → DynamoDB/S3 · EventBridge/SQS → worker Lambda

Route 53 → API Gateway → Lambda → DynamoDB/S3 for anything the caller waits on, EventBridge or SQS → worker Lambdas for everything else. Watch the 10,000 rps front door against the 1,000 default concurrency behind it, keep payloads under 6 MB by passing S3 keys, and make every asynchronous handler idempotent.

3.Event-Driven Platform

An event-driven platform replaces direct service-to-service calls with a bus. A producer announces that something happened; a router copies that event to every interested consumer's own queue; each consumer drains its queue at its own pace. The producer never learns who consumed the event, which is the whole point — new consumers are added without changing it.

Key features

  • Give every consumer its own queue, not a shared one — a consumer that stops draining then backs up alone
  • Each queue gets its own dead-letter queue, so a poison message is quarantined per consumer rather than blocking the subscription
  • Consumers see at-least-once delivery, so every handler must be idempotent — the platform gives you delivery, not exactly-once processing
  • The producer publishes a fact about the past ("OrderPlaced"), never an instruction ("ChargeCard") — the instruction belongs to whoever consumes it
  • Event schemas are a public contract: add optional fields freely, and version the event type before removing or renaming one
  • Consumers are eventually consistent by construction; a read immediately after a write may not see the consumer's effect yet

Choosing the router, and what each one is actually for

Choosing the router, and what each one is actually for
RouterSelection modelReach for it when
EventBridgeContent-based rules matched against the whole event JSONMany event types on one bus, routing on fields, archive and replay, cross-account delivery
SNSTopic subscription, with optional message filtering on attributesOne event type, high fan-out, low latency, simplest possible wiring
SQS aloneNo routing — one producer, one consumer groupYou need a buffer between two components, not a fan-out
KinesisOrdered shards, consumers track their own positionOrder matters, or several consumers must replay the same stream from a chosen point
Step FunctionsExplicit state machine you authorThe sequence itself is the product and someone has to see where an execution is
One event, one copy per consumer, one queue each

The bus decides who gets a copy. The queue in front of each worker is what stops a slow or broken consumer from affecting the producer or the other consumers.

  • A fan-out diagram, left to right.
  • On the left, three producers — the orders service, the web API, and a scheduled job — all publish to a single event bus, EventBridge or SNS, in the middle.
  • The bus matches rules and delivers one copy of the event to each of three separate SQS queues.
  • Each queue feeds its own worker: billing, search indexing, and notifications. Each worker writes to its own database or external system.
  • Each queue has its own dead-letter queue beneath it.
  • Note: the producers never learn which consumers exist, and one blocked consumer only backs up its own queue.

Signature

Producers → EventBridge/SNS → one SQS queue per consumer → workers → data stores

Producers publish facts to EventBridge or SNS; the router copies each event into one SQS queue per consumer; each consumer drains at its own pace with its own dead-letter queue. Delivery is at least once, so handlers are idempotent, and the price of the decoupling is eventual consistency.

4.Data Ingestion Pipeline

A data ingestion pipeline moves data from wherever it is produced into a shape analysts can query. Records arrive as files in S3 or as a stream in Kinesis, land untouched in a raw zone, are cleaned and reshaped by a processing step, and are written to a curated zone that reporting and dashboards read. The raw copy is kept so the processing step can be rewritten and rerun.

Key features

  • Land raw, immutable and append-only first; every later zone is derived and can be regenerated from it
  • Batch arrivals go through S3, continuous records through Kinesis — the difference is whether ordering and per-record latency matter
  • Partition curated data by the column queries filter on, usually date, so a scan reads a slice rather than the whole table
  • Columnar formats such as Parquet cut both scan time and query cost for analytics, because only the referenced columns are read
  • Ingestion is at-least-once, so the transform must be able to run twice on the same input and produce the same curated output
  • Lifecycle policies on the raw zone are what keep it affordable — raw grows forever by design

The zones, and the rule that holds for each

The zones, and the rule that holds for each
ZoneContentsWritten byRule
Raw / landingExactly what arrived, unparsedIngestion onlyNever edited in place; the only zone that cannot be rebuilt
Processed / stagingParsed, typed, deduplicatedThe transform jobDisposable — deleting it costs one rerun
CuratedPartitioned columnar tables with a published schemaThe transform jobSchema changes are versioned, because queries depend on it
ConsumptionAggregates, extracts, feature tablesDownstream jobsDerived from curated only, never from raw directly

S3 or Kinesis for the arrival step

S3 or Kinesis for the arrival step
AspectS3 as the entry pointKinesis as the entry point
Shape of arrivalWhole files, on a schedule or when producedIndividual records, continuously
Latency to curatedMinutes to hoursSeconds to minutes
OrderingNone across filesPreserved within a shard
ReplayRead the object again — it is still thereOnly within the retention period
TriggeringS3 event notification, or a scheduled jobEvent source mapping or a stream consumer
Reach for it whenExports, uploads, nightly extracts, backfillsClickstream, telemetry, application events
Raw, curated, consumed — and what may be rewritten
arriveland as-isreadwritequeryrerun from hereafter a fix

Sources

apps, devices, exports, third parties

S3 or Kinesis

files for batch, stream for continuous

Raw zone

immutable, never edited, never deleted early

Processing

validate, deduplicate, reshape, partition

Curated zone

partitioned Parquet, one schema, documented

Analytics

SQL queries, dashboards, models

  • Sources — apps, devices, exports, third parties
    • leads to S3 or Kinesis (arrive)
  • S3 or Kinesis — files for batch, stream for continuous
    • leads to Raw zone (land as-is)
  • Raw zone — immutable, never edited, never deleted early
    • leads to Processing (read)
  • Processing — validate, deduplicate, reshape, partition
    • leads to Curated zone (write)
    • leads to Raw zone (rerun from here after a fix)
  • Curated zone — partitioned Parquet, one schema, documented
    • leads to Analytics (query)
  • Analytics — SQL queries, dashboards, models

Signature

S3 | Kinesis → raw zone (immutable) → transform → curated zone (partitioned Parquet) → analytics

Land raw and immutable first, transform second, publish a curated zone partitioned by the column people filter on. Files and batch go through S3, continuous records through Kinesis, and both end up in the raw zone. Write whole partitions so any run is repeatable, and keep raw so a bug costs a backfill instead of the data.

5.File Processing Pipeline

A file-processing pipeline accepts an upload, does slow work on it, and tells the user when the result is ready. The upload goes straight from the browser to S3 with a presigned URL, the object landing triggers an event, a worker reads the object and writes a processed one, and a notification closes the loop. Your API never carries the bytes.

Key features

  • A presigned URL carries the credentials of the principal who generated it, so the browser uploads directly without any AWS credentials of its own
  • Presigned URLs created with IAM user credentials are valid for up to 7 days with SigV4; one created with temporary role credentials expires when the role session does, whatever expiry you asked for
  • The S3 console caps presigned URL expiry between 1 minute and 12 hours; the CLI and SDKs allow up to 7 days
  • S3 replaces an existing object if the uploaded key already exists, so the key itself has to be unique per upload
  • S3 event notifications are delivered at least once, so the worker must tolerate seeing the same object twice
  • Route the event through SQS rather than invoking the worker directly, so a burst of uploads becomes queue depth instead of throttling

Each hop: what it is for, and what it must survive

Each hop: what it is for, and what it must survive
HopPurposeFailure it has to handle
API issues a presigned URLGrant one specific PUT without giving out credentialsThe user never uploads — the job row stays pending and is expired by a sweeper
Browser PUTs to S3Move bytes without touching your computeNetwork failure mid-upload — the object simply does not appear
S3 event notificationTurn "an object exists" into a messageDelivered more than once for one object
SQS queueAbsorb a burst and decouple worker scaling from upload rateA backlog, which is visible and drains, rather than throttling
WorkerDo the slow work and write the processed objectBeing killed mid-work; the message reappears after the visibility timeout
Dead-letter queueQuarantine an object the worker cannot processA poison file blocking every later upload
NotificationClose the loop with the userBeing sent twice — so it is keyed on the job, not on the message
The API hands out permission; S3 carries the bytes

Steps 1 and 2 are the only synchronous ones. Everything from step 4 onward happens after the browser has been told the upload succeeded.

  • A numbered pipeline diagram in two rows.
  • Step 1: the browser asks the API for an upload URL, and the API returns a presigned S3 URL after recording the job as pending.
  • Step 2: the browser PUTs the file directly to the S3 uploads bucket. The file never passes through the API.
  • Step 3: the object landing in S3 emits an event notification.
  • Step 4: the event goes to an SQS queue, which buffers bursts.
  • Step 5: a worker — Lambda or an ECS task — reads the object, does the slow work, and writes a processed object to a second prefix.
  • Step 6: the worker marks the job complete and publishes a notification to the user.
  • A dead-letter queue hangs off the SQS queue for objects the worker repeatedly fails on.

Signature

API (presigned URL) → browser PUT → S3 → event → SQS → worker → processed object → notify

The API issues a presigned URL and records the job; the browser uploads straight to S3; the object event goes through SQS to a worker; the worker writes a processed object and notifies. Every hop is at-least-once, so claim the job with a conditional write and write a deterministic output key.

Remember

  • Security groups reference the group above them, never a CIDR inside the VPC — then adding an AZ or a task changes no rules.
  • API Gateway defaults to 10,000 requests per second and Lambda to 1,000 concurrent executions, so a burst can pass the front door and be throttled behind it.
  • S3 event notifications and SQS both deliver at least once; a conditional claim plus a deterministic output key is what makes a worker safe.
  • A presigned URL expires with the credentials that signed it, so a role-signed URL cannot outlive the role session however long an expiry you asked for.

Pro tip

  • Test a proposed architecture by asking what has to be redeployed when a new consumer or a new file type arrives — the answer is the coupling, stated honestly.
  • Partition the raw zone by when data arrived and the curated zone by when the event happened: the first makes reruns a prefix, the second makes queries a prefix.

Next up

  • AWS for Django / Python Backends
  • AWS for Full-Stack / React Applications
  • AWS Senior-Level System Design

Sources

  • Lambda quotas — docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
  • Download and upload objects with presigned URLs — docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
  • Requirements for using SSL/TLS certificates with CloudFront — docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cnames-and-https-requirements.html
  • Exactly-once processing in Amazon SQS — docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues-exactly-once-processing.html
  • Checked against official AWS documentation 2026-09-04

Spotted a mistake on this page? Tell us.