Filter concepts by levelShowing all levels.

AWS · Architecture Review

AWS Architecture Review Checklist

Six passes over a running system, each closed by evidence rather than by an assurance

  • Review the accountthe diagram does not drift; the account does
  • Failures, not featuresevery control needs a date it was exercised
  • Measure before advisingattribute latency, group the bill by driver
  • The 3 a.m. testcould a stranger recover it from what is written?

1.The Security Pass — Six Questions, Six Pieces of Evidence

The security pass of a review asks six questions: who can do what, is the data encrypted, where do the secrets live, what can reach this over the network, is every action logged, and when was the code last scanned. Each question is answered with evidence you can point at — a policy, a key id, a security group rule, a scan date — not with a sentence about intent.

Key features

  • Least privilege is measured against access actually used, not against intent — IAM Access Analyzer reports permissions granted and never exercised
  • Encryption is two independent questions: at rest (which key, in whose account) and in transit (where TLS terminates, and whether the hop behind it is also encrypted)
  • A credential in an environment variable, an AMI, a container image or a repository is a finding whatever the current access list says
  • Network isolation is checked in the security groups: 0.0.0.0/0 on port 22, 3306 or 5432 fails the pass regardless of what the diagram draws
  • Logging counts as a control only when the trail lands in an account the workload cannot write to, and CloudTrail covers every Region
  • Vulnerability management needs a cadence and a named owner — Amazon Inspector findings with nobody assigned are a report, not a process

The security pass — what to run, what closes the question

The security pass — what to run, what closes the question
CheckWhat to look atPasses whenFails when
Least privilegeIAM Access Analyzer findings; policies attached to production rolesNo external access finding is unexplained, and no production policy grants `*` on `*`Wildcard actions on production resources, or long-lived user access keys instead of roles
Encryption at restRDS, EBS, S3 and DynamoDB encryption settings, and the key each usesEvery data store names a key, and sensitive stores name a customer-managed keyA store with encryption off, or nobody able to say which key protects it
Encryption in transitLoad balancer listeners, database `require_ssl`, service-to-service callsTLS on every hop that leaves an instance, including the hop behind the load balancerTLS terminated at the edge and plain HTTP the rest of the way
SecretsTask definitions, Lambda environment variables, AMIs, images, repository historyEvery credential resolves from Secrets Manager or Parameter Store at run timeA credential visible in an environment variable, an image layer, or a commit
Network isolationSecurity groups, NACLs, subnet route tables, public IP assignmentData tiers accept traffic only from an application security group, not a CIDR0.0.0.0/0 on an administrative or database port anywhere in the account
LoggingCloudTrail trails, VPC flow logs, load balancer access logs, log destinationsAn organization trail writes to a separate log account the workload cannot alterTrails writing into the same account they audit, or single-Region coverage
Vulnerability managementAmazon Inspector findings, base image ages, patch cadenceA stated cadence, a named owner, and a most-recent scan date inside itScanning enabled with no owner, or a base image nobody has rebuilt this quarter
Six security questions, and what closes each one

Each question is closed by an artefact, not by an assurance. If the middle column is empty, the question is still open.

  • Six rows, each with a question on the left, the evidence that closes it in the middle, and the failing answer on the right.
  • Least privilege: closed by an Access Analyzer report showing no unused permissions; fails on "we follow least privilege".
  • Encryption: closed by a named KMS key id and a TLS policy; fails on "everything is encrypted".
  • Secrets: closed by a Secrets Manager ARN and a rotation date; fails on a credential in an environment variable.
  • Network isolation: closed by security group rules with no 0.0.0.0/0 on admin or database ports; fails on "it is in a private subnet".
  • Logging: closed by an organization trail in a separate log account; fails on "CloudTrail is on".
  • Vulnerability management: closed by a scan date and a named owner; fails on "Inspector is enabled".

Signature

privilege · encryption · secrets · network · logging · vulnerabilities

Six questions: privilege, encryption, secrets, network, logging, vulnerabilities. Each is closed by an artefact — a policy, a key id, a secret ARN, a security group rule, a trail in another account, a scan date with an owner. Review the account rather than the diagram, because only one of the two drifts.

2.The Reliability Pass — Name the Failure, Then Find the Control

The reliability pass works backwards from failures rather than forwards from features. Name a thing that can break — a zone, an instance, a disk, a dependency, a duplicate message — and ask which control handles it and when that control was last exercised. A control nobody has tested is a plan, and plans fail differently from mechanisms.

Key features

  • Multi-AZ is the production baseline: two zones minimum for every tier, and capacity that survives losing one of them
  • A health check that only proves the process is listening will keep routing traffic to an instance whose database connection pool is exhausted
  • A backup is only a control once a restore has been timed — an untested backup gives you an RPO number and no RTO number
  • Retries without a bound and without jitter turn a slow dependency into a self-inflicted load spike on that dependency
  • Every retry needs idempotency behind it, because at-least-once delivery and client retries both produce duplicates you cannot detect from the request alone
  • Graceful degradation is a designed behaviour: what the user sees when a non-critical dependency is down has to be decided before the incident

The reliability pass — the seven checks and what a weak answer looks like

The reliability pass — the seven checks and what a weak answer looks like
CheckThe questionPasses whenWeak answer
Multi-AZWhich tiers span two zones, and does the remaining zone have the capacity?Compute, database and cache each span two zones, sized so one zone can carry the load"It is multi-AZ" for a two-task service where one zone is running at 100%
Health checksDoes the check fail when the instance cannot actually serve a request?The path exercises the database and any critical dependency, and has its own timeoutA `/health` route that returns 200 from the web framework and touches nothing
BackupsWhat is backed up, how far back, and how long does a restore take?Retention is stated, and the last restore has a measured duration and a date"Automated snapshots are enabled" with no restore ever performed
FailoverWhat happens to in-flight connections when the primary is replaced?The application reconnects, the pool recycles, and the drill has been run"RDS handles it" — while the connection pool holds dead sockets for minutes
RetriesHow many, with what backoff, and what stops the retry storm?A bounded count, exponential backoff with jitter, and a circuit breaker"The SDK retries" with the default configuration nobody has looked at
IdempotencyWhat happens when the same request is processed twice?A key derived from the request, stored, and checked before any side effect"Duplicates are rare" — which is a statement about volume, not correctness
Graceful degradationWhich dependencies are optional, and what does the user see without them?Optional dependencies are listed, each with a decided fallback behaviourEvery dependency treated as required, so any one of them is a full outage
Each failure, its control, and the proof it works

Read left to right. A row with a control and no proof is the row that will surprise you, because the mechanism has never been observed working.

  • Seven rows pairing a failure with the control that answers it and the proof that the control works.
  • A whole Availability Zone fails: answered by multi-AZ across two zones; proved by a zone-evacuation drill.
  • One instance goes sick: answered by a deep health check plus automatic replacement; proved by terminating an instance and watching traffic move.
  • Data is deleted or corrupted: answered by backups with a retention window; proved by a timed restore.
  • The primary database fails: answered by Multi-AZ failover; proved by a forced failover with the application running.
  • A call fails once: answered by bounded retries with backoff and jitter; proved by a fault-injection experiment.
  • A retry arrives twice: answered by an idempotency key; proved by replaying the same request and comparing the result.
  • A non-critical dependency is down: answered by a designed degraded mode; proved by disabling that dependency in a drill.

Signature

multi-AZ · health checks · backups · failover · retries · idempotency · degradation

List the failures first — zone, instance, data, primary, one bad call, a duplicate, an optional dependency — then write the control beside each and the date it was last exercised. Multi-AZ without spare capacity, a health check that touches nothing, and a backup nobody has restored are all controls that exist only on paper.

3.The Performance Pass — Find the Bottleneck Before Judging the Design

The performance pass starts with one question: where does the time actually go? Until a latency breakdown exists, every recommendation is a guess. Once it exists, the six levers — compute size, cache, connection pool, database tuning, CDN, and moving work out of the request — sort themselves by which part of the breakdown they shrink.

Key features

  • The bottleneck is found in a trace, not in a design document — X-Ray or OpenTelemetry attributes latency per hop and ends the argument
  • Right-sizing compute is a two-way check: a CPU-bound task on a memory-heavy instance and an idle instance three sizes too big are both findings
  • A connection pool sized per process multiplies by worker count, so a small instance count can exhaust a database's connection limit
  • A CDN removes latency only for the requests that reach it — a cache behaviour that forwards every cookie has a hit rate near zero
  • Database tuning starts with the slow query log and an execution plan, and index changes are measured on real cardinality rather than sample data
  • Work that does not have to finish before the response — email, thumbnails, indexing, webhooks — belongs behind a queue, which is the only lever that removes time rather than shortening it

The six performance levers, what each one fixes, and when it is the wrong answer

The six performance levers, what each one fixes, and when it is the wrong answer
LeverFixesCheckWrong answer when
Right computeCPU or memory starvation, and money spent on idle capacityCPU and memory utilisation at peak, against the instance familyUtilisation is already low — the time is being spent waiting, not computing
CachingRepeated reads of data that changes slower than it is requestedHit rate, and the staleness the requirement actually allowsThe read is unique per user per request, so nothing is ever reused
Connection poolingTime spent waiting for a free connection, and database connection exhaustionPool size × worker count × task count against the engine's `max_connections`The pool is idle — the connections are being held by slow queries, not by demand
Database tuningQueries scanning far more rows than they returnThe slow query log, then an execution plan on production-shaped dataThe query is already indexed and the volume itself is the problem
CDNRound trips for anything cacheable, and origin loadCache hit ratio, and which headers and cookies the cache key includesEvery response is personalised, so the cache key is unique per request
Async processingWork the user does not need to wait forWhich steps in the handler could complete after the responseThe user genuinely needs the result before the page can render
Where the milliseconds go, and which lever moves each one

The bar is one request. Each segment has its own lever, and a lever aimed at a small segment cannot produce a large improvement.

  • A horizontal bar representing a 900 millisecond request, split into five segments with the lever for each named underneath.
  • Edge and network, 40 milliseconds: the lever is a CDN and a cache behaviour that does not forward every cookie.
  • Waiting for a database connection, 310 milliseconds: the lever is connection pool sizing or a proxy.
  • The query itself, 260 milliseconds: the lever is an index, a rewrite, or an instance with more memory.
  • Application compute, 130 milliseconds: the lever is right-sizing the compute or the code path itself.
  • Sending an email inline, 160 milliseconds: the lever is a queue, which removes the segment rather than shrinking it.
  • Footnote: adding a cache in front of the query targets 260 of 900 milliseconds, while moving the email out targets 160 and costs nothing to run.

Command

text
measure first → attribute the time per hop → pull the lever that targets the largest segment → measure again

Measure, attribute, then choose. The six levers are compute size, cache, pool, database tuning, CDN and async — and each one targets a specific segment of the request. The cheapest improvement is usually the last one: work moved out of the request entirely, which removes time rather than shortening it.

4.The Cost Pass — Read the Bill by Driver, Not by Total

A cost review looks at what the money is being spent on rather than how much it is. Sort the bill by line item, and the top three usually include something nobody designed on purpose — a NAT gateway processing traffic that could take a VPC endpoint, logs retained forever, or storage sitting in the wrong class. Those are architecture findings, not procurement findings.

Key features

  • NAT gateways charge per hour and per gigabyte processed, so a workload pulling from S3 or DynamoDB through NAT is paying for a path a free gateway endpoint would carry
  • Cross-Availability-Zone traffic between your own instances is billable, which makes chatty service-to-service calls a topology decision with a price
  • CloudWatch Logs charges for ingestion and for storage, so a debug-level log left on in production is a recurring bill rather than a one-off
  • S3 lifecycle rules move objects between classes automatically — without them, data written once and never read stays in Standard forever
  • Rightsizing is judged on peak utilisation over a representative window, not on an average that hides the peak the instance was sized for
  • A budget with an alert is the control that makes every other finding recoverable, because it converts a surprise into a notification

The cost pass — six checks, and the finding each one produces

The cost pass — six checks, and the finding each one produces
CheckWhat to look atTypical findingThe fix
RightsizingPeak CPU and memory over a representative window, per instance familyAn instance sized for a launch that never came, running at 6%Move down a size, or change family to match the actual bottleneck
AutoscalingWhether capacity follows demand, or is fixed at the peakA fixed fleet sized for the busiest hour, paid for all twenty-fourTarget tracking on the metric that actually drives load
Storage lifecycleS3 storage class distribution, EBS snapshots, old AMIsObjects written once and never read, still in Standard after two yearsA lifecycle rule per bucket prefix, plus snapshot expiry
Data transferCross-AZ traffic between your own services, and egress to the internetA chatty service pair split across zones for availability it does not needZone-aware routing where it is safe, or fewer round trips
NAT and loggingNAT gateway processed bytes; CloudWatch Logs ingested bytes per groupS3 traffic routed through NAT, and debug logging left on in productionGateway endpoints for S3 and DynamoDB; log level and retention per group
Budgets and alertsWhether a budget exists, and who receives the alertNo budget at all, so the first signal is the invoiceA budget per account with a forecast alert to a named owner
Where an unreviewed bill actually goes

The three shaded segments are architecture choices rather than capacity choices. They are the ones a review can remove rather than shrink.

  • A stacked bar of a monthly bill divided into six segments, with the design-driven ones highlighted.
  • Compute, 38 percent: addressed by rightsizing and autoscaling.
  • NAT gateway data processing, 19 percent: addressed by S3 and DynamoDB gateway endpoints, which are free.
  • CloudWatch Logs ingestion, 14 percent: addressed by log level and retention.
  • S3 storage, 12 percent: addressed by lifecycle rules moving cold objects out of Standard.
  • Databases, 11 percent: addressed by rightsizing and commitments.
  • Cross-zone data transfer, 6 percent: addressed by topology and by keeping chatty calls inside a zone.
  • Footnote: the second, third and fourth segments total 45 percent and were never designed, only accumulated.

Command

text
group the bill by usage type → separate capacity spend from accumulated architecture → remove before you shrink → leave a budget alert behind

Group the bill by usage type, then split the top lines into capacity you chose and architecture that accumulated. NAT data processing, log ingestion and storage class are the three that recur, all three are removable rather than merely shrinkable, and a budget alert is what stops the next one being discovered on an invoice.

5.The Operations Pass — Could Someone Else Run This at 3 a.m.?

The operations pass has one governing question: could an engineer who did not build this system diagnose and recover it at three in the morning? Everything else — the dashboard, the alarms, the runbook, the rollback, the audit trail — is a component of that answer, and each one either exists as an artefact or does not.

Key features

  • A dashboard is for the first two minutes of an incident: request rate, error rate, latency, saturation, and the health of each dependency on one screen
  • An alarm nobody acts on trains people to ignore alarms, so every alarm needs a documented action or it should be a dashboard line instead
  • A runbook is written for someone who has not seen the system: the symptom, the check, the command, and what the command should print
  • Deployment safety is measured by how fast you can go back, not by how carefully you go forward — an untested rollback is not a rollback
  • Incident response needs a named role per incident, not a crowd: who is deciding, who is investigating, who is telling people
  • Auditability answers who changed what and when, which is a CloudTrail question — and it only works if the trail is outside the account being audited

The operations pass — six checks, each with a test you can run in daylight

The operations pass — six checks, each with a test you can run in daylight
CheckThe questionHow to test it nowFails when
DashboardsCan one screen tell you what is broken in two minutes?Open it and name the current error rate and slowest dependencyThe answer needs three consoles and a log query
AlarmsDoes every alarm have an action and an owner?List the alarms and read what each one says to doAlarms exist with no documented action, or fire weekly and are ignored
RunbooksCan someone who did not build it follow the steps?Hand it to another team and watch them work an old incidentThe runbook says "investigate the logs" instead of naming the query
Deployment safetyHow long does it take to get back to the previous version?Roll back a deployment in staging and time itRollback is theoretical, or blocked by a forward-only migration
Incident responseWho decides, who investigates, who communicates?Read the last incident record and check the roles were namedEveryone joins the call and nobody is deciding
AuditabilityCan you say who changed what, and when, without asking anyone?Pick a change from last week and find it in CloudTrailThe trail is in the same account, short-retention, or Region-limited
The 3 a.m. path, and where each artefact is used

Every box is a place an incident can stall. The artefact underneath is the thing that stops it stalling there.

  • A five-step incident path drawn left to right, with the artefact required at each step underneath.
  • Step 1, the page arrives: requires an alarm with a documented action, otherwise the responder does not know if it matters.
  • Step 2, orient: requires one dashboard with rate, errors, latency, saturation and dependency health, otherwise the responder hunts across consoles.
  • Step 3, diagnose: requires a runbook keyed by symptom with the exact commands, otherwise the responder reads code during the incident.
  • Step 4, act: requires a tested rollback and a documented degraded mode, otherwise the only option is to keep debugging under pressure.
  • Step 5, account for it: requires CloudTrail in a separate account, otherwise nobody can say what changed just before the incident began.
  • Footnote: the question the pass answers is whether someone who did not build the system can complete all five steps.

Command

text
dashboard (2 min) → alarm with an action → runbook by symptom → tested rollback → named roles → CloudTrail elsewhere

Ask whether a stranger could recover this at 3 a.m., then check the five artefacts that decide the answer: a two-minute dashboard, alarms with actions, a runbook keyed by symptom with real commands, a rollback that has been timed, and a CloudTrail outside the account. Test it by handing the runbook to another team in daylight.

6.The Sustainability Pass — Provisioned Capacity Nobody Is Using

The sustainability pass looks for capacity that is provisioned and not doing work: instances running at single-digit utilisation, storage nobody reads, environments left on overnight, and jobs that recompute results that have not changed. It overlaps heavily with the cost pass, and the useful difference is that it also counts the work itself — a query that scans a hundred times more data than it returns is a finding even when the bill is small.

Key features

  • Utilisation is the headline measure: a fleet averaging 8% CPU is doing the same work as a smaller fleet, using more of everything to do it
  • Demand alignment means capacity follows the load curve — a non-production environment running overnight and at weekends is provisioned for hours nobody is present
  • Lifecycle management removes data that nothing reads: old snapshots, unattached volumes, superseded AMIs, and log groups with no retention set
  • Efficiency of the work counts too — a nightly job that reprocesses every record to update the few that changed does many times the work the result needs
  • The pillar is a trade-off like the others: consolidating onto fewer, busier instances reduces waste and reduces headroom, and that exchange belongs in the review notes

The sustainability pass — what to look for and what it usually turns out to be

The sustainability pass — what to look for and what it usually turns out to be
QuestionWhere to lookWhat it usually is
Is anything running at very low utilisation?CloudWatch CPU and memory at peak, per Auto Scaling groupA fleet sized for a launch that never arrived
Does non-production shut down?Instance running hours for dev and staging accountsEnvironments running 168 hours a week for 40 hours of use
Is storage still read?S3 storage class mix, EBS snapshot ages, unattached volumesObjects written once, never read, still in Standard
Do log groups expire?CloudWatch log group retention settingsRetention unset, which means never expires
Is any job doing more work than the result needs?Scheduled jobs, their input size, and how many rows they changeA nightly full reprocess where an incremental pass would do
What is the trade-off being made?The review notes themselvesConsolidation that removes waste and removes headroom with it
Four questions, and the finding each one produces

Is the capacity being used?

Utilisation at peak

single-digit CPU is a finding, not a safety margin

Unattached volumes

provisioned, billed, and serving nothing

Does capacity follow demand?

Overnight and weekends

non-production environments left running

Fixed fleet at peak size

sized for the busiest hour, paid for all of them

Is the data still needed?

Snapshots and AMIs

superseded copies with no expiry rule

Log groups

no retention set means retained forever

Is the work itself efficient?

Full reprocessing

recomputing everything to update a few rows

Scan-heavy queries

reading far more than the result contains

  • Is the capacity being used?
    • Utilisation at peak — single-digit CPU is a finding, not a safety margin
    • Unattached volumes — provisioned, billed, and serving nothing
  • Does capacity follow demand?
    • Overnight and weekends — non-production environments left running
    • Fixed fleet at peak size — sized for the busiest hour, paid for all of them
  • Is the data still needed?
    • Snapshots and AMIs — superseded copies with no expiry rule
    • Log groups — no retention set means retained forever
  • Is the work itself efficient?
    • Full reprocessing — recomputing everything to update a few rows
    • Scan-heavy queries — reading far more than the result contains

Signature

utilisation · demand alignment · lifecycle · efficient work

Look for provisioned capacity doing no work: low utilisation, environments running when nobody is there, storage nothing reads, and jobs recomputing what has not changed. Then write down what the efficiency costs you — usually headroom — because a pass that only records savings is not a review.

Remember

  • A question is closed by an artefact — a policy, a key id, a restore time, a scan date. An assurance closes nothing.
  • Write the reliability section as failures with controls beside them; any failure with an empty control column is a decision made by accident.
  • Group the bill by usage type. NAT data processing, log ingestion and storage class recur, and all three are removable rather than merely shrinkable.
  • Hand the runbook to another team in daylight. The author cannot fail the operability test, because the missing knowledge is already in their head.

Pro tip

  • Three commands cover most of the security pass: security groups filtered on 0.0.0.0/0, describe-trails for multi-Region coverage, and Access Analyzer active findings.
  • Run the passes in order. A performance or cost finding on a system that fails the security or reliability pass is a distraction from the finding that matters.

Next up

  • AWS Interview-Level Questions
  • AWS Well-Architected Framework
  • Production Debugging on AWS

Sources

  • AWS Well-Architected Framework — Security Pillar — docs.aws.amazon.com/wellarchitected/latest/security-pillar/welcome.html
  • AWS Well-Architected Framework — Reliability Pillar — docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/welcome.html
  • Checked against official AWS documentation 2026-09-05

Spotted a mistake on this page? Tell us.