beginnerintermediateadvanced

AWS · Section 11

Load Balancing

Level: intermediate
Read: 30 min
Concepts: 5

A load balancer is the single entry point that spreads traffic across healthy targets — but which type, how requests get routed, and where those targets actually live are all deliberate design choices. This section covers ALB vs NLB and when each fits, the listener/rule/target-group chain that decides where a request goes, why a stateless application is what makes scaling behind a load balancer free, cross-zone balancing and health-check failure scenarios, and the standard internet-to-private-target architecture.

This section

AWS Elastic Load Balancing IntroductionStephane Maarek

What is true here

ALB operates at Layer 7 (HTTP-aware, path/host routing); NLB operates at Layer 4 (extreme throughput, static IPs, no HTTP visibility).
A listener catches connections on a protocol/port; its rules pick a target group; each target group tracks its own targets' health independently.
A stateless application (no data living in only one target's memory) is what makes Auto Scaling and rolling deployments safe without client-visible impact.
ALB always cross-zone balances; NLB defaults to off — an uneven target count across AZs needs it explicitly enabled.
The standard design places the load balancer in public subnets and the application in private subnets, reachable only from the load balancer's security group.

What you will be able to do

Choose between ALB and NLB based on what an application's routing and performance needs actually are
Trace a request through a listener, its rules, and a target group to the specific target that serves it
Explain why statelessness is what makes Auto Scaling and rolling deployments safe
Decide whether cross-zone load balancing needs to be explicitly enabled for a given load balancer type
Design the standard internet → load balancer → private target architecture, including the app tier's outbound NAT needs

From an internet request to a healthy, stateless target

ALB or NLB chosen

by what the app actually needs

Listener → rules → target group

Cross-zone balanced, health-checked

Stateless private target

any target can serve any request

Load Balancing

ALB vs NLB, the listener/rule/target-group chain, why statelessness matters, cross-zone balancing, and the standard public-to-private design.

Application Load Balancer vs Network Load Balancer

coreintermediate

An ALB understands HTTP — it can route by URL path or hostname. An NLB operates purely at the connection level, handling extreme throughput and static IPs but with no visibility into the HTTP request at all.

Think of it as

ALB is a receptionist who reads what you're asking for and sends you to the right department. NLB is a turnstile that lets traffic through fast based only on which door you approached, with no idea what you're carrying.

text

ALB: Layer 7 (HTTP) — path/host routing, content-aware
NLB: Layer 4 (TCP/UDP) — extreme throughput, static IP, protocol-agnostic

What we're doing: See a routing decision an ALB can make that an NLB structurally cannot.

alb-path-routing.txttext

Listener rule (ALB):
  IF path starts with /api/*  → target-group: api-service
  IF path starts with /admin/* → target-group: admin-service
  ELSE                         → target-group: web-frontend

2: This decision requires reading the HTTP request path — a capability that exists only at Layer 7.
3: A single ALB is routing three logically distinct services from one entry point, based purely on URL structure.

Why this works: An NLB forwards TCP connections based on IP/port alone — it has no concept of an HTTP path or host header to route on, because that information does not exist yet at the transport layer it operates on.

Choosing NLB for a web application that needs path-based routing

Wrong

text

# "We need high performance, so let's use NLB for our microservices."

Better

text

# If routing needs to inspect the HTTP request (path, host, headers),
# that requires ALB — NLB has no visibility into that layer at all

What you see: Path-based routing to different microservices cannot be configured at all on the load balancer — every listener rule option available is IP/port-based only.

Why: NLB's performance advantage comes precisely from not inspecting the HTTP payload — that same property is what makes content-based routing impossible at that layer, regardless of configuration effort.

ALB (Layer 7) vs NLB (Layer 4)

ALB

+Reads the HTTP request — path, host, headers
+Routes by URL structure
+Integrates with Cognito/OIDC auth

NLB

−Forwards by IP/port alone — no HTTP visibility
−Extreme throughput, ultra-low latency
−Supports static/Elastic IPs

When each load balancer type fits

When each load balancer type fits
Need	Choice
Path/host-based routing for a web app	ALB
Millions of requests/sec, ultra-low latency	NLB
A fixed IP address for allowlisting	NLB (supports static/Elastic IPs)
Non-HTTP TCP/UDP protocols	NLB
User authentication at the load balancer	ALB (integrates with Cognito/OIDC)

Together

bash

aws elbv2 create-load-balancer --name web-alb --type application --subnets subnet-0a subnet-0b

Remember: ALB is Layer 7 (HTTP-aware, path/host routing) — the default for web apps. NLB is Layer 4 (TCP/UDP, no HTTP visibility) — for extreme throughput, static IPs, or non-HTTP protocols.

Listeners, rules, and target groups

coreintermediate

A listener checks incoming connections on a port; its rules decide which target group handles each one; a target group is the actual pool of instances/IPs/Lambdas the traffic goes to, each with its own health check.

Think of it as

A listener is the front desk that takes every call on one phone line. Its rules are the instructions for which department to transfer each call to. A target group is that department's own roster of people who can actually take the call — and each roster is checked separately for who is currently available.

text

Listener (port 443, TLS) → Rules (priority order) → Target group → Targets (health-checked)

What we're doing: See TLS termination change what the target actually receives.

tls-termination.txttext

Client → HTTPS (encrypted) → ALB listener :443
ALB decrypts, forwards → HTTP (plain) → target :8080

# unless the target group's protocol is also HTTPS,
# re-encrypting for the hop to the target

2: The ALB terminates TLS at the listener — decryption happens here, using a certificate the ALB itself holds.
5: By default the second hop (ALB to target) is plain HTTP unless the target group is explicitly configured for HTTPS end-to-end.

Why this works: Terminating TLS at the load balancer moves certificate management to one place (ACM-issued certs on the ALB) instead of every target — but it also means traffic between the ALB and targets is unencrypted by default, worth knowing explicitly rather than assuming.

Assuming a target group's health check reflects the whole load balancer's health

Wrong

text

# "The ALB is healthy" — checked by pinging the ALB's own DNS name once

Better

text

# Health checks are per target group — check the specific target group
# serving the affected route, not the load balancer as a whole

What you see: One microservice behind the ALB is failing its health check and receiving no traffic, while the ALB itself and every other service behind it remain completely healthy.

Why: A load balancer with multiple target groups (one per microservice, say) tracks health independently per group — one group failing does not indicate anything about the load balancer's own health or any other group's.

From an incoming request to a healthy target

Listener :443

TLS termination happens here

Listener rules

path/host match, priority order

Target group

own health check settings

Healthy target

The pieces and what each one is responsible for

The pieces and what each one is responsible for
Piece	Responsible for
Listener	Checking for connections on a protocol/port
Listener rule	Deciding which target group handles a given request
Target group	The pool of targets and their own health check settings
Health check	Per target group — determines which targets receive traffic
Sticky sessions	Cookie-based binding of a client to one target

Together

bash

aws elbv2 create-target-group --name api-tg --protocol HTTP --port 8080 \
  --health-check-path /healthz --vpc-id vpc-0abc123

Remember: A listener catches connections, its rules pick a target group, and each target group tracks its own targets' health independently — TLS termination at the ALB means the ALB-to-target hop is plain HTTP by default.

Why statelessness makes scaling behind a load balancer easy

standardintermediate

If any target can correctly answer any request — because nothing needed for that request lives only in one target's memory — then adding, removing, or replacing targets never breaks anything mid-flight. That is what "stateless" buys you.

Think of it as

A call center where any agent can pull up a caller's full history from a shared system, versus one where only the agent you first spoke to remembers anything about your case — the shared-history call center can freely add or remove agents without ever dropping a caller's context.

text

Stateless: any target serves any request  →  add/remove targets freely
Stateful (local memory): request must return to the SAME target  →  scaling breaks things

What we're doing: See what happens to an in-flight user session when a target is replaced, comparing a stateless design to one holding local state.

target-replacement.txttext

# Stateless — session in ElastiCache, reachable by every target
target-1 replaced by Auto Scaling → target-2 reads the same session from ElastiCache
→ user sees no interruption

# Stateful — session held in target-1's local memory only
target-1 replaced by Auto Scaling → session data is gone, it existed only there
→ user is unexpectedly logged out

2: The replacement target has the exact same access to session state as the one it replaced — nothing was lost because nothing lived only on the old target.
6: The session existed nowhere else — replacing the instance that held it destroys it, with no other target able to recover it.

Why this works: Auto Scaling, rolling deployments, and even routine instance health replacement all assume any target can be swapped for another at any time — that assumption is only safe if the application itself has no state that exists in exactly one place.

Storing session data in local instance memory "just for now"

Wrong

python

sessions = {}   # in-process dict, lost if this specific instance restarts or is replaced

Better

python

session = redis_client.get(f"session:{session_id}")   # any target can read the same data

What you see: Sessions are randomly lost during deployments or Auto Scaling events, and the failure rate correlates exactly with how often instances get replaced — not with any application bug.

Why: Local in-process state is invisible to every other target and to the load balancer — nothing about the infrastructure knows it needs to be preserved, so ordinary, expected events like a rolling deployment silently destroy it.

Remember: A stateless application has no data that lives in only one target's memory — that single property is what makes Auto Scaling, rolling deployments, and load balancer routing all safe without special-casing.

Cross-zone balancing and health-check failures

coreintermediate

With cross-zone load balancing on, every load balancer node distributes evenly across ALL targets in ALL enabled AZs, not just the targets in its own AZ — without it, an AZ with fewer targets gets disproportionately less traffic per target.

Think of it as

Without cross-zone balancing, each city's dispatch office only sends calls to agents in that same city, even if one city has far fewer agents than another — those few agents get overloaded while agents elsewhere sit idle.

text

Without cross-zone: AZ-a node → only AZ-a targets
With cross-zone:    AZ-a node → targets in ANY enabled AZ

What we're doing: See the traffic-per-target imbalance that results from an uneven target count across AZs, without cross-zone balancing.

imbalance.txttext

AZ-a: 1 target   — receives 100% of AZ-a's incoming traffic alone
AZ-b: 4 targets  — split AZ-b's incoming traffic four ways

# Each load balancer node gets roughly equal traffic, but the single
# target in AZ-a ends up handling far more traffic per-target than
# any one target in AZ-b

2: This single target absorbs everything routed to AZ-a, with no other target in that AZ to share the load.
3: These four targets each handle a quarter of AZ-b's traffic — a much lighter individual load than the AZ-a target.

Why this works: Cross-zone load balancing fixes exactly this — it lets any load balancer node send to any target, regardless of AZ, so the actual per-target traffic evens out across the whole fleet instead of being pinned to whichever AZ the request happened to land in.

Assuming NLB has cross-zone balancing on by default, like ALB

Wrong

text

# "It's a load balancer, so it must already balance evenly across every AZ."

Better

text

# NLB defaults to cross-zone OFF — check and explicitly enable it if
# target counts are uneven across AZs

What you see: Targets in an AZ with fewer instances run consistently hotter than targets elsewhere, and the imbalance is not explained by anything in the application itself.

Why: ALB and NLB do not share this default — ALB always balances cross-zone, but NLB requires it to be explicitly enabled. Assuming NLB behaves like ALB here is a natural but incorrect assumption if only one of the two has been used before.

Without cross-zone: uneven targets, uneven load per target

AZ-a — 1 target

gets 100% of AZ-a traffic alone

runs hot

AZ-b — 4 targets

splits AZ-b traffic 4 ways

each target runs light

Remember: ALB always cross-zone balances; NLB defaults to off — check target counts per AZ, and make sure health-check timing matches real application startup, not an assumed instant-ready state.

Designing internet → load balancer → private targets

coreintermediate

The standard web architecture: the load balancer sits in public subnets (reachable from the internet), while the actual application instances sit in private subnets, reachable only from the load balancer's security group — never directly from the internet.

Think of it as

A store's public storefront (the load balancer, in public subnets) versus its stockroom (the application, in private subnets) — customers interact with the storefront, and staff bring exactly what's needed from the stockroom, but nobody walks into the stockroom directly from the street.

text

Internet → ALB (public subnet, sg: 0.0.0.0/0:443)
         → App targets (private subnet, sg: source = ALB's security group only)

What we're doing: Compare the app tier's security group rule for this design against a rule that accidentally allows direct internet access.

app-tier-sg.jsonjson

{
  "IpPermissions": [
    { "IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080,
      "UserIdGroupPairs": [{ "GroupId": "sg-0alb456" }] }
  ]
}

3: The source is the load balancer's security group, not a CIDR block — the app tier is reachable only through the load balancer, by construction.

Why this works: Even if the app tier were mistakenly placed in a public subnet with a public IP, this security group rule alone would still block direct internet access — but the private-subnet placement is a second, independent layer that holds even if the security group is ever misconfigured.

Placing the application tier in the same public subnet as the load balancer

Wrong

text

# ALB and app instances both in the same public subnet, "for simplicity"

Better

text

# ALB in public subnets, app instances in private subnets — two
# independent layers instead of relying on the security group alone

What you see: A later security group change (or an instance launched with a public IP by mistake) makes the application directly internet-reachable, bypassing the load balancer entirely.

Why: Public subnet placement plus a public IP is enough for direct internet reachability, regardless of what a security group says at any given moment — putting the app tier in a private subnet removes that path structurally, not just by policy.

The standard public-to-private web architecture

Internet

Load balancer

public subnet

App targets

private subnet

Remember: Load balancer in public subnets, application in private subnets, target security group scoped to the load balancer's security group as its only source — this layering holds even if any one piece is later misconfigured.