Rate Limiting

Key features

• Fixed window: count requests in a clock-aligned interval (e.g. per-minute); simplest to implement, but the counter resets abruptly at the window boundary
• The fixed-window boundary-burst problem: because two counters are independent, a caller can send the full limit at the very end of one window and the full limit again at the very start of the next, doubling the effective rate in a short span straddling the boundary
• Sliding window: evaluates the limit over a continuously moving time range (e.g. "the last 60 seconds from now") rather than a clock-aligned bucket, closing the boundary-burst gap at the cost of more state (a log of timestamps, or a weighted blend of the current and previous fixed windows)
• Token bucket: a bucket refills with tokens at a fixed rate up to a cap; each request consumes one token, and unused tokens accumulate — so it naturally allows a burst up to the bucket size after an idle period, then throttles to the refill rate
• Leaky bucket: requests queue into a bucket that drains (is "leaked") at a constant, fixed rate; it smooths bursty arrivals into steady output but can add queuing delay, and requests are dropped if the bucket is full when they arrive

Command

fixed window:    count++ in bucket(now / window_size); reset at boundary
sliding window:   count requests where timestamp > (now - window_size)
token bucket:      tokens = min(cap, tokens + elapsed * refill_rate); allow if tokens >= 1
leaky bucket:      queue.push(request); drain queue at fixed_rate; drop if queue full

Key features

• An in-process counter (a plain variable or in-memory map inside the app) only sees requests routed to that specific instance — it cannot see traffic another instance is handling for the same caller
• With N stateless instances behind a load balancer, an in-process-only limiter effectively multiplies the intended per-caller limit by up to N
• Shared-state approach: every instance checks and increments a counter in a fast shared store (typically Redis, using an atomic INCR + EXPIRE or a Lua script) so the count reflects total traffic across all instances
• Edge-level approach: enforce the limit at a single chokepoint all traffic passes through before reaching any app instance — an API gateway, reverse proxy, or CDN/edge network — so there is only ever one counter, by construction, no cross-instance coordination needed
• The two approaches are not mutually exclusive: a system might rate-limit coarsely at the edge (cheap, catches obvious abuse early) and enforce a finer per-user or per-endpoint limit with shared state closer to the application

Command

# in-process (broken once there is more than one instance)
counter = 0  # lives in this process's memory only
if counter >= limit: reject()
counter += 1

# shared-state (correct across instances) — see the Redis
# INCR + EXPIRE pattern already covered under Redis for
# distributed caching, rather than re-deriving it here

Key features

• Per-user (or per-account): the most precise scope for authenticated abuse — one user cannot exceed their limit regardless of how many IPs or devices they use — but requires the caller to already be authenticated before the limit can apply
• Per-IP: works for unauthenticated endpoints (login, signup, password reset) where there is no user identity yet, but a shared IP (corporate NAT, mobile carrier NAT, public Wi-Fi) means many legitimate users can trip one IP-based limit together, and a distributed attacker rotating IPs evades it entirely
• Per-API-key: the standard scope for third-party/programmatic access — ties the limit to a specific integration or client application rather than a human, and keys can carry different limits (tiered plans) unlike a blanket per-user limit
• Per-endpoint: the same caller can have different limits on different routes — a cheap read endpoint tolerates a high limit, while an expensive write, search, or password-reset-email endpoint gets a much tighter one, independent of who is calling it
• Real systems typically combine scopes rather than picking one: e.g. a login endpoint might limit by IP (catch a single-source brute force) AND by account username (catch credential stuffing spread across many IPs targeting one account)

Command

# a limit is a (key, count, window) triple — the key decides the scope
key = user_id                     # per-user
key = client_ip                   # per-IP
key = api_key                     # per-API-key
key = f"{api_key}:{endpoint}"     # per-key, per-endpoint (combined scope)
key = f"{client_ip}:{username}"   # per-IP AND per-account (login example)