beginnerintermediateadvanced

System Design · Section 48

Quotas and Fairness

Level: intermediate
Read: 18 min
Concepts: 3

A hard quota caps total usage over a period; a burst limit caps instantaneous rate — two distinct controls that commonly guard the same resource at once, and a caller can fail either independently of the other. The noisy-neighbor problem is what happens without per-tenant isolation on a shared resource: one tenant's heavy usage degrades every other tenant sharing it, even while those other tenants' own traffic stays flat — a resource-sharing design flaw that a bigger shared pool only delays, never fixes. Weighted fairness, per-tenant queues, and dedicated resource isolation are the three concrete mechanisms, sitting on a cost-vs-isolation-strength spectrum from cheapest (still shared) to most expensive (fully isolated).

System Design overview

System Design for Beginners CoursefreeCodeCamp.org

What is true here

A quota bounds total consumption over a period; a burst limit bounds rate right now — different units, different failure modes, often enforced together.
Noisy neighbor: one tenant's heavy usage degrades other tenants on a shared resource despite their own traffic being unchanged.
Increasing total shared capacity delays a noisy-neighbor incident but does not fix the underlying lack of per-tenant isolation.
Weighted fairness, per-tenant queues, and dedicated resource isolation are the three concrete fixes, trading cost for isolation strength.

What you will be able to do

Distinguish a quota failure from a burst-limit failure and pick the correct fix for each
Recognize the noisy-neighbor diagnostic signature — other tenants degrading with flat traffic
Choose the right fairness mechanism for a given contention scenario's cost/isolation trade-off

Two controls, one failure mode

Quotas and burst limits as genuinely distinct controls, and the noisy-neighbor failure both exist to guard against.

Hard quotas vs burst limits

coreintermediate

A hard quota caps total usage over a fixed period — for example, 10,000 API calls per day, or 1,000 VM-hours per month. A burst limit caps the instantaneous rate of usage — for example, no more than 50 requests per second, even if the daily quota has plenty of room left. These are two different questions ("how much in total" vs "how fast right now") and a real system typically enforces both on the same resource at once: you can be well under your monthly quota and still get throttled because you tried to do too much in one second.

Think of it as

A hard quota is like a monthly data plan on a phone — 20 GB for the month, and once it is gone it is gone until the plan resets. A burst limit is like the phone's hotspot speed cap — even on day one, with the full 20 GB untouched, the connection still will not exceed a fixed number of megabits per second at any given moment. A customer can hit either wall independently: run out of data by the 10th of the month (quota exhausted, rate irrelevant), or get a slow hotspot in the first minute of use (rate capped, total data barely touched).

text

Two independent checks, both guarding the same resource:

  quota_used_this_month  <  quota_limit        (hard quota)
  tokens_in_bucket        >  0                  (burst limit,
                                                  refills over time)

A request can fail either check independently of the other.

What we're doing: Show a caller that is well under its monthly quota still getting throttled by the burst limit, and a separate caller hitting the opposite failure.

quota-vs-burst.txttext

Account: monthly quota = 1,000,000 API calls
        burst limit = 10 requests/second (token
        bucket, capacity 10, refill 10/sec)

09:00:00  Caller sends 40 requests in the same second.
          Calls used this month so far: 40 (0.004% of
          quota — nowhere near the monthly cap).
          Burst check: bucket had 10 tokens, all 10
          spent on the first 10 requests; the other 30
          requests in that same second get
          RequestLimitExceeded (rate-limited, not
          quota-limited).

09:00:01  Bucket refills to 10 tokens. Next 10 requests
          that second succeed.

--- separately, a different failure mode ---

23:58:00  A second caller has been sending a steady
          3 requests/second all month (well under the
          10/sec burst limit — never once throttled by
          rate). By 23:58 on the last day of the month
          it has made exactly 1,000,000 calls.
          Call #1,000,001 fails: monthly quota
          exhausted. The burst limit was never the
          problem for this caller.

9: The first 10 requests in that second succeed by draining the token bucket; the remaining 30 fail on the burst check even though the account has used almost none of its monthly quota.
19: A second caller that never bursts can still exhaust the separate hard quota simply by sustaining steady traffic long enough — a completely different failure with a completely different fix (wait for reset / request a quota increase, not "slow down").

Why this works: The two callers fail for opposite reasons on the same account type, and the fix for one does nothing for the other — this is the concrete reason a design has to track and report on quota-remaining and burst-capacity-remaining as two separate signals, not one combined "limit."

Returning one generic "rate limited" error for both quota and burst failures

Wrong

text

HTTP 429 Too Many Requests
{ "error": "rate limit exceeded" }

-- same error body whether the caller burst for
   one second or exhausted the whole month's quota

Better

text

HTTP 429 Too Many Requests
{ "error": "burst_limit_exceeded",
  "retryAfterSeconds": 1 }

HTTP 429 Too Many Requests
{ "error": "monthly_quota_exhausted",
  "quotaResetsAt": "2026-09-01T00:00:00Z" }

-- distinct error codes tell the caller which
   knob to adjust and how long to actually wait

What you see: Client retry logic backs off for a second (correct for a burst limit) and immediately fails again, over and over, because the real problem was a monthly quota that will not reset for three more weeks — a generic error code gives the caller no way to distinguish "retry in 1 second" from "retry in 3 weeks."

Why: A burst limit and a hard quota fail for different reasons and recover on different timescales; collapsing them into one error code forces every caller to guess which one happened, which usually produces either wasteful tight-retry loops or overly conservative backoff that is unnecessary for the burst case.

Hard quota vs. burst limit

Hard quota

+"How much total, over this period?"
+Calls/day, VM-hours/month, storage GB
+Hitting it means: wait for reset or request an increase

Burst limit

−"How fast, right now?"
−Requests/second, tokens/second
−Hitting it means: slow down, capacity returns in seconds

Hard quota vs burst limit on the same resource

Hard quota vs burst limit on the same resource
Property	Hard quota	Burst limit
Question answered	"How much total, over this period?"	"How fast, right now?"
Typical unit	Calls/day, VM-hours/month, storage GB	Requests/second, tokens/second
Resets	At the end of the period (daily/monthly)	Continuously, as the bucket refills
Hitting it means	No more capacity until the period resets or a quota increase is granted	Slow down; capacity returns within seconds

Remember: Quota = total allowed over a period (resets on a schedule); burst limit = allowed rate right now (recovers continuously). Both commonly apply to the same resource at once — check both before deciding which one to raise.

The noisy-neighbor problem

coreintermediate

The noisy-neighbor problem is what happens when a multi-tenant system has no per-tenant isolation: one tenant's heavy usage consumes so much of a shared resource (database connections, CPU, thread pool, queue capacity) that every other tenant sharing that resource slows down or fails, even though those other tenants did nothing wrong and stayed well within any usage they would consider reasonable. It is fundamentally a resource-sharing design flaw, not a capacity shortage — adding more total capacity delays the problem but does not fix it, because nothing stops the next heavy tenant from consuming the larger pool just as completely.

Think of it as

A shared apartment building with one water heater is the noisy-neighbor problem in physical form. If one unit runs three showers and a dishwasher simultaneously, every other unit in the building gets cold water — not because the building lacks water, but because nothing allocates the shared heater's capacity per unit. Installing a bigger water heater helps until a unit with even more fixtures moves in. The actual fix is giving each unit its own allocation (a flow limiter per unit) or a fair-sharing rule for the one shared heater, not an ever-larger heater.

text

Symptom pattern that identifies noisy-neighbor
(as opposed to plain overload):

  - Total resource utilization is not exhausted
  - One tenant's usage graph spikes right before
    the incident
  - Other tenants' request volume was flat/normal,
    yet THEIR latency/error rate also spiked
  - No per-tenant cap exists on the shared resource

What we're doing: Show a shared database connection pool before and after a per-tenant cap is added.

noisy-neighbor-before-after.txttext

BEFORE: single pool, 100 connections, shared by
        all tenants, no per-tenant limit

02:00  Tenant A kicks off a nightly export job that
       opens connections as fast as the app allows —
       it climbs to 95 of 100 pool connections within
       seconds.
02:00  Tenant B's normal user traffic (2-3 concurrent
       connections typically) now waits in the pool's
       queue behind A's 95; B's request latency goes
       from 40ms to 8s, then times out.
02:00  Every other tenant sharing this pool sees the
       same timeout, despite none of them increasing
       their own traffic at all.

AFTER: same pool, 100 connections, per-tenant cap
       of 10 connections enforced by the pool wrapper

02:00  Tenant A's export job requests connections past
       its cap of 10; the 11th request queues on A's
       own sub-limit instead of taking a shared slot.
02:00  Tenant B still gets connections immediately from
       the remaining 90 slots the pool never let A touch.
02:00  A's export job simply takes longer to finish
       (bounded to 10 connections' worth of throughput)
       — the cost of A's own heavy job stays with A.

5: With no per-tenant cap, tenant A's job is free to consume 95% of the shared pool in seconds.
8: Tenant B is harmed purely by sharing a resource with A — B's own traffic never changed, which is the signature of a noisy-neighbor incident rather than an overload incident.
19: After capping tenant A at 10 connections, A's job is still allowed to run (just slower), and it can no longer take slots away from other tenants — the fix contains the cost of A's load to A alone.

Why this works: The before/after makes the diagnostic signature concrete: tenant B's failure with zero change in B's own traffic is what distinguishes "noisy neighbor" from "the system is simply out of capacity," and shows that the fix is isolation, not a bigger pool.

Fixing a noisy-neighbor incident by increasing total pool size instead of adding per-tenant isolation

Wrong

text

-- incident: tenant B timed out because tenant A's
-- job took 95 of 100 connections
-- "fix": raise the pool to 300 connections

Better

text

-- add a per-tenant cap (e.g. 10 connections per
-- tenant) inside the same pool, or give each
-- tenant its own smaller pool
-- now A's job is bounded regardless of total
-- pool size

What you see: The next tenant whose job is heavy enough (or the same tenant A running an even bigger job next month) reproduces the identical incident against the new, larger pool — the postmortem for the second incident looks just like the first one, because the actual cause (no per-tenant bound) was never addressed.

Why: A bigger shared pool raises the threshold at which the problem recurs but does not remove the underlying design flaw: any single tenant can still consume an unbounded fraction of whatever the pool's new size is, so the fix only buys time, not a structural guarantee for other tenants.

A 100-connection pool: before vs. after a per-tenant cap

Before (no cap)

+Tenant A's job takes 95 of 100 connections
+Tenant B waits in the queue, times out at 8s
+Every tenant sees the same timeout — none changed their own traffic

After (cap of 10)

−Tenant A queues on its own 10-connection sub-limit
−Tenant B gets connections immediately from the other 90
−A's job runs slower — the cost stays with A alone

Before and after: a shared connection pool with 100 slots

Before and after: a shared connection pool with 100 slots
Scenario	Tenant A (heavy)	Tenants B–Z (normal)	Outcome
Before: one shared pool, no per-tenant cap	Opens 95 of 100 connections during a batch job	Each needs 1–2 connections, pool has 5 left for 25 tenants	B–Z see connection timeouts and elevated latency during A's job
After: per-tenant cap of 10 connections	Capped at 10, batch job queues or slows for A only	Still draw from the remaining 90 slots freely	B–Z see no impact; only A's own job is affected by its own load

Remember: Noisy neighbor: one tenant's heavy usage degrades other tenants on a shared resource, even while those other tenants' own traffic is unchanged. Adding total capacity delays it; only per-tenant isolation (caps, queues, or dedicated pools) actually fixes it.

The three concrete fixes

Weighted fairness, per-tenant queues, and resource isolation, and the cost/isolation trade-off between them.

Weighted fairness, per-tenant queues, and resource isolation

standardintermediate

These are the three concrete mechanisms a design reaches for once it has identified a noisy-neighbor risk. Weighted fairness (also called weighted fair queuing) shares a resource proportionally to each tenant's assigned weight instead of first-come-first-served. Per-tenant queues give each tenant its own request queue so one tenant's backlog cannot delay another tenant's requests sitting behind it. Resource isolation gives each tenant a dedicated slice of a resource (its own connection pool, its own thread pool, its own shard) so there is no shared pool left to monopolize at all.

Think of it as

Think of three different ways an airport could handle boarding when flights share a single gate agent. Weighted fairness is priority boarding by ticket class — everyone eventually boards, but first class gets a proportionally larger share of the agent's attention. Per-tenant queues is giving each flight its own line at the same gate, so a slow-moving family in flight A's line never blocks flight B's passengers standing in a separate line. Resource isolation is giving each flight its own gate entirely — no shared agent or line to contend over in the first place, at the cost of needing more gates.

text

Weighted fair share for tenant i:

  share(i) = (weight(i) / sum(all weights)) * capacity

Example: weights {A: 1, B: 3}, capacity = 100 req/s
  share(A) = (1/4) * 100 = 25 req/s
  share(B) = (3/4) * 100 = 75 req/s

What we're doing: Compare a plain shared FIFO queue against per-tenant queues under one tenant's backlog.

per-tenant-queues.txttext

SHARED FIFO QUEUE (before)
  Tenant A enqueues 10,000 slow jobs at once.
  Tenant B enqueues 1 urgent job right after.
  Worker pulls strictly in arrival order:
    B's job sits behind all 10,000 of A's jobs.
  B's job finishes only after A's entire backlog
  drains — B has no way to jump the line.

PER-TENANT QUEUES (after)
  Tenant A's 10,000 jobs go into queue_A.
  Tenant B's 1 job goes into queue_B.
  Worker pool pulls round-robin across queues
  (one from queue_A, one from queue_B, repeat):
    B's single job is picked up on the worker's
    very next turn, regardless of how deep
    queue_A is.
  A's backlog still drains completely — just no
  longer at the cost of blocking B.

6: In a single shared FIFO queue, tenant B's one job is stuck behind all 10,000 of tenant A's jobs purely because of arrival order — no relationship to B's own usage.
15: With one queue per tenant and round-robin pulling across queues, tenant B's job gets picked up within one worker cycle no matter how large tenant A's backlog is.

Why this works: The failure in the "before" case is not capacity (the workers are the same, the total work is the same) — it is purely queue ordering, which is exactly what per-tenant queues fix without needing any extra hardware.

Three fairness mechanisms, cost vs. isolation

Weighted fairness

proportional share, resource stays shared

Per-tenant queues

ordering isolated

Resource isolation

dedicated slice per tenant

Three fairness mechanisms compared

Three fairness mechanisms compared
Mechanism	What it isolates	Cost	Weakness
Weighted fairness	Share of throughput, proportional to weight	Low — resource stays shared	Requires choosing and maintaining weights per tenant
Per-tenant queues	Ordering — one tenant's backlog cannot block another's	Moderate — more queues to manage	Underlying resource (workers, connections) is still shared and can still be exhausted in aggregate
Resource isolation	Full capacity — dedicated pool/shard per tenant	High — idle capacity cannot be reused across tenants	Provisioning for peak-per-tenant wastes capacity most of the time

Remember: Weighted fairness shares one resource proportionally by weight; per-tenant queues stop one tenant's backlog from blocking another's ordering; resource isolation gives each tenant a dedicated slice with no sharing at all. Pick the cheapest one that removes the actual contention.