Quotas and Fairness

Total usage vs instantaneous rate, the noisy-neighbor failure, and the three mechanisms that prevent it

Quota vs bursttotal over time vs rate right now
Noisy neighborone tenant degrades others with flat traffic
Three fixesweighted fairness, per-tenant queues, isolation
Isolation coststrongest guarantee, least resource sharing

1.Hard quotas vs burst limits

A hard quota caps total usage over a fixed period — for example, 10,000 API calls per day, or 1,000 VM-hours per month. A burst limit caps the instantaneous rate of usage — for example, no more than 50 requests per second, even if the daily quota has plenty of room left. These are two different questions ("how much in total" vs "how fast right now") and a real system typically enforces both on the same resource at once: you can be well under your monthly quota and still get throttled because you tried to do too much in one second.

Key features

A quota bounds total consumption over a period (requests/day, VM-hours/month); a burst limit bounds instantaneous rate (requests/second) — different units, different failure modes
The two are typically enforced together on the same resource: AWS EC2 API throttling defines a token-bucket "bucket maximum capacity" (burst) and a "bucket refill rate" (sustained rate) for every API action, on top of separate account-level service quotas
Google Cloud names this split explicitly: "allocation quotas" restrict total resource held (e.g. VM count), "rate quotas" restrict consumption speed over a window (e.g. per minute)
Exhausting a burst limit is usually transient (wait a second, retry); exhausting a hard quota usually requires a period reset or an explicit quota increase request
Conflating the two in a design leads to picking the wrong lever: raising a rate limit does not help a caller that is failing because a monthly quota is exhausted, and vice versa

Hard quota vs burst limit on the same resource

Hard quota vs burst limit on the same resource
Property	Hard quota	Burst limit
Question answered	"How much total, over this period?"	"How fast, right now?"
Typical unit	Calls/day, VM-hours/month, storage GB	Requests/second, tokens/second
Resets	At the end of the period (daily/monthly)	Continuously, as the bucket refills
Hitting it means	No more capacity until the period resets or a quota increase is granted	Slow down; capacity returns within seconds

Hard quota vs. burst limit

Hard quota

"How much total, over this period?"
Calls/day, VM-hours/month, storage GB
Hitting it means: wait for reset or request an increase

Burst limit

"How fast, right now?"
Requests/second, tokens/second
Hitting it means: slow down, capacity returns in seconds

Command

text

Two independent checks, both guarding the same resource:

  quota_used_this_month  <  quota_limit        (hard quota)
  tokens_in_bucket        >  0                  (burst limit,
                                                  refills over time)

A request can fail either check independently of the other.

Quota = total allowed over a period (resets on a schedule); burst limit = allowed rate right now (recovers continuously). Both commonly apply to the same resource at once — check both before deciding which one to raise.

2.The noisy-neighbor problem

The noisy-neighbor problem is what happens when a multi-tenant system has no per-tenant isolation: one tenant's heavy usage consumes so much of a shared resource (database connections, CPU, thread pool, queue capacity) that every other tenant sharing that resource slows down or fails, even though those other tenants did nothing wrong and stayed well within any usage they would consider reasonable. It is fundamentally a resource-sharing design flaw, not a capacity shortage — adding more total capacity delays the problem but does not fix it, because nothing stops the next heavy tenant from consuming the larger pool just as completely.

Key features

Named after the "noisy neighbor" effect in multi-tenant cloud/virtualization systems, where one tenant's I/O or CPU load degrades performance for co-located tenants on shared hardware
A single unbounded connection pool, thread pool, or queue shared across tenants is the most common root cause — one tenant's burst exhausts the pool before per-tenant limits even get a chance to apply
Kubernetes documents the same class of problem at the namespace level: without a ResourceQuota per namespace, one team's workloads can consume cluster capacity that other teams' namespaces needed
The problem is invisible under light load and only appears under contention — it is easy to ship a design that "works fine" in testing because no single tenant was ever heavy enough to starve the others
It is distinct from a plain capacity shortage: even a system with abundant total capacity has a noisy-neighbor problem if any single tenant can consume an unbounded share of it

Before and after: a shared connection pool with 100 slots

Before and after: a shared connection pool with 100 slots
Scenario	Tenant A (heavy)	Tenants B–Z (normal)	Outcome
Before: one shared pool, no per-tenant cap	Opens 95 of 100 connections during a batch job	Each needs 1–2 connections, pool has 5 left for 25 tenants	B–Z see connection timeouts and elevated latency during A's job
After: per-tenant cap of 10 connections	Capped at 10, batch job queues or slows for A only	Still draw from the remaining 90 slots freely	B–Z see no impact; only A's own job is affected by its own load

A 100-connection pool: before vs. after a per-tenant cap

Before (no cap)

Tenant A's job takes 95 of 100 connections
Tenant B waits in the queue, times out at 8s
Every tenant sees the same timeout — none changed their own traffic

After (cap of 10)

Tenant A queues on its own 10-connection sub-limit
Tenant B gets connections immediately from the other 90
A's job runs slower — the cost stays with A alone

Command

text

Symptom pattern that identifies noisy-neighbor
(as opposed to plain overload):

  - Total resource utilization is not exhausted
  - One tenant's usage graph spikes right before
    the incident
  - Other tenants' request volume was flat/normal,
    yet THEIR latency/error rate also spiked
  - No per-tenant cap exists on the shared resource

Noisy neighbor: one tenant's heavy usage degrades other tenants on a shared resource, even while those other tenants' own traffic is unchanged. Adding total capacity delays it; only per-tenant isolation (caps, queues, or dedicated pools) actually fixes it.

3.Weighted fairness, per-tenant queues, and resource isolation

These are the three concrete mechanisms a design reaches for once it has identified a noisy-neighbor risk. Weighted fairness (also called weighted fair queuing) shares a resource proportionally to each tenant's assigned weight instead of first-come-first-served. Per-tenant queues give each tenant its own request queue so one tenant's backlog cannot delay another tenant's requests sitting behind it. Resource isolation gives each tenant a dedicated slice of a resource (its own connection pool, its own thread pool, its own shard) so there is no shared pool left to monopolize at all.

Key features

Weighted fair queuing (WFQ) allocates a shared resource's capacity proportionally: a flow with weight 2 gets twice the share of a flow with weight 1, and each flow is protected from starving the others — this is a real, named network-scheduling algorithm, not just an analogy
Per-tenant queues (or per-tenant sub-limits within a shared pool) mean one tenant's backlog only delays that tenant's own future requests, never another tenant's — this is the fix an unbounded FIFO shared queue is missing
Resource isolation (dedicated pools, dedicated shards, dedicated compute) removes contention structurally by giving each tenant capacity nothing else can touch — strongest guarantee, but the most expensive because idle capacity in one tenant's isolated slice cannot be borrowed by another
Kubernetes' own ResourceQuota docs note that when total cluster capacity is less than the sum of namespace quotas, contention is resolved first-come-first-served, not fairly — quotas alone are not a fairness mechanism; they only cap monopolization
These three sit on a spectrum of cost vs isolation strength: weighted fairness (cheapest, resources still shared) → per-tenant queues (moderate, ordering isolated) → dedicated resource isolation (most expensive, full isolation)

Three fairness mechanisms compared

Three fairness mechanisms compared
Mechanism	What it isolates	Cost	Weakness
Weighted fairness	Share of throughput, proportional to weight	Low — resource stays shared	Requires choosing and maintaining weights per tenant
Per-tenant queues	Ordering — one tenant's backlog cannot block another's	Moderate — more queues to manage	Underlying resource (workers, connections) is still shared and can still be exhausted in aggregate
Resource isolation	Full capacity — dedicated pool/shard per tenant	High — idle capacity cannot be reused across tenants	Provisioning for peak-per-tenant wastes capacity most of the time

Three fairness mechanisms, cost vs. isolation

Weighted fairness

proportional share, resource stays shared

Per-tenant queues

ordering isolated

Resource isolation

dedicated slice per tenant

Command

text

Weighted fair share for tenant i:

  share(i) = (weight(i) / sum(all weights)) * capacity

Example: weights {A: 1, B: 3}, capacity = 100 req/s
  share(A) = (1/4) * 100 = 25 req/s
  share(B) = (3/4) * 100 = 75 req/s

Weighted fairness shares one resource proportionally by weight; per-tenant queues stop one tenant's backlog from blocking another's ordering; resource isolation gives each tenant a dedicated slice with no sharing at all. Pick the cheapest one that removes the actual contention.

Spotted a mistake on this page? Tell us.

Read Sheet