beginnerintermediateadvanced

Django · Section 31

Concurrency and Race Conditions

Level: advanced
Read: 20 min
Concepts: 2

Recognizing the classic read-modify-write race (two requests reading the same stale value, one write silently overwriting the other with no crash and no error) and matching the right tool to the specific race — F() when no decision is needed, select_for_update() when a genuine read-then-decide step is required, a database constraint when the database itself should reject a bad outcome. The genuinely new ground this section adds: idempotency keys (a client-supplied unique value, backed by a real database constraint, that makes a retried request recognizable as a duplicate rather than being double-applied) and conditional updates (folding a precondition directly into an UPDATE's WHERE clause as one atomic statement, checking the returned row count to know whether the write actually happened).

This section

What is true here

A race condition is a silent lost update from a window between reading a value and writing based on it — dangerous specifically because nothing crashes or raises an error.
The right fix depends on the specific race: F() when no decision is needed (no read step at all), select_for_update() when a genuine decision requires reading the current value first, a database constraint when the database should reject a bad outcome after the fact.
An idempotency key (a client-supplied value with a REAL unique constraint) makes a retried request recognizable as a duplicate — get_or_create() is only genuinely atomic with that constraint in place, per Django's own documented caveat.
A conditional update (filter(pk=id, expected_state).update(new_state)) folds the precondition into the UPDATE's WHERE clause as one atomic statement, avoiding a separate read-then-check-then-write race entirely.
update()'s returned row count is the only signal for whether a specific call actually made a conditional transition — a side effect that should only happen once must be gated on that count, not run unconditionally.

What you will be able to do

Recognize a read-modify-write race condition and identify the specific window that enables it
Choose the correct concurrency tool for a given race, rather than reaching for one reflexively
Use an idempotency key correctly, backed by a real uniqueness constraint
Write a conditional update and correctly gate side effects on its returned row count

Recognizing the pattern, and the toolkit

The classic balance race, and matching the right fix to the specific race rather than reaching for one reflexively.

Recognizing the pattern, and the toolkit

coreadvanced

The classic race: two requests both READ balance=100, both independently decide to write, and the second write overwrites the first — a lost update, not a crash, which is exactly why it is dangerous (nothing fails loudly). Every fix shares the same shape: remove the window between reading and writing, either by never reading at all (F()), by locking the row for the duration of the decision (select_for_update()), or by making the database itself reject the bad outcome (a unique/check constraint) rather than trusting application code to avoid it.

Think of it as

A race condition is not about speed — it is about a WINDOW between "read a value" and "write based on it" during which another process can act on the same, now-stale, information. Request A reads balance=100, request B reads balance=100 (same identical value, because A has not written yet), A computes and writes 60, B computes and writes 40 — the correct answer (100 minus both changes) never gets written, because B's calculation was based on a value that was already stale by the time B wrote. Every fix in the toolkit closes this window a different way: F() removes the read step entirely (the database computes the new value from its own current value, atomically); select_for_update() keeps the read but locks the row so no other transaction can read-and-decide during the window; a unique/check constraint does not prevent the race at all — it lets the race happen, but makes the DATABASE refuse the outcome that would violate the rule, converting a silent lost update into a loud, catchable error.

python

# ask first: is there a window between a READ and a WRITE based on it?
# if yes — F() (no decision needed), select_for_update() (a decision IS needed),
# or a database constraint (let the database reject the bad outcome)

What we're doing: Recognize a balance-update race in existing code, and choose the right fix based on whether the operation actually needs to read-and-decide, or is a pure unconditional change.

accounts/services.pypython

def withdraw(account_id, amount):
    account = Account.objects.get(pk=account_id)   # READ — a window opens here
    account.balance -= amount                        # DECIDE, based on a value that may be stale
    account.save()                                    # WRITE — may overwrite a concurrent change

2: This read has no protection at all — another concurrent withdraw() call for the same account can read the same starting balance before this one writes.
3: The actual bug is here — the new balance is computed from a value that is only correct at the moment it was read, not necessarily at the moment it gets written.

Why this works: This function genuinely needs a DECISION (does the account have sufficient funds? that check needs the current balance) — so the correct fix is select_for_update() (lock the row, check, then write), not a plain F() expression, since F() has no way to express "only subtract if the resulting balance would be non-negative."

Assuming any function that reads then writes a value automatically has a race condition, even when Python-level concurrency (not database concurrency) is the actual environment

Wrong

python

# adding select_for_update() reflexively, even in a genuinely single-threaded,
# single-process management command with no concurrent access possible
def one_off_migration_script():
    with transaction.atomic():
        for account in Account.objects.select_for_update():
            account.balance = recalculate(account)
            account.save()

Better

python

# no lock needed — genuinely no concurrent writer exists for this script's run
def one_off_migration_script():
    for account in Account.objects.iterator():
        account.balance = recalculate(account)
        account.save(update_fields=["balance"])

What you see: Not a bug — unnecessary complexity and lock overhead added to code that never actually runs under any concurrent access at all, based on a reflexive "reads then writes, must need locking" pattern-match rather than an actual analysis of the deployment context.

Why: A race condition requires an actual SECOND concurrent actor capable of reading/writing the same data during the window — a one-off, single-run management script with no other process touching the same rows has no race to protect against, and select_for_update() there only adds lock overhead and complexity for a scenario that cannot occur. Recognizing the pattern means checking whether a genuine concurrent writer exists, not applying the fix reflexively to every read-then-write shape.

The lost-update race: two reads of the same stale value

Request A

balance row

Request B

1. read balance=100
2. read balance=100
3. write 60 (100-40)
4. write 40 (100-60) — overwrites A

The concurrency toolkit, mapped to where it is covered in depth

The concurrency toolkit, mapped to where it is covered in depth
Tool	Removes the race by	Covered in depth at
F() expressions	no read step at all — the database computes the new value from its current one	django.query-expressions.f-and-q-expressions
select_for_update()	locking the row so no other transaction can read-and-decide concurrently	django.select-for-update.row-level-locking
transaction.atomic()	grouping the read+write as one all-or-nothing unit	django.transactions.atomic-and-nested-blocks
Unique / Check constraints	letting the database reject a rule-violating outcome, after the fact	django.models.constraints-and-indexes
Idempotency keys / conditional updates	making a RETRIED request safe to repeat without double-applying	this section's own companion concept

Together

python

# the race:
account = Account.objects.get(pk=1)   # reads balance=100
account.balance -= 40                  # computed from a value that might already be stale
account.save()                         # overwrites whatever the balance actually is by now

# the fix (F(), no read step):
Account.objects.filter(pk=1).update(balance=F("balance") - 40)

Remember: A race condition is a silent lost update from a window between reading a value and writing based on it — not a crash, which is why it is dangerous. F() removes the read step entirely (best when no decision is needed); select_for_update() locks the row for a genuine read-then-decide need; a unique/check constraint lets the database reject a bad outcome after the fact. Match the specific tool to the specific race — they are not interchangeable.

Idempotency keys and conditional updates

The two tools specific to making a retried request safe to repeat without double-applying.

Idempotency keys and conditional updates

coreadvanced

An idempotency key is a client-supplied unique value (e.g. a UUID generated once per logical operation) stored with a UNIQUE constraint — a retried request with the SAME key hits the constraint and is safely recognized as a duplicate, rather than double-applying (double-charging a payment, double-decrementing stock). A conditional update adds the expected PRIOR state directly into the WHERE clause of an UPDATE (update(...).filter(status="PENDING")) — the write only applies if that precondition still holds, and the return value (rows affected) tells the caller whether it actually happened.

Think of it as

Both tools solve the same underlying problem from different angles: "what happens if this exact operation is attempted more than once" — a network retry, a user double-clicking submit, an at-least-once message queue redelivering the same job. An idempotency key makes the OPERATION itself deduplicatable — get_or_create(idempotency_key=key, defaults={...}) either creates the row (first attempt) or finds the existing one (a retry), and Django's own docs note this pattern is only truly atomic when the lookup field has a real database-level uniqueness constraint backing it. A conditional update instead makes the WRITE itself self-checking — instead of read-then-check-then-write (three steps, a race window between them), the check is folded directly into the UPDATE's WHERE clause, so the database atomically does "write only if this precondition still holds" in one single statement, and the affected-row count tells you whether it happened.

python

Charge.objects.get_or_create(idempotency_key=key, defaults={...})
Order.objects.filter(pk=id, status="PENDING").update(status="PAID")   # 0 or 1+ rows affected

What we're doing: Make a payment-confirmation endpoint safe against network retries, using an idempotency key to recognize a duplicate request and a conditional update to make the actual state transition atomic.

payments/views.pypython

def confirm_payment(request_key, order_id):
    charge, created = Charge.objects.get_or_create(
        idempotency_key=request_key, defaults={"order_id": order_id},
    )
    if not created:
        return charge   # this exact request was already processed — safe no-op

    updated = Order.objects.filter(pk=order_id, status="PENDING").update(status="PAID")
    if updated == 0:
        raise AlreadyProcessed(order_id)
    return charge

2: get_or_create(idempotency_key=request_key, ...) relies on idempotency_key having a real unique=True constraint — without it, two concurrent retries of the exact same request could both create a Charge row.
8: .filter(pk=order_id, status="PENDING").update(...) is one atomic database statement — no separate read of status happens first, so there is no window for a race between checking and writing.

Why this works: A network retry of this exact request (same request_key) hits the unique constraint on idempotency_key and is recognized as already-processed at line 2 — the conditional update at line 8 separately protects against a DIFFERENT concurrent request (e.g. a webhook AND a client-side confirmation both trying to mark the same order paid) by making the transition atomic and self-checking.

Using get_or_create() for idempotency without a real unique constraint on the key field

Wrong

python

class Charge(models.Model):
    idempotency_key = models.CharField(max_length=64)   # no unique=True!
# two concurrent retries of the SAME request can both pass get_or_create()'s
# internal get() before either has committed its create() — both create a row

Better

python

class Charge(models.Model):
    idempotency_key = models.CharField(max_length=64, unique=True)

What you see: A customer is double-charged despite the code "using get_or_create() for idempotency" — two Charge rows exist for the exact same idempotency_key, exactly the outcome the pattern was meant to prevent.

Why: Django's own documentation states plainly that get_or_create() is atomic ONLY when the lookup fields have a real database-level uniqueness constraint — without unique=True on idempotency_key, two concurrent calls can both execute the internal get() (finding nothing) before either has committed its create(), and both proceed to create a row. The uniqueness constraint is not optional decoration here; it is the entire mechanism that makes the pattern actually safe.

A retried confirm_payment() call is recognized, not re-applied

Retry request

Charge (unique key)

Order (conditional update)

1. get_or_create(idempotency_key=key)
2. created=False — already exists
3. .filter(status="PENDING").update(...)
4. 0 rows affected — safe no-op

Idempotency key vs conditional update

Idempotency key vs conditional update
Tool	Protects against	Mechanism
Idempotency key	the SAME logical operation being applied twice	a unique constraint on a client-supplied key
Conditional update	a write being applied when its precondition no longer holds	the precondition folded into the UPDATE's WHERE clause

Together

python

# idempotency key: a retried "charge $50" request is recognized, not re-charged
charge, created = Charge.objects.get_or_create(
    idempotency_key=request_key, defaults={"amount": 50, "customer": customer},
)

# conditional update: only transitions if still PENDING — returns 0 if already PAID
rows_updated = Order.objects.filter(pk=order_id, status="PENDING").update(status="PAID")
if rows_updated == 0:
    # already processed (by this request or a concurrent one) — safe to no-op

Remember: An idempotency key (a client-supplied value with a REAL unique constraint) makes a retried request recognizable as a duplicate — get_or_create() is only genuinely atomic with that constraint in place. A conditional update folds the precondition into the UPDATE's WHERE clause as one atomic statement — always check its returned row count before running a side effect that should only happen once.