beginnerintermediateadvanced

Django · Section 29

Transactions

Level: advanced
Read: 26 min
Concepts: 3

transaction.atomic() as decorator or context manager, turning Django's default autocommit behavior into one real all-or-nothing transaction — and nested atomic() blocks, which create SAVEPOINTs rather than separate transactions, so an inner block's rollback stays scoped to it while the outermost block remains the true commit boundary. The broken-transaction trap: catching a database error INSIDE an atomic() block and continuing to query in that same block raises TransactionManagementError, since the database itself considers the transaction unusable from the moment of the error — the fix is catching from outside the block, and retrying only for genuinely retriable failures (serialization conflicts, deadlocks), never a persistent data conflict like IntegrityError. transaction.on_commit() defers a side effect (email, webhook, background task) until the transaction is genuinely, successfully committed, and durable=True forces a block to be verified as the outermost transaction.

This section

Django Database Transactions / atomic() functionBugBytes

What is true here

Django defaults to autocommit — atomic() (decorator or context manager) groups a set of statements into one real, all-or-nothing transaction.
A nested atomic() block creates a savepoint, not a separate transaction — its own rollback stays scoped to it, but the outermost block is the true commit boundary, and its rollback undoes everything, inner blocks included.
Catching a database error inside an atomic() block and continuing to query in that same block raises TransactionManagementError — the database considers the transaction broken from the moment of the error; catch from outside the block instead.
Retry loops make sense for serialization failures and deadlocks (a timing conflict with another transaction) but not for a genuine IntegrityError (a persistent data conflict that will fail identically on retry).
transaction.on_commit(callback) defers a side effect until the transaction genuinely, successfully commits — never call an email/webhook/background task directly inside atomic(), since the transaction might still roll back.

What you will be able to do

Use atomic() correctly as both a decorator and a context manager, and reason about nested savepoints
Avoid the broken-transaction trap by catching database errors from outside an atomic() block
Write a correct retry loop for genuinely retriable transaction failures
Defer external side effects with on_commit(), and use durable=True only where genuinely warranted

transaction.atomic() and nested blocks

Autocommit, the decorator/context-manager forms, and how nested blocks behave as savepoints.

transaction.atomic() and nested blocks

coreintermediate

Django runs in autocommit mode by default — every query commits immediately on its own. transaction.atomic() (as a decorator or with block) groups everything inside into one real transaction: all-or-nothing, committed together on success, rolled back together on any exception. A NESTED atomic() block creates a SAVEPOINT, not a separate transaction — if the inner block raises, only ITS changes roll back (to the savepoint); the outer block continues and can still commit its own, earlier work.

Think of it as

Picture atomic() as a boundary you draw around a group of writes that must succeed or fail as one unit — a balance transfer's two UPDATEs are the textbook case: either both happen or neither does. Nesting atomic() blocks doesn't create a second, independent transaction — SQL doesn't really have "transactions inside transactions." Instead, Django uses SAVEPOINTs: a marker inside the one real transaction that lets JUST the code since that marker be undone, while everything before it stays intact and can still commit normally when the OUTER block finishes. This is what makes "try this optional step, but keep going even if it fails" possible — wrap the optional step in its own inner atomic(), catch the exception outside it, and the outer transaction's earlier work survives.

python

@transaction.atomic
def transfer(from_id, to_id, amount):
    ...

# or, scoped to part of a function:
def view(request):
    do_stuff()             # autocommit — not part of any transaction
    with transaction.atomic():
        do_more_stuff()     # atomic

What we're doing: Transfer funds between two accounts atomically, and separately attempt an optional bonus-points award that should not undo the transfer if it fails.

accounts/services.pypython

def transfer_with_bonus(from_id, to_id, amount):
    with transaction.atomic():
        Account.objects.filter(id=from_id).update(balance=F("balance") - amount)
        Account.objects.filter(id=to_id).update(balance=F("balance") + amount)

        try:
            with transaction.atomic():
                award_loyalty_points(to_id, amount)   # might fail — a separate, optional step
        except LoyaltyServiceError:
            log.warning("Loyalty points award failed, transfer still proceeds")

2: The outer atomic() is the real transaction boundary — the two balance updates commit or roll back together.
8: The inner atomic() around award_loyalty_points() creates a savepoint — if that call raises, only ITS work rolls back, not the two balance updates that already ran inside the outer block.

Why this works: Without the inner atomic() block, an exception from award_loyalty_points() would propagate up and roll back the ENTIRE outer transaction — undoing a successful funds transfer just because an unrelated, genuinely optional bonus-points step happened to fail. The nested block is what lets the two concerns fail independently.

Assuming an inner atomic() block's success means it has actually committed to the database

Wrong

python

with transaction.atomic():
    with transaction.atomic():
        create_order()   # "succeeded" — but only within the savepoint
    raise SomeUnrelatedError()   # rolls back EVERYTHING, including create_order()

Better

python

# either don't raise past the point where create_order() must survive,
# or move create_order() outside any block that might still fail:
with transaction.atomic():
    create_order()
    do_the_risky_thing_after()   # if this fails, create_order() still rolls back too — by design

What you see: Code assumes that because an inner atomic() block exited without raising, its changes are permanently saved — then a LATER exception in the same outer transaction undoes that "already succeeded" work too, surprising anyone who thought the inner block's exit was the actual commit point.

Why: Only the OUTERMOST atomic() block is a true commit boundary — an inner block finishing without error just means its savepoint was released, not that the data is durably committed. The database only actually commits (making the change permanent) when the outermost atomic() block exits successfully; anything that raises afterward, even in unrelated code, rolls back the whole transaction, inner blocks included.

A nested atomic() is a savepoint, not a separate transaction

Outer atomic()

the true commit boundary — its rollback undoes everything, inner blocks included

Inner atomic() (savepoint)

its own rollback stays scoped to just its own changes

Autocommit (no atomic())

default — every query commits immediately on its own

What rolls back, at which nesting level

What rolls back, at which nesting level
Scenario	What rolls back
Exception inside an inner atomic() block, caught there or just outside it	only the inner block's changes (to its savepoint)
Exception inside the outer atomic() block, uncaught	everything — including any inner blocks that already committed to their savepoint
No exception anywhere	nothing rolls back — the outer block commits everything on exit

Together

python

with transaction.atomic():          # outer — the real transaction
    create_parent()                  # survives even if the inner block below fails

    try:
        with transaction.atomic():   # inner — a savepoint
            generate_relationships()
    except IntegrityError:
        handle_it()                  # only the inner block's work was rolled back

    add_children()                   # runs regardless, still inside the outer transaction

Remember: Django defaults to autocommit — atomic() (decorator or context manager) is what groups statements into one real transaction. A NESTED atomic() block is a savepoint, not a separate transaction: its own rollback stays scoped to it, but the OUTERMOST block is the true commit boundary, and its rollback undoes everything, inner blocks included.

The broken-transaction trap, and retrying

Why catching an error inside atomic() makes things worse, and which failures are actually worth retrying.

The broken-transaction trap, and retrying on conflict

coreadvanced

Catching a database error (IntegrityError, an isolation-level serialization failure, a deadlock) INSIDE an atomic() block does NOT recover the transaction — the database considers it broken from that point on, and Django raises TransactionManagementError on any further query attempted inside that same block. The fix is to catch the exception OUTSIDE the atomic() block, letting the whole block roll back cleanly, then decide whether to retry from scratch.

Think of it as

Once a statement inside a transaction fails at the database level, the DATABASE itself refuses every subsequent statement in that same transaction until it's rolled back — this isn't a Django limitation, it's how PostgreSQL's own transaction machinery works, and Django's TransactionManagementError is just Django surfacing that reality early and loudly rather than letting a confusing raw database error happen later. The only way forward after a genuine database-level failure (a constraint violation, a serialization conflict, a deadlock) is to let the WHOLE atomic() block exit via the exception — rolling everything in it back — and then, if the failure is the retriable kind (serialization/deadlock, not a real data problem), start a brand NEW atomic() block from scratch.

python

for attempt in range(3):
    try:
        with transaction.atomic():
            do_the_work()
        break
    except OperationalError as e:
        if "could not serialize" not in str(e) and "deadlock detected" not in str(e):
            raise
        if attempt == 2:
            raise

What we're doing: Retry a transaction that fails due to a serialization conflict (Serializable isolation) or a deadlock, but not one that fails for an unrelated, non-retriable reason.

accounts/services.pypython

def transfer_with_retry(from_id, to_id, amount, max_attempts=3):
    for attempt in range(max_attempts):
        try:
            with transaction.atomic():
                Account.objects.filter(id=from_id).update(balance=F("balance") - amount)
                Account.objects.filter(id=to_id).update(balance=F("balance") + amount)
            return
        except OperationalError as e:
            retriable = "could not serialize" in str(e) or "deadlock detected" in str(e)
            if not retriable or attempt == max_attempts - 1:
                raise

1: The retry loop wraps the ENTIRE atomic() block, from outside it — each attempt starts a genuinely fresh transaction, not a continuation of a broken one.
8: Checking the error message for the specific retriable failure types avoids blindly retrying every database error — a real IntegrityError (e.g. a genuine constraint violation) would just fail identically on retry and should propagate immediately instead.

Why this works: Serialization failures and deadlocks are, by PostgreSQL's own design, an EXPECTED outcome under real concurrent load at stricter isolation levels — the correct response is exactly this: retry the whole transaction from scratch a bounded number of times, since the conflict was with another transaction's timing, not a real, persistent problem with this one's data.

Catching IntegrityError inside atomic() and trying to "fix and continue" in the same block

Wrong

python

with transaction.atomic():
    try:
        Order.objects.create(reference="DUP-123")
    except IntegrityError:
        pass   # "handled" the duplicate — but the transaction itself is now broken
    Order.objects.create(reference="DUP-124")   # raises TransactionManagementError, not IntegrityError

Better

python

try:
    with transaction.atomic():
        Order.objects.create(reference="DUP-123")
except IntegrityError:
    pass
Order.objects.create(reference="DUP-124")   # runs fine, fresh transaction

What you see: A confusing TransactionManagementError appears on a completely unrelated, later query — one that has nothing to do with the actual duplicate that caused the original IntegrityError — making the real root cause much harder to trace back.

Why: The moment IntegrityError was raised, the DATABASE marked the current transaction as unusable — catching the exception in Python does not un-break the transaction at the database level. Every query attempted afterward, inside that same atomic() block, hits Django's own check for this state and raises TransactionManagementError instead of a normal database error, which is Django surfacing the real problem (a broken transaction) rather than letting a stranger failure happen deeper in the database driver.

A transaction cannot recover from inside itself

Transaction open

start

DB error occurs

Broken — unusable

Rolled back cleanly

end

Fresh transaction, retried

end

Where to catch a transaction-breaking exception

Where to catch a transaction-breaking exception
Location	Result
Inside the atomic() block, then more queries in the SAME block	TransactionManagementError on the next query — worse, not better
Outside the atomic() block (wrapping the `with` statement itself)	the block rolls back cleanly; code after the except runs in a fresh state

Together

python

# WRONG — catching inside, then continuing in the same block
with transaction.atomic():
    try:
        b.save()
    except IntegrityError:
        pass          # transaction is now broken
    c.save()          # raises TransactionManagementError

# RIGHT — catching outside
try:
    with transaction.atomic():
        b.save()
except IntegrityError:
    handle_it()
c.save()   # fine — this runs in a fresh, unbroken state

Remember: Catching a database error INSIDE an atomic() block and continuing to query in that same block raises TransactionManagementError — the transaction is broken from the moment the error occurred. Catch from OUTSIDE the block instead, letting it roll back cleanly. Retry loops make sense for serialization failures/deadlocks (a timing conflict) but not for a genuine IntegrityError (a real, persistent data conflict).

on_commit(), durable=True, and ATOMIC_REQUESTS

Deferring side effects until a commit is certain, and forcing a block to be the outermost transaction.

on_commit(), durable=True, and ATOMIC_REQUESTS

standardadvanced

transaction.on_commit(callback) registers a function that runs ONLY after the transaction actually, successfully commits — never if it rolls back, and never as part of the transaction itself. This is the correct way to send an email/trigger a background task tied to a database write, since the write inside a still-open transaction isn't durable yet. durable=True forces an atomic() block to be the OUTERMOST one — a safety check that raises immediately if it's accidentally nested inside another. ATOMIC_REQUESTS (a per-database setting) wraps every view in a transaction automatically.

Think of it as

A common bug: send a confirmation email INSIDE the same atomic() block that creates the order, right after order.save(). If the transaction later rolls back for an unrelated reason (another statement later in that same block fails), the email has ALREADY been sent for an order that no longer exists in the database — on_commit() exists specifically to close that gap, by deferring the callback until Django knows FOR CERTAIN the transaction actually committed. durable=True is a much narrower safety net: some operations (rare, deliberately critical ones) must be guaranteed to actually be the outermost transaction, not silently become a savepoint because some caller wrapped them in an outer atomic() block — durable=True turns that mistake into an immediate, loud error instead of a silent, wrong nesting.

python

transaction.on_commit(lambda: send_email(order.id))
@transaction.atomic(durable=True)
def critical_operation(): ...

What we're doing: Defer sending a confirmation email until the order-creation transaction is genuinely committed, so a later failure in the same transaction can't leave an email sent for data that was rolled back.

orders/services.pypython

def place_order(customer, cart):
    with transaction.atomic():
        order = Order.objects.create(customer=customer, total=cart.total)
        transaction.on_commit(lambda: send_confirmation_email(order.id))
        reserve_inventory(order, cart)   # might still raise and roll everything back

4: The email send is deferred, not executed immediately — if reserve_inventory() below raises, the whole atomic() block rolls back, and this on_commit() callback is simply discarded, never running at all.

Why this works: A direct send_confirmation_email(order.id) call right after Order.objects.create() would run immediately, well before the transaction is known to succeed — if reserve_inventory() fails afterward and rolls everything back, the customer would have already received an email confirming an order that, from the database's perspective, never actually happened.

Calling an external side effect (email, webhook, background task) directly inside atomic(), instead of via on_commit()

Wrong

python

with transaction.atomic():
    order = Order.objects.create(customer=customer, total=total)
    send_confirmation_email(order.id)   # runs immediately, transaction might still roll back
    charge_payment_method(customer, total)   # if THIS fails, the email was already sent

Better

python

with transaction.atomic():
    order = Order.objects.create(customer=customer, total=total)
    transaction.on_commit(lambda: send_confirmation_email(order.id))
    charge_payment_method(customer, total)

What you see: A customer receives a confirmation email (or a webhook fires, or a background job starts) for an order that the database actually rolled back and never persisted, because a LATER step in the same transaction failed.

Why: Any code that runs directly inside atomic() executes immediately, with no guarantee the surrounding transaction will actually commit — on_commit() is specifically the mechanism for deferring an external, non-database side effect until Django can guarantee the transaction succeeded, which is the only point at which "the order really exists" becomes true.

on_commit() vs a plain call inside the transaction

on_commit() vs a plain call inside the transaction
Approach	Risk
send_email() called directly inside atomic()	runs even if a LATER statement in the same block causes a rollback — sends for data that was never actually saved
transaction.on_commit(send_email) inside atomic()	deferred until the commit is certain — never runs if the transaction rolls back

Together

python

with transaction.atomic():
    order = Order.objects.create(customer=customer, total=total)
    transaction.on_commit(lambda: send_confirmation_email(order.id))
    # if anything below this line raises, the order AND the email are both cancelled
    reserve_inventory(order)

Remember: on_commit(callback) defers a side effect (email, webhook, background task) until the transaction is genuinely, successfully committed — never call such side effects directly inside atomic(), since the transaction might still roll back. durable=True forces a block to be the outermost transaction, raising immediately if nested — reserve it for genuinely critical, must-not-be-nested operations, not ordinary reusable functions.