beginnerintermediateadvanced

Django · State Machines and Workflows

State Machines and Workflows

The edges you did not draw are the ones doing the work

choices constrains values, not movesthe machine is the transition table — write it as data on the model
One writer of the status fieldtransition_to() only, so a grep for `.status =` is a real audit
Lock and read in one statementselect_for_update().get() — a lock over a stale object changes nothing
Remote calls outside the transactionor a provider’s p99 becomes your database’s lock hold time
The state check is your idempotencyafter the move, the same request is a no-op — no key needed inside

1.Valid transitions — and making the invalid ones impossible

The roadmap draws two machines: a payment goes `pending → processing → succeeded`, or `→ failed`; an order goes `draft → placed → processing → fulfilled`, or `→ cancelled`. What makes that a state machine rather than a list of words is the set of transitions that are **not** drawn — `succeeded → pending` is absent, and so is `fulfilled → draft`. A `status` field with `choices` enforces the legal *values* and says nothing about legal *moves*, so the transition table has to be written down and checked somewhere.

Key features

`choices` constrains the value set, not the transitions between values
`choices` is enforced by validation, not by the database — add a `CheckConstraint` for the values
Write the edges as data: a dict of `state → set of reachable states`
One `transition_to()` method; nothing else assigns to the status field
Re-entering the current state is usually a duplicate request — succeed quietly
A genuinely unreachable move is a 409 Conflict: the caller's view of the world is stale

The roadmap's two machines, written as a transition table

The roadmap's two machines, written as a transition table
Object	From	May move to
Payment	`pending`	`processing`
Payment	`processing`	`succeeded`, `failed`
Payment	`succeeded`	— (terminal, except a modelled refund)
Payment	`failed`	— (terminal; a retry is a new payment)
Order	`draft`	`placed`, `cancelled`
Order	`placed`	`processing`, `cancelled`
Order	`processing`	`fulfilled`, `cancelled`
Order	`fulfilled`	— (terminal)

Where each rule can actually be enforced

Where each rule can actually be enforced
Rule	Enforced by	Survives `QuerySet.update()`?
the value is one of the choices	`choices` + forms/`full_clean()`	no
the value is one of the choices	`CheckConstraint`	yes — the database rejects it
the move is legal	`transition_to()` in application code	no — so nothing else may assign
no two workers move it at once	`select_for_update()`	n/a — see the next concept
a terminal state stays terminal	an empty set in the table	no

The order machine — and the moves that are absent on purpose

customerconfirmsabandonedpaymentsucceededpayment failed,or cancelledshippedcancelledbefore dispatch

draft

start

placed

processing

fulfilled

end

cancelled

end

Command

python

order.transition_to(Status.PLACED, actor=request.user)   # the only way status changes

A `status` field with `choices` constrains values, never moves — the machine is the set of *edges*, and the edges that are absent (`fulfilled → draft`) are the ones doing the work. Write the table as data on the model, add a `CheckConstraint` so the database enforces the value set that `choices` only validates in forms, and make `transition_to()` the single place the field is ever assigned so a grep is a real audit. Re-entering the current state is a duplicate request and should succeed quietly; a genuinely unreachable move is a 409.

2.Two workers, one order: locking and the transaction boundary

Checking a state and then changing it is two statements, and anything can happen between them. Two workers can both read `placed`, both decide the move to `processing` is legal, and both perform it — shipping twice, or charging twice. `select_for_update()` closes the gap by locking the row until the transaction ends, so the second worker waits and then re-reads the *new* state. The matching decision is the transaction boundary: what belongs inside it, and what must be deferred until after it commits.

Key features

`select_for_update()` must be inside `atomic()` — outside it raises `TransactionManagementError`
The lock is only useful if the state is **re-read and re-checked** after acquiring it
Keep the transaction short: lock, check, write, commit — no HTTP calls inside
Side effects that cannot be rolled back go in `transaction.on_commit()`
`select_for_update(skip_locked=True)` for worker pools: take a different row instead of waiting
An optimistic alternative: `filter(pk=…, status="placed").update(status="processing")` — check the row count

Pessimistic or optimistic — both work, differently

Pessimistic or optimistic — both work, differently
Approach	How it prevents the race	Best when
`select_for_update()`	the second reader waits for the lock	the transition does real work while holding it
conditional `update()`	the `WHERE` clause fails for the loser	the transition is a single field change
`skip_locked=True`	the second worker takes another row	a worker pool draining a queue table
`nowait=True`	the second worker fails immediately	you would rather error than queue
a database version column	the update matches on the old version	long user-facing edits ("someone else changed this")

Inside the transaction, or after it commits

Inside the transaction, or after it commits
Action	Where	Why
the status change	inside	it is the change
the audit row / domain event	inside	a change with no record is the gap to avoid
stock decrement, balance update	inside	must be consistent with the state
sending an email	`on_commit`	cannot be un-sent
enqueuing a task	`on_commit`	a worker can outrun the commit, or survive a rollback
calling a payment provider	before the transaction	never hold a row lock for a remote p99

Two workers take the same order — with the lock, and what the loser does

Worker A

Worker B

Database

1BEGIN; SELECT … FOR UPDATE (order 88)the row lock is taken here
2status = "placed"
3BEGIN; SELECT … FOR UPDATE (order 88)blocks — A holds the lock
4check: placed → processing is legal
5UPDATE status = "processing"; INSERT event
6COMMITlock released; on_commit hooks now fire
7status = "processing"B re-reads and sees the NEW value — this is the point of the lock
8check: processing → processing is a no-opreturns quietly; the order ships once

Command

python

with transaction.atomic():
    order = Order.objects.select_for_update().get(pk=pk)   # lock, then re-check

Read-check-write is a race. `select_for_update()` closes it, but only if the *same statement* both locks and reads — a lock over an object you loaded earlier checks a stale value and changes nothing. It must be inside `atomic()`; Django raises otherwise. Keep the transaction short: do slow and remote work before it, hold the lock only for the check and the writes. Everything that must be consistent with the state change goes inside the boundary; everything irreversible — email, queue messages, provider calls — goes in `on_commit` or before it. And `skip_locked=True` is the right tool when a pool of workers is draining rows.

3.Idempotency, events, and the history the transition leaves behind

Three things hang off a transition. **Idempotency**: the same request arriving twice must produce one transition, which the state check itself gives you for free — the second attempt finds the object already moved and does nothing. **Event generation**: a transition is the natural place to emit "order.fulfilled" for other parts of the system, and it must be emitted exactly as often as the transition happens. **Audit history**: a row per transition, so the object's timeline is queryable rather than inferred from a single current-status field.

Key features

The state check is the idempotency mechanism — after the move, the same request is a no-op
Emit events inside `transition_to`, so a second code path cannot move the object silently
Publish in `on_commit`: an event emitted before the commit may describe a rollback
If subscribers must never miss one, write the event as a row and relay it (outbox)
Store `from_state` *and* `to_state` on the history row — it makes the row self-contained
The `status` field becomes a cache of the latest transition, not the only record

What each of the three gives you, and where it goes

What each of the three gives you, and where it goes
Concern	Mechanism	Where
idempotency inside the workflow	the transition check itself	`transition_to()`
idempotency at the edge	a client-supplied key	the view / webhook handler
events for other systems	emit on successful transition	`on_commit`, from `transition_to()`
events that must not be lost	an outbox row + a relay	inside the transaction
domain history	a `StatusChange` row	inside the transaction
compliance audit	an `AuditEntry` row	inside the transaction

One order, read from its history table rather than its status field

Mon 09:12
draft → placed
actor: the customer · reason: checkout completed
Mon 09:12
event: order.placed
emitted on_commit — after the row is durable
Mon 09:14
placed → processing
actor: system · reason: payment succeeded
Mon 09:14
placed → processing (rejected)
a retried task: already processing, so a quiet no-op
Tue 16:40
processing → fulfilled
actor: warehouse · reason: dispatched, tracking AB123
Tue 16:40
event: order.fulfilled
one event, because there was one transition
Fri 11:02
fulfilled → cancelled (rejected)
not an edge in the table — 409, and the caller is out of date

Command

python

transaction.on_commit(lambda: publish("order.fulfilled", order.id))

The transition check *is* your idempotency mechanism inside the workflow — after the move, the same request is a no-op, so keys are only needed at the edges. Emit events from inside `transition_to` and inside `on_commit`: emitting elsewhere means a second code path moves the object silently, and emitting before the commit means announcing a change that may roll back. Where an event must never be lost, write it as an outbox row in the same transaction and let a relay publish it. Store `from_state` and `to_state` on the history row so it stands alone — and treat `status` as a cache of the last transition rather than the record itself.

Spotted a mistake on this page? Tell us.

Read Sheet