Idempotency keys, and HTTP versus application-level idempotency
coreadvancedHTTP already says `PUT` and `DELETE` are idempotent — send them twice and the server ends up in the same state. `POST` is not, which is a problem exactly where it matters most: creating a payment, an order, or a call to an external provider. The fix is application-level idempotency. The client generates a unique key per logical operation and sends it as an `Idempotency-Key` header. The server records the key before doing the work; if the same key arrives again, it returns the stored result instead of doing the work a second time. The distinction the roadmap asks for is exactly this: HTTP-level idempotency is a property of the *method* and is free; application-level idempotency is a property of *your handler* and must be built.
Think of it as
The situation this exists for is not a client bug — it is the fundamental uncertainty of a network. A client sends `POST /payments/`, the request succeeds, and the response is lost on the way back. The client now cannot tell "it worked and I did not hear" from "it never arrived", and both retrying and not retrying are wrong: one double-charges, the other loses the payment. The idempotency key resolves it by making the *operation* identifiable rather than the request. Two properties then matter. The key must be generated by the client, before the first attempt, and reused across retries — a server-generated key cannot help, because the client that never got a response has nothing to reuse. And the record must be written in the same transaction as the work, or the two can disagree: the payment commits and the key write fails, so the retry charges again. That is why "check, then do, then record" is the wrong order and a unique constraint on the key is the right mechanism — you let the database refuse the duplicate rather than asking it whether one exists.
What we're doing: Make payment creation safe to retry, letting the database refuse duplicates rather than checking for them.
- 4–5
- Requiring the header rather than defaulting to "no key" — an unkeyed payment request is a client bug, and failing loudly is better than silently accepting one that cannot be retried safely.
- 10–16
- One transaction covers the key row, the charge, and the stored response. Either all three commit or none do, which is what stops the key and the charge from disagreeing.
- 17–19
- The `IntegrityError` from the unique constraint *is* the duplicate check. Reading first and inserting after leaves a window where two concurrent retries both find nothing.
- 20–21
- Same key, different body is a client bug, not a replay — returning the first result would silently discard the second request.
- 22–23
- A key row with no stored status means the original attempt is still running. Answering 409 tells the client to wait rather than issuing a second charge.
Why this works: Letting the unique constraint decide removes the read-then-write race entirely: two simultaneous retries race to `INSERT`, exactly one wins, and the loser is routed to the replay path by the database rather than by a check that could have been stale.
Checking for the key before doing the work, in a separate step
Wrong
Better
What you see: Duplicate charges appear only under retry storms — a mobile client on a flaky connection firing two retries within milliseconds — so the bug is unreproducible locally and shows up in a customer complaint about being billed twice.
Why: Between the `exists()` and the `create()` there is a window in which another request can pass the same check. This is the classic check-then-act race, and no amount of ordering fixes it in application code; the guarantee has to come from the database. A unique constraint makes the second `INSERT` fail, which is a deterministic signal rather than a probabilistic one.
- Client → API: POST /payments/ · Idempotency-Key: 8f14e45f
- API → IdempotencyRecord: INSERT key (unique constraint) (inside the same transaction as the work)
- API → Payment provider: charge £40.00
- Payment provider → API: charged · txn_9931
- API → IdempotencyRecord: store 201 + response body, commit
- API → Client: 201 Created (the response is lost — connection reset)
- Client → API: retry: same body, SAME key 8f14e45f
- API → IdempotencyRecord: INSERT fails on the unique constraint (the database refuses the duplicate — no read-then-check race)
- IdempotencyRecord → API: the stored 201 and body
- API → Client: 201 Created — identical response, one charge
Two kinds of idempotency, often confused
Together
Remember: HTTP gives you idempotency for `PUT` and `DELETE` for free; `POST` needs you to build it. The client generates a key per logical operation and reuses it on every retry; the server records the key with a unique constraint, in the same transaction as the work, and returns the stored response on a replay. Let the constraint refuse duplicates rather than checking first — `exists()` then `create()` is a race. Same key with a different body is a client bug (409/422), and keys need a retention window and a purge.
See also: at least once delivery and consumer idempotency · http semantics and status codes · idempotency and conditional updates · atomic and nested blocks

