beginnerintermediateadvanced

Django · Contract and Integration Testing

Contract and Integration Testing

Stubs for every branch, one sandbox test for the shape, and the three properties a webhook actually has

Stub the HTTP boundary, not your own clientpatching your wrapper deletes the URL, headers and timeout from coverage
Sandbox tests invalidate stubs — they do not cover brancheskeep them few, tagged, and scheduled rather than on every commit
requests has NO default timeoutone hung provider with a fixed worker pool is your outage
Assert the retry's key, not the retry counta retry with a fresh idempotency key is a second charge with extra steps
Verify webhook signatures over the RAW bodyjson.dumps(json.loads(x)) is different bytes — and the test makes the same mistake

1.Contract tests, integration tests, and the four test doubles

A **contract test** checks that the request you send and the response you can parse match what the provider actually promises — it runs against a recorded or stubbed response and is fast. An **integration test** talks to the provider's sandbox for real, which is slow and can fail for reasons that have nothing to do with your code. You need both, and they answer different questions: the contract test asks "does my code handle this shape correctly", the integration test asks "is this shape still what they send". A **test double** is whatever you put in the provider's place — a stub returns a canned answer, a fake is a working simplified version, a spy records calls, and a mock asserts on them.

Key features

Contract test: your code against a stubbed provider — fast, deterministic, runs on every commit
Integration test: your code against the real sandbox — slow, flaky, and the only thing that can tell you a stub has gone stale
Stub → canned response · Fake → working simplified implementation · Spy → records calls · Mock → asserts on calls
Django ships a fake for email already: the `locmem` backend fills `mail.outbox`, so no email provider needs mocking in tests
`responses` registers per-URL behaviour and exposes `responses.calls`, each entry carrying `.request` and `.response`
`RequestsMock(assert_all_requests_are_fired=True)` fails a test that registered a stub it never used — a dead stub is a lie about coverage

The four doubles, and when each is the right one

The four doubles, and when each is the right one
Double	What it does	Reach for it when
Stub	returns a canned response	you are testing your branches — the default choice
Fake	a real but simplified implementation	state matters: an in-memory bucket where `put` then `get` must agree
Spy	passes through, records the calls	the call is real but you also want to assert its arguments
Mock	asserts calls were (or were not) made	the assertion is "we never charged the card twice"

Two tests, two different failures caught

Contract test — stubbed provider

Runs on every commit, in milliseconds
You choose the response, including the 402 and the timeout
Catches: your code mishandling a shape
Cannot catch: the provider changing that shape
Deterministic — no network, no rate limits, no card expiry

Integration test — real sandbox

Runs nightly, tagged, allowed to fail loudly
The provider chooses the response
Catches: a renamed field, a changed status code, a new required header
Cannot replace contract tests — you cannot make it produce every branch
Its job is to invalidate your stubs, not to cover your logic

Command

python

@responses.activate
def test_x():
    responses.post(url, json={...}, status=200)   # stub
    ...
    assert len(responses.calls) == 1              # spy

Contract tests answer "does my code handle this shape", integration tests answer "is this still the shape" — you need both, and the second one exists to invalidate the first one's stubs, not to cover your branches. Keep sandbox tests few, tagged and scheduled; keep stubbed tests exhaustive, because only a stub gives you a 500 or a timeout on demand. Put the double at the HTTP boundary rather than on your own client class, or the URL, headers and timeout you actually ship go untested.

2.Testing retry behaviour, timeouts, and idempotency

These three are one test area because they are one bug. A request times out, your code retries, and the provider charges the card twice — because the first call had in fact succeeded and only the *response* was lost. So test all three together: make the first attempt fail, assert the code retried, and assert the second attempt carried the same idempotency key. A stub library with an ordered registry gives you exactly that — register a timeout followed by a success, and the two attempts are two entries you can inspect.

Key features

`requests` applies **no timeout by default** — a hung provider holds the worker until something else kills it
Pass `timeout=(connect, read)` as a tuple: a short connect budget and a longer read budget are different decisions
A read timeout means the work may already be done — retrying without an idempotency key is how double charges happen
Register an exception instance as the stub body (`body=ConnectTimeout()`) to make a call raise instead of respond
`responses.activate(registry=OrderedRegistry)` plays registered responses in order, so attempt 1 can fail and attempt 2 succeed
Test the exhausted path too: after the final attempt, assert the state the system is left in

Four failures, and what each one licenses you to do

Four failures, and what each one licenses you to do
Failure	Did the work happen?	Safe to retry?
connect timeout	almost certainly not	yes — with the same key anyway
read timeout	unknown — possibly yes	only with an idempotency key
`500` / `503`	unknown	yes, with backoff and the same key
`409 already_charged`	yes, on a previous attempt	no — treat it as success
`402 card_declined`	yes, and it failed for good	no — retrying re-declines forever

One outbound call, every path it can take

200402 — donot retryno responsebudgetremainsattempt n+1409 onthe retrythe firstattempt DID workbudget spent

attempt n

start

success → mark paid

end

timeout / 5xx

wait, then retry with the SAME key

409 already_charged

402 declined → release stock

end

attempts exhausted → manual review

end

Command

python

@responses.activate(registry=OrderedRegistry)
def test_retry():
    responses.post(URL, body=ConnectTimeout())      # attempt 1
    responses.post(URL, json={"id": "ch_1"})        # attempt 2

A read timeout means the work may already be done, so the retry must carry the same idempotency key — and the test must assert that the two attempts' keys are equal, not merely that two attempts happened. Treat `409 already_charged` on a retry as success, never retry a permanent `402`, and always test the exhausted path, because it runs during the incident where being wrong costs most. Set `timeout=(connect, read)` on every outbound call: `requests` has no default, and one hung provider with a fixed worker pool is your outage.

3.Webhook testing, and the five integration surfaces

A webhook is the provider calling *you*, so the tests are the reverse of every other integration test: you build the request. Four of them matter — a valid signed event that does the work, a forged signature that is rejected, the same event delivered twice that must not do the work twice, and events arriving out of order. Beyond payments, a Django service usually talks to five kinds of thing: payment providers, email providers, storage APIs, other REST APIs, and message brokers. Each has a standard double, and for two of them Django or the library already ships one — email has the `locmem` backend, and storage has an in-memory or temporary-directory backend.

Key features

Verify the signature over the **raw request body** — parsing to JSON and re-serializing changes bytes and breaks the check
Delivery is at-least-once: the same event id will arrive again, so the handler must be idempotent and the test must post it twice
Order is not guaranteed — compare the event's own timestamp against what you have stored, and ignore the older one
Return 2xx fast and do the work in a task: a slow handler makes the provider retry, which multiplies the load you were already struggling with
Django's `locmem` email backend fills `mail.outbox` — a fake that ships with the framework, so email needs no mocking
Never log the raw webhook body at INFO: it routinely contains card metadata, addresses and tokens

The five surfaces and the double each one wants

The five surfaces and the double each one wants
Surface	Standard double	The test people skip
Payment providers	`responses` stub + one sandbox test	the retry that must reuse the idempotency key
Email providers	Django `locmem` → `mail.outbox`	the send that must NOT happen on rollback
Storage APIs	temp-directory or in-memory storage backend	the upload that fails halfway and leaves no orphan row
External REST APIs	`responses` at the HTTP boundary	timeout and 5xx branches, not only 200
Message brokers	eager execution, or an in-memory broker	the duplicate delivery — consumers get at-least-once too

One payment, three inbound deliveries — and the two your handler must survive

provider

webhook view

task queue

database

1POST /webhooks/payments (event evt_1)raw body + signature header
2verify signature over the RAW bodyreject before parsing anything
3INSERT WebhookEvent(evt_1)unique on event id — the dedupe point
4enqueue handle_payment(evt_1)
5200 — fast, before the work
6POST evt_1 again (retry)at-least-once: identical bytes
7INSERT evt_1 → IntegrityErroralready seen — swallow and 200
8200 — no second charge applied
9POST evt_0 (older, arrives late)ordering is not guaranteed
10occurred_at < stored → ignorethe test that delivers the pair backwards

Command

python

response = client.post(
    "/webhooks/payments/", data=raw_body, content_type="application/json",
    HTTP_X_SIGNATURE=sign(raw_body, settings.WEBHOOK_SECRET),
)

Sign and verify the **raw body** — re-serialized JSON is different bytes, and a test that makes the same mistake agrees with itself while production rejects everything. Write the forged-signature test before the happy path, deliver the same event twice and assert the side effect happened once, and deliver a related pair out of order to prove your handler uses the event timestamp rather than arrival order. Acknowledge fast and process in a task, or the provider's retries multiply the load. Across the other four surfaces, look for the double that already exists before building one: `locmem` for email, a temp-directory backend for storage, `responses` for REST, eager execution for queues.

Spotted a mistake on this page? Tell us.

Read Sheet