beginnerintermediateadvanced

Django · Section 70

Contract and Integration Testing

Level: advanced
Read: 34 min
Concepts: 3

Every external dependency gives you two independent things to get wrong, and one kind of test cannot cover both. The first is your handling: given a 402 with this body, does your code release the reservation and return the right error? That is entirely your logic, and running it against a live sandbox makes it slower and flakier without making it stronger — a stub is better, because you can produce the 402 on demand instead of hoping the sandbox does. The second is the assumption underneath: is the field still called `charge_id`, is the error still a 402? No stub can tell you, because your stub is a copy of your own belief. That is what the sandbox test is for, and why it should be small, tagged and scheduled — its job is to invalidate your stubs, not to cover your branches. Put the double at the HTTP boundary rather than on your own client class, or the URL, auth header, timeout and JSON parsing you actually ship go untested. Retries, timeouts and idempotency then turn out to be one subject rather than three, because they are one bug: a request times out, the code retries, and the provider charges twice — since the first call had in fact succeeded and only the response was lost. A read timeout tells you nothing about whether the work happened, so the retry must carry the same idempotency key, and the test has to assert that the two attempts' keys are equal rather than that two attempts occurred. The branch after the last attempt matters too: it runs during the incident where being wrong is most expensive. Webhooks reverse the direction and bring three properties people discover late. The endpoint is public and forgeable, so the signature is checked over the raw bytes before anything is parsed. Delivery is at-least-once, so the same event id will arrive again and the side effect must happen once. And order is not guaranteed, so a refund and a success can arrive backwards — which a handler trusting arrival order will happily get wrong. Across the five surfaces the roadmap names, look for the double that already ships before building one.

Django overview

Python Django Full Course for Beginners | Complete All-in-One Tutorial | 3 HoursDave Gray

What is true here

Contract tests cover branches; sandbox tests catch shape drift. Keep the second few, tagged and scheduled.
Stub at the HTTP boundary — patching your own client removes the URL, headers and timeout from coverage.
Assert that a retry reuses the idempotency key; the retry count alone proves nothing about safety.
Set timeout=(connect, read) on every outbound call — requests has no default.
Webhooks need four tests: forged signature, valid event, duplicate delivery, out-of-order pair.

What you will be able to do

Split an integration into the branches a stub should cover and the one assumption a sandbox should check
Write a retry test that would actually catch a double charge
Decide what a timeout licenses your code to do, and test the exhausted path
Test a webhook endpoint as the public, redelivered, unordered surface it really is

Four layers of integration testing, cheapest and most frequent at the top

Unit tests with a stubbed HTTP boundary

every branch — 200, 402, 409, 500, timeout. Milliseconds, every commit.

Handler tests through your own view

webhook signature, dedupe, ordering — you build the request

Wired integration tests, no provider

view + ORM + task queue together, eager execution, real database

Sandbox tests against the real provider

a handful, tagged, scheduled — they exist to invalidate the stubs above

What each kind of test proves

Contract against integration, the four doubles, and why a sandbox suite fails on days you changed nothing.

Contract tests, integration tests, and the four test doubles

coreadvanced

A **contract test** checks that the request you send and the response you can parse match what the provider actually promises — it runs against a recorded or stubbed response and is fast. An **integration test** talks to the provider's sandbox for real, which is slow and can fail for reasons that have nothing to do with your code. You need both, and they answer different questions: the contract test asks "does my code handle this shape correctly", the integration test asks "is this shape still what they send". A **test double** is whatever you put in the provider's place — a stub returns a canned answer, a fake is a working simplified version, a spy records calls, and a mock asserts on them.

Think of it as

Every external dependency gives you two independent things to get wrong, and one test kind cannot cover both. The first is your handling: given a 402 with this body, does your code refund the reservation and return the right error to the caller? That is entirely about your logic, and running it against a live sandbox makes it slower and flakier without making it stronger — a stub is strictly better, because you can produce the 402 on demand instead of hoping the sandbox produces one. The second is the assumption underneath: is the field still called `charge_id`, does the error still come back as 402, is the signature header still `Stripe-Signature`? No amount of stubbing can tell you this, because your stub is a copy of your own belief. That is why the sandbox test exists, and why it should be small, tagged, and run on a schedule rather than on every commit — its purpose is to *invalidate your stubs*, not to test your branches. Once you see the split that way, the doubles fall into place. Most of the time you want a stub. Reach for a fake when the interaction has state worth modelling — an in-memory storage backend where a `put` is visible to a later `get` catches ordering bugs a stub never will. Use a spy when you care that a call was made with the right arguments, and keep true mocks for the few places where the *absence* of a call is the assertion.

python

@responses.activate
def test_x():
    responses.post(url, json={...}, status=200)   # stub
    ...
    assert len(responses.calls) == 1              # spy

What we're doing: Cover both directions for one payment call: every branch against a stub, and one narrow assumption check against the sandbox.

billing/tests/test_charge_contract.pypython

CHARGE_URL = "https://api.payments.test/v1/charges"


@pytest.fixture
def stubbed_provider():
    with responses.RequestsMock(assert_all_requests_are_fired=True) as mock:
        yield mock


@pytest.mark.django_db
@pytest.mark.parametrize("status,body,expected", [
    (200, {"id": "ch_1", "status": "succeeded"}, "paid"),
    (402, {"error": "card_declined"}, "declined"),
    (409, {"error": "already_charged", "id": "ch_1"}, "paid"),
    (500, {"error": "internal"}, "retry_scheduled"),
])
def test_every_provider_outcome_maps_to_one_order_state(
    stubbed_provider, order, status, body, expected,
):
    stubbed_provider.post(CHARGE_URL, json=body, status=status)

    settle(order)

    order.refresh_from_db()
    assert order.state == expected


@pytest.mark.django_db
def test_a_declined_card_never_leaves_stock_reserved(stubbed_provider, order):
    stubbed_provider.post(CHARGE_URL, json={"error": "card_declined"}, status=402)

    settle(order)

    assert order.reservation.released is True
    assert Stock.objects.get(sku=order.sku).remaining == 1


@pytest.mark.integration
def test_the_sandbox_still_speaks_the_shape_we_parse():
    charge = PaymentClient(settings.SANDBOX_KEY).charge(amount=100, token="tok_visa")

    assert charge["status"] == "succeeded"
    assert isinstance(charge["id"], str)

5–7: `assert_all_requests_are_fired=True` turns a stub nobody called into a failure. Without it, a refactor that stops calling the provider entirely leaves every test green.
11–16: Four provider outcomes, four order states, one table. The 409 row is the interesting one — "already charged" must land on `paid`, not on an error, or a retry corrupts the order.
19: The stub is the point of control. You cannot ask a sandbox for a 500 on demand; you can always ask a stub.
28–34: The assertion that is about *your* system rather than the provider's: a failed charge must not leave stock held. This is exactly the branch a happy-path-only suite skips.
37–42: One narrow test against the real sandbox, tagged so it is deselected by default. It asserts the shape and nothing else — it is not trying to cover the branches above.

Why this works: The parametrized stub test covers every branch cheaply and deterministically, and the one sandbox test covers the assumption the stubs are built on. Neither can do the other's job, and the tag keeps the slow one out of the commit loop.

Running the whole payment suite against the sandbox

Wrong

python

@pytest.mark.django_db
def test_declined_card(order):
    order.card_token = "tok_chargeDeclined"     # hope the sandbox still has this token
    settle(order)
    assert order.state == "declined"
# 40 tests like this: 6 minutes, fails on their maintenance window

Better

python

# 40 stubbed tests: 0.4s, deterministic, every branch reachable
# + 1 tagged sandbox test asserting the response shape

What you see: CI takes minutes instead of seconds and goes red on days you changed nothing — provider maintenance, a rotated sandbox key, a rate limit. People start re-running CI reflexively, and then stop reading failures at all.

Why: A sandbox is a shared, stateful, network-dependent system, so a suite built on it inherits every one of those properties. Worse, it cannot produce the branches that matter: there is no reliable way to make it return a 500, or time out, or return a 409 on the second call. Stubs give you all of those on demand. Keep the sandbox for the one job stubs structurally cannot do — telling you the shape changed — and run it on a schedule, where a red result is information rather than noise.

Two tests, two different failures caught

Contract test — stubbed provider

+Runs on every commit, in milliseconds
+You choose the response, including the 402 and the timeout
+Catches: your code mishandling a shape
+Cannot catch: the provider changing that shape
+Deterministic — no network, no rate limits, no card expiry

Integration test — real sandbox

−Runs nightly, tagged, allowed to fail loudly
−The provider chooses the response
−Catches: a renamed field, a changed status code, a new required header
−Cannot replace contract tests — you cannot make it produce every branch
−Its job is to invalidate your stubs, not to cover your logic

The four doubles, and when each is the right one

The four doubles, and when each is the right one
Double	What it does	Reach for it when
Stub	returns a canned response	you are testing your branches — the default choice
Fake	a real but simplified implementation	state matters: an in-memory bucket where `put` then `get` must agree
Spy	passes through, records the calls	the call is real but you also want to assert its arguments
Mock	asserts calls were (or were not) made	the assertion is "we never charged the card twice"

Together

python

responses.post(CHARGE_URL, json={"id": "ch_1", "status": "succeeded"}, status=200)
charge_card(order)
assert len(responses.calls) == 1                       # spy half
assert responses.calls[0].request.headers["Idempotency-Key"]

Remember: Contract tests answer "does my code handle this shape", integration tests answer "is this still the shape" — you need both, and the second one exists to invalidate the first one's stubs, not to cover your branches. Keep sandbox tests few, tagged and scheduled; keep stubbed tests exhaustive, because only a stub gives you a 500 or a timeout on demand. Put the double at the HTTP boundary rather than on your own client class, or the URL, headers and timeout you actually ship go untested.

Retries, timeouts, and idempotency

Three names for one bug, and the assertions that would actually catch a double charge.

Testing retry behaviour, timeouts, and idempotency

coreadvanced

These three are one test area because they are one bug. A request times out, your code retries, and the provider charges the card twice — because the first call had in fact succeeded and only the *response* was lost. So test all three together: make the first attempt fail, assert the code retried, and assert the second attempt carried the same idempotency key. A stub library with an ordered registry gives you exactly that — register a timeout followed by a success, and the two attempts are two entries you can inspect.

Think of it as

The thing to hold onto is that a timeout tells you nothing about whether the work happened. A connect timeout means the request probably never arrived; a read timeout means it arrived, was processed, and the reply was lost on the way back. Your code cannot tell these apart, which is why "retry on timeout" and "send an idempotency key" are not two decisions — the second is the price of the first. Testing follows the same shape. It is not enough to assert a retry occurred; you have to assert what the retry *carried*. A retry with a fresh key is a second charge with extra steps, and the only test that catches it inspects the request headers of both attempts and asserts they match. Then there is the part people leave out entirely: exhaustion. Every retry policy has a last attempt, and what your code does after it is a real branch — mark the order for manual review, leave the reservation held, alert someone. That branch runs in production during exactly the incident where you can least afford it to be wrong, and it is trivially cheap to test with a stub that fails every time. Finally, set a timeout at all. `requests` has no default: a call with no `timeout=` can hang until the worker is recycled, and one slow provider becomes your outage.

python

@responses.activate(registry=OrderedRegistry)
def test_retry():
    responses.post(URL, body=ConnectTimeout())      # attempt 1
    responses.post(URL, json={"id": "ch_1"})        # attempt 2

What we're doing: Prove the retry happens, that it reuses the key, that a 409 counts as success, and that exhaustion leaves a state a human can act on.

billing/tests/test_retry_semantics.pypython

@pytest.mark.django_db
@responses.activate(registry=OrderedRegistry)
def test_a_timeout_is_retried_with_the_same_idempotency_key(order):
    responses.post(CHARGE_URL, body=ReadTimeout("timed out"))
    responses.post(CHARGE_URL, json={"id": "ch_1", "status": "succeeded"}, status=200)

    settle(order)

    assert len(responses.calls) == 2
    first, second = (c.request.headers["Idempotency-Key"] for c in responses.calls)
    assert first == second == order.idempotency_key
    order.refresh_from_db()
    assert order.state == "paid"


@pytest.mark.django_db
@responses.activate(registry=OrderedRegistry)
def test_a_409_on_the_retry_is_treated_as_success_not_failure(order):
    responses.post(CHARGE_URL, body=ReadTimeout("timed out"))
    responses.post(CHARGE_URL, json={"error": "already_charged", "id": "ch_1"}, status=409)

    settle(order)

    order.refresh_from_db()
    assert order.state == "paid"
    assert order.charge_id == "ch_1"


@pytest.mark.django_db
@responses.activate
def test_exhausting_every_attempt_leaves_the_order_for_a_human(order):
    responses.post(CHARGE_URL, body=ConnectTimeout("unreachable"))

    settle(order)

    assert len(responses.calls) == MAX_ATTEMPTS
    order.refresh_from_db()
    assert order.state == "needs_review"
    assert order.reservation.released is False      # stock stays held: it may yet be charged


@pytest.mark.django_db
@responses.activate
def test_a_declined_card_is_not_retried(order):
    responses.post(CHARGE_URL, json={"error": "card_declined"}, status=402)

    settle(order)

    assert len(responses.calls) == 1                # a decline is final, not transient

9–11: The assertion that makes this a real idempotency test. A retry with a fresh key would still leave `len(responses.calls) == 2` and still mark the order paid — and still double-charge the customer.
18–20: The branch nearly every suite is missing. After a lost response, the provider says "you already did this" — and the only correct reading of that is success, not an error to surface.
32: A single non-ordered stub replays for every attempt, which is what you want to model a provider that stays down.
38–39: Two assertions, because "handled the failure" is not enough: the state must be one a human can act on, and stock must stay held while the charge outcome is genuinely unknown.
42–49: The negative retry test. A 402 is a permanent answer, so retrying it wastes the budget and delays telling the customer their card was refused.

Why this works: Together these four pin the whole policy: what is retried, what is not, what a retry carries, and what is left behind when retrying stops. Each one is a branch that only ever runs during an incident, which is exactly why it needs a test rather than a code review.

Asserting the retry count and not the retry payload

Wrong

python

def test_timeout_is_retried(order):
    ...
    settle(order)
    assert len(responses.calls) == 2      # green with a fresh key on attempt 2

Better

python

def test_timeout_is_retried(order):
    ...
    settle(order)
    keys = {c.request.headers["Idempotency-Key"] for c in responses.calls}
    assert len(responses.calls) == 2
    assert len(keys) == 1                 # the retry reused the key

What you see: Customers are charged twice, and the pattern only shows up during provider slowness — so it is invisible in normal conditions and arrives in a batch during an incident.

Why: The count proves your retry loop runs; it says nothing about whether the retry is safe. If each attempt generates a fresh idempotency key, the provider sees two unrelated charge requests and honours both. Asserting the *set* of keys has size one is what makes the test about idempotency rather than about the loop, and it costs one line.

One outbound call, every path it can take

attempt n

start

success → mark paid

end

timeout / 5xx

wait, then retry with the SAME key

409 already_charged

402 declined → release stock

end

attempts exhausted → manual review

end

Four failures, and what each one licenses you to do

Four failures, and what each one licenses you to do
Failure	Did the work happen?	Safe to retry?
connect timeout	almost certainly not	yes — with the same key anyway
read timeout	unknown — possibly yes	only with an idempotency key
`500` / `503`	unknown	yes, with backoff and the same key
`409 already_charged`	yes, on a previous attempt	no — treat it as success
`402 card_declined`	yes, and it failed for good	no — retrying re-declines forever

Together

python

response = session.post(url, json=payload, timeout=(3.05, 10),
                        headers={"Idempotency-Key": order.idempotency_key})

Remember: A read timeout means the work may already be done, so the retry must carry the same idempotency key — and the test must assert that the two attempts' keys are equal, not merely that two attempts happened. Treat `409 already_charged` on a retry as success, never retry a permanent `402`, and always test the exhausted path, because it runs during the incident where being wrong costs most. Set `timeout=(connect, read)` on every outbound call: `requests` has no default, and one hung provider with a fixed worker pool is your outage.

Webhooks, and the five surfaces

The direction where the provider calls you — public, redelivered, unordered — plus the standard double per surface.

Webhook testing, and the five integration surfaces

coreadvanced

A webhook is the provider calling *you*, so the tests are the reverse of every other integration test: you build the request. Four of them matter — a valid signed event that does the work, a forged signature that is rejected, the same event delivered twice that must not do the work twice, and events arriving out of order. Beyond payments, a Django service usually talks to five kinds of thing: payment providers, email providers, storage APIs, other REST APIs, and message brokers. Each has a standard double, and for two of them Django or the library already ships one — email has the `locmem` backend, and storage has an in-memory or temporary-directory backend.

Think of it as

Treat a webhook endpoint as a public, unauthenticated, hostile-input view that happens to be given a shared secret, because that is exactly what it is. Anyone on the internet can POST to it. The signature check is therefore the first thing in the view and the first thing in the test suite, and the negative test — a body whose signature does not match — is worth more than the positive one, because the positive one will be written anyway. The second property is that delivery is at-least-once. Providers retry when your response is slow or non-2xx, and a retry is byte-identical to the original, so "process this event" must be safe to run twice. The test for that is not subtle: post the same payload twice, assert the side effect happened once. The third is ordering, which people discover late — `payment.succeeded` and `payment.refunded` can arrive in either order, and a handler that trusts arrival order will happily un-refund a payment. Guard with the event's own timestamp or sequence number, and test it by delivering the pair backwards. Across the five surfaces, the useful instinct is to ask what already exists before writing a double: Django gives you `locmem` for email, `django-storages` and `tmp_path` cover storage, `responses` covers REST, `CELERY_TASK_ALWAYS_EAGER` or a fake broker covers queues, and only payments generally need real stubbing work.

python

response = client.post(
    "/webhooks/payments/", data=raw_body, content_type="application/json",
    HTTP_X_SIGNATURE=sign(raw_body, settings.WEBHOOK_SECRET),
)

What we're doing: The four webhook tests that matter, written against the view rather than the handler.

billing/tests/test_webhooks.pypython

def deliver(client, payload, secret=None):
    raw = json.dumps(payload).encode()
    return client.post(
        reverse("payment-webhook"), data=raw, content_type="application/json",
        HTTP_X_SIGNATURE=sign(raw, secret or settings.WEBHOOK_SECRET),
    )


@pytest.mark.django_db
def test_a_forged_signature_is_refused_and_changes_nothing(client, order, event):
    response = deliver(client, event, secret="not-the-real-secret")

    assert response.status_code == 400
    order.refresh_from_db()
    assert order.state == "pending"
    assert not WebhookEvent.objects.exists()


@pytest.mark.django_db
def test_a_valid_event_marks_the_order_paid(client, order, event):
    assert deliver(client, event).status_code == 200

    order.refresh_from_db()
    assert order.state == "paid"
    assert order.charge_id == event["data"]["charge_id"]


@pytest.mark.django_db
def test_the_same_event_delivered_twice_pays_the_order_once(client, order, event):
    assert deliver(client, event).status_code == 200
    assert deliver(client, event).status_code == 200      # the provider's retry

    assert WebhookEvent.objects.filter(event_id=event["id"]).count() == 1
    assert LedgerEntry.objects.filter(order=order).count() == 1


@pytest.mark.django_db
def test_a_late_older_event_does_not_undo_a_newer_one(client, order):
    refunded = make_event("payment.refunded", occurred_at="2026-09-05T10:05:00Z")
    succeeded = make_event("payment.succeeded", occurred_at="2026-09-05T10:00:00Z")

    deliver(client, refunded)
    deliver(client, succeeded)        # older, delivered second

    order.refresh_from_db()
    assert order.state == "refunded"  # not resurrected as paid

1–6: One helper signs the exact bytes it posts. Signing a re-serialized dict instead is the single most common reason a webhook test passes while production rejects every event.
10–16: The negative test first. It asserts three things — the status, that nothing changed, and that no event row was written — because "rejected" must mean rejected before any work.
28–33: At-least-once, made concrete. Both deliveries must return 200 (a non-2xx makes the provider retry harder), and the ledger must contain exactly one entry.
37–46: Ordering. Delivered in this sequence a naive handler sets `paid` last and the customer keeps both the refund and the goods. The guard is the event's own `occurred_at`, not arrival order.
44: The comment is the assertion in English. Reviewers who skim the name and the comment can tell what breaks without reading the handler.

Why this works: These four cover the properties a webhook endpoint actually has — public and forgeable, delivered more than once, and unordered — rather than the one property it appears to have, which is "the provider tells us what happened".

Verifying the signature against re-serialized JSON

Wrong

python

payload = json.loads(request.body)
expected = sign(json.dumps(payload), settings.WEBHOOK_SECRET)   # bytes changed
if expected != request.headers["X-Signature"]:
    return HttpResponse(status=400)

Better

python

expected = sign(request.body, settings.WEBHOOK_SECRET)          # the raw bytes
if not hmac.compare_digest(expected, request.headers.get("X-Signature", "")):
    return HttpResponse(status=400)

payload = json.loads(request.body)                              # parse after

What you see: Every real event is rejected with a 400 while the test suite passes — because the test signs a re-serialized body too, so both sides make the same mistake and agree with each other.

Why: `json.dumps()` is not the inverse of `json.loads()`: key order, whitespace and number formatting can all differ from what the provider sent, and any single byte of difference changes the HMAC. The signature is over the bytes on the wire, so verification has to use `request.body` before parsing. Use `hmac.compare_digest` rather than `==` as well — a plain comparison returns early on the first differing byte, which leaks timing information about the expected value.

One payment, three inbound deliveries — and the two your handler must survive

provider

webhook view

task queue

database

1. POST /webhooks/payments (event evt_1)raw body + signature header
2. verify signature over the RAW bodyreject before parsing anything
3. INSERT WebhookEvent(evt_1)unique on event id — the dedupe point
4. enqueue handle_payment(evt_1)
5. 200 — fast, before the work
6. POST evt_1 again (retry)at-least-once: identical bytes
7. INSERT evt_1 → IntegrityErroralready seen — swallow and 200
8. 200 — no second charge applied
9. POST evt_0 (older, arrives late)ordering is not guaranteed
10. occurred_at < stored → ignorethe test that delivers the pair backwards

The five surfaces and the double each one wants

The five surfaces and the double each one wants
Surface	Standard double	The test people skip
Payment providers	`responses` stub + one sandbox test	the retry that must reuse the idempotency key
Email providers	Django `locmem` → `mail.outbox`	the send that must NOT happen on rollback
Storage APIs	temp-directory or in-memory storage backend	the upload that fails halfway and leaves no orphan row
External REST APIs	`responses` at the HTTP boundary	timeout and 5xx branches, not only 200
Message brokers	eager execution, or an in-memory broker	the duplicate delivery — consumers get at-least-once too

Together

python

@override_settings(EMAIL_BACKEND="django.core.mail.backends.locmem.EmailBackend")
def test_receipt_body(...):
    ...
    assert mail.outbox[0].to == ["buyer@example.com"]

Remember: Sign and verify the **raw body** — re-serialized JSON is different bytes, and a test that makes the same mistake agrees with itself while production rejects everything. Write the forged-signature test before the happy path, deliver the same event twice and assert the side effect happened once, and deliver a related pair out of order to prove your handler uses the event timestamp rather than arrival order. Acknowledge fast and process in a task, or the provider's retries multiply the load. Across the other four surfaces, look for the double that already exists before building one: `locmem` for email, a temp-directory backend for storage, `responses` for REST, eager execution for queues.