beginnerintermediateadvanced

Django · Section 80

Email Systems

Level: intermediate
Read: 28 min
Concepts: 3

Django has one email API, and most of what looks like separate features are parameters on it. `send_mail()` is the one-liner and returns the number of messages delivered — `1` or `0`, since it can only send one. `EmailMessage` is the object form, the one that has cc, bcc, `reply_to`, headers and attachments; `EmailMultiAlternatives` adds `attach_alternative(html, "text/html")`, which is the entirety of what "HTML email" means. The plain-text body stays, deliberately: a client that will not render HTML needs something to show, and a multipart message with no text half is a routine spam signal. Underneath sits a backend chosen by `EMAIL_BACKEND`, and swapping it is how the same code writes to stdout in development, to one file per message when you want to open the markup in a browser, and to `mail.outbox` in tests — with no call site changing. The second idea is where delivery runs. Sending inside a view puts somebody else's SMTP server on your request path, so a slow relay becomes slow signups and an unavailable relay becomes failed ones. Commit the row, then enqueue with `transaction.on_commit` — enqueueing inside the transaction lets a worker outrun its own data, or survive a rollback and email a customer about an order that never existed. Pass ids rather than instances, retry only the transient failures with backoff *and* jitter so a recovering relay does not meet the entire backlog at once, and never retry a refused address. The third idea corrects what `send()` proves: your relay accepted the message. Delivery, bounces and complaints arrive later by webhook, so keep a row per delivery keyed by the provider message id. A hard bounce is SMTP's 5xx family and is permanent — suppress the address, because repeatedly mailing dead addresses is what erodes deliverability for the mail that matters; a soft bounce is 4xx and is worth retrying. And treat opens as a weak signal: the pixel is fetched by privacy proxies, not only by people.

Django overview

Python Django Full Course for Beginners | Complete All-in-One Tutorial | 3 HoursDave Gray

What is true here

One API: send_mail for the trivial case, EmailMessage when you need headers or attachments.
HTML mail is an alternative attached to a plain-text body, never a replacement for it.
EMAIL_BACKEND makes the transport an environment concern instead of a code concern.
Commit first, enqueue with on_commit, retry transient failures only — with jitter.
send() returning 1 is custody, not delivery; bounces and complaints arrive by webhook.

What you will be able to do

Send a two-part email with attachments over a single connection
Test mail without a network, by asserting against `mail.outbox`
Keep a slow or dead relay from failing the request that triggered the email
Tell a permanent bounce from a temporary one, and act on the difference

From a view to a mailbox — and the three places the path can end

The view

creates the order and commits — nothing more

transaction.on_commit(…)

the queue call waits for a successful commit

Enqueued inside the transaction

worker outruns the row, or a rollback still emails

Worker task

idempotent: checks status before doing anything

EmailMultiAlternatives

text body + HTML alternative + attachment

EMAIL_BACKEND

smtp · console · filebased · locmem · dummy

Your relay accepts it

send() returns 1 — this is custody, not delivery

Transient failure

backoff + jitter, capped, then a dead-letter queue

Delivered

reported later, by webhook

Hard bounce (5xx)

suppress the address — do not send again

The API, and the backend beneath it

One message object, four transports, and what "HTML email" actually is.

The email API, and the backend underneath it

coreintermediate

Django ships one email API with two entry points. `send_mail(subject, message, from_email, recipient_list)` is the one-liner, and it returns the number of messages delivered — `1` or `0`, because it can only send one. `EmailMessage` is the object you build when you need more than that: cc, bcc, `reply_to`, extra headers, attachments. `EmailMultiAlternatives` adds `attach_alternative(html, "text/html")`, which is how you send an HTML email that still has a plain-text body. Underneath both sits a **backend**, chosen by the `EMAIL_BACKEND` setting, and swapping it is how the same code writes to your console in development and to an SMTP server in production.

Think of it as

Think of it as three layers you can substitute independently. At the top is a message — a subject, a body, some recipients, maybe an HTML alternative and some attachments — and that object knows nothing about how it travels. In the middle is a connection, an open channel to whatever actually accepts mail. At the bottom is the backend, the code that implements that channel: SMTP in production, console or file-based while you are building, `locmem` in tests where every message lands in `django.core.mail.outbox` instead of a network. Because the message does not know which backend it will meet, none of your application code changes between those environments — one setting does. The layer that catches people out is the connection. `send_messages()` will open a connection if one is not open and close it afterwards, so a loop that calls `send_mail()` five hundred times performs five hundred SMTP handshakes. Open one connection with `get_connection()`, pass it to every message, and the documented behaviour changes: a connection you opened manually is left open, so the handshake happens once. The other thing worth internalising early is that "HTML email" is not a separate API. It is a plain-text message with an HTML *alternative* attached, and the two-part shape is deliberate — a client that will not or cannot render HTML still has something to show, and a mail body with no text part is a well-known spam signal.

python

send_mail(subject, message, from_email, recipient_list, fail_silently=False, html_message=None)

What we're doing: Send a receipt as a two-part email — plain text and HTML rendered from templates — with the PDF attached, over a single connection.

billing/mail.pypython

from django.core.mail import EmailMultiAlternatives, get_connection
from django.template.loader import render_to_string


def build_receipt(order, pdf_bytes, connection=None):
    context = {"order": order, "total": order.total}

    message = EmailMultiAlternatives(
        subject=f"Receipt for order {order.reference}",
        body=render_to_string("billing/receipt.txt", context),
        to=[order.customer_email],
        reply_to=["support@example.test"],
        headers={"X-Order-Reference": order.reference},
        connection=connection,
    )
    message.attach_alternative(
        render_to_string("billing/receipt.html", context), "text/html"
    )
    message.attach(f"receipt-{order.reference}.pdf", pdf_bytes, "application/pdf")
    return message


def send_receipts(orders):
    connection = get_connection()
    connection.open()               # opened here, so it is NOT closed per message
    try:
        messages = [build_receipt(o, render_pdf(o), connection) for o in orders]
        return connection.send_messages(messages)
    finally:
        connection.close()

# send_messages() returns the number of successfully delivered messages.

9–10: `body` stays plain text. The HTML is an *alternative* to it, not a replacement — a client that will not render HTML still has something to show.
16: `attach_alternative(content, mimetype)` is the whole of "HTML email". There is no separate HTML message class.
19: `attach(filename, content, mimetype)` takes bytes already in memory. `attach_file(path)` is the variant that reads from disk. Omit the mimetype and Django guesses it from the filename.
24: Opening the connection yourself is the difference between one SMTP handshake and one per message — `send_messages()` only opens and closes implicitly when it finds the connection closed.
30: The return value is a count, not a boolean. Comparing it against `len(messages)` is how you notice a partial failure at all.

Why this works: One handshake carries the whole batch, every recipient gets a body their client can render, and the count that comes back is the only evidence you have that anything was accepted.

Sending HTML as the body instead of as an alternative

Wrong

python

send_mail(subject, html, from_email, [to])
# body is now a wall of markup: no text part at all

Better

python

send_mail(subject, text, from_email, [to], html_message=html)
# or EmailMultiAlternatives + attach_alternative(html, "text/html")

What you see: Some recipients see raw `<table>` markup instead of a message, and delivery rates to strict providers drop without any error being raised.

Why: A mail body is plain text unless the message declares an HTML alternative, so passing markup as the body sends the markup itself as the text a client displays. `send_mail` already has `html_message=` for this, and `EmailMultiAlternatives` is the general form. Keeping a real text part is not only about old clients: a multipart message whose text half is missing is a routine spam heuristic, so the two-part shape protects deliverability as well as readability.

Three substitutable layers — and the one setting that swaps the bottom one

Your code

builds a message: subject, text body, HTML alternative, attachments

EmailMessage / EmailMultiAlternatives

knows nothing about how it will travel — that is the point

Connection

get_connection() — open it yourself and one handshake serves the whole batch

Backend — EMAIL_BACKEND

smtp in production · console/filebased in dev · locmem in tests · dummy for never

An SMTP server, stdout, a file, or a list

the only layer that differs between your laptop and production

The backends, and what each one is actually for

The backends, and what each one is actually for
`EMAIL_BACKEND`	Where mail goes	Use it for
`...backends.smtp.EmailBackend`	a real SMTP server	production — this is the default
`...backends.console.EmailBackend`	stdout	local development; you read the mail in the runserver log
`...backends.filebased.EmailBackend`	one file per message in `EMAIL_FILE_PATH`	local development when you want to open the HTML in a browser
`...backends.locmem.EmailBackend`	`django.core.mail.outbox`, a list	tests — assert on `len(mail.outbox)` and on the message
`...backends.dummy.EmailBackend`	nowhere	a load test, or an environment that must never send

Together

python

# settings/dev.py
EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"

# tests assert against the outbox the locmem backend fills
from django.core import mail
assert len(mail.outbox) == 1
assert mail.outbox[0].subject == "Your receipt"

Which class to reach for

Which class to reach for
You need	Use	Why
one plain message, nothing special	`send_mail(...)`	returns 1 or 0; no object to build
cc, bcc, `reply_to`, headers	`EmailMessage(...)`	`send_mail` exposes none of these
an HTML version as well	`EmailMultiAlternatives`	`attach_alternative(html, "text/html")`
a file on the message	`.attach()` / `.attach_file()`	content in memory vs a path on disk
many messages at once	`connection.send_messages([...])`	one handshake, not one per message

Together

python

from django.core.mail import get_connection

connection = get_connection()
connection.open()                      # opened by you, so it stays open
connection.send_messages(messages)     # one handshake for the whole batch
connection.close()

Remember: `send_mail` returns a count (1 or 0), not a boolean; `EmailMessage` is the object form; `EmailMultiAlternatives.attach_alternative(html, "text/html")` is all "HTML email" means — the plain-text body stays. `EMAIL_BACKEND` swaps SMTP for console, filebased, locmem or dummy without touching a single call site, so tests assert on `mail.outbox`. Open a connection with `get_connection()` before a batch, or you pay one SMTP handshake per message. Treat `fail_silently=True` as data loss you agreed to.

Off the request path

Commit, enqueue, retry the transient failures only — and never block a signup on a relay.

Delivery belongs in the background

coreintermediate

Sending mail inside a view puts a third party on your request path. An SMTP handshake to a relay that is having a slow day turns a 200 ms signup into a 30 s one, and if the relay is down the signup fails even though the account was created. The fix the roadmap states outright — **do not block critical requests on slow email delivery** — is to commit the database change, enqueue a task, and return. The worker then sends the message and can retry on failure, which a request cannot.

Think of it as

Separate two questions that a synchronous `send_mail()` collapses into one: did the thing happen, and did the customer hear about it? The first is your transaction and belongs in the request. The second depends on a network, a relay, a provider queue and a mailbox you do not control, so it belongs in a job that is allowed to take minutes and fail twice. Once they are split, the failure modes stop being coupled: a relay outage delays receipts instead of rejecting checkouts. The ordering inside the request is where this goes wrong, and it is worth being precise. Enqueue *after* the transaction commits, using `transaction.on_commit`, because a task queued mid-transaction can be picked up by a worker before the row it refers to is visible — or after a rollback that means the row will never exist. That produces an email about an order nobody placed, which is worse than a late email. On the worker side, retries need the same discipline any other retry needs: an exponential delay with jitter so a relay coming back does not meet your entire backlog at once, a cap on attempts, and a distinction between errors worth retrying (a timeout, a 5xx, a rate limit) and errors that will never succeed (a malformed address, a hard bounce). The last piece is the provider choice. An HTTP API and SMTP reach the same destination, but the API returns a provider message id you can correlate later, which is what makes the delivery-status half of this section possible at all.

python

transaction.on_commit(lambda: send_welcome_email.delay(user.id))

What we're doing: A delivery task that survives a flaky relay: bounded retries with jitter, permanent failures separated from transient ones, and a record of what the provider accepted.

notifications/tasks.pypython

from celery import shared_task
from django.db import transaction
from smtplib import SMTPRecipientsRefused, SMTPServerDisconnected


@shared_task(
    bind=True,
    autoretry_for=(SMTPServerDisconnected, TimeoutError),
    retry_backoff=True,        # 1s, 2s, 4s, 8s …
    retry_jitter=True,         # spread the retries of a whole backlog
    retry_kwargs={"max_retries": 5},
)
def send_receipt(self, delivery_id):
    delivery = EmailDelivery.objects.select_related("order").get(pk=delivery_id)

    if delivery.status in {"sent", "cancelled"}:
        return                                    # already done: a duplicate run

    try:
        message = build_receipt(delivery.order, render_pdf(delivery.order))
        accepted = message.send()
    except SMTPRecipientsRefused:
        delivery.mark_permanently_failed("address refused")
        return                                    # never retried

    delivery.status = "sent" if accepted else "not_accepted"
    delivery.save(update_fields=["status", "updated_at"])


def queue_receipt(order):
    delivery = EmailDelivery.objects.create(order=order, status="queued")
    transaction.on_commit(lambda: send_receipt.delay(delivery.id))

7–11: Backoff spreads the retries of one message; jitter spreads the retries of the whole backlog. Without jitter, every task queued during an outage retries in the same second and knocks the relay back over.
13: The task takes an id. Passing the model instance would serialise a snapshot taken before the commit — the worker must re-read current state.
16–17: The idempotency guard. Queues deliver at least once, so this task will occasionally run twice; checking the status first is what stops the customer getting two receipts.
21–24: A refused recipient is separated from a dropped connection and never retried. Five attempts at an address that does not exist is five guaranteed failures and a worse sender reputation.
30: `on_commit` is the load-bearing line: enqueueing before the commit races the worker against your own transaction, and a rollback would leave a task referring to a row that never existed.

Why this works: The request returns as soon as the row is committed, a slow relay costs a delayed email rather than a failed signup, and every failure either retries with a widening gap or stops permanently on purpose.

Enqueueing inside the transaction instead of after it commits

Wrong

python

with transaction.atomic():
    order = Order.objects.create(...)
    send_receipt.delay(order.id)     # a worker may pick this up NOW

Better

python

with transaction.atomic():
    order = Order.objects.create(...)
    transaction.on_commit(lambda: send_receipt.delay(order.id))

What you see: Intermittent `DoesNotExist` in the worker under load, and — worse and rarer — a receipt emailed for an order whose transaction later rolled back.

Why: A broker accepts the message the moment `.delay()` is called, so a free worker can start before your transaction commits. It then queries a row that is not visible yet and raises `DoesNotExist`, which looks like a flaky worker rather than an ordering bug. The rarer case is worse: if the transaction rolls back, the task is still queued, and it emails a customer about something that never happened. `transaction.on_commit` defers the call until after a successful commit, which makes both impossible.

The same signup, with the relay 30 seconds slow

Sending inside the view

+The user waits for your relay, not for your database
+A relay outage fails the signup — the account is rolled back
+No retry is possible: the request is already over
+Worker threads are held open by a network you do not own
+A gunicorn timeout kills it mid-send; did it go out or not?

Committing, then enqueueing

−The user waits for one INSERT
−A relay outage delays the email; the signup still succeeds
−The worker retries with backoff, and gives up into a DLQ
−Request threads return immediately
−The task is idempotent, so a duplicate run is harmless

What to do with each failure the send can produce

What to do with each failure the send can produce
Failure	Retry?	What to do instead
connection timeout, relay refused	yes	backoff + jitter; it is almost always transient
provider 5xx	yes	same — capped, then dead-letter
provider 429 (rate limited)	yes	honour the retry-after the provider gives you
invalid address / 5xx hard bounce	no	mark the address undeliverable; retrying cannot help
template raised while rendering	no	a code bug — fail loudly, do not retry it 25 times
exhausted every attempt	—	dead-letter queue, and an alert; do not drop it silently

Together

python

try:
    message.send()
except (SMTPServerDisconnected, SMTPConnectError) as exc:
    raise self.retry(exc=exc)          # transient — backoff + jitter
except SMTPRecipientsRefused:
    EmailAddress.objects.filter(address=to).update(undeliverable=True)
    return                             # permanent — retrying cannot help

Remember: Commit first, enqueue second, and do it with `transaction.on_commit` — a task queued mid-transaction can outrun its own row or survive a rollback. Pass ids, not instances. Retry only transient failures, with backoff *and* jitter so a recovering relay does not meet the whole backlog at once, and never retry a refused address. Make the task idempotent, because at-least-once delivery means it will sometimes run twice. The rule the section states plainly: do not block a critical request on somebody else's mail server.

After send() returns

Bounces, complaints and the delivery lifecycle your relay never told you about.

Bounces, and what "sent" actually means

standardintermediate

A successful `send()` means your relay accepted the message, not that anyone received it. Everything after that — the destination server accepting or rejecting it, the mailbox being full, a spam filter discarding it — happens minutes later and reaches you only if you ask for it. A **bounce** is the rejection notice. A **hard bounce** is permanent (the address does not exist) and the address must never be mailed again; a **soft bounce** is temporary (mailbox full, server busy) and is worth retrying. Providers report both, along with delivered/opened/complained, by calling a webhook you expose.

Think of it as

Treat "sent" as the first state in a lifecycle rather than the end of one. The moment your code hands the message off, all you know is that a relay took custody — an SMTP 250 on your own hop. The interesting outcomes arrive afterwards and out of band, so the only way to know them is to store a row per delivery, key it by the provider message id, and let inbound events move it along. That row is what turns "did they get the invoice?" from a shrug into a query. The classification that matters most is permanent versus temporary, because it decides whether the address is still usable. SMTP already draws that line by reply-code family: 5xx is a permanent negative reply and 4xx is transient, which is exactly the hard/soft distinction under a different name. Acting on it is not politeness, it is deliverability — mailbox providers score senders on how often they are handed addresses that do not exist, and a system that keeps retrying dead addresses degrades delivery for the mail that *does* matter. So a hard bounce should mark the address undeliverable at the point where every future send checks it, and a spam complaint should suppress that address for marketing while leaving genuinely transactional mail — a password reset the person asked for — alone. The last trap is the open/click signal. Open tracking works by embedding a pixel, and privacy proxies that fetch every image on arrival make opens read as high and near-instant. It is a weak signal, and it is the wrong one to build "did they see it?" logic on.

python

EmailDelivery.objects.filter(provider_message_id=mid).update(status="bounced")

What we're doing: A webhook endpoint that records provider events idempotently and suppresses an address the moment it hard-bounces.

notifications/webhooks.pypython

PERMANENT = {"bounce_hard", "invalid_address"}
TERMINAL_RANK = {"queued": 0, "accepted": 1, "delivered": 2,
                 "bounce_soft": 2, "bounce_hard": 3, "complaint": 3}


@require_POST
@csrf_exempt
def provider_events(request):
    event = verify_and_parse(request)          # signature check first — see §88

    delivery = EmailDelivery.objects.filter(
        provider_message_id=event["message_id"]
    ).first()
    if delivery is None:
        return HttpResponse(status=200)        # unknown id: ack, do not 500

    # Webhooks arrive at least once and out of order. Never move backwards.
    incoming = TERMINAL_RANK.get(event["type"], 0)
    if incoming <= TERMINAL_RANK.get(delivery.status, 0):
        return HttpResponse(status=200)

    with transaction.atomic():
        delivery.status = event["type"]
        delivery.status_at = event["occurred_at"]
        delivery.save(update_fields=["status", "status_at"])

        if event["type"] in PERMANENT:
            EmailAddress.objects.update_or_create(
                address=delivery.to_address,
                defaults={"suppressed_at": timezone.now(),
                          "reason": event["type"]},
            )

    return HttpResponse(status=200)

9: Verification comes before parsing. An unauthenticated bounce endpoint lets anyone suppress any address — see the webhook section for how the signature check works.
11–15: An unknown message id returns 200, not an error. Providers retry non-2xx responses, so a 500 here turns one unmatched event into a stream of them.
17–20: The ordering guard. Providers deliver at least once and out of order, so a late `delivered` can arrive after a `bounce_hard`; ranking the states stops a stale event resurrecting a dead address.
27–31: Suppression is a row in a table every send path consults — not a flag set somewhere in the mail helper. If suppression is not checked at send time, recording it changes nothing.

Why this works: Every event is safe to receive twice, no event can move a delivery backwards, and a hard bounce ends in a suppression row that later sends actually read.

One message, and the states it moves through after your code stops looking

queued

start

accepted by your relay (send() returned 1)

delivered

end

deferred — soft bounce (4xx)

bounced — hard bounce (5xx)

end

complained (spam)

end

address suppressed

end

The delivery lifecycle after `send()` returns

The delivery lifecycle after `send()` returns
Event	What it means	What your system should do
`accepted`	your relay took custody	the only thing `send()` proves
`delivered`	the receiving server accepted it	the honest definition of success
`bounced` (hard, 5xx)	the address is permanently unusable	suppress it; stop all future sends
`deferred` (soft, 4xx)	temporarily undeliverable	the provider retries; you usually wait
`complained`	marked as spam by the recipient	suppress marketing; keep requested mail
`opened` / `clicked`	a tracking pixel or link was fetched	weak signal — proxies inflate it

Together

python

EmailDelivery.objects.filter(provider_message_id=event["message_id"]).update(
    status=event["type"], status_at=timezone.now()
)

Remember: `send()` returning 1 means your relay accepted the message — it is not delivery. Store a row per delivery keyed by the provider message id, or later events have nothing to attach to. Hard bounce (SMTP 5xx) is permanent: suppress the address, because repeatedly mailing dead addresses is what damages the deliverability of the mail you care about. Soft bounce (4xx) is transient. Webhooks are at-least-once and unordered, so rank the states and refuse to move backwards. Treat opens as a weak signal — privacy proxies fetch the pixel for the recipient.