beginnerintermediateadvanced

Django · Section 83

File Exports and Imports

Level: intermediate
Read: 32 min
Concepts: 3

Exports are a question about where bytes accumulate. An `HttpResponse` builds the whole body in your process before sending anything, so a large export is a memory peak that multiplies with every concurrent user and a long silence that a proxy may time out. `StreamingHttpResponse` takes an iterator instead, and Django's documented CSV recipe pairs it with a tiny `Echo` class whose `write()` returns the value rather than buffering it — that is what turns `csv.writer` into a generator. Two consequences follow and both bite: the response has no `content` attribute, and because headers are sent before the generator runs, an exception inside it cannot become a 500 — the user simply receives a truncated file that looks complete. So do everything fallible before the response object exists, and stream the queryset too with `.iterator()`, since streaming a materialised list saves nothing. Past a certain size the whole approach stops fitting: a streaming export still occupies a request worker for minutes, and `.xlsx` cannot stream at all because its zip directory is written last. Then the export becomes a job and, importantly, a *row* — requested, building, ready or failed — which is what the UI polls and what lets a failure be visible and retryable instead of arriving as a short file. Build to a temporary file rather than a buffer, because worker memory is usually tighter than web memory and a deterministic OOM turns retries into a loop. Store the result under a random key, hand out short-lived signed URLs so object storage serves the bytes, and link emails to a page rather than to the URL — a signed URL is a bearer credential, and email is a bad place to keep one. Imports invert the problem: untrusted input in bulk. Validate the entire file before writing any of it, reusing a `Form` per row so coercion and error messages come for free, and report every problem with its row number so the user fixes the spreadsheet once. Then choose a failure policy deliberately — all-or-nothing for a file that means one indivisible change, per-row with a reject report for a bag of independent records — because the accidental middle leaves a prefix applied and no record of where it stopped. Finally, make a re-upload harmless: a business key from the data, a `UniqueConstraint` to enforce it, and an upsert to apply it, never a check-then-create that races.

Django overview

Python Django Full Course for Beginners | Complete All-in-One Tutorial | 3 HoursDave Gray

What is true here

Streaming keeps memory flat and the connection alive, but forfeits the ability to fail properly.
Past some size an export is a job and a status row, not a response.
A signed URL is a bearer credential: expire it, randomise the key, do not email it.
Validate the whole file first; report every error with its row number.
Idempotency is a business key plus a UniqueConstraint plus an upsert.

What you will be able to do

Export a million rows without a memory spike or a proxy timeout
Know exactly when to stop streaming and start queueing
Give a user every error in their file in one pass
Make a re-uploaded file update rather than duplicate

Which mechanism fits — by how big the file is and whether failure must be reportable

failure must be reportablea truncated file is tolerable

small / boundedlarge / unbounded

HttpResponse: builds in memory, but an error is an honest 500 — fine for a few hundred rows
StreamingHttpResponse: flat memory and an immediate first byte; an error truncates silently
FileResponse: the file already exists — nothing to generate, nothing to fail mid-way
async job + signed URL: the only option that is both unbounded in size and honest about failure
.xlsx (any size): cannot stream at all — the zip directory is written last, so it must be built offline
import: validate then apply: two passes, so a bad row is reported instead of half-applied

Formats, and where the bytes accumulate

CSV, JSON Lines and Excel — and the `Echo` class that makes streaming work.

Formats, and streaming instead of building in memory

coreintermediate

An `HttpResponse` builds the whole body in memory before anything is sent, so a 400 MB export needs 400 MB of process memory and the browser sees nothing until it is finished. `StreamingHttpResponse` takes an iterator instead and sends rows as they are produced. Django's documented CSV recipe pairs it with a tiny `Echo` class whose `write()` returns the value rather than storing it, which turns `csv.writer` into a generator. The format choice is separate: CSV for tabular data, JSON when structure matters, and Excel only when a person will actually open it in Excel.

Think of it as

The question behind this concept is where the bytes accumulate. With a regular response they accumulate in your process — the queryset materialises, the rows render, the string grows, and only then does the first byte leave. That is fine for a hundred rows and it is a memory incident for a million, and the failure is not gradual: several concurrent exports multiply the same peak, so an endpoint that has been fine for a year falls over the day two people click it at once. Streaming inverts the shape. The response holds an iterator, the WSGI or ASGI server pulls from it, and each row is formatted, written and discarded, so memory stays roughly flat regardless of row count. There is a second benefit the documentation calls out directly: bytes start flowing immediately, so a proxy or load balancer sees an active connection instead of a long silence, and does not time it out while you build. Two consequences follow that catch people. Because there is no body until the iterator runs, there is no `content` attribute — anything that expects one, including some middleware, will not work with a streaming response. And because the status line and headers are sent before the generator has produced anything, an exception raised *inside* the generator cannot become a 500: the client has already been told the request succeeded and simply receives a truncated file. That makes it worth doing the risky work — the count, the permission check, the parameter validation — before the response object is created, while an error can still be an error. On format: CSV is a stream of rows and suits this model perfectly; JSON needs care because a single array is one big value, so stream it as JSON Lines or write the brackets yourself; and modern `.xlsx` is a zip archive whose central directory is written at the end, so it does not stream at all. Excel is the reason the third concept exists.

python

StreamingHttpResponse((writer.writerow(r) for r in rows), content_type="text/csv")

What we're doing: Stream a CSV export of an arbitrary number of orders, with every fallible step done before the response object exists.

reporting/views.pypython

import csv
from django.http import StreamingHttpResponse


class Echo:
    """Implements just the write method of the file-like interface: it
    RETURNS the formatted row instead of storing it, which is what turns
    csv.writer into a generator rather than a buffer."""

    def write(self, value):
        return value


HEADERS = ["reference", "placed_at", "customer", "total", "currency"]


def export_orders(request):
    # Everything that can fail happens BEFORE the response exists — while a
    # failure can still be a 403 or a 400 rather than a truncated file.
    form = ExportFilterForm(request.GET)
    if not form.is_valid():
        return HttpResponseBadRequest(form.errors.as_json())

    queryset = (
        Order.objects.for_user(request.user)          # authorisation, not filtering
        .filter(**form.cleaned_data)
        .select_related("customer")                   # or the generator N+1s
        .order_by("pk")
    )

    def rows():
        yield HEADERS
        for order in queryset.iterator(chunk_size=2000):
            yield [
                order.reference,
                order.placed_at.isoformat(),          # ISO-8601, not a local reading
                order.customer.display_name,
                str(order.total),                     # str(), never float()
                order.currency,
            ]

    writer = csv.writer(Echo())
    filename = f"orders-{timezone.localdate():%Y-%m-%d}.csv"
    return StreamingHttpResponse(
        (writer.writerow(row) for row in rows()),
        content_type="text/csv",
        headers={"Content-Disposition": f'attachment; filename="{filename}"'},
    )

5–11: The whole trick. `csv.writer` calls `write()` on whatever it is given; returning the value instead of buffering it means `writerow()` hands back the formatted line, which the generator can then yield.
20–22: Validation before the response object. Once `StreamingHttpResponse` is returned, the status is already 200 and a later error can only truncate the file.
24–27: `for_user` scopes the export to what this user may see, and `select_related` is not optional here — without it every row inside the generator issues another query, and the export gets slower the longer it runs.
31: `.iterator(chunk_size=2000)` streams from the database rather than materialising the queryset, which is the other half of keeping memory flat. Streaming the response but loading all the rows achieves nothing.
34–37: ISO-8601 for the timestamp so the file is unambiguous across zones, and `str()` on the `Decimal` because `float()` would introduce binary rounding into a money column.

Why this works: Memory stays flat whether the export is a thousand rows or a million, the download begins immediately, and every error that can be reported properly is reported before the first byte is sent.

Streaming the response but not the queryset

Wrong

python

rows = list(Order.objects.all())          # 800,000 rows, all in memory
return StreamingHttpResponse(writer.writerow(r) for r in rows)

Better

python

rows = Order.objects.all().iterator(chunk_size=2000)
return StreamingHttpResponse(writer.writerow(r) for r in rows)

What you see: Memory use is identical to the non-streaming version, and the response still takes a minute to produce its first byte — the streaming machinery is there but does nothing.

Why: The generator is only lazy if what it iterates over is lazy. A `list()` — or a plain queryset loop, which fills the result cache — materialises everything before the first `yield`, so the peak memory and the initial delay are exactly what they were before. `.iterator()` fetches in chunks and does not populate the result cache, which is what makes the laziness reach all the way to the database.

One export of 800,000 orders, two response classes

HttpResponse

+Every row is materialised before the first byte leaves
+Peak memory scales with the export — and multiplies per concurrent user
+The proxy sees a silent connection and may time it out
+The user stares at a blank tab for the whole build
+An error mid-build is at least an honest 500

StreamingHttpResponse

−Each row is written and discarded
−Memory stays roughly flat at any row count
−Bytes flow immediately, so the connection stays visibly alive
−The download starts at once
−But: headers are already sent, so an error truncates instead of 500-ing

The three formats, and how each behaves under streaming

The three formats, and how each behaves under streaming
Format	Streams?	Use it when
CSV	yes — a row is a complete unit	tabular data; the default for a data export
JSON Lines (`.jsonl`)	yes — one object per line	structure matters and the file is large
JSON (one array)	awkward — it is one value	small responses, or an API rather than a file
`.xlsx`	no — the zip directory is written last	a person will genuinely open it in Excel
`.xlsx` via a job	n/a — built offline	the honest way to deliver a large spreadsheet

Together

python

# JSON Lines streams; a single JSON array does not.
def rows():
    for order in queryset.iterator(chunk_size=2000):
        yield json.dumps({"ref": order.reference, "total": str(order.total)}) + "\n"

Which response class to return

Which response class to return
Class	Body is	Reach for it when
`HttpResponse`	a string, fully built	small and bounded — a few hundred rows
`StreamingHttpResponse`	an iterator	generated data of unknown or large size
`FileResponse`	an open file object	a file that already exists on disk or in storage
a redirect to a signed URL	nothing — the CDN serves it	large files: keep your process out of the transfer

Together

python

return FileResponse(open(path, "rb"), as_attachment=True, filename="orders.csv")

Remember: `HttpResponse` accumulates the whole body in memory; `StreamingHttpResponse` takes an iterator and stays flat, and it also keeps the connection visibly alive so a proxy does not time it out. Django's `Echo` class — a `write()` that returns instead of buffering — is what turns `csv.writer` into a generator. Stream the queryset too, with `.iterator()`, or the laziness stops at the database. And remember the trade: headers go out first, so an exception inside the generator truncates the file rather than raising a 500. Do everything fallible before the response object exists.

When streaming runs out

The export becomes a job and a status row, and the file is served by storage.

When streaming is not enough: async exports and signed URLs

coreadvanced

Streaming keeps memory flat, but it still holds a request open for the entire build and still cannot report an error once it has started. Past some size — or for a format like `.xlsx` that cannot stream at all — the export becomes a **job**: the request creates an export record and returns immediately, a worker builds the file into object storage, and the user is handed a **signed URL**, a time-limited link that grants access to that one object without your application serving the bytes.

Think of it as

Think about which of three things is actually scarce. In a streaming export the scarce resource is the request slot: one worker thread is occupied for minutes, and a handful of concurrent exports can consume a pool that the rest of the site needs. In an async export the scarce resource moves to the queue, where it belongs — jobs are meant to be slow, they can be retried, and their failures are visible instead of arriving as a truncated file. The third scarce resource is bandwidth through your process, and that is what signed URLs remove: the file lives in object storage, the storage service serves it directly, and your application never reads those bytes at all. The user-facing shape that follows is a small state machine, and being explicit about it is what makes the feature comprehensible: an export is *requested*, then *building*, then *ready* with a download link, or *failed* with a reason. That row is the thing the UI polls, the thing the email links to, and the thing that lets a user ask for the same report twice without building it twice. Two properties of the link itself are worth getting right. It should expire — an export contains real data, and a link with no expiry is a permanent credential sitting in somebody's browser history and, if the report was emailed, in a mailbox and on a mail server. And the object key should be unguessable, because expiry protects the link but not the object: a predictable path like `exports/1042.csv` invites someone to try `1043`. Generate the key randomly and store it on the export row. Finally, authorisation belongs in two places, not one. Creating the export must check what this user may see, because the file is built from a query; and re-issuing a download link must check it again, because ownership and permissions can change between the build and the download.

python

return JsonResponse({"id": export.id}, status=202)   # not the file

What we're doing: The job half of an async export: build to a temporary file, upload under a random key, and record success or a readable failure.

reporting/tasks.pypython

import tempfile
from uuid import uuid4
from django.core.files.storage import default_storage


@shared_task(bind=True, max_retries=3)
def build_export(self, export_id):
    export = Export.objects.get(pk=export_id)

    if export.status == "ready":
        return                       # at-least-once delivery: do not rebuild

    Export.objects.filter(pk=export_id).update(status="building")

    try:
        # A temp file, not a bytes buffer: 900 MB of rows must not be resident.
        with tempfile.NamedTemporaryFile("w+", suffix=".csv", newline="") as handle:
            writer = csv.writer(handle)
            writer.writerow(HEADERS)
            rows = 0
            for order in export.queryset().iterator(chunk_size=2000):
                writer.writerow(serialise(order))
                rows += 1

            handle.flush()
            handle.seek(0)
            # Random key: expiry protects the link, not a guessable path.
            key = f"exports/{uuid4().hex}.csv"
            default_storage.save(key, File(handle))

    except Exception as exc:
        Export.objects.filter(pk=export_id).update(
            status="failed", error=str(exc)[:500]
        )
        raise                        # re-raise so the job retries and is visible

    Export.objects.filter(pk=export_id).update(
        status="ready", storage_key=key, row_count=rows, completed_at=timezone.now()
    )
    notify_ready.delay(export_id)    # the email links to the export page, not the file

10–11: The idempotency guard. Queues deliver at least once, so this task will occasionally run twice; without the check, the second run rebuilds a finished export and replaces a key the user may already be downloading.
16–17: A temporary *file*, not an in-memory buffer. Streaming to disk keeps the worker's memory flat for the same reason `StreamingHttpResponse` does for a request.
27: A random key rather than `exports/{export.id}.csv`. A signed URL expires but does not conceal the key, so a sequential path invites walking the range.
31–35: The failure is recorded on the row *and* re-raised. Recording it gives the user a status other than "building forever"; re-raising is what lets the job retry and what puts the error in front of you.
39: The notification links to the export page, not to a signed URL. A link in an email outlives the message, and permissions are re-checked when the page issues a fresh one.

Why this works: The request returns immediately, a 900 MB build never occupies memory or a request worker, failures are visible and retryable, and the download link is short-lived and issued only after a fresh permission check.

Emailing the signed URL itself

Wrong

python

send_mail(subject, f"Your export: {signed_url}", ...)
# the credential now lives in a mailbox, a mail server, and any forward of it

Better

python

send_mail(subject, f"Your export is ready: {site_url}/exports/{export.id}", ...)
# the page authenticates, re-checks permission, then signs a short-lived URL

What you see: An export of customer data remains downloadable by anyone holding the email — including after the recipient leaves the company, and after their access was revoked.

Why: A signed URL is a bearer credential: whoever holds it can fetch the object, with no further authentication. Email is a poor place to store credentials — it is retained, searchable, forwarded, and often synced to personal devices. Linking to a page instead keeps authentication in the loop and moves the permission check to download time, which is the only point at which it can reflect the user's *current* access rather than their access when the report was queued.

A 900 MB export, from click to download — with your process out of the transfer

Browser

Django

Queue

Worker

Object storage

1. POST /exports (filters)
2. authorise + validate, create Export(status="requested")the only work done inside the request
3. on_commit → build_export.delay(id)
4. 202 Accepted {id, status}returns in milliseconds
5. deliver job
6. status = "building"; stream rows to a temp filea failure here is a retryable job, not a truncated download
7. upload to exports/<random>.csv
8. status = "ready", store key + row count
9. GET /exports/<id> (poll, or follow an email link)
10. re-check permission, then sign a short-lived URLownership can change between build and download
11. 302 → signed URL
12. GET the object directlythe bytes never pass through Django

Three ways to deliver an export, and what each one costs

Three ways to deliver an export, and what each one costs
Approach	Holds a request for	Errors are
`HttpResponse`	the whole build, at full memory	an honest 500
`StreamingHttpResponse`	the whole build, at flat memory	a silently truncated file
async job + signed URL	milliseconds	a failed row, retryable, with a reason
async job + emailed link	milliseconds	the same — plus the link is now in a mailbox

Together

python

export = Export.objects.create(user=request.user, params=params, status="requested")
transaction.on_commit(lambda: build_export.delay(export.id))
return JsonResponse({"id": export.id, "status": export.status}, status=202)

What a signed URL does and does not protect

What a signed URL does and does not protect
Property	Signed URL	Note
grants access without a login	yes	that is the point — and the risk
expires	yes, at the time you choose	minutes for a link in a page; hours for one in an email
limited to one object	yes	the signature covers the key
survives being forwarded	yes, until expiry	treat the URL itself as the credential
hides an unguessable key	no	randomise the key — do not use the export id
re-checks permissions	no	so re-check them when you issue the link

Together

python

key = f"exports/{uuid4().hex}.csv"     # random, not exports/1042.csv

Remember: Streaming fixes memory but still holds a request worker for the whole build and still cannot report a mid-file error. Past that point, make the export a job and the export a *row*: requested → building → ready/failed, so the UI has something to poll and a failure has somewhere to live. Build to a temporary file, not a buffer — worker memory is usually tighter than web memory, and a deterministic OOM turns retries into a loop. Store the object under a random key, hand out short-lived signed URLs so storage serves the bytes, and link emails to a page rather than to the URL, because a signed URL is a bearer credential.

The other direction

Untrusted input in bulk: validate everything first, then decide what a bad row costs.

Imports: validation, partial failure, and being safe to re-upload

coreadvanced

An import is untrusted input in bulk, so the first decision is what happens when row 4,000 of 10,000 is invalid. There are only two honest answers — reject the whole file, or accept the good rows and report the bad ones — and you have to choose deliberately, because "it depends what the exception does" is how half a file gets imported. The second decision is what happens when the same file is uploaded twice. That is answered by a **stable business key** in the file plus a uniqueness rule in the database, which turns a re-upload into an update rather than a duplicate.

Think of it as

Validate the whole file before writing any of it. That single ordering choice removes most of the difficulty, because it separates "is this file acceptable?" from "apply it", and only the second half needs a transaction. It also gives the user something far more useful than the first error: a complete list of every problem, with row numbers, so they fix the spreadsheet once instead of ten times. Reusing a Django `Form` per row is the cheap way to get there — you already have field types, coercion, `cleaned_data` and per-field error messages, and reinventing that per import is how imports end up accepting `"12/01/2026"` as a date without anyone knowing which month it meant. On the all-or-nothing question, pick by what the file *means*. A file that represents one indivisible change — a payroll run, a price list where a half-applied update is incoherent — is all-or-nothing, and one `atomic()` around the apply phase gives you that. A file that is a bag of independent records — new contacts, stock counts — is better applied per valid row, with a report of the rejects, because failing 9,998 good rows over two typos wastes everyone's time. What is not acceptable is the accidental middle: a loop with no transaction that stops at the first exception, leaving a prefix applied and no record of where it stopped. Idempotency is the last piece, and it is what makes the inevitable "did that upload work?" re-send harmless. It needs a key that comes from the data rather than from the file — an SKU, an invoice number, an external id — because row position changes when someone sorts the spreadsheet. Enforce it with a `UniqueConstraint` so the database is the thing guaranteeing it, then use an upsert so a second import updates rather than duplicates. Recording a hash of the file itself is a useful second layer: it lets you recognise a byte-identical re-upload instantly and tell the user "this file was already processed" rather than silently doing nothing.

python

Product.objects.bulk_create(objs, update_conflicts=True,
                            unique_fields=["tenant", "sku"], update_fields=["price"])

What we're doing: Import a product price list: validate the whole file first, apply valid rows as batched upserts, and return a report a person can act on.

catalogue/imports.pypython

class ProductRowForm(forms.Form):
    """One row of the CSV. Django's field types do the coercion and the
    error messages, so the import inherits both instead of reinventing them."""

    sku = forms.CharField(max_length=32)
    name = forms.CharField(max_length=200)
    price = forms.DecimalField(max_digits=9, decimal_places=2, min_value=0)
    active = forms.BooleanField(required=False)


def import_products(tenant, file_bytes, all_or_nothing=False):
    digest = hashlib.sha256(file_bytes).hexdigest()
    if ImportRun.objects.filter(tenant=tenant, digest=digest, status="applied").exists():
        return Report(applied=0, errors=[], note="This exact file was already imported.")

    # ---- Pass 1: validate everything. Nothing is written in this pass. ----
    valid, errors = [], []
    reader = csv.DictReader(io.StringIO(file_bytes.decode("utf-8-sig")))
    for number, raw in enumerate(reader, start=2):        # row 1 is the header
        form = ProductRowForm(raw)
        if form.is_valid():
            valid.append(form.cleaned_data)
        else:
            for field, messages in form.errors.items():
                errors.append(f"row {number}: {field}: {'; '.join(messages)}")

    if errors and all_or_nothing:
        return Report(applied=0, errors=errors)          # nothing written at all

    # ---- Pass 2: apply. Batched, and idempotent on (tenant, sku). ----
    applied = 0
    for chunk in batched(valid, 1000):
        objects = [
            Product(tenant=tenant, sku=row["sku"], name=row["name"],
                    price=row["price"], active=row["active"])
            for row in chunk
        ]
        with transaction.atomic():
            Product.objects.bulk_create(
                objects,
                update_conflicts=True,
                unique_fields=["tenant", "sku"],         # the UniqueConstraint
                update_fields=["name", "price", "active"],
            )
        applied += len(objects)

    ImportRun.objects.create(tenant=tenant, digest=digest,
                             status="applied", rows=applied)
    return Report(applied=applied, errors=errors)

12–14: The file hash recognises a byte-identical re-upload — the "did that work?" second attempt — and answers it honestly instead of doing the work again.
19–25: Every row is validated and every error collected. Stopping at the first one means the user fixes a typo, re-uploads, and discovers the next typo: ten round trips instead of one.
18: `utf-8-sig` strips the byte-order mark Excel writes. Without it the first column name arrives as `\ufeffsku`, every row fails on a missing field, and the file looks fine in a text editor.
27–28: The policy decision is explicit and taken before anything is written — not implied by where an exception happens to be raised.
36–43: The upsert. `update_conflicts` with `unique_fields` matching the `UniqueConstraint` makes a re-import update the existing product rather than raising or duplicating — idempotency enforced by the database, not by a racy `.exists()` check.

Why this works: The user gets every error at once with row numbers, a re-upload updates instead of duplicating, memory stays flat over a large file, and the all-or-nothing case really does write nothing.

Checking for an existing row instead of upserting

Wrong

python

for row in rows:
    if not Product.objects.filter(tenant=t, sku=row["sku"]).exists():
        Product.objects.create(tenant=t, **row)     # races a concurrent import
# two uploads at once → duplicates, or IntegrityError halfway through

Better

python

Product.objects.bulk_create(objs, update_conflicts=True,
                            unique_fields=["tenant", "sku"],
                            update_fields=["name", "price"])

What you see: Duplicate products appear when two people import at the same time, or an `IntegrityError` aborts an import that had already applied several thousand rows.

Why: A check-then-write is two statements with a gap between them, and anything can happen in that gap — including the same check succeeding in another connection. The database is the only place that can make uniqueness true, which is what the `UniqueConstraint` is for; an upsert then resolves the conflict atomically in a single statement. It is also one round trip per batch instead of two per row, so the correct version is the faster one.

Two passes over one file — and why nothing is written during the first

Upload: products.csv, 10,000 rows

Hash the bytes

seen before? tell the user, do not re-apply

Pass 1 — validate every row

a Form per row; nothing is written

2 rows invalid

row 4,000: price "N/A" · row 7,412: sku blank

Which policy?

decided up front, not by what the exception does

All-or-nothing: reject the file

zero rows written; the user gets both errors

Pass 2 — apply the 9,998 valid rows

batched upserts inside atomic()

Upsert on (tenant, sku)

UniqueConstraint decides insert vs update

Report: 9,998 applied, 2 rejected

with row numbers and reasons

Two honest failure policies, and the one to avoid

Two honest failure policies, and the one to avoid
Policy	Right when	Implementation
all-or-nothing	the file is one indivisible change	validate all → one `atomic()` around the apply
partial, with a report	rows are independent records	validate all → apply valid rows → return the rejects
(avoid) stop at first error	never — it is not a decision	a prefix is applied and nobody knows where it stopped

Together

python

if policy == "all_or_nothing" and errors:
    return Report(applied=0, errors=errors)      # nothing was written

Where each guarantee actually comes from

Where each guarantee actually comes from
Guarantee	Mechanism	Why not elsewhere
field types are right	a `Form` per row	hand-rolled parsing accepts ambiguous dates silently
no duplicates on re-upload	`UniqueConstraint` + upsert	a `.exists()` check races two concurrent uploads
a batch is all-or-nothing	`transaction.atomic()`	a loop without one leaves a prefix applied
memory stays flat	chunked reads + `bulk_create`	a 200 MB CSV read into a list is 200 MB resident
the same file is recognised	a stored hash of the bytes	filenames are not identity

Together

python

class Meta:
    constraints = [
        UniqueConstraint(fields=["tenant", "sku"], name="uniq_product_sku_per_tenant"),
    ]

Remember: Validate the entire file before writing any of it, with a `Form` per row so you inherit coercion and messages — and report every error with its row number, not just the first. Choose all-or-nothing or per-row explicitly; the accidental middle leaves a prefix applied and nobody knows where. Idempotency comes from a business key in the *data* (SKU, invoice number), enforced by a `UniqueConstraint` and applied with an upsert — never a check-then-create, which races. Read in chunks and batch the writes, because an upload's size is the user's choice, not yours.