Filter concepts by levelShowing all levels.

Django · Section 50

DRF Pagination

Level
advanced
Read
28 min
Concepts
3

Pagination cuts a list response into pages, and the three shipped strategies differ in one thing: how a page is addressed. `PageNumberPagination` (`?page=3`) and `LimitOffsetPagination` (`?limit=50&offset=100`) both compile to `LIMIT … OFFSET …`, and `OFFSET n` is not a seek — the database produces and discards n rows first, so cost grows with depth. Because an offset names a position in a moving result set, an insert between two requests shifts everything after it, letting a row appear on two pages or on none. Both classes wrap results in `{count, next, previous, results}`, where `count` is a separate `SELECT COUNT(*)` run on every request. `CursorPagination` replaces the position with a value: the cursor encodes the last row's ordering value, so the next page is a `WHERE` comparison an index serves in constant time, and concurrent writes cannot shift it. That requires an ordering field which is unchanging, non-nullable, not a float, indexed, and unique — in practice a compound ordering ending in the primary key — and costs you `count` and the ability to jump to an arbitrary page. Custom classes are mostly three attributes: `page_size` for the default, `page_size_query_param` to let clients change it, and `max_page_size` as the ceiling, which is inert without the second and essential with it. Overriding `get_paginated_response()` changes the envelope; overriding `get_paginated_response_schema()` alongside it is what keeps the generated OpenAPI document true.

What is true here

  1. OFFSET n produces and discards n rows, so deep pages are slow while shallow pages on the same endpoint are instant.
  2. An offset is a position in a moving set — concurrent inserts cause duplicated and skipped rows across page boundaries.
  3. A cursor is a value, giving flat cost and consistency at the price of count and random page access.
  4. Cursor ordering must be unchanging, non-null, non-float, indexed, and unique — end it with the primary key.
  5. DEFAULT_PAGINATION_CLASS without PAGE_SIZE leaves pagination inert, and page_size_query_param without max_page_size is unbounded.

What you will be able to do

  • Predict the database cost of a paginated request from its depth and page size
  • Explain when cursor pagination is the right answer, and what capability it removes
  • Choose an ordering that keeps pages stable under concurrent writes
  • Write a custom pagination class with a bounded, client-settable page size
  • Change the response envelope without silently invalidating the generated schema
Addressing a page by position, or by value

Offset — "how many rows in?"

  • +Cost grows with depth: OFFSET 44950 produces and discards 44,950 rows.
  • +count costs a separate SELECT COUNT(*) on every request.
  • +A concurrent insert shifts every later position — rows repeat or vanish between pages.
  • +Can jump to any page, which is what a numbered-page UI needs.
  • +Works with any deterministic ordering.

Cursor — "after which row?"

  • −Constant cost at any depth — an index seek, not a scan-and-discard.
  • −No count field, and therefore no COUNT(*) per request.
  • −A concurrent insert changes nothing: the boundary is a value, not a position.
  • −Cannot jump to page N — forward and backward only.
  • −Ordering must be unchanging, non-null, indexed, and unique.
  • Offset — "how many rows in?"
    • Cost grows with depth: OFFSET 44950 produces and discards 44,950 rows.
    • count costs a separate SELECT COUNT(*) on every request.
    • A concurrent insert shifts every later position — rows repeat or vanish between pages.
    • Can jump to any page, which is what a numbered-page UI needs.
    • Works with any deterministic ordering.
  • Cursor — "after which row?"
    • Constant cost at any depth — an index seek, not a scan-and-discard.
    • No count field, and therefore no COUNT(*) per request.
    • A concurrent insert changes nothing: the boundary is a value, not a position.
    • Cannot jump to page N — forward and backward only.
    • Ordering must be unchanging, non-null, indexed, and unique.

Offset-based pagination

Page numbers, limit/offset, the shared metadata envelope, and what OFFSET really costs.

PageNumberPagination, LimitOffsetPagination, and the metadata

coreintermediate

Pagination cuts a list response into pages. `PageNumberPagination` numbers them — `?page=3` — and `LimitOffsetPagination` takes a window — `?limit=50&offset=100`. Both wrap the results in the same metadata envelope: `count` (how many rows matched in total), `next` and `previous` (full URLs, or `null` at the ends), and `results` (the rows themselves). Both are opt-in: `DEFAULT_PAGINATION_CLASS` and `PAGE_SIZE` are both `None` out of the box, so an unpaginated list endpoint returns every row it matched — which is fine on a table of twenty and a serious problem on a table of two million.

Think of it as

Both of these classes compile to `LIMIT … OFFSET …`, and understanding what `OFFSET` actually does explains almost everything about their behaviour. `OFFSET 44950` is not a seek to row 44,951 — the database must produce the first 44,950 rows in sorted order and throw them away before it can return anything. So the cost of a page grows with how deep the page is, and page 900 of a large table is genuinely a slow query while page 1 of the same endpoint is instant. The second consequence is subtler: an offset is a position in a *result ordering*, not a handle on a row. If a row is inserted or deleted between two requests, every position after it shifts, so page 2 can repeat an item page 1 already returned, or skip one entirely. Neither problem is a bug in DRF; they are what offset pagination is. `count` deserves the same scrutiny — it is a separate `SELECT COUNT(*)` over the filtered set on every single request, which on a large table can cost more than fetching the page did. Those three facts — depth cost, drift, and the count query — are the reasons cursor pagination exists.

python
class OrderPagination(PageNumberPagination):
    page_size = 50
    page_size_query_param = "page_size"
    max_page_size = 200

What we're doing: Paginate an orders endpoint, let clients ask for a bigger page within a cap, and drop the count query where it is not worth paying for.

orders/pagination.py + orders/views.pypython
class OrderPagination(PageNumberPagination):
    page_size = 50
    page_size_query_param = "page_size"
    max_page_size = 200


class FastCountPagination(OrderPagination):
    def get_paginated_response(self, data):
        # No count query: the client gets links, not a total.
        return Response({
            "next": self.get_next_link(),
            "previous": self.get_previous_link(),
            "results": data,
        })


class OrderViewSet(viewsets.ModelViewSet):
    serializer_class = OrderSerializer
    pagination_class = OrderPagination
    ordering = ["-created_at", "-id"]
3
Without `page_size_query_param`, `?page_size=100` is ignored silently — the client asks and nothing changes.
4
`max_page_size` is the only thing standing between an exposed `page_size` parameter and `?page_size=100000`. Naming the first without the second is the common mistake.
8–14
Overriding the envelope to drop `count` removes a full `SELECT COUNT(*)` per request. Worth doing when the client only walks forward and the total is decoration.
20
A deterministic sort with `-id` as the tiebreaker. Without it, two orders sharing a `created_at` can swap places between requests, and a row lands on both page 1 and page 2.

Why this works: The page size cap and the tiebreaking sort are the two lines that make offset pagination behave predictably; the count override is the one to reach for when profiling shows the total costs more than the page.

Setting `DEFAULT_PAGINATION_CLASS` and assuming the API is paginated

Wrong

python
REST_FRAMEWORK = {
    "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
}
# GET /orders/  ->  a JSON array of all 1.2M orders. No envelope, no error.

Better

python
REST_FRAMEWORK = {
    "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
    "PAGE_SIZE": 50,
}

What you see: Endpoints return a bare list rather than the `{"count", "next", "previous", "results"}` envelope, and one request eventually exhausts the worker's memory. In development, with a hundred seeded rows, everything looks correct.

Why: `PageNumberPagination.page_size` defaults to the `PAGE_SIZE` setting, which is itself `None`. A page size of `None` means "do not paginate", so the class is installed and inert. Nothing warns about this, because an unpaginated response is a legitimate configuration — the failure only appears at a data volume development never sees.

What each page actually asks the database for

Page 1 — cheap

The database reads the first 50 rows in order and stops. An index on the ordering column serves this directly.

Page 2 — still cheap

Fifty rows are produced and discarded before the fifty that are returned. Barely measurable at this depth.

Page 900 — expensive

Forty-four thousand nine hundred and fifty rows are produced, sorted, and thrown away to return fifty. Cost grows linearly with depth.

And the response shape never changes

The client sees the same envelope whether the page cost 8ms or 12s — which is exactly why deep pagination goes unnoticed until it is a production incident.

  1. Page 1 — cheap — The database reads the first 50 rows in order and stops. An index on the ordering column serves this directly.
  2. Page 2 — still cheap — Fifty rows are produced and discarded before the fifty that are returned. Barely measurable at this depth.
  3. Page 900 — expensive — Forty-four thousand nine hundred and fifty rows are produced, sorted, and thrown away to return fifty. Cost grows linearly with depth.
  4. And the response shape never changes — The client sees the same envelope whether the page cost 8ms or 12s — which is exactly why deep pagination goes unnoticed until it is a production incident.

The two offset-based classes

The two offset-based classes
Property`PageNumberPagination``LimitOffsetPagination`
Parameters`?page=3``?limit=50&offset=100`
Page size attribute`page_size``default_limit`
Client-settable size`page_size_query_param` (off by default)`limit_query_param` (`limit`, on by default)
Upper bound`max_page_size``max_limit` (`None` by default)
Out of range404empty `results`, `count` still correct
Best forUIs with numbered pagesclients that want an arbitrary window

Together

python
REST_FRAMEWORK = {
    "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
    "PAGE_SIZE": 50,          # without this, the class above does nothing
}

Remember: Both offset classes compile to `LIMIT … OFFSET …`, and `OFFSET n` produces and discards n rows — so page cost grows with depth, and an insert between requests shifts every later position, letting a row appear twice or not at all. `count` is a separate `COUNT(*)` on every request. Setting `DEFAULT_PAGINATION_CLASS` without `PAGE_SIZE` leaves pagination inert, and exposing `page_size_query_param` without `max_page_size` hands the caller your memory limit.

See also: cursor pagination and stable ordering · custom pagination and page size limits · query validation and complexity controls

Advertisement

Cursor pagination and stable ordering

Addressing a page by value, the constraints that makes on the ordering field, and what you give up.

Cursor pagination, stable ordering, and large datasets

coreadvanced

Cursor pagination replaces "give me page 900" with "give me what comes after this row". The `next` link carries an opaque cursor encoding the ordering value of the last row returned, and the next query becomes `WHERE created_at < <cursor> ORDER BY created_at DESC LIMIT 50` — which an index serves in the same time whether it is the first page or the ten-thousandth. It also fixes drift: because the boundary is a value rather than a position, rows inserted while a client is paging do not shift anything, so no item is returned twice or skipped. The price is that you cannot jump to an arbitrary page, and the ordering field has to satisfy real constraints — unchanging, effectively unique, non-nullable, and indexed.

Think of it as

Offset pagination asks "how many rows in?" and cursor pagination asks "after which row?". That single change is what makes both of offset's problems disappear at once, because both problems come from positions being relative to a set that keeps moving. A value-based boundary is absolute: `created_at < 2026-09-04T10:00:00Z` means the same thing regardless of what was inserted a second ago. This is also why the constraints on the ordering field are not fussiness — they are what makes the boundary well-defined. A field that changes value breaks the boundary (a row can move from behind the cursor to in front of it and be served twice). A field with duplicates makes the boundary ambiguous, so rows sharing a value can be dropped at a page edge — which is why the practical form is always a compound ordering ending in the primary key. A nullable field has no defined position relative to the cursor. And without an index, the `WHERE` clause scans, which throws away the entire performance argument. The trade you accept in return is real: no `count`, no page numbers, no jumping. For an activity feed, an audit log, or an export, none of those matter. For a table UI showing "page 4 of 87", they are the whole feature — and that UI is the honest reason to keep offset pagination.

python
class EventPagination(CursorPagination):
    page_size = 100
    ordering = ("-occurred_at", "-id")

What we're doing: Paginate a high-volume event log so page cost stays flat and a concurrent insert never duplicates a row.

events/models.py + events/pagination.py + events/views.pypython
class Event(models.Model):
    occurred_at = models.DateTimeField(default=timezone.now, editable=False)
    level = models.CharField(max_length=16, choices=Level.choices)

    class Meta:
        indexes = [models.Index(fields=["-occurred_at", "-id"])]
        ordering = ["-occurred_at", "-id"]


class EventCursorPagination(CursorPagination):
    page_size = 100
    page_size_query_param = "page_size"
    max_page_size = 500
    ordering = ("-occurred_at", "-id")


class EventViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = EventSerializer
    queryset = Event.objects.all()
    pagination_class = EventCursorPagination
    filter_backends = [DjangoFilterBackend]
2
`editable=False` on the ordering field enforces the "unchanging" requirement at the model level, so no serializer or admin form can move a row across an existing cursor.
6
The composite index matches the ordering exactly, including direction. Without it the `WHERE (occurred_at, id) < (…)` comparison scans, and cursor pagination performs no better than offset.
12–13
A client-settable page size needs both attributes: `page_size_query_param` names the parameter, `max_page_size` bounds it. Either one alone does nothing useful.
14
The `-id` tiebreak is what makes the ordering unique. Events sharing a timestamp — common when a batch job writes many at once — would otherwise be dropped at a page boundary.
21
Filtering still runs before pagination, so `?level=error` narrows the set and the cursor walks the filtered ordering. The cursor stays valid only for the same filter parameters.

Why this works: Flat page cost and no duplicates are exactly what a log or feed needs, and the client loses only page numbers and a total — neither of which an "load more" interface uses.

Cursor-paginating on a non-unique timestamp

Wrong

python
class EventCursorPagination(CursorPagination):
    ordering = "-occurred_at"     # a batch import writes 400 events at the same instant

Better

python
class EventCursorPagination(CursorPagination):
    ordering = ("-occurred_at", "-id")

What you see: Rows go missing — not at random, but specifically at page boundaries, and only for timestamps shared by more than one row. A client walking the feed sees 100, 100, 97, 100 items and no error.

Why: The cursor encodes a value, and the next query asks for rows strictly beyond it. When several rows share that value, the boundary cannot distinguish them, so the ones that happened to fall after the cut are skipped on the next page. Adding the primary key as a final ordering key makes the tuple unique, which restores a well-defined boundary. This is why the DRF docs specify "unique, or nearly unique" — and why "nearly" is not good enough in practice.

The same insert, under offset and under cursor pagination
Client
API
Database
Another writer
  1. 1. GET /events/?page=1 (offset)
  2. 2. LIMIT 50 OFFSET 0
  3. 3. rows 1–50 · newest is event #900
  4. 4. INSERT a new eventeverything shifts down one position
  5. 5. GET /events/?page=2
  6. 6. LIMIT 50 OFFSET 50
  7. 7. row 50 repeats — it moved into page 2
  8. 8. GET /events/?cursor=cD0yMDI2… (cursor)
  9. 9. WHERE (occurred_at, id) < (cursor) LIMIT 50an index seek — same cost at any depth
  10. 10. the next 50, no repeat and no gap
  1. Client → API: GET /events/?page=1 (offset)
  2. API → Database: LIMIT 50 OFFSET 0
  3. Database → Client: rows 1–50 · newest is event #900
  4. Another writer → Database: INSERT a new event (everything shifts down one position)
  5. Client → API: GET /events/?page=2
  6. API → Database: LIMIT 50 OFFSET 50
  7. Database → Client: row 50 repeats — it moved into page 2
  8. Client → API: GET /events/?cursor=cD0yMDI2… (cursor)
  9. API → Database: WHERE (occurred_at, id) < (cursor) LIMIT 50 (an index seek — same cost at any depth)
  10. Database → Client: the next 50, no repeat and no gap

Offset versus cursor, on the properties that actually differ

Offset versus cursor, on the properties that actually differ
PropertyPage number / limit-offsetCursor
Cost of the 1000th pagegrows linearly with depththe same as the first page
Insert during pagingshifts positions — duplicates or skipsno effect — the boundary is a value
`count` in the responseyes, and it costs a `COUNT(*)`no
Jump to page Nyesno
Ordering constraintsany deterministic sortunchanging, unique, non-null, indexed
Fitsadmin tables, numbered-page UIsfeeds, logs, exports, sync APIs

Together

python
class EventPagination(CursorPagination):
    page_size = 100
    ordering = ("-occurred_at", "-id")   # unique because of the pk tiebreak

Remember: Cursor pagination trades random access for constant cost and consistency: the boundary is a value, not a position, so depth is free and a concurrent insert cannot duplicate or skip a row. The ordering field must be unchanging, non-nullable, not a float, indexed, and unique — which in practice means ending the ordering with the primary key. You give up `count` and page numbers, so choose per endpoint: feeds, logs and exports get cursors; numbered-page tables keep offsets.

See also: page number and limit offset pagination · custom pagination and page size limits · indexes · meta ordering

Advertisement

Custom classes and size limits

The four attributes that control page size, and changing the envelope without breaking the schema.

Custom pagination classes and page-size limits

standardintermediate

A custom pagination class is usually a three-line subclass: set `page_size`, name `page_size_query_param` if clients may change it, and set `max_page_size` to bound what they can ask for. Beyond that, overriding `get_paginated_response(data)` changes the envelope — to nest the metadata, to add a `total_pages` field a front-end needs, or to drop `count` and its `COUNT(*)`. Overriding `get_paginated_response_schema()` alongside it keeps the generated OpenAPI document honest. Set the class per view with `pagination_class`, or globally with `DEFAULT_PAGINATION_CLASS`; set `pagination_class = None` on a view that genuinely must return everything.

Think of it as

Two limits are in play and they are easy to confuse. `page_size` is the default — what a client gets when it asks for nothing. `max_page_size` is the ceiling — what a client cannot exceed when it asks for something. Neither has any effect on the other, and `max_page_size` does nothing at all unless `page_size_query_param` has made the size caller-controlled in the first place. Pick the ceiling from what one request may cost the server, not from what feels generous: the page is fetched, instantiated as model objects, serialized, and rendered, all held in memory simultaneously, so the ceiling is really a memory budget. As for the envelope, changing it is cheap and the cost is entirely downstream — every client, every SDK, and the OpenAPI schema encode its shape, so a custom envelope is a decision to make once, early, and apply project-wide rather than per endpoint.

python
class StandardPagination(PageNumberPagination):
    page_size = 50
    page_size_query_param = "page_size"
    max_page_size = 200

    def get_paginated_response(self, data):
        return Response({"meta": {...}, "data": data})

What we're doing: Standardise one envelope across the project — metadata nested under `meta`, rows under `data` — without breaking the generated schema.

common/pagination.pypython
class EnvelopePagination(PageNumberPagination):
    page_size = 50
    page_size_query_param = "page_size"
    max_page_size = 200

    def get_paginated_response(self, data):
        return Response({
            "meta": {
                "count": self.page.paginator.count,
                "page": self.page.number,
                "total_pages": self.page.paginator.num_pages,
                "page_size": self.get_page_size(self.request),
            },
            "links": {"next": self.get_next_link(), "previous": self.get_previous_link()},
            "data": data,
        })

    def get_paginated_response_schema(self, schema):
        return {
            "type": "object",
            "properties": {
                "meta": {
                    "type": "object",
                    "properties": {
                        "count": {"type": "integer"},
                        "page": {"type": "integer"},
                        "total_pages": {"type": "integer"},
                        "page_size": {"type": "integer"},
                    },
                },
                "links": {
                    "type": "object",
                    "properties": {
                        "next": {"type": "string", "nullable": True},
                        "previous": {"type": "string", "nullable": True},
                    },
                },
                "data": schema,
            },
        }
7–13
`num_pages` and `count` both come from the paginator that has already run, so this richer envelope costs no extra query.
12
`get_page_size(self.request)` returns the size actually applied after the `max_page_size` clamp — echoing back what the client asked for would be a lie when it asked for more.
18
Without this override the OpenAPI document still describes `{count, next, previous, results}`, so generated clients would be wrong in a way no test catches.
37
`schema` is the array schema DRF already built for the rows — pass it through rather than rebuilding it, so it stays in step with the serializer.

Why this works: One envelope class in `DEFAULT_PAGINATION_CLASS` means every list endpoint answers the same shape, and overriding the schema method alongside the response method keeps the documentation and the runtime from drifting apart.

Every line of a custom pagination class, and what it decides

class OrderPagination(CursorPagination): page_size = 50 page_size_query_param = "page_size" max_page_size = 200 ordering = "-created_at"

CursorPagination

the strategy — Decides how a page is addressed at all — by cursor here, or by page number / limit-offset. Changing this changes the response envelope and the client contract.

page_size = 50

the default size — What a client that asks for nothing receives. Leave it unset and it falls back to the PAGE_SIZE setting, which is None — meaning no pagination.

page_size_query_param = "page_size"

the opt-in to a client-set size — Until this is named, ?page_size=100 is ignored silently. Naming it hands size control to the caller.

max_page_size = 200

the memory budget — The only bound on the line above. Inert without it, and essential with it — the page is held as rows, model instances, and serialized data at once.

ordering = "-created_at"

the cursor boundary — Cursor-only, and required — it must be indexed, unchanging and effectively unique, or pages skip rows at their edges.

  • Whole: class OrderPagination(CursorPagination): page_size = 50 page_size_query_param = "page_size" max_page_size = 200 ordering = "-created_at"
  • CursorPagination — the strategy: Decides how a page is addressed at all — by cursor here, or by page number / limit-offset. Changing this changes the response envelope and the client contract.
  • page_size = 50 — the default size: What a client that asks for nothing receives. Leave it unset and it falls back to the PAGE_SIZE setting, which is None — meaning no pagination.
  • page_size_query_param = "page_size" — the opt-in to a client-set size: Until this is named, ?page_size=100 is ignored silently. Naming it hands size control to the caller.
  • max_page_size = 200 — the memory budget: The only bound on the line above. Inert without it, and essential with it — the page is held as rows, model instances, and serialized data at once.
  • ordering = "-created_at" — the cursor boundary: Cursor-only, and required — it must be indexed, unchanging and effectively unique, or pages skip rows at their edges.

The four attributes that decide the page-size behaviour

The four attributes that decide the page-size behaviour
AttributeDefaultWhat it does
`page_size``PAGE_SIZE` setting (`None`)the size when the client asks for nothing — `None` means no pagination at all
`page_size_query_param``None`the parameter name clients use; unset means the size is fixed
`max_page_size``None`the ceiling on a requested size; inert unless the parameter above is named
`page_query_param``"page"`the page-number parameter name

Together

python
class StandardPagination(PageNumberPagination):
    page_size = 50
    page_size_query_param = "page_size"
    max_page_size = 200

Remember: `page_size` is the default, `max_page_size` is the ceiling, and the ceiling does nothing until `page_size_query_param` makes the size caller-controlled. Choose the ceiling as a memory budget — the page exists as rows, model instances, and serialized data simultaneously. If you override `get_paginated_response()`, override `get_paginated_response_schema()` in the same class, or your OpenAPI document quietly describes an envelope you no longer return.

See also: page number and limit offset pagination · cursor pagination and stable ordering

Advertisement