TheLazy.Dev
Portfolio GitHub

Blog

django |python

Pagination made easy with Django Rest Framework

sankalpjonna

CH S Sankalp jonna

Posted on January 10, 2021

Pagination made easy with Django Rest Framework

While building any application, the need for fetching a small subset of a large result-set derived from querying a database table is a common occurrence. 

In most cases a user would want quick access to the most recent entries in the result-set, while the older entries are only fetched when the user scrolls up/down or selects the next page of the table. 

Pagination is therefore core to any application but there are several ways to implement it using Django. You could go with the Paginator class that comes out of the box if all you need is the bare minimum, but if you are looking for a more feature rich solution, Django Rest Framework is the way to go.

How pagination should work

Any type of pagination can be implemented by following a simple API response pattern. An API must return these 4 details.

  1. A count of the number of entries in the complete result-set.
  2. A list of entries which represents one page of the result-set.
  3. A URL that can be used to fetch the next page, if the next page exists.
  4. A URL that can be used to fetch the previous page if the previous page exists.

Previously I demonstrated how CRUD applications can be built using DRF with an apple notes like application as a reference and I believe the same example can be used to demonstrate pagination.

Let's say that there are 10 notes in the database, but I only want to look at the latest 2 notes at a time, the API response for the note listing API should look like this

{    
    "count": 10,
    "next": "http://localhost:8000/note/all?p=2",
    "previous": null,
    "results": [
        {
            "id": 10,
            "title": "Note 10",
            "content": "content 10",
            "last_udpated_on": "2021-01-09T01:44:33.645706Z",
            "is_active": true,
            "created": "2021-01-09T01:44:33.645745Z"
        },
        {
            "id": 9,
            "title": "Note 9",
            "content": "content 9",
            "last_udpated_on": "2021-01-09T01:44:29.487257Z",
            "is_active": true,
            "created": "2021-01-09T01:44:29.487295Z"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

This can be achieved using DRF with 2 lines of code.

Paginating an API response

The quickest way is to set up pagination globally and apply it on all the APIs where a query set is returned as a response.

First make sure you have Django Rest Framework installed in your application, then add these lines of code to settings.py

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': 2,
}
Enter fullscreen mode Exit fullscreen mode

That's it, we're done! You will now notice that all the APIs that return a query set as a response have automatically become paginated. If this solves your problem then great, but if you want to dig deeper into the rich solutions offered by DRF, please keep reading.

Overriding pagination for each API view

If you would like to override individual views with their own pagination settings, this can be done by writing a custom paginator which inherits from any of the existing paginators that DRF provides.

You can then include a pagination_class in your API view as shown below.

from django.shortcuts import render, get_object_or_404

from rest_framework.viewsets import ModelViewSet
from .models import Note
from .serializers import NoteSerializer
from .paginations import CustomPagination
# Create your views here.


class NoteViewSet(ModelViewSet):
    serializer_class = NoteSerializer
    pagination_class = CustomPagination

    def get_object(self):
        return get_object_or_404(Note, id=self.request.query_params.get("id"))

    def get_queryset(self):
        return Note.objects.filter(is_active=True).order_by('-last_udpated_on')

    def perform_destroy(self, instance):
        instance.is_active = False
        instance.save()
Enter fullscreen mode Exit fullscreen mode

To create a custom pagination class, first create a paginations.py file in your app folder and create a class that inherits from an existing DRF paginator.

from rest_framework import pagination

class CustomPagination(pagination.PageNumberPagination):
    page_size = 2
    page_size_query_param = 'page_size'
    max_page_size = 50
    page_query_param = 'p'
Enter fullscreen mode Exit fullscreen mode

The parameters given in this custom paginator class will enable you to provide a page_size query parameter to determine the size of each page. If this parameter exists, it will override the default page size of 2. The page number itself would be indicated using the query parameter p. 

Therefore, calling the URL http://localhost:8000/note/all?p=2&page_size=3 will now result in this response:

{
    "count": 10,
    "next": "http://localhost:8000/note/all?p=3&page_size=3",
    "previous": "http://localhost:8000/note/all?page_size=3",
    "results": [
        {
            "id": 7,
            "title": "Note 7",
            "content": "content 7",
            "last_udpated_on": "2021-01-09T01:44:21.560544Z",
            "is_active": true,
            "created": "2021-01-09T01:44:21.560582Z"
        },
        {
            "id": 6,
            "title": "Note 6",
            "content": "content 6",
            "last_udpated_on": "2021-01-09T01:44:17.883444Z",
            "is_active": true,
            "created": "2021-01-09T01:44:17.883483Z"
        },
        {
            "id": 5,
            "title": "Note 5",
            "content": "content 5",
            "last_udpated_on": "2021-01-09T01:44:13.344068Z",
            "is_active": true,
            "created": "2021-01-09T01:44:13.344107Z"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

This is because of setting a page size of 3 and fetching the second page in the result-set.

Types of paginations offered by DRF

DRF offers 3 types of paginations that you can choose from based on your use case. You can also set any of these types as the default pagination class in settings.py

  • Page number pagination
  • Limit and offset pagination
  • Cursor pagination

Page number pagination

This type of pagination is to used to fetch any arbitrary page from the result-set. How it should be used has already been demonstrated in the above example.

Limit and offset pagination

Simulates a database query where you retrieve part of a result-set from a table by providing a limit and an offset. Limit will constrain the maximum rows returned in the query while offset represents the starting position of the rows with reference to the complete size of the result-set.

from rest_framework import pagination

class CustomPagination(pagination.LimitOffsetPagination):
    default_limit = 2
    limit_query_param = 'l'
    offset_query_param = 'o'
    max_limit = 50
Enter fullscreen mode Exit fullscreen mode

The limit_query_param and offset_query_param will indicate the names of the query params to be used for limit and offset. You can set a default_limit if no query parameter is provided and a max_limit to restrict the max number of rows returned in a page.

If this paginator class is used, the response returned by the URL http://localhost:8000/note/all?l=3&o=3 would be something like this.

{
    "count": 10,
    "next": "http://localhost:8000/note/all?l=3&o=6",
    "previous": "http://localhost:8000/note/all?l=3",
    "results": [
        {
            "id": 7,
            "title": "Note 7",
            "content": "content 7",
            "last_udpated_on": "2021-01-09T01:44:21.560544Z",
            "is_active": true,
            "created": "2021-01-09T01:44:21.560582Z"
        },
        {
            "id": 6,
            "title": "Note 6",
            "content": "content 6",
            "last_udpated_on": "2021-01-09T01:44:17.883444Z",
            "is_active": true,
            "created": "2021-01-09T01:44:17.883483Z"
        },
        {
            "id": 5,
            "title": "Note 5",
            "content": "content 5",
            "last_udpated_on": "2021-01-09T01:44:13.344068Z",
            "is_active": true,
            "created": "2021-01-09T01:44:13.344107Z"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

This is because we have specified the limit as 3 which indicates that the page size is 3 and an offset of 3 means that the starting point of this page should be after the first 3 rows in the result-set.

Cursor pagination

Gives you a database cursor of the exact page you are looking at and therefore it becomes very efficient when querying a large dataset, but the limitation is that the ordering cannot be changed and must be unique.

To order the results, one must use a field that is set once during creation and is relatively unique. When this type of pagination is used, we cannot fetch an arbitrary page and can only move forward or backwards in a result-set.

Also, this pagination will not return a count of the entries because there is no query being run on the entire result-set.

from rest_framework import pagination
from rest_framework.response import Response
from collections import OrderedDict

class CustomPagination(pagination.CursorPagination):
    page_size = 2
    cursor_query_param = 'c'
    ordering = '-id'
Enter fullscreen mode Exit fullscreen mode

custom_query_param indicates what query parameter to use to represent a cursor and ordering indicates the field based on which pagination will be applied. Since this field has to have a fixed order and the values must be unique, I have chosen the primary key in descending as the ordering. 

This is what the response returned by this type of pagination looks like:

{
    "next": "http://localhost:8000/note/all?c=cD05",
    "previous": null,
    "results": [
        {
            "id": 10,
            "title": "Note 10",
            "content": "content 10",
            "last_udpated_on": "2021-01-09T01:44:33.645706Z",
            "is_active": true,
            "created": "2021-01-09T01:44:33.645745Z"
        },
        {
            "id": 9,
            "title": "Note 9",
            "content": "content 9",
            "last_udpated_on": "2021-01-09T01:44:29.487257Z",
            "is_active": true,
            "created": "2021-01-09T01:44:29.487295Z"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

Clicking on the next URL returns

{
    "next": "http://localhost:8000/note/all?c=cD03",
    "previous": "http://localhost:8000/note/all?c=cj0xJnA9OA%3D%3D",
    "results": [
        {
            "id": 8,
            "title": "Note 8",
            "content": "content 8",
            "last_udpated_on": "2021-01-09T01:44:25.712231Z",
            "is_active": true,
            "created": "2021-01-09T01:44:25.712268Z"
        },
        {
            "id": 7,
            "title": "Note 7",
            "content": "content 7",
            "last_udpated_on": "2021-01-09T01:44:21.560544Z",
            "is_active": true,
            "created": "2021-01-09T01:44:21.560582Z"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

Custom pagination

If you prefer to have the count, previous link and next link in the HTTP headers while the data only contains the list of results, you can make these customisations by overriding the get_paginated_response method in the pagination class that you inherit from any of the above 3 mentioned types of paginations. 

from rest_framework import pagination
from rest_framework.response import Response
from collections import OrderedDict

class CustomPagination(pagination.PageNumberPagination):
    page_size = 2
    page_size_query_param = 'page_size'
    max_page_size = 50
    page_query_param = 'p'

    def get_paginated_response(self, data):
        response = Response(data)
        response['count'] = self.page.paginator.count
        response['next'] = self.get_next_link()
        response['previous'] = self.get_previous_link()
        return response
Enter fullscreen mode Exit fullscreen mode

You will now see the links in headers and only the actual data in the response

HTTP 200 OK

Allow: GET, HEAD, OPTIONS

Content-Type: application/json

Vary: Accept

count: 10

next: http://localhost:8000/note/all?p=2

previous: None


[

    {

        "id": 10,

        "title": "Note 10",

        "content": "content 10",

        "last_udpated_on": "2021-01-09T01:44:33.645706Z",

        "is_active": true,

        "created": "2021-01-09T01:44:33.645745Z"

    },

    {

        "id": 9,

        "title": "Note 9",

        "content": "content 9",

        "last_udpated_on": "2021-01-09T01:44:29.487257Z",

        "is_active": true,

        "created": "2021-01-09T01:44:29.487295Z"

    }

]
Enter fullscreen mode Exit fullscreen mode




Closing notes

It takes a very low amount of code to paginate any list API using Django Rest Framework and therefore I would highly recommend using it over the default paginator class that Django provides out of the box.

DRF itself inherits from the default paginator class so you can save yourself some time. 

Originally posted on my blog

💖 💪 🙅 🚩
sankalpjonna
CH S Sankalp jonna

Posted on January 10, 2021

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Custom Django Python Migration
django Custom Django Python Migration

November 28, 2024

PWA and Django #3: Online and offline resources in a PWA - Developing Progressive Web Applications with Django
django PWA and Django #3: Online and offline resources in a PWA - Developing Progressive Web Applications with Django

November 27, 2024

I built a Learning Management System with Django
webdev I built a Learning Management System with Django

November 26, 2024

Building a RESTful API from Scratch 🚀
undefined Building a RESTful API from Scratch 🚀

November 25, 2024

Django accounts management app (3), forgot password and account details
django Django accounts management app (3), forgot password and account details

November 25, 2024