How to do caching in Django applications to speed up your website. by Esther Vati | October, 2022

Improve the speed of your application by using different technologies

Unsplash. Photo by Scott Graham on

In the first three parts of this series to improve performance in your Django applications, we focused on database, code optimization, and frontend optimization. In Part 4, we’ll focus on ways to improve the speed of Django applications. We will cover the following:

What is Caching?Importance of CachingTypes of Caching in DjangoHow to Perform Caching in Django Applications with Django-redisView CachingPer site CachingTemplate Fragment Caching

Several factors can contribute to slow applications, including slow API calls and database queries. As applications grow in size, fetching data from a database can become very expensive and slow. A better way to speed things up is to store the most requested data or data that doesn’t change for each user in a cache. It is easy and fast to retrieve the data stored in memory.

Caching is the process by which data received is stored in the cache, so subsequent requests are received from the cache rather than from the source.

Let’s say you have an online bookstore that provides an overview of books and their descriptions; Initially, the application works fine, but as you add more books to the application, the site starts getting stuck and becomes slow.

The database has to run queries to get the book details with each and every user who visits the site. A better way to serve your users would be to cache the data for a certain period of time. During this period, users will be served data from the cache, and the database will be free to perform other tasks.

With good caching, you can free the database from high load. It also frees your application server from overloading and makes it possible to serve websites to more people with the same hardware. It also reduces page load times, which is always good.

Caching also forces websites to request and deliver data with minimum latency. If your application provides multiple API calls for data that does not change, it is more efficient to cache the data so that subsequent calls are received from the cache.

Before you can use caching in your Django application, you must set up the cache best suited to your application needs. There are different levels of caching, based on:

Memory caching Database caching File system caching.

Memory caching stores the cached data in memory. The most efficient type of memory cache supported by Django is Memcached. Memcached is a fast memory-based cache server that can handle high loads of data and hence increases performance in Django applications and, at the same time, reduces database load.

Memcached caching is suitable for dynamic websites and can also be used to share cache across multiple servers. However, it is not permanent storage as it is susceptible to data loss when the server crashes.

It is also not suitable for caching large files.

To set Memcached as your default backend, you need to set it up by adding the following configuration settings.py of your Django application:

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.PyMemcacheCache',
'LOCATION': '127.0.0.1:11211',
}
}

You also need to have Memcached installed on your server. BACKEND key specifies the caching backend, whereas LOCATION Defines where the specified backend is running.

In LOCATION Installation above, localhost is specified as the cache location on port 11211. Memcached supports caching across multiple servers. If you are using multiple servers to cache the data, you specify the host server as shown below:

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.PyMemcacheCache',
'LOCATION': [
'172.19.26.240:11211',
'172.19.26.242:11211',
]
}
}

One thing to note about Memcached is that it does not persist data, so if Memcached is restarted, the cache becomes empty again and needs to be refilled.

Database caching involves using a database to store cached data.

To set up database caching, specify the backend in your cache settings.py file, as shown below:

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
'LOCATION': 'cache_table',
}
}

LOCATION Specifies the name of the database table where the cached data will be stored. to make table cache_table Specified above, run the following command:

python manage.py createcachetable

As the name suggests, this caching involves serializing and storing individual cache values ​​in a file. Unlike other types of caching that require a lot of configuration, file system caching requires that you specify the path to the directory used to store the cache, as follows:

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
'LOCATION': '/var/tmp/django_cache,
}
}

In the above code, the cache is stored in the directory var/tmp/django_cache, The directory must already exist and be writable and readable. Alternatively, the system user should be able to create it.

Local memory caching is the default cache used by Django if no other caching is configured. This gives fast local memory cache capacity but possibly consumes a lot of memory and is, therefore, unsuitable in production. However, local memory caching is suitable for production if you have enough RAM on your server. Add the following CACHE configuration to set up local memory caching in settings.py file.

Local memory will cache the data in the memory of the web server process that is running Django, so there is no need for a separate cache in your server.

Memcached and file system caching have the same speed and memory usage, but with this file system caching, you don’t need to set up a Memcached server because it only uses the memory of the server process in which Django is running.

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
'LOCATION': 'ecom',
}
}

Even though Django supports multiple caching backends, such as memcached and others mentioned above, Redis is the most popular caching backend. According to Django Developer Survey 2021, Redis is used by over 47% of Python developers.

Redis is an in-memory caching backend that allows you to cache data in terms of key-value pairs. We will be using Redis as our caching backend for the rest of this tutorial.

django-redis is a redis cache backend for Django. You need to have Redis installed on your operating system. Follow the Getting Started guide on the Redis website to install Redis Server and make sure it is working as expected.

In this section, we will create a Django application and demonstrate how to implement different levels of caching in a Django application, such as the following:

  • per site caching
  • view caching
  • template caching

The first step is to create a project directory where your files will reside. Next, create a virtual environment in the project folder with the venv library. Virtual environments are required to separate project dependencies from system packages.

create a virtual environment in env folder:

python -m venv env

Activate the virtual environment:

source env/bin/activate

Create a new Django project named django_redis:

django-admin startproject django_redis

Install the required dependencies, namely django and django-redis with pip:

pip install django django-redis

We’re going to build an application that displays product records, and then we’ll query the cache to see different ways to use the cache to retrieve data faster.

Next, create a Django app called ecommerce,

cd django_redis
django admin startapp ecommerce

add app ecommerce In the list of installed apps settings.pyas shown below:

INSTALLED_APPS = [
# …
'ecommerce'
]

Add cache configuration to settings.py file by setting cache BACKENDas shown below:

CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://127.0.0.1:6379/1",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
}
}
}

By default, redis is on port 6379,

In models.pyAdd the following models:

since we have a ImageField Which requires the Pillow library, install the Pillow package with Pillow:

pip install pillow

Do the migration (required for pillows):

python manage.py makemigrations
python manage.py migrate

To add some sample data, we’ll be using the Django admin site. Let’s start by entering two models ecommerce/admin.py file, as shown below.

#ecommerce/admin.pyfrom django.contrib import admin
from .models import Book,Category
# Register your models here.admin.site.register(Book)
admin.site.register(Category)

Next, create a super user that will give us access to add data to the Django admin site:

python manage.py createsuperuser

Next, start the development server:

python manage.py runserver

Navigate to Admin Site https://127.0.0.1:8000and add some data.

In ecommerce/views.pyWrite a function that retrieves all book entries from the database:

Django will automatically search for templates in the app’s template directory. add a template book_list.html In the following directory structure.

ecommerce/ templates/ ecommerce/ book_list.html

Next, add the following code to book_list.htmlwhich renders the data from the view:

Next, map the view to a URL urls.py file, as shown below:

The Django Debug Toolbar is a Django query monitoring tool that allows developers to perform a variety of checks in your Django application by measuring how long it takes to run a query. By installing the Django debug toolbar, we can compare the time it takes to run database queries and retrieve templates if there is no cache and the cache mechanism is introduced.

Install Django Debug Toolbar with pip:

pip install django-debug-toolbar

Add the Django Debug Toolbar to the List of Installed Apps in settings.py,

INSTALLED_APPS = [
# …
'debug_toolbar',
]

Update urls.py To include the URL of the django-debug-toolbar file as follows:

import debug_toolbarurlpatterns = [
path('admin/', admin.site.urls),
path('books', book_list),
path('__debug__/', include(debug_toolbar.urls)),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

add DebugToolbarMiddleware,

MIDDLEWARE = [
# …
'debug_toolbar.middleware.DebugToolbarMiddleware'
# …
]

for MIDDLEWARE is important. You should include the debug toolbar middleware as soon as possible. However, it must come after any other middleware that encodes the content of the response, such as a class:~django.middleware.gzip.GZipMiddleware,

Now, if you navigate http://127.0.0.1:8000/booksYou can see that the view performed a total of 11 SQL queries which took 0.77ms.

This may seem time-consuming, but as the application grows, so do the SQL queries, and with each request, if the data does not change often, there is no point in querying the exact data for each user. .

View caching is a more efficient method of caching as it only caches the results of individual Django views. This is done by decorating the scene cache_page from the decorator django.views.decorator.cache,

cache_page The decorator takes one parameter, the timeout in seconds.

let’s apply cache_page the decorator book_list View and cache data for 15 minutes.

Navigate to the page that displays the list of books, and you should see that no SQL queries were run because the data is now being retrieved from the cache. The view does not touch the database hence freeing up the database to perform other important tasks.

CPU time also decreases from 68.18ms to 7.20ms. This represents about ten times faster for processing the CPU request.

Per-site caching is another way to implement caching in Django applications. Per-site caching caches everything in your Django application. But first, you need to add some middleware configuration to the middleware settings, namely:

django.middleware.cache.UpdateCacheMiddleware And django.middleware.cache.FetchFromCacheMiddleware,

MIDDLEWARE = [
# …
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
# …
]

Both sets of middleware enable site-wide caching in your Django site. UpdateCacheMiddleware is responsible for updating the response cache for each successful response (status 200), whereas FetchCacheMiddleware Responsible for updating the page from the cache.

You also need to add the following settings to it settings.py file.

CACHE_MIDDLEWARE_ALIAS  = ' ' # cache alias
CACHE_MIDDLEWARE_SECONDS = 600 # number of seconds each page should be cached.
CACHE_MIDDLEWARE_KEY_PREFIX = '' # name of site if multiple sites are used

Template caching involves caching parts of Django templates by specifying the cached fragment and cache time.

To cache template fragments, apply {% load cache %} Tags where caching is required. {% load cache %} Will cache the appended content for the specified amount of time. For example, we can implement a cache for our book list template.

In the code above, we apply the cache to the div fragment which displays the list of books.

As you can see from the above screenshot, the database runs only 2 SQL queries and takes 0.44 ms.

In this tutorial, you learned the different types of caching available in Django and how to use the different cache backends in your Django application. You’ve also learned how to view per-site, templates, and caching to speed up the performance of your Django site.

Leave a Comment