Some application admins need to know which user performed what action in the application. We also felt the need of such tracking hence we started developing the access log system for our application.

In this article, we will see how to develop the server access logging app for the Django project.

We will be storing the below information:

• URL/link visited.
• Request method, get, or post.
• Get or Post Data
• Referrer
• The IP address of the visitor
• Session ID

What is an access log:

Why access log:

• Origin of request i.e. referrer
• Location of visitor
• What page or link is visited most
• In case of an audit, which visitor clicked on which page and searched what. etc.

Access Logs:

We strongly recommend to use a virtual environment for python or Django project development.

To log the information, we need to create a model.

Model:

from django.db import models


class AccessLogsModel(models.Model):
    sys_id = models.AutoField(primary_key=True, null=False, blank=True)
    session_key = models.CharField(max_length=1024, null=False, blank=True)
    path = models.CharField(max_length=1024, null=False, blank=True)
    method = models.CharField(max_length=8, null=False, blank=True)
    data = models.TextField(null=True, blank=True)
    ip_address = models.CharField(max_length=45, null=False, blank=True)
    referrer = models.CharField(max_length=512, null=True, blank=True)
    timestamp = models.DateTimeField(null=False, blank=True)

    class Meta:
        app_label = "django_server_access_logs"
        db_table = "access_logs"

Middleware:

from .models import AccessLogsModel
from django.conf import settings
from django.utils import timezone


class AccessLogsMiddleware(object):

    def __init__(self, get_response=None):
        self.get_response = get_response
        # One-time configuration and initialization.

    def __call__(self, request):
        # create session
        if not request.session.session_key:
            request.session.create()

        access_logs_data = dict()

        # get the request path
        access_logs_data["path"] = request.path

        # get the client's IP address
        x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
        access_logs_data["ip_address"] = x_forwarded_for.split(',')[0] if x_forwarded_for else request.META.get('REMOTE_ADDR')
        access_logs_data["method"] = request.method
        access_logs_data["referrer"] = request.META.get('HTTP_REFERER',None)
        access_logs_data["session_key"] = request.session.session_key

        data = dict()
        data["get"] = dict(request.GET.copy())
        data['post'] = dict(request.POST.copy())

        # remove password form post data for security reasons
        keys_to_remove = ["password", "csrfmiddlewaretoken"]
        for key in keys_to_remove:
            data["post"].pop(key, None)

        access_logs_data["data"] = data
        access_logs_data["timestamp"] = timezone.now()

        try:
            AccessLogsModel(**access_logs_data).save()
        except Exception as e:
            pass

        response = self.get_response(request)
        return response

In the file above, we are doing the following things:

1. Since we are using the session key to uniquely identifying the request, we need to create a session if the session key doesn't exist.
2. Get the path i.e. URL which user/visitor visited.
3. Collect, IP address, the request method, referrer URL, and session key.
4. Collect the post and get data and remove sensitive information like a password. You may edit the logic as per your requirement.
5. Store the data with the timestamp in the table.

Settings:

Add your app in installed app's list in settings.py file.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django_server_access_logs'
]

Add your middleware class in the middleware classes list.

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django_server_access_logs.logging_middleware.AccessLogsMiddleware',
]

Make migrations to create the Model table in the database.

Complete code is available on Github.

Future Work:

• Ignore hits on static and media URLs.
• Log user_id  if the user is logged in.

Data Cleanup:

You might want to create a management command and schedule it to delete the data periodically.

create a command and use the below code for cleanup.

from django.core.management.base import BaseCommand, CommandError
import datetime
from django.utils import timezone
from .models import AccessLogsModel


class Command(BaseCommand):

    help = 'Clean the user access logs older than x days'

    def add_arguments(self, parser):
        pass

    def handle(self, *args, **options):
        days_to_keep_data = 7
        now = timezone.now()
        back_date = now - datetime.timedelta(days=days_to_keep_data)
        AccessLogsModel.objects.filter(timestamp__lt=back_date).delete()
        

Must read article for middleware :

How to develop a distributable Django app to block the crawling IP addresses