tips   16   74979
How to upload and process the CSV file in Django

In this article we will discuss how to upload a csv file and then process the content without storing file on server. 

One approach could be uploading the file, storing it in upload directory and then reading the file.

Another approach could be uploading file and reading it directly from post data without storing it in memory and displaying the data.

We will work with the later approach here.


Uploading CSV file:

First create HTML form to upload the csv file. Use below code for the same.

<form action="{% url "myapp:upload_csv" %}" method="POST" enctype="multipart/form-data" class="form-horizontal"> 
{% csrf_token %}
<div class="form-group">
    <label for="name" class="col-md-3 col-sm-3 col-xs-12 control-label">File: </label>
    <div class="col-md-8">
        <input type="file" name="csv_file" id="csv_file" required="True" class="form-control">
    </div>                    
</div>
<div class="form-group">                    
    <div class="col-md-3 col-sm-3 col-xs-12 col-md-offset-3" style="margin-bottom:10px;">
         <button class="btn btn-primary"> <span class="glyphicon glyphicon-upload" style="margin-right:5px;"></span>Upload </button>
    </div> 
</div>
</form>


Important: Do not forget to include enctype="multipart/form-data"  in form.

upload csv in django.png


Add a URL in URLpatterns.

url(r'^upload/csv/$', views.upload_csv, name='upload_csv'),
Create a function in views with the name upload_csv .


Process the CSV file:

In view function, get the file from post data and process it. I used below code in my project.

def upload_csv(request):
	data = {}
	if "GET" == request.method:
		return render(request, "myapp/upload_csv.html", data)
    # if not GET, then proceed
	try:
		csv_file = request.FILES["csv_file"]
		if not csv_file.name.endswith('.csv'):
			messages.error(request,'File is not CSV type')
			return HttpResponseRedirect(reverse("myapp:upload_csv"))
        #if file is too large, return
		if csv_file.multiple_chunks():
			messages.error(request,"Uploaded file is too big (%.2f MB)." % (csv_file.size/(1000*1000),))
			return HttpResponseRedirect(reverse("myapp:upload_csv"))

		file_data = csv_file.read().decode("utf-8")		

		lines = file_data.split("\n")
		#loop over the lines and save them in db. If error , store as string and then display
		for line in lines:						
			fields = line.split(",")
			data_dict = {}
			data_dict["name"] = fields[0]
			data_dict["start_date_time"] = fields[1]
			data_dict["end_date_time"] = fields[2]
			data_dict["notes"] = fields[3]
			try:
				form = EventsForm(data_dict)
				if form.is_valid():
					form.save()					
				else:
					logging.getLogger("error_logger").error(form.errors.as_json())												
			except Exception as e:
				logging.getLogger("error_logger").error(repr(e))					
				pass

	except Exception as e:
		logging.getLogger("error_logger").error("Unable to upload file. "+repr(e))
		messages.error(request,"Unable to upload file. "+repr(e))

	return HttpResponseRedirect(reverse("myapp:upload_csv"))
 

In the above code we are performing below actions:

      - If this is a GET request then render the upload csv html file.

      - If this is a POST request then proceed.

      - First check if file name is not ending with .csv then this is not the valid file. You may implement you own checks as well.

      - Then we check if file is too large. If these tests fail, we return to html form page with appropriate error message. For displaying error/success messages, we are using messages framework. Please import required modules.

      - Then we read the file and split the content by new line character.

      - Iterate over each line and split the line using comma.

      - We are assuming that our csv file have 4 columns of data. We stored the data in a dictionary and then pass the data dictionary to a form.

      - If form is valid then we proceed to save the form and hence creating entry in DB.

      - If form is not valid, or any other error is thrown then we log the error in log file.


      Read here about logging the errors in log files. This might be useful on live servers where debug is set to false.


        Please provide your inputs.


        tips   16   74979
        16 comments on 'How To Upload And Process The Csv File In Django'
        Login to comment

        Bhavik Gorajiya Sept. 26, 2018, 4:50 p.m.
        Thank You for this post.I have tried this code in my project but it gives me following error messagename 'logging' is not definedcan you help me?
        Mohammad Asrar A Oct. 15, 2018, 2:26 p.m.
        Thank you for this post.I have tried for my project.But I am finding it difficult to relate forms and also not able to understand this Line "forms = EventsForms(data_dict)".Can you please help
        Jason Ngu Nov. 10, 2018, 6:01 p.m.
        hello please i am getting this error Exception Type: TypeError at /upload/csv/Exception Value: uploadFile() takes 0 positional arguments but 1 was givenplease how can i fix this?
        Raza Sheikh March 11, 2019, 12:19 p.m.
        how i import the csv file data on the django project and simple display in html tags (no on the forms html tags)? for news article post
        Md. Shadd March 21, 2019, 9:04 p.m.
        HI,Thanks for such a good guide.I've followed it through, but couldn't display the output. How can I display the output?
        Vishesh Arora July 3, 2019, 5:42 a.m.
        Hello! Thanks for such a simple guide. I have run the server without any error but when I click on Submit after uploading the particular file it gets refreshed to No File Chosen screen and I can't find any trace of the uploaded csv file being saved in the working directory. Can you please help me to access the csv data?
        Rolph July 23, 2019, 7:11 p.m.
        I am very confused with what to replace the various references of myapp with?
        Jesus July 31, 2019, 4:38 p.m.
        works! but show Unable to upload file. IndexError('list index out of range')
        Joe Oct. 16, 2019, 12:01 a.m.
        Hi i am new to django and would like to ask, is all this code done on the django virtual environment?
        Abhishek Dec. 9, 2019, 6:05 p.m.
        form = EventsForm(data_dict)In this above line its showing undefined variable EventsForm. Please provide me the solution for this.
        Sindhura Dec. 31, 2019, 1:02 p.m.
        Hi,I am working on a tool...in which I need to upload the data which I have in the format of dictionary into database when I click a button to upload data on html page.could u help me with it
        Rathod Jan. 7, 2020, 12:32 p.m.
        Hi can you share the model and form file, and one example of CSV file
        Komo Feb. 6, 2020, 12:56 p.m.
        Hi, I successfully followed above but I only two records are being uploaded. Only the first two items in my CSV
        Prerana N G Feb. 22, 2020, 10:01 a.m.
        form = EventsForm(data_dict) Here I'm getting an error How to resolve?Thanks in advance.And also as You have mentioned above ,can You share the link for"One approach could be uploading the file, storing it in upload directory and then reading the file."
        Efi March 3, 2020, 11:18 p.m.
        hi , so my csv file a list of numbers i want to send a message to, i am trying to read the csv file from a path on my server , the process it send my message to these numbers how do i go about it , thanks
        Randhir Singh April 2, 2020, 2:23 p.m.
        Could i get complete project code for this?

        Related Articles:
        atexit with example
        There is an option in python where you can execute a function when the interpreter terminates. Here we will see how to use atexit module....
        Improve Your Python Practices: Debugging, Testing, and Maintenance
        improving your python skills, debugging, testing and practice, pypi...
        How to start with Python Programming - A beginner's guide
        starting with python programming, how to start learning python programming, novice to expert in python, beginner to advance level in python programming, where to learn python programming, become expert in python programming...
        Adding Robots.txt file to Django Application
        Adding robots.txt file in your Django application, Easiest way to add robots.txt file in Django, Django application robots.txt file, Why should you add robots.txt file in your Django Application,...
        DigitalOcean Referral Badge

        © 2022-2023 Python Circle   Contact   Sponsor   Archive   Sitemap