We know how to return CSV file as response in Django. In most of the cases building such CSV doesn't take much time and response is returned immediately. 

However in some exceptional cases, you might require to return a large CSV file in response.  

In such case one the below thing can happen:
- Generating large file may take more time and hence web server or load balancer may drop the connection due to timeout.
- Generating large file may eat up all the memory available to process

To avoid above situation, we can generate and stream the large CSV files in Django. For this we will use StreamingHttpResponse class.

Note: It is recommended to perform the time taking tasks asynchronously. We should not include such tasks in request response cycle. Web worker remain blocked during the whole download process. If you have fewer number of workers and more simultaneous requests, users may not be able to access your application.

Code:

First lets understand the writer method of csv.

The csv.writer() method accepts a file-like object and returns a writer object which converts the user's data into delimited strings on the given file-like object. See the code example below.

import csv

numbers = [
    [1, 2, 3],
    [4, 5, 6]
]

with open('pythoncircle-dot-com.csv', 'w') as f:
    writer = csv.writer(f)
    for row in numbers:
        writer.writerow(row)

writerow method will take care of comma and escaping part and write the row on to file-like object.

If you look at the source code of StreamingHttpResponse class, you will see that its __init__ method accepts the streaming_content as first parameter which should be an iterable of bytestrings. To create streaming_content we need to use modified version of write method.

Here in our streaming-csv code, we will not actually write the row on file but will return it as it is.

For this we need to create a dummy file-like object with write method which will return the value. 

Here is a class with write method.

class DummyFile:
    def write(self, value_to_write):
        return value_to_write

Now create the writer object using object of this DummyFile class. 

writer = csv.writer(DummyFile())

Now pass multiple values to this writer object's writerow method one by one.

rows = [[str(i), str(2*i), str(3*i)] for i in range(50000)]
data = [writer.writerow(row) for row in rows]

Now create and return response (Streaming response to be specific) using this data.

response = StreamingHttpResponse(data, content_type="text/csv")
response['Content-Disposition'] = 'attachment; filename="pythoncircle-dot-com.csv"'
return response

Complete code will look something like below.

view.py file:

import csv
from django.http import StreamingHttpResponse


class DummyFile:
    def write(self, value_to_write):
        return value_to_write


def large_csv(request):
    rows = ([str(i), str(2 * i), str(3 * i)] for i in range(555555))
    writer = csv.writer(DummyFile())
    data = [writer.writerow(row) for row in rows]
    response = StreamingHttpResponse(data, content_type="text/csv")
    response['Content-Disposition'] = 'attachment; filename="pythoncircle-dot-com.csv"'
    return response

Create appropriate entry in urls.py file to trigger large_csv view. Once you hit the url in browser, file will start downloading.

Host your Django Apps for free.