In Django, the streaming response StreamingHttpResponse is a great thing to generate a large file quickly and memory efficiently.
One of the current projects for streaming responses is Eventsource, which is used to improve the feeling of slowness generated by users when communicating across systems. I won't go into detail on this one.
There is also one that generates a large csv file.
When the Django process is in gunicorn or uwsgi and other web containers, if the response over a certain period of time did not return, it will be terminated by the web container , although we can lengthen the timeout time of the web container to bypass the problem, but after all, it is still treating the symptom but not the root cause. To fundamentally solve this problem, Python's generator, Django framework provides StreamingHttpResponse this streaming response is very helpful!
And in csv, Chinese handling is also crucial to make sure that opening csv with excel doesn't mess up the code or something. In order to save space, I will paste all the code together. The actual use of the project in accordance with the planning placed ha
Up Code:
from __future__ import absolute_import
import csv
import codecs
import cStringIO
class Echo(object):
def write(self, value):
return value
class UnicodeWriter:
"""
A CSV writer which will write rows to CSV file "f",
which is encoded in the given encoding.
"""
def __init__(self, f, dialect=, encoding="utf-8", **kwds):
# Redirect output to a queue
= ()
= (, dialect=dialect, **kwds)
= f
= (encoding)()
def writerow(self, row):
([handle_column(s) for s in row])
# Fetch UTF-8 output from the queue ...
data = ()
data = ("utf-8")
# ... and reencode it into the target encoding
data = (data)
# write to the target stream
value = (data)
# empty queue
(0)
return value
def writerows(self, rows):
for row in rows:
(row)
from import View
from import StreamingHttpResponse
class ExampleView(View):
headers=['Some','Table header']
def get(self,request):
result = [['First line','Data 1'],
['Second line','Data 2']]
echoer = Echo()
writer = UnicodeWriter(echoer)
def csv_itertor():
yield codecs.BOM_UTF8
yield ()
for column in result:
yield (column)
response = StreamingHttpResponse(
(row for row in csv_itertor()),
content_type="text/csv;charset=utf-8")
response['Content-Disposition'
] = 'attachment;filename=""'
return response