Open the file in universal-newline mode using the CSV Django module

Question

I am trying to access a model.filefield in Django to parse a CSV file in Python using the csv module. It's working on Windows, but on Mac it gave me this:

Exception Type: Error

Exception Value: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

This is the code:

myfile = customerbulk.objects.all()[0].fileup

mydata = csv.reader(myfile)
    for email,mobile,name,civilid in mydata:
        print email,mobile,name,civilid

what is this customerbulk.objects.all()[0].fileup thing. Is it a filename on a model? — SingleNegationElimination
– SingleNegationElimination, Commented Jul 17, 2011 at 21:42
fileup = models.FileField(verbose_name="CsvFile",upload_to='ExcelFiles') if i make a small query like customerbulk.objects.get(pk=1) — mohd
– mohd, Commented Jul 17, 2011 at 21:49
The exact reason of using rU (It's related to the open() function and is not csv specific.): In addition to the standard fopen() values mode may be 'U' or 'rU'. Python is usually built with universal newlines support; supplying 'U' opens the file as a text file, but lines may be terminated by any of the following: the Unix end-of-line convention '\n', the Macintosh convention '\r', or the Windows convention '\r\n'. docs.python.org/2/library/functions.html#open — zengr
– zengr, Commented Jun 4, 2013 at 2:53

Peter Mortensen · Accepted Answer · 2016-02-05 16:51:02Z

151

I finally found the solution:

mypath = customerbulk.objects.get(pk=1).fileup.path
o = open(mypath,'rU')
mydata = csv.reader(o)

edited Feb 5, 2016 at 16:51

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Jul 18, 2011 at 21:41

mohd

2,6342 gold badges17 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

timbo Over a year ago

I believe you can simplify this. It seems odd to seek to the start after just opening the file. The following helped me overcome the same issue from an Excel spreadsheet export to CSV using the defaults: data = csv.reader(open(FILENAME, 'rU'), quotechar='"', delimiter = ',')

rd108 Over a year ago

Yes, there's no need to seek to beginning of file just after opening. The >>>o = open(mypath, 'rU') bit, with the 'rU' flag is the important magic that worked in my case.

Xiao Over a year ago

PEP278 explained what rU stands for:

In a Python with universal newline support open() the mode     parameter can also be "U", meaning "open for input as a text file     with universal newline interpretation".  Mode "rU" is also allowed,     for symmetry with "rb".

Naymesh Mistry Over a year ago

@Xiao +1 for linking to the original PEP which explains the rationale.

Tim Diels Over a year ago

In Python >= 3, use newline instead, the default is newline=None which is like newline='' except it also translates the newlines to \n. I'm unsure which of these is equivalent to mode='U'

Collectives™ on Stack Overflow

Open the file in universal-newline mode using the CSV Django module

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related