Does python 3.1.3 support unicode in csv module?

Question

I have been using python 2.6. While I was writing a python program to process the query result ( in csv format ) from sql server. I found it does not support unicode.

When I run the program with csv file, a error poped up saying:

    for row in csvReader:
Error: line contains NULL byte

After I save the csv file in ANSI/ASCII format with Ultraedit, the program is running okay.

I tried to include the encoding option, but it failed:

csvReader = csv.reader(open(fname, mode='rb', encoding='unicode'), delimiter=',')
TypeError: 'encoding' is an invalid keyword argument for this function

csvReader = csv.reader(open(fname, mode='rb', encoding='utf-8'), delimiter=',')
TypeError: 'encoding' is an invalid keyword argument for this function

I wonder if python 3 support this unicode reading. It can save me a lot of work.

Python 3.1.3's open definitely supports and encoding= argument, so either you're using a different version of Python, or you have accidentally overwritten the open function. — David Wolever
– David Wolever, Commented Feb 29, 2012 at 7:02
I said I am using python 2.6. So Python 3.1.3 won't have this problem? — lamwaiman1988
– lamwaiman1988, Commented Feb 29, 2012 at 7:05
Python 3 still have this problem. Apparently there are a NUL byte there messing up python, no matter it is python 3 or python 2. — lamwaiman1988
– lamwaiman1988, Commented Feb 29, 2012 at 7:26
Oh, sorry, it wasn't clear that you were using Python 2.6. See my updated answer. — David Wolever
– David Wolever, Commented Feb 29, 2012 at 19:00

David Wolever · Accepted Answer · 2012-02-29 19:04:10Z

7

Python 3 definitely supports unicode. My guess is that you specified the wrong (or no?) encoding when you opened the CSV file for reading. See: http://docs.python.org/release/3.1.3/library/functions.html#open

And try something like:

reader = csv.reader(open("foo.csv", encoding="utf-8"))

Edit: If you are using Python 2.6, you can achieve the same result with:

import codecs
reader = csv.reader(codecs.open("foo.csv", encoding="utf-8"))

HOWEVER if you're getting null bytes, your file may be encoded using "utf-16", so try that if the file can't be decoded using utf-8.

edited Feb 29, 2012 at 19:04

answered Feb 29, 2012 at 6:55

David Wolever

156k94 gold badges365 silver badges513 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

lamwaiman1988 Over a year ago

I tried to specified the encoding, but it return an error. Please check my edit.

Community · Accepted Answer · 2017-05-23 12:08:01Z

2

A similar question is already answered Python CSV error: line contains NULL byte

Also, Try to open it in 'rb' mode instead of 'rU'

edited May 23, 2017 at 12:08

CommunityBot

11 silver badge

answered Feb 29, 2012 at 6:57

jerrymouse

18k20 gold badges83 silver badges101 bronze badges

Collectives™ on Stack Overflow

Does python 3.1.3 support unicode in csv module?

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related