UnicodeDecodeError when using pymssql with freetds

Question

I want to get a field in SQL server 2008 from python 2.6. Here is my freeTDS .conf file:

[ARGSERVER03]
    host = 192.168.1.3
    port = 1433
    tds version = 7.0

Here is the code:

conn = pymssql.connect(host='192.168.1.3', user='****', password='****', database='TrafficMonitor', as_dict=True, charset='UTF-8')
i = 0
cur.execute('SELECT * FROM dbo.tblTrafficCounterData')
while i < 10:
    car = cur.fetchone_asdict()
    if car is None:
        break
    c = car['Class']
    print c
    i = i + 1

But it gives:

UnicodeDecodeError: 'utf8' codec can't decode byte 0xd3 in position 0: invalid continuation byte

The Unicode field is in Persian. The trace back is for line car = cur.fetchone_asdict()

[Edit]

I have checked for database collation in database properties from sql server management studio and it is:

Arabic_CI_AS

But when I use that in charset it gives:

LookupError: unknown encoding: Arabic_CI_AS

This error seems to be very similar to this: stackoverflow.com/questions/9090915/… — Thanasis Petsas
– Thanasis Petsas, Commented Apr 4, 2012 at 12:45

John Paulett · Accepted Answer · 2012-04-04 13:14:03Z

4

Are you extremely sure that SQL Server is using UTF-8 (indicated by your charset='UTF-8')? Typically most SQL Server instances I have run into use a Microsoft encoding (not UTF-8), such as cp1252 (in the U.S.).

A few things that may help you discover the correct encoding:

SELECT DATABASEPROPERTYEX('dbname', 'Collation') SQLCollation

answered Apr 4, 2012 at 13:14

John Paulett

15.9k4 gold badges48 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

John Paulett Over a year ago

Arabic_CI_AS is the SQL Server name for the collation, it only provides a hint as to the Python encoding to use. Try using cp1256 as the charset (cp1256 is for Windows Arabic, as shown at docs.python.org/library/codecs.html#standard-encodings)

Collectives™ on Stack Overflow

UnicodeDecodeError when using pymssql with freetds

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related