UnicodeEncode Error in deployed Python Application

Question

I have a simple python/pyqt application that is inserting data into a SQLITE database table, some of the entries contain special characters such as umlauts. I did the development using eclipse, tested it and managed to insert all the data without any errors. I then decided to create an executable, of my application. I am using py2exe to create the executable. My setup.py looks as follows

from distutils.core import setup
import py2exe
import os
includes=["sqlite3","sip","PyQt4", "PyQt4.QtGui"]
excludes=[]
packages=[]
dll_excludes=['libgdk-win32-1.0-0.dll', 'libgobject-2.0-0.dll', 'tcl84.dll', 'tk84.dll']

setup(

      options={"py2exe": {"compressed": 2,
                          "optimize": 2,
                          "includes": includes,
                          "excludes": excludes,
                          "packages": packages,
                          "dll_excludes": dll_excludes,
                          "bundle_files": 2,
                          "dist_dir": "dist",
                          "xref": False,
                          "skip_archive": False,
                          "ascii": False,
                          "custom_boot_script": '',
                          }
               },
      windows=['MainWindow.py'],     
      )

It seems to work in as far as I get an executable application in a "dist" folder, when I run it, it loads fine, When I insert the same data, it falls over at the special characters(umlauts). of 100 entries I successfully insert 60, the first occurrence of an accented character causes the executable to fall over with the error message

(, UnicodeEncodeError('ascii', u'Citro\xebnCX', 5, 6, 'ordinal not in range(128)'), )

Just so happens I was inserting CitroenCX into the table, the 'e' being the french accented character

I don't know why when I run the python/pyqt application from my eclipse environment, I get no errors, but once I create an executable i get this error.

What changes should I make in my setup file

Thanks. I applied the solution

 def setencoding():
    encoding = "ascii" 
    if 0:
        import locale
        loc = locale.getdefaultlocale()
        if loc[1]:
            encoding = loc[1]
    if 0: #changes comes here, change 0 to 1
        encoding = "undefined" #the encoding you want
    if encoding != "ascii":
        sys.setdefaultencoding(encoding)

and it worked for me. Not an original solution, saw it on SO How to set default encoding in Python (setdefaultencoding() function does not exist)?

to exclude py2exe from the equation, you could try to run python MainWindow.py in a console and compare it with python MainWindow.py >output. The former might succeed while the latter should fail. — jfs
– jfs, Commented Dec 13, 2013 at 0:19
modifying by hand predefined setencoding() function in a system-wide site.py is a terrrible terrible idea. It changes how all python scripts work on your system in a such way that only occasionally would break them. Many scripts won't expect your changes. It would be a nightmare to troubleshoot random breakage in programs that you even didn't know they were implemented in Python. Don't do it. Fix your code instead. If you don't know how, ask. — jfs
– jfs, Commented Dec 13, 2013 at 17:40

Mark Ransom · Accepted Answer · 2013-12-12 23:32:57Z

1

The difference is in the default encoding selected by the different environments. Any I/O of Unicode to stdin or stdout will trigger an encode or decode using the default encoding. If that encoding is 'ascii' then any accented character will fail.

You can check the defaults to be sure:

import sys
sys.getdefaultencoding()
sys.stdin.encoding
sys.stdout.encoding

answered Dec 12, 2013 at 23:32

Mark Ransom

310k44 gold badges423 silver badges660 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user595985 Over a year ago

Thanks. your response got me on the right track. I eventually looked at How to set default encoding in Python (setdefaultencoding() function does not exist)? stackoverflow.com/questions/7105441/…

Collectives™ on Stack Overflow

UnicodeEncode Error in deployed Python Application

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related