Python script encoding trouble when run in CRON job

Question

As many hopefully can relate, this encoding problem is driving me mental. I would really appreciate some light on this!

End goal is to be able to run the same script.py from both terminal and cron, and from cron with > stdout.txt. And needless to say, I'm having serious encoding troubles.

My script.py runs fine from terminal thus: python script.py
It throws an error, however, when run from terminal thus: python script.py > stdout.txt
It throws same error when run in cron, either way.

I have a python script, entered in crontab -e as root.

This is my script.py header:

#!/usr/bin/python
# -*- coding: utf-8 -*-

This is my cron entry:

* * * * * python /home/ubuntu/parrot/script.py > /home/ubuntu/parrot/stdout.txt

This is my stdout.txt (the relevant part):

Unexpected error!  (<type 'exceptions.UnicodeDecodeError'>, UnicodeDecodeError('ascii', 'blabla some weird text n\xc3\xa5r end', 54, 55, 'ordinal not in range(128)'))

This is my env from terminal (the relevant part):

LANG=en_US.UTF-8

This is my env from cron (the relevant part):

LANG=en_US.UTF-8

This is the (first) line in script.py throwing the error:

print 'Posting @%s: %s' % (statusObj.user.screen_name.encode('ascii', 'replace'), statusObj.text.encode('utf-8', 'replace'))

Edit: sys.getdefaultencoding() returns ascii

Any help is greatly appreciated!

If this isn't the same problem as in stackoverflow.com/questions/2224130/… it is close. — msw
– msw, Commented Aug 7, 2012 at 1:26
possible duplicate of Setting the correct encoding when piping stdout in python — Mechanical snail
– Mechanical snail, Commented May 31, 2013 at 19:17
Possible duplicate of UnicodeEncodeError only when running as a cron job — Ariel M.
– Ariel M., Commented May 20, 2016 at 1:01

monkut · Accepted Answer · 2012-08-07 01:11:07Z

1

If you have control over the statusObj you should check the relevant code where data is being parsed into the object and try to get the input as clean as possible.

You want to make sure your string is decoded to unicode before you try to encode it.

If not you can try:

# try to get the string into unicode
screen_name = unicode(statusObj.user.screen_name) 
post = unicode(statusObj.text) # probably an error here?
output_str = u"Posting @{name}: {post}".format(name=screen_name, post=post)
print output_str.encode("utf8", "replace") # encode the unicode string on

answered Aug 7, 2012 at 1:11

monkut

44.1k26 gold badges133 silver badges161 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

knutole Over a year ago

but my code already works from CLI. so it must be something with the encoding environment of cron?? i will try tho. thank you!

knutole Over a year ago

i just added sys.stdout = codecs.getwriter('utf8')(sys.stdout) to the top and deleted the .encode'ings altogether. problem solved! thanks a lot all! =)

Collectives™ on Stack Overflow

Python script encoding trouble when run in CRON job

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related