Python hex string read in from file no longer interpreted as hex

Question

I'm using python 2.7

I have a file that contains hex strings:

    \x01\x02\x93AAA
    \x02\x02\x93AAA
    \x03\x02\x93AAA
....

They are a mix of unprintable characters, represented as hex, and printablecharacters represented in ascii.

I am reading them into a map based on their line number. Then trying to send it over a socket. I am receiving inconsistent results when I send from my map versus as a straight string.

Example code:

print mymap[0]
mysocket.sendall(mymap[0])
mysocket.sendall("\x01\x02\x93AAA")

With these two send attempts, in wireshark, I can see the first one when sending from mymap is being sent in ASCII. As in it sends '\', 'x', '0', '1' instead of interpreting that as a hex 0x01. Where as the second line sends it correctly as hex data.

The print mymap[0] prints exactly the same string ("\x01\x02\x93AAA"), so why is one sending it as ASCII, and one sending it as hex?

I want it to send as hex, but when using my map It keeps sending it as ASCII, yet just hardcoding a string in sends as hex. Help?

Edit: I updated the title of this question to reflect what the underlying problem actually was. For those happening upon this later. My original thought was it had to do with reading the data from a map, when in reality the issue was reading it in from a file.

Daniel · Accepted Answer · 2016-03-23 22:15:24Z

1

In Python code '\x01' is an escape sequence, in a text file it is only text. You have to unescape the string before sending.

import re

def unescape(text):
    return re.sub(r'\\x([0-9a-fA-F]{2})', lambda g: chr(int(g.group(1),16)), text)

answered Mar 23, 2016 at 22:15

Daniel

42.9k4 gold badges57 silver badges82 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Without Me It Just Aweso Over a year ago

Daniel, you are my hero. I've spent so much time hitting my head against a wall on this.. That worked perfectly

ShadowRanger Over a year ago

Nitpicking: Won't this fail with a legitimate input like \\x40 which is supposed to be one backslash, followed by x40, but instead gets interpreted as \@ (changing \x40 to @)? I'd think careful use of ast.literal_eval would be safer. Maybe something like fmt, repl = ('"""{}"""', '"') if text.endswith("'") else ("'''{}'''", "'"), return ast.literal_eval(fmt.format(text.replace(repl, '\\' + repl))) (the replace escapes the quote characters, the format wraps to a string literal, then let Python take over).

Collectives™ on Stack Overflow

Python hex string read in from file no longer interpreted as hex

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related