Write to file bytes and strings

Question

I have to create files which have some chars and hex value in little-endian encoding. To do encoding, I use:

pack("I", 0x01ddf23a)

and this give me:

b':\xf2\xdd\x01'

First problem is that, this give me bytes string which I cannot write to file. Second one is that \x3a is turn to ':'. What I expect, is write to file \x3a\xf2\xdd\x01 as bytes not as chars.

What I tried:

>>> a=0x01ddf23a

>>> str(pack("I", a))
"b':\\xf2\\xdd\\x01'" <= wrong

>>> pack("I", a).hex()
'3af2dd01             <= I need '\x' before each byte

>>> pack("I", a).decode()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 1: invalid continuation byte

Changing open() from "w" to "wb" force me to write only bytes, but I want to writes lots of strings and few bytes, eg.:

Hello world
^I^M^T^B
End file

I know I can simple do this:

fs.open("file" "w")
fs.write("Hello world")
fs.write("\x3a\xf2\xdd\x01")
fs.write("End file")
fs.close()

But this is make my byte value 0x01ddf23a hard to read and there is easy to make some mistake when changing this value in that form.

Martijn Pieters · Accepted Answer · 2016-09-23 15:28:49Z

7

You are producing bytes, which can be written to files opened in binary mode without issue. Add b to the file mode when opening and either use bytes string literals or encode your strings to bytes if you need to write other data too:

with open("file", "wb") as fs:
    fs.write(b"Hello world")  # note, a byte literal!
    fs.write(pack("I", 0x01ddf23a))
    fs.write("End file".encode('ASCII'))  # encoded string to bytes

The alternative would be to decode your binary packed data to a text string first, but since packed data does not, in fact, contain decodable text, that approach would require contortions to force the binary data to be decodable and encodable again, which only works if your file encoding was set to Latin-1 and severely limits what actual text you could add.

A bytes representation will always try to show printable characters where possible. The byte \x3a is also the correct ASCII value for the ':' character, so in a bytes representation the latter is preferred over using the \x3a escape sequence. The correct value is present in the bytes value and would be written to the file entirely correctly:

>>> b'\x3a'
b':'
>>> b'\x3a' == b':'
True
>>> b':'[0]
58
>>> b'\x3a'[0]
58
>>> hex(58)
'0x3a'

answered Sep 23, 2016 at 15:28

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Borgo Over a year ago

Perfect! Exactly what I need :) First, easy to use bytes string literal with files in binary mode, an second good explanation that if python show me some ascii in bytes string literal, it doesn't mean that it will print me ascii character.

Collectives™ on Stack Overflow

Write to file bytes and strings

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related