How to read double, float and int values from binary files in python?

Question

I have a binary file that was created in C++. The first value is double and the second is integer. I am reading the values fine using the following code in C++.

double dob_value;
int integer_value;

fread(&dob_value, sizeof(dob_value), 1, fp);
fread(&integer_value, sizeof(integer_value), 1, fp);

I am trying to read the same file in python but I am running into issues. My dob_value is 400000000.00 and my integer_value 400000. I am using following code in python for double.

def interpret_float(x):
    return struct.unpack('d',x[4:]+x[:4])
with open(file_name, 'rb') as readfile:
     dob = readfile.read(8)
     dob_value = interpret_float(dob)[0]
     val = readfile.read(4)
     test2 = readfile.read(4)
     integer_value = int.from_bytes(test2, "little")

My dob_value is 400000000.02384186 . My question is where is this extra decimals coming from? Also, how do I get the correct integer_value? With above code, my integer_value is 1091122467. I also have float values after integer but I haven't looked into that yet.

@DOOM I have uploaded the file here: link This file has double,integer and float in the same order. — someoneonboard1
– someoneonboard1, Commented Mar 24, 2020 at 23:43
can you change in interpret_float the return statement to struct.unpack('d', x) — DOOM
– DOOM, Commented Mar 24, 2020 at 23:52

Suthiro · Accepted Answer · 2020-03-25 07:38:12Z

2

If the link goes broken and just in case the test.bin contains
00 00 00 00 84 D7 B7 41 80 1A 06 00 70 85 69 C0.

Your binary contains correct 41B7D78400000000 hexadecimal representation of 400000000.0 in the first 8 bytes. Running

import binascii
import struct
fname = r'test.bin'
with open(fname, 'rb') as readfile:
    dob = readfile.read(8)
    print(struct.unpack('d', dob)[0])
    print(binascii.hexlify(dob))

outputs

>> 400000000.0
>> b'0000000084d7b741'

which is also correct little endian representation of the double.
When you swap parts, you get

print(binascii.hexlify(dob[4:]+dob[:4]))
>> b'84d7b74100000000'

and if you check the decimal value, it will give you 5.45e-315, not what you expect. Moreover,

struct.unpack('d', dob[4:]+dob[:4])[0]
>>5.44740625e-315

So I'm not sure how you could get 400000000.02384186 from the code above. However, to obtain 400000000.02384186 using your test.bin, just skip the four bytes in the beginning:

with open(fname, 'rb') as readfile:
    val = readfile.read(4)
    dob = readfile.read(8)
    dob = dob[4:]+dob[:4]
    print(binascii.hexlify(dob))
    print(struct.unpack('d', dob)[0])
>>b'801a060084d7b741'
>>400000000.02384186

Binary value 0x41B7D78400061A80 corresponds to 400000000.02384186. So you first read incorrect bytes, then incorrectly swap parts and get a result close to what you expect. Considering integer value, the 400000 is 0x00061A80, which is also present in the binary, but you definitely read past that bytes, since you used them for double, so you get wrong values.

edited Mar 25, 2020 at 7:38

answered Mar 25, 2020 at 6:03

Suthiro

1,3001 gold badge9 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

gamesun Over a year ago

b'0000000084d7b741' convert to little endian is b'41b7d78400000000', not swap half parts, the endian conversion is reverse every bytes. And struct support big/little endian by using struct.unpack('>d', dob), struct.unpack('<d', dob)

Suthiro Over a year ago

Could you please clarify, you think that the answer contains mistake(s) or you just added some info? Thanks.

gamesun Over a year ago

<1> mistake: dob[4:]+dob[:4] is not endian conversion. <2> add info: struct module support endian conversion.

Suthiro Over a year ago

@gamesun <1> I suppose you misread something? Do you mean the line with "which is also correct little endian representation of the double. When you swap parts"? If so, it's my fault with bad formatting, I will add a newline.

someoneonboard1 Over a year ago

Thank you @Suthiro. You pointed out the problem. Actual file I was reading had:

x="\x00\x00\x00\x00\x00\x84\xd7\xb7A\x80\x1a\x06\x00#5\tA"\x0e\xa0@\x9a\xe5\x06A\xac<1@%=\xf1@\xa4E\x0c?9\xd0\xc7@\xcf\xa8\xaa\xbf\x8a\x9f\x95@\xfaV\x1c\xc0\x9b\x187@\x9d\xd0 \xc05DR?\xb3r\xd9\xbfE\xea\xa0\xbf7d+\xbefXJ\xc0\xa5"

I was supposed to ignore the first '\x00' but I was ignoring all '\x00' which was causing the problem.

Collectives™ on Stack Overflow

How to read double, float and int values from binary files in python?

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related