Weird float to integer conversion issue in python

Question

For a calculation in program that I wrote that involves finite algebraic fields, I needed to check whether (2**58-1)/61 was an integer. However python seems to indicates that it is, while it is not.

For example -

>>> (2**58-1)/61
4725088133634619.0

Even when using numpy functions this issue appears -

>>> np.divide(np.float64(2**58)-1,np.float64(61))
4725088133634619.0

This happens although python does calculate 2**58 correctly (I assume this issue is general, but I encountered it using these numbers).

Thierry Lathuille · Accepted Answer · 2021-01-03 15:32:29Z

3

If you use normal / division, your result is a float, with the associated limited precision. The result gets rounded, and in your case, it gets rounded to 4725088133634619.0 - but that doesn't prove that it is an integer.

If you want to check if the result of the division by 61 is an integer, test if the remainder of the division by 61 is 0, using the modulo operator:

>>> (2**58-1) % 61
45

As you can see, it isn't.

answered Jan 3, 2021 at 15:32

Thierry Lathuille

24.4k10 gold badges49 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

proton Over a year ago

Thank you! But howcome when using np.float64 the number is still rounded?

luther Over a year ago

@proton: This number is exactly the right size for float precision to go down to the ones digit. The .0 is superfluous and is there only to show that it's type is float.

alex_noname · Accepted Answer · 2021-01-03 16:24:08Z

1

As for float limited precision mentioned by @Thierry Lathuille, Python's float uses 64 bits and is double-precision that provides 53 bits for mantissa (the same is true for np.float64). That means that not all numbers > 2**53 are representable using the float, we have a loss of precision. For example, 2**53 == 2**53 + 1 is true in double precision. More detailed here:

https://en.wikipedia.org/wiki/Double-precision_floating-point_format

Is floating point math broken?

edited Jan 3, 2021 at 16:24

answered Jan 3, 2021 at 15:48

alex_noname

33.3k6 gold badges95 silver badges110 bronze badges

Comments

Amit · Accepted Answer · 2021-01-03 16:00:50Z

0

Correct answers have already been given. I am just adding another approach (which is not much different from what has already been said).

What you may want to do, due to inherent limitations of representation errors, is use divmod( ) for python and numpy.divmod( ) for numpy. That way you can check the quotient and the remainder.

print(divmod((2**58-1),61))

Gives quotient and remainder as

(4725088133634618, 45)

In numpy you may want to use similar, divmod, function but the numbers should be np.int type and not np.float type (due to representation errors mentioned above).

np.divmod(np.int64(2**58)-1,np.int8(61))

The above gives quotient and remainder as.

(4725088133634618, 45)

answered Jan 3, 2021 at 16:00

Amit

2,1381 gold badge10 silver badges15 bronze badges

Collectives™ on Stack Overflow

Weird float to integer conversion issue in python

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related