Python can indeed left shift a bit by large integers
1L << 100
# 1267650600228229401496703205376L
But NumPy seemingly has a problem:
a = np.array([1,2,100])
output = np.left_shift(1L,a)
print output
# array([ 2, 4, 68719476736])
Is there a way to overcome this using NumPy's left_shift operation? Individually accessing the array elements gives the same incorrect result:
1L << a[2]
# 68719476736
Python long values aren't the same type as the integers held in a. Specifically, Python long values are not limited to 32 bits or 64 bits but instead can take up an arbitrary amount of memory.
On the other hand, NumPy will create a as an array of int32 or int64 integer values. When you left-shift this array, you get back an array of the same datatype. This isn't enough memory to hold 1 << 100 and the result of the left-shift overflows the memory it's been allocated in the array, producing the incorrect result.
To hold integers that large, you'll have to specify a to have the object datatype. For example:
>>> np.left_shift(1, a.astype(object))
array([2, 4, 1267650600228229401496703205376L], dtype=object)
object arrays can hold a mix of different types, including the unlimited-size Python long/integer values. However, a lot of the performance benefits of homogeneous datatype NumPy arrays will be lost when using object arrays.
int, but the docs use integer somewhat as a synonym for int). longs act as you described. That said, the code here uses long types and python uses long if your literal is too big for int. Your statement just confused me, and I wanted to spare others to look it up at docs.python.org/2/library/…
long type has disappeared (I see now that the question is tagged Python 2.7).
long type has disappeared in Python 3" may mislead. In Python 3 the built-in int type has no theoretical upper limit on size, so essentially long has been integrated into int.Given the array a you create, the elements of output are going to be integers. Unlike Python itself, integers in a numpy darray are of fixed size with a defined maximum and minimum value.
numpy is arriving at the value given (2 ** 36) by reducing the shift modulo the length of the integer, 64 bits (100 mod 64 == 36).
This is an issue with 64-bit Python too, except that the integer at which this becomes a problem is larger. if you run a[2]=1L << a[2] you will get this Traceback :
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long
So as the traceback says Python int too large to convert to C long , so you need to change the array type (that is actually C structure ) to an python object :
>>> a = np.array([1,2,100],dtype='O')
>>> a[2]=1L << a[2]
>>> a
array([1, 2, 1267650600228229401496703205376L], dtype=object)
