For example, the shape of the array is 5,4.
a = np.random.randint(10, size= (5, 4))
a =
[[1 4 5 0]
[3 1 5 1]
[4 8 0 9]
[8 1 5 8]
[6 4 7 4]]
I want the array to be reshaped as:
a =
[[1 4]
[3 1]
[4 8]
[8 1]
[6 4]
[5 0]
[5 1]
[0 9]
[5 8]
[7 4]]
My original array size is around 200 GB and of shape 80000*480600. I have tried to use remap mode but it is very slow.
Use numpy.hsplit and numpy.concatenate:-
>>> a = np.random.randint(10, size= (5, 4))
>>> a
array([[8, 5, 8, 9],
[9, 5, 6, 3],
[5, 3, 8, 7],
[9, 0, 9, 9],
[0, 7, 8, 0]])
>>> t = np.hsplit(a, 2)
>>> t
[array([[8, 5],
[9, 5],
[5, 3],
[9, 0],
[0, 7]]), array([[8, 9],
[6, 3],
[8, 7],
[9, 9],
[8, 0]])]
>>> np.concatenate([t[0], t[1]])
array([[8, 5],
[9, 5],
[5, 3],
[9, 0],
[0, 7],
[8, 9],
[6, 3],
[8, 7],
[9, 9],
[8, 0]])
hsplit and reshape also works
np.reshape(np.hsplit(a, 2), (-1,2))
Out[99]:
array([[1, 4],
[3, 1],
[4, 8],
[8, 1],
[6, 4],
[5, 0],
[5, 1],
[0, 9],
[5, 8],
[7, 4]])
This can be done with a reshape and transpose (and a final reshape)
In [195]: arr = np.arange(20).reshape(5,4)
In [196]: arr
Out[196]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19]])
In [197]: arr.reshape(5,2,2)
Out[197]:
array([[[ 0, 1],
[ 2, 3]],
[[ 4, 5],
[ 6, 7]],
[[ 8, 9],
[10, 11]],
[[12, 13],
[14, 15]],
[[16, 17],
[18, 19]]])
In [198]: arr.reshape(5,2,2).transpose(1,0,2)
Out[198]:
array([[[ 0, 1],
[ 4, 5],
[ 8, 9],
[12, 13],
[16, 17]],
[[ 2, 3],
[ 6, 7],
[10, 11],
[14, 15],
[18, 19]]])
Identifying the right transpose (or swapaxes) may require a bit of trial and error.
In [199]: arr.reshape(5,2,2).transpose(1,0,2).reshape(-1,2)
Out[199]:
array([[ 0, 1],
[ 4, 5],
[ 8, 9],
[12, 13],
[16, 17],
[ 2, 3],
[ 6, 7],
[10, 11],
[14, 15],
[18, 19]])
The equivalent with split and concatenate:
In [200]: np.concatenate(np.hsplit(arr,2), axis=0)
Out[200]:
array([[ 0, 1],
[ 4, 5],
[ 8, 9],
[12, 13],
[16, 17],
[ 2, 3],
[ 6, 7],
[10, 11],
[14, 15],
[18, 19]])
The transpose route should be faster.
