I thought that NumPy arrays always consume less memory than Python lists. However, when I tested this with an empty list, the pure Python list had 56 bytes and the NumPy array was 112 bytes in size. Why?
1 Answer
I reopened this because the duplicate focused on how np.reshape produces a view and changes what getsizeof sees. Here the issue is the size of a list versus an array.
Let me illustrate: (posting an image of this code is not good SO style. We prefer copy-n-paste code )
Your list and array:
In [458]: alist = [1,2,3,4,5,7,'a','b','c','@']
In [459]: alist
Out[459]: [1, 2, 3, 4, 5, 7, 'a', 'b', 'c', '@']
In [460]: arr = np.array(alist)
In [461]: arr
Out[461]: array(['1', '2', '3', '4', '5', '7', 'a', 'b', 'c', '@'], dtype='<U21')
Note the dtype. The array contains strings, not numbers.
In [462]: arr.nbytes
Out[462]: 840
In [463]: import sys
In [464]: sys.getsizeof(arr)
Out[464]: 952
getsizeof gets that 840, plus 112 'overhead'. For regular arrays, getsizeof gives a reasonable number, but really isn't needed.
But for the list:
In [465]: sys.getsizeof(alist)
Out[465]: 136
We can get the 840 bytes by checking the length and dtype:
In [466]: len(arr)
Out[466]: 10
In [467]: 4*21*10
Out[467]: 840
For the list, 'overhead' is 56, and the rest is storage for pointers - 10 of them.
In [468]: sys.getsizeof([])
Out[468]: 56
In [469]: 56+80
Out[469]: 136
Lists can also have memory for 'growth space'. getsizeof does not measure the memory used by the objects pointed to. In this case, the small integers already exist, and don't require any additional memory. The strings take up an extra 50 bytes each. Lists can store objects of various types, including other lists and dicts and arrays, etc. getsizeof tells us nothing about those.
The array could have been given a different dtype, with a reduction in memory:
In [470]: arr1 = np.array(alist,'U1')
In [471]: arr1
Out[471]: array(['1', '2', '3', '4', '5', '7', 'a', 'b', 'c', '@'], dtype='<U1')
In [472]: arr1.nbytes
Out[472]: 40
In [473]: sys.getsizeof(arr1)
Out[473]: 152
In sum, to get anything useful from getsizeof you have to understand how the object/class is stored, and just what that function measures. Neither is a trivial topic for a python beginner. Well, the beginner should learn, soon if not later, how lists and arrays are stored.
1 Comment
chepner
Because the size of a NumPy array also includes the contents of the array, an accurate comparison must take into account the size of all objects recursively contained in the list. Even ignoring arrays,
[1] and [[1,2,3,4,5,6,7,8,9]] both have the same size according to getsizeof, because each is a list with a single reference to some other object.
sys.getsizeis not measuring all of the list memory use.