Why is the memory usage of a Python list smaller than expected?

Question

enter image description here

As seen in the picture. 50 000 000 records only take 404M memory, why? Since one record takes 83 Bytes, 50 000 000 records should take 3967M memory.

>>> import sys
>>> a=[]
>>> for it in range(5*10**7):a.append("miJ8ZNFG9iFqiQQohvyTWwqsij2rJCiZ7v"+str(it))
... 
>>> print(sys.getsizeof(a)/1024**2)
404.4306411743164
>>> print(sys.getsizeof("miJ8ZNFG9iFqiQQohvyTWwqsij2rJCiZ7v"))
83
>>> print(83*5*10**7/1024**2)
3957.7484130859375
>>>

Someone else had a similar query as you did but went a bit further, so this is more of a related thread: Deep version of sys.getsizeof — metatoaster
– metatoaster, Commented Jan 17, 2019 at 3:08

ShadowRanger · Accepted Answer · 2019-01-17 03:05:34Z

5

sys.getsizeof only reports the cost of the list itself, not its contents. So you're seeing the cost of storing the list object header, plus (a little over) 50M pointers; you're likely on a 64 bit (eight byte) pointer system, thus storage for 50M pointers is ~400 MB. Getting the true size would require sys.getsizeof to be called for each object, each object's __dict__ (if applicable), etc., recursively, and it won't be 100% accurate since some of the objects (e.g. small ints) are likely shared; this is not a rabbit hole you want to go down.

answered Jan 17, 2019 at 3:05

ShadowRanger

158k12 gold badges221 silver badges315 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

purplecity Over a year ago

yes. 64bit. I care about the actual storage(obtain list contents). so it mean if a list that have 50 000 000 records. actually all related storage is 3957 + 404 M?

ShadowRanger Over a year ago

@purplecity: Well, your records are 83 bytes, plus the length of the stringified int you're adding on, so a bit larger than that, more like 4339 + 404 M, but yes, that's roughly correct.

Collectives™ on Stack Overflow

Why is the memory usage of a Python list smaller than expected?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related