List memory usage

Question

I am trying to improve the memory usage of my script in python, therefore I need to know what's RAM usage of my list. I measure the memory usage with

print str(sys.getsizeof(my_list)/1024/1024)

which hopefully would give me the size of the list in RAM in Mb.

it outputs 12 Mb, however in top command I see that my script uses 70% of RAM of 4G laptop when running.

In addition this list should contain a content from file of ~500Mb.

So 12Mb is unrealistic.

How can I measure the real memory usage?

What's the object type contained in my_list? As Python's document shows: "Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to." You may refer this to see an example usage of getsizeof — WDan
– WDan, Commented Dec 25, 2013 at 10:09

brainLoop · Accepted Answer · 2021-10-26 06:48:35Z

34

sys.getsizeof only take account of the list itself, not items it contains.

According to sys.getsizeof documentation:

... Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to. ...

Use Pympler:

>>> import sys
>>> from pympler.asizeof import asizeof
>>>
>>> obj = [1, 2, (3, 4), 'text']
>>> sys.getsizeof(obj)
48
>>> asizeof(obj)
176

Note: The size is in bytes.

edited Oct 26, 2021 at 6:48

brainLoop

4,2602 gold badges12 silver badges13 bronze badges

answered Dec 25, 2013 at 10:06

falsetru

371k69 gold badges770 silver badges660 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user16168 Over a year ago

thank you very much, however it fails with MemoryError

ArtificiallyIntelligent Over a year ago

this is very helpful. most of the answer thinks getsizeof is the true memory usage.

Birdy40 · Accepted Answer · 2019-04-28 23:37:57Z

0

Here is a code snippet that shows two suggested answers. However, the use of pympler gives what I believe to be the correct and more succinct answer to my question. Thank you falsetru :-)

import sys
from pympler.asizeof import asizeof

tuple1 = ('1234','2019-04-27','23.4658')
tuple2 = ('1563','2019-04-27','19.2468')
klist1 = [tuple1]
klist2 = [tuple1,tuple2]
# The results for the following did not answer my question 
print ("sys.getsizeof(klist1): ",sys.getsizeof(klist1))
print ("sys.getsizeof(klist2): ",sys.getsizeof(klist2))    
# The results for the following give a quite reasonable answer
print ("asizeof(klist1): ",asizeof(klist1))
print ("asizeof(klist2): ",asizeof(klist2))

answered Apr 28, 2019 at 23:37

Birdy40

3671 gold badge6 silver badges15 bronze badges

Collectives™ on Stack Overflow

List memory usage

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related