Do I need to free memory when using big arrays in python

Question

For my python script I need to read in big arrays of up to a billion elements. I read one array in and process it. After processing I do not need the input anymore. Then I read the second array in and process it and again don't need the input anymore. Is it necessary to explicitly free arrays I won't ever call again after processing it or is this done automatically by the garbage collection?

At the moment I only have 4 arrays and not yet any memory issues, will this become problematic if I have 100 of those arrays?

Charlie · Accepted Answer · 2018-04-01 11:51:36Z

1

Have a read over this article on Garbage Collection in Python. It states that you do not explicitly have to free the memory, as this is done automatically.

However with such large objects, you may need to take a bit more care. Use the del keyword to remove all references to an object, making it no longer usable and marking it as collectible to the garbage collector.

Look at the answers to this question, they go into detail about when to use the del keyword.

For example:

# Get the data.
someList = get_list()

# Do your processing.
do_something_with(someList)

# Free the reference to the list.
del someList

Alternatively, instead of deleting and reallocating memory for the list, you could reuse the variable. This is likely to increase the speed of your program due to a decrease in memory allocation.

# Get the data.
someList = get_list()

# Do your processing.
do_something_with(someList)

# Get the data.
someList = get_next_list()

# Do your processing.
do_something_with(someList)

# Free the reference to the list.
del someList

answered Apr 1, 2018 at 11:51

Charlie

3,10224 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jay_ESE Over a year ago

Thank you for your explanation!

Collectives™ on Stack Overflow

Do I need to free memory when using big arrays in python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related