Can Python list assignment by slice cause overflow?

Question

About a decade ago, when I played around with C++ in my high school, I learnt about pointers, and memory overflows. In C++, sometimes, using a pointer to expand (or contract) an array in place can cause it to run over (or under) the memory allotted, and cause weird behaviour. I am interpreting slice assignment in Python to be something similar to assignment by pointers:

a[:] = list(range(10000))  # Similar to using pointers in C++,
# because memory location remains unchanged.

So how does Python avoid overflows (or underflows)?

Of course, in C++, we initialize each array to a specified size, and pointer-based assignments violating that size are horrible programming practice.

"I am interpreting slice assignment in Python to be something similar to assignment by pointers" It isn't. More like "pointers with guard rails that will not let you overrun/underrun". Like in Java. You can only pass a legal index, not any arbitrary positive or negative index. — smci
– smci, Commented Apr 4, 2018 at 20:34
Python manages its memory behind the scenes to prevent memory overflows. You may get an out of memory exception but that's still well defined behavior. — Mark Ransom
– Mark Ransom, Commented Apr 4, 2018 at 20:34
It is a mistake to assume that just because you are modifying an existing list, your new storage must go into the same space in memory. — khelwood
– khelwood, Commented Apr 4, 2018 at 20:35
Incidentally, the things you're saying about C++ indicate that you need to stop using so many arrays and get used to vectors, which also resize automatically. — user2357112
– user2357112, Commented Apr 4, 2018 at 20:36
In (pure) Python you don't need to worry about memory at all. At least to a point where you run out of said memory, but that's another problem. — ForceBru
– ForceBru, Commented Apr 4, 2018 at 20:36

Terry Jan Reedy · Accepted Answer · 2018-04-04 20:40:03Z

1

Python lists are resizable arrays. Most of the time, there is extra space to allow for growth (and shrinkage) without reallocation. When more space is needed, or too much is wasted, the internal space is reallocated. This can result in a MemoryError if more space is needed and not available. The details depend on the implementation, OS, and OS settings.

answered Apr 4, 2018 at 20:40

Terry Jan Reedy

19.3k3 gold badges44 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Kartik Over a year ago

Can you link to a source or two where I can read and learn more? Thanks.

Terry Jan Reedy Over a year ago

stackoverflow.com/questions/3917574/…; laurentluce.com/posts/python-list-implementation For more, search SO for [python] list implementation or web for python list implementation, which is what I did.

Collectives™ on Stack Overflow

Can Python list assignment by slice cause overflow?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related