C++ new/delete: deconstruct object from pointer array might not release memory?

Question

I am having a strange C++ new/delete question which is at some circumstance I delete (deconstruct) an object from pointer array but the memory seems not to be released. Please see the code and comments below.

If run all the steps from Step-1 to Step-6, the memory is NOT released after Step-4 or Step-5.

If run Step-1, Step-2, Step-3, Step-4 and Step-6 (comment and skip Step-5), the memory is NOT released after Step-4 or Step-6.

If run Step-3, Step-4 and Step-6 (comment and skip Step-1, Step-2 and Step-5), the memory can be released as expected after Step-4 or Step-6.

If run all the steps from Step-1 to Step-6, but at Step-3 the new string size is larger than previous size at Step-1, the memory can be released as expected after Step-4 or Step-5 or Step-6.

So in general, if NOT run Step-1 and Step-2, everything seems fine. But If run Step-1 and Step-2 before, something wired happens that the memory is NOT released unless the new string size is larger than previous size at Step-1.

#include <iostream>
#include <list>
#include <unistd.h>

struct MyClass {
    std::string str;
    MyClass() {}
    ~MyClass() {}
};

int main(int argc, char* argv[]) {
    std::list<MyClass*> mylist;

    // Step-1: create 100 MyClass pointer array,
    // then construct 20 MyClass objects with 1MB's string for each
    // and put them into a list
    MyClass** pt1 = new MyClass*[100];
    for (int i = 0; i < 20; ++i) {
            std::string tmp_str(1024*1024, 'a'); // 1MB
            pt1[i] = new MyClass();
            pt1[i]->str = tmp_str;
            mylist.push_back(pt1[i]);
    }
    std::cout << "Step-1: creating done: " << mylist.size() << std::endl;
    sleep(10); // now check the memory usage of this process, it should use
               // about 20MB memory


    // Step-2: delete all the MyClass objects from the list,
    // then delete pt1
    while (1) {
            std::list<MyClass*>::iterator it = mylist.begin();
            if (it == mylist.end())
                    break;
            delete *it;
            mylist.erase(it);
    }
    delete [] pt1;
    pt1 = NULL;
    std::cout << "Step-2: deleting done, left: " << mylist.size()
            << std::endl;
    sleep(10); // now check the memory usage (RSS) of this process, 
               // it should reduce about 20MB memory 


    // Step-3: create another 100 MyClass pointer array,
    // then construct 10 MyClass objects with 1MB's string for each
    // and put them into a list
    MyClass** pt2 = new MyClass*[100];
    for (int i = 0; i < 10; ++i) {
            std::string tmp_str2(1024*1024, 'b');
            pt2[i] = new MyClass();
            pt2[i]->str = tmp_str2;
            mylist.push_back(pt2[i]);
    }
    std::cout << "Step-3: creating done: " << mylist.size() << std::endl;
    sleep(10); // now check the memory usage (RSS) of this process
               // it should use about 10MB memory


    // Step-4: delete 4 MyClass objects from the list, NOT all of them
    int j = 0;
    while (1) {
            std::list<MyClass*>::iterator it = mylist.begin();
            if (it == mylist.end() || ++j == 5)
                    break;
            delete *it;
            mylist.erase(it);
    }
    std::cout << "Step-4: deleting done, left: " << mylist.size()
            << std::endl;
    sleep(10); // now check the memory usage (RSS) of this process,
               // we expect it should reduce about 4MB memory,
               // but it still uses about 10MB memory and seems
               // no memory is freed.


    // Step-5: delete all the left MyClass objects from the list
    while (1) {
            std::list<MyClass*>::iterator it = mylist.begin();
            if (it == mylist.end())
                    break;
            delete *it;
            mylist.erase(it);
    }
    std::cout << "Step-5: deleting done, left: " << mylist.size()
            << std::endl;
    sleep(10); // now check the memory usage (RSS) of this process,
               // we expect it should reduce about 10MB memory,
               // but it still uses about 10MB memory and seems
               // no memory is freed.


    // Step-6: delete pt2
    delete [] pt2;
    pt2 = NULL;
    std::cout << "Step-6: deleting array done" << std::endl;
    sleep(10); // now check the memory usage (RSS) of this process,
               // if we run Step-5 before, then the memory will reduce
               // about 10MB,
               // but if we didn't run Step-5 before, then the memory will 
               // still be about 10MB and seems no memory is freed. 
    return 0;
}

Kirill Kobelev · Accepted Answer · 2016-12-21 03:08:38Z

1

There is a difference between the memory allocated on the user level (new or malloc) and memory allocated by the OS for the process. There is layer of code in the C++ library itself and memory management code inside the process provided by the OS (on Windows it is called NT Heap). This means that when you release memory with delete or free, this intermediate layer of code may immediately release this memory to OS or may not, expecting that you will want to allocate it again. This makes sense because there is good chance that this will improve the overall performance.

This behavior is system specific. On Windows it may look one way, on say iOS very different.

Also note that the way how you store pointers in your application, i.e. in array of pointers, in a vector or list does not matter. OS has no idea of this.

edited Dec 21, 2016 at 3:08

answered Dec 21, 2016 at 3:02

Kirill Kobelev

10.6k6 gold badges34 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sam Varshavchik · Accepted Answer · 2016-12-21 03:04:20Z

0

now check the memory usage (RSS) of this process,

This gives you how much memory the operating system has allocated to your process.

Deleting objects, with delete (this also applies to free()ed memory that was malloc()ed also, but for the purpose of this answer I will only reference new and delete) does not necessarily return the released memory to the operating system. You will not necessarily see this reflected in the process's RSS.

The shown code appears to be in order. Eventually, it diligently deletes all newed objects. This does not guarantee, however, that the process will actually discard the allocated memory, and return it to the operating system. Typically, released memory gets placed into an internal pool of available memory blocks, that's maintained by the C++ library. When objects gets created via new, additional memory is acquired from the operating system only if no sufficiently large contiguous block of memory can be found in the pool of available memory that can be recycled in this manner.

In other words, there's nothing wrong with the shown code.

Traditional operating systems typically allocate memory to the running process in moderately to large-sized chunks; anywhere from several kilobytes to several megabytes at a time. Creating a small object, a few bytes long, typically results in one such chunk getting allocated from the operating system, a small piece of it getting used to construct the new object, then the remaining memory placed in an internal pool of available memory, and used to construct any objects that get newed later.

Conversely, deleteing an object places its memory into an internal pool of available memory. If sufficient number of objects get deleteed that frees up a contiguous block that comprises one or more pages of memory, it is possible that those pages will be released back to the operating system, but that's not typical, and rare. Once a process allocates memory, it stays gobbled up by the process until it terminates.

answered Dec 21, 2016 at 3:04

Sam Varshavchik

119k6 gold badges109 silver badges164 bronze badges

3 Comments

David-Z Over a year ago

Thanks, Sam. Like you said, "Typically, released memory gets placed into an internal pool of available memory blocks, that's maintained by the C++ library". When will those memory blocks be released back to OS? From my testing program, If I try to sleep as long as possible after Step-4, those memory seems not to be released back to OS.

Sam Varshavchik Over a year ago

It's entirely up to the C++ library. The last paragraph in my answer offers one possibility.

David-Z Over a year ago

Thanks, Sam. I also find similar info as you mentioned: cplusplus-soup.com/2010/01/…

Collectives™ on Stack Overflow

C++ new/delete: deconstruct object from pointer array might not release memory?

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related