Linux c application memory usage

Question

I have C Linux application which continuously allocates and frees memory (around 200 alloc/free per sec) using malloc, calloc, realloc & free functions. Even though all allocated memory are freed (verified by wrapping *alloc and free), the VmSize, VmRSS & VmData numbers are keep on increasing and finally application is getting killed by OOM killer.

Why the VmSize, VmRSS & VmData are keep on increasing? if it is Memory management issue, any pointers to avoid this?

I saw this Problem usage memory in C, but the answers are not explaining the OOM behavior.

I strongly suggest you run it through valgrind rather than relying on macro'd log output. It is remarkably efficient at pointing out where potential leaks are, and it sounds like you have one, whether you agree or not, its worth checking. — WhozCraig
– WhozCraig, Commented May 16, 2013 at 10:11
The first tool I used was valgrind, but it didn't help as there is no leak from application code — Jibin Scaria
– Jibin Scaria, Commented May 16, 2013 at 10:14
Are you using any 3rd-party libs in your code outside of the runtime library implementation? — WhozCraig
– WhozCraig, Commented May 16, 2013 at 10:15
no only standard libs, libc, xml2, pthread, z, m, rt etc. One more thing I forgot to mention is that the allocations are of random sizes — Jibin Scaria
– Jibin Scaria, Commented May 16, 2013 at 10:18
The random sizes shouldn't matter unless your random sizes are statistically ever-increasing, as you would end up with a likely pretty putrid heap fragmented to the heavens. — WhozCraig
– WhozCraig, Commented May 16, 2013 at 10:29

Community · Accepted Answer · 2017-05-23 11:49:16Z

0

You should first use valgrind (to debug potential hard to find memory leaks or misbehavior). Don't forget to compile with gcc -Wall -g (then, when it works, use -Wall -O); of course, improve your code till no warnings are given.

You could probably (if the algorithms fit) try to (usefully) allocate memory zone of e.g. either power of two, or 3 times a power of two [perhaps minus 2 or 3 words]; at least try to avoid too many different random sizes of allocation.

You may want to try using Boehm's conservative garbage collector - i.e. replacing all your malloc with GC_MALLOC (or GC_MALLOC_ATOMIC & strdupwith GC_STRDUP), your free with GC_FREE, etc...

At least for testing purposes, use setrlimit(2) perhaps thru the bash ulimit builtin. You want RLIMIT_AS - possibly with RLIMIT_DATA (setting these limits sensibly avoids the OOM killer and makes your mmap -called by malloc- fail when memory is exhausted).

You may want to compile with GCC 4.8 which accepts -fsanitize=address.

You could also implement your own application specific garbage collector (read that wikipage, it gives you insights and terminology); a mark & compact algorithm will fight fragmentation.

See also this question about memory fragmentation. Look into the Plug tool.

edited May 23, 2017 at 11:49

CommunityBot

11 silver badge

answered May 16, 2013 at 11:36

Basile Starynkevitch

231k18 gold badges323 silver badges578 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jibin Scaria Over a year ago

Thanks for the useful links. But I decided to tryout with a custom memory allocator to avoid allocations and frees.

Basile Starynkevitch Over a year ago

Feel free to upvote and/or accept my answer, and tell us if your allocator solves the issue (maybe not)...

Collectives™ on Stack Overflow

Linux c application memory usage

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related