Valgrind on Python2.4 : Huge amount of memory 'possibly lost'

Question

I have written some code which works without errors. The code uses MySQLdb for (frequent) database access and uses 4 different databases. Also a couple of log files are generated and logging module is used. The real worry is that when running Valgrind, i get the following

==7840== LEAK SUMMARY:
==7840==    definitely lost: 29 bytes in 1 blocks
==7840==    indirectly lost: 0 bytes in 0 blocks
==7840==      possibly lost: 1,104,793 bytes in 8,865 blocks
==7840==    still reachable: 70,684 bytes in 2,194 blocks
==7840==         suppressed: 0 bytes in 0 blocks

The biggest leak is

==7840== 393,216 bytes in 1 blocks are possibly lost in loss record 1,585 of 1,585
==7840==    at 0x4005903: malloc (vg_replace_malloc.c:195)
==7840==    by 0x204929E: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x2054833: PyString_InternInPlace (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x20A0362: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209FB15: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x20A0075: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209FB15: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x20A0068: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x20A04CC: PyMarshal_ReadObjectFromString (in /usr/lib /libpython2.4.so.1.0)
==7840==    by 0x20A1D20: PyMarshal_ReadLastObjectFromFile (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209AA63: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209CB7E: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209D9B2: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209DE71: ??? (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x209E087: PyImport_ImportModuleEx (in /usr/lib/libpython2.4.so.1.0)
==7840==    by 0x207DB2D: ??? (in /usr/lib/libpython2.4.so.1.0)

So you see, It shows almost 1MB of memory as possibly lost. Is there some real problem with my code, or is it use of MySQLdb that gives such behaviour. Also to minimise it, should I explicilty free up objects (inc file, database connections) or is there a python module that I could look into?

Why are you using a 7-year-old python version? That's almost as bad using PHP3 instead of PHP5 (ok, Python 2.4 is still much better than PHP3 compared to the current versions, but anyway) — ThiefMaster
– ThiefMaster, Commented Apr 19, 2011 at 11:44
I use Scientific Linux and that's the version they ship it with. — RedBaron
– RedBaron, Commented Apr 19, 2011 at 11:48
Did you use suppression file for valgrind provided with Python distribution (Misc/valgrind-python.supp)? — Denis Otkidach
– Denis Otkidach, Commented Apr 19, 2011 at 12:00

Denis Otkidach · Accepted Answer · 2011-04-19 12:15:18Z

5

Python uses own memory allocator on top of malloc which causes problems when using valgrind. See Misc/README.valgrind for detailed explanation. Assuming you are not going to rebuild Python the solution is to use Misc/valgrind-python.supp as suppression file and uncomment the lines in it that suppress the warnings for PyObject_Free and PyObject_Realloc.

answered Apr 19, 2011 at 12:15

Denis Otkidach

33.5k8 gold badges86 silver badges102 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

RedBaron Over a year ago

I had thought so.... but I also ran a 'naive' test(if test can be used)...I started system monitor and noted amount of memory currently used...then ran my code.... it took up some 5 MB of memory.... then when the code exited, the memory usage came back to pre-program level...I'm not a 100% sure, but if there were a big memory leak the memory usage would not have fallen back...right??

Denis Otkidach Over a year ago

This depends on what you mean by running code. When process exits its memory is freed even if there were memory leaks. While python may not return memory to system (reserve it for future use) when objects are destroyed and the code has no memory leaks. The only reliable way to detect memory leaks requires recompilation without PyMalloc.

Collectives™ on Stack Overflow

Valgrind on Python2.4 : Huge amount of memory 'possibly lost'

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related