0

I have a bash script that loops through the results of find for a directory containing hundreds of subdirectories and thousands of files and runs a number of commands on each file, mostly sed and grep.

I understand that this obviously computationally intensive. The problem I'm running into is that it uses a ton of memory which isn't freed up when the script is finished. This script is running inside of cygwin. Closing cygwin does not free up any memory, I need to reboot Windows to get it back.

Is there a command I can run (either in Windows or cygwin) to free up memory used by the script?

Improve this question
10
  • I'm no expert, but try purge in cygwin. Works in Linux. Commented Jan 5, 2014 at 15:55
  • @dangenet purge? What is it? There is no such utility in Debian repository. To clear FS cache on GNU/Linux you may use # sync && echo 3 > /proc/sys/vm/drop_caches Commented Jan 5, 2014 at 16:26
  • @dangenet Oh, I found. It’s tool from OS X, is not it? Commented Jan 5, 2014 at 16:31
  • cygwin doesn't recognize the command, dangenet is probably right about it being an OSX utility. Commented Jan 5, 2014 at 16:34
  • 1
    Assuming that you run the script as something like bash ./script, all memory should be given back to the system after the bash-process ends, except (a) cygwin/bash has serious bug or (b) not all subprocesses started are actually really gone. Which measure exactly are you using that lets you know your RAM is all taken and no longer recoverable by Windows? Commented Jan 5, 2014 at 17:26

1 Answer 1

Reset to default
3

Modern operating systems tend to use RAM for two main purposes:

  • Processes
  • Filesystem cache

In GNU/Linux, most of the times, you see a LOT of ram used, but most of it is actually just filesystem cache, which is discarded when memory is needed for real applications, on the fly, without any problems.

Example: my laptop has 7868MB of RAM. In total 6284 megs are used, but only 2441 are used by real programs.

So, to answer your question:

  1. I have no idea (I cant see your code) if there are processes being spawned and kept alive, investigate on this, if you're sure there is none, go ahead.

  2. Lookup in Windows (your OS) which command is used to discard filesystem cache. WARNING: empyting filesystem cache will probably make your computer slow for a moment or two, since operating system will have to re-read some of its components and some executable files from hard disk. (is there a way to tell Windows not to cache filesystem syscalls coming from a certain user/process/etc ?)

Please not that despite I am trying to help, I could be totally missing the point (I have no way to see your code, I'm just guessing basing on "symptoms" you described).

Hope it helps,

Snoopy

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.