I am running an externally compiled C++ program using RunProcess, and I encountered a strange thing: when I run the RunProcess command many times, it takes increasingly more and more time for identical evaluation. How comes? Can I do something to prevent this?
Minimal example of this behavior
We can see this behavior running simply date (as noted by Jason B.):
ListPlot@Table[First@AbsoluteTiming@Do[RunProcess["date"], 5], 400]
We measure the time it takes to run RunProcess["date"] five times over 400 experiments. The code yields the following:
As you can see, despite running an identical command, the running time gets longer the more times we do it.
Question
So I have two questions:
- Why is this happening?
- Is there a better way to use external code if I need its output many times? I realise that running new process over and over again as a function call is not the most efficient way to go, but is there a better way if I am using a code which I don't understand and don't want to touch in any way?
System specifications
- System: macOS Monterey 12.1
- Mathematica version: 12.3.1
(Original example of this question)
I took the following C++ code
#include <algorithm>
using namespace std;
int main(int argc, char const *argv[])
{
double product;
for (int i = 0; i < 1000000; ++i)
{
product = rand() * rand();
}
return 0;
}
I compiled it using g++ clutter.cpp -std=c++11 -o clutter. In Mathematica, I ran the following code:
times = {}
Module[{timeStart, time},
timeStart = Now;
Do[RunProcess["<PATH>/clutter"], 10];
time = Now - timeStart;
AppendTo[times, time];
]& /@ Range[1000];
That is, I ran 1000$\times$10 calls to RunProcess, and measured the time for each 10 calls. (If you want to run this, you might want to lower the numbers, this took 254 seconds together). The times were as in the figure:

As you can see, the time more than doubles throughout the experiment.


AppendToin order to isolate the contributions. See: 127644. $\endgroup$Now-tsbe evaluated beforeAppendTomaking the append process outside of the timing? I see your point, and I'll make it clearer, but considering other experiments I have done, I am pretty sure that is not the problem. $\endgroup$ListPlot@Table[First@AbsoluteTiming@Do[RunProcess["date"], 5], 400]$\endgroup$