22

Let us suppose that we have some problem and at least two solutions for it. And what we want to achieve - is to compare effectiveness for them. How to do this? Obviously, the best answer is: do tests. And I doubt there's a better way when it comes to language-specific questions (for example "what is faster for PHP: echo 'foo', 'bar' or echo('foo'.'bar')").

Ok, now we'll assume that if we want to test some code, it's equal to test some function. Why? Because we can wrap that code to function and pass it's context (if any) as it's parameters. Thus, all we need - is to have, for example, some benchmark function which will do all stuff. Here's very simple one:

function benchmark(callable $function, $args=null, $count=1)
{
   $time = microtime(1);
   for($i=0; $i<$count; $i++)
   {
      $result = is_array($args)?
                call_user_func_array($function, $args):
                call_user_func_array($function);
   }
   return [
      'total_time'   => microtime(1) - $time,
      'average_time' => (microtime(1) - $time)/$count,
      'count'        => $count
   ];
}

-this will fit our issue and can be used to do comparative benchmarks. Under comparative I mean that we can use function above for code X, then for code Y and, after that, we can say that code X is Z% faster/slower than code Y.

The problem

Ok, so we can easily measure time. But what about memory? Our previous assumption "if we want to test some code, it's equal to test some function" seems to be not true here. Why? Because - it's true from formal point, but if we'll hide code inside function, we'll never be able to measure memory after that. Example:

function foo($x, $y)
{
   $bar = array_fill(0, $y, str_repeat('bar', $x));
   //do stuff
}

function baz($n)
{
   //do stuff, resulting in $x, $y
   $bee = foo($x, $y);
   //do other stuff
}

-and we want to test baz - i.e. how much memory it will use. By 'how much' I mean 'how much will be maximum memory usage during execution of function'. And it is obvious that we can not act like when we were measuring time of execution - because we know nothing about function outside of it - it's a black box. If fact, we even can't be sure that function will be successfully executed (imagine what will happen if somehow $x and $y inside baz will be assigned as 1E6, for example). Thus, may be it isn't a good idea to wrap our code inside function. But what if code itself contains other functions/methods call?

My approach

My current idea is to create somehow a function, which will measure memory after each input code's line. That means something like this: let we have code

$x = foo();
echo($x);
$y = bar();

-and after doing some thing, measure function will do:

$memory = memory_get_usage();
$max    = 0;

$x = foo();//line 1 of code
$memory = memory_get_usage()-$memory;
$max    = $memory>$max:$memory:$max;
$memory = memory_get_usage();

echo($x);//second line of code
$memory = memory_get_usage()-$memory;
$max    = $memory>$max:$memory:$max;
$memory = memory_get_usage();

$y = bar();//third line of code
$memory = memory_get_usage()-$memory;
$max    = $memory>$max:$memory:$max;
$memory = memory_get_usage();

//our result is $max

-but that looks weird and also it does not answer a question - how to measure function memory usage.

Use-case

Use-case for this: in most case, complexity-theory can provide at least big-O estimation for certain code. But:

  • First, code can be huge - and I want to avoid it's manual analysis as long as possible. And that is why my current idea is bad: it can be applied, yes, but it will still manual work with code. And, more, to go deeper in code's structure I will need to apply it recursively: for example, after applying it for top-level I've found that some foo() function takes too much memory. What I will do? Yes, go to this foo() function, and.. repeat my analysis within it. And so on.
  • Second - as I've mentioned, there are some language-specific things that can be resolved only by doing tests. That is why having some automatic way like for time measurement is my goal.

Also, garbage collection is enabled. I am using PHP 5.5 (I believe this matters)

The question

How can we effectively measure memory usage of certain function? Is it achievable in PHP? May be it's possible with some simple code (like benchmark function for time measuring above)?

5
  • As you said this highly depends on the memory management and it is very important if Garbage Collection is used: php.net/manual/de/features.gc.php Commented Nov 14, 2013 at 8:47
  • @ZoolWay you're right. I've updated. I'm using PHP 5.5 and gc is enabled, of course Commented Nov 14, 2013 at 8:48
  • fyi: echo(a,b); is a parse error while echo a,b; is not. Commented Nov 14, 2013 at 9:21
  • Yes, copy-pasted from echo('foo'.'bar') - fixed, thank you Commented Nov 14, 2013 at 9:23
  • 1
    You could use a PROFILER or something like NEW RELIC to monitor exactly that, afaik. Those tools trace memory etc. (per class, file, function, thread etc.) on a lower software level, so you don't even have to touch your code. Commented Nov 14, 2013 at 9:41

5 Answers 5

11

After @bwoebi proposed great idea with using ticks, I've done some researching. Now I have my answer with this class:

class Benchmark
{
   private static $max, $memory;

   public static function memoryTick()
   {
      self::$memory = memory_get_usage() - self::$memory;
      self::$max    = self::$memory>self::$max?self::$memory:self::$max;
      self::$memory = memory_get_usage();
   }

   public static function benchmarkMemory(callable $function, $args=null)
   {
      declare(ticks=1);
      self::$memory = memory_get_usage();
      self::$max    = 0;

      register_tick_function('call_user_func_array', ['Benchmark', 'memoryTick'], []);
      $result = is_array($args)?
                call_user_func_array($function, $args):
                call_user_func($function);
      unregister_tick_function('call_user_func_array');
      return [
         'memory' => self::$max
      ];
   }
}

//var_dump(Benchmark::benchmarkMemory('str_repeat', ['test',1E4]));
//var_dump(Benchmark::benchmarkMemory('str_repeat', ['test',1E3]));

-so it does exactly what I want:

  • It is a black box
  • It measures maximum used memory for passed function
  • It is independent from context

Now, some background. In PHP, declaring ticks is possible from inside function and we can use callback for register_tick_function(). So my though was - to use anonymous function which will use local context of my benchmark function. And I've successfully created that. However, I don't want to affect global context and so I want unregister ticks handler with unregister_tick_function(). And that is where troubles are: this function expects string to be passed. So you can not unregister tick handler, which is closure (since it will try to stringify it which will cause fatal error because there's no __toString() method in Closure class in PHP). Why is it so? It's nothing else, but a bug. I hope fix will be done soon.

What are other options? The most easy option that I had in mind was using global variables. But they are weird and also it is side-effect which I want to avoid. I don't want to affect context. But, really, we can wrap all that we need in some class and then invoke tick function via call_user_func_array(). And call_user_func_array is just string, so we can overcome this buggy PHP behavior and do the whole stuff succesfully.

Update: I've implemented measurement tool from this. I've added time measurement and custom callback-defined measurement there. Feel free to use it.

Update: Bug, mentioned in this answer, is now fixed, so there's no need in trick with call_user_func(), registered as tick function. Now closure can be created and used directly.

Update: Due to feature request, I've added composer package for this measurement tool.

Sign up to request clarification or add additional context in comments.

2 Comments

Why not register_tick_function([$this, 'memoryTick']); and unregister_tick_function([$this, 'memoryTick']);?
Because of the bug it was not possible (now that is fixed already)
8
declare(ticks=1); // should be placed before any further file loading happens

That should say already all what I will say.

Use a tick handler and print on every execution the memory usage to a file with the file line with:

function tick_handler() {
    $mem = memory_get_usage();
    $bt = debug_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS, 2)[0];
    fwrite($file, $bt["file"].":".$bt["line"]."\t".$mem."\n");
}
register_tick_function('tick_handler'); // or in class: ([$this, 'tick_handler']);

Then look at the file to see how the memory varies in time, line by line.

You also can parse that file later by a separate program to analyse peaks etc.

(And to see how the possible peaks are by calling internal functions, you need to store the results into a variable, else it'll be already freed before the tick handler will measure the memory)

4 Comments

in order to make this code work, i needed to remove the [1] from the line that begins with $bt and add [0] to the fwrite calls to $bt like $bt[0]['line'] (also changed the " to ').
@MikeiLL yes, then you're using some elder version of PHP.
thank you. PHP 5.5.3 (cli) (built: Sep 18 2013 14:31:13) . Is there a standard for how often servers should be updated?
Here is the updated link to the documentation for what they are talking about: php.net/manual/en/control-structures.declare.php @PauloFreitas Your link is dead.
2

You can use the XDebug and a patch for XDebug which provides memory usage information

if this is not possible, you can always use memory_get_peak_usage() which i think would fit better than memory_get_usage()

4 Comments

No, I can not use memory_get_usage() because if I will use comparisons consecutive, it will fail. Sample: var_dump(benchmarkMemory('str_repeat', ['test',1E4])); var_dump(benchmarkMemory('str_repeat', ['test',1E2])); - it will take upper limit from first call and skip anything for second because first call already reached peak limit
Even if you call gc_collect_cycles in between?
How it will help? Logically, I see no difference, but tested just in case - see fiddle (0 is well-expected result for second call)
and what about XDebug with the patch?
0

This might not be exactly what you are looking for, but you could probably use XDebug to get at that information.

Comments

0

Just stumbled across

http://3v4l.org/

Although they don't provide details on how the benchmarks, respectively the taking of measures is implemented - don't think many people have over 100 PHP versions running in parallel on a VM beneath their desk ;)

1 Comment

I believe they're using some external features (so, profilers) - which I'm able to use too as well, but, in terms of the question, I was interesting of - how to measure memory properly from inside PHP itself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.