Profiling a python 3.6 module from the command line

Question

I have a python project where I execute the app as a module using the -m flag. So something like:

python -m apps.validate -i input.mp4

Now, I want to profile it using the command line. So the inline examples suggest invoking cProfile itself a module. However, I cannot do something like:

python -m cProfile apps.validate -i input.mp4

However, this results in the error "No such file or directory". I cannot just go to the apps directory and launch validate.py due to relative imports.

Is there a way to profile a module on the command line?

There's a "New in version 3.7" note in the cProfile online documentation that says a -m option was added to cProfile in that version. This is in addition to the Python interpreter's own -m option, This means that something like python -m cProfile -m apps.validate -i input.mp4 ought to work (if you're using Python 3.7+). — martineau
– martineau, Commented Jan 31, 2019 at 17:41
I see. Unfortunately, my stuff needs python 3.6 as some of the other libraries are not working with 3.7. I guess I will need to use it in code. — Luca
– Luca, Commented Jan 31, 2019 at 17:53
You could look at the source code for profile (and cProfile) and see how support for the new -m option was added. It might even be possible to use that version of it with an earlier version of the Python interpreter (depending on what other changes were made). — martineau
– martineau, Commented Jan 31, 2019 at 18:02

kuangchen · Accepted Answer · 2019-08-23 08:57:34Z

8

Instead of running cProfile in shell, maybe you can use cProfile in your python script by adding some code in apps.validate or creating a new script and import apps.validate like this. Maybe some typo below :)

import cProfile
import sys

def run_validate(args): 
    # run your apps.validate code with shell arguments args here
    pass

if __name__ == '__main__':
    pr = cProfile.Profile()
    pr.enable()
    run_validate(*sys.argv)
    pr.disable()
    pr.print_stats()

then just run the original script: python -m apps.validate -i input.mp4

answered Aug 23, 2019 at 8:57

kuangchen

961 silver badge2 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jay Over a year ago

pr.dump_stats(filename) would save it to a file.

Collectives™ on Stack Overflow

Profiling a python 3.6 module from the command line

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related