python traceback - performance problems

Question

we have a python script that uses a c library to call some low level functions. Because of code structure reasons we want to store the stack trace/call stack before each c call.

To do this we use traceback.extract_stack() to extract the call stack of python. When an exception occurs later on, we use traceback.format_list(abc) on each element to format and print the stack trace.

The problem is that the function extract_stack is too slow. I slows down our code from 1.7 seconds to 11 seconds.

Is there any function to store the stack trace in order to be able to use/print it later? The stacktrace getter must be very fast. The format function can be slow, this is no problem.

Example:

Stack Trace:
  - LXScript: '_LXS:TOOL:RUNLX'
      File "_LXS:TOOL:RUNLX", line 13, in <module>
      File "lxs", line 1, in <module>
  - UNIFACE ACTIVATE: 'ACTQREC_SVC' 'EXECLXSRP'
  - LXScript: '_DATATRT:ACTQREC:EXECUTE@Main:run'
      File "_LXS:TOOL:RUNLX", line 13, in <module>
      File "lxs", line 1, in <module>
      File "<string>", line 63, in run
      File "<string>", line 97, in __doAll
      File "<string>", line 127, in __do
      File "_DATATRT:ACTQREC:EXECUTE", line 7, in do
  - UNIFACE ACTIVATE: 'ACTQ_CSVC' 'EXECBYREC'
  - LXScript: 'TOOL:ACTQ:SYNLAB_DATA@Main:runOnBeforeExec'
      File "TOOL:ACTQ:SYNLAB_DATA", line 1, in <module>
    ImportError: No module named 'localls'

Why not get the stack trace after the exception, so the overhead is only in exceptional circumstances? — Open AI - Opting Out
– Open AI - Opting Out, Commented Jan 5, 2017 at 10:01
maybe you could use the limit parameter to only get a part of the context. In the example above, maybe you don't need all the stack but only the 5 deeper calls. Should speed up operations a bit. — Jean-François Fabre
– Jean-François Fabre ♦, Commented Jan 5, 2017 at 10:05
@PeterWood Because we don't have the exsiting stacktrace when we enter the c interface from the non python code (uniface). — Denny Weinberg
– Denny Weinberg, Commented Jan 5, 2017 at 12:01
By using a limit of 2, I come to 2,1 seconds. Compared to 1,7 and 11 it makes a big difference!! — Denny Weinberg
– Denny Weinberg, Commented Jan 5, 2017 at 12:10

Denny Weinberg · Accepted Answer · 2017-03-15 12:41:14Z

3

I resolved the problem!!!

We can use stt_obj = sys._getframe().f_back to get the "<frame>?" And then we can use estt_obj = traceback.extract_stack(f=stt_obj) and traceback.format_list(estt_obj) to get the stack trace as string list.

edited Mar 15, 2017 at 12:41

answered Jan 5, 2017 at 12:27

Denny Weinberg

2,6161 gold badge25 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Glycerine Over a year ago

Could you show where elem and self exist to read STT_OBJ?

Glycerine Over a year ago

Fantastic thank you - this really works, f is a frame! I wish that was clearer in the docs

Collectives™ on Stack Overflow

python traceback - performance problems

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related