Let's say I've already imported a python module in the interpreter. How can I get the abstract syntax tree of the imported module (and any functions and classes within it) within the interpreter? I don't want to have to re-parse the source files. Thanks!
asked Jan 27, 2011 at 2:35
Heinrich Schmetterling
6,91413 gold badges43 silver badges56 bronze badges
-
4"I don't want to have to re-parse the source files." Why not? That's a pretty silly restriction. Is there a reason?S.Lott– S.Lott2011-01-27 02:53:29 +00:00Commented Jan 27, 2011 at 2:53
-
it's inefficient, seems like there should be a way to do so with the already-loaded module.Heinrich Schmetterling– Heinrich Schmetterling2011-01-27 03:14:30 +00:00Commented Jan 27, 2011 at 3:14
-
I don't think you can get an AST without re-parsing. I believe Python parses the file, converts it to bytecode, and then throws away the AST, all at import time. In fact, if a pyc or pyo file is present, it may never build an AST at all, instead directly loading bytecode. I bet you can find and disassemble the Python bytecode if you want, but that's probably less helpful.Walter Mundt– Walter Mundt2011-01-27 03:36:29 +00:00Commented Jan 27, 2011 at 3:36
-
4"it's inefficient"? Really? The cost is microscopic and almost unmeasurable. What are you doing?S.Lott– S.Lott2011-01-27 03:39:38 +00:00Commented Jan 27, 2011 at 3:39
Add a comment
|
2 Answers
Maybe you find some inspiration in this recipe:
A function that outputs a human-readable version of a Python AST.
Python 2 option (as compiler is removed in Python 3): Use compiler combined with inspect (which, of course, still uses the source):
>>> import compiler, inspect
>>> import re # for testing
>>> compiler.parse(inspect.getsource(re))
Module('Support for regular expressions (RE). \n\nThis module provides ...
Python 3:
>>> import ast, inspect
>>> ast.parse(inspect.getsource(re))
<_ast.Module at 0x7fdcbd1ac550>
3 Comments
Jordan Dimov
In Python 3 the
compiler module has been deprecated. Use ast.parse instead.
pschanely
Also consider using
textwrap.dedent(inspect.getsource(re)) which will handle functions defined inside other functions or classes. (otherwise the extra leading whitespace will make the parser mad)
Nathan Chappell
@pschanely your comment is why I actually came here
There doesn't seem to be a way to get an AST except from source code. If you explain why you don't want to re-parse the source files, maybe we can help with that.
2 Comments
Heinrich Schmetterling
it just seemed unnecessary and unwieldy to me and i don't necessarily know the paths.
Ned Batchelder
If you've imported a module m, then the file it came from is
m.__file__. It should be straightforward to re-read the source. As The MYYN points out, inspect has methods that are helpful also.