I need to get all the variables in a python program, but I am not allowed to run the program. I can however run it by calling it from a main program and analyze the local variables.
3 Answers
ast is your friend. It allows you to take advantage of the actual Python parser, but not actually execute the parsed code.
That said, this will only give you a first-order approximation - since Python allows dynamic creation of variables, static analysis can't predict 100% of all variable creation.
Comments
You can't do this. The variables in a Python program depend on the execution path. They're all instantiated dynamically. For example.
if path == 0:
x = 1
else:
z = 1
w = 2
And let's say path is something read out of a file or from the user. Can you specify how many variables will be defined after this block is executed?
A more pathological case would be something like this
if path == 0:
locals()['x'] = 1
This will create a local variable x but it's using an assignment statement so you can't even catch it using the ast module.
5 Comments
Tadeck
But you can surely extract list of variables from such script / module / program. As a result you will get a list of variable names, but without guarantee they are all variables accessible within that module, and without guarantee that they are defined in every place of the script. So "you can't do this" is not entirely right.
Noufal Ibrahim
There are hairier situations which will break that as well. I've updated my answer.
machine yearning
+1 for that tricky stuff, I didn't see that coming like the bad guys from Home Alone 2.
Tadeck
@NoufalIbrahim: Yes, of course there are worse cases, but your example is incorrect. First of all you should use
globals() instead of locals(), as globals() is a dictionary containing global variables (and global variables in Python means the variables accessible by the whole module). In addition, locals() documentation says: "Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.".Noufal Ibrahim
Thanks for the pointer. I just wanted to point out that "creation" of variables might not be due to assignments.
Please correct me if I'm wrong, but I'd say anything with an AST is likely overkill here. Isn't a variable anything that appears on the left-hand side of an assignment statement?
Edit: read the manual here and here
It would seem that "variables" are a specific instance of identifiers to which values can be bound. In the context of the Python execution model, these are known as "objects" which are bound to "names".
If a name is bound in a block, it is a local variable of that block. If a name is bound at the module level, it is a global variable. (The variables of the module code block are local and global.) If a variable is used in a code block but not defined there, it is a free variable.
And here's the sweet stuff (my emphasis):
The following constructs bind names: formal parameters to functions, import statements, class and function definitions (these bind the class or function name in the defining block), and targets that are identifiers if occurring in an assignment, for loop header, in the second position of an except clause header or after as in a with statement. The import statement of the form from ... import * binds all names defined in the imported module, except those beginning with an underscore. This form may only be used at the module level.
So if you can cover those bases, you're set! :)
I assume that any code you analyze with this is going to be fairly rudimentary Python code; otherwise you may want to go with an AST as suggested above.
3 Comments
Noufal Ibrahim
Needn't be. Check my answer for a particularly pathological case.
machine yearning
Sure, but I assumed that under his circumstances he can ignore "pathalogical" cases where, for example, he's using function which returns a list of variable references. Anyway, I'm aware that this is in general a difficult or unsolvable problem in many languages; however as with most difficult or unsolvable problems it becomes almost trivial if you can sufficiently constrain the domain.
Noufal Ibrahim
Agreed. Although, it's good to be aware about the impossibility of a 100% solution.
class <class_name>,def <func_name>and<var_name> = <sth_else>not enclosed in class or function definitions (so no indentation or indentation not within class/function definition). Is it a homework?from <module> import *), you will need to look into<module>module and get__all__list containing the list of variable names imported using the asterisk (if there is no such list, you will need to perform similar action for the imported module, as for the one in which you are importing).