6

I'm facing a very strange issue on Python 3.6. In the middle of my code, I call import pdb; pdb.set_trace() to debug some code.

And then I'm not able to debug properly, for instance:

(Pdb) abc = 3
(Pdb) [abc for _ in range(2)]
*** NameError: name 'abc' is not defined
(Pdb) [abc, abc]
[3, 3]

It seems like whenever I use list comprehensions, there is an issue of variable not defined. However, if I call the debugger right after I open Python, I do not observe this behavior, everything runs fine.

Any ideas why I'm having this issue?

Improve this question
1
  • There is work around to execute the command likc this. eval("(lambda: x)()", vars()) Commented Jan 29, 2019 at 4:54

4 Answers 4

Reset to default
11

This is filed as b.p.o. issue #21161, but closed as "won't fix".

The root issue is exactly what user2357112's answer says: Comprehensions1 work by defining and then running a hidden nested function, but functions defined in pdb inside a frame aren't real nested functions and can't access variables from the frame's scope.

There are workarounds (some of which are mentioned in the b.p.o issue, or places linked from that issue), but they're all just clever variations on one of these:

  • (lambda abc: [abc for _ in range(2)])(abc) (In other words, define a function and explicitly pass the locals' values as arguments instead of capturing them.)
  • [abc_ for abc_ in [abc] for _ in range(2)] (In other words, use the fact that the outermost iterable is an argument.2)
  • [loc['abc'] for loc in [locals()] for _ in range(2)] (In other words, use locals()['name'] instead of name, and get the locals passed as above.)
  • Use exec plus any of the known just-as-horrible workarounds for comprehensions in exec.
  • Don't use a comprehension, as user2357112 suggests.

1. In Python 2.x, this is not true for list comprehensions, only set and dict comprehensions and generator expressions. But Python 3 changed list comprehensions to make them consistent with the others.

2. The hidden nested function for a comprehension captures almost all names from the enclosing scope—but the outermost iterable (the one in the first for) is special; it's actually a parameter to the hidden function, and the expression is evaluated in the enclosing scope and passed as the argument value.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

[locals()['abc'] for _ in range(2)] doesn't work - the workarounds involving locals() need to call locals in the outer scope, not inside the comprehension's scope.
@user2357112 Oops, you're right; you need to pass in a locals dict; I'll edit.
So confusing when you are trying to debug an issue, and the debugger works differently to your the code. Now you have to debug the debugger. I hope this gets fixed.
7

This happens because list comprehensions are (mostly) evaluated in a nested scope, and nested scopes created in pdb can't access the local variables of the scope being inspected. They can access globals, though, and when you launch pdb immediately after opening Python, you're running it in a global scope, so the abc you create is global.

This also happens with list comprehensions in exec and in class statements. Unfortunately, there isn't really a better workaround than "don't use list comprehensions there".

Improve this answer

2 Comments

Is this specific to Python 3? I do not observe the issue with Python 2.
@dhokas: List comprehensions aren't evaluated in a nested scope in Python 2. I believe you'll still see it with genexps and dict/set comprehensions in Python 2, though.
6

You could try the following code:

(Pdb) !import code; code.interact(local=vars())
Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 17:14:51) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> [abc for _ in range(2)]

Use Ctrl-D to return to the regular pdb prompt.

Improve this answer

Comments

3

Similar to Panfeng Li's answer, but a simpler way to do this is to use interact in pdb.

What follows is an example of pdb failing and interact working in the same context.

Python 3.6.3 (default, Oct  4 2017, 06:09:05)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def do_comp():
...   abc = 3
...   import pdb; pdb.set_trace()
...
>>> do_comp()
--Return--
> <stdin>(3)do_comp()->None
(Pdb) [abc for _ in range(2)]
*** NameError: name 'abc' is not defined
(Pdb) interact
*interactive*
>>> [abc for _ in range(2)]
[3, 3]
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.