Why does Python's filterwarning module regex not work as expected?

Question

According to the Python warnings documentation the filterwarnings pattern is action:message:category:module:line, and the module part should allow for matching specific modules against a regex. I'm trying to make this work to filter specific warnings for third party library modules, but the feature doesn't seem to work.

Minimal example

A simpler way to reproduce the issue is to create a file

my_module.py

print("\d")

and then run PYTHONWARNINGS="error,ignore:::.*" python -c "import my_module" from the same directory. Clearly the regex .* should match the module name my_module, but somehow it doesn't. The question is why?

Original example

As an example: Importing rosbag package (some package that comes with the Robot Operating System (ROS) distribution) triggers an error when running in "strict" report-warnings-as-errors mode:

$ PYTHONWARNINGS='error' python -c "import rosbag"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/<PATH-TO-ROS-DISTRIBUTION>/lib/python3/dist-packages/rosbag/__init__.py", line 33, in <module>
    from .bag import Bag, Compression, ROSBagException, ROSBagFormatException, ROSBagUnindexedException
  File "/<PATH-TO-ROS-DISTRIBUTION>/lib/python3/dist-packages/rosbag/bag.py", line 1568
    matches = re.match("#ROS(.*) V(\d).(\d)", version_line)
                       ^
SyntaxError: invalid escape sequence \d

That makes sense, it should use a raw string literal.

Now I'm trying to use a module regex to silence the warning from the rosbag module. For the purpose of the example I'm specifying the filterwarnings via PYTHONWARNINGS but other methods like via pytests settings result in the same behavior.

I'm getting the following unexpected behavior:

using PYTHONWARNINGS='error,ignore:invalid escape sequence::' works. Of course that filters all "invalid escape sequence" warnings. So let's add a module regex at the end to be module specific...
using PYTHONWARNINGS='error,ignore:invalid escape sequence::rosbag.*' should do exactly that, but the filter doesn't seem to match. The warning is still reported as error as with plain error.
even using PYTHONWARNINGS='error,ignore:::.*', which should in theory match any module, fails to match. Only removing the regex entirely like PYTHONWARNINGS='error,ignore:::' matches, but of course that is essentially equivalent to just PYTHONWARNINGS='ignore'.

Any ideas why the module regex is not matching at all?! Is this a Python distribution bug? I'm on Python 3.8.10 / Ubuntu 20.04.

The documentation states: "The module field matches the (fully-qualified) module name; this match is case-sensitive." (The syntax for PYTHONWARNINGS is the same as for -W). As far as I can see, this is not a regex, but the full string. I just made a couple of quick tests that confirm that. None of the components use regex-matching. — MrBean Bremen
– MrBean Bremen, Commented Feb 14, 2022 at 20:20
I tried the plain module name as well of course, but it is not matching either. The actual warning documentation states: module is a string containing a regular expression that the module name must match. The docs even show an example error:::mymodule[.*], which demonstrates that it is supposed to be a regex. — bluenote10
– bluenote10, Commented Feb 15, 2022 at 9:32
You are right, that is what is stated - I had overlooked this. With PYTHONWARNINGS I couldn't get any regex to work (while full module names worked, though I haven't tested with your specific module). — MrBean Bremen
– MrBean Bremen, Commented Feb 15, 2022 at 10:13

yut23 · Accepted Answer · 2022-05-11 01:23:30Z

8

+500

As of Python 3.10, this is the intended behavior. Warning filters set through -W and PYTHONWARNINGS currently only match literal strings for the message and module.

This is a known discrepancy in the Python documentation, and is tracked in this CPython issue. There were some discussions and a PR to add regex support, but these ultimately went nowhere. This open PR adds further clarification to the docs.

answered May 11, 2022 at 1:23

yut23

3,09313 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

bluenote10 Over a year ago

Interesting. At least under Python 3.8 I cannot get it to work even when not using a regex and trying to match the module name literally. I.e., PYTHONWARNINGS="error,ignore:::my_module" python -c "import my_module" also errors (same with adding a .py suffix), which is still not what the adapted docs specify.

reddot · Accepted Answer · 2022-08-19 16:40:40Z

0

For ones looking for a workaround to suppress warnings in third-party modules and keep shown for developed packages and/or modules. Say I have developing mypackage. I use a shell wrapper runner script that does following:

FLT="$(cd "./src"; find mypackage/ -name '*.py' | \
    sed -e 's/\.py//g' -e 's/\/__init__//g' -e 's/\//./g' | \
    { while read module; do
        echo "default:::$module"
    done; echo "default:::__main__"; } | paste -sd,)"

export PYTHONWARNINGS="ignore,$FLT"

answered Aug 19, 2022 at 16:40

reddot

1,00511 silver badges17 bronze badges

Collectives™ on Stack Overflow

Why does Python's filterwarning module regex not work as expected?

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related