Python fails to import package from bazel git_repository() dependency

Question

I have two git repos with python code.

https://github.com/SurferJeffAtGoogle/synthtool/tree/bazel contains two build targets:

~/gitrepos/synthtool$ bazel query ...
//tests:synthtool_test
//synthtool:synthtool

https://github.com/SurferJeffAtGoogle/nodejs-vision/tree/bazel contains one build target:

~/gitrepos/nodejs-vision$ bazel query ...
//:synth

//tests:synthtool_test and //:synth both depend on //synthtool:synthtool.

I can run the test and the test code successfully imports synthtool:

~/gitrepos/synthtool$ bazel test ...
INFO: Analyzed 2 targets (4 packages loaded, 230 targets configured).
INFO: Found 1 target and 1 test target...
INFO: Elapsed time: 7.477s, Critical Path: 7.07s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
//tests:synthtool_test                                                   PASSED in 7.0s

Executed 1 out of 1 test: 1 test passes.

However, when I try running //:synth it fails to import synthtool:

~/gitrepos/nodejs-vision$ bazel run //:synth
Starting local Bazel server and connecting to it...
INFO: Analyzed target //:synth (17 packages loaded, 272 targets configured).
INFO: Found 1 target...
INFO: Deleting stale sandbox base /usr/local/google/home/rennie/.cache/bazel/_bazel_rennie/dadb642c5feb5ae86ab3
1dcb8a48c54a/sandbox
Target //:synth up-to-date:
  bazel-bin/synth
INFO: Elapsed time: 2.555s, Critical Path: 0.03s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Traceback (most recent call last):
  File "/usr/local/google/home/rennie/.cache/bazel/_bazel_rennie/dadb642c5feb5ae86ab31dcb8a48c54a/execroot/nodejs_vision/bazel-out/k8-fastbuild/bin/synth.runfiles/nodejs_vision/synth.py", line 18, in <module>
    import synthtool.gcp as gcp
ModuleNotFoundError: No module named 'synthtool.gcp'
rennie@rennie:~/gitrepos/nodejs-vision$

Both the targets have the exact same line of code:

import synthtool.gcp as gcp

It succeeds for the tests but not for //:synth.

I see that the right files were fetched and end up in the blaze output directories, but synth.py fails to import them:

~/gitrepos/nodejs-vision$ find -L bazel-* -name gcp
bazel-bin/synth.runfiles/nodejs_vision/external/synthtool/synthtool/gcp
bazel-bin/synth.runfiles/synthtool/synthtool/gcp
bazel-nodejs-vision/external/synthtool/synthtool/gcp
bazel-nodejs-vision/bazel-out/k8-fastbuild/bin/synth.runfiles/nodejs_vision/external/synthtool/synthtool/gcp
bazel-nodejs-vision/bazel-out/k8-fastbuild/bin/synth.runfiles/synthtool/synthtool/gcp
bazel-out/k8-fastbuild/bin/synth.runfiles/nodejs_vision/external/synthtool/synthtool/gcp
bazel-out/k8-fastbuild/bin/synth.runfiles/synthtool/synthtool/gcp

Why is that? How do I fix it?

And strangely, running python in the bazel output directory succeeds: ~/gitrepos/nodejs-vision/bazel-nodejs-vision$ python synth.py — Jeffrey Rennie
– Jeffrey Rennie, Commented Feb 6, 2020 at 0:22

Jon Brandvein · Accepted Answer · 2020-02-06 22:29:45Z

This looks like an instance where the repo name (@synthtool) is shadowing the package name (//synthtool). The runfiles tree of your failing target looks something like this:

<runfiles>/synthtool/synthtool/foo.py
     ^          ^        ^       ^
   root         |        |       |
           repo root     |       |
                      package    |
                              modules

The Python rules add each repo root to the PYTHONPATH, as well as the runfiles root itself. (See for instance #7067.) This means that both the first and second path fragments above are on your PYTHONPATH. You want import synthtool to be resolved relative to the repo root, so that it grabs the package, but instead it's getting resolved relative to the runfiles root, so it tries to import the repo root.

You should be able to mitigate this by renaming the package and/or repository.

Collectives™ on Stack Overflow

Python fails to import package from bazel git_repository() dependency

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related