tensorflow build fails with "missing dependency" error

Question

I'm completely new to bazel and tensorflow so the solution to this may be obvious to someone with some experience. My bazel build of tensorflow fails with a "missing dependency" error message. Here is the relevant sequence of build commands and output:

(tf-gpu)kss@linux-9c32:~/projects> git clone --recurse-submodules https://github.com/tensorflow/tensorflow tensorflow-nogpu
Cloning into 'tensorflow-nogpu'...
remote: Counting objects: 16735, done.
remote: Compressing objects: 100% (152/152), done.
remote: Total 16735 (delta 73), reused 0 (delta 0), pack-reused 16583
Receiving objects: 100% (16735/16735), 25.25 MiB | 911.00 KiB/s, done.
Resolving deltas: 100% (10889/10889), done.
Checking connectivity... done.
Submodule 'google/protobuf' (https://github.com/google/protobuf.git) registered for path 'google/protobuf'
Cloning into 'google/protobuf'...
remote: Counting objects: 30266, done.
remote: Compressing objects: 100% (113/113), done.
remote: Total 30266 (delta 57), reused 0 (delta 0), pack-reused 30151
Receiving objects: 100% (30266/30266), 28.90 MiB | 1.98 MiB/s, done.
Resolving deltas: 100% (20225/20225), done.
Checking connectivity... done.
Submodule path 'google/protobuf': checked out '0906f5d18a2548024b511eadcbb4cfc0ca56cd67'
(tf-gpu)kss@linux-9c32:~/projects> cd tensorflow-nogpu/
(tf-gpu)kss@linux-9c32:~/projects/tensorflow-nogpu> ./configure 
Please specify the location of python. [Default is /home/kss/.venv/tf-gpu/bin/python]: 
Do you wish to build TensorFlow with GPU support? [y/N] 
No GPU support will be enabled for TensorFlow
Configuration finished
(tf-gpu)kss@linux-9c32:~/projects/tensorflow-nogpu> bazel build -c opt //tensorflow/tools/pip_package:build_pip_package                                                        
Sending SIGTERM to previous Bazel server (pid=8491)... done.
....
INFO: Found 1 target...
ERROR: /home/kss/.cache/bazel/_bazel_kss/b97e0e942a10977a6b42467ea6712cbf/external/re2/BUILD:9:1: undeclared inclusion(s) in rule '@re2//:re2':
this rule is missing dependency declarations for the following files included by 'external/re2/re2/perl_groups.cc':
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/stddef.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/stdarg.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/stdint.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/x86intrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/ia32intrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/mmintrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/xmmintrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/mm_malloc.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/emmintrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/immintrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/fxsrintrin.h'
  '/usr/lib64/gcc/x86_64-suse-linux/4.8/include/adxintrin.h'.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 144.661s, Critical Path: 1.18s
(tf-gpu)kss@linux-9c32:~/projects/tensorflow-nogpu>

The version of bazel I'm using is release 0.1.4, I'm running on openSUSE 13.2. I confirmed that the header files do exist which is probably expected:

(tf-gpu)kss@linux-9c32:~/projects/tensorflow-nogpu> ll /usr/lib64/gcc/x86_64-suse-linux/4.8/include/stddef.h
-rw-r--r-- 1 root root 13619 Oct  6  2014 /usr/lib64/gcc/x86_64-suse-linux/4.8/include/stddef.h

Note for anyone who finds this question:

Use Damien's answer below except that you have to use --crosstool_top rather than --crosstool. Also if you are building for GPU acceleration you will also need to modify the CROSSTOOL file in the tensorflow repo like:

(tf-gpu)kss@linux-9c32:~/projects/tensorflow-gpu> git diff third_party/gpus/crosstool/CROSSTOOL | cat
diff --git a/third_party/gpus/crosstool/CROSSTOOL b/third_party/gpus/crosstool/CROSSTOOL
index dfde7cd..b63f950 100644
--- a/third_party/gpus/crosstool/CROSSTOOL
+++ b/third_party/gpus/crosstool/CROSSTOOL
@@ -56,6 +56,7 @@ toolchain {
   cxx_builtin_include_directory: "/usr/lib/gcc/"
   cxx_builtin_include_directory: "/usr/local/include"
   cxx_builtin_include_directory: "/usr/include"
+  cxx_builtin_include_directory: "/usr/lib64/gcc"
   tool_path { name: "gcov" path: "/usr/bin/gcov" }

   # C(++) compiles invoke the compiler (as that is the one knowing where

Damien Martin-Guillerez · Accepted Answer · 2016-02-08 16:42:00Z

4

You should tweak the C++ compiler.

To do so, here's the best way to proceed: edit the file tools/cpp/CROSSTOOL (https://github.com/bazelbuild/bazel/blob/master/tools/cpp/CROSSTOOL) from your package path directory (should be in ~/.bazel/base_workspace, can be found with bazel info package_path) to add a line cxx_builtin_include_directory: /usr/lib64/gcc around line 100 (see https://github.com/bazelbuild/bazel/blob/master/tools/cpp/CROSSTOOL#L101).

Then echo "build --crosstool=//tools/cpp:toolchain" >>~/.bazelrc and then retries to build.

Sorry for the mess, we are working on making C++ toolchain work better out of the box.

answered Feb 8, 2016 at 16:42

Damien Martin-Guillerez

2,38014 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Damien Martin-Guillerez Over a year ago

(another way to do it would be to compile a custom version of bazel with that change in the CROSSTOOL file).

Keith Over a year ago

Many thanks for your help. I tried the modifications you suggest (in the answer not the comment) and the compilation worked. I had to make a minor change to "--crosstool_top" rather than "--crosstool" which isn't recognized by the build command.

tmsimont Over a year ago

This got me pretty far, but I still hit a wall: ` C++ compilation of rule '//tensorflow/python:py_func_lib' failed: gcc failed: error executing command` -- any idea why?

Keith Over a year ago

@tmsimont this is just a shot in the dark, but did you install the python development libs?

tmsimont Over a year ago

Hi Keith -- that's a solid shot. I found out that i needed to update numpy in order for this to work, but now I'm still stuck with what looks like another gcc issue.. I opened up an issue on github

Dmitry Baksheev · Accepted Answer · 2017-03-10 08:13:30Z

Bazel complaints of system header files because compiler uses -MD (as opposed to -MMD) flag when generating dependences. While using -MD is reasonable for an environment that changes often, listing dependency on system header files causes the 'missing dependency declarations' errors.

What helped me was converting the '-MD' flag into '-MMD' flag in the compiler wrapper files third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc.tpl just before 'subprocess.call([CPU_COMPILER]...)':

cpu_compiler_flags = ['-MMD' if flag == '-MD' else flag for flag in cpu_compiler_flags]

and third_party/sycl/crosstool/computecpp.tpl, similar place:

computecpp_device_compiler_flags = ['-MMD' if flag == '-MD' else flag for flag in computecpp_device_compiler_flags]

Collectives™ on Stack Overflow

tensorflow build fails with "missing dependency" error

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related