Trying to install mpi4py, getting error: Failed to build wheels

Question

edit: i tried to install this requirements file contains these and the errors occur after

Collecting mpi4py==3.1.5
  Downloading mpi4py-3.1.5.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 4.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
...
Building wheels for collected packages: mpi4py, pycocotools, diffdist, mup, deepspeed
  Building wheel for mpi4py (pyproject.toml) ... error

the file

    pillow==9.4.0
    opencv-python==4.8.1.78
    pyyaml==6.0.1
    json_tricks==3.17.3
    yacs==0.1.8
    scikit-learn==1.3.1
    pandas==2.0.3
    timm==0.4.12
    numpy==1.26.4
    einops==0.8.0
    fvcore==0.1.5.post20221221
    transformers==4.34.0
    sentencepiece==0.1.99
    ftfy==6.1.1
    regex==2023.10.3
    nltk==3.8.1
    mpi4py==3.1.5
    vision-datasets==0.2.2
    cython==3.0.2
    pycocotools==2.0.7
    diffdist==0.1
    #pyarrow==13.0.0
    #cityscapesscripts==2.2.2
    #shapely==1.8.0
    scikit-image==0.21.0
    mup==1.0.0
    accelerate==0.23.0
    kornia==0.7.0
    deepspeed==0.10.3
    #wandb==0.15.12
    infinibatch==0.1.1
    open-clip-torch==2.26.1
    git+https://github.com/MaureenZOU/detectron2-xyz.git
    #gradio==3.42.0
    #torch==2.3.1 #2.0.1 
    #torchvision==0.15.2 
    #torchaudio==2.0.2
    #torch==2.1.0
    #torchvision==0.16.0

I am trying to install the mpi4py library, but it is giving me the following error:

**(biomedparse) vusynhathoang@Nevermore:/mnt/d/AI/BiomedParse$ pip install mpi4py**
Collecting mpi4py
  Using cached mpi4py-4.0.1.tar.gz (466 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: mpi4py
  Building wheel for mpi4py (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for mpi4py (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [316 lines of output]
      running bdist_wheel
      running build
      running build_src
      using Cython 3.0.11
      cythonizing 'src/mpi4py/MPI.pyx' -> 'src/mpi4py/MPI.c'
      running build_py
      creating build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/run.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__init__.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__main__.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/bench.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/typing.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      creating build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_base.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/aplus.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_core.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__init__.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__main__.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/util.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/pool.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/server.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      creating build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/dtlib.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/sync.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/__init__.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/pool.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/pkl5.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/__main__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/typing.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__init__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/MPI.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/run.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/bench.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/py.typed -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/libmpi.pxd -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/MPI.pxd -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__init__.pxd -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/MPI_api.h -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/MPI.h -> build/lib.linux-x86_64-cpython-39/mpi4py
      creating build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/pycapi.h -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/futures/__main__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__init__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/pool.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/util.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/server.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/aplus.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_base.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_core.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/util/__init__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/pool.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/sync.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/pkl5.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/dtlib.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      running build_ext
      MPI configuration: [mpi] from 'mpi.cfg'
      MPI C compiler:    /usr/bin/mpicc
      MPI C++ compiler:  /usr/bin/mpicxx
      checking for MPI compile and link ...
      /usr/bin/mpicc -pthread -B /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/vusynhathoang/miniconda3/envs/biomedparse/include -I/home/vusynhathoang/miniconda3/envs/biomedparse/include -fPIC -O2 -isystem /home/vusynhathoang/miniconda3/envs/biomedparse/include -fPIC -I/home/vusynhathoang/miniconda3/envs/biomedparse/include/python3.9 -c _configtest.c -o _configtest.o
      success!
      removing: _configtest.c _configtest.o
      /usr/bin/mpicc -pthread -B /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/vusynhathoang/miniconda3/envs/biomedparse/include -I/home/vusynhathoang/miniconda3/envs/biomedparse/include -fPIC -O2 -isystem /home/vusynhathoang/miniconda3/envs/biomedparse/include -fPIC -I/home/vusynhathoang/miniconda3/envs/biomedparse/include/python3.9 -c _configtest.c -o _configtest.o
      /usr/bin/mpicc -pthread -B /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat _configtest.o -o _configtest
      /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat/ld: warning: libopen-rte.so.40, needed by /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so, not found (try using -rpath or -rpath-link)
      /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat/ld: warning: libopen-pal.so.40, needed by /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so, not found (try using -rpath or -rpath-link)


**AT LEAST 50 ERRORs like these in between.**


      /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to `opal_hash_table_set_value_uint64'
      /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to `MPIR_being_debugged'
      /home/vusynhathoang/miniconda3/envs/biomedparse/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to `opal_list_t_class'
      collect2: error: ld returned 1 exit status
      failure.
      removing: _configtest.c _configtest.o
      error: Cannot link MPI programs. Check your configuration!!!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mpi4py
Failed to build mpi4py
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (mpi4py)

I tried installing it with both pip and conda, I am still getting ther error. I also tried the following answers:

But still no luck!

I need to know the way you installed openmpi to recreate the problem. — Lubomyr
– Lubomyr, Commented Jan 11 at 22:27

Lubomyr · Accepted Answer · 2025-01-11 22:25:51Z

0

Looks like it's a problem with your openmpi installation.

Possible solution:

remove openmpi
run pip install mpi4py
run suggested command to install openmpi
again pip install mpi4py.

answered Jan 11 at 22:25

Lubomyr

3492 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Otis Willie Jan 12 at 10:50

sudo apt-get remove --purge openmpi* libopenmpi-dev sudo apt-get update sudo apt-get install openmpi-bin openmpi-common libopenmpi-dev pip install mpi4py these right? :C it still gave me this /home/vsnh/miniconda3/envs/biomedparse/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to a thousand files collect2: error: ld returned 1 exit status failure. removing: _configtest.c _configtest.o error: Cannot link MPI programs. Check your configuration!!! ERROR: Failed building wheel for mpi4py

Otis Willie Jan 12 at 10:50

Failed to build mpi4py ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (mpi4py)

Otis Willie · Accepted Answer · 2025-01-12 11:33:58Z

0

I USED THIS AND IT DID WORK

conda install -c conda-forge mpi4py openmpi

answered Jan 12 at 11:33

Otis Willie

94 bronze badges

Comments

Lubomyr · Accepted Answer · 2025-01-12 13:28:29Z

0

Create a python -m venv .venv environment and try to install the package there.

answered Jan 12 at 13:28

Lubomyr

3492 silver badges12 bronze badges

Collectives™ on Stack Overflow

Trying to install mpi4py, getting error: Failed to build wheels

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related