ipyparallel with mpi cannot find engines

Question

I'm trying to get a basic ipyparallel environment working using mpi4py as described in the ipyparallel documentation. After starting the ipcluster, I load ipython and try to create a client but it has no IDs and accessing the directview returns a NoEnginesRegistered exception.

Steps I take to get to this point:

Create a new environment: conda create --name=ipyparallel and source activate ipyparallel
Install ipyparallel and mpi4py: conda install ipyparallel mpi4py
Create a new ipython profile: ipython profile create --parallel --profile=mpi
Edit ~/.ipython/profile_mpi/ipcluster_config.py and add c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'
Launch cluster with ipcluster start --profile=mpi

Then I launch ipython and run the following:

import ipyparallel as ipp
c = ipp.client(profile="mpi")
c[:] # <-- NoEnginesRegistered exception

Step 5 reports that "Engines appear to have started successfully" and I can see that a process named "mpiexec" is running. Strangely, I tried these same steps on another machine with the same OS and there it worked with no problems. What am I missing?

m3wolf · Accepted Answer · 2016-12-20 20:55:17Z

0

I solved the problem for anyone coming here with a similar problem. During installation I had added the notebook extension to jupyter's global config. Not sure why that caused this problem but it's fixed now. Outside of a conda environment, I ran:

sudo pip install ipyparallel
sudo jupyter nbextension disable --py ipyparallel
sudo jupyter nbextension uninstall --py ipyparallel
sudo pip uninstall ipyparallel

and then inside the conda environment I can connect to the ipyparallel engines.

answered Dec 20, 2016 at 20:55

m3wolf

1144 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ipyparallel with mpi cannot find engines

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related