How to run python3 on databricks?

Question

I try to run my machine-learning code on databricks(community version) and need to use the Orange3 data-mining library. However, when I tried to create the orange3 library, it gives an error like this:

Error: java.lang.RuntimeException: Installation failed with message: Collecting orange3 Downloading Orange3-3.8.0.tar.gz (34.0MB) Complete output from command python setup.py egg_info: Orange requires Python >= 3.4 ---------------------------------------- mesg: ttyname failed: Inappropriate ioctl for device Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-DnpIcU/orange3/

I guess that means I need python3 to install the library, but the default version on databricks is python2. I wonder How I can change it to python3.

I also searched on databricks forum and find this post(https://docs.databricks.com/api/latest/examples.html#create-a-python-3-cluster).

However, I still don't quite understand how to create the python3-cluster. Does it mean I need to use Cluster API to customize a cluster, which need to charge extra money?

Appreciate any help!!

Silvio · Accepted Answer · 2019-10-28 01:55:29Z

3

Python 3 is now the default when creating clusters and there's a UI dropdown to switch between 2 or 3 on older runtimes. 2 will no longer be supported on Databricks Runtime 6+.

The docs give more details on the various Python settings.

In regards to specific versions, it depends on the Runtime you're using.

For instance:

5.5 LTS runs Python 3.5
5.5 LTS ML runs Python 3.6
5.5 with Conda runs Python 3.7
6.0 and 6.1 both run 3.7

edited Oct 28, 2019 at 1:55

answered Jan 6, 2018 at 4:03

Silvio

4,22726 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Silvio Over a year ago

btw, you can clone the terminated cluster to create a new python 3 cluster without having to use the API (restarts are not supported on community edition)

Josh Rosen Over a year ago

Update: there is now a UI for selecting the Python version when creating a cluster! See docs.databricks.com/user-guide/clusters/python3.html for more details.

Eric Bellet Over a year ago

How can user python 3.5 or python 3.7?

Collectives™ on Stack Overflow

How to run python3 on databricks?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related