installing tensorflow_transform and apache_beam on Datalab

Question

I'm going over these example from google-cloud Coursera courses, and although they worked till a few weeks ago, I can't install tf.transform or apache_beam on Datalab anymore.

https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/feateng/tftransform.ipynb

https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/06_structured/4_preproc_tft.ipynb

When installing tensorflow_transform I get the following errors:

%bash
pip install --upgrade --force tensorflow_transform==0.6.0

twisted 18.7.0 requires PyHamcrest>=1.9.0, which is not installed. datalab 1.1.3 has requirement six==1.10.0, but you'll have six 1.11.0 which is incompatible. gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible. proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible. apache-airflow 1.9.0 has requirement bleach==2.1.2, but you'll have bleach 1.5.0 which is incompatible. apache-airflow 1.9.0 has requirement funcsigs==1.0.0, but you'll have funcsigs 1.0.2 which is incompatible. google-cloud-monitoring 0.28.0 has requirement google-cloud-core<0.29dev,>=0.28.0, but you'll have google-cloud-core 0.25.0 which is incompatible. proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible. pandas-gbq 0.3.0 has requirement google-cloud-bigquery>=0.28.0, but you'll have google-cloud-bigquery 0.25.0 which is incompatible. googledatastore 7.0.1 has requirement httplib2<0.10,>=0.9.1, but you'll have httplib2 0.11.3 which is incompatible. googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible. Cannot uninstall 'dill'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

Salma R · Accepted Answer · 2018-09-09 01:16:06Z

2

The tensorflow version on my Datalab instance was 1.4. I had to add this one line of code to update tensorflow to 1.10.1

%bash
pip install --upgrade --force-reinstall pip==10.0.1
pip install tensorflow==1.10.1
pip install tensorflow_transform

my environment:

apache-airflow==1.9.0
apache-beam==2.6.0
tensorflow==1.10.1
tensorflow-metadata==0.9.0
tensorflow-tensorboard==0.4.0rc3
tensorflow-transform==0.8.0

edited Sep 9, 2018 at 1:16

answered Sep 8, 2018 at 0:22

Salma R

2042 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lak · Accepted Answer · 2018-08-30 03:34:06Z

1

The current version of Datalab uses TensorFlow 1.8, so please change the notebook cell in question to:

%bash
pip uninstall -y google-cloud-dataflow
pip install --upgrade --force tensorflow_transform==0.8.0 apache-beam[gcp]

I've updated and checked in the two notebooks linked above.

Another problem might be that you are using Python 2. Datalab by default now uses Python 3 and your pip install (above) happens in Python 3 even if the kernel is Python 2 because %%bash opens up a new shell in which the conda activate of Python 2 has not happened.

To make sure the pip install happens in Python 2, change your pip install of apache-beam[gcp] as follows:

%%bash
source activate py2env
conda install -y dill pytz  # do this for all the distutils complaints 
pip uninstall -y google-cloud-dataflow
pip install --upgrade --force tensorflow_transform==0.8.0 apache-beam[gcp]

edited Aug 30, 2018 at 3:34

answered Aug 28, 2018 at 22:26

Lak

4,30624 silver badges38 bronze badges

2 Comments

Salma R Over a year ago

Hi Lak, thanks for you answer. I can now import apache_beam, but for tensorflow_transform, i get this error:

AttributeErrorTraceback (most recent call last) <ipython-input-1-a151d3498be0> in <module>()       1 import datetime       2 import apache_beam as beam ----> 3 import tensorflow_transform as tft  AttributeError: 'module' object has no attribute 'uint64'

Salma R Over a year ago

now the tensorflow_transform import works, but not apache_beam. getting some other version incompatibility errors, too long to post here in the comment. Some are "httplib2 , PyHamcrest, six, oauth2client, bleach, funcsigs"

Collectives™ on Stack Overflow

installing tensorflow_transform and apache_beam on Datalab

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related