Install local python libary in azure ml studio environment

Question

I am trying to run a Machine Learning training script I wrote in azure by using:

env = Environment.from_conda_specification("experiment_env", "my_environment.yml")
script_config = ScriptRunConfig(source_directory=experiment_folder,
                                script='path_to_file/classifier_train.py',
                                arguments=arguments,
                                environment=env,
                                docker_runtime_config=DockerConfiguration(use_docker=True)

my problem is, that in order to run the code of classifier_train.py I need to call

pip install -e .

since I need code from a local package I wrote (the setup.py for this package is in the experiment folder).

Can anyone tell me how I can run pip install -e . after azure installed all other package when building the environment and before running the classifier_train.py script?

Sairam Tadepalli · Accepted Answer · 2022-10-27 06:13:24Z

1

To install local libraries on azure ML environment, we need to create the Data Science VM on the local machine and connect it with the workspace subscription details.

Create ML studio workspace and download the JSON file which consists of the details of the workspace need to be connected to the VM.

Create a python virtual environment on the local machine
Either virtualenv or conda is fine
Activate the virtual environment
Install the Azure Machine Learning Python SDK
Configure the Azure ML Studio with local machine

Open the Jupyter Notebook installed and configured. Create a virtual environment.
conda install notebook ipykernel – enable all the ipykernal things
ipython kernel install --user --name <myenv> --display-name "Python (myenv)" – creating a kernel
Launch the jupyter notebook server

Now we need to get the ARM template by following the procedure taken from MS Docs

Using the following code block create Windows DSVM

az vm create --resource-group YOUR-RESOURCE-GROUP-NAME --name YOUR-VM-NAME --image microsoft-dsvm:dsvm-windows:server-2016:latest --admin-username YOUR-USERNAME --admin-password YOUR-PASSWORD --authentication-type password

Using the following code block create Ubuntu DSVM

az vm create --resource-group YOUR-RESOURCE-GROUP-NAME --name YOUR-VM-NAME --image microsoft-dsvm:linux-data-science-vm-ubuntu:linuxdsvmubuntu:latest --admin-username YOUR-USERNAME --admin-password YOUR-PASSWORD --generate-ssh-keys --authentication-type password

Create conda environment:

conda create -n py310 python=310

Activate the environment and install the libraries locally which are directly impacted in azure ml platform

conda activate py310
pip install azure-ai-ml

answered Oct 27, 2022 at 6:13

Sairam Tadepalli

1,6891 gold badge5 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mayool Over a year ago

Thanks for the awesome answer! I am sure your solution would work... I solved the problem myself a couple of days ago by adding: - pip: - -e git+github.com/myrepo#egg=myenvname to the environment.yml file

Collectives™ on Stack Overflow

Install local python libary in azure ml studio environment

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related