0

We are building some tutorial notebooks (for an average technical audience). We have problems on Google Colab due to incompatibilities of some of the packages we use and the preinstalled versions of some packages (scipy for instance). We can't fix all those incompatibilities but we know the set of versions without conflicts (we use it to create a conda environment and a virtualenv to run everything).

So the simplest option would to create a custom kernel on Google Colab based on this set of versions. Is it possible on the free version of Google Colab ?

I found posts from 2017-2019 stating that it was not possible but maybe it is now.

Notes:

  • I found some advanced solutions(here or here) but this is too complicated for our audience.
  • Similarly, local kernels are too complicated.
Improve this question
7
  • Jupyter is open source and the Jupyter community has solutions to this if your computational needs are modest? MyBinder can provide temporary sessions without the need to log in or create any account. You can even use an environment.yml file to create an environment definition. See here and here for examples and details. (Be aware the notes can be a little outdated.) I don't have a very current conda version I can point you are but these two versions using requirements.txt ... Commented Apr 7 at 17:48
  • <continued> may give you some ideas: This one has scipy listed and you can launch it from here. Here shows you can use Python 3.12 if you'd like newer Python even. (That is based on the demo for JupyterLab 4.3 here which uses whatever is default Python for the MyBinder infrastructure at the time.) Commented Apr 7 at 17:51
  • Thanks for your suggestion. In fact we also test our tutorials on MyBinder (which takes advantage of the conda environment) but it is sometime slow and the VM have less computational power. So we want to offer participants the ability to use Colab. Commented Apr 8 at 8:29
  • That's great to hear and totally understandable. (A lot of folks aren't aware of it.) Except when there is an occasional hiccup because of working out some unique aspects of the new system, you should find it much better in terms of speed these days. Commented Apr 8 at 14:45
  • 1
    Interesting! We will advertise MyBinder a bit more then. Commented Apr 10 at 12:57

1 Answer 1

Reset to default
2

Solution

Here is an approach to use specific Python Version customizing google colab kernel through Miniconda.

Workable Timestamp: 4/10/2025

I will demonstrate below how to create a Python 3.9 environment within google colab using Miniconda.

Here are steps

  1. Create the google colab file under specific folder.

    Here I create custom_env_colab/py-3.9.ipynb upder the root folder of gcloud for demonstrating.

  2. In the file py-3.9.ipynb, we create four cells like below

    Cell 1: Specify Parameters

    # Parameters
%env PYTHONPATH =                             
YOUR_GOOGLE_DRIVE_FOLDER = "custom_env_colab" 
PYTHON_VERSION = "3.9"                        

# Connect to google drive place
from google.colab import drive
drive.mount('/content/drive')

# Forward to working dir
import os
os.chdir(f"/content/drive/MyDrive/{YOUR_GOOGLE_DRIVE_FOLDER}")

    Cell 2: Build virtulenv with Miniconda

    !wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
!sudo chmod +x Miniconda3-latest-Linux-x86_64.sh
!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local
!conda update --yes conda

import sys
sys.path.append(f'/usr/local/lib/python{PYTHON_VERSION}/site-packages')

!conda create -n myenv python={PYTHON_VERSION} --yes
!apt-get install libtinfo5 # Fix

    Cell 3: Attach to virtualenv and install package

    %%shell
eval "$(conda shell.bash hook)"
conda activate myenv

python -m pip install scikit-learn
# python -m pip install -r your-requirements.txt

    Cell 4: Execute your own code like below:

    %%shell
eval "$(conda shell.bash hook)"
conda activate myenv

python your-code.py

However this method has limitations.

With this method, you need to activate the created environment at the start of each cell execution, which is equivalent to running your command in the terminal.

i.e. Cell will be like below to execute your code:

%%shell
eval "$(conda shell.bash hook)"
conda activate myenv

python main.py
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

I'm not sure which part of your answer works and which part is not working. Could you clarify this? In any case, it seems VERY complicated.
I revised the answer to avoid confusion caused by the ideas I had already tried. I included the results of those attempts in the comment below, hoping it’s clearer now.
I thought about possibly setting up a Jupyter Notebook server inside the created Miniconda environment, and then letting users connect to that server. This way, they could use the Jupyter Notebook interface on a customized Python version within Google Colab. However, after trying it out, I found that it's not possible to access Google Colab’s global IP, so it didn’t work.
Your method might work but it's too complicated for my case. I will test it in a clean project to accept your answer if it works.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.