python install without pip on ubuntu

Question

I need to install a Python package in a custom Docker container that I'm building from the official 'ubuntu' Docker image, so I want to minimize how much space this uses. Python3 installs fine and runs, but for some reason, pip is not included.

So I installed via apt install python3-pip, this works but it is a whopping 300 megs and takes a couple of minutes to install (apparently because it installs a sh*load of stuff to build binary packages from gcc etc).

Of course I could uninstall python3-pip from the image after installing the dependencies I want, and additionally use apt autoremove to get rid of 299 megs. However this takes another minute.

So although the above works, it significantly increases the build time of my Docker image. So I tried to see if there was a way of installing the dependency without pip:

I tried downloading the dependency's .tar.gz from PyPI, extracted, and tried python3 setup.py install, but this gets me an odd error:

Traceback (most recent call last):
  File "setup.py", line 59, in <module>
    from distutils import log
ImportError: cannot import name 'log'

I thought perhaps I need to install setuptools, or upgrade distutils.

I tried to use get-pip.py from the official site but that failed too:

Traceback (most recent call last):
  File "get-pip.py", line 20890, in <module>
    main()
  File "get-pip.py", line 197, in main
    bootstrap(tmpdir=tmpdir)
  File "get-pip.py", line 82, in bootstrap
    import pip._internal
  File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/__init__.py", line 40, in <module>
  File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/cli/autocompletion.py", line 8, in <module>
  File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/cli/main_parser.py", line 8, in <module>
  File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/cli/cmdoptions.py", line 17, in <module>
  File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/locations.py", line 10, in <module>
ImportError: cannot import name 'sysconfig'

which is very weird because if I start python3, import sysconfig works fine.

I also tried apt install python-pyyaml (the dependency I need in my Docker image) but that doesn't seem to exist.

So I'm out of options.

pip actually really needs all this (the C compiler and other build tools) because pip packages might contain C code or link with binary packages. — digitalarbeiter
– digitalarbeiter, Commented Nov 2, 2018 at 21:52
Yes I know why these are necessary to cover all possible build/install cases, just seems overkill for one or 2 libraries which don't need C (AFAIK). There should be a "light" version of pip that is for pure Python packages, it would probably be 1/100 the size. — Oliver
– Oliver, Commented Nov 3, 2018 at 3:30

Floating Octothorpe · Accepted Answer · 2020-08-26 12:34:22Z

4

I ran into a similar issue and wanted to give an alternative solution.

On Ubuntu 20.04 build-essential and python3-dev are recommended packages for python3-pip, therefore you can use the --no-install-recommends option to skip them:

RUN apt update -y && \
    apt install python3 python3-pip --no-install-recommends -y && \
    apt clean

This took my image from 420MB to 165MB, and obviously the build time was also quicker.

Note: this will work fine for pure-Python packages, but you will likely need build-essential and python3-dev if you want to compile anything

Useful links

answered Aug 26, 2020 at 12:34

Floating Octothorpe

5555 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

digitalarbeiter · Accepted Answer · 2018-11-02 21:45:33Z

2

The Debian and Ubuntu package for PyYAML is called python-yaml, not python-pyyaml.

sudo apt install python-yaml

or

sudo apt install python3-yaml

respectively.

(It seems to be common in Debian/Ubuntu package names to drop any additional "py" prefix that a Python package might have: it's apt install python-tz instead of python-pytz, too. They don't seem to like the py-redundancy.)

answered Nov 2, 2018 at 21:45

digitalarbeiter

2,35514 silver badges16 bronze badges

3 Comments

Oliver Over a year ago

+1 for a way to completely obviate the need for installing pip, I did not know that! but my question still stands: if you have a pure Python package, is there a way to install without full pip like just setuptools or something, I was hoping the errors in my question give hints about something I did wrong.

Oliver Over a year ago

I added answer that address that addresses the question for cases where a python-something apt install does not exist

Addison Klinke Over a year ago

In Ubuntu 18.04 this is not providing the Python module for me. Has the package name been changed since 2018?

Anthon · Accepted Answer · 2018-11-03 06:03:11Z

I have been installing pip from the pip-get.py in Docker (Ubuntu) containers for a few years without problem. For me it is the best way to not-get pip-out-of-date warnings or (at some point, some time ago) SSL related errors.

So the second part of your answer is close, but your python install seems a bit too minimal, you need sysconfig as provided by python-distutils. You can try this rather minimal Dockerfile:

FROM ubuntu:latest

MAINTAINER Anthon

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y \
  python3 \
  python3-distutils \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# this gets you the latest pip
COPY pip/get-pip.py /tmp/get-pip.py
RUN python3 /tmp/get-pip.py

RUN pip3 install pyyaml

which I ran using this Makefile:

doit:   pip/get-pip.py
        docker build .

pip/get-pip.py:
        -@mkdir pip
        curl https://bootstrap.pypa.io/get-pip.py -o pip/get-pip.py

(those need to be TAB characters on the indented lines) to make sure pip-get.py is available from the context (you can of course download it from within the Dockerfile, but that is not necessary). That ends in a succesful PyYAML install, but it will be slow.

I recommend you start using ruamel.yaml (disclaimer: I am the author of that package), by changing the last line of the Dockerfile to read:

RUN pip3 install ruamel.yaml

Apart from many bugfixes in the original PyYAML code it is based upon, ruamel.yaml supports YAML 1.2 and YAML 1.1 (replaced in 2009 and the version PyYAML supports), and installs the appropriate version from a .whl file, so you'll have the fast C loader available in your container (PyYAML doesn't do that).

You can load a YAML file using the C-loader in ruamel.yaml using:

from pathlib import Path
from ruamel.yaml import YAML


path = Path('yourfile.yaml')
yaml = YAML(typ='safe')
data = yaml.load(path)

The first error in your question is also caused by python-distutils not having been installed in your container.

Oliver · Accepted Answer · 2019-01-22 22:05:38Z

When there is no apt install something that is available for a python package, here is how to do it. Thanks to @Anthon and @digitalarbeiter as their answers provided important information to arrive at solution.

To install via a setup.py file (esp. useful in Ubuntu Docker container):
- it was sufficient for me to
```
apt install python3-distutils
<download package, tar xvf, cd to folder>
python3 setup.py install
```
- This method of installation only works for pure Python packages (should not not be a surprise), which means that Python packages that have non-pure Python dependencies may not install or, if they do, will have some functionality unavailable.
- Note that even before installing python3-distutils, python3 -m distutils worked; this implies that the builtin distutils, that comes with Python3 via apt install python3, is not the full distutils; I did not know that, is this fact ever mentioned anywhere?
To install pip without the gcc toolchain: it was sufficient for me to
```
apt install python3-distutils
wget https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
```
Then pip install pyyaml completed. It seemed to install from a .tar.gz so it is the pure Python implementation too. Not surprising. This technique is useful if a package is not installable via apt install python3-<package>

The above methods required only a few megs of disk space.

A couple other missing pieces of the puzzle for me were:

apt install python3-<something>:
- I missed the fact that many Python packages are distributed this way in Debian, handy for packages that have a C implementation (or C dependencies) since no compilation necessary.
- AND I didn't know that for packages that are called py<something> on PyPI, the apt install is python3-<something> NOT python3-py<something>. Unfortunately apt search pyyaml was no help here.
apt search <something>: I had sort of forgotten about it because bash on (desktop) Ubuntu automatically suggests the right package to download when a command is not found.
- In particular apt search yaml yields 81 packages that provide YAML read/write in several languages (Python 2, Python 3, nodejs, Java, Go, Ruby, Erlang, Lua, Perl, C, C++, Clojure), linter, schema validator, etc.
- Multiple search terms are AND'ed so apt search yaml python3 showed the python3-yaml that I missed.
- Unfortunately apt search pyyaml produces no results, eventhough the Source and Homepage fields of apt show python3-yaml contain the word "pyyaml". I could not find a way to make search include those fields.

Collectives™ on Stack Overflow

python install without pip on ubuntu

4 Answers 4

Useful links

Comments

3 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Useful links

Comments

3 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related