Install Numpy Requirement in a Dockerfile. Results in error

Question

I am attempting to install a numpy dependancy inside a docker container. (My code heavily uses it). On building the container the numpy library simply does not install and the build fails. This is on OS raspbian-buster/stretch. This does however work when building the container on MAC OS.

I suspect some kind of python related issue, but can not for the life of me figure out how to make it work.

I should point out that removing the pip install numpy from the requirements file and using it in its own RUN statement in the dockerfile does not solve the issue.

The Dockerfile:

FROM python:3.6
ENV PYTHONUNBUFFERED 1
ENV APP /app
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN mkdir $APP
WORKDIR $APP
ADD requirements.txt .
RUN pip install -r requirements.txt
COPY . .

The requirements.txt contains all the project requirements, amounf which is numpy.

Step 6/15 : RUN pip install numpy==1.14.3
 ---> Running in 266a2132b078
Collecting numpy==1.14.3
  Downloading https://files.pythonhosted.org/packages/b0/2b/497c2bb7c660b2606d4a96e2035e92554429e139c6c71cdff67af66b58d2/numpy-1.14.3.zip (4.9MB)
Building wheels for collected packages: numpy
  Building wheel for numpy (setup.py): started
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...

EDIT:

So after the comment by skybunk and the suggestion to head to official docs, some more debugging on my part, the solution wound up being pretty simple. Thanks skybunk to you go all the glory. Yay.

Solution:

Use alpine and install python install package dependencies, upgrade pip before doing a pip install requirements.

This is my edited Dockerfile - working obviously...

FROM python:3.6-alpine3.7

RUN apk add --no-cache --update \
    python3 python3-dev gcc \
    gfortran musl-dev \
    libffi-dev openssl-dev

RUN pip install --upgrade pip

ENV PYTHONUNBUFFERED 1
ENV APP /app

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN mkdir $APP
WORKDIR $APP

ADD requirements.txt .
RUN pip install -r requirements.txt

COPY . .

skybunk · Accepted Answer · 2025-02-22 07:11:06Z

8

To use Numpy on python3 here, we first head over to the official documentation to find what dependencies are required to build Numpy.

Mainly these 5 packages + their dependencies must be installed:

Python3 - 70 mb
Python3-dev - 25 mb
gfortran - 20 mb
gcc - 70 mb
musl-dev -10 mb (used for tracking unexpected behaviour/debugging)

An POC setup would look something like this -

Dockerfile:

FROM gliderlabs/alpine
ADD repositories.txt /etc/apk/repositories

RUN apk add --no-cache --update \
    python3 python3-dev gcc \
    gfortran musl-dev

ADD requirements-pip.txt .
RUN pip3 install --upgrade pip setuptools && \
    pip3 install -r requirements-pip.txt

ADD . /app
WORKDIR /app
ENV PYTHONPATH=/app/
ENTRYPOINT python3 testscript.py

repositories.txt

http://dl-5.alpinelinux.org/alpine/v3.4/main

requirements-pip.txt

numpy

testscript.py

import numpy as np

def random_array(a, b):
    return np.random.random((a, b))

a = random_array(2,2)
b = random_array(2,2)
print(np.dot(a,b))

To run this - clone alpine, build it using "docker build -t gliderlabs/alpine ."

Build and Run your Dockerfile

docker build -t minidocker .
docker run minidocker

Output should be something like this-

[[ 0.03573961 0.45351115]
[ 0.28302967 0.62914049]]

edited Feb 22 at 7:11

answered Sep 3, 2019 at 3:55

skybunk

8632 gold badges12 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Kevin Smith Over a year ago

Awesome thanks. The official docs seemed to guide me in the right direction, when I bothered looking into it to my shame. I had to apply a minor add on the your suggested Dockerfile, adding- [libffi-dev] and [openssl-dev] to the list of required packages for a successful completion of my specific Dockerfile (Mostly because I was using the "ciffy" package as well. So YES this answer solved my problem.

Soorena Over a year ago

cloned the Github repo, followed the instructions and it failed.

Antony Hatchkins · Accepted Answer · 2022-02-02 02:44:22Z

1

Another method is to install from the 'slim' distribution of python (based on debian):

FROM python:slim
CMD pip install numpy

123Mb

This results in a smaller image than that of alpine:

FROM python:3-alpine3.9
RUN apk add --no-cache py3-numpy

187MB

Plus it gives better support for other whl libraries for slim is based on a glibc library (against which all the wheels are built) while apline uses musl (incompatible with the wheels), so all packages will have to be either apk added or compiled from sources.

edited Feb 2, 2022 at 2:44

answered Feb 1, 2022 at 15:07

Antony Hatchkins

34.4k13 gold badges121 silver badges114 bronze badges

1 Comment

NFeruch - FreePalestine Over a year ago

this worked for me: I couldn't install in python:3.10-alpine but it works for python:3.10-slim!

Antony Hatchkins · Accepted Answer · 2022-02-03 06:00:41Z

1

From the error logs, it does not seem that it is from numpy. but you can install numpy before the requirment.txt and verify if it's working.

FROM python:3.6
RUN pip install numpy==1.14.3

Build

docker build -t numpy .

Run and Test

docker run numpy bash -c "echo import numpy as np > test.py ; python test.py"

So you will see no error on import.

or You can try numpy as an alpine package

FROM python:3-alpine3.9
RUN apk add --no-cache py3-numpy

Or better to post the requirement.txt.

edited Feb 3, 2022 at 6:00

Antony Hatchkins

34.4k13 gold badges121 silver badges114 bronze badges

answered Sep 3, 2019 at 2:40

Adiii

61.1k10 gold badges180 silver badges174 bronze badges

2 Comments

Kevin Smith Over a year ago

To be honest this did work as well, additionally it also wound up being more dependancy based issues than that solely for numpy. Thanks for the insight.

Adiii Over a year ago

the purpose is to solve your issue either me or someone else. Glad to know :)

RicHincapie · Accepted Answer · 2021-01-06 14:43:37Z

0

I had lot of trouble with this issue using FROM python:3.9-buster and pandas.

My requirements.txt had the python-dev-tools, numpy and pandas, along with other packages.

I always got something like this when attempting to build:

preluded by:

and by:

Following hints by Adiii in this thread, I did some debug and found out that this actually works and builds a perfectly running container:

RUN pip3 install NumPy==1.18.0

RUN pip3 install python-dev-tools

RUN pip3 install pandas

RUN pip3 install -r requirements.txt

So, giving a specific RUN layer to the pip3 installing pandas solved the problem!

answered Jan 6, 2021 at 14:43

RicHincapie

4,1831 gold badge28 silver badges34 bronze badges

Collectives™ on Stack Overflow

Install Numpy Requirement in a Dockerfile. Results in error

4 Answers 4

2 Comments

1 Comment

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related