1

I am using Poetry to install a python project using Poetry in a Docker container. Below you can find my Docker file, which used to work fine until recently when I switched to a new version of Poetry (1.2.1) and the new recommended Poetry installer:

# pull official base image
FROM ubuntu:20.04

ENV PATH = "${PATH}:/home/poetry/bin"
ENV APP_HOME=/home/app/web

RUN apt-get -y update && \
    apt upgrade -y && \
    apt-get install -y \
        python3-pip \
        curl \
        netcat \
        gunicorn && \
    rm -fr /var/lib/apt/lists


# alias python2 to python3
RUN ln -s /usr/bin/python3 /usr/bin/python

# Install Poetry
RUN mkdir -p /home/poetry && \
    curl -sSL https://install.python-poetry.org | POETRY_HOME=/home/poetry python -

# Cleanup
RUN apt-get remove -y curl && \
    apt-get clean

RUN pip install --upgrade pip && \
    pip install cryptography && \
    pip install psycopg2-binary

# create directory for the app user
# create the app user
# create the appropriate directories
RUN adduser --system --group app && \
    mkdir -p $APP_HOME/static-incdtim && \
    mkdir -p $APP_HOME/mediafiles

# copy project
COPY . $APP_HOME

WORKDIR $APP_HOME

# Install Python packages
RUN poetry config virtualenvs.create false
RUN poetry install --only main

# copy entrypoint-prod.sh
COPY ./entrypoint.incdtim.prod.sh $APP_HOME/entrypoint.sh
RUN chmod a+x $APP_HOME/entrypoint.sh

# chown all the files to the app user
RUN chown -R app:app $APP_HOME

# change to the app user
USER app

# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.sh"]

The poetry install works fine, I attached to a running container and run it myself and found that it works without problems. However, when I open a Python console and try to import a module (django) which is installed by the Poetry project, the module is not found. Please note that I am installing my project in the system environment (poetry config virtualenvs.create false). I verified, and there is only one version of python installed in the docker container. The specific error I get when trying to import a python module installed by Poetry is: ModuleNotFoundError: No module named xxxx

3
  • This Dockerfile seems quite long, and in the following text you refer to a Python source file that imports a module. Can you edit the question to include a minimal reproducible example: on the one hand, remove the parts of the Dockerfile that aren't necessary, and on the other, include the Python source and the actual error message you get? Commented Oct 8, 2022 at 16:20
  • What is the name of the module that is missing? Is it a module in your code, or a module that should be installed by poetry? Commented Oct 10, 2022 at 8:05
  • It turns out this is a known bug in Poetry: github.com/python-poetry/poetry/issues/6459 Commented Oct 10, 2022 at 17:04

2 Answers 2

1

Although this is not an answer, it is too long to fit within the comment section. It is rather a piece of advice:

  • declare your ENV at the top of the Dockerfile to make it easier to read.
  • merge the multiple RUN commands together to avoid creating useless intermediate layers. In the particular case of apt-get install, this will also prevent you from installing a package which dates back from the first "apt-get update". Indeed, since the command line has not changed Docker will not re-execute the command and thus not refresh the package list..
  • avoid making a copy of all the files in "." when you previously copy some specific files to specific places..

Here, you Dockerfile could rather look like:

# pull official base image
FROM ubuntu:20.04

ENV PATH = "${PATH}:/home/poetry/bin"
ENV HOME=/home/app
ENV APP_HOME=/home/app/web

RUN apt-get -y update && \
    apt upgrade -y && \
    apt-get install -y \
        python3-pip \
        curl \
        netcat \
        gunicorn && \
    rm -fr /var/lib/apt/lists

# alias python2 to python3
RUN ln -s /usr/bin/python3 /usr/bin/python

# Install Poetry
RUN mkdir -p /home/poetry && \
    curl -sSL https://install.python-poetry.org | POETRY_HOME=/home/poetry python -

# Cleanup
RUN apt-get remove -y \
        curl && \
    apt-get clean

RUN pip install --upgrade pip && \
    pip install cryptography && \
    pip install psycopg2-binary

# create directory for the app user
# create the app user
# create the appropriate directories
RUN mkdir -p /home/app && \
    adduser --system --group app && \
    mkdir -p $APP_HOME/static-incdtim && \
    mkdir -p $APP_HOME/mediafiles

WORKDIR $APP_HOME

# copy project
COPY . $APP_HOME

# Install Python packages
RUN poetry config virtualenvs.create false && \
    poetry install --only main

# copy entrypoint-prod.sh
RUN cp $APP_HOME/entrypoint.incdtim.prod.sh $APP_HOME/entrypoint.sh && \
    chmod a+x $APP_HOME/entrypoint.sh && \
    chown -R app:app $APP_HOME

# change to the app user
USER app

# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.sh"]

UPDATE:

Let's get back to your question. Having your program running okay when you "run it yourself" does not mean all the dependencies are met. Indeed, this can mean that your module has not been imported yet (and thus has not triggered the ModuleNotFoundError exception yet).

In order to validate this theory, you can either:

  1. create a simple application which imports the failing module and then quits. If the import succeeds then there is something weird indeed.
  2. list the installed modules with poetry show --latest. If the package is listed, then there is something weird indeed.

If none of the above indicates the module is installed, that just means the module is not installed and you should update your Dockerfile to install it.

NOTE: I do not know much about poetry, but you may want to have a list external dependencies to be met for your application. In the case of pip3, the list is expressed as a file named requirement.txt and can be installed with pip3 install -r requirement.txt.

Sign up to request clarification or add additional context in comments.

3 Comments

pip install commands can (and probably should) be joined together as well here. You can use COPY --chown=... instruction to avoid separate RUN chown later. Is there any way to avoid re-downloading dependencies using poetry, like putting pip install -r requirements.txt before copying project data to rely on layer caching?
I have updated the Docker file as suggested, problem persists.
I didn't expect it to solve your issue, it was rather a piece of advice. I updated the answer because I bet the module is just not installed in the container..
1

It turns out this is known a bug in Poetry: https://github.com/python-poetry/poetry/issues/6459

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.