7

I'm trying to build a Docker container that should install a series of python packages from a requirements.txt file. One of the entries is a python package hosted on a private GitHub repository. To install it, I've created a pair of SSH keys and added the public one as a Deploy Key to the GitHub repository.

However, when I'm building the container I'm getting this error:

ERROR: Command errored out with exit status 128: git clone -q 'ssh://****@github.com:organization/my-package' /tmp/pip-install-e81w4wri/my-package Check the logs for full command output.

I've tried to debug the error by changing the pip install command of the docker file with RUN git clone [email protected]:organization/my-package.git and it did work fine.

What does this error mean and how can I solve it? I could clone it and install it with a dedicated command, but if possible I'd like to keep all requirements in a single place. Thanks!

This is the Dockerfile that I'm using:

FROM joyzoursky/python-chromedriver:3.7-alpine3.8 as base

FROM base as builder

RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add bash gcc gfortran build-base git wget freetype-dev libpng-dev openblas-dev openssh-client
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h

# copy requirements
RUN mkdir /install
WORKDIR /install
COPY ./requirements.txt /var/www/requirements.txt

### GITHUB SSH KEY ###
COPY ./keys/deploy_key_private .
RUN mkdir /root/.ssh && mv deploy_key_private /root/.ssh/id_rsa
RUN eval $(ssh-agent) && \
    ssh-add /root/.ssh/id_rsa && \
    ssh-keyscan -H github.com >> /etc/ssh/ssh_known_hosts

RUN pip install --upgrade pip && pip install --prefix=/install -r /var/www/requirements.txt --log logs.txt

FROM base
COPY --from=builder /install /usr/local
# KEEP ON BUILDING THE CONTAINER

The package is listed in the requirements.txt as git+ssh://[email protected]:organization/my-package@master#egg=my_package

If relevant, here is the traceback from pip:

Exception information:
2020-07-30T11:56:55,329 Traceback (most recent call last):
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
2020-07-30T11:56:55,329     status = self.run(options, args)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
2020-07-30T11:56:55,329     return func(self, options, args)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
2020-07-30T11:56:55,329     reqs, check_supported_wheels=not options.target_dir
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
2020-07-30T11:56:55,329     discovered_reqs.extend(self._resolve_one(requirement_set, req))
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
2020-07-30T11:56:55,329     abstract_dist = self._get_abstract_dist_for(req_to_install)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
2020-07-30T11:56:55,329     abstract_dist = self.preparer.prepare_linked_requirement(req)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 469, in prepare_linked_requirement
2020-07-30T11:56:55,329     hashes=self._get_linked_req_hashes(req)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 239, in unpack_url
2020-07-30T11:56:55,329     unpack_vcs_link(link, location)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 99, in unpack_vcs_link
2020-07-30T11:56:55,329     vcs_backend.unpack(location, url=hide_url(link.url))
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 733, in unpack
2020-07-30T11:56:55,329     self.obtain(location, url=url)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 641, in obtain
2020-07-30T11:56:55,329     self.fetch_new(dest, url, rev_options)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/git.py", line 230, in fetch_new
2020-07-30T11:56:55,329     self.run_command(make_command('clone', '-q', url, dest))
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 774, in run_command
2020-07-30T11:56:55,329     log_failed_cmd=log_failed_cmd)
2020-07-30T11:56:55,329   File "/usr/local/lib/python3.7/site-packages/pip/_internal/vcs/versioncontrol.py", line 166, in call_subprocess
2020-07-30T11:56:55,329     raise SubProcessError(exc_msg)
2020-07-30T11:56:55,329 pip._internal.exceptions.SubProcessError: Command errored out with exit status 128: git clone -q 'ssh://****@github.com:organization/my-package' /tmp/pip-install-e81w4wri/my-package Check the logs for full command output.
Improve this question
3
  • I'm not too familiar with Alpine, but usually the keys for known hosts don't go into /etc/ssh/ssh_known_hosts, but ~/.ssh/known_hosts. Commented Jul 30, 2020 at 14:09
  • Thanks for your comment, I tried to change the path but unfortunately it didn't solve the issue. I don't necessarily have to stick with Alpine, do you think the issue might be solved with a different base image? Commented Jul 30, 2020 at 14:17
  • Not sure, I don't think so. Does this post answer help: stackoverflow.com/questions/9617336/…? In short, try adding git configuration with user name and user email. Commented Jul 30, 2020 at 15:05

1 Answer 1

Reset to default
6

[email protected]:organization/my-package.git is a valid SSH URL. ssh://[email protected]:organization/my-package.git is not. ssh://[email protected]/organization/my-package.git would be.

As in here, you can add GIT_SSH_COMMAND='ssh -v' pip install ... to see exactly what is going on.

You might need:

git config --global url."ssh://[email protected]/".insteadOf ssh://[email protected]:

The OP arabinelli reports in the comments having to use the following line in requirements.txt:

git+ssh://[email protected]/my-organization/my-repo-name@master#egg=my_package_dir

Aug. 2022, Jako adds in the comments:

This worked for me with a private BitBucket repository:

git+ssh://[email protected]/my-organization/my-repo-name@master#egg=my_project&subdirectory=subdir1

                                      ^^^^^^^

I had to specify the subdirectory 'subdir1'

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

It was an URL issue indeed, changing it in the requirements.txt solved it. Thanks!!
@arabinelli could you tell me exactly how the url was structured in the requirements.txt file?
This is the line as it appears on the requirements.txt (I had to replace sensitive info as it's a private package): git+ssh://[email protected]/my-organization/my-repo-name@master#egg=my_package_dir
@arabinelli Thank you for this feedback. I have included your comment in the answer for more visibility.
@Jako Thank you for the feedback. I have included your comment in the answer for more visibility.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.