Gitlab CI failing to install regex from Pipfile.lock

Question

I'm trying to run my tests via Gitlab CI, but am having problems getting the environment set up. I'm developing on Windows, which is why the lockfile is being regenerated. I've tried various combinations of pipenv commands, but it all seems to come down to this regex issue. I did dump the output with --support, and I see the regex hashes in Gitlab are different from my windows version, so the lock file did get successfully regenerated for the 3.7-slim container.

.gitlab-ci.yml (relevant test section)

test-python3.7:
  stage: UnitTest
  image: python:3.7-slim
  timeout: 0h 10m
  script:
  - pip install pipenv
  - pipenv lock --dev
  - pipenv install --dev
  - cd src
  - pipenv run coverage run -m unittest discover && coverage report -m
  coverage: '/TOTAL.+ ([0-9]{1,3}%)/'
  retry:
    max: 2
    when:
    - stuck_or_timeout_failure

Gitlab CI log:

$ pipenv lock --dev
Creating a virtualenv for this project…
Pipfile: /builds/michjnich/sudoku/Pipfile
Using /usr/local/bin/python (3.7.5) to create virtualenv…
Already using interpreter /usr/local/bin/python
Using base prefix '/usr/local'
New python executable in /root/.local/share/virtualenvs/sudoku-BhtRYaDu/bin/python
Installing setuptools, pip, wheel...
done.

Virtualenv location: /root/.local/share/virtualenvs/sudoku-BhtRYaDu
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (b2e415)!
$ pipenv install --dev
Installing dependencies from Pipfile.lock (b2e415)…
An error occurred while installing regex==2019.11.1 --hash=sha256:15454b37c5a278f46f7aa2d9339bda450c300617ca2fca6558d05d870245edc7 --hash=sha256:1ad40708c255943a227e778b022c6497c129ad614bb7a2a2f916e12e8a359ee7 --hash=sha256:5e00f65cc507d13ab4dfa92c1232d004fa202c1d43a32a13940ab8a5afe2fb96 --hash=sha256:604dc563a02a74d70ae1f55208ddc9bfb6d9f470f6d1a5054c4bd5ae58744ab1 --hash=sha256:720e34a539a76a1fedcebe4397290604cc2bdf6f81eca44adb9fb2ea071c0c69 --hash=sha256:7caf47e4a9ac6ef08cabd3442cc4ca3386db141fb3c8b2a7e202d0470028e910 --hash=sha256:7faf534c1841c09d8fefa60ccde7b9903c9b528853ecf41628689793290ca143 --hash=sha256:b4e0406d822aa4993ac45072a584d57aa4931cf8288b5455bbf30c1d59dbad59 --hash=sha256:c31eaf28c6fe75ea329add0022efeed249e37861c19681960f99bbc7db981fb2 --hash=sha256:c7393597191fc2043c744db021643549061e12abe0b3ff5c429d806de7b93b66 --hash=sha256:d2b302f8cdd82c8f48e9de749d1d17f85ce9a0f082880b9a4859f66b07037dc6 --hash=sha256:e3d8dd0ec0ea280cf89026b0898971f5750a7bd92cb62c51af5a52abd020054a --hash=sha256:ec032cbfed59bd5a4b8eab943c310acfaaa81394e14f44454ad5c9eba4f24a74! Will try again.
Installing initially failed dependencies…
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1874, in do_install
[pipenv.exceptions.InstallError]:       keep_outdated=keep_outdated
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 859, in do_install_dependencies
[pipenv.exceptions.InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Collecting regex==2019.11.1', '  Using cached https://files.pythonhosted.org/packages/fc/1d/13cc7d174cd2d05808abac3f5fb37433e30c4cd93b152d2a9c09c926d7e8/regex-2019.11.1.tar.gz', 'Building wheels for collected packages: regex', '  Building wheel for regex (setup.py): started', "  Building wheel for regex (setup.py): finished with status 'error'", '  Running setup.py clean for regex', 'Failed to build regex', 'Installing collected packages: regex', '    Running setup.py install for regex: started', "    Running setup.py install for regex: finished with status 'error'"]
[pipenv.exceptions.InstallError]: ['ERROR: Command errored out with exit status 1:', '   command: /root/.local/share/virtualenvs/sudoku-BhtRYaDu/bin/python -u -c \'import sys, setuptools, tokenize; sys.argv[0] = \'"\'"\'/tmp/pip-install-4xs4zmcl/regex/setup.py\'"\'"\'; __file__=\'"\'"\'/tmp/pip-install-4xs4zmcl/regex/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' bdist_wheel -d /tmp/pip-wheel-_y82fvl8 --python-tag cp37', '       cwd: /tmp/pip-install-4xs4zmcl/regex/', '  Complete output (19 lines):', '  running bdist_wheel', '  running build', '  running build_py', '  creating build', '  creating build/lib.linux-x86_64-3.7', '  creating build/lib.linux-x86_64-3.7/regex', '  copying regex_3/__init__.py -> build/lib.linux-x86_64-3.7/regex', '  copying regex_3/regex.py -> build/lib.linux-x86_64-3.7/regex', '  copying regex_3/_regex_core.py -> build/lib.linux-x86_64-3.7/regex', '  copying regex_3/test_regex.py -> build/lib.linux-x86_64-3.7/regex', '  warning: build_py: byte-compiling is disabled, skipping.', '  ', '  running build_ext', "  building 'regex._regex' extension", '  creating build/temp.linux-x86_64-3.7', '  creating build/temp.linux-x86_64-3.7/regex_3', '  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.7m -c regex_3/_regex.c -o build/temp.linux-x86_64-3.7/regex_3/_regex.o', "  unable to execute 'gcc': No such file or directory", "  error: command 'gcc' failed with exit status 1", '  ----------------------------------------', '  ERROR: Failed building wheel for regex', '    ERROR: Command errored out with exit status 1:', '     command: /root/.local/share/virtualenvs/sudoku-BhtRYaDu/bin/python -u -c \'import sys, setuptools, tokenize; sys.argv[0] = \'"\'"\'/tmp/pip-install-4xs4zmcl/regex/setup.py\'"\'"\'; __file__=\'"\'"\'/tmp/pip-install-4xs4zmcl/regex/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' install --record /tmp/pip-record-4ggezbwt/install-record.txt --single-version-externally-managed --compile --install-headers /root/.local/share/virtualenvs/sudoku-BhtRYaDu/include/site/python3.7/regex', '         cwd: /tmp/pip-install-4xs4zmcl/regex/', '    Complete output (19 lines):', '    running install', '    running build', '    running build_py', '    creating build', '    creating build/lib.linux-x86_64-3.7', '    creating build/lib.linux-x86_64-3.7/regex', '    copying regex_3/__init__.py -> build/lib.linux-x86_64-3.7/regex', '    copying regex_3/regex.py -> build/lib.linux-x86_64-3.7/regex', '    copying regex_3/_regex_core.py -> build/lib.linux-x86_64-3.7/regex', '    copying regex_3/test_regex.py -> build/lib.linux-x86_64-3.7/regex', '    warning: build_py: byte-compiling is disabled, skipping.', '    ', '    running build_ext', "    building 'regex._regex' extension", '    creating build/temp.linux-x86_64-3.7', '    creating build/temp.linux-x86_64-3.7/regex_3', '    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.7m -c regex_3/_regex.c -o build/temp.linux-x86_64-3.7/regex_3/_regex.o', "    unable to execute 'gcc': No such file or directory", "    error: command 'gcc' failed with exit status 1", '    ----------------------------------------', 'ERROR: Command errored out with exit status 1: /root/.local/share/virtualenvs/sudoku-BhtRYaDu/bin/python -u -c \'import sys, setuptools, tokenize; sys.argv[0] = \'"\'"\'/tmp/pip-install-4xs4zmcl/regex/setup.py\'"\'"\'; __file__=\'"\'"\'/tmp/pip-install-4xs4zmcl/regex/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' install --record /tmp/pip-record-4ggezbwt/install-record.txt --single-version-externally-managed --compile --install-headers /root/.local/share/virtualenvs/sudoku-BhtRYaDu/include/site/python3.7/regex Check the logs for full command output.']
ERROR: ERROR: Package installation failed...
ERROR: Job failed: exit code 1

The issue has not much to do with GitLab-CI or pipenv. The error is very clearly stated in the log output: gcc could not be found. It is needed to build regex. To avoid having to build it, use a Python version compatible with the pre-built wheels offered on PyPI for this project. — sinoroc
– sinoroc, Commented Nov 6, 2019 at 9:57

sinoroc · Accepted Answer · 2019-11-06 09:51:08Z

1

The log says gcc could not be found. Looks like it is required to build regex.

Either get gcc (and probably some more elements) installed in your container in order to build regex.
Or select a version of the Python interpreter for which there already are pre-built wheels available, according to the list of files for regex 2019.11.1 there are currently wheels only for Python 3.6 on Linux, not for Python 3.7 as you are currently using.

answered Nov 6, 2019 at 9:51

sinoroc

23.4k4 gold badges59 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

michjnich Over a year ago

Thanks :) I need to learn more about the packaging side of things in python I think! That would explain why I have no issues on my dev machine which is WIn10. Odd that it's there for win to 3.8 and only to 3.6 for linux ...

sinoroc Over a year ago

You could check the list of files (see link in the answer) once in a while see if new pre-built wheels are added.

michjnich Over a year ago

After running a few tests in Docker, I've found that the 3.7-stretch image already includes gcc, so that seems to solve the problem nicely :) Thanks for the help!

Collectives™ on Stack Overflow

Gitlab CI failing to install regex from Pipfile.lock

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related