Gitlab CI/CD : Pytest running with Conda

Question

Context : I'm trying to setup a gitlab CI/CD that tests my build and runs my pytest tests when pushing my code.

Problem : When I push my code, the CI/CD job fails saying :

/bin/bash: line 55: pytest: command not found
ERROR: Job failed: exit code 1

Question : How do I get rid of the error and how do I setup properly my gitlab CI/CD ?

Details : I've (partially) followed this guide, and I've made a .gitlab-ci.yml file like this :

image: continuumio/miniconda3:latest

testbuild :
  stage: build
  script:
    - conda create --name test_env --file requirements.txt
    - source activate test_env
    - python setup.py install

tests:
  stage: test
  script:
    - cd tests && pytest .

My project architecture :

$ tree -L 1
project
├── package1/
├── package2/
├── data/
├── out/
├── __pycache__
├── requirements.txt
├── setup.py
└── tests/

My requirements.txt (stripped from a lot of useless stuff, for the readers convenience sake), that has been created with the command conda list -e :

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
scikit-learn=0.20.0=py36h4989274_1
scipy=1.1.0=py36hfa4b5c9_1
# ...
setuptools=40.4.3=py36_0
pip=10.0.1=py36_0
py=1.7.0=py36_0
pytest=3.9.1=py36_0
python=3.6.6=h6e4f718_2
wheel=0.32.1=py36_0

Oh yeah I knew I forgot something, sorry. It has been edited. I did not put the whole list as it felt a lot of content for not much, but I tried to put the dependencies that mattered (first of all, pytest). — 021
– 021, Commented Oct 26, 2018 at 8:07
Probably the conda env activation is unset between stages, try to reactivate it in the test stage: cd tests && source activate test_env && pytest . — hoefling
– hoefling, Commented Oct 26, 2018 at 8:19
I did as you wrote and I got something different : $ cd tests && source activate test_env &&pytest . Could not find conda environment: test_env You can list all discoverable environments with conda info --envs. — 021
– 021, Commented Oct 26, 2018 at 8:27
Looks like gitlab does not preserve the env between stages. Take a look at GitLab CI preserve environment between build stages. — hoefling
– hoefling, Commented Oct 26, 2018 at 22:12

021 · Accepted Answer · 2019-02-11 14:32:40Z

3

I've changed my .gitlab-ci.yml to :

image: continuumio/miniconda3:latest

testbuild :
  stage: build
  script:
    - conda create --name test_env --file requirements.txt
    - source activate test_env
    - python setup.py install
    - cd tests && pytest .

Regrouping both tests and testbuild in the same part. It now works, it installs everything and runs the tests, though it feels like a bad way to do it, because I'm not doing the separation anymore.

As hoefling said in the comments, the problem is that gitlab does not preserve the environment between stages. If you really do want to separate these two, look at this : GitLab CI preserve environment between build stages

edited Feb 11, 2019 at 14:32

answered Oct 26, 2018 at 9:28

021

3434 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ryanskeith Over a year ago

I agree with this solution (as I just spent a lot of time coming to it by myself!). You could also do conda env create -f <conda_environment.yml> but I feel conda create is currently more flexible. You can actually do offline with "conda create" vs. "conda env create" (assuming you had the packages required in your base conda repo).

Collectives™ on Stack Overflow

Gitlab CI/CD : Pytest running with Conda

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related