pulling a git repo at a particular commit in python setup.py file

Question

I have a Python project where I am using the maskrcnn_benchmark project from facebook research.

In my continuous integration script, I create a virtual environment where I install this project with thee following steps:

 - git clone https://github.com/facebookresearch/maskrcnn-benchmark.git
- cd maskrcnn-benchmark
- git reset --hard 5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6
- python setup.py build develop

This works fine and installs everything in the virtual environment as it needs to be.

Now I have a setup.py for my project for packaging and deploying my app. How can I do the same in this setup.py file i.e. pull and build this repository from the particular commit hash?

Thanks to the answer below and the comments, I have the setup.py as follows now:

install_requires=[
        '5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6-0.1',
        'ninja',
        'yacs',
        'matplotlib',
        'cython==0.28.5',
        'pymongo==3.7.1',
        'scipy==1.1.0',
        'torch==1.0.0',
        'torchvision==0.2.1',
        'opencv_python==3.4.2.17',
        'numpy==1.15.1',
        'gputil==1.3.0',
        'scikit_learn==0.19.2',
        'scikit_image==0.14.0',
        'sk_video==1.1.10'
  ],

dependency_links=[
        'http://github.com/facebookresearch/maskrcnn-benchmark/tarball/master#egg=5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6-0.1'
  ],

No matter where I put the '5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6-0.1', the maskrcnn-benchmark project gets compiled first. How can I do it that the dependency and this package is installed last?

Can't use dependency_links=[ "git+ssh://[email protected]" ] ? — IMParasharG
– IMParasharG, Commented Mar 13, 2019 at 16:29
@GovindParashar I see this but can I pull the project at the specific commit? The git reset --hard step? — Luca
– Luca, Commented Mar 13, 2019 at 16:37
you can specify commit hash in the url i.e. master#egg=XXXXXXXXXXX — IMParasharG
– IMParasharG, Commented Mar 13, 2019 at 16:40
@GovindParashar Thank you. Let me try and update the thread here. Please feel free to write as an answer if you wish — Luca
– Luca, Commented Mar 13, 2019 at 16:41

IMParasharG · Accepted Answer · 2019-03-13 16:46:47Z

4

You can use dependency_links setup.py

i.e.

dependency_links =[https://github.com/GovindParashar136/spring-boot-web-jsp/tarball/master#egg=8138cc3fd4e11bde31e9343c16c60ea539f687d9]

In your case url

https://github.com/facebookresearch/maskrcnn-benchmark/tarball/master#egg=5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6

answered Mar 13, 2019 at 16:46

IMParasharG

1,9051 gold badge15 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Luca Over a year ago

So, I tried the following in the dependency links:

https://github.com/facebookresearch/maskrcnn-benchmark/tarball/master#egg=5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6-0.1'. The project has a

setup.py` file. I can see it gets downloaded but it never gets installed. Do I need to do something else to get it to install it? I added the 0.1 as I saw that it is needed to add the version number.

IMParasharG Over a year ago

you have to specify --process-dependency-links while using pip

IMParasharG Over a year ago

I have seen dependency links processing has been deprecated.

Luca Over a year ago

but I am invoking it as python setup.py build install. So not using pip directly from the command line

IMParasharG Over a year ago

Then you have add install_requires=[ '5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6-0.1' ], before dependency_links

|

Simon Walker · Accepted Answer · 2020-07-20 13:11:02Z

2

This answer suggests that including a package@ prefix to the git url will install the git commit specified:

# in setup.py
setup(
    # other fields
    install_requires=[
        "packagename@git+https://github.com/<user>/<repo>#<commit hash>",
    ],
)

so in your case:

# in setup.py
setup(
    # other fields
    install_requires=[
        "maskrcnn_benchmark@git+https://github.com/facebookresearch/maskrcnn-benchmark.git#5ec0b91cc85163ac3b58265b3f9b39bb327d0ba6",
    ],
)

answered Jul 20, 2020 at 13:11

Simon Walker

6,0916 gold badges33 silver badges34 bronze badges

2 Comments

Keiran Raine Over a year ago

This doesn't work if you want to target a commit for a package that has an entry in pypi

K.S. Over a year ago

Right, I get "Invalid requirement:" for my packet with the Pypi entry

Collectives™ on Stack Overflow

pulling a git repo at a particular commit in python setup.py file

2 Answers 2

7 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related