Deploying Python and Dependencies to Elastic Beanstalk

Question

I have two python projects that share some common libraries all organized into three git repositories: project1, project2, and common-lib. The two projects are each meant to be deployed to elastic beanstalk bundled with common-lib.

I'm trying to find the most idiomatic way to structure these projects to make it easy to develop for locally and to build a zip file for deployment using eb deploy.

Setting everything up for local development is easy. Just checkout each repo and do a python setup.py develop in common-lib to make the common libraries available in the virtualenv.

For EB deployment it would be nice to have a single setup.py command that produces an EB compatible zip file that contains the project and common-lib with a requirements.txt file that lists the pip dependencies for both. I have not yet found an easy way to do this which is a bit surprising because I imagine this is a fairly common scenario.

I can't specify the git repository for common-lib in either project1 or project2's requirements.txt file because the repository won't be reachable from AWS.

So make it reachable: stackoverflow.com/questions/34727442/… — smentek
– smentek, Commented Jan 11, 2016 at 17:57
@eddie What was your solution to this? I'm not sure, but I may be experiencing the same problem. My requirements.txt lists all my third-party dependencies, but excludes the project I'm developing myself. So I'm encountering a DistributionNotFound error where the distribution referenced is MyApp (aka the app I have written). Was your solution to make the private repository accessible via AWS using the solution provided by @smentek? — Drew Burnett
– Drew Burnett, Commented Jul 17, 2016 at 18:25
Do not do this. But, I junction (Windows) or hard-link my common packages into the multiple different ones. It is a pain to set up for developers after a pull, but works very nicely with git and EBS. — std''OrgnlDave
– std''OrgnlDave, Commented Sep 6, 2016 at 1:04
Not sure what kind of eb you're using but, why not use a docker container? Just have your DockerFile do what you do to set up your environment? — Luis F Hernandez
– Luis F Hernandez, Commented Dec 10, 2016 at 18:14

szaboat · Accepted Answer · 2016-12-13 08:49:15Z

1

For me the proper way would be to create a python package from the common lib, publish it to a private pypi server, like https://gemfury.com/l/pypi-server. Have it as a reference in requirements.txt as a python package.

Another solution can be to include the common-lib as a git submodule https://git-scm.com/docs/git-submodule. With that you will have the separation, because it will live in a separate repository and you will have a simple reference as a git submodule in your's project.

answered Dec 13, 2016 at 8:49

szaboat

5893 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Deploying Python and Dependencies to Elastic Beanstalk

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related