How to execute python scripts that import third party modules via the command line

Question

I have a py file that preforms a google search using the system arguments passed to it and opens the first five results of the search as separate browser tabs (this is an exercise from the book Automate the Boring Stuff).

I would like to execute this script via the Windows run command and have therefore created a BAT file.

Currently, when I execute the BAT file, a "module not found" error is returned.

I suspected this issue was related to the fact that the modules required by the py file were only installed in the virtual environment of my specific python project (I have a project specifically for the exercises in this book). Therefore, I installed the necessary modules directly into the environment of my main Python installation. Unfortunately, this had no effect.

I then revised my BAT file by adding a line to activate my virtual environment before the line to execute my py file. This seemed to prevent my py file from being executing by the BAT file.

I'm somewhat familiar with Python but new to BAT files and the command line in general. I've read through a basic command line tutorial but couldn't find anything that helps.

I'm not sure how to resolve this issue and if possible would like to avoid polluting my main python environment with tons of modules. Is this possible and, if so, what am I missing?

.bat file

@py.exe C:\Users\Betty\PycharmProjects\Automate_the_Boring_Stuff\12\searchpypi.py %*
@pause

.py file

#! python3
# searchpypi.py  - Opens several search results.


import sys
import webbrowser

import requests

import bs4

print('Searching...')    # display text while downloading the search result page
res = requests.get('https://google.com/search?q=' 'https://pypi.org/search/?q='
                   + ' '.join(sys.argv[1:]))
res.raise_for_status()

# Retrieve top search result links.
soup = bs4.BeautifulSoup(res.text, 'html.parser')
# Open a browser tab for each result.
linkElems = soup.select('.package-snippet')
numOpen = min(5, len(linkElems))

# Open a browser tab for each result.
for i in range(numOpen):
    urlToOpen = 'https://pypi.org' + linkElems[i].get('href')
    print('Opening', urlToOpen)
    webbrowser.open(urlToOpen)

edit (Fri Mar 05 16:41:27 2021 UTC):

Error message (copied from command line):

Traceback (most recent call last):
  File "C:\Users\Betty\PycharmProjects\Automate_the_Boring_Stuff\12\searchpypi.py", line 8, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
Press any key to continue . . .

edit (Sat Mar 06 07:39:43 2021 UTC):

Pip Details:

C:\Users\Betty>pip -V
pip 21.0.1 from c:\python38\lib\site-packages\pip (python 3.8)

C:\Users\Betty>pip3 list
Package          Version
---------------- ----------
appdirs          1.4.4
certifi          2020.12.5
chardet          4.0.0
distlib          0.3.1
filelock         3.0.12
idna             2.10
pip              21.0.1
pipenv           2020.11.15
requests         2.25.1
setuptools       41.2.0
six              1.15.0
urllib3          1.26.3
virtualenv       20.2.2
virtualenv-clone 0.5.4
wheel            0.34.2

C:\Users\Betty>

Can you check the version of pip that you are using? And then check the output of pip3 list? — Ouss
– Ouss, Commented Mar 5, 2021 at 18:46

Ouss · Accepted Answer · 2021-03-05 11:57:52Z

1

Automating everything would means:

creating a virtual environment for this Python script
sourcing it
installing dependencies
run the scrip
optional (cleanup if necessary)

There is a way to do it on Bash, but I am not sure if the same applies for the Windows shell.

First you need to put all dependencies in a requirements.txt file. Each dependency should take one line in the document.

requirements.txt's content:

webbrowser
beautifulsoup4

And the automation script.sh:

# creates a venv folder that contains a copy of python3 packages to isolate any further changes in packages from the system installation
python3 -m venv venv  
 # tell the shell to use the created virtual environment
source venv/bin/activate
# install requirements
pip3 install -r requirements.txt 
# run the script
python3 your_script_filename.py
# remove the venv
rm -rf venv
# deactivate the virtual environment
deactivate

That’s a very basic script that resembles manual invocation of the Python script. There is so much space for improvements. The venv could be placed in the OS temporary folder and can be left there for reuse next time. That will remove the need for recreating the venv and installing the requirements.

I know this is not exactly what you want but I hope it got you a little bit further

edited Mar 5, 2021 at 11:57

answered Mar 5, 2021 at 11:28

Ouss

4,0563 gold badges34 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

tripleee Over a year ago

Removing requirements.txt at the end is weird. Why would you do that?

Ouss Over a year ago

You are right. I’ve fixed it. Delete the virtual environment folder... but that comes at the cost of having to download the requirements every time...

Robert Brown Over a year ago

Thanks for the replies! I can almost understand why I would need to create an ad-hoc venv to fully automate the script and will use the provided script to continue testing. I still cannot explain why the .py script wouldn't execute if the required modules are installed in my main python environment. Any ideas about this?

Ouss Over a year ago

Can you share the exact error message as part of your question above? That would help understand where the issue might be.

Collectives™ on Stack Overflow

How to execute python scripts that import third party modules via the command line

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related