13
votes
Accepted
12
votes
Accepted
Python script to scrape and parse the Stanford Encyclopedia of Philosophy
Add a hashbang at the top of your script since it's expected to be executable.
There is no function but I didn't think it was worth adding
Functions are worth adding: the visual and performance ...
- 71.2k
10
votes
Accepted
Repeatable HTTP requests in Python
If you are making many requests, it is worth it to have a look at requests.Session. This re-uses the connection to the server for subsequent requests (and also the ...
- 41.7k
10
votes
Python network manager class using exec()
sending_request is not a great name for an event; similar to your other signals, it should describe something that happened i.e. ...
- 71.2k
10
votes
Python network manager class using exec()
You don't need to use exec().
requests has a requests.request() function that takes the name ...
- 402
10
votes
News API in Python
Naming
NewsConcerning is styled in the way PEP 8 would tell us to style a class name. As function it should be in snake case: ...
- 6,126
8
votes
Accepted
Web scraping Google Trends in Python
The idea behind your script is pretty cool and there's a couple ways to polish it up! While it certainly wouldn't cause a headache, I think that creating an entire class just to measure how long the ...
- 1,120
7
votes
Repeatable HTTP requests in Python
I'll start with the obvious:
while submissions > 0:
...
submissions -= 1
can be transformed into:
...
- 1,421
6
votes
Accepted
Python script to POST requests to a networking device
Disable hard certificate failures, fine (temporarily). But do not disable_warnings. They're there to remind you that you're doing a bad thing. You're a network ...
- 71.2k
6
votes
Accepted
Scraping web data with Python 3
I assume is_good_response is just checking for a 200 response code.
Merge is_good_response, ...
- 44.6k
5
votes
Accepted
Python: Handling different HTTP status codes using the requests library
I suggest you implement exponential back-off. For example, first try after x seconds and then twice as long as the last wait time. This is the common practice for retries. Recursion often looks ...
- 196
5
votes
Alerts by phone calls for location-relevant Israel Home Front Command alerts
always running
The requirement is perfectly sensible,
but the implementation is weird and inconvenient.
make sure that the main program is running:
...
- 43.3k
5
votes
News API in Python
String Replacements and Matching
You should use a case-insensitive regular expression using the re.I flag to do your matching. For example:
...
- 4,101
5
votes
News API in Python
single responsibility
The SRP
says that a function should do one thing well.
The expression that replaces "concerning" and other stop words
should be in a helper function.
For one thing, ...
- 43.3k
5
votes
News API in Python - Take II
Type hints
I like the usage of type hints for your functions. However, the return hint
looks wrong for:
def clean_query(text: str) -> str:
The function always ...
- 16.4k
4
votes
Accepted
Python Web scraping
Naming
Variable names should be snake_case, and should represent what they are containing. I would also use req instead of <...
- 10.8k
4
votes
Python script to scrape and parse the Stanford Encyclopedia of Philosophy
General style
Use normal comments instead of triple quoted strings. Actually """ sep2epub """ doesn't really convey anything useful ...
- 1,233
4
votes
Accepted
Printing a JSON/HTTP response from a Cisco endpoint
Since this is the entire script,
it doesn't need to exist and you can just invoke curl directly; and
we can be a little more forgiving on exception-handling best ...
- 71.2k
4
votes
Accepted
Single API call with authentication in Python
Instead of using:
esx_name = sys.argv[1]
prefer the argparse library which is more flexible and allows for variable argument positioning.
In ...
- 8,778
4
votes
Python: Handling different HTTP status codes using the requests library
First, you should reduce to one call to request. You don't need to separate by method; just pass type to the method argument.
...
- 71.2k
4
votes
Alerts by phone calls for location-relevant Israel Home Front Command alerts
Here are a few suggestions for function get_alerts:
Why is get_alerts an async Function?
I ...
- 4,101
4
votes
Alerts by phone calls for location-relevant Israel Home Front Command alerts
Context manager
One minor point: since you are opening files with a context manager (with:), file.close() is superfluous. The ...
- 8,778
4
votes
News API in Python
Just a high-level comment: A large part of the code consists of rule-based parsing of text (all the string-replace calls). This is extremely brittle (there are lots of cases that you won't cover) and ...
- 141
4
votes
Accepted
News API in Python - Take II
Reinventing the wheel
The following loop simply reinvents next.
...
- 6,126
4
votes
News API in Python - Take II
Session
This may not a big deal since you are using requests at only one place, but using a session has some benefits. You can set all your headers at session level ...
- 8,778
3
votes
Search for providers of TV Shows
PEP-8
Your code is mostly compliant with the Style Guide for Python Code. A few deviations should be corrected.
Function names should be in snake_case. So ...
- 35.3k
3
votes
Python script for Web scraping
When using Requests to fetch JSON content, it's usually more convenient to use the resonse.json() method instead of manually passing ...
- 4,221
3
votes
Accepted
Config Variables Initialization | Python
With no usage shown, no implementation for LookupDict shown and nothing other than your network type values, this is overdesigned and offers nothing beyond
...
- 71.2k
3
votes
Scrape webelements
Naming confusion
First of all there is something that is confusing in your code: bs4 does not actually represent an instance of BeautifulSoup. Pretty much any Python code that is based on ...
- 8,778
3
votes
Python Web scraping
I think you can even get rid of the regular expressions. I prefer to use the BS4 functions.
Instead of:
...
- 8,778
Only top scored, non community-wiki answers of a minimum length are eligible