2

I wrote a Python script which scrapes a website and sends emails if a certain condition is met. It repeats itself every day in a loop.

I converted the Python file to an EXE and it runs as an application on my computer. But I don't think this is the best solution to my needs since my computer isn't always on and connected to the internet.

Is there a specific website I can host my Python code on which will allow it to always run?

More generally, I am trying to get the bigger picture of how this works. What do you actually have to do to have a Python script running on the cloud? Do you just upload it? What steps do you have to undertake?

Thanks in advance!

7
  • 1
    in many WWW servers you can use cron/crontab to execute some code every day at selected time - so you don't have to use loop. So check if server have cron and it let run Python script. Mostly servers use Linux so you don't have to create EXE file. Commented Jul 20, 2019 at 17:05
  • as I remeber on ScrapingHub.com you can run one spider/scraper for free but you have to use Python's scraping framework Scrapy. See: pricing on ScrapingHub, and doc for Scrapy Commented Jul 20, 2019 at 17:09
  • @furas Thanks for your answer! I am trying to get the bigger picture. I was asking what you actually have to do to have a running python script on the web? Do you just upload it or what exactly do you have to do? Sorry for the dumb question but I don't know how it works and I couldn't really find any info elsewhere. Commented Jul 20, 2019 at 21:30
  • on some servers you can upload script using web page with admin panel and run it only in cron and script is killed if it runs too long. If you need run it every day at the same time then cron can do it and you don't need loop. Commented Jul 20, 2019 at 22:02
  • on other servers you may have access throu ssh so you have console with Linux and you can use Linux/bash commands, runs scripts in any language (Python, Perl, PHP, Ruby, etc.) or even you can compile code in C/C++. So you could even run program which works all time but still it can be better to use cron for this. If admins will have to restart server then cron will still run script after restart. Commented Jul 20, 2019 at 22:07

3 Answers 3

3

well i think one of the best option is pythonanywhere.com there you can upload your python script(script.py) and then run it and then finish. i did this with my telegram bot

Sign up to request clarification or add additional context in comments.

Comments

2

You can deploy your application using AWS Beanstalk. It will provide you with the whole python environment along with server configuration likely to be changed according to your needs. Its a PAAS offering from AWS cloud.

Comments

0

The best and cheapest solution I have found so far is to use AWS Event Bridge with AWS Lambda.

AWS Lambda allows you upload and execute any script you want in most popular programming languages without needing to pay for a server monthly.

And you can use AWS Event Bridge to trigger an execution of a Lambda function.

You only get charged for what you use in AWS Lambda and it is extremely cheap. Below is the pricing for Lambda in the AWS N. Virginia region. For most scripts, the minimum memory is more than enough. So to run a script every hour for a month that takes 5 seconds to finish, it will cost $0.00756 a month (less than a cent!).

Memory (MB) Price per 1ms
128 $0.0000000021
512 $0.0000000083
1024 $0.0000000167
1536 $0.0000000250
2048 $0.0000000333
3072 $0.0000000500
4096 $0.0000000667
5120 $0.0000000833
6144 $0.0000001000
7168 $0.0000001167
8192 $0.0000001333
9216 $0.0000001500
10240 $0.0000001667

Then you can use AWS Event Bridge to schedule to run an AWS Lambda function every minute, hour, etc.

Here are some articles to help you run any script every minute, hour, etc.

How to Create Lambda Functions in Python

How to Schedule Running an AWS Lambda Function

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.