Running Python function at defined intervals

Question

I've got a Python loop that should run every minute, do some data processing and sleep until the next minute is up. However the processing takes variable amount of time, sometimes it's close to zero when there is not much to do, sometimes it takes even 10 or 20 seconds.

To compensate for that I measure the time it takes to run the processing like this:

while True:
  time_start = time.time()
  do_something()                        # <-- This takes unknown time
  time_spent = time.time() - time_start
  time.sleep(60 - time_spent)

It kind of works but over a couple of days it still drifts away by a number of seconds. I guess it happens when the computer (small Raspberry Pi) is busy and delays the start of the loop, then it all starts slipping away.

I don't need do_something() executed exactly every minute, so no need for real-time OS or anything like that, but I don't want one delayed start affect all the subsequent ones either.

Is there some kind of scheduler that can run my function at a predefined rate? Or some more clever way to compensate for the occasional delays?

if you really want this kind of concurrency I think you have to use threads. one thread as timer and another as worker. — prhmma
– prhmma, Commented Nov 3, 2019 at 21:30
@phalanx what if the timer thread gets delayed / slips away? I don't think it's any different from the timestamp-compensation method I use. Unless you mean some (hardware-)scheduled timer? — KeepLearning
– KeepLearning, Commented Nov 3, 2019 at 21:43
Possible duplicate of In Python, how can I put a thread to sleep until a specific time? — quamrana
– quamrana, Commented Nov 3, 2019 at 21:54
@quamrana Thanks, but that's quite an over-complicated solution for a simple fixed-rate trigger that I'm after. My solution below seems to work in my case. — KeepLearning
– KeepLearning, Commented Nov 3, 2019 at 23:52

KeepLearning · Accepted Answer · 2019-11-03 22:30:39Z

1

Playing with the loop a little this seems to work quite well. The trick is to record the start time once before the loop starts, not on every iteration. That way one delayed start won't affect any future ones.

rate_sec = 60
time_start = time.time()

while True:
    print("{}".format(datetime.now()))

    # Emulate processing time 0s to 20s
    time.sleep(random.randint(0, 20))

    # Sleep until the next 'rate_sec' multiple
    delay = rate_sec - (time.time() - time_start) % rate_sec
    time.sleep(delay)

answered Nov 3, 2019 at 22:30

KeepLearning

3792 gold badges3 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

weulerfilho · Accepted Answer · 2019-11-03 21:35:19Z

0

Is sleeping a pre-requisite of your project? I mean, you don't need to have your processing blocked if you want to run the task every ~1 minute.

Since you are on a Raspberry Pi, you can (and probably should) use crontab.

This will give you the most flexibility and allow you to don't have the computer sleeping idle.

To do this, open your cron tab with

crontab -e

And add the following entry

* * * * * /usr/bin/env python3 /path/to/script.py

answered Nov 3, 2019 at 21:35

weulerfilho

614 bronze badges

1 Comment

KeepLearning Over a year ago

I was considering that but there is quite a lengthy start up procedure and calibration that takes a couple of minutes on RPi, so this is not an option unfortunately. It's ok to do it during RPi boot when the program starts but not on every iteration. Thanks for the suggestion though :)

Collectives™ on Stack Overflow

Running Python function at defined intervals

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related