0

I am trying to get a python program to continuously run until a certain aws log is registered and printed. It is supposed to:

  1. Run indefinitely even if no events happen
  2. List whatever events occurs (in my case a log stating that the task is finished)
  3. Stop running

the python command looks like this: python3 watch_logs.py <log-source> --start=15:00:00

The logs are working fine, and the python script can print them out between certain time frames as long as they already exist. The program works by taking a continuously running task which prints events to the log file, and the python script should filter out events I am looking for and print them.

But, when I run the script, it wont print the event even if I can see the log entry appear in the file. If i kill the process and run it again using the same timestamp, it will find the log entry and end the script like it should.

the code is fairly short: 

logs = get_log_events(
                log_group=log_group,
                start_time=start_time,
                end_time=end_time
            )
while True:
    for event in logs:
        print(event['message'].rstrip())
        sys.exit("Task complete")

Any insight why this is happening would help a lot. I am fairly new to python

Improve this question
3
  • 5
    What exactly is logs in that code? This example is not complete. Commented Aug 7, 2019 at 14:21
  • If your code ever would reach the section shown, there could be 3 possible actions: 1. logs is an empty collection or iterator: infinite loop due to while True: - 2. logs is a collection with just one element: something would be printed, program would terminate. - 3. logs is a generator function that did not yet generate anything: your program would wait until something is generated. - it looks like the first case is true, and logs is just an empty collection. Commented Aug 7, 2019 at 14:44
  • logs refers to a connection to an aws logging service. It vomits out thousands of events every minute. The code is built to connect to it, filter what I need, and print out the filtered items. I added the snippet that defines this. It might be useful. Commented Aug 8, 2019 at 8:11

4 Answers 4

Reset to default
1

The value in the variable logs is old when the file is updated. You need to update this variable. For example if you were to use logs = myfile.read() at start of your script, the value in the logs variable would be a snapshot of that file at that time.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks. this comment helped lead me to the answer! I put the variable that defines the logs into the loop as well, so that was being defined at each cycle. I'll update my post
0

Try storing event['message'].rstrip() in a variable and checking with an if statement if it corresponds to the log you want to find

Improve this answer

Comments

0

If you don't want to read the file each time through the loop, you should have a look at pygtail (https://pypi.org/project/pygtail/).

Improve this answer

Comments

0

I was overthinking the problem. I put the log variable inside the loop so it was being defined at each cycle


while True:
    logs = get_log_events(
        log_group=log_group,
        start_time=start_time,
        end_time=end_time
        )
        for event in logs:
            print(event['message'].rstrip())
            sys.exit("Task complete")
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.