1

I'm having issues with my program just closing at random stages and am not sure why.

At first, I thought it was because it was getting an error but I added an error handle. still for some reason it just closes after say a few days of running and no error is displayed. code below

import requests
import lxml.html as lh
import sys
import time
from clint.textui import puts, colored

API_URL = "http://urgmsg.net/livenosaas/ajax/update.php"

class Scraper (object):
    id_stamp = 0

    def __init__(self, timeout, recent_messages=True):
        self.timeout = timeout
        self.handlers = []
        self.recent_messages = recent_messages

    def register_handler(self, handler):
        self.handlers.append(handler)
        return handler

    def scrape(self):
        try:
            resp = requests.get(API_URL, params={'f': self.id_stamp}).json()
        except requests.exceptions.ConnectionError as e:
            puts("Error encountered when connecting to urgmsg: ", newline=False)
            puts(colored.red(e.__class__.__name__), newline=False)
            puts(" " + e.message)
            return

        if not resp['updated']:
            return

        old_id_stamp = self.id_stamp
        self.id_stamp = resp['IDstamp']
        # if old_id_stamp is 0, this is the first scrape
        # which will return a whole bunch of recent past messages
        if not self.recent_messages and old_id_stamp == 0: return

        # Pager messages are returned newest to oldest, we want to
        # process them oldest to newest
        frags = lh.fragments_fromstring(resp['data'])[::-1]
        for frag in frags:
            msg = PagerMessage(frag)
            for handler in self.handlers:
                handler(msg)

    def run(self):
        while True:
            self.scrape()
            time.sleep(self.timeout)

class PagerMessage:
    def __init__(self, fragment):
        children = fragment.getchildren()
        self.datetime = children[0].text
        self.text = children[1].text
        # channel starts with `- `
        self.channel = children[1].getchildren()[0].text[2:]
        self.response = 'CFSRES' in self.text
    def __str__(self):
        return "{} [{}]: {}".format(self.channel, self.datetime, self.text)

if __name__ == "__main__":
    scraper = Scraper(5)
    @scraper.register_handler
    def handler(msg):
        puts(colored.yellow(msg.channel), newline=False)
        puts(" [", newline=False)
        puts(colored.green(msg.datetime), newline=False)
        puts("] ", newline=False)
        if msg.response:
            puts(colored.red(msg.text))
        else:
            puts(msg.text)
    scraper.run()

Have I set this part out wrong ?

except requests.exceptions.ConnectionError as e:
                puts("Error encountered when connecting to urgmsg: ", newline=False)
                puts(colored.red(e.__class__.__name__), newline=False)
                puts(" " + e.message)
                return
5
  • It would be hard to answer, because we don't even know what happens. And it's impossible to reproduce the problem. Try to add logging, use broad exception cases and this will help you to know your error. Commented Jun 14, 2015 at 7:17
  • OK will read up on how to do logging and how would one add and use a broad exception case? Commented Jun 14, 2015 at 7:19
  • Just add extra except: with no exception type in the end. Commented Jun 14, 2015 at 7:20
  • I'll add as solved if you add that in answer area as I can see what you mean about everyone getting an error Commented Jun 14, 2015 at 7:21
  • I really think, that it's an answer. It is just an advice to trace your error. Commented Jun 14, 2015 at 8:18

1 Answer 1

1

As suggested by @sobolevn change

except: as e:
                puts("Error encountered", newline=False)
                puts(colored.red(e.__class__.__name__), newline=False)
                puts(" " + e.message)
                return
Sign up to request clarification or add additional context in comments.

1 Comment

Sorry can't format from mobile will change once home.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.