Python - Scrape Javascript using bs4 and print out the value

Question

So I have been trying to create a script where there is a countdown with epoch time which I later gonna convert it.

The html is following:

<script type="text/javascript">
                new Countdown('countdown_timer', '1547161260', 'https://stackoverflow.com/');
            </script>

and I started to scrape it which I managed to scrape using:

try:
    time_countdown_tag = bs4.find_all('script', {'type': 'text/javascript'})
except Exception:
    time_countdown_tag = []

for countdown in time_countdown_tag:
    if 'new Countdown' in countdown.text.strip():
        print(countdown)

which my output is:

<script type="text/javascript">
                new Countdown('countdown_timer', '1547161260', 'https://stackoverflow.com/');
            </script>

However what I want to print out in this case is the number inside the params which is 1547161260 - I would appreciate all kind of help to be able to be able to only print out the number (epoch) if it is possible?

Yunyu L. · Accepted Answer · 2019-01-05 00:50:32Z

1

You can use regular expressions to match the portion of the JS that contains a positive integer:

import re
output = """<script type="text/javascript">
                new Countdown('countdown_timer', '1547161260', 'https://stackoverflow.com/');
            </script>"""
re.findall("\d+", output)

answered Jan 5, 2019 at 0:50

Yunyu L.

7559 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Hellosiroverthere Over a year ago

Hello! Oh yeah! However I do get TypeError: expected string or bytes-like object when I try this code :O could it be becuase of the spaces \t ?

Hellosiroverthere Over a year ago

My guess is to do maybe countdown.text.strip() before doing re.findall("\d+", output)

Yunyu L. Over a year ago

That would work. I'm guessing you're passing in countdown which is an object, and countdown.text (or str(countdown)) would be the string representation.

Hellosiroverthere Over a year ago

Exactly, I guess thats what my thoughts were, It did work when I passed it as a text.

Collectives™ on Stack Overflow

Python - Scrape Javascript using bs4 and print out the value

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related