1

I’ve been trying to scrape lottery results from a website that shows draws. The data is presented in a results table, but I keep running into strange issues where sometimes the numbers are captured correctly, and other times the scraper either misses some rows or duplicates entries.

Code:

import requests
from bs4 import BeautifulSoup

url = "https://star49s.com/results/teatime"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

results = []
for row in soup.select("div.results-row"):
    numbers = [num.get_text(strip=True) for num in row.select("span.ball")]
    date = row.select_one("span.date").get_text(strip=True) if row.select_one("span.date") else None
    if numbers:
        results.append({"date": date, "numbers": numbers})

print(results)

The output is inconsistent:

  • Sometimes it captures the latest results fine.

  • Other times it skips certain rows.

  • Occasionally, I get duplicated results when re-running the script.

I suspect the page might be partially dynamic (maybe using JavaScript rendering) or that I’m not selecting the correct elements. I’m not sure if I should switch to something like Selenium or Playwright, or if I just need to refine my CSS selectors.

What’s the most reliable way to scrape a results page like this so I can always get the latest draw numbers without missing or duplicating rows? Should I rely on a headless browser or is there a way to do this cleanly with requests + BeautifulSoup?

Improve this question
4
  • Take a look at Selenium, which can sometimes be better at handling dynamic output. Commented Aug 27 at 10:52
  • Hi @OldBoy. Can you please confirm that is star49s teatime link is allowing for scraping or not? Commented Aug 27 at 10:59
  • @Zuryab, the CSS selectors you're using don't seem to be there on the scraped HTML page. Can you attach a screenshot of which element you're trying to extract? Commented Aug 27 at 11:27
  • I have no association with that website, so I cannot possibly confirm anything about it. Commented Aug 27 at 12:00

1 Answer 1

Reset to default
3

You can grab the <script type="application/ld+json"> block with BeautifulSoup, then load it into a Python dict using json.loads(). From there, you can access the itemListElement list directly. It contains all the data.

import requests
import json
from bs4 import BeautifulSoup

url = "https://star49s.com/results/teatime"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

# Find the JSON-LD <script> tag
script_tag = soup.find("script", {"type": "application/ld+json"})

if script_tag:
    # Load JSON into a Python dictionary
    data = json.loads(script_tag.string)

    # Get the itemListElement
    results = []
    for node in data.get("@graph", []):
        if node.get("@type") == "WebPage":
            main_entity = node.get("mainEntity", {})
            if main_entity.get("@type") == "ItemList":
                results = main_entity.get("itemListElement", [])

    print(results)

results:

[
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 26-08-2025',
    'description': 'UK49s Teatime lottery result for 26-08-2025 with winning numbers: 14, 19, 24, 27, 35, 38 and booster ball: 7.',
    'datePublished': '26-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 25-08-2025',
    'description': 'UK49s Teatime lottery result for 25-08-2025 with winning numbers: 3, 16, 25, 29, 32, 38 and booster ball: 39.',
    'datePublished': '25-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 24-08-2025',
    'description': 'UK49s Teatime lottery result for 24-08-2025 with w inning numbers: 5, 8, 10, 28, 39, 47 and booster ball: 2.',
    'datePublished': '24-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    ' name': 'UK49s Teatime Results - 23-08-2025',
    'description': 'UK49s Teatime lottery result for 23-08-2025 with winning numbers: 3, 13, 15, 26, 29, 42 and booster ball: 44.',
    'datePu blished': '23-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 22-08-2025',
    'description': 'UK49s  Teatime lottery result for 22-08-2025 with winning numbers: 5, 30, 31, 34, 44, 49 and booster ball: 17.',
    'datePublished': '22-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s .com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 21-08-2025',
    'description': 'UK49s Teatime lottery result for 21-08-2025 with winning numbers: 2, 4, 3 7, 40, 43, 49 and booster ball: 44.',
    'datePublished': '21-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime  Results - 20-08-2025',
    'description': 'UK49s Teatime lottery result for 20-08-2025 with winning numbers: 11, 23, 26, 38, 39, 43 and booster ball: 32.',
    'datePublished': '20-08-2025 ',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 19-08-2025',
    'description': 'UK49s Teatime lottery resul t for 19-08-2025 with winning numbers: 22, 30, 31, 32, 35, 42 and booster ball: 14.',
    'datePublished': '19-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime '
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 18-08-2025',
    'description': 'UK49s Teatime lottery result for 18-08-2025 with winning numbers: 9, 14, 17, 24, 29, 34 and b ooster ball: 48.',
    'datePublished': '18-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 17-08-202 5',
    'description': 'UK49s Teatime lottery result for 17-08-2025 with winning numbers: 2, 8, 29, 33, 41, 49 and booster ball: 45.',
    'datePublished': '17-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 16-08-2025',
    'description': 'UK49s Teatime lottery result for 16-08-2025 with  winning numbers: 5, 13, 29, 30, 36, 49 and booster ball: 21.',
    'datePublished': '16-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset ',
    'name': 'UK49s Teatime Results - 15-08-2025',
    'description': 'UK49s Teatime lottery result for 15-08-2025 with winning numbers: 5, 15, 31, 33, 36, 43 and booster ball: 22.',
    'da tePublished': '15-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 14-08-2025',
    'description': 'UK 49s Teatime lottery result for 14-08-2025 with winning numbers: 11, 18, 20, 36, 39, 45 and booster ball: 21.',
    'datePublished': '14-08-2025',
    'inLanguage': 'en',
    'url': 'https://st ar49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s Teatime Results - 13-08-2025',
    'description': 'UK49s Teatime lottery result for 13-08-2025 with winning numbers: 14 , 17, 18, 24, 26, 37 and booster ball: 40.',
    'datePublished': '13-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  },
  {
    '@type': 'Dataset',
    'name': 'UK49s T eatime Results - 12-08-2025',
    'description': 'UK49s Teatime lottery result for 12-08-2025 with winning numbers: 8, 19, 30, 38, 48, 49 and booster ball: 16.',
    'datePublished': '12-08-2025',
    'inLanguage': 'en',
    'url': 'https://star49s.com/results/teatime'
  }
]

and then you can parse it to extract whatever data you need from the results

However, the very last <script> tag holds even more data since 23-12-2024

here's how you can get them all:

import requests
import json
import re
from bs4 import BeautifulSoup

url = "https://star49s.com/results/teatime"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

script_tag = soup.find_all("script")[-1]

if script_tag:
    # parse the embedded escaped json data using regex
    match = re.search(r'\\"sortedTea\\":(\[.*\])}\]\]', script_tag.string)
    if match:
        # First, unescape the string using json.loads
        escaped_json = match.group(1)
        unescaped_json = json.loads(f'"{escaped_json}"')  # decode escaped quotes
        # Then parse it as JSON
        results = json.loads(unescaped_json)
        print(results)

results:

[
  {
    "id": "68ade81632b3b2ea76aefbbc",
    "date": "26-08-2025",
    "draw": "Teatime",
    "time": "12:00:02",
    "numbers": [
      14,
      19,
      24,
      27,
      35,
      38
    ],
    "boosterBall": 7
  },
  {
    "id": "68ac9a1932b3b2 ea76aeec49",
    "date": "25-08-2025",
    "draw": "Teatime",
    "time": "12:15:02",
    "numbers": [
      3,
      16,
      25,
      29,
      32,
      38
    ],
    "boosterBall": 39
  },
  {
    "id": "68ab451332b3b2ea76aedcbe",
    "date": "2 4-08-2025",
    "draw": "Teatime",
    "time": "12:00:02",
    "numbers": [
      5,
      8,
      10,
      28,
      39,
      47
    ],
    "boosterBall": 2
  },
  {
    "id": "68aab4f332b3b2ea76aed5b2",
    "date": "23-08-2025",
    "draw": "Teat ime",
    "time": "01:45:02",
    "numbers": [
      3,
      13,
      15,
      26,
      29,
      42
    ],
    "boosterBall": 44
  },
  {
    "id": "68aab4f332b3b2ea76aed5b8",
    "date": "22-08-2025",
    "draw": "Teatime",
    "time": "01:45:02 ",
    "numbers": [
      5,
      30,
      31,
      34,
      44,
      49
    ],
    "boosterBall": 17
  },
  {
    "id": "68aab4f432b3b2ea76aed5bf",
    "date": "21-08-2025",
    "draw": "Teatime",
    "time": "01:45:02",
    "numbers": [
      2,
      4,
      37,
      40,
      43,
      49
    ],
    "boosterBall": 44
  },
  {
    "id": "68aab4f532b3b2ea76aed5c5",
    "date": "20-08-2025",
    "draw": "Teatime",
    "time": "01:45:02",
    "numbers": [
      11,
      23,
      26,
      38,
      39,
      43
    ],
    "boosterBall": 32
  },
  {
    "id": "68aab4f532b3b2ea76aed5cb",
    "date": "19-08-2025",
    "draw": "Teatime",
    "time": "01:45:02",
    "numbers": [
      22,
      30,
      31,
      32,
      35,
      42
    ],
    "boosterBall": 14
  },
  {
    "id": "68aab4f632 b3b2ea76aed5d1",
    "date": "18-08-2025",
    "draw": "Teatime",
    "time": "01:45:02",
    "numbers": [
      9,
      14,
      17,
      24,
      29,
      34
    ],
    "boosterBall": 48
  },
  .
  .
  .
  {
    "id": "687257592a271ce505a0ea2e",
    "date": "29-12-2024",
    "draw": "Teatime",
    "time": "18:00:45",
    "numbers": [
      2,
      11,
      13,
      18,
      20,
      22
    ],
    "boosterBall": 29
  },
  {
    "id": "687257592a 271ce505a0ea2b",
    "date": "28-12-2024",
    "draw": "Teatime",
    "time": "18:00:45",
    "numbers": [
      19,
      23,
      34,
      35,
      38,
      45
    ],
    "boosterBall": 27
  },
  {
    "id": "687257582a271ce505a0ea28",
    "date\ ": "27-12-2024",
    "draw": "Teatime",
    "time": "18:00:45",
    "numbers": [
      1,
      14,
      23,
      28,
      45,
      47
    ],
    "boosterBall": 24
  },
  {
    "id": "687257582a271ce505a0ea25",
    "date": "26-12-2024",
    "draw": "Teatime",
    "time": "18:00:45",
    "numbers": [
      3,
      15,
      17,
      23,
      24,
      28
    ],
    "boosterBall": 26
  },
  {
    "id": "687257572a271ce505a0ea22",
    "date": "24-12-2024",
    "draw": "Teatime",
    "time": "1 8:00:45",
    "numbers": [
      1,
      7,
      10,
      16,
      26,
      28
    ],
    "boosterBall": 5
  },
  {
    "id": "687257572a271ce505a0ea1f",
    "date": "23-12-2024",
    "draw": "Teatime",
    "time": "18:00:45",
    "numbers": [
      14,
      23,
      28,
      29,
      36,
      39
    ],
    "boosterBall": 22
  }
]
  • If the order of this script tag changes (i.e. if it's not the last one) then simply find all the script tags and use the same regex pattern to match the target script tag holding all the data.
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Hello @ajeeed Vema. Thanks for this code. Will this method would be work for this page also because i am trying to scrape the multiple pages from star49s: lunchtime results page
I didn't fully understand your question, but the above solution already works for https://star49s.com/results/teatime URL and gets all the data available. Now, if you mean another page on this website, then you need to check if that page's HTML contains a similar structure or not.
Yes, I just checked. the same code works for https://star49s.com/results/lunchtime as well. the only modification is to replace the regex pattern. for star49s.com/results/lunchtime, use r'\\"sortedLunch\\":(\[.*\])}\]\]'
thanks a lot. I will check it shortly and informed if data is scraping correctly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.