Python Scraping Problem with Beautifulsoup

Question

Using python I am trying to get the whole html code of a target CSS selector. The target html code looks like following and I want this whole code:

<img class="card-img-top" src="/svg/2798804.svg" width="200px" height="200px" alt="kitten animal cat " style="user-select: auto;">

My python code :

with open("links.txt", "r") as a_file:
  for line in a_file:
    stripped_line = line.strip()
    endpoint = stripped_line
    start = stripped_line.find('/tag/') + 5
    end = stripped_line.find('.html', start)
    filename = stripped_line[start:end]
    soup = BeautifulSoup(requests.get(endpoint).text)
    completeName = os.path.join(workingpath, filename + ".txt")
    with open(completeName, "w") as f_out:
        for inp in soup.select('.card-img-top'):
            print(inp["value"], file=f_out)

Getting this error when running the code:

KeyError: 'value'

Do you expect the <img> to be an <input>?

Klaus D.
– Klaus D.

2021-08-28 05:18:34 +00:00
Commented Aug 28, 2021 at 5:18 — Klaus D.
– Klaus D., Commented Aug 28, 2021 at 5:18

Corralien · Accepted Answer · 2021-08-28 05:25:42Z

1

There is no problem with BeautifulSoup. value attribute does not exist in your html fragment (because img has no such attribute, but input has it as suggested by @KlausD.)

>>> inp.attrs
{'class': ['card-img-top'],
 'src': '/svg/2798804.svg',
 'width': '200px',
 'height': '200px',
 'alt': 'kitten animal cat ',
 'style': 'user-select: auto;'}

All possible attributes for img tag can be found here.

answered Aug 28, 2021 at 5:25

Corralien

121k8 gold badges44 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Katherine Elizabeth Kath Over a year ago

Thanks @corralien I am fairly new in this field. Now it is working fine. But I need to get the 'src' and 'alt' at the same time. Then save to a text file line by line. Can you help me with that. As I said I am new and tried a few ideas but failed.

Katherine Elizabeth Kath Over a year ago

Succeeded with this "print(inp.attrs["src"], inp.attrs["alt"], sep="\n", file=f_out)"

Collectives™ on Stack Overflow

Python Scraping Problem with Beautifulsoup

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related