Python3 XML get text between tags

Question

I have the following code in Python 3. I am using the import xml.etree.ElementTree as ET for XML parsing. the webScraper grab the text from an webside but on that website there is text between the <link></link> tag, but the program returns None. I can se that the program finds all tags but where the tag result should be printed it only says None.

    result = webScrapper.scrappPart("http://www.dn.se/rss/senaste-nytt/", "body")
    root = ET.fromstring(result)
    for items in root.findall('.//item'):
        link = items.find('link')
        print(link.text)

Does anyone know how to fix this?

What is webScrapper.scrappPart()? Does it return the XML or HTML of the transformed XML? (since it mentioned "body" while there is no <body> in the XML source. Too far a guess?) — har07
– har07, Commented May 7, 2016 at 23:19

MattDMo · Accepted Answer · 2016-05-07 22:11:21Z

Since your URL is actually an RSS feed, you'd be much better off using an RSS feed parser on it, instead of trying to roll your own. Fortunately, this is why feedparser exists. Check this out:

import feedparser as fp

feed = fp.parse("http://www.dn.se/rss/senaste-nytt/")
for entry in feed["entries"]:
    print(entry["link"])

This returns

http://www.dn.se/sport/fotboll/cavani-het-i-svalt-psg/
http://www.dn.se/sport/fotbolls-em/kompany-missar-em/
http://www.dn.se/nyheter/sverige/livvaktens-slakting-fick-praktik-hos-sahlin-trots-myndighetens-avslag/
http://www.dn.se/sport/st-louis-andraperiod-avgjorde/
http://www.dn.se/nyheter/varlden/syrien-spanska-journalister-fria/
http://www.dn.se/sport/dansk-dynamit-ska-stoppa-tre-kronor/
http://www.dn.se/nyheter/sverige/mordmisstankt-slappt-ur-haktet-1/
http://www.dn.se/nyheter/varlden/ekonomiprofessor-loste-ekvation-togs-for-terrorist/
http://www.dn.se/sport/fotboll/leicester-firade-med-storseger/
http://www.dn.se/ekonomi/protester-mot-ny-granskontroll-urartade/
http://www.dn.se/sport/ishockey-vm/jimmie-ericsson-jag-ar-beredd-gora-allt-for-att-vinna/
http://www.dn.se/sport/ishockey-vm/schweiz-straffat-av-kazakstan/
http://www.dn.se/nyheter/varlden/natosoldater-dodade-i-afghanistan-2/
http://www.dn.se/sport/forsta-matchen-till-eslov/
http://www.dn.se/nyheter/sverige/drunknad-man-hittad-av-dykare/
http://www.dn.se/ekonomi/tagstopp-efter-olycka/
http://www.dn.se/sport/kristianstad-till-sm-final/
http://www.dn.se/sthlm/en-person-attackerad-med-kniv-i-centrala-stockholm/
http://www.dn.se/nyheter/sverige/inga-spar-efter-forsvunnen-22-arig-student/
http://www.dn.se/sport/fotboll/forlust-for-rydstrom-i-tranardebuten/
http://www.dn.se/nyheter/sverige/manga-grasbrander-runt-om-i-landet/
http://www.dn.se/nyheter/sverige/tre-gripna-efter-skottlossning-i-malmo/
http://www.dn.se/sport/fotboll/elfsborg-ar-med-i-toppen-igen/
http://www.dn.se/sport/em-silver-till-rissveds/

which I assume is what you're looking for.

Padraic Cunningham · Accepted Answer · 2016-05-08 10:24:59Z

You can use ElementTree just fine, you just need to pass the source and use the xpath:

from xml.etree import ElementTree as et
import requests

tree = et.fromstring(requests.get("http://www.dn.se/rss/senaste-nytt/").content)

print([x.text for x in tree.findall(".//item//link")])

Output:

['http://www.dn.se/nyheter/varlden/andlig-ledare-ihjalhackad-i-bangladesh/', 'http://www.dn.se/nyheter/sverige/tillstandet-battre-for-pakord-ettaring/', 'http://www.dn.se/ekonomi/maria-crofts-dags-att-gora-nagot-at-orattvisa-pensioner/', 'http://www.dn.se/nyheter/varlden/turkisk-militar-dodade-55-is-krigare/', 'http://www.dn.se/nyheter/varlden/massiv-fiskdod-i-sjo/', 'http://www.dn.se/nyheter/varlden/kanadabranden-i-bilder/', 'http://www.dn.se/nyheter/sverige/manga-saknas-efter-jordskred-i-kina/', 'http://www.dn.se/nyheter/sverige/fortsatt-sokande-efter-student/', 'http://www.dn.se/nyheter/sverige/en-dod-i-villabrand-8/', 'http://www.dn.se/nyheter/politik/v-vill-ta-bort-terrorstampel-pa-pkk/', 'http://www.dn.se/ekonomi/raknehjalp-pa-natet-ger-ratt-underhall/', 'http://www.dn.se/nyheter/varlden/kanadabranden-fullstandigt-okontrollerad/', 'http://www.dn.se/nyheter/varlden/attentat-mot-journalister-besvarande-for-erdogan/', 'http://www.dn.se/nyheter/varlden/superlobbyist-ska-gora-trump-serios/', 'http://www.dn.se/nyheter/vetenskap/karin-bojs-en-typisk-foralder-ar-28-ar-gammal/', 'http://www.dn.se/sport/nervos-vantan-pa-em-biljetter/', 'http://www.dn.se/ekonomi/ovantat-stort-exportfall-i-kina/', 'http://www.dn.se/ekonomi/lott-gav-35-miljarder-i-vinst-i-usa/', 'http://www.dn.se/nyheter/vetenskap/fabels-kansliga-nos-ska-ge-svar-om-massmordet/', 'http://www.dn.se/sport/johan-esk-nu-borde-idrotten-lara-ledarskap-av-naringslivet-1/', 'http://www.dn.se/sport/melker-karlsson-malskytt-for-san-jose/', 'http://www.dn.se/sport/backstrom-visade-vagen-till-washingtons-viktiga-vinst/', 'http://www.dn.se/nyheter/varlden/15-miljoner-signaturer-bekraftade/', 'http://www.dn.se/nyheter/varlden/medan-du-sov-varlden-i-korthet-8-maj-1/', 'http://www.dn.se/nyheter/varlden/karnvapen-bara-om-landet-hotas/', 'http://www.dn.se/nyheter/varlden/protester-mot-att-avskaffa-senat/', 'http://www.dn.se/nyheter/sverige/industri-brann-i-uppsala/', 'http://www.dn.se/nyheter/varlden/atta-poliser-dodade-i-attack/', 'http://www.dn.se/nyheter/varlden/tva-fast-vid-myr-utanfor-kiruna/', 'http://www.dn.se/sport/hockeyhasten-nyquist-vann-kentucky-derby/', 'http://www.dn.se/nyheter/varlden/15-miljoner-signaturer-bekraftade-i-venezuela/']

Or using lxml which can also get the source for you:

from lxml import etree

result = etree.parse("http://www.dn.se/rss/senaste-nytt/")
print(result.xpath("//item//link//text()"))

Which gives you the exact same output.

Collectives™ on Stack Overflow

Python3 XML get text between tags

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related