12

That is, all text and subtags, without the tag of an element itself?

Having

<p>blah <b>bleh</b> blih</p>

I want

blah <b>bleh</b> blih

element.text returns "blah " and etree.tostring(element) returns:

<p>blah <b>bleh</b> blih</p>

7 Answers 7

11

ElementTree works perfectly, you have to assemble the answer yourself. Something like this...

"".join( [ "" if t.text is None else t.text ] + [ xml.tostring(e) for e in t.getchildren() ] )

Thanks to JV amd PEZ for pointing out the errors.


Edit.

>>> import xml.etree.ElementTree as xml
>>> s= '<p>blah <b>bleh</b> blih</p>\n'
>>> t=xml.fromstring(s)
>>> "".join( [ t.text ] + [ xml.tostring(e) for e in t.getchildren() ] )
'blah <b>bleh</b> blih'
>>> 

Tail not needed.

Sign up to request clarification or add additional context in comments.

3 Comments

Just pointing out a typo - method name - "finall" which I think should have been "findall". Even if findall is used it results in this pastebin.com/f6de9a841. Please revise your answer.
I'm doing something similar to that, but with a for look. You are actually missing the tail.
The tail is the extra whitespace after the closing tag of the construct.
9

This is the solution I ended up using:

def element_to_string(element):
    s = element.text or ""
    for sub_element in element:
        s += etree.tostring(sub_element)
    s += element.tail
    return s

5 Comments

That would fail when there's no text or no tail, wouldn't it?
PEZ, yes, it fails when there's no text, just found it by running my code and fixed it. I have many instances of no tail and that doesn't fail. Not sure why.
Just a nitpick: += on strings is less performant. It's best to accumulate a list of strings and ''.join it at the end.
You may want to recurse and call element_to_string on the sub element again to capture all of the text, i.e for sub_element in element: s += element_to_string(sub_element)
For Python 3: ET.tostring(sub_element, encoding='unicode').
3

These are good answers, which answer the OP's question, particularly if the question is confined to HTML. But documents are inherently messy, and the depth of element nesting is usually impossible to predict.

To simulate DOM's getTextContent() you would have to use a (very) simple recursive mechanism.

To get just the bare text:

def get_deep_text( element ):
    text = element.text or ''
    for subelement in element:
        text += get_deep_text( subelement )
    text += element.tail or ''
    return text
print( get_deep_text( element_of_interest ))

To get all the details about the boundaries between raw text:

class holder: pass # this is just a way of creating a holder object
holder.element_count = 0
def get_deep_text_w_boundaries(element, depth = 0):
    holder.element_count += 1
    element_no = holder.element_count 
    indent = depth * '  '
    text1 = f'{indent}(el {element_no} tag {element.tag}: text |{element.text or ""}| - attribs: {element.attrib})' 
    print(text1)
    for subelement in element:
        get_deep_text_w_boundaries(subelement, depth + 1)
    text2 = f'{indent}(el {element_no} tag {element.tag} - tail: |{element.tail or ""}|)' 
    print(text2)
get_deep_text_w_boundaries(etree_element)

Example output:

(el 1 tag source: text |DEVANT LE | - attribs: {})
  (el 2 tag g: text |TRIBUNAL JUDICIAIRE| - attribs: {'style_no': '3'})
  (el 2 tag g - tail: ||)
(el 1 tag source - tail: | DE VERSAILLES|)

Comments

2

I doubt ElementTree is the thing to use for this. But assuming you have strong reasons for using it maybe you could try stripping the root tag from the fragment:

 re.sub(r'(^<%s\b.*?>|</%s\b.*?>$)' % (element.tag, element.tag), '', ElementTree.tostring(element))

Comments

2

Most of the answers here are based on the XML parser ElementTree, even PEZ's regex-based answer still partially relies on ElementTree.

All those are good and suitable for most use cases but, just for the sake of completeness, it is worth noting that, ElementTree.tostring(...) will give you an equivalent snippet, but not always identical to the original payload. If, for some very rare reason, that you want to extract the content as-is, you have to use a pure regex-based solution. This example is how I use regex-based solution.

Comments

0

This answer is slightly modified of Pupeno's reply. Here I added encoding type into "tostring". This issue took many hours of mine. I hope this small correction will help others.

def element_to_string(element):
        s = element.text or ""
        for sub_element in element:
            s += ElementTree.tostring(sub_element, encoding='unicode')
        s += element.tail
        return s

Comments

-4

No idea if an external library might be an option, but anyway -- assuming there is one <p> with this text on the page, a jQuery-solution would be:

alert($('p').html()); // returns blah <b>bleh</b> blih

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.