How to detect either the string contains an html (can be html4, html5, just partials of html within text)? I do not need a version of HTML, but rather if the string is just a text or it contains an html. Text is typically multiline with also empty lines
example inputs:
html:
<head><title>I'm title</title></head>
Hello, <b>world</b>
non-html:
<ht fldf d><
<html><head> head <body></body> html
You can use an HTML parser, like BeautifulSoup. Note that it really tries it best to parse an HTML, even broken HTML, it can be very and not very lenient depending on the underlying parser:
>>> from bs4 import BeautifulSoup
>>> html = """<html>
... <head><title>I'm title</title></head>
... </html>"""
>>> non_html = "This is not an html"
>>> bool(BeautifulSoup(html, "html.parser").find())
True
>>> bool(BeautifulSoup(non_html, "html.parser").find())
False
This basically tries to find any html element inside the string. If found - the result is True.
Another example with an HTML fragment:
>>> html = "Hello, <b>world</b>"
>>> bool(BeautifulSoup(html, "html.parser").find())
True
Alternatively, you can use lxml.html:
>>> import lxml.html
>>> html = 'Hello, <b>world</b>'
>>> non_html = "<ht fldf d><"
>>> lxml.html.fromstring(html).find('.//*') is not None
True
>>> lxml.html.fromstring(non_html).find('.//*') is not None
False
non-html = "<html><head> head <body></body> html" bool(BeautifulSoup(non_html, "html.parser").find()) => True? It is not an html snippetnon_html = "<html head> head <body jdkjs /body> html dkslfjglangaiowmgiowe" will pass the test :(
True, cause BeautifulSoup tries it best to parse the html and be lenient. It would transform it to <html><head> head <body></body> html</head></html>.non_html = "<ht fldf d><" will also workfldf and d are attributes and ht tag is just not closed. Nice examples, thanks :)One way I thought of was to intersect start and end tags found by attempting to parse the text as HTML and intersecting this set with a known set of acceptable HTMl elements.
Example:
#!/usr/bin/env python
from __future__ import print_function
from HTMLParser import HTMLParser
from html5lib.sanitizer import HTMLSanitizerMixin
class TestHTMLParser(HTMLParser):
def __init__(self, *args, **kwargs):
HTMLParser.__init__(self, *args, **kwargs)
self.elements = set()
def handle_starttag(self, tag, attrs):
self.elements.add(tag)
def handle_endtag(self, tag):
self.elements.add(tag)
def is_html(text):
elements = set(HTMLSanitizerMixin.acceptable_elements)
parser = TestHTMLParser()
parser.feed(text)
return True if parser.elements.intersection(elements) else False
print(is_html("foo bar"))
print(is_html("<p>Hello World!</p>"))
print(is_html("<html><head><title>Title</title></head><body><p>Hello!</p></body></html>")) # noqa
Output:
$ python foo.py
False
True
True
This works for partial text that contains a subset of HTML elements.
NB: This makes use of the html5lib so it may not work for other document types necessarily but the technique can be adapted easily.
HTMLSanitizerMixin anymore in html5lib.If all you need to know is wether or not a string contains html text then another solution not listed here would be to use a regex expression like the following:
</?\s*[a-z-][^>]*\s*>|(\&(?:[\w\d]+|#\d+|#x[a-f\d]+);)
Bear in mind that although this would be a much faster solution than using an HTML Parser, it can be potentially inaccurate depending on the complexity of html markup you're expecting.
Here is a test of the above regex for a general idea of it's coverage.
You can easily extend the built-in HTMLParser that already handles the parsing, and collect (start/end) tags, attrs, and data. To assert whether the document is valid the amount of start tags should match the amount of end tags:
from html.parser import HTMLParser
class MyHTMLParser(HTMLParser):
def __init__(self):
super().__init__()
self.start_tags = list()
self.end_tags = list()
self.attributes = list()
def is_text_html(self):
return len(self.start_tags) == len(self.end_tags)
def handle_starttag(self, tag, attrs):
self.start_tags.append(tag)
self.attributes.append(attrs)
def handle_endtag(self, tag):
self.end_tags.append(tag)
def handle_data(self, data):
print("Encountered some data :", data)
Then
>>> parser = MyHTMLParser()
>>> parser.feed("<head><title>I'm title</title></head>"
"Hello, <b>world</b>")
>>> parser.is_text_html()
True
>>> parser.feed("<ht fldf d><"
"<html><head> head <body></body> html")
>>> parser.is_text_html()
False
Check for ending tags. This is simplest and most robust I believe.
"</html>" in possibly_html
If there is an ending html tag, then it looks like html, otherwise not so much.
</html> in it. It won't work for partial HTML (like <b>some word</b>).
Expanding on the previous post I would do something like this for something quick and simple:
import sys, os
if os.path.exists("file.html"):
checkfile=open("file.html", mode="r", encoding="utf-8")
ishtml = False
for line in checkfile:
line=line.strip()
if line == "</html>"
ishtml = True
if ishtml:
print("This is an html file")
else:
print("This is not an html file")
