Regular expressions in python unicode

Question

I need to remove all the html tags from a given webpage data. I tried this using regular expressions:

import urllib2
import re
page = urllib2.urlopen("http://www.frugalrules.com")
from bs4 import BeautifulSoup, NavigableString, Comment
soup = BeautifulSoup(page)
link = soup.find('link', type='application/rss+xml')
print link['href']
rss = urllib2.urlopen(link['href']).read()
souprss = BeautifulSoup(rss)
description_tag = souprss.find_all('description')
content_tag = souprss.find_all('content:encoded')
print re.sub('<[^>]*>', '', content_tag)

But the syntax of the re.sub is:

re.sub(pattern, repl, string, count=0)

So, I modified the code as (instead of the print statement above):

for row in content_tag:
    print re.sub(ur"<[^>]*>",'',row,re.UNICODE

But it gives the following error:

Traceback (most recent call last):

File "C:\beautifulsoup4-4.3.2\collocation.py", line 20, in <module>
print re.sub(ur"<[^>]*>",'',row,re.UNICODE)
File "C:\Python27\lib\re.py", line 151, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer

What am I doing wrong?

Can you not find a minimal code example that also fails? For example, remove all non stdlib dependencies bs4 unless they are crucial. If they are, then add a tag for them. This makes the question easier to answer and more useful. — Ciro Santilli OurBigBook.com
– Ciro Santilli OurBigBook.com, Commented Nov 13, 2013 at 15:46
I know parsing HTML with RegEx is a sin, but umm, I really couldn't remove the tags any other way. Could you please suggest me a working method instead? :) — Remy
– Remy, Commented Nov 13, 2013 at 16:08

Qui · Accepted Answer · 2013-11-13 15:52:02Z

1

Last line of your code try:

print(re.sub('<[^>]*>', '', str(content_tag)))

answered Nov 13, 2013 at 15:52

Qui

1088 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Qui Over a year ago

sorry - my code is written for python 3 try print re.sub('<[^>]*>', '', str(content_tag))

Collectives™ on Stack Overflow

Regular expressions in python unicode

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related