Extracting strings surrounded by html tags from within a list?

Question

I used BeautifulSoup to parse a website and store the content. It is in this form:

records = [[[<p>data_1_1</p>], [<p>data_1_2</p>],[], [<li>data_1_3</li>]],
           [[<p>data_2_1</p>], [<p>data_2_2</p>], [], [<li>data_2_3</li>]]]

I am having trouble making this:

records = [["data_1_1", "data_1_2", "data_1_3"],
           ["data_2_1", "data_2_2", "data_2_3"]]

I tried list comprehensions:

text_records = [sum(record, []) for record in records]

but the text is still wrapped in <p> or <li> tags.

text_records = [item.string for item in sum(record, []) for record in records]

takes the text out of tags, but this gives one large list, with the same values repeated multiple times.

I know there is plenty out there on list comprehensions in python, and I've searched SO, but I can't find anything to help with this situation.

CasualDemon · Accepted Answer · 2014-01-28 13:37:48Z

1

Edit - This will work even for multiple items:

[sum([v.string for v in [item for item in record if item]], []) for record in records]

Adding the sum will make sure all the lists are combined into a single one per record.

Original:

This should work fine as long as you will always only have internal lists of a single item:

[[item[0].string for item in row if item] for row in records]

This will go through each record, make sure that the record exists with the if statement, and then append the first element of the list to the new record in it's string format.

edited Jan 28, 2014 at 13:37

answered Jan 28, 2014 at 13:09

CasualDemon

6,2083 gold badges25 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Christian Lemp Over a year ago

thank you, this worked. why item[0] though? what else is in there?

CasualDemon Over a year ago

Welcome, glad it worked for you! It's not that there are additional items in there, it's simply that it's a list object. So you have to extract the item out of the list (even if there is just one) or else it will be treated as a separate list.

Azwr · Accepted Answer · 2014-01-28 13:10:08Z

0

This will do the job just fine(although this many for loops is annoying,any suggestion is welcome).

records1 = [BeautifulSoup(k).text for i in records for j in i for k in j]

answered Jan 28, 2014 at 13:10

Azwr

7848 silver badges13 bronze badges

Collectives™ on Stack Overflow

Extracting strings surrounded by html tags from within a list?

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related