extracting values by keywords in a pandas column

Question

I have a column that is a list of dictionary. I extracted only the values by the name key, and saved it to a list. Since I need to run the column to a tfidVectorizer, I need the columns to be a string of words. My code is as follows.

def transform(s,to_extract):
return [object[to_extract] for object in json.loads(s)]

cols = ['genres','keywords']

for col in cols:
    lst = df[col]
    df[col] = list(map(lambda x : transform(x,to_extract='name'), lst))
    df[col] = [', '.join(x) for x in df[col]]

for testing, here's 2 rows.

data = {'genres': [[{"id": 851, "name": "dual identity"},{"id": 2038, "name": "love of one's life"}],
               [{"id": 5983, "name": "pizza boy"},{"id": 8828, "name": "marvel comic"}]],
    'keywords': [[{"id": 9663, "name": "sequel"},{"id": 9715, "name": "superhero"}],
                [{"id": 14991, "name": "tentacle"},{"id": 34079, "name": "death", "id": 163074, "name": "super villain"}]]
   }

df = pd.DataFrame(data)

I'm able to extract the necessary data and save it accordingly. However, I find the codes too verbose, and I would like to know if there's a more pythonic way to achieve the same outcome?

Desired output of one row should be a string, delimited only by a comma. Ex, 'Dual Identity,love of one's life'.

BENY · Accepted Answer · 2018-06-27 14:51:54Z

1

Is this what you need ?

df.applymap(lambda x : pd.DataFrame(x).name.tolist())
Out[278]: 
                                genres                   keywords
0  [dual identity, love of one's life]        [sequel, superhero]
1            [pizza boy, marvel comic]  [tentacle, super villain]

Update

df.applymap(lambda x : pd.DataFrame(x).name.str.cat(sep=','))
Out[280]: 
                             genres                keywords
0  dual identity,love of one's life        sequel,superhero
1            pizza boy,marvel comic  tentacle,super villain

edited Jun 27, 2018 at 14:51

answered Jun 27, 2018 at 14:44

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Zoozoo Over a year ago

This is great. However, due to my carelessness, I didnt specify the desired output. I need it to be a string, ie no bracets, just delimiter by a comma.

Collectives™ on Stack Overflow

extracting values by keywords in a pandas column

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related