List of lists to rows in a pandas dataframe

Question

After using my script my algorithms return the exptected outcome in a list of lists like this: pred=[[b,c,d],[b,a,u],...[b,i,o]]

I already have a dataframe that needs those values added in a new matching column. The list is exactly x long like the other columns in the frame and I just need to create a new column with all the values of the lists.

However when I try to put the list into the column I get the error:

ValueError: Length of values does not match length of index

Looking at the data, it puts the entire list into one row instead of each entry in a new row.

EDIT:

All values in the list should be put in the column namend pred

sent  token   pred
 0     a        b
 0     b        c
 0     b        d
 1     a        b
 1     b        a
 1     c        u

Solution:

x = []
for _ in pred:
  if _ is not None:
    x += _

df_new = pd.DataFrame(df)
df_new["pred"] = list(itertools.chain.from_iterable(x))

Can you show the code you have tried? As well as a sample input and expected output? — user3483203
– user3483203, Commented May 14, 2018 at 17:25

jpp · Accepted Answer · 2018-05-14 18:39:53Z

2

You can use itertools.chain, which allows you to flatten a list of lists, which you can then slice according to the length of your dataframe.

Data from @ak_slick.

import pandas as pd
from itertools import chain

df = pd.DataFrame({'sent': [0, 0, 0, 1, 1, 1], 
                   'token': ['a', 'b', 'b', 'a', 'b', 'c']})

lst = [['b','c',None],['b',None,'u'], ['b','i','o']]

df['pred'] = list(filter(None, chain.from_iterable(lst)))[:len(df.index)]

print(df)

   sent token pred
0     0     a    b
1     0     b    c
2     0     b    d
3     1     a    b
4     1     b    a
5     1     c    u

edited May 14, 2018 at 18:39

answered May 14, 2018 at 18:34

jpp

166k37 gold badges301 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mi. Over a year ago

Thank you! I had to clear NoneType entries first but this seems to work.

jpp Over a year ago

@ThelMi, I updated data for None values, just use filter(None, chain...). Note this method will remove all False values (e.g. None, 0, False, empty string, etc). Otherwise you can apply your own filtering logic.

Mi. Over a year ago

I just applied a rule to my loop before. It seems to work and my classifiers seem to use the input perfectly. Thank you.

ak_slick · Accepted Answer · 2018-05-14 18:16:29Z

1

import pandas as pd

# combine input lists
x = []
for _ in [['b','c','d'],['b','a','u'], ['b','i','o']]:
    x += _

# output into a single column
a = pd.Series(x)

# mock original dataframe
b = pd.DataFrame({'sent': [0, 0, 0, 1, 1, 1], 
                  'token': ['a', 'b', 'b', 'a', 'b', 'c']})

# add column to existing dataframe
# this will avoid the mis matched length error by ignoring anything longer 
# than your original data frame
b['pred'] = a

   sent token pred
0     0     a    b
1     0     b    c
2     0     b    d
3     1     a    b
4     1     b    a
5     1     c    u

edited May 14, 2018 at 18:16

answered May 14, 2018 at 17:28

ak_slick

1,0167 silver badges21 bronze badges

Collectives™ on Stack Overflow

List of lists to rows in a pandas dataframe

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related