Converting pandas.core.series.Series to dataframe with multiple column names

Question

My toy example is as follows:

import numpy as np
from sklearn.datasets import load_iris
import pandas as pd

### prepare data 
Xy = np.c_[load_iris(return_X_y=True)]
mycol = ['x1','x2','x3','x4','group']
df = pd.DataFrame(data=Xy, columns=mycol)
dat = df.iloc[:100,:] #only consider two species
dat['group'] = dat.group.apply(lambda x: 1 if x ==0 else 2) #two species means two groups
dat.shape
dat.head()

### Linear discriminant analysis procedure
G1 = dat.iloc[:50,:-1]; x1_bar = G1.mean(); S1 = G1.cov(); n1 = G1.shape[0]
G2 = dat.iloc[50:,:-1]; x2_bar = G2.mean(); S2 = G2.cov(); n2 = G2.shape[0] 
Sp = (n1-1)/(n1+n2-2)*S1 + (n2-1)/(n1+n2-2)*S2
a = np.linalg.inv(Sp).dot(x1_bar-x2_bar); u_bar = (x1_bar + x2_bar)/2
m = a.T.dot(u_bar); print("Linear discriminant boundary is {} ".format(m)) 

def my_lda(x):
    y = a.T.dot(x)
    pred = 1 if y >= m else 2
    return y.round(4), pred
 
xx = dat.iloc[:,:-1]
xxa = xx.agg(my_lda, axis=1)
xxa.shape
type(xxa)

We have xxa is a pandas.core.series.Series with shape (100,). Note that there are two columns in parentheses of xxa, I want convert xxa to a pd.DataFrame with 100 rows x 2 columns and I try

xxa_df1 = pd.DataFrame(data=xxa, columns=['y','pred'])

which gives ValueError: Shape of passed values is (100, 1), indices imply (100, 2). Then I continue to try

xxa2 = xxa.to_frame()
# xxa2 = pd.DataFrame(xxa) #equals `xxa.to_frame()`
xxa_df2 = pd.DataFrame(data=xxa2, columns=['y','pred'])

and xxa_df2 presents all NaN with 100 rows x 2 columns. What should I do next?

Ynjxsjmh · Accepted Answer · 2022-06-14 15:25:50Z

2

Let's try Series.tolist()

xxa_df1 = pd.DataFrame(data=xxa.tolist(), columns=['y','pred'])

print(xxa_df1)

          y  pred
0   42.0080     1
1   32.3859     1
2   37.5566     1
3   31.0958     1
4   43.5050     1
..      ...   ...
95 -56.9613     2
96 -61.8481     2
97 -62.4983     2
98 -38.6006     2
99 -61.4737     2

[100 rows x 2 columns]

answered Jun 14, 2022 at 15:25

Ynjxsjmh

30.3k7 gold badges43 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Converting pandas.core.series.Series to dataframe with multiple column names

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related