Colletting element from different DataFrames into arrays

Question

I like using nested data structures and now I'm trying to understand how to use Pandas

Here is a toy model:

a=pd.DataFrame({'x':[1,2],'y':[10,20]})
b=pd.DataFrame({'x':[3,4],'y':[30,40]})
c=[a,b]

now I would like to get:

sol=np.array([[[1],[3]],[[2],[4]]])

I have an idea to get both sol[0] and sol[1] as:

s0=np.array([item[['x']].ix[0] for item in c])
s1=np.array([item[['x']].ix[1] for item in c])

but to get sol I would run over the index and I don't think it is really pythonic...

unutbu · Accepted Answer · 2014-01-16 17:20:13Z

1

It looks like you want just the x columns from a and b. You can concatenate two Series (or DataFrames) into a new DataFrame using pd.concat:

In [132]: pd.concat([a['x'], b['x']], axis=1)
Out[132]: 
   x  x
0  1  3
1  2  4

[2 rows x 2 columns]

Now, if you want a numpy array, use the values attribute:

In [133]: pd.concat([a['x'], b['x']], axis=1).values
Out[133]: 
array([[1, 3],
       [2, 4]], dtype=int64)

And if you want a numpy array with the same shape as sol, then use the reshape method:

In [134]: pd.concat([a['x'], b['x']], axis=1).values.reshape(2,2,1)
Out[134]: 
array([[[1],
        [3]],

       [[2],
        [4]]], dtype=int64)

In [136]: np.allclose(pd.concat([a['x'], b['x']], axis=1).values.reshape(2,2,1), sol)
Out[136]: True

answered Jan 16, 2014 at 17:20

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user2988577 Over a year ago

It looks fine but I still have a doubt since I would like to use the list c and not its elements explicitly.

unutbu Over a year ago

Then use pd.concat([df['x'] for df in c], axis=1).

Collectives™ on Stack Overflow

Colletting element from different DataFrames into arrays

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related