How do I convert each array in the nested arrays as rows in the data-frames?

Question

I would like to convert each individual array in the nested arrays as each rows in the dataframes. For example: The nested arrays example is below. How do I create each numbers inside [[... ]] in rows in dataframe? There are many nested arrays such as below. Any help would be appreciated. Thank you. Just to make things easier, I want to create each array as rows in dataframe. Example for the first array needs to be laid out like this in dataframe *-0.14091441, 0.02556057, 0.10425788, ..., -0.03699904, 0.00503982, 0.08761989 So far I have tried :


pd.concat([pd.DataFrame(arrays_list[i][0]) for i in range(len(arrays_list))]).reset_index(drop=False)

arrays_list being my nested arrays.

[array([[[-0.14091441,  0.02556057,  0.10425788, ..., -0.03699904,
       0.00503982,  0.08761989]],

    [[-0.16227441,  0.03150389,  0.06440173, ..., -0.10543424,
       0.05987305,  0.04117104]],

    [[-0.11423473,  0.03741207,  0.0783961 , ..., -0.16773996,
       0.06566695,  0.0683976 ]],

    ...,

    [[-0.13789459,  0.05840103,  0.09803487, ..., -0.09256409,
       0.01833008,  0.08413954]],

    [[-0.12652887,  0.03683193,  0.06100509, ..., -0.06188103,
       0.00915053,  0.09518969]],

    [[-0.19781192,  0.05750425,  0.14811654, ..., -0.10550601,
       0.05405622,  0.13771409]]]),

 array([[[-0.0375578 ,  0.16006446,  0.07978896, ..., -0.0883253 ,
       0.0057608 ,  0.07953031]],

    [[ 0.00282089,  0.18854009,  0.01686837, ..., -0.02981209,
      -0.01220972,  0.02810074]],

    [[ 0.0333602 ,  0.21895081,  0.05255894, ..., -0.01882036,
      -0.03316848,  0.02506595]],

    ...,

    [[-0.02498044,  0.17067145,  0.03956907, ..., -0.00617604,
       0.01254308,  0.03375499]],

    [[ 0.0333602 ,  0.21895081,  0.05255894, ..., -0.01882036,
      -0.03316848,  0.02506595]],

    [[ 0.01105822,  0.20526624,  0.05087842, ..., -0.0442748 ,
      -0.08184794,  0.04356682]]]),
 array([[[-1.16298698e-01,  6.60857707e-02,  4.37349118e-02, ...,
       5.65935597e-02,  1.17720775e-01,  4.68457118e-02]],

    [[-1.70329705e-01,  7.15664029e-02,  2.13463139e-02, ...,
       8.40441436e-02,  3.38792875e-02, -9.66352411e-04]],

    [[-1.38925180e-01,  6.48617744e-02,  7.50765130e-02, ...,
       3.60708833e-02,  9.37591046e-02,  3.88324559e-02]],

    ...,

    [[-1.38925180e-01,  6.48617744e-02,  7.50765130e-02, ...,
       3.60708833e-02,  9.37591046e-02,  3.88324559e-02]],

    [[-1.55961111e-01,  7.59401619e-02,  3.65645029e-02, ...,
       9.78165418e-02,  9.37420279e-02,  5.14532737e-02]],

    [[-1.19004108e-01,  6.59743100e-02,  7.72421211e-02, ...,
       4.89978008e-02,  1.33881345e-04,  2.18758285e-02]]])]

For the purpose of this question here is an analogous mre.

from numpy.random import default_rng
rng = default_rng()

a = rng.integers(0,5,(6,1,6))
b = rng.integers(0,5,(6,1,6))
c = rng.integers(0,5,(6,1,6))
# or just
# np.random.randint(0,5,(6,1,6))
obj = [a,b,c]

what are the shapes of your arrays? Can you provide some code which generates similar (smaller) arrays as well as your expected dataframe result? — user32882
– user32882, Commented Aug 23, 2020 at 13:29
@user32882 - It is a list of three arrays with shapes of (x,1,y). — wwii
– wwii, Commented Aug 23, 2020 at 14:06
When asking questions for code that relies on data, it is important that a minimal example of the data is included in your question. Sometimes you might want to create fake data for the question - as long as it faithfully represents the actual data.. The easier you make it for us to copy and paste from your question (so that we can execute your code and test our solution) the more likely you'll get responses. - Please read minimal reproducible example. — wwii
– wwii, Commented Aug 23, 2020 at 14:41

Luc Bertin · Accepted Answer · 2020-08-23 14:26:05Z

2

Taking the reproducible example you've provided, and assuming you want one dataframe as a result (a, b and c have same shapes).

I would first stack vertically using numpy.stack() all the arrays in the list of arrays as one array.

Then I would drop the middle dimension of shape 1, by slicing it out.

Then I call pandas.DataFrame() constructor and you should have your result.

df = pd.DataFrame(np.vstack(obj)[:,0,:])

answered Aug 23, 2020 at 14:26

Luc Bertin

3464 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

wwii Over a year ago

numpy.squeeze for the xtra dimension also works.

Collectives™ on Stack Overflow

How do I convert each array in the nested arrays as rows in the data-frames?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related