Split Large Dataframe into multiple dataframes based on Column names

Question

I have a very large Dataframe with more than 3000 columns. Column names starts with items_0_**** | items_0_**** | items_1_**** | items_1_**** and so on up to items_99_*** . I want to Split my Large DataFrame in such a way that items_0_*** in one Dataframe, items_1_**** in one Dataframe and so up to items_99_**** in one DataFrame.

So Finally I need to get 100 Data Frames.

Column names starting with items_0_ must be in one Dataframe.
Column names starting with items_1_ must be in another Dataframe.
.

.

Column names starting with items_99_ must be in another Dataframe.

Any help is appreciated.

How do you want to retrieve your dataframes, from a dict?

ignoring_gravity
– ignoring_gravity

2020-01-02 10:09:02 +00:00
Commented Jan 2, 2020 at 10:09 — ignoring_gravity
– ignoring_gravity, Commented Jan 2, 2020 at 10:09
Updated my Question, Please check.

Bhanu Prakash
– Bhanu Prakash

2020-01-02 10:16:12 +00:00
Commented Jan 2, 2020 at 10:16 — Bhanu Prakash
– Bhanu Prakash, Commented Jan 2, 2020 at 10:16

ignoring_gravity · Accepted Answer · 2020-01-02 10:18:10Z

1

How do you want to retrieve your dataframes, from a dict?

If so:

my_dataframes = dict()

for i in range(100):
    my_dataframes[i] = df[[j for j in df.columns if j.startswith(f'items_{i}_')]]

Then, if you type my_dataframes[0], you'll get a dataframe with the columns items_0_****

edited Jan 2, 2020 at 10:18

answered Jan 2, 2020 at 10:10

ignoring_gravity

11.1k7 gold badges45 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Bhanu Prakash Over a year ago

Is there any way to store each of them in different csv files?

ignoring_gravity Over a year ago

You could put them all in a list, I guess? What do you want to do with them, once you have all your difference dataframes?

Bhanu Prakash Over a year ago

After I get all the different DataFrames , I need to merge them into one single DataFrame again, For Example : Columns items_0_a , items_1_a, items_2_a values should be in one single column.

Collectives™ on Stack Overflow

Split Large Dataframe into multiple dataframes based on Column names

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related